Skip to content
This repository has been archived by the owner on Mar 28, 2023. It is now read-only.

Using LOBCDER

Jump to bottom
skoulouzis edited this page Oct 17, 2015 · 3 revisions

Mounting LOBCDER manually on Linux

First of all, we need to install the davfs2 package. davfs2 provides the ability to access WebDAV resources like a typical filesystem, allowing for use by standard applications with no built-in support for WebDAV.

$sudo apt-get install davfs2

Next create, create the mount directory:

$sudo mkdir /media/$USER/lobcder
  • Now we are ready to mount lobcder.
$sudo mount -t davfs https://hostname/lobcder/dav \ /media/$USER/lobcder -o rw,uid=$USER
  • You will be asked for a username and a password.
  • Next we press 'y' to accept lobcder's certificate and lobcder is mounted on our machine.

Using LOBCDER with cadaver

For a quick access to lobcder we can also use cadaver. cadaver is a command-line WebDAV client for Unix. It supports file upload, download, on-screen display, namespace operations (move/copy), collection creation and deletion, and locking operations.

First of all, we need to install the cadaver package $sudo apt-get install cadaver

Now we can login with cadaver: $cadaver https://lobcder.vph.cyfronet.pl/lobcder/dav

You will be asked for a username and a password.

If you wish to log in to LOBCDER without typing your password or short token every time you can use the .netrc file. The .netrc file contains login and initialization information used by the auto-login process. It resides in the user's home directory.

To create it type: $echo "machine lobcder.vph.cyfronet.pl" >> $HOME/.netrc $echo "login username" >> $HOME/.netrc $echo "login password" >> $HOME/.netrc

Setting Location Preference

Since LOBCDER can use many storage backends in any location you may want to influence the replication policy for some files. This is done by setting the data-location-preference property.

This is an example of how to do it with cadaver:

  1. Create a folder:
    dav:/lobcder/dav/home/user> mkdir test

  2. Get the available backends and their exact url (you'll need it to set the data-location-preference property) check the 1avail-storage-sites property: dav:/lobcder/dav/home/user> propget test Fetching properties for test':
    custom: data-distribution = []
    custom: dri-supervised = false
    custom: dri-checksum-MD5 = null
    custom: dri-last-validation-date-ms = 0
    custom: dri-status =
    custom: description =
    custom: data-location-preference =
    custom: encrypt = []
    custom: avail-storage-sites = [swift://host1/v2.0/,webdavssl://host2/lobcder/]
    DAV: lockscope = io.milton.property.PropertySource$PropertyMetaData@3846d3c1
    custom: ttl =
    DAV: supported-report-set =
    DAV: iscollection = TRUE
    DAV: resourcetype = <DAV:collection></DAV:collection>
    DAV: creationdate = 2014-12-22T14:33:55Z
    DAV: quota-used-bytes =
    DAV: isreadonly = FALSE
    DAV: quota-available-bytes =
    DAV: getcontenttype = text/html
    DAV: name = test
    DAV: getcontentlength =
    DAV: getlastmodified = Mon, 22 Dec 2014 14:33:55 GMT
    DAV: getcreated = 2014-12-22T14:33:55Z
    DAV: getetag = "368_1919627634"
    DAV: displayname = test

  3. Now you can set the data-location-preference:

dav:/lobcder/dav/home/user> set namespace custom:
dav:/lobcder/dav/> propset test data-location-preference webdavssl://host2/lobcder/

  1. Now all files and folders under the test folder will inherit this property. This means that every file that is created under test will be replicated only towebdavssl://host2/lobcder/. If the URL is not set correctly or doesn't exist the files will be replicated according the default policy. So when putting a file in the test folder:

dav:/lobcder/dav/home/user> cd test
dav:/lobcder/dav/home/user/test/> put file

After some seconds (depending on the replication queue) if you run propget:

dav:/lobcder/dav/home/user/test/> propget file

custom: data-distribution = [webdavssl://host2/lobcder//f6291111-702a-4426-ba86-3f74755cf807-file]
custom: dri-supervised = false
custom: dri-checksum-MD5 =
custom: dri-last-validation-date-ms = 0
custom: dri-status =
custom: description =
custom: data-location-preference = webdavssl://host2/lobcder/ custom: encrypt = [[webdavssl://host2/,false]]
custom: avail-storage-sites = [swift://host1/v2.0/,webdavssl://host2]
DAV: lockscope = io.milton.property.PropertySource$PropertyMetaData@3846d3c1
custom: ttl =
DAV: supported-report-set =
DAV: iscollection = FALSE
DAV: resourcetype =
DAV: creationdate = 2014-12-22T14:45:19Z
DAV: quota-used-bytes =
DAV: isreadonly = TRUE
DAV: quota-available-bytes =
DAV: getcontenttype = application/octet-stream
DAV: name = file
DAV: getcontentlength = 37
DAV: getlastmodified = Mon, 22 Dec 2014 14:45:19 GMT
DAV: getcreated = 2014-12-22T14:45:19Z
DAV: getetag = "391_1920311634"
DAV: displayname = file

Follow redirects and resume downloads with curl

Follow redirects (returned code 302): curl -u user:pass -C - -O -L https://lobcder.vph.cyfronet.pl/lobcder/dav/file

Resume a download. This command in bash keeps trying to download a file till the download is complete: export ec=18; while [ $ec -eq 18 ]; do curl -O -C - -L --request GET -u user:pass https://lobcder.vph.cyfronet.pl/lobcder/dav/file ; export ec=$?; done`

Clone this wiki locally