Integrate shadow copy snapshots for Samba

[dajhorn]

Provide an option to hook automatic snapshots into Samba like this:

• https://wiki.samba.org/index.php/Rotating_LVM_snapshots_for_shadow_copy

[hawkowl]

This is easily done in Samba4 with the latest zfsonlinux by setting snapdir as visible and making your file shares like such in smb.conf - however you'd only be able to do any one of the snapshot times - maybe if you removed the prefixes, somehow? Might interfere with the 'keeping a certain amount of hourly/weekly'. Here's how I'm using it right now, anyway:

[data]
        path = /tank/sharedata
        read only = No
        comment = ZFS dataset with Previous Versions enabled
        vfs objects = shadow_copy2
        shadow: snapdir = .zfs/snapshot
        shadow: sort = desc
        shadow: format = zfs-auto-snap_hourly-%F-%H%M

[dajhorn]

@FransUrbo is working on Samba registry shares in his smbfs_registry-shares branch, which seems like the best way to automate this kind of thing.

Perhaps this could be implemented as a zfs set sharesmb=on,shadowcopies or zfs set shareshadow=on enhancement in the ZoL libshare.

[leecallen]

I have modified zfs-auto-snapshot for my own use to address this problem. Instead of embedding the 'label' in the snapshot name (eg, zfs-auto-snap_hourly-...), I put it in a zfs properly. I also modified the code that cleans up old snapshots to examine this property. It's passed in the SNAPSHOTS_OLD list so it should be just as fast.

I would be glad to provide this code if there is any desire to integrate it into the standard distribution. But I think making this behaviour parametric -- ie, controlled by an argument or config value -- would significantly complicate the otherwise clean code.

[dajhorn]

@leecallen, certainly. Post the code to a fork of this repository if you want to float an idea. This will make it visible to people that are already watching this repo and not put anything permanent into the ticket history.

[leecallen]

I have never done that before - submitted a change (fork) to a github project. Do I flag my changes, or will my system automatically detect changes using diff? Is there anything else I should know? (I found the basic 'howto' under Help).

[FransUrbo]

On Feb 3, 2014, at 5:30 PM, leecallen wrote:

Do I flag my changes, or will my system automatically detect changes using diff?

It will automatically find your changes when you do a 'create pull request' on
GitHub.

One note though. If upstream have done a lot of changes (let's say ten commits),
which you then pull in to your local repo and then push to YOUR repo, the pull
request will look ... 'weird'. It will contain those then commits in addition to the
commit(s) you intended, even though 'the ten' already exist upstream...

What you do then, is a 'git rebase origin/master' (or whatever the upstream
branch is called), THEN push it you your own repo.

And you can always 'take back' (delete) your pull request if it doesn't look ok.
You'll probably need to test this out a few times before you get the hang of it.
Just don't do NEW pull requests, keep it in the one and play with 'git rebase'...

Oh, and another thing you might want to consider: push your changes to your
own repo into separate branches for each pull request. That way things is kept
isolated from each other (and you'll have a record of what you've done, even
though it's not needed - I keep all my 'pull request branches' for that purpose,

even after the pull request have been accepted).

System administrators motto:
You're either invisible or in trouble.

• Unknown

[leecallen]

Done. Thanks for the help guys.

[ArakniD]

One can expose all weekly, monthly, hourly snapshots into windows if you're clever with the snapshot naming.. use a number system such as 01, 02, 03 ,04 for week/mon/hour/day etc.. then use the format property to interpret that label as part of the date-time

vfs objects = shadow_copy2
shadow: snapdir = .zfs/snapshot
shadow: sort = desc
shadow: format = zfs-auto-snap_%S-%Y-%m-%d-%H%M
shadow: localtime = yes

too easy :)

[leecallen]

Well that is really very clever. I will try it as soon as I get a chance.
Have you used it successfully?

On Sat, Jun 21, 2014 at 9:01 AM, ArakniD [email protected] wrote:

One can expose all weekly, monthly, hourly snapshots into windows if
you're clever with the snapshot naming.. use a number system such as 01,
02, 03 ,04 for week/mon/hour/day etc.. then use the format property to
interpret that label as part of the date-time

vfs objects = shadow_copy2
shadow: snapdir = .zfs/snapshot
shadow: sort = desc
shadow: format = zfs-auto-snap_%S-%Y-%m-%d-%H%M
shadow: localtime = yes

too easy :)

—
Reply to this email directly or view it on GitHub
#10 (comment)
.

Lee Allen
email: [email protected]
cell: (716) 880-0854
fax: (716) 408-8844

[ArakniD]

Sure have.. use it on my home NAS.. mainly to prevent my wife from deleting my shit :)

I've also had to patch the main script to purge old snapshots by their tag name.. so If I say keep 8 hourly.. it only removes HOURLY snapshots.. keeping 8 of them only..

this way you can say, keep 8 hourly... 7 daily 4 weekly and 6 monthly? a fairly good snapshot regime

I think I posted this under another issue too.. enjoy :0

[stuehmer]

I created some instructions in the wiki how to use shadow copies with Samba.

Can someone merge the leecallen branch into master to simplify the process?

[dajhorn]

@stuehmer, the work by @leecallen was merged into a topic branch, but ZoL itself didn't get tight Samba integration. I will revisit this issue when I do a mid-series update for Ubuntu 15.04 Vivid.

[n4ldi]

Hi all, thanks before. I need some help about destroy snapshot.
How to make zfs auto destroy only the old snapshot ? For example I have three snapshot list =
zfs-auto-snap_daily-2016-05-01-2336
zfs-auto-snap_daily-2016-05-02-2350
zfs-auto-snap_daily-2016-05-03-2491

And I need to delete/destroy the old one list "zfs-auto-snap_daily-2016-05-01-2336"
And than tomorrow will delete the old one list "zfs-auto-snap_daily-2016-05-02-2350"
and so on. And the script will be run on cron every day.

[ArakniD]

There is a --keep option. I label mine with 01, 02, 03 etc so that samba can take format and use it as the date milliseconds, but then allow the auto snap script to use the tag to destroy only the older snapshots in THAT list. Not just the oldest one..

I'll dump my commands when I get behind a terminal

[ArakniD]

this is my crontab

/etc/cron.d/zfs-auto-snapshot:*/15 * * * * root zfs-auto-snapshot -q -g --label=04 --keep=4 rpool/TRUNK/ubuntu-1
/etc/cron.daily/zfs-auto-snapshot:exec zfs-auto-snapshot --quiet --syslog --label=01 --keep=31 rpool/TRUNK/ubuntu-1
/etc/cron.hourly/zfs-auto-snapshot:exec zfs-auto-snapshot --quiet --syslog --label=00 --keep=24 rpool/TRUNK/ubuntu-1
/etc/cron.monthly/zfs-auto-snapshot:exec zfs-auto-snapshot --quiet --syslog --label=03 --keep=3 rpool/TRUNK/ubuntu-1
/etc/cron.weekly/zfs-auto-snapshot:exec zfs-auto-snapshot --quiet --syslog --label=02 --keep=8 rpool/TRUNK/ubuntu-1

This will create snapshots that look like

rpool/TRUNK/ubuntu-1@zfs-auto-snap_03-2016-01-14-2151
rpool/TRUNK/ubuntu-1@zfs-auto-snap_03-2016-02-12-2149
rpool/TRUNK/ubuntu-1@zfs-auto-snap_03-2016-03-12-2147
rpool/TRUNK/ubuntu-1@zfs-auto-snap_02-2016-04-06-2156
rpool/TRUNK/ubuntu-1@zfs-auto-snap_03-2016-04-11-2150
rpool/TRUNK/ubuntu-1@zfs-auto-snap_02-2016-04-13-2149
rpool/TRUNK/ubuntu-1@zfs-auto-snap_02-2016-04-20-2143
rpool/TRUNK/ubuntu-1@zfs-auto-snap_02-2016-04-27-2144
rpool/TRUNK/ubuntu-1@zfs-auto-snap_01-2016-04-30-2200
rpool/TRUNK/ubuntu-1@zfs-auto-snap_01-2016-05-01-2155
rpool/TRUNK/ubuntu-1@zfs-auto-snap_01-2016-05-02-2144

which if you use samba and the "previous versions" tabs, then configure smb with;

vfs objects = shadow_copy2
shadow: snapdir = .zfs/snapshot
shadow: sort = desc
shadow: format = zfs-auto-snap_%S-%Y-%m-%d-%H%M
shadow: localtime = yes

Then you can have seperate lists of snapshots which are kept on different queue depths. Success.

[realG]

Hey @ArakniD , thanks for your tips in this thread! I've mostly got shadow copies working now on my server. Right clicking on the share on my windows box, I can see the snapshots listed in the Previous Versions tab, but when I try to open one, I'm getting a "You do not have permission to access \server\tank\share@GMT-date-time" error.

I'm thinking this might be due to the permissions set on snapshots created by the zfs-auto-snapshot script? Have you (or anyone else reading this!) had to modify permissions or tweak the script in any way to be able to browse snapshot contents on a windows client?

Edit: Well, I went back to trying to figure out how to make this work after a few days' pause... and, funnily enough, I found shadow copies working perfectly fine, without changing anything in the config. No idea why it didn't work before, my best guess is that one of the recent samba patches fixed some underlying issue. I'm on Ubuntu 16.04 and it's still early days so a lot of bugfixes are being pushed out almost daily... So, for the record, the smb config settings posted in this thread seem to be working great on 16.04/Samba version 4.3.9-Ubuntu.

[AntonPointner]

Hello,
in our enviroment (newest centos and samba 4.4.4) the previous versions was not working.

In our case the problem was the format of the date/time.
I found in the vfs_shadow_copy2 man page:
The format must be compatible with the conversion specifications recognized by str[fp]time.

My experience:
In Samba the vfs object shadow_copies2 regognize only this format:
YYYY.MM.DD-hh.mm.ss

I have manually changed in the script "zfs-auto-snapshot" the line
DATE=$(date --utc +%F-%H%M) to
DATE=$(date +%Y.%m.%d-%H.%M.%S)
(for creating snapshots manually:
zfs snapshot zpool1/test@zfs-auto-snap_daily-`date -u +%Y.%m.%d-%H.%M.%S`)

Our smb.conf:
[sharename]
path = /fish/tank
read only = no
vfs objects = acl_xattr shadow_copy2
shadow: snapdir = .zfs/snapshot
shadow: sort = desc
shadow: format = zfs-auto-snap_daily-%Y.%m.%d-%H.%M.%S
shadow: localtime = yes

Now iam happy.

regards
Anton

[beren12]

any possibility for a config for this so we don't have to pull in another repo?

via https://github.com/zfsonlinux/zfs-auto-snapshot/wiki/Samba