pvaneynd: (Default)
[personal profile] pvaneynd
Now that my server at home is running FreeBSD on top of ZFS for a while now I though to setup the backups again.

Now I have 3 disks in the server, 2 are reliable and are used in the main ZFS pool (zroot), one is a POC WD 'green' disk that will die if you use it too often. So that one is my local backup disk.

My backup strategy is that I configured daily snapshots on zroot, I have the local backup disk and I'm using a USB hard disk for off-site backups.

At first I thought to use UFS2 on the third disks with rsync, like I did with Linux. Then I read a bit more about the ZFS functions and decided to use ZFS on the third disk. I created a 'fastbackup' pool and then used zfs send and zfs receive to sync the two pools. Syncing the disks was fast. Very fast indeed, about as fast as a simple 'dd' would have been.

However ZFS's magic does not end here. It has the option of sending incremental changes that happened between snapshots. So I wrote a script that makes a new snapshot ("nu" now in Dutch), sends the incremental changes to 'fastbackup' and then moved the reference snapshot forward. This to me seemed to be faster then using rsync. which would always take at least 5 minutes to declare that the filesystems were in sync. ZFS is ... faster:

+ zfs snapshot zroot/usr@nu
+ zfs send -vi zroot/usr@laatste zroot/usr@nu
+ zfs receive -Fv fastbackup/usr@nu
receiving incremental stream of zroot/usr@nu into fastbackup/usr@nu
received 597MB stream in 10 seconds (59.7MB/sec)


So that's a day worth of changes, including building and installing emacs and clisp, in 10 seconds.

Script I used below the cut.
[root@frost ~/bin]# cat fastbackup.sh
#!/usr/local/bin/bash

set -x

zpool import fastbackup || exit 1

/usr/local/bin/rsync --archive --acls --xattrs --times /etc/ /fastbackup/etc/
/usr/local/bin/rsync --archive --acls --xattrs --times /usr/local/etc/ /fastbackup/usr/local/etc/
/usr/local/bin/rsync --archive --acls --xattrs --times /var/cron/tabs/ /fastbackup//var/cron/tabs/


for fs in /usr /usr/ports /usr/src /usr/home /var /var/crash /var/db /var/db/pkg /var/empty /var/mail /var/run /Media /Media/Data /Media/Movies /Media/Music /Media/MusicVideos /Media/Pictures 
  do 
   zfs list -t snapshot zroot${fs}@nu > /dev/null 2>&1 && zfs destroy  zroot${fs}@nu
   zfs list -t snapshot fastbackup${fs}@nu > /dev/null 2>&1 && zfs destroy  fastbackup${fs}@nu
   echo snapshotting ${fs}
   zfs snapshot zroot${fs}@nu && 
     ( echo snapshop done
       zfs send -vi zroot${fs}@laatste zroot${fs}@nu | zfs receive -Fv fastbackup${fs}@nu &&
       ( echo sending done 
         zfs destroy zroot${fs}@laatste || exit
         zfs destroy fastbackup${fs}@laatste || exit
         zfs rename zroot${fs}@nu zroot${fs}@laatste || exit
         zfs rename fastbackup${fs}@nu fastbackup${fs}@laatste || exit
         echo done for ${fs} ) )
done

# now do root:
# evil hack as I do not want fastbackup to contain the zroot/ information
zfs list -t snapshot zroot@nu > /dev/null 2>&1 && zfs destroy  zroot@nu
zfs list -t snapshot fastbackup/root@nu > /dev/null 2>&1 && zfs destroy  fastbackup/root@nu
echo snapshotting  /
zfs snapshot zroot@nu && 
 ( echo snapshop done
   zfs send -vi zroot@laatste zroot@nu | zfs receive -Fv fastbackup/root@nu &&
       ( echo sending done 
         zfs destroy zroot@laatste || exit
         zfs destroy fastbackup/root@laatste || exit
         zfs rename zroot@nu zroot@laatste || exit
         zfs rename fastbackup/root@nu fastbackup/root@laatste || exit
         echo done for / ) )

zpool export fastbackup      
echo all done

Date: 2011-05-09 05:15 pm (UTC)
From: [identity profile] click0.myopenid.com
In order for this script to work, you must first create zroot${fs}@laatste
snapshotting /var
snapshop done
warning: cannot send 'zroot/var@nu': incremental source (@laatste) does not exist
cannot receive: failed to read from stream
snapshotting /tmp
snapshop done
warning: cannot send 'zroot/tmp@nu': incremental source (@laatste) does not exist
cannot receive: failed to read from stream

Date: 2011-05-13 08:44 pm (UTC)
From: [identity profile] click0.myopenid.com
I am a little changed your script.
Scripts were two - one for the initial copy sections, and the second to copy only the changes.
https://gist.github.com/971271
What to add copyrights in the scripts? or leave as is?

Profile

pvaneynd: (Default)
pvaneynd

March 2017

S M T W T F S
   1234
567891011
12131415 161718
19202122232425
262728293031 

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Mar. 25th, 2017 07:45 am
Powered by Dreamwidth Studios