pvaneynd: (Default)
[personal profile] pvaneynd
Now that my server at home is running FreeBSD on top of ZFS for a while now I though to setup the backups again.

Now I have 3 disks in the server, 2 are reliable and are used in the main ZFS pool (zroot), one is a POC WD 'green' disk that will die if you use it too often. So that one is my local backup disk.

My backup strategy is that I configured daily snapshots on zroot, I have the local backup disk and I'm using a USB hard disk for off-site backups.

At first I thought to use UFS2 on the third disks with rsync, like I did with Linux. Then I read a bit more about the ZFS functions and decided to use ZFS on the third disk. I created a 'fastbackup' pool and then used zfs send and zfs receive to sync the two pools. Syncing the disks was fast. Very fast indeed, about as fast as a simple 'dd' would have been.

However ZFS's magic does not end here. It has the option of sending incremental changes that happened between snapshots. So I wrote a script that makes a new snapshot ("nu" now in Dutch), sends the incremental changes to 'fastbackup' and then moved the reference snapshot forward. This to me seemed to be faster then using rsync. which would always take at least 5 minutes to declare that the filesystems were in sync. ZFS is ... faster:

+ zfs snapshot zroot/usr@nu
+ zfs send -vi zroot/usr@laatste zroot/usr@nu
+ zfs receive -Fv fastbackup/usr@nu
receiving incremental stream of zroot/usr@nu into fastbackup/usr@nu
received 597MB stream in 10 seconds (59.7MB/sec)

So that's a day worth of changes, including building and installing emacs and clisp, in 10 seconds.

Script I used below the cut.
[root@frost ~/bin]# cat fastbackup.sh

set -x

zpool import fastbackup || exit 1

/usr/local/bin/rsync --archive --acls --xattrs --times /etc/ /fastbackup/etc/
/usr/local/bin/rsync --archive --acls --xattrs --times /usr/local/etc/ /fastbackup/usr/local/etc/
/usr/local/bin/rsync --archive --acls --xattrs --times /var/cron/tabs/ /fastbackup//var/cron/tabs/

for fs in /usr /usr/ports /usr/src /usr/home /var /var/crash /var/db /var/db/pkg /var/empty /var/mail /var/run /Media /Media/Data /Media/Movies /Media/Music /Media/MusicVideos /Media/Pictures 
   zfs list -t snapshot zroot${fs}@nu > /dev/null 2>&1 && zfs destroy  zroot${fs}@nu
   zfs list -t snapshot fastbackup${fs}@nu > /dev/null 2>&1 && zfs destroy  fastbackup${fs}@nu
   echo snapshotting ${fs}
   zfs snapshot zroot${fs}@nu && 
     ( echo snapshop done
       zfs send -vi zroot${fs}@laatste zroot${fs}@nu | zfs receive -Fv fastbackup${fs}@nu &&
       ( echo sending done 
         zfs destroy zroot${fs}@laatste || exit
         zfs destroy fastbackup${fs}@laatste || exit
         zfs rename zroot${fs}@nu zroot${fs}@laatste || exit
         zfs rename fastbackup${fs}@nu fastbackup${fs}@laatste || exit
         echo done for ${fs} ) )

# now do root:
# evil hack as I do not want fastbackup to contain the zroot/ information
zfs list -t snapshot zroot@nu > /dev/null 2>&1 && zfs destroy  zroot@nu
zfs list -t snapshot fastbackup/root@nu > /dev/null 2>&1 && zfs destroy  fastbackup/root@nu
echo snapshotting  /
zfs snapshot zroot@nu && 
 ( echo snapshop done
   zfs send -vi zroot@laatste zroot@nu | zfs receive -Fv fastbackup/root@nu &&
       ( echo sending done 
         zfs destroy zroot@laatste || exit
         zfs destroy fastbackup/root@laatste || exit
         zfs rename zroot@nu zroot@laatste || exit
         zfs rename fastbackup/root@nu fastbackup/root@laatste || exit
         echo done for / ) )

zpool export fastbackup      
echo all done
Anonymous( )Anonymous This account has disabled anonymous posting.
OpenID( )OpenID You can comment on this post while signed in with an account from many other sites, once you have confirmed your email address. Sign in using OpenID.
Account name:
If you don't have an account you can create one now.
HTML doesn't work in the subject.


Notice: This account is set to log the IP addresses of everyone who comments.
Links will be displayed as unclickable URLs to help prevent spam.


pvaneynd: (Default)

March 2017

12131415 161718
26272829 3031 

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Oct. 17th, 2017 12:17 am
Powered by Dreamwidth Studios