Been searching for a tool to streamline ZFS replication between my storage arrays and I think I found it – it’s called syncoid.
Installation
# ubuntu apt install pv lzop mbuffer libconfig-inifiles-perl git -y # centos yum install pv lzop mbuffer perl-Config-IniFiles.noarch git -y cd /usr/src/ git clone https://github.com/jimsalterjrs/sanoid.git cp sanoid/syncoid /usr/local/sbin/
Now setup SSH keys for passwordless root login and create cronjob to fire up /usr/local/sbin/replication-syncoid.sh once a day/week/whenever suits you.
0 0 * * 1 /usr/local/sbin/replication-syncoid.sh|logger
Where /usr/local/sbin/replication-syncoid.sh is
#!/bin/bash FILESYSTEMS="apps home service vmfs" LOG=/var/log/syncoid.log for i in $FILESYSTEMS; do echo "--------------- `date` - syncing $i -------------" >> $LOG 2>&1 /usr/local/sbin/syncoid -r root@mielnet-second-tier0022:tank/$i cistern/$i >> $LOG 2>&1 echo "--------------- `date` - done $i -------------" >> $LOG 2>&1 done
No matching ciphers
You may get a message:
"no matching cipher found: client [email protected],arcfour server aes256-ctr,aes192-ctr,aes128-ctr,arcfour256,arcfour128"
most likely when you run Centos 6 on target host. You need to check for available ciphers on target host and modify syncoid script accordingly.
# ssh -Q cipher root@mielnet-second-tier0022| paste -d , -s 3des-cbc,blowfish-cbc,cast128-cbc,arcfour,arcfour128,arcfour256,aes128-cbc,aes192-cbc,aes256-cbc,[email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected],[email protected]
and paste output from above on line 39 of /usr/local/sbin/replication-syncoid.sh
$sshcipher = '-c 3des-cbc,blowfish-cbc,cast128-cbc,arcfour,arcfour128,arcfour256,aes128-cbc,aes192-cbc,aes256-cbc,[email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected],[email protected] ';
Now your sync will run OK.
Summary
It gets the job done nicely and saves me from writing my own bash-perl-voodoo-black-magic-fuckery.
Thanks Jim.