In an earlier article, I explained how to use rsync to make local backups of a Linux system. Remote backups, where you store your backed up data on a separate machine, further promote data safety by separating information both physically and geographically.
First steps
To perform secure remote backups, you must have rsync and SSH installed on both your local and your target remote machine. Rsync can use SSH as a secure transport agent.
Make sure rsync is installed by opening a terminal session and typing rsync --version on each machine. You should see a message like rsync version 2.X.X protocol version X. If you receive "command not found" or a similar message, you'll need to download and install rsync. Use your GNU/Linux distribution's package management system to do this, or download and install the source from the rsync Web site. If you're running Microsoft Windows I recommend installing cwRsync. Mac OS X comes with rsync, but if you want to try a different version, check out RsyncX.
SSH is likely to already be installed on your Mac OS X and GNU/Linux systems, while the Windows port of rsync, cwRsync, includes the key SSH programs. I'm going to assume you're running Linux or OS X on the remote machine where the backup is to be stored. Make sure your remote machine has Secure Shell Daemon (sshd) running and that the users of both machines have proper permissions to execute a backup.
To ensure that sshd is running on a remote machine, enter a terminal session and type ssh [user]@[remote.machines.address]. If all is well, you should be asked for the user's password and allowed to log in and check permissions. If the remote machine is the destination where the rsync backup will be stored, you'll want read and write permission to the destination directory.
Once you know all the necessary programs and permissions are in place, choose a directory with a few small files to use as a test backup. On the remote machine, create a destination directory to hold your backups:
rsync -avz -e ssh /some/small/directory/ remote_user@remotehost.com:/backup/destination/directory/
The trailing slash in the source directory causes rsync to copy only the contents of the source directory. Omitting the trailing slash causes rsync to copy both the directory name and its contents to the destination.
After rsync completes, you'll be left with a copy of the source files on the remote computer. Congratulations on your first remote backup! Now let's automate the process.
Automatic backups
The first step in automating remote backups is to remove any required user intervention -- namely requests for SSH passwords. To allow your systems to make an SSH connection without asking for a password, you must generate passphraseless keys. On the local machine, drop into the terminal and enter:
$ ssh-keygen -t dsa -b 2048 -f ~/rsync-key
Generating public/private dsa key pair.
Enter passphrase (empty for no passphrase): [press enter here]
Enter same passphrase again: [press enter here]
Your identification has been saved in /home/user/rsync-key.
Your public key has been saved in /home/user/rsync-key.pub.
The key fingerprint is:
8c:57:af:68:cd:b2:7c:aa:6d:d6:ee:0a:5a:a4:29:03 user@localhost
Now copy the public key to the remote machine using Secure Copy:
scp ~/rsync-key.pub user@remotehost:~
Finally, put the public key into the authorized_keys file on the remote host. SSH into the remote machine using ssh user@remotehost.com and execute:
mkdir ~/.ssh
chmod 700 ~/.ssh
mv ~/rsync-key.pub ~/.ssh/
cd ~/.ssh/
touch authorized_keys
chmod 600 authorized_keys
cat rsync-key.pub >> authorized_keys
You should now be able to SSH into the remote machine without being asked for a password. Give it a try by closing your previous SSH session and creating another one by typing ssh -i ~/rsync-key user@remotehost.
These entries with no passwords can originate from any host and execute anything. You can add additional security by limiting what the SSH connection can do via the authorized_keys file. I don't recommend employing any additional security until after your first backup in order to limit the troubleshooting process, but once you've completed that successfully, you can employ additional security by using SSH to connect to the remote machine and editing your ~/.ssh/authorized_keys file. It should look similar to:
ssh-dss AAAAB3NzaC1yc2EAAAABIwAAAIEAyNChQxw/+Da....= user@remotehost.com
To limit where connections are coming from, prefix the key with from="ip.address". To limit what command is executed, prefix the key with command="/path/to/validating/script". As an example, your secured authorized_keys file might look like:
from="192.168.0.1", command="/home/user/validate-rsync.sh" ssh-dss AAAAB3NzaC1yc2EAAAABIwAAAIEAyNChQxw/+Da....= user@remotehost.com
Finally, put something like the following in your validate-rsync.sh file:
#!/bin/sh
case "$SSH_ORIGINAL_COMMAND" in
*\&*)
echo "Rejected"
;;
*\;*)
echo "Rejected"
;;
rsync\ --server*)
$SSH_ORIGINAL_COMMAND
;;
*)
echo "Rejected"
;;
esac
Make it executable by typing: chmod +x ~/validate-rsync.sh. This will check to see if the ssh session is being used to execute an rsync backup. If it is being used for anything else, the session will be rejected and closed.
Rsync should now complete without prompting for a password if we modify the test backup we used earlier, telling it to use keys. Try it out by typing:
rsync -avz -e "ssh -i ~/rsync-key" /some/small/directory/ remote_user@remotehost.com:/backup/destination/directory/
If you're having problems, please ensure you have proper permissions to read from the source (/some/small/directory) and to write to the target (remotehost:/backup/destination). Also make sure passphraseless ssh sessions can be established between the two hosts, ~/rsync-key exists on the machine to be backed up, and that rsync is intalled on both machines.
Next: Making a real backupNote: Comments are owned by the poster. We are not responsible for their content.
but I get an error if I try:<tt>ssh -i ~/.ssh/rsync-key user@remotehost.com</tt>
The error is:<tt>rsync -avz -e "ssh -i ~/.ssh/rsync-key"<nobr> <wbr></nobr>/local/dir/ user@remotehost.com:/remote/dir/</tt>
Why would the first work but not the second?<tt>Warning: Identity file ~/.ssh/rsync-key not accessible: No such file or directory.</tt>
Using tools like <A HREF="http://www.gentoo.org/proj/en/keychain/index.xml" title="gentoo.org">keychain</a gentoo.org> or some other manipulation of ssh-agent on the backup repository and pulling the data negates the temptation to use such an insecure method.
Based in part on the above, we've put together a series of how-tos, including sample scripts, for using rsync on Linux, Windows & Mac OS X. Might be of some help to folks.
<a href="http://www.exavault.com/rsync_setup_unix_linux_bsd.shtml" title="exavault.com">rsync on Linux/Unix/BSD</a exavault.com>
<a href="http://www.exavault.com/rsync_setup_mac_osx.shtml" title="exavault.com">rsync on Mac OS X</a exavault.com>
<a href="http://www.exavault.com/rsync_setup_windows.shtml" title="exavault.com">rsync on Windows (All Versions)</a exavault.com>
These are for our online backup service, <a href="http://www.exavault.com/" title="exavault.com">ExaVault</a exavault.com>, but if you want to use them for your own purposes you are welcome to. You just need to change the name of the server (username.exavault.com) to your own server, and you need to add your keys to your authorized_keys file manually, instead of using our 'addkeys'.
validate-rsync.sh needs password?
Posted by: Anonymous Coward on November 19, 2004 06:43 PM#