Linux.com

Providing all went well, it's time to make a real backup. We'll modify the backup.sh script from my earlier article to allow our backup to be stored remotely. Create an rbackup.sh script by copying the code below into your favorite text editor. Decide on the files and directories you'd like to back up and add them to the $SOURCES variable in the script:

#!/bin/sh
# Author: Brice Burgess - bhb@iceburg.net
# rbackup.sh -- secure backup to a remote machine using rsync.

# Directories to backup. Separate with a space. Exclude trailing slash!
SOURCES="/home/wendy /home/daisy /var/mail"

# IP or FQDN of Remote Machine
RMACHINE=192.168.0.2

# Remote username
RUSER=brice

# Location of passphraseless ssh keyfile
RKEY=/home/user/rsync-key

# Directory to backup to on the remote machine. This is where your backup(s) will be stored
# Exclude trailing slash!
RTARGET="/home/user/backups/my_machine"

# Your EXCLUDE_FILE tells rsync what NOT to backup. Leave it unchanged, missing or
# empty if you want to backup all files in your SOURCES. If performing a
# FULL SYSTEM BACKUP, ie. Your SOURCES is set to "/", you will need to make
# use of EXCLUDE_FILE. The file should contain directories and filenames, one per line.
# An example of a EXCLUDE_FILE would be:
# /proc/
# /tmp/
# /mnt/
# *.SOME_KIND_OF_FILE
EXCLUDE_FILE="/path/to/your/exclude_file.txt"

# Comment out the following line to disable verbose output
VERBOSE="-v"

#######################################
########DO_NOT_EDIT_BELOW_THIS_POINT#########
#######################################

if [ ! -f $RKEY ]; then
  echo "Couldn't find ssh keyfile!"
  echo "Exiting..."
  exit 2
fi

if ! ssh -i $RKEY $RUSER@$RMACHINE "test -x $RTARGET"; then
  echo "Target directory on remote machine doesn't exist or bad permissions."
  echo "Exiting..."
  exit 2
fi

echo "Verifying Sources..."
for source in $SOURCES; do
	echo "Checking $source..."
	if [ ! -x $source ]; then
     echo "Error with $source!"
     echo "Directory either does not exist, or you do not have proper permissions."
     exit 2
   fi
done

if [ -f $EXCLUDE_FILE ]; then
EXCLUDE="--exclude-from=$EXCLUDE_FILE"
fi

echo "Sources verified. Running rsync..."
for source in $SOURCES; do

  # Create directories in $RTARGET to mimick source directory hiearchy
  if ! ssh -i $RKEY $RUSER@$RMACHINE "test -d $RTARGET/$source"; then
    ssh -i $RKEY $RUSER@$RMACHINE "mkdir -p $RTARGET/$source"
  fi

  rsync $VERBOSE $EXCLUDE -a --delete -e "ssh -i $RKEY" $source/ $RUSER@$RMACHINE:$RTARGET/$source/

done

exit 0

Change the $RMACHINE, $RTARGET, $RUSER, and $RKEY variables to appropriate values. Save the script (as rbackup.sh) to your computer and make it executable by typing: chmod +x backup.sh.

If your local machine uses Mac OS X, these methods should work for you within the terminal program. If you're using Windows, the cwRsync installer creates a file named cwrsync.cmd that you can customize as a backup script. Save this file as a batch (.bat) file.

Execute your backup script by typing ./rbackup.sh on the Linux or OS X terminal, or run your batch script from the Windows command prompt. It will take a long time for the script to complete the first time it runs, because rsync must make a copy of each file rather than solely updating changed files. Later runs will complete much faster.

If you notice something is wrong, press Ctrl-C to stop the process. Upon completion of the script, there should be a replica of your $SOURCES on the remote machine.

Automating the process

Assuming rbackup.sh ran successfully, it's time to automate the process. Use the Scheduled Task accessory to run your batch file under Windows. For Linux and OS X, use the cron daemon to schedule backups. The cron daemon uses crontab files to schedule tasks. You can edit the system's main crontab file by becoming the superuser (either by logging in as root or typing su in the terminal) and executing crontab -e to edit the file with your system's default editor.

You'll want to schedule a time for your rbackup.sh to execute. Crontab syntax is:

[minute] [hour] [day] [month] [dow] [command]

Thus, adding the line

0 4 * * * /path/to/rbackup.sh

will execute rbackup.sh at 4:00 a.m. every day, and

0 4 * * 5 /path/to/rbackup.sh

will execute rbackup.sh at 4:00 a.m. every Friday. When you've finished adding the line, save the file and exit.

Keeping multiple backups

You may want to implement multiple remote backups if there is enough space at the destination to hold them. Multiple backups, known as 'snapshots' are important to have in case you've accidently deleted files you'd like to retain. Check the size of your backup by executing:du -sh /rsync/target_directory on the machine holding your backups. You can estimate the number of days to retain by dividing the amount of space you're willing to allocate to backups by your backup size and rounding down to the integer value. If you have the space, keep snapshots of your five past backups.

I've modified the above script to accommodate for multiple backup rotation. The modifications keep a designated number of backups in the remote machine's target directory named after the date they were executed (YYYY-MM-DD_Hour-Minute). Here's the modified script:

#!/bin/sh
# Author: Brice Burgess - bhb@iceburg.net
# multi_rbackup.sh -- secure backup to a remote machine using rsync.
# Uses hard-link rotation to keep multiple backups on the remote machine.

# Directories to backup. Separate with a space. Exclude trailing slash!
SOURCES="/home/wendy /home/daisy /var/mail"

# IP or FQDN of Remote Machine
RMACHINE=192.168.0.2

# Remote username
RUSER=brice

# Location of passphraseless ssh keyfile
RKEY=/home/user/rsync-key

# Directory to backup to on the remote machine. This is where your backup(s) will be stored
# :: NOTICE :: -> Make sure this directory is empty or contains ONLY backups created by
#	                        this script and NOTHING else. Exclude trailing slash!
RTARGET="/home/user/backups/my_machine"

# Set the number of backups to keep (greater than 1). Ensure you have adaquate space.
ROTATIONS=3

# Your EXCLUDE_FILE tells rsync what NOT to backup. Leave it unchanged, missing or
# empty if you want to backup all files in your SOURCES. If performing a
# FULL SYSTEM BACKUP, ie. Your SOURCES is set to "/", you will need to make
# use of EXCLUDE_FILE. The file should contain directories and filenames, one per line.
# An example of a EXCLUDE_FILE would be:
# /proc/
# /tmp/
# /mnt/
# *.SOME_KIND_OF_FILE
EXCLUDE_FILE="/path/to/your/exclude_file.txt"

# Comment out the following line to disable verbose output
VERBOSE="-v"

#######################################
########DO_NOT_EDIT_BELOW_THIS_POINT#########
#######################################

if [ ! -f $RKEY ]; then
  echo "Couldn't find ssh keyfile!"
  echo "Exiting..."
  exit 2
fi

if ! ssh -i $RKEY $RUSER@$RMACHINE "test -x $RTARGET"; then
  echo "Target directory on remote machine doesn't exist or bad permissions."
  echo "Exiting..."
  exit 2
fi

# Set name (date) of backup.
BACKUP_DATE="`date +%F_%H-%M`"

if [ ! $ROTATIONS -gt 1 ]; then
  echo "You must set ROTATIONS to a number greater than 1!"
  echo "Exiting..."
  exit 2
fi

#### BEGIN ROTATION SECTION ####

BACKUP_NUMBER=1
# incrementor used to determine current number of backups

# list all backups in reverse (newest first) order, set name of oldest backup to $backup
# if the retention number has been reached.
for backup in `ssh -i $RKEY $RUSER@$RMACHINE "ls -dXr $RTARGET/*/"`; do
	if [ $BACKUP_NUMBER -eq 1 ]; then
		NEWEST_BACKUP="$backup"
	fi

	if [ $BACKUP_NUMBER -eq $ROTATIONS ]; then
		OLDEST_BACKUP="$backup"
		break
	fi

	let "BACKUP_NUMBER=$BACKUP_NUMBER+1"
done

# Check if $OLDEST_BACKUP has been found. If so, rotate. If not, create new directory for new backup.
if [ $OLDEST_BACKUP ]; then
  # Set oldest backup to current one
  ssh -i $RKEY $RUSER@$RMACHINE "mv $OLDEST_BACKUP $RTARGET/$BACKUP_DATE"
else
  ssh -i $RKEY $RUSER@$RMACHINE "mkdir $RTARGET/$BACKUP_DATE"
fi

# Update current backup using hard links from the most recent backup
if [ $NEWEST_BACKUP ]; then
  ssh -i $RKEY $RUSER@$RMACHINE "cp -al $NEWEST_BACKUP. $RTARGET/$BACKUP_DATE"
fi

#### END ROTATION SECTION ####

# Check to see if rotation section created backup destination directory
if ! ssh -i $RKEY $RUSER@$RMACHINE "test -d $RTARGET/$BACKUP_DATE"; then
  echo "Backup destination not available."
  echo "Make sure you have write permission in RTARGET on Remote Machin  e."
  echo "Exiting..."
  exit 2
fi

echo "Verifying Sources..."
for source in $SOURCES; do
	echo "Checking $source..."
	if [ ! -x $source ]; then
     echo "Error with $source!"
     echo "Directory either does not exist, or you do not have proper permissions."
     exit 2
   fi
done

if [ -f $EXCLUDE_FILE ]; then
EXCLUDE="--exclude-from=$EXCLUDE_FILE"
fi

echo "Sources verified. Running rsync..."
for source in $SOURCES; do

  # Create directories in $RTARGET to mimick source directory hiearchy
  if ! ssh -i $RKEY $RUSER@$RMACHINE "test -d $RTARGET/$BACKUP_DATE/$source"; then
    ssh -i $RKEY $RUSER@$RMACHINE "mkdir -p $RTARGET/$BACKUP_DATE/$source"
  fi

  rsync $VERBOSE $EXCLUDE -a --delete -e "ssh -i $RKEY" $source/ $RUSER@$RMACHINE:$RTARGET/$BACKUP_DATE/$source/

done

exit 0

Rsync is a powerful tool not only for file transfers, but also for advanced and secure backups to remote machines. If you would like to learn more about backing up with rsync, Mike Rubel provides a great tutorial and reference section on his Web site.

Share    Print    Comments