My backups with rsync
In previous posts I have introduced the use of rsync for making incremental backups, and then mentioned an event of making use of such backups. However, I have realized that I haven’t actually explained my backup scheme! Let’s go for it:
Backup plan
I make a backup of my $home directory, say /home/isilanes. Each “backup” will be a set of 18 directories:
- Current (last day)
- 7 daily
- 4 weekly
- 6 monthly
Each such dir has an apparent complete copy of how /home/isilanes looked like at the moment of making the backup. However, making use of hard links, only the new bits of info are actually written. All the parts that are redundant are written once on disk, and then linked from all the places referring to it.
Result: a 18 copies of a $home of 3.8 GB in a total of 8.7 GB (14% of the apparent size of 63 GB, and 13% of 18x the info size, 68,4 GB).
Perl script for making the backup
Update (Jun 5, 2008): You can find a much refined version of the script here. It no longer requires certain auxiliary script to be installed in the remote machine, and is “better” in general (or it should be!)
Below is the commented Perl script I use. Machine names, directories and IPs are invented. Bart is the name of my computer.
#!/usr/bin/perl -w
use strict;
my $rsync = "rsync -a -e ssh --delete --delete-excluded";
my $home = "/home/isilanes";
my $logfile = "$home/.LOGs/backup_log";
#
# $where -> where to make the backup
#
# $often -> whether this is a daily, weekly or monthly backup
#
my $where = $ARGV[0] || 'none';
my $often = $ARGV[1] || 'none';
my ($source,$remote,$destdir,$excluded,$to,$from);
# Possible "$where"s:
my @wheres = qw /machine1 machine2/;
# Possible "$often"s:
my @oftens = qw /daily weekly monthly/;
# Check remote machine:
my $pass = 0;
foreach my $w (@whats) { $pass = 1 if ($what eq $w) };
die "$what is an incorrect option for \"what\"!\n" unless $pass;
# Check how-often:
$pass = 0;
foreach my $o (@oftens) { $pass = 1 if ($often eq $o) };
die "$often is an incorrect option for \"often\"!\n" unless $pass;
# Set variables:
if ($what eq 'machine1')
{
# Defaults:
$source = $home;
$remote = '0.0.0.1';
$destdir = '/disk2/backup/isilanes/bart.home.current';
$excluded = "--exclude-from $home/.LOGs/excludes_backup.dat";
$to = 'machine1';
$from = 'bart';
}
elsif ($what eq 'machine2')
{
# Defaults:
$source = $home;
$remote = '0.0.0.2';
$destdir = '/scratch/backup/isilanes/bart.home.current';
$excluded = "--exclude-from $home/.LOGs/excludes_backup.dat";
$to = 'machine2';
$from = 'bart';
}
# Do the job:
unless ($what eq 'none')
{
unless ($often eq 'none')
{
# Connect to the remote machine, and run ANOTHER script there, making a rotation
# of the backup dirs:
system "ssh $remote \"/home/isilanes/MyTools/rotate_backups.pl $often\"";
# Actually make the backup:
system "$rsync $excluded $source/ $remote:$destdir/";
# "touch" the backup dir, to give it present timestamp:
system "ssh $remote \"touch $destdir\"";
# Enter a line in the log file defined above ($logfile):
&writelog($from,$often,$to);
};
};
sub writelog
{
my $from = ucfirst($_[0]);
my $often = $_[1];
my $to = uc($_[2]);
my $date = `date`;
open(LOG,">>$logfile");
printf LOG "home@%-10s %-7s backup at %-10s on %1s",$from,$often,$to,$date;
close(LOG);
};
As can be seen, this script relies on the remote machine having a rotate_backups.pl Perl script, located at /home/isilanes/MyTools/. That script makes the rotation of the 18 backups (moving current to yesterday, yesterday to 2-days-ago, 2-days-ago to 3-days-ago and so on). The code for that:
#!/usr/bin/perl -w
use strict;
# Whether daily, weekly or monthly:
my $type = $ARGV[0] || 'daily';
# Backup directory:
my $bdir = '/disk4/backup/isilanes/bart.home';
# Max number of copies:
my %nmax = ( 'daily' => 7,
'weekly' => 4,
'monthly' => 6 );
# Choose one of the above:
my $nmax = $nmax{$type} || 7;
# Rotate N->tmp, N-1->N, ..., 1->2, current->1:
system "mv $bdir.$type.$nmax $bdir.tmp" if (-d "$bdir.$type.$nmax");
my $i;
for ($i=$nmax-1;$i>0;$i--)
{
my $j = $i+1;
system "mv $bdir.$type.$i $bdir.$type.$j" if (-d "$bdir.$type.$i");
};
system "mv $bdir.current $bdir.$type.1" if (-d "$bdir.current");
# Restore last (tmp) backup, and then refresh it:
system "mv $bdir.tmp $bdir.current" if (-d "$bdir.tmp");
system "cp -alf --reply=yes $bdir.$type.1/. $bdir.current/" if (-d "$bdir.$type.1");
