Linux - Automate Backups with rsync and cron (Advanced Guide)
Intro
This guide demonstrates how to automate backups using rsync and cron, with advanced techniques for local and remote backups, incremental backups, and retention policies. By the end, you’ll have a robust backup solution tailored to your needs.
Step 1: Install rsync
Ensure rsync is installed on your system. For Debian-based systems:
1
2
sudo apt update
sudo apt install rsync
For Red Hat-based systems:
1
sudo yum install rsync
Step 2: Set Up SSH for Remote Backups (Optional)
To back up to a remote server, configure SSH key-based authentication:
Generate an SSH key pair if you don’t already have one:
1
ssh-keygen -t rsa -b 4096 -C "[email protected]"
Copy the public key to the remote server:
1
ssh-copy-id username@remote_server_ip
Test the connection:
1
ssh username@remote_server_ip
This ensures passwordless login for secure and automated backups.
Step 3: Create a Backup Script
Create a script to define what to back up, where, and how. This script will include logging, error handling, and optional email notifications.
Example Script
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
#!/bin/bash
# Variables
SOURCE="/home/username/"
DESTINATION="/mnt/backup_drive/"
LOGFILE="/var/log/backup.log"
EMAIL="[email protected]"
# Function to send email notifications (optional)
send_email() {
mail -s "Backup Status" "$EMAIL" < "$LOGFILE"
}
# Start backup process
echo "Backup started at $(date)" > "$LOGFILE"
rsync -av --delete --exclude="*.tmp" "$SOURCE" "$DESTINATION" >> "$LOGFILE" 2>&1
if [ $? -eq 0 ]; then
echo "Backup completed successfully at $(date)" >> "$LOGFILE"
send_email
else
echo "Backup failed at $(date)" >> "$LOGFILE"
send_email
exit 1
fi
Make the script executable:
1
chmod +x ~/backup.sh
Step 4: Schedule Backups with cron
Edit the crontab file:
1
crontab -e
Examples of Schedules:
- Daily at midnight:
1
0 0 * * * /home/username/backup.sh
- Every Sunday at 2 AM:
1
0 2 * * 0 /home/username/backup.sh
- Every hour:
1
0 * * * * /home/username/backup.sh
Save and exit. To verify the cron job is active, use:
1
crontab -l
Step 5: Advanced Backup Features
Incremental Backups with --link-dest
To save space while maintaining multiple backup versions, use incremental backups:
Create a directory structure for snapshots:
1
mkdir -p /mnt/backup_drive/{current,incremental}
Update the script to use
--link-dest:1 2 3
rsync -av --delete --link-dest=/mnt/backup_drive/current "$SOURCE" /mnt/backup_drive/incremental/$(date +%Y-%m-%d) >> "$LOGFILE" 2>&1 cp -al /mnt/backup_drive/incremental/$(date +%Y-%m-%d) /mnt/backup_drive/current >> "$LOGFILE" 2>&1
Remote Backups with SSH Tunnel:
For remote backups:
1
rsync -avz -e "ssh -i ~/.ssh/id_rsa" --delete "$SOURCE" username@remote_server:/path/to/destination >> "$LOGFILE" 2>&1
Retention Policy for Old Backups:
Use find to delete old backups automatically:
1
find /mnt/backup_drive/incremental/* -mtime +30 -exec rm -rf {} \;
This keeps backups from the last 30 days.
Step 6: Verify Backups
Regularly verify that backups are working as expected:
Check the log file:
1
cat /var/log/backup.logTest restoring files:
1
rsync -av /mnt/backup_drive/current/ /path/to/test_restore/
Step 7: Troubleshooting
Common Issues:
- Cron job not running: Ensure the script has executable permissions and uses absolute paths.
- Permission errors: Run the script as a user with sufficient permissions or use
sudo. - SSH issues: Verify SSH keys are configured correctly.
Debugging Tips:
- Add debugging options like
-vor--progresstorsync. Redirect cron errors to a log file:
1 2 3 4
crontab -e # Example with error logging: 0 0 * * * /home/username/backup.sh >> /var/log/cron_backup.log 2>&1
Conclusion
With this advanced setup, you now have a flexible and reliable backup system using rsync and cron. Customize it further based on your specific requirements, such as encryption or cloud integration. Always test your backups regularly!