How to make a daily copy of backup files from a webserver on Debian to a local NAS using rsync over SSH.
On my webserver I make a dump of most databases every day, using a command like this:
mysqldump dbName --opt --user=dbUsername --password=dbPassword | gzip > "/home/backups/dbName/dbName_`date +%d`.sql.gz"
This creates nicely gzipped files, one for each day of the month. The command is in a shell script which is executed by a cron task. As an extra backup, I wanted to have a copy of these files on a different server: my Qnap NAS. The NAS runs a version of Linux, so all required tools are present: rsync, SSH and cron. rsync will compare the local and remote directories and synchronize them as needed. It’s a smart tool: it only copies new or modified files, saving time and bandwidth.
Set up the rsync script
Log in to the NAS as the admin user, preferably over SSH with a tool like Putty.
On the NAS, I created a shell script backup.sh
containing these lines:
#!/bin/sh
rsync -ae ssh tom@www.example.com:/home/backups/ /share/toms/serverbackups/
-a
puts rsync into archive mode, which makes the operation recursive and keeps timestamps, amongst others.
-e ssh
makes rsync run through an SSH tunnel, making the operation pretty secure.
The command will sync /home/backups/
on www.example.com
with /share/toms/serverbackups
on the NAS. toms
is a shared folder on the NAS; serverbackups
is where I want to store the backups. tom
is the user on the webserver who created and owns the database backups.
You’ll need to write the script using the lovely vi
, which is the only text editor available on Qnap NAS. Or just create the script on your PC and copy it to the NAS. (Update 4-4-2016: don’t do that. It may append unwanted characters to the command, such as \#015
. Use vi on the NAS.)
Store backup.sh somewhere in a shared folder but definitely not in the root folder, as that one is recreated at every server reboot. I stored it in /share/toms/scripts/
and made it executable to admin only with
chmod u+x /share/toms/scripts/backup.sh
Check that the rsync command works by running it from the NAS command line. It should prompt for tom
‘s server password, and then start copying files.
If that works, the next step is to enable login without you entering the password all the time: authorize with keys.
SSH keys
On the NAS, run the ssh-keygen
command to generate a set of keys. It will prompt for a passphrase: don’t enter anything, just hit enter.
Copy the contents of ~/.ssh/id_rsa.pub
(the public key) on the NAS to /home/tom/.ssh/authorized_keys
on the webserver. Create the directory and file if they don’t yet exist and set tom as the owner. chmod
the directory to 0700
and the file to 0600
, so only the user tom can read them.
Next, test the password-free login:
ssh -l tom www.example.com
If that works, the next step is to automate the task.
Setting up the cron task on the Qnap NAS
Run this to add a command to the crontab’s config file:
echo "45 15 * * * /share/toms/scripts/backup.sh" >> /etc/config/crontab
The command will run the script every day at 15:45.
Finally, reload the running cron daemon with the altered config file:
crontab /etc/config/crontab
This way our added command will persist over NAS reboots, contrary to using crontab -e
.
Very good tutorial.
I got one problem though – I can’t get cron to work. I’ve tried for 2 days now, but it just won’t run!
If I bash it manually it works, but the cron won’t work.
This is the script I use: https://gist.github.com/enoch85/45eba73c49f760905bc2 and this is the line in cron:
25 23 * * * /bin/sh /share/CACHEDEV1_DATA/FSGO/backup.sh
Thanks!
There’s a whitespace in the cron line:
/bin/sh[space]/share
.Otherwise, can you get other cron jobs running?
I was stuck in the middle (after SSH key) left with the question how to combine the ssh login line with the rsync line. Not hindered with a lot of knowledge I came up with the working command line:
rsync -avvvz -e “ssh -i /homes/admin/.ssh/key_rsa” tom@www.example.com:/home/backups/ /share/toms/serrverbackups/
Bert
Great post – thank you.
Quick question though: on my qnap the owner and group are lost after transfer. it now just has a number there for both. Any suggestions what is missing?
Sorry, no…
Ok. in case I figure it out – i’ll let you know.
Hello Mike,
use –numeric-ids to get the original id of the owner and group of your server from where you sync your data.
Hello, thanks for the great tutorial. I had a couple of issues when trying to replicate your solution.
First, when I generated keys with ssh-keygen I had to relink the id_rsa and id_rsa.pub symlinks on Qnap to the newly generated files. I think this depends on the name of the admin user. Otherwise the rsync command could not log onto the server using keys.
If you need to have a complete sync’ed copy of your source structure I think it’s better to (re)write the rsync command as
rsync –delete-after -ae …
I mean same folder structure, same files, same permissions and dates.
And finally I had to check and adjust permissions over the source structure, and it seems 644 is the best choice to make rsync work without issues.
Thanks again for the tutorial.
Ciao Luciano