I just wrote a little bash script for rsync push backups of my laptop to my Synology-Diskstation at home.
Since I am a bash beginner and don't have any experience with ssh, rsync and backups I am asking for suggestions how to improve the performance, reliablility, usability and security of my script and for suggestions to improve the coding style. I have listed some problems and notes below.
The hardrive of my laptop is encrypted and the backup should be encrypted as well.
Here is the script:
#!/bin/bash bold=`tput bold` normal=`tput sgr0` dirlist=/home/myuser/bin/dirlist pass=mypassword date=`date "+%Y-%m-%dT%H_%M_%S"` echo "${bold}Mounting encrypted backup volume${normal}" ssh -n -i /home/myuser/.ssh/id_rsa [email protected] "/usr/syno/sbin/synoshare --enc_mount MyuserBackup $pass" sleep 5 echo "${bold}Checking and eventually creating dirs on backup server${normal}" ## Creating directories while read dir; do echo "Create incomplete$dir if not existent" ## http://stackoverflow.com/questions/9393038/ssh-breaks-out-of-while-loop-in-bash ssh -n -i /home/myuser/.ssh/id_rsa [email protected] "mkdir -p /volume1/MyuserBackup/rsync_Backup/incomplete$dir" done < $dirlist while read dir; do echo "${bold}Backing up $dir${normal}" time sudo rsync -avX -u --compress-level=0 --stats -h --exclude-from=/home/myuser/bin/excludelist --numeric-ids "$@" --link-dest=/volume1/MyuserBackup/rsync_Backup/current$dir --rsh="ssh -i /home/myuser/.ssh/id_rsa" $dir [email protected]:/volume1/MyuserBackup/rsync_Backup/incomplete$dir done < $dirlist echo $date echo "${bold}Updating Date-Structure${normal}" echo "mv /volume1/MyuserBackup/rsync_Backup/incomplete /volume1/MyuserBackup/rsync_Backup/back-$date" ssh -t -i /home/myuser/.ssh/id_rsa [email protected] "mv /volume1/MyuserBackup/rsync_Backup/incomplete /volume1/MyuserBackup/rsync_Backup/back-$date" echo "rm -f /volume1/MyuserBackup/rsync_Backup/current" ssh -t -i /home/myuser/.ssh/id_rsa [email protected] "rm -f /volume1/MyuserBackup/rsync_Backup/current" echo "ln -s /volume1/MyuserBackup/rsync_Backup/back-$date /volume1/MyuserBackup/rsync_Backup/current" ssh -t -i /home/myuser/.ssh/id_rsa [email protected] "ln -s /volume1/MyuserBackup/rsync_Backup/back-$date /volume1/MyuserBackup/rsync_Backup/current" echo "Showing the current dir structure:" ssh -t -i /home/myuser/.ssh/id_rsa [email protected] "ls -lhrt /volume1/MyuserBackup/rsync_Backup/" echo "${bold}Unmounting encrypted volume${normal}" ssh -t -i /home/myuser/.ssh/id_rsa [email protected] "/usr/syno/sbin/synoshare --enc_unmount MyuserBackup" cat dirlist
/home/ /etc/ /var/ cat excludelist
## Universal excludes lost+found ld.so.cache # backup text files (e.g. from Emacs) - *~ - \#*\# # Commonly distributed Windows cache - Thumbs.db # Common package managers (apt, yum) - *cache/apt/ # sshfs - box1 - box2 - box3 - box4 # Cache - .cache/ - **cache # Temporary files / cache - .local/share/Trash - .cache - .Trash - **.Trash - **.macromedia - **.thumbnails - .local/share/gvfs-metadata - .aptitude - .pulse - .pulse-cookie - .xournal - .sage/temp - tmp - .mozilla/firefox/*/Cache - .mozilla/firefox/*/minidumps # in case Fx crashes dumps will be stored in this - .mozilla/firefox/*/.parentlock # session-specific - .mozilla/firefox/*/urlclassifier3.sqlite # phishing database, recreated - .mozilla/firefox/*/blocklist.xml # blacklisted extensions - .mozilla/firefox/*/extensions.sqlite # extension database, recreated on startup - .mozilla/firefox/*/extensions.sqlite-journal - .mozilla/firefox/*/extensions.rdf - .mozilla/firefox/*/extensions.ini - .mozilla/firefox/*/extensions.cache - .mozilla/firefox/*/XUL.mfasl # cached UI data, recreated - .mozilla/firefox/*/XPC.mfasl - .mozilla/firefox/*/xpti.dat - .mozilla/firefox/*/compreg.dat - .compiz - .dbus # Private directory from ecryptfs - .Private # X Windows System - .xsession-errors* # Gnome temp stuff - .compiz*/session - .gksu.lock - **.gvfs # Common Applications # Adobe Reader - */.adobe/**/AssetCache/ - */.adobe/**/Cache/ - */.adobe/**/Temp/ - */.adobe/**/UserCache.bin # Gimp - /.gimp-*/tmp - /.gimp-*/swap # Mozilla Firefox - .mozilla/firefox/*/Cache/ - .mozilla/firefox/*/lock - .mozilla/firefox/*/.parentlock # Mozilla Thunderbird - .mozilla-thunderbird/*/lock - .mozilla-thunderbird/*/.parentlock - *recently-used* Notes and Problems
- Usually I execute the script with sudo, but sometimes it asks for the sudo password again when it comes to the next item in dirlist. I guess this is due to the fact that the sudo password is only saved for some time, 5 minutes or so. However this is bad since I don't want to control what the script is doing all the time.
Sometimes it gives something like
rsync: send_files failed to open "/home/myuser/.thunderbird/myprofile.default/session-6.json": Input/output error (5)This happens only with some "exotic" files like the file above.
Sometimes it prints errors like
rsync: mkstemp "/volume1/MyuserBackup/rsync_Backup/incomplete/home/" failed: No such file or directory (2)However I didn't find any files which were not transferred.
Is there some better way to handle the password instead of writing it in plaintext into the script?
The script seems to be really slow even if nothing changed on the disk it takes over 30 min
Currently I thin out the snapshots manually but I am thinking of writing a program which do this automatically. Something like this.
Versions
Here are the versions of the used software:
- bash: 4.2.8(1)
- rsync: 3.0.7
- synology DSM: 4.2
- OpenSSH 5.8p1 Debian-1ubuntu3
Hardware
- Disk-Station
- Laptop CPU: Dual-Core P8700
