The home user how to make backups as easy as:
#backupninja -n
*Overview
Backups the never ending problem. Follow this How-To if you have:
a) a cd/dvd burner, (to use backupninja's makecd) OR
b) two linux pc's on a LAN, (to use backupninja's rdiff) OR
c) want an example of backupninja you can adapt to use:
-- a spare hard drive be it internal or usb (you can rdiff to a local directory/partition)
-- push backups over the net to another pc/server (that you have permissions to ssh into) (rdiff if trusted or duplicity to encrypt the data)
We will install, setup and use backupninja to create backups over a LAN and/or create iso images you can burn to cd/dvd using your favourite burning program.
Note: If mixing i386/amd64 and etch/sarge then see this post
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
*Information
There is very good documentation at backupninja and a guide to usage at debianhelp.co.uk (thanks to david23 for the link) and of course:
$man backupninja
$man ninjahelper
This excellent documentation and the way backupninja integrates backup into the sytem while providing multiple ways to backup at the same time are all good reasons to use it.
This How-To will not attempt to cover all that is available in the above, rather it aims to set up a easy backup system using cd/dvd and/or another pc on a home LAN.
Next Installation
How to Backup Debian with Backupninja - home user
If using Sarge:
I would suggest the backports version as it has ninjahelper which is an ncurses helper that sets up a few things like ssh keys the easy way.
You can install the stable sarge versions of rdiff-backup, duplicity, hwinfo, mkisofs, cdrecord, dvd+rw-tools which are suggests and any depends you don't already have.
Backports instructions will show how to use backports safely.
Details:
#apt-get update
#apt-get install backupninja rdiff-backup duplicity hwinfo mkisofs cdrecord dvd+rw-tools
Next Set up
How to Backup Debian with Backupninja - home user
What good would making backups be if you never tested the restore function or didn't even know where it was? Kind of like making a boot/rescue floppy and never booting with it till the system won't start on it's own any more! Do you remember how that story ended?
Unfortunately for us there is no nice graphical or semi-graphical helper that I could find at this time in Sarge to do this. There are a few being developed but they are in alpha stage at this time, not what you want to trust data to.
So it is left to the independant tools used and mostly command line stuff ie. read the man page. This is the reason I used 'rdiff-backup' and 'makecd' methods since they can partly use standard copy commands and your favourite filemanager.
makecd
Test the cd/dvd after burning, use the check data function of the burner software during burning. Place it in another drive and mount it. Copy random files to a temp folder and see if everything is ok. Optical disks have a limited lifespan, have you checked any disks you made 6+ years ago lately?
rdiff-backup
The beauty of rdiff-backup is that it creates a mirror and diffs. For the mirror normal tools can be used to copy the data to a test directory. Try using fish in konqueror over the network to copy a file out of /backup on the host machine to a local temp folder. Enter this at the location bar : "fish://root@hostIP" or "fish://user@rhostname" you need the remote password for that user and ssh access through any firewalls. If you know of any other programs that use fish insert here.
rdiff-backup can also be called upon to do time based rollback, see the man page and be carefull, test in a temp folder and don't overwrite anything it will happily whipe out whole directory tree's if you use --force, remember linux is a unix it thinks you know what you want to do.
How to Backup Debian with Backupninja - home user
The rdiff-backup set up bellow is based around having two linux pc's on a LAN.
I have my main box and my wifes laptop connnected via a LAN HS switch to each other and to an ADSL modem/router. She keeps my backups and I keep hers which include the family /pics database that she maintains.
Or if you have a trusted friend you could keep each others backups by using rdiff-backup (secure since it uses ssh) over the net or better still duplicity which encrypts the data then you don't have to trust them. Restoring data with duplicity may prove harder since it is not a plain mirror.
However rdiff-backup may just as easily be used locally to an internal hard drive or a usb drive mounted at /backup. Use a different unit from the data source otherwise you won't protect against disk failure or a seperate partition mounted at /backup and then use makecd to create iso images to burn to cd/dvd size allowing. Take these to work and stash them in your desk in case of fire/theft.
See the makecd section if you just want to prepare iso images for burning to cd/dvd. I have not tested if backupninja splits the images to the correct size for the type of disk.
//////////////////////////////////////////////////////////////////////////
Details:
/etc/backupninja.conf
comment out (place # in front) or change these lines as needed using your favourite text editor:
#reportemail = root
usecolors = yes
now it turns out the when line "when = everyday at 01:00" is the script default and will not stop backup attempts if simply commented out. So if you want to stop these automatic backup attempts then the cron job that is run every hour needs to be commented out:
/etc/cron.d/backupninja
#0 * * * * root if [ -x /usr/sbin/backupninja ]; then /usr/sbin/backupninja; fi
__Logic__
Since this is a home machine and not likely to be on and unused at 01:00 every night I don't want a string of failed backup attempts. With this setup we will run the backup when we are good and ready ie when the source and destination machines are on and unused (only light loaded is fine too) and the correct cd/dvd is in the drive.
Have you ever read root's mail as a home user? If you do then leave it as is. Otherwise /var/log/backupninja.log is the place to look, to ensure the backup completed every time you run:
#backupninja -n
color is up to you, I have a yellow root terminal backround so I set it to 'no'
//////////////////////////////////////////////////////////////////////////
/etc/backup.d/
We have installed a version of backupninja that includes ninjahelper in the Installation section of this How-To so that is what we will use to populate this directory (roll down to next section). Or you can copy files from /usr/share/doc/backupninja/examples or get my examples here.
-Note do not leave old files lying around it this directory as backupninja will not like it and tries to execute everything that does not have a .disabled extension.
-If your text editor likes to leave .old .bak or text.txt~ laying around then clean up behind it or turn that feature off!
-Files are executed in order so :
10.sys.disabled
20.makecd.disabled
30.sh
90.rdiff
91.rdiff
will try to run: 10 (but disabled) then 20 (but disabled) then 30.sh which does things that will be used in 90.rdiff then 91.rdiff.
_______________________________________________________________________
Just run ninjahelper, as root, in your favourite term (xterm, konsole etc):
#ninjahelper
An ncurses program should appear.
Select 'new' -> 'sys' use spacebar to toggle selection:
[x] packages
[x] partitions
[x] hardware
hit enter, a new selection should appear in the main menu
: 1 '/etc/backup.d/10.sys'.
_______________________________________________________________________
Just run ninjahelper, as root, in your favourite term (xterm, konsole etc):
#ninjahelper
Select 'new' -> 'makecd' answer questions:
>cd or dvd
>y
>/home/backupninja (if you have enough space here or /tmp/backupninja)
>backup.iso
>/dev/hdx (not required as we only making iso)
>/home/your_user_name (or /backup)
>enter any excludes like /home/user/Desktop/Trash
hit enter, a new selection should appear: 2 '/etc/backup.d/20.makecd'.
Select this and press enter again then 'run'
Use your favourite burning app to burn /home/backupninja/backup.iso to a disk.
_______________________________________________________________________
Just run ninjahelper, as root, in your favourite term (xterm, konsole etc):
#ninjahelper
Select 'new' -> 'rdiff' -> 'src' add include lines:
include = /root
include = /boot
include = /etc
include = /var/backups
include = /var/lib/dpkg/status*
include = /home
#or
#include = /home/'user1'
#include = /home/'user2'
>hit enter add exclude lines:
exclude = /home/*/.gnupg
exclude = /home/*/.local/share/Trash
exclude = /home/*/.Trash
exclude = /home/*/.thumbnails
exclude = /home/*/Desktop/Trash
exclude = /home/*/.mozilla/*/*/Cache
exclude = /home/*/.mozilla/*/*/Cache.Trash
exclude = /home/*/.beagle
exclude = /home/*/.aMule
exclude = /home/*/gtk-gnutella-downloads
exclude = /home/*/.giFT
exclude = /root/*/.gnupg
exclude = /root/*/.local/share/Trash
exclude = /root/*/.Trash
exclude = /root/*/.thumbnails
exclude = /root/*/Desktop/Trash
exclude = /root/*/.mozilla/*/*/Cache
exclude = /root/*/.mozilla/*/*/Cache.Trash
>hit enter 'des' add destination lines:
60D
/backup/'hostname' (you local hostname will appear here)
IP_of_dest_host (10.0.0.2 etc or hostname like 'otherbox' as found in /etc/hosts)
root
>hit enter 'conn' this will then test the destination and transfer the ssh keys. Follow the messages all must go well here or you won't be able to transfer data to the destination. Create the directory if it asks.
3 '/etc/backup.d/90.rdiff should appear in the main menu.
_______________________________________________________________________
Finished
You are now ready to run each menu item by selecting it, press enter, and selecting 'run'.
Or to run all the enabled actions when it suites you (say weekly).
Open your favourite term (xterm, konsole etc) and do:
$su
Password:
#backupninja -n
When backupninja is finished (the prompt returns) always check that the backup has worked by reading /var/log/backupninja.log and checking the destination and testing Restore.
Make it a habit to do the backup at the intervals that you have decided, the more frequently the smaller and quicker the rdiff will run.
Backupninja will attempt to continue if the network drops out and returns. But if you 'Ctrl+C' it's ass then it obviously won't finish and rdiff-backup's need to be run initially at least twice successfully, once to create the mirror then once to create the diff so do it twice for this first time.
A first run of rdiff backup took me 1h34 (3.62GB) of pics across 100M LAN through a switch. The second run takes less than 2min.
Next Restore
#/etc/backup.d/10.sys.disabled
##### Gav - Disabled as I don't like running sfdisk every back up
##### too easy to partition the disk instead of dumping partition table.
##### See the shell script for dpkg selections and the much safer df.
##### Run only once on a new system/disk after running the other backups first!
# this config file will save various reports of vital system information.
# by default, all the reports are enabled and are saved in /var/backups.
# requires dpkg, sfdisk, and hwinfo
# (1) a list of all the packages installed and removed.
# this file can be used to restore the state of installed packages
# by running "dpkg --set-selections < dpkg-selections.txt
# (2) the partition table of all disks.
# this partition table can be used to format another disk of
# the same size. this can be handy if using software raid and
# you have a disk go bad. just replace the disk and partition it
# by running "sfdisk /dev/sdb < partitions.sdb.txt"
# (MAKE SURE YOU PARTITION THE CORRECT DISK!!!)
# (3) hardware information.
# detailed information on most important aspects of the hardware.
# here the defaults were commented out:
packages = yes
# packagesfile = /var/backups/dpkg-selections.txt
partitions = yes
# partitionsfile = /var/backups/partitions.*.txt
hardware = yes
# hardwarefile = /var/backups/hardware.txt
#/etc/backup.d/20.makecd.disabled
# TYP is cd or dvd AS WELL AS the disk inside!!
burnertype = dvd
# not yet supported
system = no
# iso or burn to cd/dvd?
isoonly = yes
# location for image file
# where you have sufficient space
backupdir = /tmp/backup
# image filename
imagefile = bak_hostname_24Jul06.iso
# cd/dvd burner device
device = /dev/hdx
# dirs/files to include in the backup
# or /home/user or what you want to backup
target = /backup
# directories/files to be excluded
exclude = /proc
exclude = /sys
exclude = /dev
#/etc/backup.d/30.sh
# This script can run most any command normally run on the command line.
# Add anything usefull that you would like to do during a backup. But all it does as it stands
#is place files in /var/backup. So these will still need to be backed up by rdiff and friends
# (1) a list of all the packages installed and removed.
# this file can be used to restore the state of installed packages
# by running "dpkg --set-selections < dpkg-selections.txt"
dpkg --get-selections > /var/backups/dpkg-selections.txt
# (2) used with along with /etc/fstab and /var/backups/partitions.hdx.txt
# to decide on how to partition a new disk during a recovery.
# possibly by running #"sfdisk /dev/sdx < partitions.hdx.txt"
# (MAKE SURE YOU PARTITION THE CORRECT DISK!!!)
df > /var/backups/df.txt
#/etc/backup.d/90.rdiff
# options = --force
# when = everyday at 02
[source]
type = local
keep = 60D
include = /root
include = /boot
include = /etc
include = /var/backups
include = /var/lib/dpkg/status*
include = /home
#or
#include = /home/'user1'
#include = /home/'user2'
#/home excludes
exclude = /home/*/.gnupg
exclude = /home/*/.local/share/Trash
exclude = /home/*/.Trash
exclude = /home/*/.thumbnails
exclude = /home/*/Desktop/Trash
exclude = /home/*/.mozilla/*/*/Cache
exclude = /home/*/.mozilla/*/*/Cache.Trash
exclude = /home/*/.beagle
exclude = /home/*/.aMule
exclude = /home/*/gtk-gnutella-downloads
exclude = /home/*/.giFT
#/root excludes
exclude = /root/*/.gnupg
exclude = /root/*/.local/share/Trash
exclude = /root/*/.Trash
exclude = /root/*/.thumbnails
exclude = /root/*/Desktop/Trash
exclude = /root/*/.mozilla/*/*/Cache
exclude = /root/*/.mozilla/*/*/Cache.Trash
######################################################
## destination section
## (where the files are copied to)
[dest]
type = remote
directory = /backup/'local_hostname'
host = 'dest IP or hostname'
user = root