/ Markus Amersdorfer:home / university / about:me /
\ Say NO to Software-Patents! \


Howto backup to SCSI-tapes using kbackup

General Information

This tool is a frontend for lower-level programs such as tar or afio, gzip, mt and other stuff you can do backups with. According to the description it supports full/incremental backups (with a little trick I got it making differential backups too) as well as multiple volumes/tapes for one backup, to mention just the two most important things.
(I did not try multiple volumes/tapes yet well enough to comment on this option any further.)

PROs:

CONs:

I would recommend kbackup for small to medium sized backups only, due to its way of restoring data. It is probably also best suited for one server, little more if you can nfs- or smb-mount shares from the other servers.
There is a version of kbackup supporting remote backup over networks, but I currently don't know anything about that. Check out the kbackup homepage for details.

This document describes how to get a weekly full and daily differential backup on a SCSI-tape-drive to work.

Add-On -- 03-07-26:
As I've already mentioned above, the Debian packages on kbackup have been orphaned ("It's not maintained anymore by its author and has lots of bugs"). The previous package maintainer Ludovic Drolez focuses on backuppc now.
Backup software other than kbackup includes Arkeia, Amanda, chiark-backup, bacula, and others. You might also want to give an rsync based approach to backup a try.

Installation

Install the Debian-package kbackup. Using Debian 3.0, you'll get KBackup 1.2.11.
You will perhaps need some additional packages such as tar or afio, anyway apt or dselect will tell you so if they are not already installed.
I recommend installing dialog as well in order to ge a nice shell-based GUI.

Configuring kbackup

Overview

Starting kbackup the first time it is a good idea to check out the option Help -- view online manual. You'll find some general information, changelogs and most importantly good descriptions for all of the possible options within kbackup. I didn't find such an extensive documentation elsewhere.

Before successfully being able to do backups or restore data, you will have to configure kbackup: You can save several different configuration settings via the menu Configuration.

Backup and Restore do exactly what you think they do.

Other Actions holds different actions such as calculating the current configuration's backup size, tape operations, and so on.

I didn't get Scheduling menu to work properly. Instead you can perfectly use cron-jobs for doing daily incremental and weekly full backups.

Detecting the tape drive

First, you should check if kbackup can work with your tape-drive.
A great option is rather well hidden: Configure KBackup -- Select device/type for backup -- Attempt to autodetect tape device
Starting this option will go through all available tape-devices (/dev/st[01]* and /dev/nst[01]*) and try to find out if it fits kbackup's needs.
/dev/nst0 as the first non-rewinding tape device is probably the correct one (if you made /dev/tape a link to /dev/nst0 previously, kbackup will of course find your device a "second" time).

After getting a positive response about the drive, here are the most important options for setting up a working backup. (Remember the detailed description of all options accessible through kbackup's Help.)

Menu "Configuration"

Configuration does what it seems to do: configure kbackup according to your needs.
As already mentioned, you can and should use different sets of configurations by simply writing and loading configuration files. You will at least need 2 configurations, one for full and one for incremental backup each.
You best start with a configuration for a full backup and afterwards save this under a second name, just changing one parameter in order to get the same configuration doing incremental backups.

Set parent directory for backup/restore

The parent directy will probably be set to "/", except for testing-purposes with which you are better off setting it to some special directory such as "/backup-tests". All further mentioned paths are relative to the parent directory.

Set file selection parameters

Use Toggle Incremental Mode [No] for full backups and [YES] for incremental backups.

Shell patterns are very useful in order to have some directories and/or files included and some not. For example set home/max/WAVs/* as exclusions and home/* as pattern for files to include. (Mind the not existing "/" at the beginning of these lines!) This way having already set "/" as parent directory you'll backup all users' home-directories, but will not save the space wasting sound-files in directory /home/max/WAVs/ or anything further down this tree.
In order to get this behaviour: Toggle use of include/exclude patterns [Yes].

Protect newer files during restore [Yes] will be important if you want to restore your data starting from the latest (incremental) backup and finishing the restore-process with the huge full backup. This seems kind of a funny direction, but generally saves you time when restoring everything.

Configure KBackup

Create report on backup can be set to [No] if it is enough for you to have log-files being sent by mail when having the jobs automatically started by cron.

According to the documentation, Compression should only be activated when using afio (which is the default) instead of tar. As compression often can safe lot's of space, I would recommend you to activate it and use afio as the archive type.

The device type should of be set to [Tape].

Toggle autorewind [Yes] will automatically rewind the tape when using the dialog-based frontend of kbackup. Doing cron-job'd backups, you will need to manually rewind the tape after the backup has been written to it (see below).

Write, Load and Delete a configuration file

These options deal with kbackup's configuration files. You have for example one for Full and Incremental backup-setting each.
When doing backups or restores via the GUI, be sure to first load the correct config-file after starting kbackup.

Menu "Backup"

I repeat: When doing backups or restores via the GUI, be sure to first load the correct config-file after starting kbackup:

  C  Configuration
  L  Load a configuration file
     [ load your e.g. differential-backups configuration ]
  <  Return to main menu
     [ < No >, don't save ]

Backup can be invoked after loading the correct configuration and will start dumping the backup onto your selected media (in my case: SCSI-tapes).
In this interactive mode using the GUI, the programm will prompt for the next volume if the data needs to be spanned over multiple tapes.

Menu "Restore"

I repeat: When doing backups or restores via the GUI, be sure to first load the correct config-file after starting kbackup:

  C  Configuration
  L  Load a configuration file
     [ load your e.g. differential-backups configuration ]
  <  Return to main menu
     [ < No >, don't save ]

Restore (after loading the correct configuration as has just been described) can either restore

The 1st option explains itself.

By chosing the 2nd option, you can restore just one file of the thousands you backuped. You can then delete all other files from the list within your standard-editor, just leaving the one you would like to be restored in the file when saving and exiting the editor. But watch out: kbackup seems to have a bug here as you will need to remove the initial "./" from the files' pathname as well as the trailing information such as file size, time, etc.

The 3rd option lets you do the same based on a list saved in the log-files. This way you can "cheat" somehow before restoring your data: create a file with the file-names in it you want to be restored (be sure to have the correct path), save it in /var/lib/kbackup/Data/[profile-name]/ with the corresponding naming-convention for log-files and gzip the file. This way, you'll be able to take this new "log" and won't have to edit the perhaps enormous list of files.
(Note: The list of logs presented to you depends on the current configuration loaded before entering the Restore-menu. If you have both a "Full" and a "Differential" backup configuration, when restoring from e.g. the latest differential backup, you have to select this configuration (see above), then enter the restore-menu -- then selecting the log-based filesystem-info only shows you all logs concerning the differential configuration (sorted according to the date of the backup).)

Menu "Other actions"

This is an interesting menu and gives you options such as rewinding the tape, verify backups and above all get the size of all data which are going to be backuped according to the currently loaded configuration.

Menu "Scheduling menu"

The menu-option is actually just for backwards-compatability. Scheduling the jobs should best be done via entries in /etc/crontab and the --auto-backup mode of kbackup's command-line options.

Hints and scripts

Same shell patterns for different configurations

When using shell patterns to do detailed backups, you would have to manage the shell patterns for at least both the Full and the Incremental backup configuration.

As the shell patterns are stored in plain text files under /var/lib/kbackup/Data/[config-name]/, you can create symbolic links in order to change the same files within different configurations:

  /var/lib/kbackup/Data/Incremental/Exclude_Files -> ../Full/Exclude_Files
  /var/lib/kbackup/Data/Incremental/Exclude_Patterns -> ../Full/Exclude_Patterns
  /var/lib/kbackup/Data/Incremental/Include_Files -> ../Full/Include_Files
  /var/lib/kbackup/Data/Incremental/Include_Patterns -> ../Full/Include_Patterns
This way when changing the patterns within the configuration "Full" it will also have effect on the configuration "Incremental" and vice versa.

Differential backups

kbackup basically supports full and incremental backups. When doing incremental backups it checks the file /var/lib/kbackup/Data/Incremental/DateStamp when the last backup was made and includes all files which have been changed since.

By making the Incremental/DateStamp a symbolic link to Full/DateStamp, you will get a backup holding all files that were changed since the last Full-backup.
As running the Incremental-backup will change the DateStamp, you will have to copy the original Full/DateStamp file to Full/DateStamp_orig before starting the incremental backup and copy Full/DateStamp_orig back to Full/DateStamp again after the incremental backup has finished.
See below for a sample shell script.

Shell script for a "Full Backup"

#!/bin/sh
# 
# This script is executed every Friday (cron) and makes a
# FULL BACKUP of all necessary data.
# 

# rewind
/bin/mt -f /dev/nst0 rewind
if [ $? != "0" ]; then
        /bin/echo "Backup:"
        /bin/echo "Error rewinding the tape before backup, ABORTING!"
        exit
fi

# BACKUP:
/usr/sbin/kbackup --auto-backup Full

# rewind and eject
/bin/mt -f /dev/nst0 rewoffl
if [ $? != "0" ]; then
        /bin/echo "Backup:"
        /bin/echo "Error at rewoffl after backup, ABORTING!"
        exit
fi

# test if there was the string "fatal error" in the log, if so, that's not good.
# (e.g. this could happen if a backup tape is set read-only)
logfile=`/bin/ls -h /var/lib/kbackup/Data/NotarFull/Log* | /usr/bin/tail -1 `
/bin/zgrep -i "fatal error" $logfile >& /dev/null
if [ $? == "0" ]; then
        /bin/echo "Backup:"
        /bin/echo "The string \"fatal error\" is in the log file."
        /bin/echo "$logfile is mailed to you."
        /usr/bin/mail -s "Backup log file" your-address@your-domain.org < $logfile
        exit
fi

/bin/echo "Full backup ok!"

Shell script for a "Differential Backup"

#!/bin/sh
# 
# This script is executed Monday to and including Thursday (cron) and does a 
# DIFFERENTIAL BACKUP to the last full backup.
# 
# This is accomplished via the trick to copy the DateStamp-file.
# (kbackup actually only supports incremental backups by default.)
# 

# rewind
/bin/mt -f /dev/nst0 rewind
if [ $? != "0" ]; then
        /bin/echo "Backup:"
        /bin/echo "Error rewinding the tape before backup, ABORTING!"
        exit
fi

# differential (part 1)
/bin/cp -f /var/lib/kbackup/Data/NotarFull/DateStamp /var/lib/kbackup/Data/NotarFull/DateStamp_DIFF-ORIG

# BACKUP:
/usr/sbin/kbackup --auto-backup NotarDifferential

# differential (part 2)
/bin/mv -f /var/lib/kbackup/Data/NotarFull/DateStamp_DIFF-ORIG /var/lib/kbackup/Data/NotarFull/DateStamp

# rewind and eject
/bin/mt -f /dev/nst0 rewoffl
if [ $? != "0" ]; then
        /bin/echo "Backup:"
        /bin/echo "Error at rewoffl after backup, ABORTING!"
        exit
fi

# test if there was the string "fatal error" in the log, if so, that's not good.
# (e.g. this could happen if a backup tape is set read-only)
logfile=`/bin/ls -h /var/lib/kbackup/Data/NotarDifferential/Log* | /usr/bin/tail -1 `
/bin/zgrep -i "fatal error" $logfile >& /dev/null
if [ $? == "0" ]; then
        /bin/echo "Backup:"
        /bin/echo "The string \"fatal error\" is in the log file."
        /bin/echo "$logfile is mailed to you."
        /usr/bin/mail -s "Backup log file" your-address@your-domain.org < $logfile
        exit
fi

/bin/echo "Differential backup ok!"

Backup-Verification

While I didn't try it (yet?), here's the part of a script I received via mail from BuSab on how to get kbackup verify the backups -- many thanks for this hint!:

TAPE=/dev/tape
MULTIBUF=/usr/sbin/multibuf
DATADIR=/data/

echo "skips the contents directory"
cat $TAPE > /dev/null
sleep 60
echo "read headers"
$MULTIBUF -r -h $TAPE | dd bs=4096k > /dev/null
sleep 60
cd $DATADIR
echo "compare tape content and hard drive content" 
$MULTIBUF -r -h $TAPE -m 100 | dd bs=4096k | afio -r -kzZ -

According to the author, this is based on section A.2.3 of the kbackup manual, "Restore my files manually?", by replacing the "-i" option of afio with a "-r" to verify the tape content.


Valid HTML 4.01! Valid CSS! Created with Vim [Blue Ribbon Campaign icon]
© Markus Amersdorfer
last modified: Tuesday, 23-Feb-2010 15:42:17 UTC