We all know back-up is important and a must have in case of critical systems.
Making a backup on tape is one of the best industry standards that is still unmatched.
In the following I am presenting a simple way to make tape back-ups using tar and LTFS on LTO5 and up tapes.
Why LTFS , because it will be easier to access the individual back-up files in case you have several automatic back-up scripts saving data on the same tape.
The back-up I am setting up has several different sets of data:
- WordPress installation directory, less then 100MB
- WordPress database dump, less then 100MB
- OwnCloud installation directory, less then 100MB
- OwnCloud database dump, less then 100MB
- OwnCloud data directory, around 900MB
1-4 will be done as a full back-up each day. 5 will be done as a full back-up on Monday morning ( 3.a.m) and incremental back-up for the other days.
LTFS on LTO5 allows a maximum of 1330MB on a tape so gives me quite a big space without having to switch tapes daily.
I can use one tape for a week with a minimum rotation of 2 tapes. Why minimum of 2 tapes ?, because you want to have at least one full back-up on hand. If you want to have a longer history for your files you can increase the size of the tape set to 4 or 5 so that you have at least one whole month of history.
The extra benefit is that I need to change the tape only on Sunday so my trips to my data-center are limited.
STEP 1: Prepare the tape (to be done each Sunday afternoon)
### Every Sunday
# Unmount the tape if needed
umount /media/lto5
# Load a new tape
# Format tape as ltfs, this will erase all data
mkltfs -d /dev/st0 --force
ltfs -o devname=/dev/st0 /media/lto5
After executing the above steps you new LTFS tape is loaded under the /media/lto5 directory.
STEP 2: Setup the cron job for back-ups 1-4 (to be done once)
Add the following script under /etc/cron.daily
#!/bin/bash
# Path to binaries
TAR=/bin/tar
TAR_ARGS="--totals --blocking-factor=1024"
NOWYYYYMMDD=$(date -I)
OWNCLOUDTARNAME=$NOWYYYYMMDD.owncloud.tar
WORDPRESSTARNAME=$NOWYYYYMMDD.wordpress.tar
# owncloud directory backup
$TAR -czvf "/media/storage/backup/owncloud/$(date '+%F')-owncloud.tgz" /media/storage/www/html/owncloud/owncloud
# owncloud database backup
mysqldump -u homeclouddbuser -pxxxxx homeclouddb | gzip > "/media/storage/backup/owncloud/$(date '+%F')-ownclouddb.sql.gz"
# wordpress directory backup
$TAR -czvf "/media/storage/backup/wordpress/$(date '+%F')-wordpress.tgz" /media/storage/www/html/owncloud/wordpress
# wordpress database backup
mysqldump -u wpuser -pxxxxx wordpress | gzip > "/media/storage/backup/wordpress/$(date '+%F')-wordpressdb.sql.gz"
if mountpoint -q /media/lto5; then
$TAR $TAR_ARGS -cpf /media/lto5/$OWNCLOUDTARNAME "/media/storage/backup/owncloud/$(date '+%F')"*
$TAR $TAR_ARGS -cpf /media/lto5/$WORDPRESSTARNAME "/media/storage/backup/wordpress/$(date '+%F')"*.....
else
echo "Tape not loaded skipping copy to tape"
fi
This is a very simple script that will perform 1-4 back-ups daily and dump the contents as tar files on the LTFS tape.
Note:
- we make first a dump of the data on another back-up location, on another phzsical disk “/media/storage/backup” for both wordpress and owncloud. This is to ensure we have a handy back-up for this critical elements on the server itself
- In a second step the already dumped data is tar-ed to the LTFS tape
- TAR_ARGS=”–totals –blocking-factor=1024″ that ensure that a proper speed of writing to the LTO tape is maintained and avoid shoe-shining. The blocking factor of 1024 is best for the LTO5 tapes . For higher LTO versions higher values may be more appropriate.
STEP 3: Setup the cron job for back-ups 5 (to be done once)
#!/bin/bash
# Log base directory
LOGBASE=/root/log
# Backup dirs; do not prefix /
BACKUP_ROOT_DIR="data"
BACKUP_PATH="/media/storage/www/"
# Get todays day like Mon, Tue and so on
NOW=$(date +"%a")
NOWYYYYMMDD=$(date -I)
# tar file names template
TARNAMEFULL=$NOWYYYYMMDD.full.tar
TARNAMEPART=$NOWYYYYMMDD.part.tar
# Exclude file and tar args
TAR_ARGS="--totals --blocking-factor=1024"
EXCLUDE_CONF=/root/.backup.exclude.conf
# Backup Log file
LOGFIILE=$LOGBASE/$NOW.backup.log
# Path to binaries
TAR=/bin/tar
# ------------------------------------------------------------------------
# Excluding files when using tar
# Create a file called $EXCLUDE_CONF using a text editor
# Add files matching patterns such as follows (regex allowed):
# home/gvoina/iso
# home/gvoina/*.cpp~
# ------------------------------------------------------------------------
[ -f $EXCLUDE_CONF ] && TAR_ARGS="-X $EXCLUDE_CONF"
#### Custom functions #####
# Make a full backup
full_backup(){
if mountpoint -q /media/lto5; then
local old=$(pwd)
cd $BACKUP_PATH
echo $old
echo "Writing file:"
echo $TARNAMEFULL
$TAR $TAR_ARGS -cpf /media/lto5/$TARNAMEFULL $BACKUP_ROOT_DIR
cd $old
else
echo "Tape not available skipping backup"
fi
}
# Make a partial backup
partial_backup(){
if mountpoint -q /media/lto5; then
local old=$(pwd)
cd $BACKUP_PATH
echo "Writing file:"
echo $TARNAMEPART
# Copy only new data
$TAR $TAR_ARGS -cpf /media/lto5/$TARNAMEPART -N "$(date -d '1 day ago')" $BACKUP_ROOT_DIR
cd $old
else
echo "Tape not available skipping backup"
fi
}
#### Main logic ####
# Make sure log dir exits
[ ! -d $LOGBASE ] && $MKDIR -p $LOGBASE
# Okay let us start backup procedure
# If it is Monday make a full backup;
# For Mue to Sun make a partial backup
# Weekend no backups
case $NOW in
Mon) full_backup;;
Tue|Wed|Thu|Fri|Sat|Sun) partial_backup;;
*) ;;
esac > $LOGFIILE 2>&1
This is a simple script that will guess the day of the week and then proceeds to create a full or incremental back-up.
Note:
- TAR_ARGS=”–totals –blocking-factor=1024″ that ensure that a proper speed of writing to the LTO tape is maintained and avoid shoe-shining. The blocking factor of 1024 is best for the LTO5 tapes . For higher LTO versions higher values may be more appropriate.
- Partial back-up is done by using a simple tar filter -N “$(date -d ‘1 day ago’)” that will take into account only the files changed in the last day. You can implement more complicated methods but this is a simple straight forward good enough method.
So remember, backing up on tape is still the gold standard of back-ups.
