CoGe Backups

All CoGe source code and data is automatically backed-up to the iPlant Data Store on a weekly basis:

• MySQL Databases:  Three most-recent weekly snapshots of the MySQL databases are saved.
  • coge main
    
  • cogelinks
  • wikidb

• Files:  A single weekly snapshot of the files are saved.
  
  • Wiki web directory
    
  • SVN repository
    
  • Source data directory
    

The backup script shown below is run as a cron job on the CoGe server.

#!/bin/bash

echo `date` "Backup started"

ICMD=/usr/local/bin
DAYS_UNTIL_DELETE=21
VERSION=$(date '+%Y%m%d')
LOCAL=/storage/root/backup
REMOTE=backup

#
# Dump databases and copy to remote iRODS location
#
LOCAL_MYSQL=$LOCAL/mysql_$VERSION
mkdir -p $LOCAL_MYSQL
echo `date` "Dumping MySQL databases"
mysqldump -u XXXXXXXX -pXXXXXXXX wikidb -c | gzip -9 > $LOCAL_MYSQL/wikidb.sql.gz
mysqlhotcopy -u XXXXXXXX -p XXXXXXXX cogelinks $LOCAL_MYSQL
mysqlhotcopy --port=XXXX -u XXXXXXXX -p XXXXXXXX coge $LOCAL_MYSQL
echo `date` "Pushing MySQL databases to iRODS"
$ICMD/iput -bfr $LOCAL_MYSQL $REMOTE

#
# Remove old database backups
#
echo `date` "Deleting old backups (local & remote)"
LOCAL_DELETIONS=`find $LOCAL/mysql_* -maxdepth 1 -type d -mtime +$DAYS_UNTIL_DELETE`
if [ -n "$LOCAL_DELETIONS" ];
then
    echo delete local $LOCAL_DELETIONS
    rm -rf $LOCAL_DELETIONS
fi
for d in `$ICMD/ils backup | grep 'mysql_' | sed 's/.*\(mysql_.*\)/\1/'`
do
    if [ ! -d $LOCAL/$d ];
    then
        echo delete iRODS backup/$d
        $ICMD/irm -r backup/$d
    fi
done

#
# Sync data directories with remote iRODS location
#
echo `date` "Syncing data directories with iRODS"
$ICMD/irsync -rs /opt/apache2/cogepedia i:$REMOTE/cogepedia
# NOTE: iRODS doesn't like the .d at the end of SVN.d
$ICMD/irsync -rs /storage/SVN.d i:$REMOTE/SVN
$ICMD/irsync -rs /storage/coge/data/genomic_sequence/ i:backup/genomic_sequence

echo `date` "Backup completed"