Harley77-Backup

From Zimbra :: Wiki

Jump to: navigation, search
   Article-alert.png  - This article is a community contribution and may include unsupported customizations for network edition customers.



My Wonderful adventure in backing stuff Up

This is a work in progress, it may and probably does contain errors, misspellings, bad code, and bad grammar. Be warned.

Ok, so, I originally had a "Cold Backup Server" that I setup using some info gleaned from the forums. The solution that was provided didn't quite work for me. For one, it didn't provide a fast enough fail over AND due to misspellings and other steps left out of the script, it didn't work either.

I decided to try something that was probably not the ideal way, but, it was effective. I used a Dell workstation with a big HDD and a decent amount of ram, installed ubuntu 6.02lts (same as our live server) and installed Zimbra. The following is from my own documentation of this process.


Contents

Install Ubuntu 6.0 LTS Server

I'm not going into detail on how to install ubuntu, but, rest assured it is a required step. One thing to note however, the hostname should be identical to your LIVE server.

After you have ubuntu installed, you may proceed to the next step.

Add the “universe” depository to the apt-get updater

  • Open /etc/apt/sources.list – “pico /etc/apt/sources.list”
  • Remove the # in front of :
       deb http://us.archive.ubuntu.com/ubuntu/ dapper universe
       deb-src http://us.archive.ubuntu.com/ubuntu/ dapper universe
  • Ctrl-X and press Y to save.
  • Run
apt-get update

Install dnsmasq

Run the following commands

apt-get install dnsmasq
pico /etc/dnsmasq.conf
  • Add the following lines:
address=/your.maildomain.com/X.X.X.X
mx-host=maildomain.com,your.maildomain.com,10
mx-host=your.maildomain.com,your.maildomain.com,15
mx-target=your.maildomain.com
  • Press Ctrl+X and Y to save

Install Zimbra Prerequisites

  • Run
apt-get install libidn11 fetchmail libxml2 libstdc++5 openssl libltdl3 perl
  1. Set the DNS Resolver to point to itself.
  • Type
mv /etc/resolv.conf resolv.conf.BU
pico /etc/resolv.conf
  • Add the following:
Nameservers  X.X.X.X  #where x is your local ip
    • Press “ctrl+X” then Y to save.


Install the zimbra version that matches what is currently running on the LIVE server.

  • Copy the .tgz file downloaded from Zimbra.com .
  • Copy the file to a directory on the server. I’ve been using /ZIMBRAINSTALL/
  • Decompress the file: “gunzip FILENAME”
  • Extract the resulting tar file: “tar –xvf FILENAME”
  • Go into the directory that was created. “cd FILENAME”


Ok, I stop at this point because this is where my original and new methodologies begin to take different roads.

To make a long story short, I was rsyncing the /opt/zimbra directory from my live server to the local system. It did this once every hour. I didn't shutdown anything, I just had it syncing. The standby server was not active (meaning, zimbra was not running). However, thanks to a modified version of the script provided by fisch09 in the forums, I could quickly switch it from cold standby to Hot mode with a single command.

This past Christmas eve, our mail server crashed. I was 1500 miles away at the time and was able to remote into the backup machine, run the script, reboot and was back in action. So, it DOES work, but I've never been comfortable it. This fear was realized when we had a LDAP corruption and that corruption was synced with the backup server. This put an end to THAT particular setup.

This method also meant that after my Zimbra install, I would essentially move it aside and sync over the existing install. Since that is no longer the case, the installing of zimbra now takes on a few more steps. For this, the Disaster recovery Wiki entry was very helpful. I will be detailing that below and adding in a few things that were not quite covered. So, I revisited fisch09's method. Basically, taking backups from the live server, syncing them over and restoring them on the backup system. So, now to continue our setup.

Update

After more review, research, scripting, testing, more testing, and then more testing after that; I've come up with a Hybrid approach that takes the best of both worlds.

Basically, we ARE going to sync the /opt/zimbra directory (and any directories that you are using that exist outside that directory) to a cold standby server. This initial sync will give you a mirror of the live server as it sits, it however still has some side effects that we will address. This also keeps your incremental/full backups synced as well. So, in the event that the sync is BAD, we can still fall back on the zimbra made backups. I've divided this into two methods.

Installing Zimbra for Cold-Standby Operation (Method 1)

Use this method if you are NOT going to rsync your existing install to the cold standby server.

The process is pretty much identical to the method described for Disaster Recovery of the Network Edition. The following is essentially a cut and paste of the internal documentation I made for the company I work for.

  • Go to the extracted Zimbra install Directory and Run the installer
./install.sh
    • Pay attention to any errors.
    • It will Confirm what packages you are installing.
  • Use whatever packages you use on the Live Server
    • After this, it will test to make sure the DNS is working properly

If it isn’t you need to trouble shoot the dnsmasq settings and resolv.conf

    • Assuming all goes ok, it will stop at the config screen where you need to set Additional parameters. See next step.

IMPORTANT – At this step, you need to set the system to match the EXISTING server your backing up. You need to set:

  1. LDAP Passwords: There are 5 different ones. All of them are identical (typically).To find the existing passwords:
  • Log into the existing email server.
su zimbra
zmlocalconfig -s | grep -i pass
  • Write down the password used for the LDAP_ entries.
  1. Admin Password
  2. SPAM and HAM Email Addresses
  3. Admin email address
  4. wiki document email Address.
11) Enable default backup schedule:        no
r) Start servers after configuration        no
  1. Apply the config and continue install.

After it’s complete, we need to sync the backup directory and zimlets.

Sync the Backup Directory

  1. Establish a directory to place the backup files. I’m using an identical structure as the source server.
  • Type: mkdir /DRIVE2/backup
  • Type: chmod 777 /DRIVE2/backup
  1. To sync the files, use the following command.
rsync -e 'ssh -ax -c blowfish' -av 192.192.100.15:/DRIVE2/backup/  /DRIVE2/backup/
  1. Enter the root password for the source server when prompted. This will take awhile if it’s the first time.

Sync the Zimlets

  1. We need to sync the zimlets on the source server with the new target server. Use the following command:
rsync -e 'ssh -ax -c blowfish' -av 192.192.100.15:/opt/zimbra/mailboxd/webapps/service/zimlet 
/opt/zimbra/mailboxd/webapps/service/

Since we haven’t yet, Lets reboot. It may not be absolutely needed, but It seems to solve issues I sometimes get after a clean install..

Once rebooted,

su zimbra

Now we are going to restore our LDAP to the latest incremental backup. Type:

ldap start
  • in the list, will be a bunch of directories with a INCR prefix. Look for the newest one. Copy the whole directory name, this is our incremental backup label.

For this example, we will use: incr-20090203.200008.925

zmrestoreldap -lb incr-20090203.200008.925
  • This should show an output of the accounts being restored. Look for any errors. It will be done quickly.
  • LDAP Server should be started at this point.
mysql.server start
zmmailboxdctl start
  1. Now to restore the main message store.
zmrestore -restoreToIncrLabel incr-20090204.170011.969 -a "all"  -t /Path/To/Backup
  1. It won’t give you any feedback, so you need to open a second ssh session:
  2. Login and Type:
tail –f /opt/zimbra/log/mailbox.log
  1. You can monitor its progress here. This will take a while. About 1 – 2 hours.

Once the restore is complete:

zmcontrol stop
zmcontrol start
  1. Use the IP address of the server to access it and login. Verify the data.


Hybrid Backup - Cold Standby Server (Method 2)

This method involves:

1. Installing a dummy copy of Zimbra (still must be the identical version your using on your live server).

2. Syncing your existing live Zimbra installation to the Cold Backup server. This includes your backups.

3. Running the mysql database recovery script/procedure to keep the database from getting out of sequence.


Using my 3rBackup and a new DumpandRestore script, This whole thing can be automated when put in a cron job. Currently, I have about 60 users with ~38GB of data in total. When ran hourly, the whole process from Sync to Database Repair takes about 11 mins. This doesn't stress the live server at all so I have it running on an hourly basis. I also do hourly incremental backups so I have something to fall back on in case the synced copy is screwed.

In addition, I use the change.zimbra.status.sh script to switch the system over to live mode rapidly in the event of a mail server crash. Below are the scripts used and how.


The files involved:

3rbackup.sh -- The syncing script. This file is used by pairing it with a config file that defines the connection to the live server.

example: ./3rbackup.sh ../SERVERNAME/SERVERNAMEVAR.TXT 

SERVERNAMEVAR.TXT -- This is the variable file. It can be named whatever you want.

dumprestore.sh -- This script dumps out the entire database to files, drops them, deletes the ibdata and ib_log files, then reimports the data. This keeps you from getting the out of sequence errors in sql after doing a live sync. Is it smart? Well, it works for me.

mirror.sh -- Wraps all the scripts up into a single command that can be called in a cronjob.

sshsetup.sh -- This is not used during the backup, but can be used to create a ssh key to be used by rsync. Without setting up a key-pair, you CANNOT automate this in cron.


First, we need to create directories for these files to be placed. By default, I use the following. I like to Capitalize certain things because it makes it easy to find later.

/BACKUP/scripts <-- 3rbackup.sh, mirror.sh, sshsetup.sh, and dumprestore.sh
/BACKUP/SERVERNAME/ <-- SERVERNAMEVAR.TXT  (you can call SERVERNAME whatever you want.)
/BACKUP/SERVERNAME/logs/ <-- logfiles will be written here. 

You can have multiple SERVERNAME directories with multiple VAR files to backup/sync from multiple locations.


3rbackup.sh

#!/bin/bash
# Remote Rsync and Rotation
# Brian Davidson bsdavidson@gmail.com 
# v2.5.9  01/28/2008
# 
# For SSH use, this script works best when a public key is shared between the two computers
# that are involved. See my sshkeygencopy.sh script to assist in this or use 3rsetup.sh
# that will walk you through the complete process of setting up a new machine to be backed up.
#

# Credit to www.mikerubel.org for inspiration and technique. 
#
# Variables that this script uses are pulled from a separate file. Type the path and file name
# as an argument when running this script. ex. ./3rbackup.sh /backup/mailserver/var.txt  
# For logging to a file, add a 1 on the end of the command string. ex. ./3rbackup.sh /backup/mailserver/var.txt 1
#
# Run setup3r.sh to create a variable file, otherwise,
# A Fill-in-the-blank copy of the Variable file can be found under the file: blankvar.txt

# Enable the next line for debugging
#set -x 

# ******************************  Program and Path Setup **********************************
# The defaults should work for most cases, but you never know. They're here for your convenience. 

# Path to Rsync (VERY IMPORTANT!!!)
RSYNC="/usr/bin/rsync"

# Path to Mail program
MAIL="/usr/bin/mail"

# Path to samba mount
MOUNT="/bin/mount"

# Path to Novell Mount and ipx_configure
NCPMOUNT="/usr/bin/ncpmount"
IPXCONFIGURE="/usr/bin/ipx_configure"

# Path to umount command
UMOUNT="/bin/umount"

# SSH Program path
SSH="/usr/bin/ssh"

# The following sets the results log directory. It is shared among all backups and is used to store
# the overall pass/fail log of a days backups. It's usefull when running a series of diffrent backups in a given day/night
# since you can see in one log whether there were problems with any of them. 
# ex. RESULTSLOGDIR="/BACKUP/resultslog"
RESULTSLOGDIR="/BACKUP/resultslog"

# This is the email address that will appear in the FROM part of any emails sent to the backup administrator. 
SERVERMAIL="email@youremail.com"


# ******************************  END Program and Path Setup **********************************


# _____________________________________________________________________________
# The following should not need to be altered

if [ -f /root/coldstandby ]; then
   echo "server is in cold standby mode, proceeding with backup"
     else
   echo "The server is in Live operation, backup aborted."
  exit 127
   fi




RUNTIMECHECK=1

if [ "$2" = "" ]; then
clear
fi


# Checking out the argument to make sure its all good. 
if ! [ "$1" ]; then
 echo "Please specify a config file to load!"
 exit 127
fi

if [ -f $1 ]; then
 echo "Config File OK"
else
 echo "Invalid config file specified!"
 exit 127
fi

# Loading the variables..
source $1

# Loading any override variables
if [ -f "override.txt" ]; then
echo "Loading any override vars"
source override.txt
fi


#This is used for defining log files.
logdate=`date +'%m%d%Y'`;
logtime=`date +'%R'`;

# Results Log - Records the overall summary at the end of the backup. 
# Basically a single line that represents a Pass or Fail.
# Its filename is based on the date so any backups that are run on the same date will append to this file. 

RESULTSLOG="$RESULTSLOGDIR/r$logdate.log"
if [ -d $RESULTSLOGDIR ]; then
echo "Skipping Resultlog directory creation (already present)"
else
mkdir $RESULTSLOGDIR
fi
echo "Writing Backup Result log to $RESULTSLOG"
touch $RESULTSLOG

# This determines if the detailed log output goes to screen or a file. If "1" is used as the second variable when running
# the backup command, then it is all logged to a file, else, it goes to the screen with no logfile kept. 
if [ "$2" = "1" ]; then
 LOGFILE="$LOGDIR/$BUNAME$logdate.log"
 LOGERR="$LOGDIR/error$BUNAME$logdate.log"
 echo "Logging to file $LOGDIR/$BUNAME$logdate.log"
 # IO redirection for logging.
 touch $LOGFILE
 exec 6>&1
 # Saves stdout.
 exec > $LOGFILE
 touch $LOGERR
 exec 7>&2
 # Saves stderr.
 exec 2> $LOGERR
else
 echo "Logging output to screen"
fi


echo "--------   Starting Backup for $BUNAME     ------------"


if [ "$SNAPSHOTNUM" = "0" ]; then
 RTARGET=""$BUDIR"/"
else
 RTARGET=""$BUDIR"/snapshot.0/"

# The following should auto create the snapshot.0 directory if it doesn't exist. 
# Its not foolproof so I have an additional check later to make sure
# everything is all kosher. 

if [ -d $BUDIR ]; then
 if [ -d $RTARGET ]; then
  echo "Snapshot creation not needed" 
 else
 	mkdir $RTARGET
 fi
fi
fi


if [ $BUTYPE = "SAMBA" ]; then
 echo "Mounting SAMBA Source"
$UMOUNT "$SMBMOUNT" 2> /dev/null  
$MOUNT -t smbfs -o username=$USERNAME,password=$PASSWORD $SSOURCE $SMBMOUNT
 		if [ "$?" = "0" ]; then
 			echo "Samba Mounted OK"
 			echo "
-------------- $SMBMOUNT directory listing -----------------------"
 			ls $SMBMOUNT
 			echo "------------------------------------------------------------------		

Testing individual sources to be backed up... 
"
 			# Loop though the sources and test each one
		   NUMCOUNT=1
			 eval SRCLIST=\$SOURCE$NUMCOUNT
			 cd $SMBMOUNT
			 until [ "$SRCLIST" = "" ]
 			 do
 			  if [ -d "$SRCLIST" ]; then
 			   echo "$SRCLIST OK"
 			  else
 			   echo "$SRCLIST FAIL"
 			   SMBERRORDETECT=1
 			  fi
 			  NUMCOUNT=`expr $NUMCOUNT + 1`
			  eval SRCLIST=\$SOURCE$NUMCOUNT
			   done
 			else
 			 SMBERRORDETECT=1
 		 fi
fi

if [ $BUTYPE = "NOVELL" ]; then
 echo "Mounting NOVELL Source"
 $UMOUNT "$SMBMOUNT" 2> /dev/null
# Pre-connection stuff to help ensure the novell mount works properly.
 $IPXCONFIGURE --auto-primary=on
 $IPXCONFIGURE --auto-interface=on
 echo "Waiting 5 Seconds for IPX"
 sleep 5
 $NCPMOUNT -S "$SSOURCE"  -U "$USERNAME" -P "$PASSWORD" "$SMBMOUNT"
 		if [ "$?" = "0" ]; then
 			echo "Novell Mounted OK"
 			echo "
-------------- $SMBMOUNT directory listing -----------------------"
 			ls $SMBMOUNT
 			echo "------------------------------------------------------------------
Testing individual sources to be backed up...			
"
 			# Loop though the sources and test each one
		   NUMCOUNT=1
			 eval SRCLIST=\$SOURCE$NUMCOUNT
			 SRCLIST2=$SMBMOUNT$SRCLIST
			 cd $SMBMOUNT
			 until [ "$SRCLIST" = "" ]
 			 do
 			  if [ -d "$SRCLIST2" ]; then
 			  echo "$SRCLIST OK"
 			  else
 			  echo "$SRCLIST FAIL"
 			  NOVERRORDETECT=1
 			  fi
 			  NUMCOUNT=`expr $NUMCOUNT + 1`
			  eval SRCLIST=\$SOURCE$NUMCOUNT
			  SRCLIST2=$SMBMOUNT$SRCLIST
			   done
 			else
 			NOVERRORDETECT=1
 		fi
fi

echo " 
"

# Checking to make sure some of the paths and variables are valid
echo "*******************   SELF TESTING  ************************" 

if [ -d $BUDIR ] ; then
 echo "Backup Directory: OK" 
else
 echo "Backup Directory: FAIL" 
 ERRORDETECT=1
fi

if [ -d $RTARGET ] ; then 
 echo "Target Directory: OK" 
else
 echo "Target Directory: FAIL" 
 ERRORDETECT=1
fi

if [ -d $LOGDIR ] ; then
 echo "Log Directory: OK" 
else
 echo "Log Directory: FAIL" 
 ERRORDETECT=1
fi

if ! [ "$EXCLUDES" ]; then
 echo "Excludes File: N/A" 
 EXCLUDESCMD=""
 else
  if [ -f $EXCLUDES ] ; then
   echo "Excludes File: OK" 
   EXCLUDESCMD="--exclude-from="$EXCLUDES"" 
 else
  echo "Excludes File: FAIL" 
  ERRORDETECT=1
 fi
fi

if ! [ "$INCLUDES" ]; then
 echo "Includes File: N/A" 
 INCLUDESCMD=""
 else
  if [ -f $INCLUDES ] ; then
   echo "Includes File: OK" 
   INCLUDESCMD="--include-from="$INCLUDES"" 
 else
  echo "Includes File: FAIL" 
  ERRORDETECT=1
 fi
fi

if [ $BUTYPE = "SSH" ] ; then
 if [ ! -f $RKEY ] ; then
  echo "SSH KEY: FAIL"
  ERRORDETECT=1
 else
 if ! $SSH -2 -T -i $RKEY "$USERNAME"@$RHOST "test -x ~/"; then 
  echo "SSH LOGIN: FAIL" 
  ERRORDETECT=1
 else
  echo "SSH LOGIN: OK" 
  fi
 fi
fi

if [ "$BUTYPE" = "SAMBA" ] ; then
 if [ "$SMBERRORDETECT" = "1" ]; then
  echo "Samba Connection: FAIL" 
  ERRORDETECT=1
 else
  echo "Samba Connection: OK" 
 fi
fi

if [ "$BUTYPE" = "NOVELL" ] ; then
 if [ "$NOVERRORDETECT" = "1" ]; then
  echo "Novell Connection: FAIL" 
  ERRORDETECT=1
 else
  echo "Novell Connection: OK" 
 fi
fi

echo "*******************   END TESTING  ************************

" 

# If any errors are detected, it will add it to the Overall backup log and also create or append to an existing
# Error Log file. At the end of the backup, it will send two emails to the admin, one that indicates a backup error
# and another that gives the overall listing of Pass/Fail. 

if [ "$ERRORDETECT" = "1" ] ; then
 echo "Errors were detected, please check the error log above and verify" 
 echo "that your settings are correct" 
  if [ "$2" = "1" ]; then
    echo "$logtime - Backup $BUNAME Error - FAILED TO START " >> $RESULTSLOG
    echo "$logtime - Backup $BUNAME Error - FAILED TO START " >> $LOGERR
  fi
 exit 1
else
 echo "ALL PASS" 
fi
# END CHECKS
#******************************************************************
#----------------------Backup Rotation-----------------------------
#******************************************************************
#Only run if snapshot count is 1 or higher
if [ "$SNAPSHOTNUM" = "0" ]; then
 echo "Backup set for NO rotation, skipping..." 
else
 DAYS=$SNAPSHOTNUM
 echo "Backup is configured for a $DAYS day rotation" 

 echo "Begin Rotation Procedure ..... " 
 if [ -d $BUDIR/snapshot.$DAYS ] ; then
  echo "Deleting oldest snapshot" 
 rm -Rf $BUDIR/snapshot.$DAYS 
 DAYS=`expr $DAYS - 1`
 else
 DAYS=`expr $DAYS - 1`
 fi ;

 echo "Verifying/Creating Snapshot directories" 
 until [ "$DAYS" = "0" ]
 do
 if [ -d $BUDIR/snapshot.$DAYS ] ; then
 echo "snapshot.$DAYS exists"
 else
  echo "snapshot.$DAYS Doesnt exist, Creating.. " 
  mkdir $BUDIR/snapshot.$DAYS ; 
 fi ;
 DAYS=`expr $DAYS - 1`
 done

 #Reset DAYS Variable
 DAYS=$SNAPSHOTNUM
 DAYS=`expr $DAYS - 1`

 echo "Shifting snapshots to their older slots" 
 until [ "$DAYS" = "0" ]
 do
  echo "snapshot.$DAYS ==> snapshot.`expr $DAYS + 1`"
  mv $BUDIR/snapshot.$DAYS $BUDIR/snapshot.`expr $DAYS + 1` ; 
  DAYS=`expr $DAYS - 1`
 done

# I'm using a hard link as opposed to the --link-dest method since the results have proven (at least in my tests) 
# to be more reliable and consistant with its behavior.
echo "Making a hard link from snapshot.0 to snapshot.1"
if [ -d $BUDIR/snapshot.0 ] ; then
cp -al $BUDIR/snapshot.0 $BUDIR/snapshot.1 ;
fi ; 


echo "Rotation Complete" 

fi
# --------------------- Rotation Complete ------------------------

# Time to compile the sources into a single variable that's rsync compatible. 
# reseting the count
NUMCOUNT=1
# Establish our first entry and
# Set the initial Source Directory in the Var 
eval SRCLIST=\$SOURCE$NUMCOUNT

# Replace the space with a ? for compatibility with rsync (applies to Novell and Samba only).
if [ $BUTYPE == "SAMBA" -o $BUTYPE == "NOVELL" ]; then
SRCLIST=`echo "$SRCLIST" | sed s'/ /?/'`
fi

# Adding  so if the source has a space, rsync won't have a fit (SSH ONLY). 
if [ $BUTYPE == "SSH" ]; then
SRCLIST="'$SRCLIST'"
fi

# This is the first SOURCE Directory. 
SOURCEVAR="$SMBMOUNT/$SRCLIST"


# The Idea here is to take each source (SOURCE1, SOURCE2, SOURCE3, ect) and append it to a single SOURCEVAR. 
until [ "$SRCLIST" = "" ]
do
# Since we have already established SOURCEVAR1 earlier, I'm adding a 1 to make SOURCEVAR2. 
NUMCOUNT=`expr $NUMCOUNT + 1`
# I'm now refrencing the Next SOURCE in the list. 
eval SRCLIST=\$SOURCE$NUMCOUNT
# echo "$SRCLIST"
 # If the SOURCEVAR does NOT exist, I know that we have reached the end of the list. Otherwise, I will have to modify
 # the path a little so it will work properly when RSYNC gets ahold of it. 
 if [ "$SRCLIST" != "" ]; then
  if [ $BUTYPE == "SAMBA" -o $BUTYPE == "NOVELL" ]; then 
   # Replace the space with a ? for compatibility with rsync (applies to Novell and Samba only).
   SRCLIST=`echo "$SRCLIST" | sed s'/ /?/'`
  fi
  if [ $BUTYPE == "SSH" ]; then
   # Adding  so if the source has a space, rsync won't have a fit 
   SRCLIST="'$SRCLIST'"
  fi
 # I now take the existing SOURCEVAR and append the next SOURCE
 SOURCEVAR="$SOURCEVAR $SRCLIST"
fi
done

# Added to show the sources as rsync will see it. Good for debugging and show you that it did things right. 
echo "Compiled Source(s):"
echo "$SOURCEVAR

"

# -------------------------   Compile complete  --------------------------

# -------------------------   Backup Actually Begins  --------------------------

echo "Performing Sync. This may take a few minutes to several hours depending on data size." 
echo "Rsync Started `date`" 
if [ "$BUTYPE" = "LOCAL" ] ; then
 $RSYNC $EXCLUDESCMD $RVAR $SOURCEVAR $RTARGET 
fi

if [ "$BUTYPE" = "SAMBA" ] ; then
 cd $SMBMOUNT
  $RSYNC $EXCLUDESCMD $RVAR $SOURCEVAR $RTARGET
  # This backs us out of the directory so it can be dismounted. 
  cd /
  $UMOUNT "$SMBMOUNT"
  fi

if [ "$BUTYPE" = "NOVELL" ] ; then
 cd $SMBMOUNT
  $RSYNC $EXCLUDESCMD $RVAR $SOURCEVAR $RTARGET
  # This backs us out of the directory so it can be dismounted. 
  cd /
  $UMOUNT "$SMBMOUNT"
fi

if [ "$BUTYPE" = "SSH" ] ; then
 # A fine looking command if I say so myself. 
 $RSYNC $EXCLUDESCMD $RVAR -e "$SSH -T -i $RKEY" "$USERNAME"@$RHOST:"$SOURCEVAR" "$RTARGET" 
fi

# -------------------------   Backup Actually Ends  --------------------------

# Setting the permissions to make it web-readable. I use this in combination with a PHP Filemanager so I can login from any workstation
# to download a needed file. It's not required for anything other than that, so if you don't need it you can set
# the option CHMODDIR in the var file to "1".  
#if [ "$CHMODDIR" = "" ]; then
# chmod -R 755 $RTARGET
#fi

# Make an empty file with the name of the date/time to identify the date the snapshot was made.
# This is just to have a hard indicator of the actual date/time if the file/dir timestamp gets messed up.
# It can be excluded in the VAR file by setting NOTIMEFILE="1"
if [ "$NOTIMEFILE" = "" ]; then
 echo "Creating timestamp file" 
 rm -Rf $RTARGET/backupdate*
 touch $RTARGET/"backupdate$logdate-$logtime"
 echo "Rsync Ended `date`" 
fi


# This is the end-of-backup log handler.
if [ "$2" = "1" ]; then
 if [ -s "$LOGERR" ]; then
  echo "Error Reported!"
  echo "$logdate $logtime - BACKUP $BUNAME ERROR. LOGGED ERRORS DURING BACKUP" >> $RESULTSLOG
  echo "`cat $LOGERR`" | mail -s "$BUNAME ERROR REPORTED" -r "$SERVERMAIL" $NOTIFY
 else
  echo "No Errors"
  echo "$logdate $logtime - BACKUP $BUNAME SUCCESSFUL" >> $RESULTSLOG
  rm -Rf $LOGERR
 fi
fi

# The END. I hope you had a great time, I know I did. 
exit 0


dumprestore.sh

#!/bin/bash
# Dump and restore. Use with Rsynced Zimbra to resolve the out-of-sequence issues.
# This should be run as root.

# Remove the # below for debugging
# set -x

# Put your mySQL ROOT password below.

SQLPASSWD="YOURROOTSQLPASSWORD"

clear

echo "
**** Starting mysql.server ****"
su - zimbra -c mysql.server start

echo ".......Waiting a bit for mysql to fully start."
sleep 20


if [ -f /tmp/mysql.db.list ]; then
rm -Rf /tmp/mysql.db.list
else
echo " "
fi

echo "
**** Creating database list ****"
su - zimbra -c "/opt/zimbra/bin/mysql -NB -e 'show databases' | grep mbox > /tmp/mysql.db.list"


if [ -d /tmp/mysql.sql ]; then
echo " "
else
su - zimbra -c 'mkdir /tmp/mysql.sql'
fi

echo "
**** Dumping Zimbra database ****"
su - zimbra -c "/opt/zimbra/mysql/bin/mysqldump zimbra -S /opt/zimbra/db/mysql.sock -u root --password=$SQLPASSWD > /tmp/mysql.sql/zimbra.sql"


echo "
**** Dumping mailboxes ****"
for db in `< /tmp/mysql.db.list`;
do
su - zimbra -c "/opt/zimbra/mysql/bin/mysqldump $db -S /opt/zimbra/db/mysql.sock -u root --password=$SQLPASSWD > /tmp/mysql.sql/$db.sql"
echo -n "#"
done

echo -e " "
echo  "
**** Dropping mailboxes ****"
for db in `< /tmp/mysql.db.list`;
do
/opt/zimbra/bin/mysql -e "drop database $db"
echo -n "#"
done

echo -e " "
echo "
**** Dropping zimbra database ****"
/opt/zimbra/bin/mysql -e 'drop database zimbra'


echo "
**** Stopping Sql Server ****"
su - zimbra "mysql.server stop"

echo "
**** Deleting ibdata1 and ib_log files ****"
rm /opt/zimbra/db/data/ibdata1
echo -n "Pew! "
rm /opt/zimbra/db/data/ib_*
echo -n "Pew! Pew!"
echo "
Files deleted"


echo "
**** Starting mysql.server ****"
su - zimbra 'mysql.server start'

echo "....... Waiting for mysql to fully start."
sleep 20

echo "
**** Creating Zimbra database ****"
/opt/zimbra/bin/mysql -e "create database zimbra character set utf8"


echo "
**** Creating Mailbox Databases ****"
for db in `< /tmp/mysql.db.list`;
do
/opt/zimbra/bin/mysql -e "create database $db character set utf8"
echo -n  "#"
done


echo "
**** Importing Zimbra Database ****"
/opt/zimbra/bin/mysql zimbra < /tmp/mysql.sql/zimbra.sql



echo "
**** Importing Mailbox Databases ****"

for sql in /tmp/mysql.sql/mbox*
do
/opt/zimbra/bin/mysql `basename $sql .sql` < $sql
echo -n  "#"
done

echo -e " "
echo "
**** Stopping mysql.server ****"
su - zimbra 'mysql.server stop'

echo "********************* All Finished! ***********************"

mirror.sh

#!/bin/bash
# Mirror Master Script

# This script runs a sequence of scripts that mirrors the copy of zimbra on the live server
# and then performs a database recovery to keep the logfile sequence correct.

# Enable the next line for debugging
# set -x

echo "------------------ Making sure mysql.server is stopped ------------------"
su - zimbra -c mysql.server stop
sleep 20

/BACKUP/scripts/3rbackup.sh /BACKUP/SERVER/SERVERVAR.TXT 1
/BACKUP/scripts/3rbackup.sh /BACKUP/SERVER/SERVERVAR.TXT 1
/BACKUP/scripts/dumprestore.sh
echo "ALL DONE!"


sshsetup.sh

#!/bin/bash
#SSH Keygen and Copy
#Brian Davidson v1.2  1/22/2007
#This is to semi-automate the process of creating and copying a public key to linux machines that
#will need a passphraseless key. This will allow backup scripts to rsync and copy
#without the need of a password (thus they can run unattended). 
#This same code is incorporated into setup3r.sh. 


clear

echo "#######################  SSH KEYGEN AND COPY   ############################"

echo "Will you be Generating a NEW KEY?"
echo "REMEMBER! If you generate a new key, you must run the copy procedure for ALL Linux based machines"
echo "that you will be backing up via RSync over SSH."
echo "Type Y or N"
read type
echo " "
echo "Enter the Username of the machine that we will be connecting to"
echo "Ex. root, harvyh, or winnie rumplestien"
read USER
echo "Enter the IP Address of the remote machine"
read REMOTEIP
echo " "


# The following generates a new passphraseless ssh key.

if [ "$type" = "Y" -o "$type" = "y"  ]; then
echo "###################################################"
echo "      PRESS ENTER ON THE FOLLOWING 2 PROMPTS       "
echo "You will have to wait a minute before the first prompt"
echo "###################################################"
if [ ! -d ~/rsynckey ]; then
mkdir ~/rsynckey
chmod 700 ~/rsynckey
fi
ssh-keygen -t dsa -b 2048 -f ~/rsynckey/rsync-key
fi;
clear

echo "######  COPYING FILES TO REMOTE MACHINE   ######"
echo " "
echo "IF prompted with: Are you sure you want to continue connecting: Type *YES*"
echo "### Enter the password for $USER on $REMOTEIP when prompted ###"
echo " "
scp ~/rsynckey/rsync-key.pub "$USER"@$REMOTEIP:~

echo " "
echo "##################################################"
echo "      Enter the Password again                    "
echo "##################################################"
ssh "$USER"@$REMOTEIP "mkdir -p ~/.ssh;chmod 700 ~/.ssh;mv -f ~/rsync-key.pub ~/.ssh/;cd ~/.ssh/;touch authorized_keys;chmod 600 authorized_keys;cat rsync-key.pub >> authorized_keys; rm ~/.ssh/rsync-key.pub"

echo "Thats all folks"


SERVERVAR.TXT

# This VAR file and it's generator were created by Brian Davidson (bsdavidson@gmail.com)

# Config file used by the 3rbackup.sh script.
# This file is not intended to be ran alone.
# Usage example: 3rbackup.sh ../BACKUP/THISFILEVAR.TXT


# Email Address of Who to notify incase something goes wrong
# This requires Sendmail to be operational.
NOTIFY=your@emailaddress.com

#Short name of the machine or data being backed up.
# ex. server1, userdocs, etc.
BUNAME=SERVERNAME

# This number is how many days back you want in your rotation.
SNAPSHOTNUM=0

# Enter a Backup Type.
# SSH   = Source files are obtained via SSH connection
BUTYPE=SSH

# Note: DO NOT REMOVE QUOTES AROUND VARIABLES. DO NOT LEAVE A SPACE AFTER OPEN QUOTE OR BEFORE CLOSING QUOTE.

# -------  Common Paths ----------

# Path to excludes file: ex. /path/to/excludes.txt
EXCLUDES=""

# rsync variables: ex.-avz --delete --delete-excluded
RVAR="-avz --delete --delete-excluded --progress"

# The Target directory to store the snapshots -- DO NOT PUT A TRAILING SLASH!
BUDIR="/opt/zimbra"

# Path(s) to directories to be backed up: ex.
# SOURCE1=/Stuff/to/backup
# If you only want the files and directores under the source path backed up, add a trailing slash.
# Ex. SOURCE1=/stuff/to/backup/
# For additional Sources, Add them using SOURCE2= , SOURCE3=, ect.


SOURCE1="/opt/zimbra/"


#The Directory where you want your logfiles to be stored.
LOGDIR=/BACKUP/ZSYNC/LOGS/
# ------- End Common Paths ----------

# ------- Time Stamp File -----------
# Set this variable to 1 to prevent the creation of a file in the DATA dir that
# has it's name set to the date and time of the backup operation.
NOTIMEFILE=""

# ----- End Time Stamp File ---------


# ------- Common Network Info ----------
# Login Username. ex. root, briand, bob somebody, etc.
USERNAME="yourrootpassword"

# ----- End Common Network Info --------


# ----------- SSH CONFIG -------------
# Enter the following info if your going to use SSH to retrieve the Data
# Path to SSH Key File: ex. ~/rsynckey/rsync-key
# Please note there are NO quotation marks
RKEY=~/rsynckey/rsync-key

# Host IP address: ex. 192.168.1.200
RHOST=123.123.123.123


# --------------  DO NOT EDIT BELOW THIS LINE --------------------

if ! [ "$RUNTIMECHECK" ]; then
echo "This script must be called from 3rbackup.sh"
echo "It will not function on its own."
fi


Verified Against: Ubuntu 6.0 LTS Date Created: 2/13/2009
Article ID: http://wiki.zimbra.com/index.php?title=Harley77-Backup Date Modified: 11/29/2011
Personal tools