Scripts to sync to a remote Zimbra backup machine

Revision as of 00:25, 17 July 2007 by Costa-101 (talk | contribs) (Brief tutorial on implementing a cost-effective backup solution for a Zimbra server.)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Synchronising to a Remote Backup Machine

Scenario

A single remote machine is running the Zimbra Open Source Edition (OSE), providing services for a whole load of users (user1, user2, userx) in various domains (domain1, domain2 and domainy).

The machine has a public IP address and is in operation 24 hours-a-day 7 days-a-week.

The source machine, in this fictitious example, will be known as 'zimbra.server.com' and the backup machine as 'backup.server.com'.

Aims

  • To provide a cost-effective backup solution, ensuring the main server (zimbra.server.com) is on-line at all times.
  • The backup solution must stay up-to-date, so in case of an emergency, users do not lose large chunks of their emails.
  • The solution must be customisable, automated and require a minimal amount of time for administrators to implement.

System Requirements

  • Imapsync
  • Zimbra OSE
  • Reasonable amount of disk space to ensure room for emails and their attachments
  • Familiarity with crontab

Time To Implement

After Zimbra and imapsync installations - just the time needed to input your user names and passwords and setup a cron job.

So, depending on your experience of Linux and Zimbra installations and the size of your user base, anything between say 20 minutes to .....

Top

Plan of action

  • Install Zimbra OSE on a local machine, in this case it will be known as 'backup.server.com' which does not have to be on the same Linux platform or even the same Zimbra version ... but of course, for the sake of completion it may as well be a mirrored setup. The 'backup.server.com' can be implemented anywhere .. a local LAN, a second remote site or right next to our source server 'zimbra.server.com'. In this scenario the zimbra.server.com is located in the docklands, London and our backup.server.com is located out in the country at an office.
  • Install imapsync.

Step 1

  • Create a directory named 'Zimbra_Scripts' and change into that directory.
mkdir Zimbra_Scripts; cd Zimbra_Scripts

Top

Step 2

  • Create and edit a file called 'imap_users' and enter the domain names, the user names and passwords of all your domains and users on zimbra.server.com, using the syntax as follows (obviously substituting domains, users and passwords appropriately):
nano imap_users
domains=(
	domain1 \
	domain2 \
	domainy)

users=(
	user1@domain1 \
	user2@domain1 \
	userx@domain1 \
	user1@domain2 \
	user2@domain2 \
	userx@domain2 \
	user1@domainy \
	user2@domainy \
	userx@domainy)

passwords=(
	u1d1passwd \
	u2d1passwd \
	uxd1passwd \
	u1d2passwd \
	u2d2passwd \
	uxd2passwd \
	u1dypasswd \
	u2dypasswd \
	uxdypasswd)

Top

Step 3

  • Create another file in the 'Zimbra_Scripts' folder - called 'zimbra_migration' and enter the main body of our code here.

This code recreates the domains currently residing on zimbra.server.com onto the backup.server.com, ready to receive the emails in our next step.

For admins with a lot of domains, recreating them one-by-one can be a time consuming process.

N.B. - The first line does start with a period . and ensures the file 'imap_users' is included into 'zimbra_migration' script.

This was done in regards to object oriented programming, for portability and security.

nano zimbra_migration
. imap_users

for ((i = 0 ; i < ${#domains[@]} ; i++ ))
do
echo $i: /opt/zimbra/bin/zmprov cd ${domains[$i]}
done

for ((i = 0 ; i < ${#users[@]} ; i++ ))
do
echo $i: /opt/zimbra/bin/zmprov ca ${users[$i]}
/opt/zimbra/bin/zmprov ca ${users[$i]} ${passwords[$i]}
done
Breakdown

Uses two for loops, iterating through the array of domains firstly.

A little debug code, shows the id number of the domain in the loop and then creates the domain by calling
zmprov cd

The domain value is taken from the 'domains' array located in the 'imap_users' file.

In the second loop, again a little debug code lets us know where we are in the list of users and which user is being created.

zmprov ca
creates the user, taking its arguments from the 'users' array in the 'imap_users' file, and the password from the same array index position of the 'passwords' array, again located in the 'imap_users' file.

Top

Step 4

Set the permissions on all of the files so that we can execute the files correctly:

chmod +x *

So you are now 'good-to-go' - you can invoke this script by running, from within the 'Zimbra_Scripts' directory, the command:

./zimbra_migration

Top

Cronjob

So now to automate the sync on a regular basis, between source and destination servers.

Create a file for our cron job called 'imapsync_crontab' - again within the 'Zimbra_Scripts' directory.

nano imapsync_crontab

Edit it to include the following:

. imap_users #if in the same folder as imap_users else full-path to imap_users

src_srv = zimbra.server.com
dest_srv = backup.server.com

for ((i = 0 ; i < ${#users[@]} ; i++ ))
do
/usr/bin/imapsync --noauthmd5 --syncinternaldates --buffersize 65535000 --subscribe \
--host1 $src_srv --ssl1 --user1 ${users[$i]} --password1 ${passwords[$i]} --nofastio1 --authmech1 LOGIN \
--host2 $dest_srv --ssl2 --user2 ${users[$i]} --password2 ${passwords[$i]} --nofastio2 --authmech2 LOGIN
done
Breakdown

We set the source server address as 'src_srv' and the destination server address as 'dest_srv' - for use in the call to imapsync.

A single for loop iterates through the array containing our list of 'users' and their passwords.

The following imapsync options are used:

  • noauthmd5 - no use of md5
  • syncinternaldates - to keep the correct dates and times emails were sent/received
  • buffersize 65535000 - ensures if a large attachment is encountered there is room in the buffer
  • subscribe - automatically creates any additional user folders on the destination machine (keeping the account directory structure)
  • host1 - this is our source server 'zimbra.server.com'
  • host2 - this is our destination server 'backup.server.com'
  • ssl1/2 - uses a secure connection on both machines
  • nofastio1/2 - helps to reduce errors when syncing
  • authmech1/2 - the method used to authenticate at both ends (LOGIN is commonly successful)
  • password1/2 - should probably be the same on both machines


Create a cron job by using the following command (firstly setting the editor to use nano):

export EDITOR=$(type -P nano);crontab -e

Now, set an hourly synchronisation to the destination server. This assumes the 'Zimbra_Scripts' directory is located in '/opt/Zimbra_Scripts'.

The output is emailed to admin@zimbra.server.com.

MAILTO=admin@zimbra.server.com
@hourly /opt/Zimbra_Scripts/imapsync_crontab

This ensures that users emails are backed up on an hourly basis.

N.B. - if you have a large number of users, it would be advisable to run the job at greater intervals. This will alleviate system resources and ensure no overlapping.

Top

Created by Costa-101


External Links

Jump to: navigation, search