Scripts to sync to a remote Zimbra backup machine: Difference between revisions

Line 9: Line 9:
The source machine, in this fictitious example, will be known as 'zimbra.server.com' and the backup machine as 'backup.server.com'.
The source machine, in this fictitious example, will be known as 'zimbra.server.com' and the backup machine as 'backup.server.com'.


====''** Notice **''====


Syncing between platforms  favours exporting your ldap from the main server, achieved via:
''(this example works from x86_64 => x86)''
''''The following commands should execute on the main/source server''''
Logging in as zimbra user
<code><pre>su - zimbra</pre></code>
Export LDAP data to an ldif file
<code><pre>openldap/sbin/slapcat -f /opt/zimbra/conf/slapd.conf -l /tmp/zimbra-ldap.ldif</pre></code>
''''The following command should execute on the destination/backup server''''
Logging in as zimbra user
<code><pre>su - zimbra</pre></code>
Shutting zimbra server down
<code><pre>zmcontrol stop</pre></code>
Manually check for any imbra services - (kill if neccessary)
<code><pre>ps auxx | grep slapd</pre></code>
The following removes any current data - you could just move it until back in the 'safe zone'.
<code><pre>rm -f openldap-data/*</pre></code>
Import user accounts and passwords from the LDAP file zimbra-ldap.ldif
<code><pre>openldap/sbin/slapadd -f /opt/zimbra/conf/slapd.conf -l /tmp/zimbra-ldap.ldif</pre></code>
That's that you may now proceed to run your imapsync_crontab file
(It is suggested and personally recommended to do things this way - however - the choice is your own and the instructions on the rest of this page also successfully work)


===Aims===
===Aims===

Revision as of 23:54, 6 November 2007

Synchronising to a Remote Backup Machine

Scenario

A single remote machine is running the Zimbra Open Source Edition (OSE), providing services for a whole load of users (user1, user2, userx) in various domains (domain1, domain2 and domainy).

The machine has a public IP address and is in operation 24 hours-a-day 7 days-a-week.

The source machine, in this fictitious example, will be known as 'zimbra.server.com' and the backup machine as 'backup.server.com'.


Aims

  • To provide a cost-effective backup solution, ensuring the main server (zimbra.server.com) is on-line at all times.
  • The backup solution must stay up-to-date, so in case of an emergency, users do not lose large chunks of their emails.
  • The solution must be customisable, automated and require a minimal amount of time for administrators to implement.

System Requirements

Time To Implement

After Zimbra and imapsync installations - just the time needed to input your user names and passwords and setup a cron job.

So, depending on your experience of Linux and Zimbra installations and the size of your user base, anything between say 20 minutes to .....

Top

Plan of action

  • Install Zimbra OSE on a local machine, in this case it will be known as 'backup.server.com' which does not have to be on the same Linux platform or even the same Zimbra version ... but of course, for the sake of completion it may as well be a mirrored setup. The 'backup.server.com' can be implemented anywhere .. a local LAN, a second remote site or right next to our source server 'zimbra.server.com'. In this scenario the zimbra.server.com is located in the docklands, London and our backup.server.com is located out in the country at an office.
  • Install imapsync.

Step 1

  • Create a directory named 'Zimbra_Scripts' and change into that directory.
mkdir Zimbra_Scripts; cd Zimbra_Scripts

Top

Step 2

  • Create and edit a file called 'imap_users' and enter the domain names, the user names and passwords of all your domains and users on zimbra.server.com, using the syntax as follows (obviously substituting domains, users and passwords appropriately):
nano imap_users
domains=(
	domain1 \
	domain2 \
	domainy)

users=(
	user1@domain1 \
	user2@domain1 \
	userx@domain1 \
	user1@domain2 \
	user2@domain2 \
	userx@domain2 \
	user1@domainy \
	user2@domainy \
	userx@domainy)

passwords=(
	u1d1passwd \
	u2d1passwd \
	uxd1passwd \
	u1d2passwd \
	u2d2passwd \
	uxd2passwd \
	u1dypasswd \
	u2dypasswd \
	uxdypasswd)

Top

Step 3

  • Create another file in the 'Zimbra_Scripts' folder - called 'zimbra_migration' and enter the main body of our code here.

This code recreates the domains currently residing on zimbra.server.com onto the backup.server.com, ready to receive the emails in our next step.

For admins with a lot of domains, recreating them one-by-one can be a time consuming process.

N.B. - The first line does start with a period . and ensures the file 'imap_users' is included into 'zimbra_migration' script.

This was done in regards to object oriented programming, for portability and security.

nano zimbra_migration
. imap_users

for ((i = 0 ; i < ${#domains[@]} ; i++ ))
do
echo $i: /opt/zimbra/bin/zmprov cd ${domains[$i]}
done

for ((i = 0 ; i < ${#users[@]} ; i++ ))
do
echo $i: /opt/zimbra/bin/zmprov ca ${users[$i]}
/opt/zimbra/bin/zmprov ca ${users[$i]} ${passwords[$i]}
done
Breakdown

Uses two for loops, iterating through the array of domains firstly.

A little debug code, shows the id number of the domain in the loop and then creates the domain by calling

zmprov cd

The domain value is taken from the 'domains' array located in the 'imap_users' file.

In the second loop, again a little debug code lets us know where we are in the list of users and which user is being created.

zmprov ca

creates the user, taking its arguments from the 'users' array in the 'imap_users' file, and the password from the same array index position of the 'passwords' array, again located in the 'imap_users' file.

Top

Step 4

Set the permissions on all of the files so that we can execute the files correctly:

chmod +x *

So you are now 'good-to-go' - you can invoke this script by running, from within the 'Zimbra_Scripts' directory, the command:

./zimbra_migration

Top

Cronjob

So now to automate the sync on a regular basis, between source and destination servers.

Create a file for our cron job called 'imapsync_crontab' - again within the 'Zimbra_Scripts' directory.

nano imapsync_crontab

Edit it to include the following:

. imap_users #if in the same folder as imap_users else full-path to imap_users

src_srv = zimbra.server.com
dest_srv = backup.server.com

for ((i = 0 ; i < ${#users[@]} ; i++ ))
do
/usr/bin/imapsync --noauthmd5 --syncinternaldates --buffersize 65535000 --subscribe \
--host1 $src_srv --ssl1 --user1 ${users[$i]} --password1 ${passwords[$i]} --nofastio1 --authmech1 LOGIN \
--host2 $dest_srv --ssl2 --user2 ${users[$i]} --password2 ${passwords[$i]} --nofastio2 --authmech2 LOGIN
done
Breakdown

We set the source server address as 'src_srv' and the destination server address as 'dest_srv' - for use in the call to imapsync.

A single for loop iterates through the array containing our list of 'users' and their passwords.

The following imapsync options are used:

  • noauthmd5 - no use of md5
  • syncinternaldates - to keep the correct dates and times emails were sent/received
  • buffersize 65535000 - ensures if a large attachment is encountered there is room in the buffer
  • subscribe - automatically creates any additional user folders on the destination machine (keeping the account directory structure)
  • host1 - this is our source server 'zimbra.server.com'
  • host2 - this is our destination server 'backup.server.com'
  • ssl1/2 - uses a secure connection on both machines
  • nofastio1/2 - helps to reduce errors when syncing
  • authmech1/2 - the method used to authenticate at both ends (LOGIN is commonly successful)
  • password1/2 - should probably be the same on both machines


Create a cron job by using the following command (firstly setting the editor to use nano):

export EDITOR=$(type -P nano);crontab -e

Now, set an hourly synchronisation to the destination server. This assumes the 'Zimbra_Scripts' directory is located in '/opt/Zimbra_Scripts'.

The output is emailed to admin@zimbra.server.com.

MAILTO=admin@zimbra.server.com
@hourly /opt/Zimbra_Scripts/imapsync_crontab

This ensures that users emails are backed up on an hourly basis.

N.B. - if you have a large number of users, it would be advisable to run the job at greater intervals. This will alleviate system resources and ensure no overlapping.

Top

Created by Costa-101


External Links

Jump to: navigation, search