ZCS to ZCS rsync Migration

(Redirected from Moving ZCS to New Server)

ZCS to ZCS rsync Migration

   KB 20739        Last updated on 07/11/2015  




0.00
(0 votes)
Article-check.png  - This is certified documentation and is protected for editing by Zimbra Employees & Moderators only.
 - This article is a Work in Progress, and may be unfinished or missing sections.
Zimbra-coldbackup-001.png

Using rsync to migrate from an old ZCS mailstore to a new ZCS mailstore can be required for multiple reasons:

1. Migrating from a 32-bit server to a 64-bit server

2. Abandoning an old system because it is damaged, corrupted, broken, etc.

3. Running continuous rsync copies of a ZCS mailstore for DR purposes

4. Moving from an Operating System that is EOL to one that is current, for example:

  • Moving from RHEL 4 to RHEL 5
  • Moving from RHEL 5 to RHEL 6
  • Moving from Ubuntu 8 to Ubuntu 10
  • Moving from Ubuntu 10 to Ubuntu 12

For the purposes of this documentation, we will call the two platforms the "Old Server" and the "New Server".

Requirements

IMPORTANT: The ZCS release you install on the New Server must be the same ZCS Version as installed on the Old Server. The server can have a different operating system, but the ZCS Version must be identical.

The new server hardware must meet the requirements described in the Installation Prerequisites section of the ZCS Single Server Installation Guide. Install the new operating systems, making any necessary OS configuration modifications as described in the installation guide.

Please note too: this method is not required for ancillary systems, such as LDAP nodes, MTAs, or Proxies - all of these nodes can have new systems added while the old system is still running, and therefore do not require the rsync methods described here. Use Rolling Upgrade methods for non-mailstores systems. This rsync method is specifically for ZCS mailstores.

Migration Steps

For the migration, you would perform the following high-level steps to minimize downtime:

New Server:

  • 1. Prepare Operating System - this includes patching, updating, and tuning.
  • 2. Install ZCS Version - this MUST be the exact same ZCS Version as on the Old Server. The bit level (32-bit vs. 64-bit) or the Operating System build (RHEL 5, RHEL 6, Ubuntu 12, etc.) may be different, but the ZCS version must be exactly the same.

Old Server:

  • 1. Rsync'ing large amounts of data. Since rsync can sync data incrementally, you can therefore rsync the majority of the data in advance of any downtime.
    • The actual amount of downtime is then limited to a final rsync and then the final setup steps. For the second-to-last rsync run, it is a good idea to time it - this will give you a good idea of the amount of downtime required.
  • 2. Backup LDAP and MYSQL data for importing into new server.

You do the following on the New Server:

  • Prepare the new server
  • Restore the LDAP data
  • Edit the localconfig.xml file to update the MySQL and LDAP password values to be the same as those configured on the old server
  • Copy the latest backup files from the Old Server to the New Server
  • Delete the MySQL data that is set up in the initial installation of ZCS
  • Copy various files from the Old Server to the New Server
  • Prepare and run a new backup for the New Server

Notes on rsync:

1. In order to use rsync between servers, you can either use the xinetd method, or the ssh method. Since many servers allow only ssh for accessing servers these days, you may find that method required. Always run rsync as root, so that it can set the proper permissions and ownership on all files:

xinetd method

This site has some good instructions for setting up rsync with xinetd: http://people.virginia.edu/~ll2bf/docs/nix/rsync.html

(on Old Server)

rsync --delete -axvzKHS [source-location] [newserver-IP-address]:[destination-location]

ssh method

This site has some good instructions for setting up rsync using ssh: http://oreilly.com/pub/h/38

Please note that root must be allowed to login on the destination server via ssh in order for rsync over ssh to work:

(on Old Server)

rsync -e ssh --delete -axvzKHS [source-location] [newserver-IP-address]:[destination-location]

2. From here on out, we will provide examples for the ssh method.

3. Test that rsync works properly:

(on Old Server)
cd /tmp
touch testfile
rsync -e ssh -axvzKHS testfile [newserver-IP-address]:/tmp
(enter root password if necessary)

Then check if the file /tmp/testfile exists on the New Server.

Until rsync is working properly, do not proceed further.

Preparing the Old Server

1. Block client access to the server's IP address with firewall rules.

2. Create a backup directory on each of the Old Server and New Server, for use in copying data. As root, type the following:

On Old Server:

mkdir /backup

On New Server:

mkdir /backup
mkdir /backup/old

3. On both servers, as user root, type:

chown zimbra:zimbra /backup

4. Stop the Old Server, type

su - zimbra
zmcontrol stop

5. On ZCS 6.0 and later, backup the LDAP config database, as user zimbra, type

/opt/zimbra/libexec/zmslapcat -c /backup

6/ Backup the LDAP data, as zimbra, type

/opt/zimbra/libexec/zmslapcat /backup

7. Change the hostname and IP address on the old server to something else. Do not turn off the server.

Preparing the New Server

Install ZCS

The ZCS installation on the new server must be configured exactly as the ZCS configuration on the original server. You go through the complete menu driven installation process, making changes to the configuration setting to match the settings on the original server.

a. Copy your ZCSLicense.xml file to a directory on the new server. You will not be able to complete the ZCS installation if the license is not on the new server.

b. Follow the directions in the ZCS single server installation guide to install ZCS. Make sure that you configure the same domain, hostname, passwords as on the old server. During ZCS install, the following settings must be changed to match the original server settings:

  • Zimbra LDAP Server - For Domain to create - identify the same default domain as on the original server.
  • Zimbra Mailbox Server - An administrator's account is automatically created.
  • Make sure that the account name for Admin user to create is the same name as on the original server.
  • Set the admin password.
  • Change the Spam training user and the Non-spam (HAM) training user account names to be the same as the spam account names on the original server.
  • Global Document Account - This account name is automatically generated. Change the Global Document Account name to be the same account name as on the original server.
  • Change any other settings on the new server to match the configuration on the original server.

ZCS is installed on the New Server

1. To continue, stop the ZCS services, type the following:

su - zimbra
zmcontrol stop

ZCS 5.0.x LDAP Import

Restore the LDAP data to the New Server. As zimbra, type

1. First, move aside the default LDAP data on the New Server, as user zimbra:

su - zimbra
mv -f /opt/zimbra/openldap-data/* /backup/old

2. Copy the file:

/opt/zimbra/openldap-data/DB_CONFIG

from the Old Server to the New Server:

/opt/zimbra/openldap-data/DB_CONFIG

Note: If this file does not exist, or is empty, creating it may improve ldap performance; see the Performance Tuning Guide for more information.

3. Create the necessary directory structure. Type the following as user root:

mkdir -p /opt/zimbra/openldap-data/logs /opt/zimbra/openldap-data/accesslog/db /opt/zimbra/openldap-data/accesslog/logs

Set the correct permissions on the directory:

chown -R zimbra:zimbra /opt/zimbra/openldap-data

4. Copy from the Old Server to the New Server the LDAP data file:

/backup/ldap.bak

5. Import the LDAP Data:

/opt/zimbra/openldap/sbin/slapadd -q -b "" -f /opt/zimbra/conf/slapd.conf -cv  -l /backup/ldap.bak

ZCS 6.0.x-7.2.x LDAP Import

1. Restore the LDAP data to the New Server. As user root:

mv -f /opt/zimbra/data/ldap/config/* /backup/old
mv -f /opt/zimbra/data/ldap/hdb/* /backup/old

2. If this is an ldap master with replicas:

mv -f /opt/zimbra/data/ldap/accesslog/* /backup/old

3. Create new directories:

mkdir -p /opt/zimbra/data/ldap/hdb/db /opt/zimbra/data/ldap/hdb/logs

4. If this is an ldap master with replicas:

mkdir -p /opt/zimbra/data/ldap/accesslog/db /opt/zimbra/data/ldap/accesslog/logs

5. Copy this file from the Old Server:

/opt/zimbra/data/ldap/hdb/db/DB_CONFIG

To the New Server:

/opt/zimbra/data/ldap/hdb/db/DB_CONFIG

Note: If this file does not exist, or is empty, creating it may improve ldap performance; see the Performance Tuning Guide for more information.

6. Set the permissions correctly:

chown -R zimbra:zimbra /opt/zimbra/data/ldap

7. Copy the LDAP config data from the Old Server to the New Server:

/backup/ldap-config.bak

8. Copy from LDAP data the Old Server server to the New Server:

/backup/ldap.bak

9. Import the LDAP config:

/opt/zimbra/openldap/sbin/slapadd -q -n 0 -F /opt/zimbra/data/ldap/config -cv -l /backup/ldap-config.bak

10. Import the LDAP data:

/opt/zimbra/openldap/sbin/slapadd -q -b "" -F /opt/zimbra/data/ldap/config -cv  -l /backup/ldap.bak

ZCS 8.0.x-8.5.x LDAP Import

1. On the New Server, move aside the default LDAP data:

(as root)
cd /opt/zimbra/data/ldap
mv -f mdb /backup/old

2. On the New Server, create the new directories

mkdir -p mdb/db

WARNING: ONLY REMOVE AND RELOAD THE CONFIG DB IF ABSOLUTELY NECESSARY. Reloading the config db should rarely be necessary (unless you customized the config). To do so though and only if necessary, you would need to clear the config/ directory and import the config data:

cd /opt/zimbra/data/ldap
mv -f config /backup/old
mkdir config

ZCS 8: Reloading the accesslog DB would only apply to a master or multi-master scenario. If necessary:

cd /opt/zimbra/data/ldap
mv acccesslog /backup/old
mkdir -p accesslog/db

3. Set the permissions correctly:

chown -R zimbra:zimbra /opt/zimbra/data/ldap

4. Export the data from the Old Server:

(as user zimbra)

Example of main database export:

/opt/zimbra/libexec/zmslapcat /backup

Example of configuration database export:

/opt/zimbra/libexec/zmslapcat -c /backup

Example of an accesslog database export (8.0.2 and later):

/opt/zimbra/libexec/zmslapcat -a /backup

5. Copy the files from /backup/ from the Old Server to the New Server, put them in /backup on the New Server

6. Import the data to the New Server:

  • WARNING: ONLY REMOVE AND RELOAD THE CONFIG DB IF ABSOLUTELY NECESSARY. If the config data had been removed (if /opt/zimbra/data/ldap/config is empty), import the LDAP config data:

As the user zimbra: su - zimbra

/opt/zimbra/libexec/zmslapadd -c /backup/ldap-config.bak
  • Import the production LDAP database. The last argument is the full path to the data export file:
/opt/zimbra/libexec/zmslapadd /backup/ldap.bak
  • If the New Server is an LDAP Master or Multi-Master (MMR), import the accesslog LDAP database via the zmslapadd command. The last argument is the full path to the export:
/opt/zimbra/libexec/zmslapadd -a /backup/ldap-accesslog.bak

zmlocalconfig attributes from Old Server

There are some configuration options from the zmlocalconfig configuration file that you may want to transfer from the Old Server to the New Server.

1. Set the Passwords the same on the New Server as the Old Server

For all versions:

Edit /opt/zimbra/conf/localconfig.xml to update the following with the values from the localconfig.xml file on the Old Server:

a. zimbra_mysql_password
b. mysql_root_password
c. zimbra_logger_mysql_password (Note: Transfer/copy this value to the New Server only if available from the old Old Server.)
d. zimbra_ldap_password
e. ldap_root_password
f. ldap_postfix_password
g. ldap_amavis_password
h. ldap_nginx_password
i. ldap_replication_password

You can retrieve these passwords from the Old Server with the following command:

su - zimbra
zmlocalconfig -x -s | grep password

2. There may be other customized configuration values in zmlocalconfig on the Old Server. Check for these with this command on the Old Server, as user zimbra:

su - zimbra
zmlocalconfig -n

On the New Server, you can edit those same values as needed:

zmlocalconfig -e [attribute-name]='[attribute-value]'

See here for more details on zmlocalconfig: zmlocalconfig

Import the MySQL Data

Delete the MySQL data on the New Server and copy the MySQL data from the Old Server.

1. On the New Server, move the default database aside:

mv -f /opt/zimbra/db/data/* /backup/old

2. Copy the MySQL data to the New Server:

(on Old Server)
rsync --delete -axvzKHS /opt/zimbra/db/data/* [newserver-IP-address]:/opt/zimbra/db/data/

Note: if you are having any problems moving over the MySQL data as raw data files, you can instead export the data from the Old Server and import it to the New Server, as described here: http://wiki.zimbra.com/wiki/MySQL_Backup_and_Restore

Copy the Message Blob Data

Copy the following files from the Old Server to the New Server

1. On New Server, move aside default files:

mv -f /opt/zimbra/store/* /backup/old
mv -f /opt/zimbra/index/* /backup/old

2. Rsync all files to the new server, as user root:

(on Old Server)
rsync -e ssh --delete -axvzKHS /opt/zimbra/store/* [newserver-IP-address]:/opt/zimbra/store/
rsync -e ssh --delete -axvzKHS /opt/zimbra/index/* [newserver-IP-address]:/opt/zimbra/index/

3. Be sure to rsync over any other message volumes needed, such as if using HSM or secondary volumes:

(on Old Server)
rsync -e ssh --delete -axvzKHS /opt/zimbra/hsm/* [newserver-IP-address]:/opt/zimbra/hsm/

Copy Commercial SSL Certificates (if used)

1. If the Old Server had a customized keystore, you may want to copy that keystore to the New Server. In most cases, simply installing the same SSL certificates from the Old Server is sufficient. For example, using these steps to install the correct certificate on the New Server: zmcertmgr examples

2. If you do want to copy over the keystore from the Old Server to the New Server, perhaps because you installed a new certs or CA certs, you can do the following:

a. If the original server was running with commercial certificates, copy those over as well.

b. The jetty keystore (http, pop, and imap) is at the following location:

/opt/zimbra/mailboxd/etc/keystore

Copy this to the New Server and put in place. First, move the default keystore:

mv -f /opt/zimbra/mailboxd/etc/keystore /backup/old

Then copy the desired keystore into place and make sure the permissions are correct:

mv /backup/keystore /opt/zimbra/mailboxd/etc/keystore
chown zimbra:zimbra /opt/zimbra/mailboxd/etc/keystore

c. When transferring the keystore file, be sure to transfer the keystore password to the new system otherwise the mailbox server will not start.

Run on Old Server:

zmlocalconfig -s mailboxd_keystore_password

Set the password the same on New Server:

zmlocalconfig -e mailboxd_keystore_password='thepassword'

d. If any root certificates were added to the cacerts keystore, that is this file:

/opt/zimbra/java/jre/lib/security/cacerts

The cacerts password is generally the default of "changeit".

e. The postfix certificates (smtp) are at these locations:

/opt/zimbra/conf/smtpd.crt
/opt/zimbra/conf/smtpd.key

If you are using the certificates for nginx, perdition, or ldap (slapd) they are also in these locations:

/opt/zimbra/conf/

They should normally be identical to the postfix certificate files.

f. Finally, if necessary, you can copy the entire /opt/zimbra/ssl directory manually if necessary. Follow these steps if manual installation of commercial SSL certificates are required: Multi-Node Commercial Certificates

Clear Unneeded Backups from the New Server

1. Remove any unneeded backup sessions because these sessions are no longer needed.

mv -f /opt/zimbra/backup/* /backup/old
mv -f /opt/zimbra/redolog/* /backup/old

2. On New Server, create new backup and redolog directories:

(as root)
mkdir /opt/zimbra/backup/sessions
mkdir /opt/zimbra/backup/tmp
mkdir /opt/zimbra/redolog/archive
chown zimbra:zimbra /opt/zimbra/backup/sessions /opt/zimbra/backup/tmp
chown zimbra:zimbra /opt/zimbra/redolog/archive

Fix the Permissions

Confirm that all permissions are correct on the new server:

1. As root, run the zmfixperms script to repair any potential permissions problems with files under /opt/zimbra:

(as root)
/opt/zimbra/libexec/zmfixperms

2. For best results, run the -extended option to confirm correct perms for all files, although this may take longer:

(as root)
/opt/zimbra/libexec/zmfixperms -extended

Start the Services on the New Server

1. Start ZCS

(as user zimbra)
zmcontrol start

2. Now run a full backup

zmbackup -f -a all

3. Remove the firewall rules and allow client access to the new server.

Reference Documentation



Keywords: Migration, migrating, 32bit, 64bit, architecture, moving


Verified Against: Zimbra Collaboration Server 8.0 Date Created: 01/18/2014
Article ID: http://wiki.zimbra.com/index.php?title=ZCS_to_ZCS_rsync_Migration Date Modified: 07/11/2015



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »

Jump to: navigation, search