Zimbra NG Modules/Zimbra NG Backup/Incremental server to server migration with Zimbra NG Backup




NG Incremental Server to Server Migration

Overview

This guide describes how to perform an incremental migration using server to server migration methodology with full backups created by Zextras Suite / Zimbra Plus Suite and restoring with NG Backup or Zimbra Plus Suite backup modules. It's specifically designed for the migration of a production environment, minimizing downtime and aiming to be transparent for the users.

For environments that are running version 6.0.7 to 8.7.x, you will need to install Zextras Suite 2.4.11 onto each mailbox server to create a backup that can be migrated. It is not recommended to use Zextras Migration tool do to a number of known issues. If a licensed Zextras Suite or Zimbra Plus Suite is installed and active, you can create the migration backup using your current setup.

Overview of the Migration Process

The following is a high level description of the incremental migration process:

  • Create a Destination server (The new server that will replace the current production server)
  • Take a SmartScan backup of the source server (Production System).
  • Restored the SmartScan backup to the destination server.
  • Take a new SmartScan backup of the source server (This will only capture changes that occurred after the last SmartScan.)
  • Restored the new SmartScan backup to the destination server (This will only restore the changes).
  • Repeat this until there are minimal changes and you are ready to switch servers.
  • Replace source system (System in Production) with destination server (New System)
  • Take one more SmartScan backup of the source server which will obtain that last group of changes
  • Restored the SmartScan backup.

What Will be Migrated

  • Emails and Email Folders.
  • Contacts and Address Books.
  • Appointments and Calendars.
  • Tasks and Tasklists.
  • Files and Briefcases.
  • Share informations.
  • User Preferences.
  • User Settings.
  • Class of Service Settings.
  • Domain Settings.

What will Not be Migrated

  • Server settings (migrated for reference but not restored).
  • Global Settings (migrated for reference but not restored).
  • Customizations (Postfix, Jetty, Server Branding, etc...).
  • Items moved or deleted during the process will not be moved or deleted on the destination server.
  • Preferences (e.g. passwords) changed during the process will be reset upon each import.


warning

This migration process is not designed to set up a server-to-server mirroring. Using multiple imports to create a mirrored copy of the source server won't create a mirrored copy since no deletions are performed by the Import process.


We're here to help! Zimbra Community members who need assistance with upgrading can reach out to sales@zimbra.com, and the Zimbra Sales team will assist you with all your upgrade needs

PRE-MIGRATION CHECKS

Please carefully read the NG Backup FAQs pages before performing the migration process.

Servers Description

  • Source Server: any Zimbra server can be the source of your migration, as long as the running Zimbra version is 6.0.7 or higher.
  • Destination Server: any Zimbra server can be the destination of your migration, as long as the Zimbra version for Network addition is 8.8.6 or higher with NG Modules enabled or Zimbra Suite Plus is installed and enabled.

Be sure to have root access to both servers, and have a look at External Restore for some import performance tips.

Destination Server Setup

The destination server should be the latest version on the latest supported OS that meets your current and future needs. For the latest Zimbra documentation Click Here.

You will need to define the best method on how to replace the source server with the destination server. Each company has different needs so Zimbra Support does not recommend one method over another.

Key wiki's:

warning

Zimbra community members who have an active support contract, to obtain support, you must be upgrading to a supported version. Zimbra recommends upgrading to the latest release.


Storage

  • Source server: The Zextras Suite or Zimbra Suite Plus backup must be installed and enabled on the source server, make sure you have an amount of free disk space comparable to the size of the /opt/zimbra/store/ folder (the exported data is compressed through the gzip algorithm, and all zimbra items are deduplicated, usually reducing the size of the export to around 70% of the original size).
  • Destination server: Make sure you have an amount of free space greater than the size of the /opt/zimbra/store/ and of the "export" folders on the source server combined

Data Transfer

There are a number of ways that you can choose to transfer the backup. If network speed isn’t an issue, rsync is Zimbra’s choice to transfer the backup between the servers. Also remote mount or physical move of the drive will significantly shorten the transfer between the two servers.

DNS

If you are changing the server hostname, setting the TTL value of your MX record to 300 on your "real" DNS will allow a fast switch between source and destination servers.

SET-UP

All the CLI commands in this guide must be executed as the Zimbra user unless otherwise specified.

Step 1: Destination Server Setup

Setup the destination server or cluster that meets your current and future needs.Click Here for the latest Zimbra Documentation

Step 2: Source Server Check

In order to avoid any possible data-related issues during the migration, we recommend running the following checks on the source Mailbox server/s:

  • ‘’’Identifying Blob Issues:’’’ To prevent restoring issues do to missing or corrupted blobs, we recommend doing a blob check on the volumes or accounts that are being migrated using the blob check that comes with the Zextras Legacy suite which does a more in-depth verification than zmblobcheck. The following are some examples of how to run the command

Blob coherency check on selected volumes:

zxsuite hsm doCheckBlobs start volumes <volume1> <Volume2>

Blob coherency check on mailbox’s using mailbox_ids:

zxsuite hsm doCheckBlobs start mailboxes < mailbox_id>. <mailbox_id>, <mailbox_id>, ….

You can list all of the accounts in a single request and the Blob check will verify each account. You can also script the Blob check to check the desired accounts.

  • ’’’Reindex Mailboxes’’’ - Reindexing all mailboxes is suggested to insure there is no existing messages within each mail account that could cause parsing issues during the migration. Because indexing is resource and time intensive, we recommend performing the indexing before installing Zextras or Zimbra Plus Suite. Also it's recommended scripting the reindexing to limit the number of accounts that will be index at once to match the resources on the source system.
  zmprov reIndexMailbox(rim) {name@domain|id} {start|status|cancel} [{types|ids} {type or id} [,type or id...]]

Step 3: Server Setup

Source Server: To take a backup that is compatible with the NG backup module, you will need to install either Zextras Suite or Zimbra Suite Plus on your source mailbox server. Please note:

  • Versions older than 6.0.8 must upgrade to 6.0.8 before using this migration method.
  • Network and Open Source editions running 6.0.8 to 8.7.11 will need to use the 2.4.12 legacy installer
  • Open Source editions running 8.0.0 and higher can also use the latest release of Zextras Suite or Zimbra Suite Plus.
  • Zextras Suite newer than 2.4.12 will not install on network addition.


warning

ZxPowerstore (HSM) is enabled as default during the installation of the Zextras Suite or Zimbra Suite Plus. Before installing, we recommend creating a backup just incase. Community members who are running Network Edition and have an active support contract, you must only enable the backup module. Enabling other Modules on your source server (Admin and Mobile) could change the behavior of your system and IS NOT supported by Zimbra Support. <more info Needed>


Run this commands as root within the root directory.

wget http://download.zextras.com/zextras_suite-legacy.tgz
tar zxvf zextras_suite-latest.tgz
cd zextras_suite-[version]
./install.sh all

Disable the Real Time Scanner on both servers:

zxsuite backup setProperty ZxBackup_RealTimeScanner false

A dedicated device for the data export is strongly recommended in order to improve the export performance and to lower the impact on the performance of the running system.

Such device must be mounted on the /opt/zimbra/backup/ path and the Zimbra user must have r/w permissions.

Step 4: Data Export (SmartScan)

Run a SmartScan on the source server/s:

zxsuite backup doSmartScan

All data will be exported to the default backup path (/opt/zimbra/backup/zextras/).

Data export (SmartScan) via the NG Administration Zimlet

Informational:

There are two backup options:

  • SmartScan must be executed by the system admin and will take a backup then incremental changes each time it's re-ran.
  • Real-Time backup will actively backup each action at the time the action it taken.

Real Time backup will not support this migration, this is the reason we set ZxBackup_RealTimeScanner to FALSE above

Step 5: Data Synchronization

Using rsync, copy the data contained within /opt/zimbra/backup/zextras/ to a directory in the destination server (make sure the Zimbra user has r/w permissions on such folder). Use a terminal multiplexer like screen or tmux, this process command might need A LOT of time depending on network speed and amount of data involved.

[run this command as Root]
rsync -avH /opt/zimbra/backup/zextras/ root@desinationserver:/path/for/the/data/

Or mount the folder containing the backup to the destination server.

warning

When you move or mount the exported data to the destination server make sure that the destination folder is not ZxBackup's backup path to avoid any current or future issues with the ZxBackup module.

First Importing into Source Server

Import all exported data to the destination Mailbox server:

zxsuite backup doExternalRestore /path/for/the/data/

This will import all accounts within the backup. Based on the amount of data, the initial restore process can take significant amount of time.

You also can import your data using the NG Administration Zimlet following this guide. While importing via the NG Administration Zimlet be sure to remove all System Accounts (like GalSync, Ham, Spam, Quarantine etc.) from the account list within the import window.

When importing from the command line, system accounts from the source server will need to be deleted during the post migration check list. DO NOT delete accounts during the incremental restore process. If accounts are deleted during the restore process, restore issues could occur.

warning

Do not edit nor delete the Map Files. Doing so could cause item duplication issues and will disrupt your migration process.

Incremental Data Migration

Once the initial restore has completed, majority of the data has been imported to the destination server. Since the source server is still active, new SmartScan needs to be done to migrate the changes that occurred since the first migration by following the same steps in section 4 5 and 6 above.

This step may need to be performed a number of times based on the amount of time and data received between the scans.

If there are more than one mailbox server's in your setup; planned correctly more than 90% of your data will be restored in the initial restore. If the mailbox server share the same storage, staggering the restores will lower disk IO and network traffic. Once the initial restores are completed on all mailbox servers, the following restores will be much faster and less intensive.

DESTINATION SERVER ACTIVATION

Moving Destination Server into Active Server Position

It's time to make the destination server / cluster the active system/s using the process you selected at the beginning of this migration. Once the destination server / cluster is live, you will need to verify the there's no issues by doing the following tests:

  • Able to access the mail account from inside and outside of your company network.
  • Restored data is present in the accounts.
  • Mail is received from external senders
  • Mail is sent to external senders
  • Clients (IMAP, POP, active-sync, etc) can access the account

Last Incremental Migration

Once the destination server / cluster is active and tested, it is time to obtain the last incremental backup. This backup should be minimal since the time between the last restore and implementing the destination server / cluster should have occurred in a timely manor. To do the last SmartScan, you will need to follow the same steps in section 4 5 and 6 above for each mailbox server

Post Migration Checklist

Once the destination server / cluster is active and the last SmartScan has completed, it's time to perform the following actions:

  • Verify Share Consistency
  • Remove Legacy system accounts
  • Recreate Domain GAL-Sync accounts
  • Message Deduplication

Shares Consistency

The following command will check for shares inconsistencies:

zxsuite backup doCheckShares

Should this command report any inconsistency, the

zxsuite backup doFixShares

This Command will parse the import mapfile used as the first argument and fix any broken share. Mapfiles can be found in the Backup Path of the destination server as "map_[source_serverID]".

Legacy System Accounts

Delete any imported legacy system accounts from the Zimbra Administration Console. You can verify the accounts that are being used by running:

zmprov gacf zimbraSpamIsNotSpamAccount
zmprov gacf zimbraSpamIsSpamAccount
zmprov gacf zimbraAmavisQuarantineAccount
zmprov gd <Domain.com> |grep -l gal 

You need to recreate GalSync accounts on all imported domains. Please see the GalSync Accounts wiki for detail instructions.

Message Deduplication

To save storage space, it is highly recommended to run the message deduplication operation to consolidate all messages that have more than one instance to a single blob. This can be done by running the following command:

zxsuite hsm doDeduplicate {volume_name} [attr1 value1 [attr2 value2...]]

At this time your system should be up and running, you want to insure that you enable the backup module

zxsuite backup setProperty ZxBackup_RealTimeScanner TRUE



Zimbra NG Modules

logo.png

Latest Version: 8.8

Zimbra NG Modules Resources

Here you can find useful resources for your Zimbra NG Modules



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »

Jump to: navigation, search