Zimbra NG Modules/Zimbra NG Backup/Incremental server to server migration with Zimbra NG Backup: Difference between revisions

Line 100: Line 100:
There are a number of ways that you can choose to transfer the backup. If network speed isn’t an issue, rsync is Zimbra’s choice to transfer the backup between the servers.  Also remote mount or physical move of the drive will significantly shorten the transfer between the two servers.
There are a number of ways that you can choose to transfer the backup. If network speed isn’t an issue, rsync is Zimbra’s choice to transfer the backup between the servers.  Also remote mount or physical move of the drive will significantly shorten the transfer between the two servers.


===DNS===
====DNS====
If you are changing the server hostname, setting the TTL value of your MX record to 300 on your "real" DNS will allow a fast switch between source and destination servers.
If you are changing the server hostname, setting the TTL value of your MX record to 300 on your "real" DNS will allow a fast switch between source and destination servers.



Revision as of 23:41, 12 March 2018




NG Incremental Server to Server Migration

Overview

This guide describes how to perform an incremental migration using server to server migration methodology with full backups created by Zextras Suite / Zimbra Plus Suite and restoring with NG Backup or Zimbra Plus Suite backup modules. It's specifically designed for the migration of a production environment, minimizing the downtime and aiming to be transparent for the users.

Overview of the Migration Process

The following is a high level description of the incremental migration process.

  • Create a Destination server (The new server that will replace the current production server)
  • Take a SmartScan backup of the source server (Production System).
  • Restored the SmartScan backup to the destination server.
  • Take a new SmartScan backup of the source server (This will only capture changes that occurred after the last SmartScan.)
  • Restored the new SmartScan backup to the destination server (This will only restore the changes).
  • Repeat this until there are minimal changes and you are ready to switch servers.
  • Replace source system (System in Production) with destination server (New System)
  • Take one more SmartScan backup of the source server which will obtain that last group of changes
  • Restored the SmartScan backup.

What will be migrated

  • Emails and Email Folders.
  • Contacts and Address Books.
  • Appointments and Calendars.
  • Tasks and Tasklists.
  • Files and Briefcases.
  • Share informations.
  • User Preferences.
  • User Settings.
  • Class of Service Settings.
  • Domain Settings.

What will NOT be migrated

  • Server settings (migrated for reference but not restored).
  • Global Settings (migrated for reference but not restored).
  • Customizations (Postfix, Jetty, Server Branding, etc...).
  • Items moved or deleted during the process will not be moved or deleted on the destination server.
  • Preferences (e.g. passwords) changed during the process will be reset upon each import.


warning

This migration process is not designed to set up a server-to-server mirroring. Using multiple imports to create a mirrored copy of the source server won't create a mirrored copy since no deletions are performed by the Import process.


We're here to help! Zimbra Community members who need assistance with upgrading can reach out to sales@zimbra.com, and the Zimbra Sales team will assist you with all your upgrade needs

PRE-MIGRATION CHECKS

Please carefully read the NG Backup FAQs pages before performing the migration process.

Servers Description

  • Source Server: any Zimbra server can be the source of your migration, as long as the running Zimbra version is 6.0.7 or higher.
  • Destination Server: any Zimbra server can be the destination of your migration, as long as the Zimbra version for Network addition is 8.8.6 or higher with NG Modules enabled or Zimbra Suite Plus is installed and enabled.


Be sure to have root access to both servers, and have a look at External Restore for some import performance tips.

warning

Zimbra community members who have an active support contract, to obtain support, you must be upgrading to a supported version. Zimbra recommends upgrading to the latest release..

Destination Server Setup

The destination server should be the latest version on the latest supported OS that meets your current and future needs. For the latest Zimbra documentation Click Here.

You will need to define the best method on how to replace the source server with the destination server. Each company has different needs so Zimbra Support does not recommend one method over another.

Key wiki's:

Storage

  • Source server: The Zextras Suite or Zimbra Suite Plus backup must be installed and enabled on the source server, make sure you have an amount of free disk space comparable to the size of the /opt/zimbra/store/ folder (the exported data is compressed through the gzip algorithm, and all zimbra items are deduplicated, usually reducing the size of the export to around 70% of the original size).
  • Destination server: Make sure you have an amount of free space greater than the size of the /opt/zimbra/store/ and of the "export" folders on the source server combined

Data Transfer

There are a number of ways that you can choose to transfer the backup. If network speed isn’t an issue, rsync is Zimbra’s choice to transfer the backup between the servers. Also remote mount or physical move of the drive will significantly shorten the transfer between the two servers.

DNS

If you are changing the server hostname, setting the TTL value of your MX record to 300 on your "real" DNS will allow a fast switch between source and destination servers.

The Set-Up

All the CLI commands in this guide must be executed as the Zimbra user unless otherwise specified.

Step 1: Destination Server Setup

Setup the destination server or cluster that meets your current and future needs.

Step 2: Source Server Check

In order to avoid any possible data-related issues during the migration, we recommend running the following checks on the source Mailbox server/s:

  • ‘’’Identifying Blob Issues:’’’ To prevent restoring issues do to missing or corrupted blobs, we recommend doing a blob check on the volumes or accounts that are being migrated. We recommend using the blob check that comes with the Zextras Legacy suite which does a more in-depth verification than zmblobcheck. The following are some examples of how to run the command

Blob coherency check on selected volumes:

zxsuite hsm doCheckBlobs start volumes <volume1> <Volume2>

Blob coherency check on mailbox’s using mailbox_ids:

zxsuite hsm doCheckBlobs start mailboxes < mailbox_id>. <mailbox_id>, <mailbox_id>, ….

You can list all of the accounts in a single request and the Blob check will verify each account. You can also script the Blob check to check the desired accounts.

  • ’’’Reindex all mailboxes’’’ - Reindexing all mailboxes is suggested to insure there is no existing messages within each mail account that could cause parsing issues during the migration. Because indexing is resource and time intensive, we recommend performing the indexing before installing Zextras or Zimbra Plus Suite. Also we recommend scripting the reindexing to limit the number of accounts that will be index at once to match the source system resources.
  zmprov reIndexMailbox(rim) {name@domain|id} {start|status|cancel} [{types|ids} {type or id} [,type or id...]]

Step 3: Server Setup

Source Server: To be able to take a backup that is compatible with the NG backup module, you will need to install either Zextras Suite or Zimbra Suite Plus on your source mailbox server. Please note:

  • Versions older than 6.0.8 must upgrade to 6.0.8 before using this migration method.
  • Network and Open Source editions running 6.0.8 to 8.7.11 will need to use the 2.4.12 legacy installer
  • Open Source editions running 8.0.0 and higher can also use the latest release of Zextras Suite or Zimbra Suite Plus.
  • Zextras Suite newer than 2.4.12 will not install on network addition.


Warning!

HSM is enabled as default, when installing Zextras Suite or Zimbra Suite Plus. If you have <more info Needed>

Community members who are running Network Edition and have an active support contract, you must only enable the backup module. Enabling other Modules on your source server (Admin and Mobile) could change the behavior of your system and IS NOT supported by Zimbra Support.

[run this commands as Root]

wget http://download.zextras.com/zextras_suite-legacy.tgz
tar zxvf zextras_suite-latest.tgz
cd zextras_suite-[version]
./install.sh all

Disable the Real Time Scanner on both servers:

zxsuite backup setProperty ZxBackup_RealTimeScanner false

A dedicated device for the data export is strongly recommended in order to improve the export performance and to lower the impact on the performance of the running system.

Such device must be mounted on the /opt/zimbra/backup/ path and the Zimbra user must have r/w permissions on it

Step 4: Data Export (SmartScan)

Run a SmartScan on the source server/s:

zxsuite backup doSmartScan

All data will be exported to the default backup path (/opt/zimbra/backup/zextras/). Data export (SmartScan) via the NG Administration Zimlet

Step 5: Data Synchronization

Using rsync, copy the data contained in the /opt/zimbra/backup/zextras/ to a directory in the destination server (make sure the Zimbra user has r/w permissions on such folder). Use a terminal multiplexer like screen or tmux, this process command might need A LOT of time depending on network speed and amount of data involved.

[run this command as Root]
rsync -avH /opt/zimbra/backup/zextras/ root@desinationserver:/path/for/the/data/

Or mount the folder containing the backup to the destination server.

Warning!
When you move or mount the exported data to the destination server make sure that the destination folder is not ZxBackup's backup path to avoid any current or future issues with the ZxBackup module.

First Importing into Source Server

Import all exported data to the destination Mailbox server:

zxsuite backup doExternalRestore /path/for/the/data/

This will import all accounts within the backup. Based on the amount of data, the initial restore process can take significant amount of time.

You also can to import your data using the NG Administration Zimlet following this guide. While importing via the NG Administration Zimlet be sure to remove all System Accounts (like GalSync, Ham, Spam, Quarantine etc.) from the imported account list.

When importing from the command line, system accounts from the source server will need to be deleted during the post migration check list. Do Not delete accounts during the incremental restore process. If accounts are deleted during the restore process, restore errors could occur.

Warning!
Do not edit nor delete the Map Files. Doing so could cause item duplication issues and will disrupt your migration process.

Incremental Data Migration

Once the initial restore has completed, majority of the data has been imported to the destination server. Since the source server is still active, a new SmartScan needs to be done to migrate the changes that occurred since the first migration by following the same steps in section 4 5 and 6.

This step may need to be performed a number of times based on the amount of time and data received between the scans.

Activating Destination Server

Implementing Distention Server as Active Server

It's time to make the destination server / cluster the active system/s using the process you selected at the beginning of this migration. Once the destination server / cluster is live, you will need to verify the there is no issues by doing the following tests:

  • Able to access the mail account from inside and outside of your company network.
  • Restored data is present in the accounts.
  • Mail is received from external senders
  • Mail is sent to external senders
  • Clients (IMAP, POP, active-sync, etc) can access the account

Last Incremental Migration

Once the destination server / cluster is active and tested, it is time to obtain the last incremental backup. This backup should be minimal since the time between the last restore and implementing the destination server / cluster should have occurred in a timely manor. To do the last SmartScan, you will need to follow the same steps in section 4 5 and 6.


Post Migration Checklist

Once the destination server / cluster is active and the last SmartScan has completed, it's time to perform the following actions:

  • Verify Share Consistency
  • Remove Legacy system accounts
  • Recreate Domain GAL-Sync accounts
  • Message Deduplication

Shares Consistency

The following command will check for shares inconsistencies:

zxsuite backup doCheckShares

Should this command report any inconsistency, the

zxsuite backup doFixShares

This Command will parse the import mapfile used as the first argument and fix any broken share. Mapfiles can be found in the Backup Path of the destination server as "map_[source_serverID]".

Legacy System Accounts

Delete any imported legacy system accounts from the Zimbra Administration Console. You can verify the accounts that are being used by running:

zmprov gacf zimbraSpamIsNotSpamAccount
zmprov gacf zimbraSpamIsSpamAccount
zmprov gacf zimbraAmavisQuarantineAccount
zmprov gd <Domain.com> |grep -l gal 

You need to recreate GalSync accounts on all imported domains. Please see the GalSync Accounts wiki for detail instructions.

Message Deduplication

To save storage space, it is highly recommended to run the message deduplication operation to consolidate all messages that have more than one instance to a single blob. This can be done by running the following command:

zxsuite hsm doDeduplicate {volume_name} [attr1 value1 [attr2 value2...]]

At this time your system should be up and running, you want to insure that you enable the backup module

zxsuite backup setProperty ZxBackup_RealTimeScanner TRUE



Zimbra NG Modules

logo.png

Latest Version: 8.8

Zimbra NG Modules Resources

Here you can find useful resources for your Zimbra NG Modules



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »

Jump to: navigation, search