Individual Mailbox Restore from Snapshot
Individual Mailbox Restore from Snapshot
This procedure was developed in a limited lab environment. The concept should be applied with caution by experienced system administrators that understand both the snapshot technologies of their SAN and virtualization vendors. Your implementation may have different assumptions as to how things work, so all steps should be tested first in a non-production environment.
Purpose and Concepts
Zmbackup is a tool developed to provide full and incremental backups of mail servers. One of the great features of zmbackup is the ability to restore individual mailboxes up to specific points in time for recovery of accidentally deleted messages or folders. Individual mailbox restores are common in enterprise environments. However, in spite of optimization of Zmbackup with features like auto-grouping, it is still possible that the window for backups is smaller than the time required to complete backups and individual restores of very large mailboxes can take a long time. In this case, administrators of larger email systems can utilize snapshot functionality (either of the SAN or a virtualized environment) to maintain backups for the purposes of disaster recovery or individual mailbox restores.
The architecture of the Zimbra environment dictates two prerequisites for restoring from snapshots.
- Since all configuration information for mailbox servers and individual accounts resides in ldap, a Zimbra ldap server must be restored as part of this process and the restored mailbox server must be able to communicate with the restored ldap server. One key piece of data stored in ldap is the mapping of user accounts to mailbox servers. Also, the restored mailbox server will not start without connectivity to an ldap server. Related to this, LDAP will only have information for production servers and production accounts (with a direct correlation of which accounts are on which mailbox servers). Therefore it is not possible to bring up a Zimbra mailbox server with a different hostname to restore data for an individual account from a server that has a different name.
- The snapshots must have consistency between the blob store and the mysql database. If /opt/zimbra/store and /opt/zimbra/db are on different luns within the SAN, they must be snapshotted together as a "snapshot group" to use one vendor's terminology. There is still the possibility that mysql data is cached in memory and not written to disk during the snapshot, resulting in an inconsistent state in the snapshot, but it should be reasonable to expect in the case of restoring accidentally deleted data that the data was consistent on disk during most past snapshots.
The good news is that Zimbra server(s) (mailbox + ldap) can be brought online based on a snapshot assuming the new server(s) have different IP addresses, but still use the same name without affecting normal production environments. From the restored mailbox server using the Zimbra Web Client, mail, folders, contacts, appointments, etc. can be exported using the export tool in preferences as a tgz file and the tgz file with the required data can be imported to the production environment.
Restore from VM Snapshot
- Create a Virtual Machine from a snapshot.
- Start the new VM. If prompted, indicate the machine was "copied", not moved. This should initialize the new virtual server network settings as DHCP instead of static addressing. Because the new server has a different IP address that what is in /etc/hosts, Zimbra services will not start on the new machine.
- Edit the IP Address settings including name resolution if needed, the /etc/hosts configuration (be sure the ldap server resolves to the snapshot image and NOT the production ldap server), and zmlocalconfig
- Start Zimbra
If the /etc/hosts file contains entries to the snapshot version of the ldap server and the snapshot version of the mailbox server, neither server should be communicating with the production environment, nor available for accidental participation in normal mail activities.
Now, browse to the Admin console of the Zimbra Mailbox server on the new IP address at port 7071 to confirm Zimbra is running on the restored server. Select the account you need to restore and click "View Mail". The action will attempt to open another tab in your browser with a redirect to the Public Service host name for your domain. Click in the address bar and change the servername to the IP address of the restored Zimbra Mailbox server. This will allow you to view the mailbox of the account on the restored server. From Preferences, select Import/Export. Use the Zimbra Export search capabilities to export a tar file with the exact contents of the mailbox (folders, mail items, calendar entries, address book entries) needed to be restored. The tar file can be copied to the production server and imported by the user to recover lost items.
Restore from SAN based Snapshot
SAN based snapshot technologies allow you to take a picture(snapshot) of the current data on disk (LUN) for a given disk. For the purposes of this article, the technologies demonstrated are Virtual Machines on ESXi 4.1 as the host server and OpenFiler as the SAN.
This shows the ESXi host server with two virtual machines, a Zimbra LDAP Server and a Zimbra Mailbox/MTA Server.
This shows the OpenFiler SAN Configuration. All virtual disks reside on the vmstore1 volume using a single LUN.
From the SAN Management Console, take a snapshot of the existing volume. I have named the snapshot "vmstore2", set the snapshot size to 20 Gb, and selected Yes to share the snapshot. For this article, this single snapshot includes all storage for the Zimbra LDAP server and the Zimbra Mailbox/MTA server.
Next, we will map the snapshot as a new LUN to the existing iSCSI target so that the ESXi server can see it.
On the ESXi server, it is may be necessary to perform a rescan of the Storage Adapters to see the new LUN.
Click Storage on the Configuration tab of the ESXi server,
- Click Add Storage
- Select Disk/LUN and click Next (it may take a moment to recognize the new LUN)
- Select the new LUN and click Next (make sure it is the LUN assigned from the Snapshot)
- Select "Assign a new Signature" and click next
- Click Next and Finished to add the Snapshot as a new storage on the ESXi host.
You can now Browse the storage and add the virtual machines to inventory.
- Start the new VM. When prompted, indicate the machine was "copied", not moved. This should initialize the new virtual server network settings as DHCP instead of static addressing. Because the new server has a different IP address that what is in /etc/hosts, Zimbra services will not start on the new machine.
- Start Zimbra on each VM created from a snapshot.
If the /etc/hosts file contains entries to the snapshot version of the ldap server and the snapshot version of the mailbox server, neither server should be communicating with the production environment, nor available for accidental participation in normal mail activities. An administrator who knows the IP address of the restored servers should be able to login to the administrator console, view mail of an individual account (will need to modify the redirect string), and export data from the account.