Revision as of 17:42, 16 August 2008 by Ajcody (talk | contribs) (New page: ==Archive & Discovery== ===Actual Archive & Discovery Notes Homepage=== Ajcody-Notes-Archive-Discovery ===References for A&D=== The two main A&D references are : http://wiki.zimbra...)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)


Archive & Discovery

Actual Archive & Discovery Notes Homepage


References for A&D

The two main A&D references are :



Multi-Server & New Mailstore A&D Setup

I also created a RFE for documentation on this.


The following is a very rough draft document I made for multi-server / new mailstore A&D setups.

Ajcody Multi-Server & New Mailstore A&D Setup Homepage

   KB 2545        Last updated on 2008-08-16  

(0 votes)
24px ‎  - This is Zeta Alliance Certified Documentation. The content has been tested by the Community.

Actual Multi-Server & New Mailstore A&D Setup Homepage

Please see Ajcody-Notes-Archive-Discovery-Mailstore-Setup

Issues That Have Caused Confusion

What Gets Installed Where?
RFE To Clear Up The Confusion
zimbra-archive package/rpm - Mailstores

zimbra-archive (the package/rpm you see from the installer) should be installed on all mailstores which you want to use for cross mailbox search. This also sets the zimbraComponentAvailable archiving config attribute which allows the mta(s) to turn on archiving. zimbra-archive is not installed directly on the mta, it's just enabled.

Note, you install zimbra-archive on a mailbox server but the service runs on the mta node.

MTA's - Require Configuration

If you add zimbra-archiving to an existing install you need to :

  • Install zimbra-archiving on one or more of your mailbox servers
  • Then set zimbraServiceInstalled archiving and zimbraServiceEnabled archiving on all the mta servers
  • Restart the mta services

For example:

zmprov ms mta.example.com +zimbraServiceInstalled archiving +zimbraServiceEnabled archiving

On the mta server:

zmmtactl restart

To confirm the /opt/zimbra/conf/amavisd.conf was modified correctly, you should see on the mta:

#$archive_quarantine_method = 'smtp:[]:10025'; 

Was uncommented out:

$archive_quarantine_method = 'smtp:[]:10025';

You'll be able to then notice in the /var/log/zimbra.log file if the redirect to the A&D account is happening [once A&D accounts are setup that is]. Example uses example.com.archive as the archive domain I setup for the A&D accounts :

grep "example.com.archive" /var/log/zimbra.log
 Dec 11 13:38:52 mta-server amavis[1978]: (01978-19) SEND via SMTP: <> -> 
    BODY=7BIT 250 2.6.0 Ok, id=01978-19, from MTA([]:10025): 250 2.0.0 Ok: queued as 
 Dec 11 13:38:52 mta-server postfix/lmtp[21864]: 5ADF8F120C4: 
    to=<user-20081211@example.com.archive>, relay=archive.example.com[X.X.X.93]:7025, 
    delay=0.07, delays=0/0/0/0.06, dsn=2.1.5, status=sent (250 2.1.5 OK)
Enabling Amavis And Archiving With 8.5+ While Antispam And AntiVirus Are Disabled

With 808 and 8.5 , archiving should be able to run without as/av being enabled.

For example, disabling antispam and antivirus but enabling amavis [required] and archiving on a mta server [note - this server had the full mta package already installed on it and had antivirus, antispam, [amavis], and postfix running on it] :

[zimbra@850-mta1 ~]$ zmcontrol status | egrep 'amavis|antispam|antivirus|archiving'
	amavis                  Running
	antispam                Running
	antivirus               Running

## NOTE , the below command will disable antispam & anitvirus for this "test". ##

[zimbra@850-mta1 ~]$ zmprov ms `zmhostname` +zimbraServiceEnabled archiving \
 +zimbraServiceEnabled amavis -zimbraServiceEnabled antispam -zimbraServiceEnabled antivirus

[zimbra@850-mta1 ~]$ zmcontrol restart                
Host 850-ldap1.zimbra.homeunix.com
	Stopping vmware-ha...Done.
	Stopping ldap...Done.
Host 850-ldap1.zimbra.homeunix.com
	Starting ldap...Done.
	Starting zmconfigd...Done.
	Starting dnscache...Done.
	Starting logger...Done.
	Starting convertd...Done.
	Starting mailbox...Done.
	Starting memcached...Done.
	Starting proxy...Done.
	Starting amavis...Done.
	Starting opendkim...Done.
	Starting archiving...Done.
	Starting snmp...Done.
	Starting spell...Done.
	Starting mta...Done.
	Starting stats...Done.
	Starting service webapp...Done.
	Starting zimbra webapp...Done.
	Starting zimbraAdmin webapp...Done.
	Starting zimlet webapp...Done.
[zimbra@850-ldap1 ~]$ zmcontrol status | egrep 'amavis|antispam|antivirus|archiving'
	amavis                  Running
	archiving               Running
zimbra_xmbxsearch zimlet

For 5.x installs, this zimbra_xmbsearch zimlet will get configured on each mailstore that you install the zimbra-archive package on. The documentation in various places might cause confusion on this matter, because for the 4.x releases it was a separate step.

You should find the zimlet already located at /opt/zimbra/zimlets-network/zimbra_xmbxsearch.zip

After the installation, you should see when you go to the admin web console on the mailstore you install the zimbra-archive package on that the cross-mailbox search zimlet is there. It shows up in two locations:

  • Left Pane : Configuration > Admin Extensions > zimbra_xmbxsearch
  • Left Pane : Tools > Search Mail

If you wanted this zimlet to also be available on a server that didn't have the zimbra-archiving packaged installed you could then deploy it on that server.

cd /opt/zimbra/zimlets-network/
zmzimletctl deploy zimbra_xmbxsearch.zip
## ls the directory and confirm the full name - you might need this:
zmzimletctl deploy com_zimbra_xmbxsearch.zip

The How-To

Reference Documents




This install how-to assumes you have an existing LDAP/Mailstore/MTA server(s) for your normal production environment, the Zimbra license and logger are installed on the primary ZCS server(s), and that you are NOT running the proxy module.

Example archive mailstore hostname is : archive.example.com

Example primary ZCS hostname is : mail.example.com

Preliminary Items

Things to do or check before install:

  • DNS entry for new mailstore and primary ZCS server(s) can resolve to it.
  • DNS configured properly on mailstore server.
  • Master Root LDAP Server mail.example.com
  • Master Root LDAP Password
    • On LDAP server do : su – zimbra ; zmlocalconfig –s | grep ldap_root_password
  • Master LDAP port – default is 389
  • SMTP Server
Installation Of New Mailstore That Will Have A&D
Install Modules
  • Type y to install the zimbra-store, zimbra-archiving and zimbra-spell (optional) packages.
    • Do Not Install MTA! These Instructions Do Not Take That Into Account.
    • When zimbra-spell is installed the zimbra-apache package is also installed.
  • Installing: zimbra-core zimbra-store zimbra-apache zimbra-spell
Modify Configuration

Press Enter to modify the system. The selected packages are installed on the server.

At this point the Main menu displays the default entries for the Zimbra component you are installing.

To expand the menu to see the configuration values type x and press Enter.

To navigate the Main menu, select the menu item to change. You can modify any of the defaults.

  • Common Configuration
    • LDAP
      • Ldap master host: [set this to the FQDN of your LDAP server]
      • Ldap port: 389 [set this if your LDAP server isn’t using default]
      • Ldap Admin password: [this is your LDAP servers Root LDAP password]
        • On LDAP server do : su – zimbra ; zmlocalconfig –s ldap_root_password
      • TimeZone: [set this]
  • For zimbra-store
    • Set the Admin Password
      • +License filename: UNSET [if you see this, then something is wrong with your
    • LDAP configuration. It should of pulled the license info from the LDAP server.
    • Set the SMTP host

Type r to return to the Main menu, if you aren’t there already.

When the mailbox server is configured, return to the Main menu and type a to apply the configuration changes.

Press Enter to save the configuration data.

When Save Configuration data to a file appears, press Enter.

The next request is where to save the files. To accept the default, press Enter.

To save the files to another directory, enter the directory and then press Enter.

When “The system will be modified - continue?” appears type y and press Enter.

The server is modified.

Installing all the components and configuring the server can take a few minutes.

When Installation complete - press return to exit displays, press Enter.

The installation of the mailbox server is complete.

After Install

Confirm server status

su – zimbra ; zmcontrol status

Populate the ssh keys, on each server in your environment

su - zimbra ;  zmupdateauthkeys 

The key is updated on /opt/zimbra/.ssh/authorized_keys.

Upgrading A Zimbra Server For An Archive & Discovery Mailstore
Adding Package For A&D

This will retain your current settings for the system. Your server will experience downtime during the upgrade.

untar zcs*.tar that matches your existing system

 cd zcs-version-directory
  choose upgrade
  select zimbra-archiving

The upgrade of the mailbox server is complete.

After Upgrade

Confirm server status

su – zimbra ; zmcontrol status

Note, zimbra-archiving only runs as a service on a MTA server.

Populate the ssh keys, on each server in your environment

su-zimbra ;  zmupdateauthkeys 

The key is updated on /opt/zimbra/.ssh/authorized_keys.

Configure Zimbra For Use Of The New Mailstore and A&D

Example A&D mailstore hostname is : archive.example.com

  • Go to your primary admin console url. [https://[example.com]:7071/zimbraAdmin]
  1. Confirm you see the new mailstore under Configuration > Servers
    1. Under Configuration > Servers > [MTA servername(s)] > Services
      1. [each MTA server needs this]
      2. You’ll see a box for Archiving and Discovery
        1. Check this to enable the MTA server(s) for Archiving and Discovery. If this is grayed out, run the command below (modified for your server) on your one of your mailstores.
          • This effectively does:
          • zmprov ms mta.example.com +zimbraServiceInstalled archiving  +zimbraServiceEnabled archiving
            • Remember, zmprov uses the variable below. A mta only server can't be set for localhost, change it to point to a mailstore.
            •  [root@mta ~]# zmlocalconfig | grep zmprov
            •     zimbra_zmprov_default_soap_server = localhost
            •  [root@mta ~]# zmlocalconfig -e zimbra_zmprov_default_soap_server=mailstore.example.com
Configuring Your COS's - Normal COS's and Archiving COS

It's recommended that the archive accounts be created on a dedicated mailstore. You can limit what mailstores are used for new account creation by restricting what mailstores are used under "Server Pool" within a COS. Your normal COS's should excluded your archive mailstores and your archive COS should only have archive mailstores selected.

You'll also want to make sure your archive COS isn't using any 'features' that aren't necessary for archive accounts. No point in consuming certain license features when they aren't needed.

  1. Go to Configuration > Class of Service > default [or your primary domain] > Server Pool
    1. You’ll want to make sure it’s limited to the correct server pools
      1. Your new mailstore for A&D should be unchecked.
    2. Click on New for a new Class of Server (COS)
      1. Call it archive or something similar
        1. Under Server Pool > Limit > have only the new mailstore checked
    3. Make sure your not using licensed features [EWS for example] that aren't necessary for your archive accounts.
Setup Initial A&D With First Account - Creation Of The Archive Domain
Revisit To COS - Naming Scheme Of Archive Accounts

When archive accounts are created they use the zimbraArchiveAccountNameTemplate variable from the COS. The default is:

$ zmprov gc default | grep -i archive
  zimbraArchiveAccountDateTemplate: yyyyMMdd
  zimbraArchiveAccountNameTemplate: ${USER}-${DATE}@${DOMAIN}.archive

I, personally, don't like the use of the $DATE variable in this. I change my ARCHIVE COS to use the normal username but retain the .archive for the domain.

zmprov mc archive zimbraArchiveAccountNameTemplate '${USER}@${DOMAIN}.archive'

Bug to be aware of:

The Creation

On server with zmarchiveconfig (most likely mailstore you installed A&D on) and as zimbra (su – zimbra) do the following to setup your first A&D account.

format : zmarchiveconfig –s servername enable user@example.com archive-cos <cos>

example :

zmarchiveconfig –s archive.example.com enable account@example.com archive-cos archive


If the above command doesn't seem to create the archive account/domain. Drop the use of [ -s servername ]. Basically, just run this on the A&D mailstore:
zmarchiveconfig enable account@example.com archive-cos archive

The above command will create the mail domain for the archive accounts using the template defaults, user@example.com to make example.com.archive

On your main ldap server or where ever you usually access the zimbra admin web console, login to the admin web console.

  1. Confirm the archive domain was setup.
    1. Configuration > Domains > [domainname].archive > General
    2. Confirm or adjust the archive domain to use the right COS
      1. Configuration > Domains > [domainname].archive > General Information
        1. Change “Default Class of Service” to your COS [archive], if needed for your configuration.
  2. Now check for the new archive account you made
    1. Address > Accounts
    2. Click on account and hit the edit button
    3. In the top summary section you'll be able to confirm the COS and Mail Server being used for the account.
      1. NOTE, if it's showing the account is on the primary mailstore and NOT the A&D mailstore.
        1. Remove the A&D account
          •  zmprov ra [user]@[domainname].archive
        2. Add the account back again using the zmarchiveconfig command
          • zmarchiveconfig enable account@example.com archive-cos archive
        3. Now confirm, as above, that the account is using the A&D mailstore.
          • This might be a bug related to the archive domain being created for the first time.

Send the primary account a test email and then shortly afterwards do a "View Mail" within the admin console for the archive account. You should see the archive message in the archive account.

Error: unknown document: EnableArchiveRequest

If you get this error when trying to create the archive account "Error: unknown document: EnableArchiveRequest" you most likely needed to install a new license for A&D and have not restart the mailboxd services . Updating the license is not enough, you'll need to restart ZCS on the mailstores also.

See the following bug:

RFE's On Archive Accounts
Testing Of Archive Mail Flow

Send the primary account a test email and then shortly afterwards do a "View Mail" within the admin console for the archive account. You should see the archive message in the archive account.

You should confirm mail flow copies occur with the following:

  1. Inbound
    1. External Account (email) to the primary zimbra account setup for archive.
    2. A zimbra account that ISN'T the archive account in question to the primary account setup for archive.
  2. Outbound
    1. With primary account setup for archive, send an email to an external email address.
    2. With primary account setup for archive, send an email to another internal zimbra email address.
Archive Account Isn't Getting Email

Let's double check everything was done correctly up above.

Assumption on syntax of account creation:

zmarchiveconfig enable user@example.com archive-cos archive

Let's check what actually was done:

zmprov ga user@example.com | grep -i archive
 amavisArchiveQuarantineTo: user-20081211@example.com.archive
 zimbraArchiveAccount: user-20081211@example.com.archive
 zimbraArchiveAccountNameTemplate: ${USER}-${DATE}@${DOMAIN}.archive

It should reference an account that's like, if you are using the archive templates:


that account should exist and reference lmtp, rather than smtp, for the transport:

zmprov ga user-20081211@example.com.archive | grep -i trans
  zimbraMailTransport: lmtp:archive.example.com:7025
Checking Logs For Archive Operations

On the mta-server, you should find a reference to the archive account in /var/log/zimbra.log

grep archive /var/log/zimbra.log
 Dec 11 13:38:52 mta-server amavis[1978]: (01978-19) SEND via SMTP: <> -> 
    BODY=7BIT 250 2.6.0 Ok, id=01978-19, from MTA([]:10025): 250 2.0.0 Ok: queued as 
 Dec 11 13:38:52 mta-server postfix/lmtp[21864]: 5ADF8F120C4: 
    to=<user-20081211@example.com.archive>, relay=archive.example.com[X.X.X.93]:7025, 
    delay=0.07, delays=0/0/0/0.06, dsn=2.1.5, status=sent (250 2.1.5 OK)

On the archive-server, you should find reference to the delivery in /opt/zimbra/log/mailbox.log

grep archive /opt/zimbra/log/mailbox.log
 2008-12-11 14:45:32,923 INFO  [LmtpServer-9] 
  [name=user-20081211@example.com.archive;mid=7;] mailop - Adding Message: id=257,
  Message-ID=<1350363939.41021229024728317.JavaMail.root@EXTERNAL-MTA.DOMAIN.com>, parentId=-1,
  folderId=2, folderName=Inbox.
Mass Accounts Configuration
Update 5/28/15, I filed the following RFE:
  • "Redesign archive account creation process [autocreate, mass creation/enabling, etc]"


Update, our 6.0 release will have a zmarchiveconfig -f command for batch processing from a file input.


One could put all the accounts in a txt file and then use a for-loop to process the account@example.com variable.

zmprov -l gaa > /tmp/accounts.txt

Remove any accounts you've already done and those not necessary for archiving (ex. admin, ham, spam, etc.)

You can give gaa other options, look at zmprov help account. For example, you could also narrow this down to a dump of accounts in a domain:

zmprov -l gaa [DOMAIN] > /tmp/accounts.txt

Note, the below uses the above setup for A&D - You'll need to modify for your environment.

for i in `cat /tmp/accounts.txt`
zmarchiveconfig –s archive.example.com enable $i archive-cos archive
sleep 3

You can be tailing /opt/zimbra/log/mailbox.log on the archive server to watch the progress.

Searches After Configuration Is Done

Please see Ajcody-Server-Misc-Topics#Cross_Mailbox_Searches_and_Tracing

Searches Limited To 500 or 1000 Maximum Results

See Ajcody-Server-Misc-Topics#Searches_Limited_To_500_or_1000_Maximum_Results

Jump to: navigation, search