Ajcody-Notes-Archive-Discovery-Mailstore-Setup

From Zimbra :: Wiki

Revision as of 16:58, 26 October 2011 by Ajcody (Talk | contribs)

Jump to: navigation, search
Attention.png - This article is NOT official Zimbra documentation. It is a user contribution and may include unsupported customizations, references, suggestions, or information.

Contents

Actual Multi-Server & New Mailstore A&D Setup Homepage

Please see Ajcody-Notes-Archive-Discovery-Mailstore-Setup

Issues That Have Caused Confusion

What Gets Installed Where?
zimbra-archive package/rpm - Mailstores

zimbra-archive (the package/rpm you see from the installer) should be installed on all mailstores which you want to use for cross mailbox search. This also sets the zimbraComponentAvailable archiving config attribute which allows the mta(s) to turn on archiving. zimbra-archive is not installed directly on the mta, it's just enabled.

Note, you install zimbra-archive on a mailbox server but the service runs on the mta node.

MTA's - Require Configuration

If you add zimbra-archiving to an existing install you need to :

  • Install zimbra-archiving on one or more of your mailbox servers
  • Then set zimbraServiceInstalled archiving and zimbraServiceEnabled archiving on all the mta servers

For example:

zmprov ms mta.example.com +zimbraServiceInstalled archiving +zimbraServiceEnabled archiving
zimbra_xmbxsearch zimlet

For 5.x installs, this zimbra_xmbsearch zimlet will get configured on each mailstore that you install the zimbra-archive package on. The documentation in various places might cause confusion on this matter, because for the 4.x releases it was a separate step.

You should find the zimlet already located at /opt/zimbra/zimlets-network/zimbra_xmbxsearch.zip

After they installation, you should see when you goto the admin web console on the mailstore you install the zimbra-archive package on that the cross-mailbox search zimlet is there. It shows up in two locations:

  • Left Pane : Configuration > Admin Extensions > zimbra_xmbxsearch
  • Left Pane : Tools > Search Mail

If you wanted this zimlet to also be available on a server that didn't have the zimbra-archiving packaged installed you could then deploy it on that server.

cd /opt/zimbra/zimlets-network/
zmzimletctl deploy zimbra_xmbxsearch.zip
## ls the directory and confirm the full name - you might need this:
zmzimletctl deploy com_zimbra_xmbxsearch.zip

The How-To

Reference Documents

http://www.zimbra.com/docs/ne/latest/multi_server_install/multi-server_install.5.1.html

http://www.zimbra.com/docs/ne/latest/administration_guide/Archiving.16.1.html

Assumptions

This install how-to assumes you have an existing LDAP/Mailstore/MTA server(s) for your normal production environment, the Zimbra license and logger are installed on the primary ZCS server(s), and that you are NOT running the proxy module.

Example archive mailstore hostname is : archive.example.com

Example primary ZCS hostname is : mail.example.com

Preliminary Items

Things to do or check before install:

  • DNS entry for new mailstore and primary ZCS server(s) can resolve to it.
  • DNS configured properly on mailstore server.
  • Master Root LDAP Server mail.example.com
  • Master Root LDAP Password
    • On LDAP server do : su – zimbra ; zmlocalconfig –s | grep ldap_root_password
  • Master LDAP port – default is 389
  • SMTP Server
Installation Of New Mailstore That Will Have A&D
Install Modules
  • Type y to install the zimbra-store, zimbra-archiving and zimbra-spell (optional) packages.
    • Do Not Install MTA! These Instructions Do Not Take That Into Account.
    • When zimbra-spell is installed the zimbra-apache package is also installed.
  • Installing: zimbra-core zimbra-store zimbra-apache zimbra-spell
Modify Configuration

Press Enter to modify the system. The selected packages are installed on the server.

At this point the Main menu displays the default entries for the Zimbra component you are installing.

To expand the menu to see the configuration values type x and press Enter.

To navigate the Main menu, select the menu item to change. You can modify any of the defaults.

  • Common Configuration
    • LDAP
      • Ldap master host: [set this to the FQDN of your LDAP server]
      • Ldap port: 389 [set this if your LDAP server isn’t using default]
      • Ldap Admin password: [this is your LDAP servers Root LDAP password]
        • On LDAP server do : su – zimbra ; zmlocalconfig –s ldap_root_password
      • TimeZone: [set this]
  • For zimbra-store
    • Set the Admin Password
      • +License filename: UNSET [if you see this, then something is wrong with your
    • LDAP configuration. It should of pulled the license info from the LDAP server.
    • Set the SMTP host

Type r to return to the Main menu, if you aren’t there already.

When the mailbox server is configured, return to the Main menu and type a to apply the configuration changes.

Press Enter to save the configuration data.

When Save Configuration data to a file appears, press Enter.

The next request is where to save the files. To accept the default, press Enter.

To save the files to another directory, enter the directory and then press Enter.

When “The system will be modified - continue?” appears type y and press Enter.

The server is modified.

Installing all the components and configuring the server can take a few minutes.

When Installation complete - press return to exit displays, press Enter.

The installation of the mailbox server is complete.

After Install

Confirm server status

su – zimbra ; zmcontrol status

Populate the ssh keys, on each server in your environment

su - zimbra ;  zmupdateauthkeys 

The key is updated on /opt/zimbra/.ssh/authorized_keys.

Upgrading A Zimbra Server For An Archive & Discovery Mailstore
Adding Package For A&D

This will retain your current settings for the system. Your server will experience downtime during the upgrade.

untar zcs*.tar that matches your existing system

 cd zcs-version-directory
 ./install
  choose upgrade
  select zimbra-archiving

The upgrade of the mailbox server is complete.

After Upgrade

Confirm server status

su – zimbra ; zmcontrol status

Note, zimbra-archiving only runs as a service on a MTA server.

Populate the ssh keys, on each server in your environment

su-zimbra ;  zmupdateauthkeys 

The key is updated on /opt/zimbra/.ssh/authorized_keys.

Configure Zimbra For Use Of The New Mailstore and A&D

Example A&D mailstore hostname is : archive.example.com

  • Go to your primary admin console url. [https://[example.com]:7071/zimbraAdmin]
  1. Confirm you see the new mailstore under Configuration > Servers
    1. Under Configuration > Servers > [MTA servername(s)] > Services
      1. [each MTA server needs this]
      2. You’ll see a box for Archiving and Discovery
        1. Check this to enable the MTA server(s) for Archiving and Discovery. If this is grayed out, run the command below (modified for your server) on your one of your mailstores.
          • This effectively does:
          • zmprov ms mta.example.com +zimbraServiceInstalled archiving  +zimbraServiceEnabled archiving
            • Remember, zmprov uses the variable below. A mta only server can't be set for localhost, change it to point to a mailstore.
            •  [root@mta ~]# zmlocalconfig | grep zmprov
            •     zimbra_zmprov_default_soap_server = localhost
            •  [root@mta ~]# zmlocalconfig -e zimbra_zmprov_default_soap_server=mailstore.example.com
  2. Go to Configuration > Class of Service > default [or your primary domain] > Server Pool
    1. You’ll want to make sure it’s limited to the correct server pools
      1. Your new mailstore for A&D should be unchecked.
    2. Click on New for a new Class of Server (COS)
      1. Call it archive or something similar
        1. Under Server Pool > Limit > have only the new mailstore checked
Setup Initial A&D With First Account - Creation Of The Archive Domain
Revisit To COS - Naming Scheme Of Archive Accounts

When archive accounts are created they use the zimbraArchiveAccountNameTemplate variable from the COS. The default is:

$ zmprov gc default | grep -i archive
  zimbraArchiveAccountDateTemplate: yyyyMMdd
  zimbraArchiveAccountNameTemplate: ${USER}-${DATE}@${DOMAIN}.archive

I, personally, don't like the use of the $DATE variable in this. I change my ARCHIVE COS to use the normal username but retain the .archive for the domain.

zmprov mc archive zimbraArchiveAccountNameTemplate '${USER}@${DOMAIN}.archive'

Bug to be aware of:

The Creation

On server with zmarchiveconfig (most likely mailstore you installed A&D on) and as zimbra (su – zimbra) do the following to setup your first A&D account.

format : zmarchiveconfig –s servername enable user@example.com archive-cos <cos>

example :

zmarchiveconfig –s archive.example.com enable account@example.com archive-cos archive

NOTE

If the above command doesn't seem to create the archive account/domain. Drop the use of [ -s servername ]. Basically, just run this on the A&D mailstore:
zmarchiveconfig enable account@example.com archive-cos archive

The above command will create the mail domain for the archive accounts using the template defaults, user@example.com to make example.com.archive

On your main ldap server or where ever you usually access the zimbra admin web console, login to the admin web console.

  1. Confirm the archive domain was setup.
    1. Configuration > Domains > [domainname].archive > General
    2. Confirm or adjust the archive domain to use the right COS
      1. Configuration > Domains > [domainname].archive > General Information
        1. Change “Default Class of Service” to your COS [archive], if needed for your configuration.
  2. Now check for the new archive account you made
    1. Address > Accounts
    2. Click on account and hit the edit button
    3. In the top summary section you'll be able to confirm the COS and Mail Server being used for the account.
      1. NOTE, if it's showing the account is on the primary mailstore and NOT the A&D mailstore.
        1. Remove the A&D account
          •  zmprov ra [user]@[domainname].archive
        2. Add the account back again using the zmarchiveconfig command
          • zmarchiveconfig enable account@example.com archive-cos archive
        3. Now confirm, as above, that the account is using the A&D mailstore.
          • This might be a bug related to the archive domain being created for the first time.

Send the primary account a test email and then shortly afterwards do a "View Mail" within the admin console for the archive account. You should see the archive message in the archive account.

Testing Of Archive Mail Flow

Send the primary account a test email and then shortly afterwards do a "View Mail" within the admin console for the archive account. You should see the archive message in the archive account.

You should confirm mail flow copies occur with the following:

  1. Inbound
    1. External Account (email) to the primary zimbra account setup for archive.
    2. A zimbra account that ISN'T the archive account in question to the primary account setup for archive.
  2. Outbound
    1. With primary account setup for archive, send an email to an external email address.
    2. With primary account setup for archive, send an email to another internal zimbra email address.
Archive Account Isn't Getting Email

Let's double check everything was done correctly up above.

Assumption on syntax of account creation:

zmarchiveconfig enable user@example.com archive-cos archive

Let's check what actually was done:

zmprov ga user@example.com | grep -i archive
 amavisArchiveQuarantineTo: user-20081211@example.com.archive
 zimbraArchiveAccount: user-20081211@example.com.archive
 zimbraArchiveAccountNameTemplate: ${USER}-${DATE}@${DOMAIN}.archive

It should reference an account that's like, if you are using the archive templates:

user-[date]@example.com.archive

that account should exist and reference lmtp, rather than smtp, for the transport:

zmprov ga user-20081211@example.com.archive | grep -i trans
  zimbraMailTransport: lmtp:archive.example.com:7025
Checking Logs For Archive Operations

On the mta-server, you should find a reference to the archive account in /var/log/zimbra.log

grep archive /var/log/zimbra.log
 Dec 11 13:38:52 mta-server amavis[1978]: (01978-19) SEND via SMTP: <> -> 
    <user-20081211@example.com.archive>,ENVID=AM.8ISxcrQG8uAj.20081211T193852Z@mailstore.example.com 
    BODY=7BIT 250 2.6.0 Ok, id=01978-19, from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 
    5ADF8F120C4
 Dec 11 13:38:52 mta-server postfix/lmtp[21864]: 5ADF8F120C4: 
    to=<user-20081211@example.com.archive>, relay=archive.example.com[X.X.X.93]:7025, 
    delay=0.07, delays=0/0/0/0.06, dsn=2.1.5, status=sent (250 2.1.5 OK)

On the archive-server, you should find reference to the delivery in /opt/zimbra/log/mailbox.log

grep archive /opt/zimbra/log/mailbox.log
 2008-12-11 14:45:32,923 INFO  [LmtpServer-9] 
  [name=user-20081211@example.com.archive;mid=7;] mailop - Adding Message: id=257,
  Message-ID=<1350363939.41021229024728317.JavaMail.root@EXTERNAL-MTA.DOMAIN.com>, parentId=-1,
  folderId=2, folderName=Inbox.
Mass Accounts Configuration
Update, our 6.0 release will have a zmarchiveconfig -f command for batch processing from a file input.

CHECK YOUR AVAILABLE LICENSES BEFORE YOU PROCEED!!

One could put all the accounts in a txt file and then use a for-loop to process the account@example.com variable.

zmprov gaa > /tmp/accounts.txt

Remove any accounts you've already done and those not necessary for archiving (ex. admin, ham, spam, etc.)

You can give gaa other options, look at zmprov help account. For example, you could also narrow this down to a dump of accounts in a domain:

zmprov gaa [DOMAIN] > /tmp/accounts.txt

Note, the below uses the above setup for A&D - You'll need to modify for your environment.

for i in `cat /tmp/accounts.txt`
do
zmarchiveconfig –s archive.example.com enable $i archive-cos archive
sleep 3
done

You can be tailing /opt/zimbra/log/mailbox.log on the archive server to watch the progress.

Searches After Configuration Is Done

Please see Ajcody-Server-Misc-Topics#Cross_Mailbox_Searches_and_Tracing

Personal tools