Ajcody-Notes-Archive-Discovery: Difference between revisions

Line 807: Line 807:


<pre>
<pre>
[zimbra@ldap2 ~]$ zmprov -l gaa
$ zmprov -l gaa
admin@ldap2.zimbra.DOMAIN.com
admin@ldap2.zimbra.DOMAIN.com
spam.gktt7hym36@ldap2.zimbra.DOMAIN.com
spam.gktt7hym36@ldap2.zimbra.DOMAIN.com
Line 819: Line 819:
testuser2@test2.zimbra.DOMAIN.com
testuser2@test2.zimbra.DOMAIN.com


[zimbra@ldap2 ~]$ zmprov -l sa "amavisArchiveQuarantineTo=*"
$ zmprov -l sa "amavisArchiveQuarantineTo=*"
user1@ldap2.zimbra.DOMAIN.com
user1@ldap2.zimbra.DOMAIN.com
$ zmprov ga user1@`zmhostname` | egrep -i 'zimbraArchiveEnabled|amavisArchiveQuarantineTo\
|zimbraArchiveAccount:|zimbraIsSystemResource'
amavisArchiveQuarantineTo: user1-archive1@ldap2.zimbra.homeunix.com.archive, user1-archive2@ldap2.zimbra.homeunix.com.archive
zimbraArchiveAccount: user1-archive1@ldap2.zimbra.homeunix.com.archive
zimbraArchiveAccount: user1-archive2@ldap2.zimbra.homeunix.com.archive
zimbraArchiveEnabled: TRUE
zimbraIsSystemResource: FALSE


[zimbra@ldap2 ~]$ zmprov fc -a all ; zmsoap -z GetLicenseRequest | egrep 'TotalAccounts|ArchivingAccounts'
[zimbra@ldap2 ~]$ zmprov fc -a all ; zmsoap -z GetLicenseRequest | egrep 'TotalAccounts|ArchivingAccounts'

Revision as of 23:31, 13 February 2015

Attention.png - This article is NOT official Zimbra documentation. It is a user contribution and may include unsupported customizations, references, suggestions, or information.

Archive & Discovery

Actual Archive & Discovery Notes Homepage

Ajcody-Notes-Archive-Discovery

References for A&D

Updated A&D Documentation In Admin Manual

Please see:

Zimbra ZSC Version 8.x

http://www.zimbra.com/docs/ne/8.0.2/administration_guide/wwhelp/wwhimpl/js/html/wwhelp.htm#href=ZCS_8_0_2_Admin_Guide.Archiving_and_Discovery.html

The version 7.x manual is more comprehensive. Version 8 Documentation is missing a lot of content available in Version 7 documentation.

Zimbra ZSC Version 7.x

http://www.zimbra.com/docs/ne/latest/administration_guide/wwhelp/wwhimpl/js/html/wwhelp.htm#href=ZCS_Admin_Guide_7.2.1.Zimbra_Archiving_and_Discovery.html

Older Notes About ZAD

Important Note:

I think we need to clean up the documentation that refers to this "download" as no longer being necessary as it's bundled in the zcs-network edition tar ball now. I found an old email to the development team where I inquired about this - I didn't get a response from on it at the time.

If this assumption is true [Like I see with the 5.x setups I've helped with] then I think we need to adjust the documents for 4.5.x stuff to state:

You'll need to download your version of 4.5.x zcs network edition tarball, untar it, and then run the install script . Which effectively will allow you to now select the "archive" package which will also include the search zimlet [zmbxsearch].


The two main A&D references are :

http://wiki.zimbra.com/index.php?title=ZAD

http://www.zimbra.com/pdf/Zimbra%20Archiving%20and%20Discovery%20Release%20Notes.pdf

But I Just Want Some BCC's To Happen - Not All This A&D Stuff

Please see Ajcody-MTA-Postfix-Topics#Automatic_BCC instead then.

But We Have A Non-Zimbra MTA - zimbraSmtpHostname & zimbraMtaRelayHost Pointed To External Device

Please see this RFE I made:

  • "RFE: Ability to handle A&D when another MTA systems is being specified"

Notable Bugs Or RFE's

Update Jan 22, 2015 . I filed these while testing against ZCS 8.6 .


Older Bug/RFE's I've noted:

Archive Stop Working After Upgrade Or Disabling Anti-Spam - Anti-Virus

Prior to JP/8.5 , the archiving feature required the anti-spam or anti-virus service to be enabled. If you disabled both of them, then amavis as a whole was disabled. This is required for archiving to work. The following bug resolves this for JP/8.5 - it makes amavis its own service:

Also note, my testing on ZCS 8.0.7 did confirm that disabling av/as [therefore, amavis] will cause archiving to not work. This is very confusing since zmcontrol status AND the admin console will show that the service for "archiving" is running - though archiving is not actually working since amavis is disabled.

Update Jan 22, 2015. New bug filed to deal with other upgrade issues we discovered.

First - Short & Sweet How-To On CLI With Single ZCS 8.6 Server

In this example, I'll turn off the antivirus and antispam service in 8.6 and then enable amavis and archiving. Create an archive cos and then archive user. This was a single ZCS 8.6 server with all services installed during the installation - just using default values. This was NOT an upgrade to ZCS 8.6 . 

[zimbra@ldap2 ~]$ zmcontrol -v 
Release 8.6.0_GA_1153.RHEL6_64_20141215151258 RHEL6_64 NETWORK edition. 

[zimbra@ldap2 ~]$ zmprov gs `zmhostname` | grep Service | egrep 'amavis|antivirus|antispam|archiving' 
zimbraServiceEnabled: amavis 
zimbraServiceEnabled: antivirus 
zimbraServiceEnabled: antispam 
zimbraServiceEnabled: archiving 
zimbraServiceInstalled: amavis 
zimbraServiceInstalled: antivirus 
zimbraServiceInstalled: antispam 
zimbraServiceInstalled: archiving 

[zimbra@ldap2 ~]$ zmprov ms `zmhostname` +zimbraServiceInstalled archiving +zimbraServiceEnabled archiving 

[zimbra@ldap2 ~]$ zmprov ms `zmhostname` -zimbraServiceEnabled antispam -zimbraServiceEnabled antivirus 

[zimbra@ldap2 ~]$ zmprov gs `zmhostname` | grep Service | egrep 'amavis|antivirus|antispam|archiving' 
zimbraServiceEnabled: amavis 
zimbraServiceEnabled: archiving 
zimbraServiceInstalled: amavis 
zimbraServiceInstalled: antivirus 
zimbraServiceInstalled: antispam 
zimbraServiceInstalled: archiving 


!! Note - zmmtactl restart didn't cause the amavisd.conf file to change, hence why I then went with zmcontrol restart. 

[zimbra@ldap2 ~]$ zmcontrol restart 

[zimbra@ldap2 ~]$ grep archive_quar /opt/zimbra/conf/amavisd.conf 
archive_quarantine_method => undef, # Don't run archiving a second time 
$archive_quarantine_method = 'smtp:[127.0.0.1]:10025'; 
$archive_quarantine_to = undef; 


[zimbra@ldap2 ~]$ zmprov cc archive 
71f9d7f4-54cc-4bf3-a6e1-94a1fc38a129 

[zimbra@ldap2 ~]$ zmarchiveconfig enable user1@`zmhostname` archive-cos archive 

[zimbra@ldap2 ~]$ vi /tmp/email.txt 

[zimbra@ldap2 attrs]$ cat /tmp/email.txt
To: user1@ldap2.zimbra.DOMAIN.com
Subject: Test For Archive - CLI
From: admin@ldap2.zimbra.DOMAIN.com
test

[zimbra@ldap2 ~]$ /opt/zimbra/postfix/sbin/sendmail -t < /tmp/email.txt 

[zimbra@ldap2 ~]$ grep archive /var/log/zimbra.log 

Jan 15 16:09:43 ldap2 amavis[29283]: (29283-01) h9nw0WdCl1Tw(h9nw0WdCl1Tw) SEND from <> -> <user1-20150115@ldap2.zimbra.DOMAIN.com.archive>,
ENVID=AM.h9nw0WdCl1Tw.20150115T210943Z@ldap2.zimbra.DOMAIN.com BODY=7BIT 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued 
as 2DE506445C 
Jan 15 16:09:43 ldap2 amavis[29283]: (29283-01) Passed CLEAN {RelayedInbound,Archived}, <zimbra@ldap2.zimbra.DOMAIN.com> -> 
<user1@ldap2.zimbra.DOMAIN.com>, quarantine: user1-20150115@ldap2.zimbra.DOMAIN.com.archive, Message-ID: 
<20150115210943.087106438E@ldap2.zimbra.DOMAIN.com>, mail_id: h9nw0WdCl1Tw, Hits: -, size: 351, queued_as: 3049C64466, 138 ms 
Jan 15 16:09:45 ldap2 postfix/lmtp[3478]: 2DE506445C: to=<user1-20150115@ldap2.zimbra.DOMAIN.com.archive>, 
relay=ldap2.zimbra.homeunix.com[192.168.1.172]:7025, delay=2.2, delays=0/0.02/0.17/2, dsn=2.1.5, status=sent (250 2.1.5 Delivery OK)
When Was A ZCS Service Enabled Or Disabled

Note - single ZCS server deployment example. Have to double check on multi-server deployment if the command needs to be tweaked.

Check Current Status Of Services Enabled-Disable and Installed-NotInstalled

Example is done on the host in question: 

zmprov -l gs `zmhostname` | egrep 'zimbraServiceEnabled|zimbraServiceInstall'

The -l is used in case mailboxd is the service in question, -l makes the call to ldap to check the configuration of the server.

Check Backups And What LDAP Has Stored

Do the following, adjust the backup path if your not using the default path: 

 for i in `find /opt/zimbra/backup/sessions/ -name ldap.bak.gz -print`;  \
 do echo $i; zgrep zimbraServiceEnabled $i; done

Will show you something like : 

zimbraServiceEnabled: antivirus
zimbraServiceEnabled: antispam
zimbraServiceEnabled: opendkim
zimbraServiceEnabled: logger
zimbraServiceEnabled: mailbox
zimbraServiceEnabled: mta
zimbraServiceEnabled: convertd
zimbraServiceEnabled: stats
zimbraServiceEnabled: snmp
zimbraServiceEnabled: ldap
zimbraServiceEnabled: spell
zimbraServiceEnabled: archiving
Check The History Of The Service Status In zimbra-stats.log

In /var/log/ you'll see zimbra-stats.log and archives of that file, for example - zimbra-stats.log-20140714 . Here's an example of a zgrep for the archive service: 

[zimbra@zcs807 log]$ zgrep archiv zimbra-stats* | tail
zimbra-stats.log:Jul 22 08:24:08 zcs807 zimbramon[27645]: 27645:info: 2014-07-22 08:24:01, 
 STATUS: zcs807.us.zimbralab.com: archiving: Running
zimbra-stats.log:Jul 22 08:26:09 zcs807 zimbramon[28689]: 28689:info: 2014-07-22 08:26:01, 
 STATUS: zcs807.us.zimbralab.com: archiving: Running
zimbra-stats.log-20140714:Jul 15 14:40:19 zcs807 zimbramon[21519]: 21519:info: 2014-07-15 14:40:01, 
 STATUS: zcs807.us.zimbralab.com: archiving: Stopped
zimbra-stats.log-20140714:Jul 15 14:42:06 zcs807 zimbramon[24998]: 24998:info: 2014-07-15 14:42:01, 
 STATUS: zcs807.us.zimbralab.com: archiving: Stopped
zimbra-stats.log-20140714:Jul 15 14:44:09 zcs807 zimbramon[29250]: 29250:info: 2014-07-15 14:44:01, 
 STATUS: zcs807.us.zimbralab.com: archiving: Running
zimbra-stats.log-20140714:Jul 15 14:46:08 zcs807 zimbramon[32188]: 32188:info: 2014-07-15 14:46:01, 
 STATUS: zcs807.us.zimbralab.com: archiving: Running
zimbra-stats.log-20140714:Jul 15 14:48:26 zcs807 zimbramon[3174]: 3174:info: 2014-07-15 14:48:07, 
 STATUS: zcs807.us.zimbralab.com: archiving: Running
zimbra-stats.log-20140714:Jul 15 14:50:15 zcs807 zimbramon[6466]: 6466:info: 2014-07-15 14:50:01, 
 STATUS: zcs807.us.zimbralab.com: archiving: Running
zimbra-stats.log-20140714:Jul 15 14:52:10 zcs807 zimbramon[10260]: 10260:info: 2014-07-15 14:52:02, 
 STATUS: zcs807.us.zimbralab.com: archiving: Running
zimbra-stats.log-20140714:Jul 15 14:54:10 zcs807 zimbramon[19004]: 19004:info: 2014-07-15 14:54:01, 
 STATUS: zcs807.us.zimbralab.com: archiving: Running
Check The Logger/RRD Data And Service History Uptime

Let's pull the stats from the logger database going back to Jan 1st of this year. Here's an example on how to do that and what to expect. Please send me a copy of the output also. Notice in the example below the lines that have ",," for column 9 - which is for archiving. This is how I can tell when the service was first enabled and running. Ref for timestamp: http://www.epochconverter.com/

Note: The columns are identified by: 

timestamp,opendkim,zmconfigd,mailbox,proxy,spell,logger,antivirus,archiving,snmp,ldap,memcached,
  convertd,stats,antispam,mta

Please adjust the commands below to suit your needs. I use `zmhostname` below in the command rather than typing out the localhosts hostname. Also, see https://www.gnu.org/software/coreutils/manual/html_node/Examples-of-date.html for examples on the --date string.


For this example, I'm looking to see when the "archiving" service was running 100%. Archiving is the 9th column - '$9 == 1' and the 1 means it was available 100% for the time period. I searched a year's worth of data by using this part below in the examples : "-s `date +%s --date="12 month ago"` -e `date +%s --date="1 minute ago"`" 

 /opt/zimbra/libexec/zmrrdfetch -f zmstatuslog -h `zmhostname` -s `date +%s \
 --date="12 month ago"` -e `date +%s --date="1 minute ago"` | awk -F, '$9 == 1'

timestamp,opendkim,zmconfigd,mailbox,proxy,spell,logger,antivirus,archiving,snmp,ldap,memcached,
  convertd,stats,antispam,mta
 [cut]
1405454400,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
1405458000,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
1405461600,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
1405465200,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
1405468800,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
1405472400,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
1405476000,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
1405479600,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
 [cut]

For this example, I'm looking to see when the "archiving" service was NOT enabled or running at all. Archiving is the 9th column - '$9 == ""' and the absence of a number value means it was UNAVAILABLE 100% for the time period. 

 /opt/zimbra/libexec/zmrrdfetch -f zmstatuslog -h `zmhostname` -s `date +%s \
 --date="12 month ago"` -e `date +%s --date="1 minute ago"` | awk -F, '$9 == ""' | head

timestamp,opendkim,zmconfigd,mailbox,proxy,spell,logger,antivirus,archiving,snmp,ldap,memcached,
  convertd,stats,antispam,mta
 [cut]
1388538000,,,,,,,,,,,,,,,
1388541600,,,,,,,,,,,,,,,
1388545200,,,,,,,,,,,,,,,
1388548800,,,,,,,,,,,,,,,
1388552400,,,,,,,,,,,,,,,
1388556000,,,,,,,,,,,,,,,
1388559600,,,,,,,,,,,,,,,
1388563200,,,,,,,,,,,,,,,
1388566800,,,,,,,,,,,,,,,
1388570400,,,,,,,,,,,,,,,
 [cut]

For this example, I'm looking to see when the "zmconfigd" service was NOT 100% available during the time period BUT was greater than 0% of the time. Zmconfigd is the 3rd column - '$3 ~ 0' and by using ~ [NOT] 0 , I can see when it was greater than 0 [100% UNAVAILABLE] but not 1 [100% available]. 

 /opt/zimbra/libexec/zmrrdfetch -f zmstatuslog -h `zmhostname` -s `date +%s \
 --date="12 month ago"` -e `date +%s --date="1 minute ago"` | awk -F, '$3 ~ 0'

timestamp,opendkim,zmconfigd,mailbox,proxy,spell,logger,antivirus,archiving,snmp,ldap,memcached,
  convertd,stats,antispam,mta
1400670000,0.926591142901235,0.882009858611111,0.962549540740741,0.962548086419753,0.926603197839506,
  1,,,0.964051042283951,1,0.962549647839506,0.962515237345679,0.292491091358025,,0.926590777777778
1400756400,1,0.968276889722222,1,1,1,1,,,1,1,1,1,1,,1
1401678000,0.9323037375,0.937399301388889,1,0.964109132777778,0.932308961111111,1,,,0.932305973888889,
  1,0.964109800277778,1,0.932309575,,0.932302677222222
1401692400,0.999979394166667,0.9999799825,1,0.9999993925,0.999979395277778,1,,,0.999979393333333,1,
  0.9999993975,1,0.999979395555556,,0.999979393333333
1401966000,1,0.964610447777778,1,1,1,1,,,1,1,1,1,1,,1
1402052400,0.999603703008394,0.999722196864111,0.999615163422937,0.999602498779275,0.999597924831049,
 0.999619443248889,,,0.99959995491499,0.999626231525528,0.99961180945684,0.999632334525742,
 0.999591523017581,,0.999607517027221
1402765200,1,0.966623691358025,1,1,0.966623606481481,1,1,,0.966623565432099,1,1,1,0.966623647530864,
 1,0.966623437345679
1402768800,0.999471892261905,0.999587652037617,0.999491736309524,0.999464918452381,0.999464358928571,
 0.999495648214286,0.999508635119048,,0.999464445238095,0.999496001785714,0.999483839285714,
 0.999503813095238,0.999461091071429,0.99951724702381,0.999478128571429
1402776000,0.9999910775,0.999972398888889,0.999993851388889,0.999991085555555,0.999972394722222,
 0.999991074166667,0.999991234166667,,0.999972394722222,0.999995651111111,0.999991075833333,
 0.999991235277778,0.999972394722222,0.999991233333333,0.999972392777778
1403175600,0.999841834656085,0.999841587301587,0.999841991402116,0.999841788690476,0.999841692791005,
 0.999842046957672,0.999842196097884,,0.999841743386243,0.999842097222222,0.999841937830688,
 0.99984214484127,0.999841638227513,0.999842260251323,0.999841883267196
1405076400,1,0.964354991388889,1,1,1,1,1,,1,1,1,1,1,1,1
1405080000,1,0.946143796944445,1,1,1,1,1,,1,1,1,1,1,1,1
1405105200,1,0.987032061666667,1,1,1,1,1,,1,1,1,1,1,1,1
1405116000,1,0.982143184166667,1,1,1,1,1,,1,1,1,1,1,1,1
1405130400,1,0.967078515555556,1,1,1,1,1,,1,1,1,1,1,1,1
1405141200,1,0.920520648333333,1,1,1,1,1,,1,1,1,1,1,1,1
1405144800,1,0.9539186725,1,1,1,1,1,,1,1,1,1,1,1,1
1405148400,1,0.981826185277778,1,1,1,1,1,,1,1,1,1,1,1,1
1405162800,1,0.996330343333333,1,1,1,1,1,,1,1,1,1,1,1,1

An example of a zmcontrol restart 

 /opt/zimbra/libexec/zmrrdfetch -f zmstatuslog -h `zmhostname` -s `date +%s \
 --date="15 minute ago"` -e `date +%s --date="1 minute ago"`

timestamp,opendkim,zmconfigd,mailbox,proxy,spell,logger,antivirus,archiving,snmp,ldap,memcached,
  convertd,stats,antispam,mta
1405621920,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
1405621950,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
1405621980,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
1405622010,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
1405622040,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
1405622070,0.105026416666667,0.1050323,1,1,0.105029991666667,1,0.105016,0.105017466666667,0.105028733333333,
  1,1,1,0.10503115,0.105014483333333,0.105025008333333
1405622100,0.105026416666667,0.1050323,1,1,0.105029991666667,1,0.105016,0.105017466666667,0.105028733333333,
  1,1,1,0.10503115,0.105014483333333,0.105025008333333
1405622130,0.105026416666667,0.1050323,1,1,0.105029991666667,1,0.105016,0.105017466666667,0.105028733333333,
  1,1,1,0.10503115,0.105014483333333,0.105025008333333
1405622160,0.105026416666667,0.1050323,1,1,0.105029991666667,1,0.105016,0.105017466666667,0.105028733333333,
  1,1,1,0.10503115,0.105014483333333,0.105025008333333
1405622190,,0,,,,,,,,,,,,,
1405622220,,0,,,,,,,,,,,,,
1405622250,,0,,,,,,,,,,,,,
1405622280,,0,,,,,,,,,,,,,
1405622310,,0,,,,,,,,,,,,,
1405622340,,0.893621444444444,,,,,,,,,,,,,
1405622370,,0.893621444444444,,,,,,,,,,,,,
1405622400,,0.893621444444444,,,,,,,,,,,,,
1405622430,,1,,,,,,,,,,,,,
1405622460,,1,,,,,,,,,,,,,
1405622490,,1,,,,,,,,,,,,,
1405622520,,1,,,,,,,,,,,,,
1405622550,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
1405622580,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
Check The audit.log File For Service Enable Changes

There is the /opt/zimbra/log/audit.log file that will note a change in a service - for example [ use: zgrep zimbraService /opt/zimbra/log/audit* ]: 

audit.log:2014-07-21 12:15:44,745 INFO 
 [qtp1097575009-2632:https://127.0.0.1:7071/service/admin/soap/ModifyServerRequest]
 [name=zimbra;ip=127.0.0.1;ua=zmprov/8.0.7_GA_6029;] security - cmd=ModifyServer; 
  name=zcs807.us.DOMAIN.com; -zimbraServiceEnabled=archiving;
audit.log:2014-07-21 12:16:51,609 INFO 
 [qtp1097575009-2640:https://127.0.0.1:7071/service/admin/soap/ModifyServerRequest]
 [name=zimbra;ip=127.0.0.1;ua=zmprov/8.0.7_GA_6029;] security - cmd=ModifyServer; 
  name=zcs807.us.DOMAIN.com; +zimbraServiceEnabled=archiving;
Check The zmsetup logs For Changes During Installation/Re-installs

In regards to the /opt/zimbra/log/zmsetup.* log files, here's an example from a test machine :

[Someone rerunning the installer/zmsetup, archiving was enabled - greping' for archiv below] 

zmsetup.07152014-150104.txt:Tue Jul 15 14:51:43 2014 enabled packages zimbra-logger zimbra-store 
  zimbra-mta zimbra-convertd zimbra-core zimbra-apache zimbra-archiving zimbra-proxy zimbra-snmp 
  zimbra-ldap zimbra-spell
zmsetup.07152014-150104.txt:Tue Jul 15 14:52:02 2014 archiving is enabled
zmsetup.07152014-150104.txt:Tue Jul 15 14:52:29 2014 checking isEnabled zimbra-archiving
zmsetup.07152014-150104.txt:Tue Jul 15 14:52:29 2014 zimbra-archiving is enabled
zmsetup.07152014-150104.txt:Tue Jul 15 14:54:43 2014 Updating cached config attribute for Server 
  zcs807.us.DOMAIN.com: zimbraServiceInstalled=archiving
zmsetup.07152014-150104.txt:Tue Jul 15 14:54:43 2014 *** Running as zimbra user: /opt/zimbra/bin/zmprov 
  -r -m -l ms zcs807.us.DOMAIN.com zimbraServiceInstalled 'antivirus' zimbraServiceInstalled 'antispam'
  zimbraServiceInstalled 'opendkim' zimbraServiceInstalled 'archiving' zimbraServiceInstalled 'logger' 
  zimbraServiceInstalled 'mailbox' zimbraServiceInstalled 'mta' zimbraServiceInstalled 'convertd' 
  zimbraServiceInstalled 'stats' zimbraServiceInstalled 'proxy' zimbraServiceInstalled 'snmp' 
  zimbraServiceInstalled 'ldap' zimbraServiceInstalled 'spell'
zmsetup.07152014-150104.txt:Tue Jul 15 14:54:46 2014 Updating cached config attribute for Server 
  zcs807.us.DOMAIN.com: zimbraServiceEnabled=archiving
zmsetup.07152014-150104.txt:Tue Jul 15 14:54:46 2014 *** Running as zimbra user: 
  /opt/zimbra/bin/zmprov -r -m -l ms zcs807.us.DOMAIN.com zimbraServiceEnabled 'antivirus' 
  zimbraServiceEnabled 'archiving' zimbraServiceEnabled 'antispam' zimbraServiceEnabled 'opendkim' 
  zimbraServiceEnabled 'logger' zimbraServiceEnabled 'mailbox' zimbraServiceEnabled 'mta' 
  zimbraServiceEnabled 'convertd' zimbraServiceEnabled 'stats' zimbraServiceEnabled 'proxy' 
  zimbraServiceEnabled 'snmp' zimbraServiceEnabled 'ldap' zimbraServiceEnabled 'spell'
zmsetup.07152014-150104.txt:Tue Jul 15 14:55:30 2014 *** Running as zimbra user: /opt/zimbra/bin/zmprov 
  -r -m -l mcf +zimbraComponentAvailable archiving
zmsetup.07152014-150104.txt: Stopping archiving...Done.
zmsetup.07152014-150104.txt: Starting archiving...Done.
zmsetup.07152014-150104.txt:Tue Jul 15 15:00:06 2014 com_zimbra_archive...
zmsetup.07152014-150104.txt:Tue Jul 15 15:00:06 2014 *** Running as zimbra user: 
  /opt/zimbra/bin/zmzimletctl -l deploy zimlets-network/com_zimbra_archive.zip
zmsetup.07152014-150104.txt:[] INFO: Deploying Zimlet com_zimbra_archive in LDAP.
zmsetup.07152014-150104.txt:[] INFO: Installing Zimlet com_zimbra_archive on this host.
zmsetup.07152014-150104.txt:[] INFO: Upgrading Zimlet com_zimbra_archive to 7.1.0
zmsetup.07152014-150104.txt:[] INFO: Enabling Zimlet com_zimbra_archive

Showing the ldap backup data in regards to the archiving service being disabled and then enabled after the 07/15 setup : 

$ zgrep "zimbraServiceEnabled:" full-20140716.070025.112/ldap/* incr-20140714.070016.382/ldap/*

  :from the day after the re-install:
full-20140716.070025.112/ldap/ldap.bak.gz:zimbraServiceEnabled: opendkim
full-20140716.070025.112/ldap/ldap.bak.gz:zimbraServiceEnabled: archiving <<
full-20140716.070025.112/ldap/ldap.bak.gz:zimbraServiceEnabled: logger
full-20140716.070025.112/ldap/ldap.bak.gz:zimbraServiceEnabled: mailbox
full-20140716.070025.112/ldap/ldap.bak.gz:zimbraServiceEnabled: mta
full-20140716.070025.112/ldap/ldap.bak.gz:zimbraServiceEnabled: convertd
full-20140716.070025.112/ldap/ldap.bak.gz:zimbraServiceEnabled: stats
full-20140716.070025.112/ldap/ldap.bak.gz:zimbraServiceEnabled: proxy
full-20140716.070025.112/ldap/ldap.bak.gz:zimbraServiceEnabled: snmp
full-20140716.070025.112/ldap/ldap.bak.gz:zimbraServiceEnabled: ldap
full-20140716.070025.112/ldap/ldap.bak.gz:zimbraServiceEnabled: spell

  :from the day before the re-install:
incr-20140714.070016.382/ldap/ldap.bak.gz:zimbraServiceEnabled: logger
incr-20140714.070016.382/ldap/ldap.bak.gz:zimbraServiceEnabled: mailbox
incr-20140714.070016.382/ldap/ldap.bak.gz:zimbraServiceEnabled: mta
incr-20140714.070016.382/ldap/ldap.bak.gz:zimbraServiceEnabled: convertd
incr-20140714.070016.382/ldap/ldap.bak.gz:zimbraServiceEnabled: stats
incr-20140714.070016.382/ldap/ldap.bak.gz:zimbraServiceEnabled: snmp
incr-20140714.070016.382/ldap/ldap.bak.gz:zimbraServiceEnabled: ldap
incr-20140714.070016.382/ldap/ldap.bak.gz:zimbraServiceEnabled: spell
incr-20140714.070016.382/ldap/ldap.bak.gz:zimbraServiceEnabled: proxy
incr-20140714.070016.382/ldap/ldap.bak.gz:zimbraServiceEnabled: opendkim
Check The Zimbra Users BASH History File and .zmprov_history

You could check your .bash_history file to see if the command is still referenced there - if it goes back long enough for your purposes. I think the default lines the history file will remember is 500. 

[zimbra@zcs807 ~]$ zmprov ms `zmhostname` -zimbraServiceEnabled archiving 
[zimbra@zcs807 ~]$ zmcontrol status | grep -i arch 
[zimbra@zcs807 ~]$ zmprov gs `zmhostname` | grep Enabled | grep archiving 
[zimbra@zcs807 ~]$ zmprov ms `zmhostname` +zimbraServiceEnabled archiving 
[zimbra@zcs807 ~]$ zmcontrol status | grep -i arch 
archiving Running 
[zimbra@zcs807 ~]$ zmprov gs `zmhostname` | grep Enabled | grep archiving 
zimbraServiceEnabled: archiving 

[zimbra@zcs807 ~]$ grep archiving .bash_history 
zmprov ms `zmhostname` +zimbraServiceInstalled archiving +zimbraServiceEnabled archiving 
zmprov ms `zmhostname` -zimbraServiceEnabled archiving 
zmprov gs `zmhostname` | grep Enabled | grep archiving 
zmprov ms `zmhostname` +zimbraServiceEnabled archiving 
zmprov gs `zmhostname` | grep Enabled | grep archiving 
grep archiving .bash_history

Note, there is also a /opt/zimbra/.zmprov_history you can also review

Getting Duplicate Emails In The Archive Account

Please see the following:

Archiving Issues When Using 3rd Party MTA's

Please see the following:

Pre-Deployment Type Questions

How To Add OLD Data Or Retroactively Add Data To A&D Account

You'll want to have A&D all setup and then you can use imapsync to 'sync' the old data from the one account into the A&D account.

Recommendations:

  • Confirm imap is enabled on the mailstores
  • Setup a sub-folder in A&D account to hold data that existed prior to the A&D account being in use.
  • Test the various imapsync flags you'll want to use first before running against all your accounts.

See also:

Can we set the archive settings for only sent mail?

Question: Can we set the archive settings for only sent mail?

Answer: No, archiving is done for any message that passes through the MTA for an account that has archiving enabled. You can setup separate MTA's for in/out bound delivery and enable archiving for only the inbound servers to achieve this goal.

Can we set the archive settings for only form port 25?

Question: Can we set the archive settings for only form port 25?

Answer: No, there are no per port level configurations. You can configure which accounts have archiving enabled and you can enable/disable the service on a per server level.

If Zimbra implemented an archive feature, is something in the following MTA files changes?

Question: If Zimbra implemented an archive feature, is something in the following MTA files changed? [ zmmta.cf , main.cf , main.cf.default , master.cf.in , master.cf ]

Answer: Nothing is specifically changed in the files you listed but settings in zmmta.cf determine how the postfix content filter is configured when archiving is enabled. Archiving functionality is handled by the amavis process on the Zimbra MTA.

Using One Archive Account For Multiple Accounts

You could also include other variable's here if they are needed, ex. - archive-cos COS_NAME 

zmarchiveconfig enable user1@domainname archive-address user-archives@domainname.archive
zmarchiveconfig enable user2@domainname archive-address user-archives@domainname.archive archive-create FALSE
zmarchiveconfig enable user3@domainname archive-address user-archives@domainname.archive archive-create FALSE

You'll see the mapping in the primary email account settings that state what archive account is being used: 

zmprov ga user1@example.com zimbraArchiveAccount
 zimbraArchiveAccount: user-archives@domainname.archive

zmprov ga user2@example.com zimbraArchiveAccount
 zimbraArchiveAccount: user-archives@domainname.archive

zmprov ga user3@example.com zimbraArchiveAccount
 zimbraArchiveAccount: user-archives@domainname.archive


External Host For Archive Accounts Setup

I installed 5.0.11 on a test box, selecting the archiving package during the installation.

The main domain was "zimbra.INTERNAL.com" and the servername is mail3.zimbra.INTERNAL.com .

'Note, I did have an external MTA relay server setup for zimbra. I was doing this test from home.

Once it was done, I then created a test account - ajcody@zimbra.INTERNAL.com

I then ran this from the CLI [on a multi-server environment, this is ran on the mta server]: 

zmprov ms mail3.zimbra.INTERNAL.com +zimbraServiceInstalled archiving  +zimbraServiceEnabled archiving

I confirm that I can send emails to the external account that I will be using with the zimbra admin account:

  • Logged into zimbra admin web console , accounts > admin > View Mail
    • Composed new message for ajcody@EXTERNAL-DOMAIN.com

I confirmed the external account received the email and did a reply back. Again, confirming now that the admin account gets the email as well from the external account.

Created the archive account to be used: 

zmarchiveconfig enable ajcody@zimbra.INTERNAL.com archive-address ajcody@EXTERNAL-DOMAIN.com archive-create false

I then stopped/started zimbra. 

zmcontrol stop
zmcontrol start

Confirmed that amavis (antispam & antivirus) and archive show up as running process: 

[zimbra@mail3 ~]$ zmcontrol status
Host mail3.zimbra.INTERNAL.com
	antispam                Running
	antivirus               Running
	archiving               Running
	ldap                    Running
	logger                  Running
	mailbox                 Running
	mta                     Running
	snmp                    Running
	spell                   Running
	stats                   Running

Checked my zimbra account to confirm the archive variables: 

[zimbra@mail3 ~]$ zmprov ga ajcody@`hostname -d` | egrep -i archive  
amavisArchiveQuarantineTo: ajcody@EXTERNAL-DOMAIN.com
zimbraArchiveAccount: ajcody@EXTERNAL-DOMAIN.com
zimbraArchiveAccountDateTemplate: yyyyMMdd
zimbraArchiveAccountNameTemplate: ${USER}-${DATE}@${DOMAIN}.archive

Tests I then performed:

  • I sent an email from the zimbra admin account to the ajcody@zimbra.INTERNAL.com account. Confirmed that the external account [ajcody@EXTERNAL-DOMAIN.com] received a copy.
  • I sent an email to ajcody@zimbra.INTERNAL.com from my ajcody@BUSINESS.com account and confirmed a copy went to ajcody@EXTERNAL-DOMAIN.com .
  • I sent an email from ajcody@zimbra.INTERNAL.com to the zimbra admin account, again confirming a copy went to ajcody@EXTERNAL-DOMAIN.com .

What didn't work as expected:

  • Messages to/from the two accounts don't create copies. If ajcody@zimbra.INTERNAL.com sends to ajcody@EXTERNAL-DOMAIN.com a "copy" will not be generated and vis-versa .
  • The subject lines weren't altered as stated in the official A&D document.
    • p3 , "When a message is received to a mailbox with archiving enabled, a copy of the message is sent to the archive mailbox with the text “to be archived” added to the subject line."

I've sent off a request about these two items to the developers to get their feedback on them.

Multi-Server & New Mailstore A&D Setup

I also created a RFE for documentation on this.

http://bugzilla.zimbra.com/show_bug.cgi?id=25135

The following is a very rough draft document I made for multi-server / new mailstore A&D setups.

  1. Zimbra Tech Center
  2. Zeta Alliance
  3. Ajcody-Notes-Archive-Discovery

Ajcody Multi-Server & New Mailstore A&D Setup Homepage

   KB 2545        Last updated on 2015-02-13  




0.00
(0 votes)
24px ‎  - This is Zeta Alliance Certified Documentation. The content has been tested by the Community.


Actual Multi-Server & New Mailstore A&D Setup Homepage

Please see Ajcody-Notes-Archive-Discovery-Mailstore-Setup

Issues That Have Caused Confusion

What Gets Installed Where?
RFE To Clear Up The Confusion
zimbra-archive package/rpm - Mailstores

zimbra-archive (the package/rpm you see from the installer) should be installed on all mailstores which you want to use for cross mailbox search. This also sets the zimbraComponentAvailable archiving config attribute which allows the mta(s) to turn on archiving. zimbra-archive is not installed directly on the mta, it's just enabled.

Note, you install zimbra-archive on a mailbox server but the service runs on the mta node.

MTA's - Require Configuration

If you add zimbra-archiving to an existing install you need to :

  • Install zimbra-archiving on one or more of your mailbox servers
  • Then set zimbraServiceInstalled archiving and zimbraServiceEnabled archiving on all the mta servers
  • Restart the mta services

For example: 

zmprov ms mta.example.com +zimbraServiceInstalled archiving +zimbraServiceEnabled archiving

On the mta server: 

zmmtactl restart

To confirm the /opt/zimbra/conf/amavisd.conf was modified correctly, you should see on the mta: 

#$archive_quarantine_method = 'smtp:[127.0.0.1]:10025';

Was uncommented out: 

$archive_quarantine_method = 'smtp:[127.0.0.1]:10025';

You'll be able to then notice in the /var/log/zimbra.log file if the redirect to the A&D account is happening [once A&D accounts are setup that is]. Example uses example.com.archive as the archive domain I setup for the A&D accounts : 

grep "example.com.archive" /var/log/zimbra.log
 Dec 11 13:38:52 mta-server amavis[1978]: (01978-19) SEND via SMTP: <> -> 
    <user-20081211@example.com.archive>,ENVID=AM.8ISxcrQG8uAj.20081211T193852Z@mailstore.example.com 
    BODY=7BIT 250 2.6.0 Ok, id=01978-19, from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 
    5ADF8F120C4
 Dec 11 13:38:52 mta-server postfix/lmtp[21864]: 5ADF8F120C4: 
    to=<user-20081211@example.com.archive>, relay=archive.example.com[X.X.X.93]:7025, 
    delay=0.07, delays=0/0/0/0.06, dsn=2.1.5, status=sent (250 2.1.5 OK)
Enabling Amavis And Archiving With 8.5+ While Antispam And AntiVirus Are Disabled

With 808 and 8.5 , archiving should be able to run without as/av being enabled.

For example, disabling antispam and antivirus but enabling amavis [required] and archiving on a mta server [note - this server had the full mta package already installed on it and had antivirus, antispam, [amavis], and postfix running on it] : 

[zimbra@850-mta1 ~]$ zmcontrol status | egrep 'amavis|antispam|antivirus|archiving'
	amavis                  Running
	antispam                Running
	antivirus               Running

## NOTE , the below command will disable antispam & anitvirus for this "test". ##

[zimbra@850-mta1 ~]$ zmprov ms `zmhostname` +zimbraServiceEnabled archiving \
 +zimbraServiceEnabled amavis -zimbraServiceEnabled antispam -zimbraServiceEnabled antivirus

[zimbra@850-mta1 ~]$ zmcontrol restart                
Host 850-ldap1.zimbra.homeunix.com
	Stopping vmware-ha...Done.
[cut]
	Stopping ldap...Done.
Host 850-ldap1.zimbra.homeunix.com
	Starting ldap...Done.
	Starting zmconfigd...Done.
	Starting dnscache...Done.
	Starting logger...Done.
	Starting convertd...Done.
	Starting mailbox...Done.
	Starting memcached...Done.
	Starting proxy...Done.
	Starting amavis...Done.
	Starting opendkim...Done.
	Starting archiving...Done.
	Starting snmp...Done.
	Starting spell...Done.
	Starting mta...Done.
	Starting stats...Done.
	Starting service webapp...Done.
	Starting zimbra webapp...Done.
	Starting zimbraAdmin webapp...Done.
	Starting zimlet webapp...Done.
[zimbra@850-ldap1 ~]$ zmcontrol status | egrep 'amavis|antispam|antivirus|archiving'
	amavis                  Running
	archiving               Running
zimbra_xmbxsearch zimlet

For 5.x installs, this zimbra_xmbsearch zimlet will get configured on each mailstore that you install the zimbra-archive package on. The documentation in various places might cause confusion on this matter, because for the 4.x releases it was a separate step.

You should find the zimlet already located at /opt/zimbra/zimlets-network/zimbra_xmbxsearch.zip

After the installation, you should see when you go to the admin web console on the mailstore you install the zimbra-archive package on that the cross-mailbox search zimlet is there. It shows up in two locations:

  • Left Pane : Configuration > Admin Extensions > zimbra_xmbxsearch
  • Left Pane : Tools > Search Mail

If you wanted this zimlet to also be available on a server that didn't have the zimbra-archiving packaged installed you could then deploy it on that server. 

cd /opt/zimbra/zimlets-network/
zmzimletctl deploy zimbra_xmbxsearch.zip
## ls the directory and confirm the full name - you might need this:
zmzimletctl deploy com_zimbra_xmbxsearch.zip

The How-To

Reference Documents

http://www.zimbra.com/docs/ne/latest/multi_server_install/multi-server_install.5.1.html

http://www.zimbra.com/docs/ne/latest/administration_guide/Archiving.16.1.html

Assumptions

This install how-to assumes you have an existing LDAP/Mailstore/MTA server(s) for your normal production environment, the Zimbra license and logger are installed on the primary ZCS server(s), and that you are NOT running the proxy module.

Example archive mailstore hostname is : archive.example.com

Example primary ZCS hostname is : mail.example.com

Preliminary Items

Things to do or check before install:

  • DNS entry for new mailstore and primary ZCS server(s) can resolve to it.
  • DNS configured properly on mailstore server.
  • Master Root LDAP Server mail.example.com
  • Master Root LDAP Password
    • On LDAP server do : su – zimbra ; zmlocalconfig –s | grep ldap_root_password
  • Master LDAP port – default is 389
  • SMTP Server
Installation Of New Mailstore That Will Have A&D
Install Modules
  • Type y to install the zimbra-store, zimbra-archiving and zimbra-spell (optional) packages.
    • Do Not Install MTA! These Instructions Do Not Take That Into Account.
    • When zimbra-spell is installed the zimbra-apache package is also installed.
  • Installing: zimbra-core zimbra-store zimbra-apache zimbra-spell
Modify Configuration

Press Enter to modify the system. The selected packages are installed on the server.

At this point the Main menu displays the default entries for the Zimbra component you are installing.

To expand the menu to see the configuration values type x and press Enter.

To navigate the Main menu, select the menu item to change. You can modify any of the defaults.

  • Common Configuration
    • LDAP
      • Ldap master host: [set this to the FQDN of your LDAP server]
      • Ldap port: 389 [set this if your LDAP server isn’t using default]
      • Ldap Admin password: [this is your LDAP servers Root LDAP password]
        • On LDAP server do : su – zimbra ; zmlocalconfig –s ldap_root_password
      • TimeZone: [set this]
  • For zimbra-store
    • Set the Admin Password
      • +License filename: UNSET [if you see this, then something is wrong with your
    • LDAP configuration. It should of pulled the license info from the LDAP server.
    • Set the SMTP host

Type r to return to the Main menu, if you aren’t there already.

When the mailbox server is configured, return to the Main menu and type a to apply the configuration changes.

Press Enter to save the configuration data.

When Save Configuration data to a file appears, press Enter.

The next request is where to save the files. To accept the default, press Enter.

To save the files to another directory, enter the directory and then press Enter.

When “The system will be modified - continue?” appears type y and press Enter.

The server is modified.

Installing all the components and configuring the server can take a few minutes.

When Installation complete - press return to exit displays, press Enter.

The installation of the mailbox server is complete.

After Install

Confirm server status 

su – zimbra ; zmcontrol status

Populate the ssh keys, on each server in your environment 

su - zimbra ;  zmupdateauthkeys

The key is updated on /opt/zimbra/.ssh/authorized_keys.

Upgrading A Zimbra Server For An Archive & Discovery Mailstore
Adding Package For A&D

This will retain your current settings for the system. Your server will experience downtime during the upgrade.

untar zcs*.tar that matches your existing system 

 cd zcs-version-directory
 ./install
  choose upgrade
  select zimbra-archiving

The upgrade of the mailbox server is complete.

After Upgrade

Confirm server status 

su – zimbra ; zmcontrol status

Note, zimbra-archiving only runs as a service on a MTA server.

Populate the ssh keys, on each server in your environment 

su-zimbra ;  zmupdateauthkeys

The key is updated on /opt/zimbra/.ssh/authorized_keys.

Configure Zimbra For Use Of The New Mailstore and A&D

Example A&D mailstore hostname is : archive.example.com

  • Go to your primary admin console url. [https://[example.com]:7071/zimbraAdmin]
  1. Confirm you see the new mailstore under Configuration > Servers
    1. Under Configuration > Servers > [MTA servername(s)] > Services
      1. [each MTA server needs this]
      2. You’ll see a box for Archiving and Discovery
        1. Check this to enable the MTA server(s) for Archiving and Discovery. If this is grayed out, run the command below (modified for your server) on your one of your mailstores.
          • This effectively does:
          • zmprov ms mta.example.com +zimbraServiceInstalled archiving  +zimbraServiceEnabled archiving
            • Remember, zmprov uses the variable below. A mta only server can't be set for localhost, change it to point to a mailstore.
            •  [root@mta ~]# zmlocalconfig | grep zmprov
            •     zimbra_zmprov_default_soap_server = localhost
            •  [root@mta ~]# zmlocalconfig -e zimbra_zmprov_default_soap_server=mailstore.example.com
Configuring Your COS's - Normal COS's and Archiving COS

It's recommended that the archive accounts be created on a dedicated mailstore. You can limit what mailstores are used for new account creation by restricting what mailstores are used under "Server Pool" within a COS. Your normal COS's should excluded your archive mailstores and your archive COS should only have archive mailstores selected.

You'll also want to make sure your archive COS isn't using any 'features' that aren't necessary for archive accounts. No point in consuming certain license features when they aren't needed.

  1. Go to Configuration > Class of Service > default [or your primary domain] > Server Pool
    1. You’ll want to make sure it’s limited to the correct server pools
      1. Your new mailstore for A&D should be unchecked.
    2. Click on New for a new Class of Server (COS)
      1. Call it archive or something similar
        1. Under Server Pool > Limit > have only the new mailstore checked
    3. Make sure your not using licensed features [EWS for example] that aren't necessary for your archive accounts.
Setup Initial A&D With First Account - Creation Of The Archive Domain
Revisit To COS - Naming Scheme Of Archive Accounts

When archive accounts are created they use the zimbraArchiveAccountNameTemplate variable from the COS. The default is: 

$ zmprov gc default | grep -i archive
  zimbraArchiveAccountDateTemplate: yyyyMMdd
  zimbraArchiveAccountNameTemplate: ${USER}-${DATE}@${DOMAIN}.archive

I, personally, don't like the use of the $DATE variable in this. I change my ARCHIVE COS to use the normal username but retain the .archive for the domain. 

zmprov mc archive zimbraArchiveAccountNameTemplate '${USER}@${DOMAIN}.archive'

Bug to be aware of:

The Creation

On server with zmarchiveconfig (most likely mailstore you installed A&D on) and as zimbra (su – zimbra) do the following to setup your first A&D account.

format : zmarchiveconfig –s servername enable user@example.com archive-cos <cos>

example : 

zmarchiveconfig –s archive.example.com enable account@example.com archive-cos archive

NOTE

If the above command doesn't seem to create the archive account/domain. Drop the use of [ -s servername ]. Basically, just run this on the A&D mailstore:
zmarchiveconfig enable account@example.com archive-cos archive

The above command will create the mail domain for the archive accounts using the template defaults, user@example.com to make example.com.archive

On your main ldap server or where ever you usually access the zimbra admin web console, login to the admin web console.

  1. Confirm the archive domain was setup.
    1. Configuration > Domains > [domainname].archive > General
    2. Confirm or adjust the archive domain to use the right COS
      1. Configuration > Domains > [domainname].archive > General Information
        1. Change “Default Class of Service” to your COS [archive], if needed for your configuration.
  2. Now check for the new archive account you made
    1. Address > Accounts
    2. Click on account and hit the edit button
    3. In the top summary section you'll be able to confirm the COS and Mail Server being used for the account.
      1. NOTE, if it's showing the account is on the primary mailstore and NOT the A&D mailstore.
        1. Remove the A&D account 
          •  zmprov ra [user]@[domainname].archive
        2. Add the account back again using the zmarchiveconfig command 
          • zmarchiveconfig enable account@example.com archive-cos archive
        3. Now confirm, as above, that the account is using the A&D mailstore.
          • This might be a bug related to the archive domain being created for the first time.

Send the primary account a test email and then shortly afterwards do a "View Mail" within the admin console for the archive account. You should see the archive message in the archive account.

Error: unknown document: EnableArchiveRequest

If you get this error when trying to create the archive account "Error: unknown document: EnableArchiveRequest" you most likely needed to install a new license for A&D and have not restart the mailboxd services . Updating the license is not enough, you'll need to restart ZCS on the mailstores also.

See the following bug:

RFE's On Archive Accounts
Testing Of Archive Mail Flow

Send the primary account a test email and then shortly afterwards do a "View Mail" within the admin console for the archive account. You should see the archive message in the archive account.

You should confirm mail flow copies occur with the following:

  1. Inbound
    1. External Account (email) to the primary zimbra account setup for archive.
    2. A zimbra account that ISN'T the archive account in question to the primary account setup for archive.
  2. Outbound
    1. With primary account setup for archive, send an email to an external email address.
    2. With primary account setup for archive, send an email to another internal zimbra email address.
Archive Account Isn't Getting Email

Let's double check everything was done correctly up above.

Assumption on syntax of account creation: 

zmarchiveconfig enable user@example.com archive-cos archive

Let's check what actually was done: 

zmprov ga user@example.com | grep -i archive
 amavisArchiveQuarantineTo: user-20081211@example.com.archive
 zimbraArchiveAccount: user-20081211@example.com.archive
 zimbraArchiveAccountNameTemplate: ${USER}-${DATE}@${DOMAIN}.archive

It should reference an account that's like, if you are using the archive templates: 

user-[date]@example.com.archive

that account should exist and reference lmtp, rather than smtp, for the transport: 

zmprov ga user-20081211@example.com.archive | grep -i trans
  zimbraMailTransport: lmtp:archive.example.com:7025
Checking Logs For Archive Operations

On the mta-server, you should find a reference to the archive account in /var/log/zimbra.log 

grep archive /var/log/zimbra.log
 Dec 11 13:38:52 mta-server amavis[1978]: (01978-19) SEND via SMTP: <> -> 
    <user-20081211@example.com.archive>,ENVID=AM.8ISxcrQG8uAj.20081211T193852Z@mailstore.example.com 
    BODY=7BIT 250 2.6.0 Ok, id=01978-19, from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 
    5ADF8F120C4
 Dec 11 13:38:52 mta-server postfix/lmtp[21864]: 5ADF8F120C4: 
    to=<user-20081211@example.com.archive>, relay=archive.example.com[X.X.X.93]:7025, 
    delay=0.07, delays=0/0/0/0.06, dsn=2.1.5, status=sent (250 2.1.5 OK)

On the archive-server, you should find reference to the delivery in /opt/zimbra/log/mailbox.log 

grep archive /opt/zimbra/log/mailbox.log
 2008-12-11 14:45:32,923 INFO  [LmtpServer-9] 
  [name=user-20081211@example.com.archive;mid=7;] mailop - Adding Message: id=257,
  Message-ID=<1350363939.41021229024728317.JavaMail.root@EXTERNAL-MTA.DOMAIN.com>, parentId=-1,
  folderId=2, folderName=Inbox.
Mass Accounts Configuration
Update 5/28/15, I filed the following RFE:
  • "Redesign archive account creation process [autocreate, mass creation/enabling, etc]"

https://bugzilla.zimbra.com/show_bug.cgi?id=99710

Update, our 6.0 release will have a zmarchiveconfig -f command for batch processing from a file input.

CHECK YOUR AVAILABLE LICENSES BEFORE YOU PROCEED!!

One could put all the accounts in a txt file and then use a for-loop to process the account@example.com variable. 

zmprov -l gaa > /tmp/accounts.txt

Remove any accounts you've already done and those not necessary for archiving (ex. admin, ham, spam, etc.)

You can give gaa other options, look at zmprov help account. For example, you could also narrow this down to a dump of accounts in a domain: 

zmprov -l gaa [DOMAIN] > /tmp/accounts.txt

Note, the below uses the above setup for A&D - You'll need to modify for your environment. 

for i in `cat /tmp/accounts.txt`
do
zmarchiveconfig –s archive.example.com enable $i archive-cos archive
sleep 3
done

You can be tailing /opt/zimbra/log/mailbox.log on the archive server to watch the progress.

Searches After Configuration Is Done

Please see Ajcody-Server-Misc-Topics#Cross_Mailbox_Searches_and_Tracing

Searches Limited To 500 or 1000 Maximum Results

See Ajcody-Server-Misc-Topics#Searches_Limited_To_500_or_1000_Maximum_Results

Special Circumstance Case Scenarios

Need Primary Account Going To Two "Archive" Targets

Currently it isn't possible, I have submitted an RFE for this though:

I also submitted an RFE to clarify the difference/intention between amavisArchiveQuarantineTo and zimbraArchiveAccount

Using Legal Intercept For Secondary Target

Note - since we are setting the legal intercept against the ARCHIVE account and not the production account, we avoid what might be an issue with legal intercept's limitations. See :

Other references: 

Primary Archive Account Setup
[zimbra@zcs807 ~]$ zmarchiveconfig create archive-test@zcs807.us.DOMAIN.com archive-address archive-test-manual@zcs807.us.DOMAIN.com.archive

Setup For The Legal Intercept One We'll Use - You could also use the 'normal' account creation process, but those defaults also will show up in the GAL.
[zimbra@zcs807 ~]$ zmarchiveconfig create archive-test2@zcs807.us.DOMAIN.com archive-address archive-test-manual@zcs807.us.DOMAIN.com.archive

[zimbra@zcs807 ~]$ zmprov ga archive-test@zcs807.us.DOMAIN.com | egrep -i 'archive|amavisArchiveQuarantine'      
 [cut - notes] 
confirm that amavisArchiveQuarantineTo & zimbraArchiveAccount are using only ONE of the 'archive' accounts, the primary archive account. Not the one we'll setup later to be the legal intercept target.

[zimbra@zcs807 ~]$ zmprov ma archive-test@zcs807.us.DOMAIN.com amavisArchiveQuarantineTo archive-test-manual@zcs807.us.DOMAIN.com.archive
[zimbra@zcs807 ~]$ zmprov ma archive-test@zcs807.us.DOMAIN.com zimbraArchiveAccount archive-test-manual@zcs807.us.DOMAIN.com.archive
 [note] by using just the variable name and not including a + or - in front of it, we'll set it to only be one variable if there were multiple ones set.

[zimbra@zcs807 ~]$ zmprov ga archive-test@zcs807.us.DOMAIN.com | egrep -i 'archive|amavisArchiveQuarantine'                                                       # name archive-test@zcs807.us.DOMAIN.com
amavisArchiveQuarantineTo: archive-test-manual@zcs807.us.DOMAIN.com.archive
cn: archive-test
mail: archive-test@zcs807.us.DOMAIN.com
sn: archive-test
uid: archive-test
zimbraArchiveAccount: archive-test-manual@zcs807.us.DOMAIN.com.archive
zimbraArchiveAccount: archive-test-manual2@zcs807.us.DOMAIN.com.archive
zimbraArchiveAccountDateTemplate: yyyyMMdd
zimbraArchiveAccountNameTemplate: ${USER}-${DATE}@${DOMAIN}.archive
zimbraArchiveEnabled: TRUE
zimbraMailDeliveryAddress: archive-test@zcs807.us.DOMAIN.com

Set Up your Legal Intercept Variable Now To The Production Account
[zimbra@zcs807 ~]$ zmprov ma archive-test@zcs807.us.DOMAIN.com zimbraInterceptAddress archive-test-manual2@zcs807.us.DOMAIN.com.archive

For Testing - I'm purging the Inbox on all three test accounts.
[zimbra@zcs807 ~]$ zmmailbox -z -m archive-test-manual2@zcs807.us.DOMAIN.com.archive ef /Inbox
[zimbra@zcs807 ~]$ zmmailbox -z -m archive-test-manual@zcs807.us.DOMAIN.com.archive ef /Inbox
[zimbra@zcs807 ~]$ zmmailbox -z -m archive-test@zcs807.us.DOMAIN.com ef /Inbox

Create An Email That We Can Send From The CLI
[zimbra@zcs807 ~]$ cat /tmp/email-archive.txt
To: archive-test@zcs807.us.DOMAIN.com
From: admin@zcs807.us.DOMAIN.com
Subject: Test message
test

[zimbra@zcs807 ~]$ /opt/zimbra/postfix/sbin/sendmail -t < /tmp/email-archive.txt  ; tail -f /var/log/zimbra.log | grep archive-test                                  Jul 18 11:10:27 zcs807 amavis[31986]: (31986-02) ESMTP:[127.0.0.1]:10024 /opt/zimbra/data/amavisd/tmp/amavis-20140718T105435-31986-BmFt0jbd: <zimbra@zcs807.us.DOMAIN.com> -> <archive-test@zcs807.us.DOMAIN.com> SIZE=341 Received: from zcs807.us.DOMAIN.com ([127.0.0.1]) by localhost (zcs807.us.DOMAIN.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP for <archive-test@zcs807.us.DOMAIN.com>; Fri, 18 Jul 2014 11:10:27 -0600 (MDT)
Jul 18 11:10:27 zcs807 amavis[31986]: (31986-02) Checking: qeJtNl1F-X9w <zimbra@zcs807.us.DOMAIN.com> -> <archive-test@zcs807.us.DOMAIN.com>
Jul 18 11:10:27 zcs807 amavis[31986]: (31986-02) SEND from <> -> <archive-test-manual@zcs807.us.DOMAIN.com.archive>,ENVID=AM.qeJtNl1F-X9w.20140718T171027Z@zcs807.us.DOMAIN.com BODY=7BIT 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as E7236E05EA
Jul 18 11:10:27 zcs807 amavis[31986]: (31986-02) FWD from <zimbra@zcs807.us.DOMAIN.com> -> <archive-test@zcs807.us.DOMAIN.com>,BODY=7BIT 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as EDAEBE0B7C
Jul 18 11:10:27 zcs807 amavis[31986]: (31986-02) Passed CLEAN {RelayedInbound,Archived}, <zimbra@zcs807.us.DOMAIN.com> -> <archive-test@zcs807.us.DOMAIN.com>, quarantine: archive-test-manual@zcs807.us.DOMAIN.com.archive, Message-ID: <20140718171027.C8881E0B7B@zcs807.us.DOMAIN.com>, mail_id: qeJtNl1F-X9w, Hits: -, size: 341, queued_as: EDAEBE0B7C, 122 ms
Jul 18 11:10:27 zcs807 postfix/smtp[12085]: C8881E0B7B: to=<archive-test@zcs807.us.DOMAIN.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=0.22, delays=0.09/0/0/0.12, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as EDAEBE0B7C)
Jul 18 11:10:28 zcs807 postfix/lmtp[12088]: E7236E05EA: to=<archive-test-manual@zcs807.us.DOMAIN.com.archive>, relay=zcs807.us.DOMAIN.com[10.137.27.38]:7025, delay=0.15, delays=0.02/0.01/0/0.12, dsn=2.1.5, status=sent (250 2.1.5 Delivery OK)
Jul 18 11:10:28 zcs807 postfix/lmtp[12090]: EDAEBE0B7C: to=<archive-test@zcs807.us.DOMAIN.com>, relay=zcs807.us.DOMAIN.com[10.137.27.38]:7025, delay=0.21, delays=0.01/0.01/0/0.19, dsn=2.1.5, status=sent (250 2.1.5 Delivery OK)
Jul 18 11:10:28 zcs807 postfix/smtp[12093]: 1C4ADE05EA: to=<archive-test-manual2@zcs807.us.DOMAIN.com.archive>, relay=127.0.0.1[127.0.0.1]:10030, delay=0.13, delays=0.05/0.01/0.01/0.06, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 2B480E0B9F)
Jul 18 11:10:28 zcs807 postfix/smtp[12096]: 2B480E0B9F: to=<archive-test-manual2@zcs807.us.DOMAIN.com.archive>, relay=127.0.0.1[127.0.0.1]:10025, delay=0.09, delays=0.06/0.01/0/0.02, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 3B6C6E05EA)
Jul 18 11:10:28 zcs807 postfix/lmtp[12088]: 3B6C6E05EA: to=<archive-test-manual2@zcs807.us.DOMAIN.com.archive>, relay=zcs807.us.DOMAIN.com[10.137.27.38]:7025, delay=0.15, delays=0.02/0/0/0.13, dsn=2.1.5, status=sent (250 2.1.5 Delivery OK)

Let's Confirm That All Three Accounts Show A New Message
[zimbra@zcs807 ~]$ zmmailbox -z -m archive-test-manual@zcs807.us.DOMAIN.com.archive gaf | grep Inbox                                                                         2  mess           1           1  /Inbox
[zimbra@zcs807 ~]$ zmmailbox -z -m archive-test-manual2@zcs807.us.DOMAIN.com.archive gaf | grep Inbox
  2  mess           1           1  /Inbox
[zimbra@zcs807 ~]$ zmmailbox -z -m archive-test@zcs807.us.DOMAIN.com gaf | grep Inbox
  2  mess           1           1  /Inbox

[zimbra@zcs807 ~]$ zmmboxsearch -m archive-test@zcs807.us.DOMAIN.com,archive-test-manual@zcs807.us.DOMAIN.com.archive,archive-test-manual2@zcs807.us.DOMAIN.com.archive -q "Test Message"
1)
ID: 12d3b4cf-61b8-4ba7-805f-be1d9ab6c0c2:283
Subject: Test message
Address:
 [f]  <admin@zcs807.us.DOMAIN.com>
Date: 1405703428000
Size: 1041
Conv: -283
Flags: u
Folder: 2
Fragment: test

2)
ID: 0b283721-857e-4bdc-967a-c5f9e7ec7db2:257
Subject: Test message
Address:
 [f]  <admin@zcs807.us.DOMAIN.com>
Date: 1405703428000
Size: 1269
Conv: -257
Flags: u
Folder: 2
Fragment: test

3)
ID: 3cb67670-1ad9-48c0-8f9b-372adaa2fe3f:258
Subject: Intercepted message for archive-test@zcs807.us.DOMAIN.com: Test message
Address:
 [f] Postmaster <postmaster@zcs807.us.DOMAIN.com>
Date: 1405703428000
Size: 2769
Conv: -258
Flags: au
Folder: 2
Fragment: Intercepted message for archive-test@zcs807.us.DOMAIN.com. Operation=add message, folder=Inbox, folder ID=2.
Using Another Archive Account Against the Primary Archive Account

To Do

Domain Admin Searches For ZCS6

For setting up domain admin rights, please see the following : User:Vladimir-DomainAdminSetup-v6

And you'll see about half way down a screen shot of the "Rights" panel view -- the screen shot shows the ACL right called :

adminConsoleCrossMailboxSearchRights --- Admin Console Cross Mailbox Search Rights

You'll also need to make sure domain admins have the following ACL right :

AdminLoginAs

service.UNKNOWN_DOCUMENT Error When Trying Search

The service.UNKNOWN_DOCUMENT error generally means that there is no handler for specified document. Try redeploying the zimlet, restarting the mailboxd service and let us know how it goes. The xmbxsearch zimlet is located in /opt/zimbra/zimlets-network directory. 

 su - zimbra
 cd /opt/zimbra/zimlets-network
 zmzimletctl undeploy zimbra_xmbxsearch.zip
 zmzimletctl deploy zimbra_xmbxsearch.zip
 zmmailboxdctl restart

How To Turn Off / Disable ALL A&D Processing From Admin Web Console

From a "server" level, you would uncheck the "Archiving and Discovery" box in the admin web console under: 

Configurations > Servers > [MTA's] > right pane > Services tab

This would stop the mta's from splitting the emails for archive.

How To Turn Off / Disable A&D For An Account

Disabling A&D at a user level, requires it to be done via the CLI: 

zmarchiveconfig disable user@domainname

What Accounts Are Set For Archive And Checking Against License Usuage

This example is done on a ZCS 8.6 server. The license usage is based upon the 'parent' account having amavisArchiveQuarantineTo set to something.  : 

$ zmprov -l gaa
admin@ldap2.zimbra.DOMAIN.com
spam.gktt7hym36@ldap2.zimbra.DOMAIN.com
ham.0zrc899s@ldap2.zimbra.DOMAIN.com
virus-quarantine.o944yi8wwl@ldap2.zimbra.DOMAIN.com
galsync.cs_37n5wp@ldap2.zimbra.DOMAIN.com
user1@ldap2.zimbra.DOMAIN.com
user1-archive1@ldap2.zimbra.DOMAIN.com.archive
user1-archive2@ldap2.zimbra.DOMAIN.com.archive
galsync@test2.zimbra.DOMAIN.com
testuser2@test2.zimbra.DOMAIN.com

$ zmprov -l sa "amavisArchiveQuarantineTo=*"
user1@ldap2.zimbra.DOMAIN.com

$ zmprov ga user1@`zmhostname` | egrep -i 'zimbraArchiveEnabled|amavisArchiveQuarantineTo\
|zimbraArchiveAccount:|zimbraIsSystemResource'

amavisArchiveQuarantineTo: user1-archive1@ldap2.zimbra.homeunix.com.archive, user1-archive2@ldap2.zimbra.homeunix.com.archive
zimbraArchiveAccount: user1-archive1@ldap2.zimbra.homeunix.com.archive
zimbraArchiveAccount: user1-archive2@ldap2.zimbra.homeunix.com.archive
zimbraArchiveEnabled: TRUE
zimbraIsSystemResource: FALSE


[zimbra@ldap2 ~]$ zmprov fc -a all ; zmsoap -z GetLicenseRequest | egrep 'TotalAccounts|ArchivingAccounts'
    <attr name="ArchivingAccountsLimit">10000</attr>
    <attr name="TotalAccounts">3</attr>
    <attr name="ArchivingAccounts">1</attr>

Need To Find A&D Accounts On Wrong MailStore To Move Them

Searching For Accounts On A Particular Mailstore

To do ldapsearch for the account, and see if there are any related entries in ldap. 

su - zimbra
source ~/bin/zmshutil
zmsetvars
ldapsearch -x -H $ldap_master_url -D $zimbra_ldap_userdn -w $zimbra_ldap_password "zimbraMailHost=YOURMAILSTORE.COM" | grep "archive.domain.com"

Renaming Production Domain And Want To Also Rename Archive Domain

This was tested against ZCS 608p1.

There is no 'automatic' process to rename the associated targeted archive email addresses of the user or the archive domain when changing/renaming the domain.

  1. Get all the account details from the production users in regards to the archive acct they are using
    • zimbraMailDeliveryAddress & amavisArchiveQuarantineTo
    • Example via CLI. My initial production domain was primary.DOMAIN.com and the archive domain was primary.DOMAIN.com.archive :
      • $ zmprov ga arch-prod@primary.DOMAIN.com zimbraArchiveAccount
        • [Returns] zimbraArchiveAccount: arch-prod@primary.DOMAIN.com.archive
      • $ zmprov ga arch-prod@primary.DOMAIN.com amavisArchiveQuarantineTo
        • [Returns] amavisArchiveQuarantineTo : arch-prod@primary.DOMAIN.archive
  2. Prepare a script that will redo those variables, you'll run it after the domain renames.
    • Example via CLI. My initial archive domain was primary.zimbra.homeunix.com.archive :
      • $ zmprov ma arch-prod@primary71.DOMAIN.com zimbraArchiveAccount arch-prod@primary71.DOMAIN.com.archive
      • $ zmprov ma arch-prod@primary71.DOMAIN.com amavisArchiveQuarantineTo arch-prod@primary71.DOMAIN.archive
    • Since this is ran after the domain renames, both references in the command about a domain will use the 'new' domainnames.
    • Ref: Bulk_Provisioning
  3. Put domains into maintenance mode during a schedule downtime window
  4. Rename/forward the production domain
  5. Rename/forward the archive domain
  6. Run the script you had prepared earlier that will reset the user variable to their new archive domain.
  7. Put domains into active mode:
    • Example [notice I have the 'new' domainname here] :
      • $ zmprov md primary71.DOMAIN.com zimbraDomainStatus active
      • $ zmprov md primary71.DOMAIN.com.archive zimbraDomainStatus active
  8. Send a test message to a prod address and you can see/confirm the transaction in the /var/log/zimbra.log on the mta .


Confirmation of things working [examples]: 

### From /var/log/zimbra.log on the MTA ###
Nov 17 14:42:27 mail71 amavis[7210]: (07210-01) SEND via SMTP: <> -> <arch-prod@primary71.DOMAIN.com.archive>,
  ENVID=AM.BYPyb4z8Cl0N.20101117T204227Z@mail71.DOMAIN.com BODY=7BIT 250 2.0.0 Ok, id=07210-01, 
  from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as C54F623E0E
Nov 17 14:42:27 mail71 postfix/smtpd[7446]: DDA5323E10: client=localhost[127.0.0.1]
Nov 17 14:42:27 mail71 postfix/cleanup[7262]: DDA5323E10: 
 message-id=<201011172042.oAHKgKG1024723@mail3.zimbra.homeunix.com>
Nov 17 14:42:27 mail71 postfix/qmgr[7530]: DDA5323E10: from=<root@mail3.DOMAIN.com>, 
 size=1644, nrcpt=1 (queue active)
Nov 17 14:42:27 mail71 amavis[7210]: (07210-01) FWD via SMTP: <root@mail3.DOMAIN.com> -> 
 <arch-prod@primary71.DOMAIN.com>,BODY=7BIT 250 2.0.0 Ok, id=07210-01, 
 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as DDA5323E10

You can also use the CLI to confirm delivery into the archives & production accounts Inbox. For example: 

$ zmmailbox -z -m arch-prod@primary71.DOMAIN.com gaf | grep Inbox
         2  mess           3           3  /Inbox

$ zmmailbox -z -m arch-prod@primary71.DOMAIN.com.archive gaf | grep Inbox
         2  mess           3           3  /Inbox

On Why You Have To Change The User Variables

I confirmed in my test that you do have to reset the user variables rather than just relying on the archive domain having a global forwarding. This is what shows up in /var/log/zimbra.log if you just use the domain rename/forwarding and don't reconfigure the user variable. 

 
Nov 17 14:27:05 mail71 postfix/error[26043]: 8E5DD23E0E: to=<arch-prod@primary.DOMAIN.com.archive>, 
 relay=none, delay=0.04, delays=0/0/0/0.03, dsn=5.0.0, status=bounced (User unknown in virtual alias table)

sending directly to archive account, works: 

Nov 17 14:37:28 mail71 amavis[7209]: (07209-01) FWD via SMTP: <root@mail3.DOMAIN.com> -> 
 <arch-prod@primary71.DOMAIN.com.archive>,BODY=7BIT 250 2.0.0 Ok, id=07209-01, 
from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 1FCA523E0E
Retrieved from "http://wiki.zimbra.com/index.php?title=Ajcody-Notes-Archive-Discovery&oldid=57004"
Jump to: navigation, search