Glenno-Notes

- This article is NOT official Zimbra documentation. It is a user contribution and may include unsupported customizations, references, suggestions, or information.

Output in mailbox.log when mailboxd stops and starts

Shutdown:

2010-08-31 16:00:06,751 INFO  [Shutdown] [] log - Shutdown hook executing
2010-08-31 16:00:07,274 INFO  [Shutdown] [] StatsImageServlet - Servlet StatsImageServlet shutting down
2010-08-31 16:00:07,275 INFO  [Shutdown] [] FileUploadServlet - Servlet FileUploadServlet shutting down
2010-08-31 16:00:07,277 INFO  [Shutdown] [] PublicICalServlet - Servlet PublicICalServlet shutting down
2010-08-31 16:00:07,277 INFO  [Shutdown] [] account - Servlet PreAuthServlet shutting down
2010-08-31 16:00:07,277 INFO  [Shutdown] [] mailbox - Servlet UserServlet shutting down
2010-08-31 16:00:07,277 INFO  [Shutdown] [] ContentServlet - Servlet ContentServlet shutting down
2010-08-31 16:00:07,277 INFO  [Shutdown] [] soap - Servlet AdminServlet shutting down
2010-08-31 16:00:07,278 INFO  [Shutdown] [] TcpServer/7025 - LmtpServer initiating shutdown
2010-08-31 16:00:07,419 INFO  [Shutdown] [] TcpServer/110 - Pop3Server initiating shutdown
2010-08-31 16:00:07,420 INFO  [Pop3Server] [] TcpServer/110 - finished accept loop
2010-08-31 16:00:07,521 INFO  [Shutdown] [] TcpServer/110 - Pop3Server shutting down idle thread pool
2010-08-31 16:00:07,521 INFO  [Shutdown] [] TcpServer/995 - Pop3SSLServer initiating shutdown
2010-08-31 16:00:07,522 INFO  [Pop3SSLServer] [] TcpServer/995 - finished accept loop
2010-08-31 16:00:07,720 INFO  [Shutdown] [] TcpServer/995 - Pop3SSLServer shutting down idle thread pool
2010-08-31 16:00:07,721 INFO  [Shutdown] [] TcpServer/143 - ImapServer initiating shutdown
2010-08-31 16:00:07,721 INFO  [ImapServer] [] TcpServer/143 - finished accept loop
2010-08-31 16:00:07,816 INFO  [Shutdown] [] TcpServer/143 - ImapServer shutting down idle thread pool
2010-08-31 16:00:07,816 INFO  [Shutdown] [] TcpServer/993 - ImapSSLServer initiating shutdown
2010-08-31 16:00:07,816 INFO  [ImapSSLServer] [] TcpServer/993 - finished accept loop
2010-08-31 16:00:07,877 INFO  [Shutdown] [] TcpServer/993 - ImapSSLServer shutting down idle thread pool

....
2010-08-31 16:00:09,172 WARN  [Shutdown] [] ZimbraHttpConnectionManager - shutting down http client idle connection reaper thread
2010-08-31 16:00:09,173 INFO  [Shutdown] [] soap - Servlet SoapServlet shutting down
2010-08-31 16:00:09,342 INFO  [Shutdown] [] log - Shutdown hook complete

Startup:

2010-08-31 16:05:26,172 INFO  [main] [] soap - Servlet SoapServlet starting up
2010-08-31 16:05:26,501 INFO  [main] [] soap - Adding service AccountService to SoapServlet
2010-08-31 16:05:26,861 INFO  [main] [] soap - Adding service MailService to SoapServlet
2010-08-31 16:05:27,940 INFO  [main] [] soap - Adding service IMService to SoapServlet
2010-08-31 16:05:27,986 INFO  [main] [] misc - version=6.0.8_GA_2637 release=20100811142254 builddate=20100811-1424 buildhost=build01.lab.zimbra.com
2010-08-31 16:05:27,989 INFO  [main] [] misc - LANG environment is set to: C
2010-08-31 16:05:27,989 INFO  [main] [] misc - System property java.home=/opt/zimbra/jdk1.6.0_21/jre
2010-08-31 16:05:27,989 INFO  [main] [] misc - System property java.runtime.version=1.6.0_21-b06
2010-08-31 16:05:27,989 INFO  [main] [] misc - System property java.version=1.6.0_21
2010-08-31 16:05:27,989 INFO  [main] [] misc - System property java.vm.info=mixed mode
2010-08-31 16:05:27,989 INFO  [main] [] misc - System property java.vm.name=Java HotSpot(TM) Server VM
2010-08-31 16:05:27,989 INFO  [main] [] misc - System property java.vm.version=17.0-b16
2010-08-31 16:05:27,989 INFO  [main] [] misc - System property os.arch=i386
2010-08-31 16:05:27,989 INFO  [main] [] misc - System property os.name=Linux
2010-08-31 16:05:27,989 INFO  [main] [] misc - System property os.version=2.6.9-22.EL
2010-08-31 16:05:27,989 INFO  [main] [] misc - System property sun.arch.data.model=32
2010-08-31 16:05:27,989 INFO  [main] [] misc - System property sun.cpu.endian=little
2010-08-31 16:05:27,989 INFO  [main] [] misc - System property sun.cpu.isalist=
2010-08-31 16:05:27,990 INFO  [main] [] misc - System property sun.os.patch.level=unknown
2010-08-31 16:05:28,117 INFO  [main] [] system - Setting mysql connector property: maxActive=100
2010-08-31 16:05:28,139 INFO  [main] [] system - Setting mysql connector property: maxActive=100
2010-08-31 16:05:31,018 INFO  [main] [] sqltrace - Setting slow SQL threshold to 2000ms
...
2010-08-31 16:05:40,890 INFO  [LmtpServer] [] TcpServer/7025 - starting accept loop
2010-08-31 16:05:40,905 INFO  [Pop3Server] [] TcpServer/110 - starting accept loop
2010-08-31 16:05:41,062 INFO  [Pop3SSLServer] [] TcpServer/995 - starting accept loop
2010-08-31 16:05:41,106 INFO  [ImapServer] [] TcpServer/143 - starting accept loop
2010-08-31 16:05:41,218 INFO  [ImapSSLServer] [] TcpServer/993 - starting accept loop
...
2010-08-31 16:05:45,017 INFO  [main] [] log - Started SslSelectChannelConnector@0.0.0.0:443
2010-08-31 16:05:45,020 INFO  [main] [] log - Started SslSelectChannelConnector@0.0.0.0:7071
2010-08-31 16:05:45,020 INFO  [main] [] log - Started SelectChannelConnector@0.0.0.0:7072

SSL Cert Installation: Unmatching certificate and private key error

If you're seeing this error:

root@zimbra:/tmp# /opt/zimbra/bin/zmcertmgr verifycrt comm /opt/zimbra/ssl/zimbra/commercial/commercial.key commercial.crt commercial_ca.crt
** Verifying commercial.crt against /opt/zimbra/ssl/zimbra/commercial/commercial.key
XXXXX ERROR: Unmatching certificate (commercial.crt) and private key (/opt/zimbra/ssl/zimbra/commercial/commercial.key) pair. 

It may be that the commercial.crt doesn't match the signature from commercial.key. The following two hashes should match, but don't:

# openssl x509 -noout -modulus -in commercial.crt | openssl md5
f4033675a216bbbe0712917e3281fe3c
# openssl rsa -noout -modulus -in commercial.key | openssl md5
76991327ac5bdae0840270dffeb4b214

Here is an example of two matching certificate and key files deployed for LDAP:

# openssl x509 -noout -modulus -in /opt/zimbra/conf/slapd.crt | openssl md5
59688017a0601a37a07d8fcd4ec9908c
# openssl rsa -noout -modulus -in /opt/zimbra/conf/slapd.key | openssl md5
59688017a0601a37a07d8fcd4ec9908c

If you have the CSR you can check to which key it belongs:

openssl req -noout -modulus -in server.csr | openssl md5

For a full explanation of the above commands: http://kb.wisc.edu/middleware/page.php?id=4064

Installing a Thawte SSL Certificate in ZCS 5.x and 6.x

Use the following instructions to install a commercial Thawte SSL certificate. All the commands below should be performed as the root user (not zimbra).

1. Begin by generating a Certificate Signing Request (CSR).

# /opt/zimbra/bin/zmcertmgr createcsr comm -new –subject "/C=US/ST=CO/L=Broomfield/O=VMware/OU=Zimbra Collaboration Suite" –subjectAltNames host.example.com

2. Next, submit the CSR to the Thawte and get a commercial certificate in X.509/PEM format.

https://search.thawte.com/support/ssl-digital-certificates/index?page=content&id=SO13187

Save the new certificate to a temporary file (e.g. /tmp/comm.crt).

3. Determine which SSL product you bought from Thawte.

https://search.thawte.com/support/ssl-digital-certificates/index?page=content&id=SO1498&actp=LIST&viewlocale=en_US

Depending on which it is, click the corresponding ApacheSSL from the link above. For the directions below, we'll assume it's plain SSL Web Server:

https://search.thawte.com/support/ssl-digital-certificates/index?page=content&id=SO14822

4. Download the Intermediate Certificate Authority in Apache format (X.509/PEM):

The Intermediate CA you download depends on which Thawte SSL product was bought. Below, we're assuming SSL Web Server:

https://search.thawte.com/support/ssl-digital-certificates/index?page=content&actp=CROSSLINK&id=AR1374

# cd /tmp && wget https://search.thawte.com/library/VERISIGN/ALL_OTHER/thawte%20ca/SSL_CA_Bundle.pem

5. Download the Root Certificate Authority:

https://www.thawte.com/roots/index.html

Again, the Root CA you download depends on the Thawte SSL product purchased. The Root CA below is for SSL Web Server:

# cd /tmp && wget https://www.thawte.com/roots/thawte_Premium_Server_CA.pem

6. Combine Root and Intermediate CAs:

cat /tmp/thawte_Premium_Server_CA.pem /tmp/SSL_CA_Bundle.pem > /tmp/commercial_ca.crt

The order of files matters: first the Root CA followed by the Intermediate CA.

Look at commercial_ca.crt and make sure there are line breaks (there should be no more than 64 characters per line). If the file is not formatted correctly, Zimbra won't be able to import it.

7. Verify:

# /opt/zimbra/bin/zmcertmgr verifycrt comm /opt/zimbra/ssl/zimbra/commercial/commercial.key /tmp/comm.crt /tmp/commercial_ca.crt

8. Deploy:

# /opt/zimbra/bin/zmcertmgr deploycrt comm /tmp/comm.crt /tmp/commercial_ca.crt

Bulk delete items from all users matching query via zmmailbox

Suppose you have the following scenario:

We have an administrative assistant who left the company (admin_assistant@example.com). She had created reoccurring appointments on several people's calendars which now cannot be removed. Restoring the administrative assistant's account is not an option since she was removed a while back and our oldest backup doesn't contain her account. Any suggestions on how to remove the appointments?

As an administrator, you can script zmmailbox to search all accounts for appointments with an organizer of admin_assistant@example.com, and then delete those appointments.

#!/bin/bash
# tested on ZCS 6.0.8

if [ `whoami` != "zimbra" ]; then
  echo "You must be the zimbra user to run this script."
  exit 0
fi

# change query below to what you want to match
# the example searches for a body matching "Organizer: admin_assistant@example.com"
query="\"\\\"Organizer: admin_assistant@example.com\\\"\""

# change to search the type(s) of items you want to match
# types are: message,conversation,contact,appointment,document,task,wiki
type="appointment"

for acct in `zmprov -l gaa`
# if you want to search specific accounts, use the for loop below
# for acct in "99@test2.test" "100@test2.test" "101@test2.test" "102@test2.test" "103@test2.test"
do
  echo "Searching account $acct..."
  ITEMS=`zmmailbox -z -m $acct search -t $type "$query" | awk '{ if (NR>4 && NF) {print $2}}' | tr '\n' ,`
  if [ -n "$ITEMS" ]; then
    echo " Deleting item(s) " $ITEMS
    # Remove echo after verifying the search results
    echo zmmailbox -z -m $acct di $ITEMS
  fi
done

Credit to: King0770-Notes-Removal_of_Bad_Contact_Address

Deciphering zmstat charts

MySQL: InnoDB Buffer Pool Hit Rate

In general, should stay above 995.

Bad

Better

Ideal

Mailboxd: Minor Garbage Collection Time and Mailboxd: Major Garbage Collection Time

If the GC time starts getting over a few percent, especially repeatedly, then it's something that users will notice.