Difference between revisions of "Troubleshooting Course Content Rough Drafts-Zimbra Logs"

Line 80: Line 80:
  
 
This will write debug postfix logs to /var/log/zimbra.log
 
This will write debug postfix logs to /var/log/zimbra.log
 +
 +
=== SpamAssassin ===
 +
* Debug logging for SpamAssassin controlled by “zimbraAmavisSALogLevel”, this writes "sa_debug" to /opt/zimbra/conf/amavisd.conf.
 +
* Debug logs get written to /var/log/zimbra.log
  
 
=== Amavis ===
 
=== Amavis ===

Revision as of 12:18, 27 February 2015

Introduction

Verified Against: Zimbra Collaboration Suite 8.6 Date Created: 01/22/2015
Article ID: https://wiki.zimbra.com/index.php?title=Troubleshooting_Course_Content_Rough_Drafts-Zimbra_Logs Date Modified: 2015-02-27



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »

Log Files

Brief list of the major log files in Zimbra and their significance.

  • /opt/zimbra/log/mailbox.log - Zimbra Mailboxd (jetty) writes all mailbox activity to this log file. This activity can be anything from moving a message to deletion of an account.
  • /opt/zimbra/log/zmmailboxd.out – Mailboxd(jetty) logs to this file during startup.
  • /opt/zimbra/log/audit.log – Logs authentication activity.
  • /opt/zimbra/log/sync.log – Mobile sync operations get logged to this log file.
  • /opt/zimbra/log/clamd.log – Clamd (Antivirus daemon) logs all antivirus activity to this file.
  • /opt/zimbra/log/freshclam.log – Antivirus definition update.
  • /opt/zimbra/log/mysql_error.log – Mysql errors and startup messages. This log should be looked up to determine if innodb corruption has occurred.
  • /opt/zimbra/log/myslow.log – Consists sql statements which took more than long_query_time seconds (zimbra default – 1second) to execute and required atleast min_examined_row_limit (zimbra default - 0) rows to be examined.
  • /opt/zimbra/log/nginx.access.log - NGINX writes information about client requests after the request is processed.
  • /opt/zimbra/log/nginx.log – Nginx error log which defaults to “info” in Zimbra.
  • /opt/zimbra/log/spamtrain.log – Spam training log.
  • /opt/zimbra/log/ews.log – EWS server side activity.
  • /var/log/zimbra.log – All MTA (Postfix, amavisd, antispam, antivirus,cyrus-sasl)/Ldap logs go to this file.
  • /var/log/zimbra-stats.log – Logger data gets logged to this file.

Log Rotation

Two different log rotation daemons handle log rotation of the zimbra log files.

  • Log4j - Log4j is responsible for logging to mailbox.log, audit.log, sync.log, synctrace.log,webxml.log and ews.log. For config : /opt/zimbra/conf/log4j.properties

Example, below the are the config options for the rotation of /opt/zimbra/log/mailbox.log

log4j.appender.LOGFILE=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.LOGFILE.RollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy
log4j.appender.LOGFILE.RollingPolicy.FileNamePattern=/opt/zimbra/log/mailbox.log.%d{yyyy-MM-dd}
log4j.appender.LOGFILE.File=/opt/zimbra/log/mailbox.log
log4j.appender.LOGFILE.layout=com.zimbra.common.util.ZimbraPatternLayout
log4j.appender.LOGFILE.layout.ConversionPattern=%d %-5p [%t] [%z] %c{1} - %m%n
  • Logrotate.d - The OS daemon logrotate used to rotate other log files.
Config file : /etc/logrotate.d/zimbra 

Files rotated by logrotate : zimbra.log, zimbra-stats.log, myslow.log, zmconfigd.log, zmconfigd.log, clamd.log, freshclam.log, nginx.log, nginx_access.log, cbpolicyd.log and spamtrain.log.

Debug Logging

Mailbox

  • Global debug logging : Doing this will enable debug logging for any log4j property.
Edit - /opt/zimbra/conf/log4j.properties.in
log4j.logger.zimbra.imap=DEBUG
Restart mailboxd – zmmailboxdctl restart
  • Per user debug logging :
addAccountLogger : zmprov aal user@domain.com zimbra.imap debug
removeAccountLogger : zmprov ral user@domain.com zimbra.imap

To list all possible loggers -

zmprov help log

Ldap

  • Ldap debug logging – Controlled by “ldap_common_loglevel”, this writes the config ldap attribute "olcLogLevel".

Examples -

 zmlocalconfig –e ldap_common_loglevel=256
         or
 zmlocalconfig –e ldap_common_loglevel=“stats”
 Restart -  ldap stop; ldap start.  

For various log levels please see -

http://www.openldap.org/software/man.cgi?query=slapd.conf&apropos=0&sektion=0&manpath=OpenLDAP+2.4-Release&format=html

Postfix

  • Edit /opt/zimbra/postfix/conf/master.cf.in
Change - smtp      inet  n       -       n       -       -       smtpd
To - smtp      inet  n       -       n       -       -       smtpd –vv

Restart of postfix required

zmmtactl restart

This will write debug postfix logs to /var/log/zimbra.log

SpamAssassin

  • Debug logging for SpamAssassin controlled by “zimbraAmavisSALogLevel”, this writes "sa_debug" to /opt/zimbra/conf/amavisd.conf.
  • Debug logs get written to /var/log/zimbra.log

Amavis

  • Amavisd debug logging - Controlled by zimbra attribute "zimbraAmavisLogLevel", which writes value of "$log_level" in file /opt/zimbra/conf/amavisd.conf
zmprov ms `zmhostname` "zimbraAmavisLogLevel" 5

This will write debug log to - /var/log/zimbra.log

Proxy

  • Controlled by zimbra attribute zimbraReverseProxyLogLevel.
zmprov ms `zmhostname` zimbraReverseProxyLogLevel debug
zmnginxctl restart

It will log a lot, so you can set it back to info at any time:

 zmprov ms `zmhostname` zimbraReverseProxyLogLevel info

This will write debug logs to /opt/zimbra/log/nginx.log

ZCO Logs

  • For all but the simplest queries, ZCO Dev needs ZCO logs, dumps and event viewer logs. Here's how to gather them:
Disable Windows Search 
start->run->services->Windows Search->Properties->Stop + “Disabled” 
Start Outlook 
Disable Foreign Addins i.e. any addins that dont start with "Zimbra" 
OL2003: Tools->Options->Other->Advanced Options->COM Add-Ins->Remove any addins that don't start  with "Zimbra" 
OL2007: Tools->Trust Center->Add-ins->Manage COM Addins->Go->Untick any addins that don't start with "Zimbra" 
OL2010: File->Options->Add-ins->Manage COM Addins->Go->Untick any addins that don't start with "Zimbra" 
OL2013: as OL2010 
Close Outlook 
  • Empty ZCOLogs Directory (to avoid sending ZCO Dev stale/irrelevant logs)
  • Set Log Settings (using ZCOLogCtl.exe) to the following -
 Click the Default button 
 Max size of single log file : 50Mb 
 Preserve Log File Backups ticked 
 Max Size of ZCOLogs directory UNticked

ZCO Trouble Shooting

  • Start Outlook + Repro the Issue
  • Take detailed notes of the steps
  • If possible, make a note of the time the problem ocurred
  • If the problem relates to a particular item, include its ZimbraID and/or Subject
  • Include Screen shots; Screen video if possible
  • Close Outlook. If Outlook hung, grab a Hang Dump:
  • Task Manager->Processes Tab->Right click OUTLOOK.EXE->Create Dump File
  • Export Outlook Events from the Windows Event Viewer:
  • Start Event Viewer
  • Select "Windows Logs" | Application from the left pane.
  • In Actions (right pane) select "Filter Current Log..."
  • In the Filter dialog select
  • select "Last 7 days" from the "Logged:" dropdown
  • check Outlook in the "Event sources:"
  • they will probably also need to limit output to the current "User:"
  • Click OK
  • Save with "Save Filtered Log File...", and pass us the resulting .evtx file
  • Check logs before sending to ZCO Dev
  • Must cover the repro period (note that each log file name includes the date and time at which a client session started)
  • Check for obvious issues (e.g. server unreachable)
## ERROR ## Usually indicates an unexpected problem 
## WARNING ## Usually benign but worth knowing 
Jump to: navigation, search