Logger (ZCS 6.0.x and later)

Logger in version 6.0 (GnR)

Description

Logger is a non-essential ZCS service that processes logs produced by Zimbra, and associated services. Logs are processed into a format that is readily displayed in the Zimbra Admin Console indicating real-time service status, MTA/spam/virus traffic and performance statistics.

What's changed since 5.0 (and earlier)

The logger service has been completely redesigned and redeveloped for ZCS 6.0

The service no longer relies on mysql and most command-line utilities related to the old service are no longer available.

zmmsgtrace has been removed with no plan for a replacement.

Architecture and Implementation

zmlogger

This process is launched through zmloggerctl and runs on a pipe from zmlogswatch, it tails /var/log/zimbra.log and captures information from the 'zimbramon' and 'mailboxd' services and parses out statistics information.

Zmlogger reads an sqlite database to determine the type of each stat counter and mappings to RRD file:data-columns and writes the stat counters to the appropriate RRD file(s).

Log entries that are unknown are spooled into an 'unprocessed.log' file

zmlogprocess

Zmlogprocess is launched out of cron on the logger host every 5 minutes and reads the 'unprocessed.log' created by zmlogger. Upon launch, zmlogprocess renames 'unprocessed.log' so that zmlogger can start spooling to a new file.

This job reads MTA data from unprocessed.log: MTA traffic and volume, SPAM and Virus infections. The data is then sent back to zmlogger as stat counters via syslog.

zmstat

Zmstats is also integrated into the logger service by broadcasting its stat counters over syslog in addition to using the old CSV files.

Stats will still be readable using the zmstat-chart tool in addition to live, real-time data displayed in the admin console.

syslog

The logger service will use syslog as a mechanism for aggregating data to a centralized server. Messages are generally sent in the format of

 message-type: headers(csv):: data-columns(csv)

Messages over 900 characters in length are broken up using a uuid eliding scheme. This scheme is required due to many syslogd implementations restricting message length to be 1022 bytes or shorter. The scheme works by appending :::uuid to prepending :::uuid to messages that are split. The UUID allows easily matching long messages together in the event multiple long messages occur at once (common).

sqlite

SQLite is used as an integration database. It maps dns names to zmhostname when there are mismatches, stores information about stat counter types and a mapping of a stat counter to an RRD file's datasource column.

Schema

Our integration database schema is as follows:

• hosts: columns (id, dns_hostname, zm_hostname); generated dynamically as logger works

Self-explanatory

• rrds: columns (id, host_id, col_name, col_name_19, col_num, csv_file, rrd_file); also generated dynamically as logger works
  • host_id is the host for which this column is mapping
  • col_name_19 is the name of the column in the rrd file
  • col_num is the index of the column in the rrd_file
  • csv_file is the grouping of the stat counter
  • rrd_file is a number that, combined with the host_id, determines the rrd filename

• datatypes: columns (csv_file, col_name, col_type); static mappings table (coming in beta2)
  • csv_file is the grouping of the stat counter
  • col_name is the column for which this is determining the datatype
  • col_type is the type of the column GAUGE, DERIVED, COUNTER or ABSOLUTE (rrd types)

• config: columns (schema_version); single row table used for schema versioning (coming in beta2)

zmloggerhostmap

This is a command used to manipulate the DNS hostname to zmhostname mappings when there is a mismatch in the information reported to Logger

rrd

RRDs are used to store stat counter information, it manages aggregation and averaging. Generally, one RRD file is generated per group of statistics and information is retained for 2 years in varying levels of granularity (from 30 second to 1 hour resolution).

zmrrdfetch

This is an internal command that ZCS will use to fetch and combine data from the RRD files. The information will often be used to form graphs and charts in the admin console.