Zmstats

Article Information
This article applies to the following ZCS versions. ZCS 5.0

Zmstats

Zmstats is how Zimbra exposes its performance metrics and statistics to the world. The information covers a wide array of data: disk usage, cpu utilization, java statistics, zimbra counters and beyond.

---

Zmstats consists of the following components and scripts:

• zmstatctl (all zmstat collection scripts are located in /opt/zimbra/libexec)
  • zmstat-fd
  • zmstat-df
  • zmstat-cpu
  • zmstat-proc
  • zmstat-io
  • zmstat-vm
  • zmstat-convertd
  • zmstat-mtaqueue
  • zmstat-mysql
• zmstat-chart
  • zmstat-chart-config

Running zmstats

• On the server where the stats were produced, make sure that the zmstat-chart.xml is provided. When running zmdiaglog, the zmstats and zmstat-chart-config are automatically produced]

# su - zimbra
$ zmstat-chart-config > /tmp/zmstat-chart.xml

• Make a charts directory:

$ mkdir ~/zmstat/2010-06-01/charts

• Then produce the stats:

$ zmstat-chart -c /tmp/zmstat-chart.xml -s ~/zmstat/2010-06-01 -d ~/zmstat/2010-06-01/charts

zmstatctl

zmstatctl is used to start and stop and various zmstat-* data logging scripts

zmstat-chart

zmstat-chart reads an XML configuration file generated by zmstat-chart-config and generates a set of HTML and PNG graph images suitable for rapidly diagnosing problems and load issues.

zmstat-chart-config.xml

Used to control what is graphed and how

Examples:

 <chart title="Mailboxd: JVM Heap Used"
        category="mailboxd"
        infile="mailboxd.csv"
        outfile="mboxd-heap-used.png"
        yAxis="MB">
   <plot data="heap_used" legend="total" divisor="1m"/>
 </chart>

The above defines a chart that reads the counter "heap_used" out of mailboxd.csv. It takes that counter and graphs it to a file "mboxd-heap-used.png". There will be a resulting graph with a yAxis labelled "MB" and "heap_used" divided by 1million (megabytes) will be graphed over time.

Multiple plots can be placed onto a single chart through the use of additional <plot> elements.

See /opt/zimbra/conf/zmstat-chart.xml for more examples

Individual stat scripts and their related counters

mailboxd.csv

Contains the following statistics written by the mailbox server. Counters that are not cumulative are reset once per minute.

lmtp_rcvd_msgs	Number of messages received over LMTP
lmtp_rcvd_bytes	Number of bytes of message data received over LMTP
lmtp_rcvd_rcpt	Number of LMTP recipients
lmtp_dlvd_msgs	Number of messages delivered over LMTP (incremented for each successful delivery to a recipient)
lmtp_dlvd_bytes	Number of bytes of message data delivered (incremented for each successful delivery to a recipient)
db_conn_count	Number of times that a database connection was requested
db_conn_ms_avg	Latency of getting a database connection
ldap_dc_count	Number of times an LDAP directory context was requested
ldap_dc_ms_avg	Latency of getting an LDAP directory context

zmstat-fd: fd.csv

Captures file descriptor usage on the system

• fd_count: current number of open file descriptors

zmstat-df: df.csv

Captures disk usage

• path: mount point
• disk: device
• disk_use: space used
• disk_pct_used: percentage used
• disk_space: total space

zmstat-cpu: cpu.csv

• cpu:user: total user time
• cpu:nice: total nice process time
• cpu:sys: total system time
• cpu:idle: total idle time
• cpu:iowait: total time in iowait
• cpu:irq: total time in irq
• cpu:softirq: total time in softirq
• cpuN:XXX: same as above, but per individual core/cpu

zmstat-io: io.csv and io-x.csv

• dev:tps: transactions per second
• dev:kB_read/s: read rate
• dev:kB_wrtn/s: write rate
• dev:kB_read: bytes read
• dev:kB_wrtn: bytes written

• dev:rrqm/s: read requests merged per second queued to device
• dev:wrqm/s: write requests merged per second queued to device
• dev:r/s: reads per second
• dev:w/s: writes per second
• dev:rkB/s read rate
• dev:wkB/s: write rate
• dev:avgrq-sz: average size (sectors) of requests
• dev:avgqu-sz: average queue length
• dev:await: average wait time for requests to be served
• dev:svctm: average time to service requests
• dev:%util: percentage of CPU time / bandwidth utilization of device

zmstat-proc: proc.csv

• user: total user time
• sys: total system time
• idle: total idle time
• iowait: total iowait time
• PROC-total-cpu: total cpu usage for PROC
• PROC-utime: usertime for PROC
• PROC-stime: system time for PROC
• PROC-totalMB: total memory footprint for PROC
• PROC-rssMB: resident-set-size of PROC
• PROC-sharedMB: shared memory of PROC
• PROC-process-count: number of threads/subprocesses

zmstat-vm: vm.csv

The output of vmstat(1) is recorded for reviewable statistics.

• r:
• b:
• swpd:
• free:
• buff:
• cache:
• si:
• so:
• bi:
• bo:
• in:
• cs:
• us:
• sy:
• id:
• wa:
• st:
• ...: other counters reported by vmstat

zmstat-mtaqueue: mtaqueue.csv

• KBytes: kilobytes queued by the mta
• requests: number of items queued by the mta

zmstat-convertd: convertd.csv

Collects CPU statistics for the convertd process (NE only).

• utime: user time for convertd
• stime: system time for convertd
• cputime: user + system time total
• rss: resident-set-size memory usage
• threads: number of threads for convertd
• processes: number of processes for convertd

Only if IO Accounting is enabled in the Linux kernel

• rchar: bytes read
• wchar: bytes written
• read_bytes: bytes read from disk
• write_bytes: bytes written to disk

zmstat-mysql: mysql.csv

Columns for mysql.csv are derived from the values of the query "SHOW GLOBAL STATUS". Refer to the mysql administration manual for further elaboration on the meanings of all its counters.

Related: Server Monitoring