https://wiki.zimbra.com/index.php?title=Monitoring_Zimbra_Collaboration_-_InfluxDB,_Telegraf_and_Grafana&feed=atom&action=historyMonitoring Zimbra Collaboration - InfluxDB, Telegraf and Grafana - Revision history2024-03-28T23:17:15ZRevision history for this page on the wikiMediaWiki 1.39.0https://wiki.zimbra.com/index.php?title=Monitoring_Zimbra_Collaboration_-_InfluxDB,_Telegraf_and_Grafana&diff=64913&oldid=prevJorge de la Cruz: /* Script to collect information from Zimbra Collaboration */2017-12-08T17:08:50Z<p><span dir="auto"><span class="autocomment">Script to collect information from Zimbra Collaboration</span></span></p>
<table style="background-color: #fff; color: #202122;" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 17:08, 8 December 2017</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l23">Line 23:</td>
<td colspan="2" class="diff-lineno">Line 23:</td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>In order to collect information from the Zimbra Collaboration mail queues, as well as to calculate messages sent, etc., I have used the pflogsumm itself, and with it I have modified the output to be in InfluxDB format.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>In order to collect information from the Zimbra Collaboration mail queues, as well as to calculate messages sent, etc., I have used the pflogsumm itself, and with it I have modified the output to be in InfluxDB format.</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>You [https://<del style="font-weight: bold; text-decoration: none;">github</del>.com/jorgedlcruz/zimbra-grafana<del style="font-weight: bold; text-decoration: none;">/blob</del>/master/zimbra_pflogsumm.pl can download the script from here] and locate it in the following route, I have called it zimbra_pflogsumm. pl and make it executable:</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>You [https://<ins style="font-weight: bold; text-decoration: none;">raw.githubusercontent</ins>.com/jorgedlcruz/zimbra-grafana/master/zimbra_pflogsumm.pl can download the script from here] and locate it in the following route, I have called it zimbra_pflogsumm. pl and make it executable:</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div> /opt/zimbra/common/bin/zimbra_pflogsumm.pl</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div> /opt/zimbra/common/bin/zimbra_pflogsumm.pl</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div> chmod +rwxr+xr+x /opt/zimbra/common/bin/zimbra_pflogsumm.pl</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div> chmod +rwxr+xr+x /opt/zimbra/common/bin/zimbra_pflogsumm.pl</div></td></tr>
</table>Jorge de la Cruzhttps://wiki.zimbra.com/index.php?title=Monitoring_Zimbra_Collaboration_-_InfluxDB,_Telegraf_and_Grafana&diff=64912&oldid=prevJorge de la Cruz: /* Script to collect information from Zimbra Collaboration */2017-12-08T17:08:26Z<p><span dir="auto"><span class="autocomment">Script to collect information from Zimbra Collaboration</span></span></p>
<table style="background-color: #fff; color: #202122;" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 17:08, 8 December 2017</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l23">Line 23:</td>
<td colspan="2" class="diff-lineno">Line 23:</td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>In order to collect information from the Zimbra Collaboration mail queues, as well as to calculate messages sent, etc., I have used the pflogsumm itself, and with it I have modified the output to be in InfluxDB format.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>In order to collect information from the Zimbra Collaboration mail queues, as well as to calculate messages sent, etc., I have used the pflogsumm itself, and with it I have modified the output to be in InfluxDB format.</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>You [https://github.com/jorgedlcruz/zimbra-grafana/blob/master/zimbra_pflogsumm.pl can download the script from here] and locate it in the following route, I have called it zimbra_pflogsumm. pl</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>You [https://github.com/jorgedlcruz/zimbra-grafana/blob/master/zimbra_pflogsumm.pl can download the script from here] and locate it in the following route, I have called it zimbra_pflogsumm. pl <ins style="font-weight: bold; text-decoration: none;">and make it executable:</ins></div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div> /opt/zimbra/common/bin/zimbra_pflogsumm.pl</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div> /opt/zimbra/common/bin/zimbra_pflogsumm.pl</div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"> chmod +rwxr+xr+x /opt/zimbra/common/bin/zimbra_pflogsumm.pl</ins></div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br/></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>You can prove that it returns content to you, and that the telegraph user can run it by launching:</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>You can prove that it returns content to you, and that the telegraph user can run it by launching:</div></td></tr>
</table>Jorge de la Cruzhttps://wiki.zimbra.com/index.php?title=Monitoring_Zimbra_Collaboration_-_InfluxDB,_Telegraf_and_Grafana&diff=64440&oldid=prevJorge de la Cruz: Created page with "{{BC|Community Sandbox}} __FORCETOC__ <div class="col-md-12 ibox-content"> =Monitoring Zimbra Collaboration - InfluxDB, Telegraf and Grafana= {{KB|{{Unsupported}}|{{ZCS 8.8}}|..."2017-09-08T14:51:05Z<p>Created page with "{{BC|Community Sandbox}} __FORCETOC__ <div class="col-md-12 ibox-content"> =Monitoring Zimbra Collaboration - InfluxDB, Telegraf and Grafana= {{KB|{{Unsupported}}|{{ZCS 8.8}}|..."</p>
<p><b>New page</b></p><div>{{BC|Community Sandbox}}<br />
__FORCETOC__<br />
<div class="col-md-12 ibox-content"><br />
=Monitoring Zimbra Collaboration - InfluxDB, Telegraf and Grafana=<br />
{{KB|{{Unsupported}}|{{ZCS 8.8}}|{{ZCS 8.7}}|{{ZCS 8.6}}|}}<br />
<br />
On this Wiki, we will see how we can monitor Zimbra by using Grafana, InfluxDB and Telegraf, this dashboard is a very early version, v0.1 even. Your feedback is always welcome.<br />
<br />
At the end of this Wiki you can have a Dashboard similar to this one:<br />
<br />
[[File:Zimbra-grafana-001.png|800px]]<br />
<br />
==Infrastructure Topology==<br />
Here is the topology followed in this design, in it we have:<br />
<br />
* The Zimbra Collaboration environment on the left side of the image, with Telegraf's agent that collects information that we pass on to you through scripts<br />
* The InfluxDB database where all these metrics received from Telegraf are stored<br />
* The Grafana Dashboard that displays the information stored in InfluxDB<br />
<br />
[[File:Tig-monitor-logic.png|800px]]<br />
<br />
==Script to collect information from Zimbra Collaboration==<br />
In order to collect information from the Zimbra Collaboration mail queues, as well as to calculate messages sent, etc., I have used the pflogsumm itself, and with it I have modified the output to be in InfluxDB format.<br />
<br />
You [https://github.com/jorgedlcruz/zimbra-grafana/blob/master/zimbra_pflogsumm.pl can download the script from here] and locate it in the following route, I have called it zimbra_pflogsumm. pl<br />
/opt/zimbra/common/bin/zimbra_pflogsumm.pl<br />
<br />
You can prove that it returns content to you, and that the telegraph user can run it by launching:<br />
sudo -u telegraf /opt/zimbra/common/bin/zimbra_pflogsumm.pl /var/log/zimbra.log<br />
<br />
If everything went well, you can see a result similar to this one, but with your numbers of course:<br />
<pre>zimbra-stats received=76<br />
zimbra-stats delivered=327<br />
zimbra-stats forwarded=0<br />
zimbra-stats deferred=1<br />
zimbra-stats bounced=5<br />
zimbra-stats rejected=6<br />
zimbra-stats reject_warnings=0<br />
zimbra-stats held=0<br />
zimbra-stats discarded=0<br />
zimbra-stats bytes_received=11889<br />
zimbra-stats bytes_delivered=32229<br />
zimbra-stats senders=7<br />
zimbra-stats sending_hosts_domains=5<br />
zimbra-stats recipients=73<br />
zimbra-stats recipient_hosts_domains=6</pre><br />
<br />
==Telegraf configuration in Zimbra Collaboration agents==<br />
In the servers of Zimbra Collaboration we will have to install Telegraf, you have all the steps in [https://www.jorgedelacruz.es/2016/07/07/en-busca-del-dashboard-perfecto-influxdb-telegraf-y-grafana-parte-ii/ the following Blog post], once we have Telegraf installed, we will have to configure our additional scripts at the end of the file, this way my part at the end shows the scripts of Zimbra and the others to monitor the state of the server:<br />
<pre># Read metrics about cpu usage<br />
[[inputs.cpu]]<br />
## Whether to report per-cpu stats or not<br />
percpu = true<br />
## Whether to report total system cpu stats or not<br />
totalcpu = true<br />
## Comment this line if you want the raw CPU time metrics<br />
fielddrop = ["time_*"]<br />
<br />
# Read metrics about disk usage by mount point<br />
[[inputs.disk]]<br />
ignore_fs = ["tmpfs", "devtmpfs"]<br />
<br />
# Read metrics about disk IO by device<br />
[[inputs.diskio]]<br />
<br />
# Get kernel statistics from /proc/stat<br />
[[inputs.kernel]]<br />
<br />
# Read metrics about memory usage<br />
[[inputs.mem]]<br />
<br />
# Get the number of processes and group them by status<br />
[[inputs.processes]]<br />
<br />
# Read metrics about swap memory usage<br />
[[inputs.swap]]<br />
<br />
# Read metrics about system load & uptime<br />
[[inputs.system]]<br />
<br />
[[inputs.procstat]]<br />
exe = "memcached"<br />
prefix = "memcached"<br />
<br />
[[inputs.procstat]]<br />
exe = "java"<br />
prefix = "java"<br />
<br />
[[inputs.procstat]]<br />
exe = "mysqld"<br />
prefix = "mysqld"<br />
<br />
[[inputs.procstat]]<br />
exe = "slapd"<br />
prefix = "slapd"<br />
<br />
[[inputs.procstat]]<br />
exe = "nginx"<br />
prefix = "nginx"<br />
<br />
[[inputs.net]]<br />
<br />
[[inputs.exec]]<br />
commands = ["/opt/zimbra/common/bin/zimbra_pflogsumm.pl -d today /var/log/zimbra.log"]<br />
name_override = "zimbra_stats"<br />
data_format = "influx"<br />
<br />
[[inputs.exec]]<br />
commands = ["sed 's/........................//' /opt/zimbra/jetty/webapps/zimbra/downloads/.git/HEAD"]<br />
name_override = "zimbra_stats"<br />
data_format = "value"<br />
data_type = "string"<br />
</pre><br />
<br />
Once this is done, we'll be able to reboot the service:<br />
service telegraf restart<br />
<br />
==Importing Dashboard in Grafana==<br />
To make your life simpler, I leave you the. json de Grafana ready to be imported, you can download it from here:<br />
* https://grafana.com/dashboards/2846/<br />
I want to thank Lex Rivera for his work, who is the creator of the rest of the Dashboard for the Linux, CPU, Disk, RAM, etc. metrics.<br />
<br />
The steps to import them are as follows, go to our Dashboards, and press Import:<br />
<br />
[[File:Grafana-esxi-014.png|800px]]<br />
<br />
The following screen will appear, and we can either upload the downloaded. json file, or enter the ID of the Zimbra Dashboard: 2846, we will automatically appear the following, select our datasource and give it to Import.<br />
<br />
[[File:Zimbra-grafana-002.png |800px]]<br />
<br />
After just a few minutes we can start to see a lively Dashboard as follows:<br />
<br />
[[File:Zimbra-grafana-001.png|800px]]<br />
<br />
==Related articles==<br />
* https://github.com/jorgedlcruz/zimbra-grafana<br />
* https://grafana.com/dashboards/2846<br />
<br />
{{Article Footer|Zimbra Collaboration Suite 8.8, 8.7|09/09/2017}}<br />
[[Category: Logger]]<br />
[[Category: Logging]]</div>Jorge de la Cruz