Alternative Way to monitor Zimbra service status from an external "monitor" server

Revision as of 22:49, 14 August 2010 by Richardteachout (talk | contribs)

I needed a way to monitor a cluster of Zimbra servers (in our case, we are running this on OSS editions, and on NE editions as well, so it has been fairly well tested, but use at your own risk - of course) from a remote monitor server. We didn't need details, disk space, just "were the services fully up and running" for the entire cluster. So we found a "poor mans" way to do it, which has worked fantastic for us, so I thought I would share.

Steps to create an external monitor server to monitor Zimbra service status Using a Logging Server

Install and Configure Apache

First, install Apache (and php is needed for this example), and configure it to run on an alternate port (it is also suggested you firewall this to only be allowed from authorized hosts, but that is beyond this concept page).

We have a dedicated server running a mailbox store, with NO users on it, and running logger with centralized syslog from all the server in our cluster of 10 machines, so we did this on that server, as it makes the most sense. You could do this on every/any server running logger however..

Then add something like:

<Virtualhost 12.34.yourip.here:81>
        ServerAdmin     support@yourdomain.com
        DocumentRoot    /opt/zimbra/customstat/www
        ServerName      stat01.yourdomain.com
        ErrorLog        /opt/zimbra/customstat/logs/error_log
        CustomLog       /opt/zimbra/customstat/logs/access_log common
       DirectoryIndex index.php
</VirtualHost>

Then make the directory /opt/zimbra/customstat/ (owned by zimbra is OK, just be sure you have a chmod 744 on the directory and a chmod 755 on the files you are about to run)

Create 3 files

Create 3 files.

Config file

The first is a small config file, so name it "config.php"

<?php
//Zimbra Logger Service mySQL Connection Information
$host      = "127.0.0.1:7307";
$mysqluser = "zimbra";
//Change this to call the zimbra function to get the password.. maybe
//to get the password, run    "/opt/zimbra/bin/zmlocalconfig -s -m nokey zimbra_logger_mysql_password" and put it below
$password = "";
?>

Status page

The second is the actual status page. name it "status.php"

<?php

include 'config.php';

if(!$dbconnect = mysql_connect("$host", "$mysqluser", "$password")) {
   print "Connection failed to the host 'localhost'.";
   exit;
} // if
if (!mysql_select_db('zimbra_logger')) {
   print "Cannot connect to database 'zimbra_logger'";
   exit;
} // if

$table_id = 'service_status';
$query = "SELECT * FROM $table_id";

$dbresult = mysql_query($query, $dbconnect);

// create a new XML document
$doc = new DomDocument('1.0');

// create root node
$root = $doc->createElement('root');
$root = $doc->appendChild($root);

// process one row at a time
while($row = mysql_fetch_assoc($dbresult)) {
 // add node for each row
  $occ = $doc->createElement($table_id);
  $occ = $root->appendChild($occ);
 // add a child node for each field
  foreach ($row as $fieldname => $fieldvalue) {
  $child = $doc->createElement($fieldname);
    $child = $occ->appendChild($child);
   $value = $doc->createTextNode($fieldvalue);
    $value = $child->appendChild($value);
  } // foreach
} // while
// get completed xml document
$xml_string = $doc->saveXML();

// Uncomment below if you are going to use this interface as an XML report
//if($_GET['raw']==1){
//        print $xml_string;
//        exit;
//}


//print $xml_string;
//exit;
$xml_doc = new DomDocument;
$xml_doc->loadXML($xml_string);

$xp = new XsltProcessor();
// create a DOM document and load the XSL stylesheet
  $xsl = new DomDocument;
  $xsl->load('status.xsl');
  // import the XSL styelsheet into the XSLT process
  $xp->importStylesheet($xsl);

 // transform the XML into HTML using the XSL file
  if ($html = $xp->transformToXML($xml_doc)) {
      echo $html;
  } else {
      trigger_error('XSL transformation failed.', E_USER_ERROR);
  } // if


?>

Layout page

The third is the layout page, name it status.xsl

<?xml version='1.0'?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">

<xsl:output method='html'/>


<!-- param values may be changed during the XSL Transformation -->
<xsl:param name="title">Server Status for IPmonitor</xsl:param>
<xsl:param name="script">test2.php</xsl:param>
<xsl:param name="numrows">0</xsl:param>
<xsl:param name="curpage">1</xsl:param>
<xsl:param name="lastpage">2</xsl:param>
<xsl:param name="script_time">0.2744</xsl:param>

<xsl:template match="root">

<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/>
<title><xsl:value-of select="$title"/></title>
    <style type="text/css">
      <![CDATA[
      <!--
        caption { font-weight: bold; }
        TH {      font-family: MS Sans Serif; font-size: 10pt;background-color: #CCCCCC;border-left: solid 1 #000000;
            border-top: solid 1 #000000;border-bottom: solid 1 #000000;border-right: none;padding-left: 5px;padding-right: 5px;padding-top: 2px;padding-bottom: 2px;}
        .footer { font-family: MS Sans Serif; font-size: 10pt;background-color: #CCCCCC;border-left: solid 1 #000000;
            border-top: solid 1 #000000;border-bottom: solid 1 #000000;border-right: none;padding-left: 5px;padding-right: 5px;padding-top: 2px;padding-bottom: 2px;}
        tr.odd {  background: #eeeeee; }
        tr.even { background: #dddddd; }
        .center { text-align: center; }
      -->
      ]]>
    </style>

</head>

<body>

<table border="1">
<caption><xsl:value-of select="$title"/></caption>
<thead>
<tr>
<th>Server.Service.Status</th>
<th>Time</th>
</tr>
</thead>
    <tbody>
          <xsl:apply-templates select="service_status">
            <xsl:sort select="server" />
            <xsl:sort select="service" />
          </xsl:apply-templates>
        <tr>
        <td colspan="2" class="footer">
        Status of 1 eq Running<br/>
        Status of 0 eq Not Running</td>
        </tr>
    </tbody>
</table>
</body>
</html>

</xsl:template>
<xsl:template match="service_status">
  <tr>
        <xsl:attribute name="class">
      <xsl:choose>
        <xsl:when test="position()mod 2">odd</xsl:when>
        <xsl:otherwise>even</xsl:otherwise>
      </xsl:choose>
    </xsl:attribute>
    <td><xsl:value-of select="server"/>.<xsl:value-of select="service"/>.<xsl:value-of select="status"/></td>
    <td><xsl:value-of select="time"/></td>
  </tr>
</xsl:template>
</xsl:stylesheet>

Start Monitoring Server

Now, just startup apache, and visit the new page you have created: You should get a table like this: (although it is actually in a real html table based on xsl above, but i couldn't figure that out in this wiki)

Server Status for IPmonitor 
Server.Service.Status Time 
ldap01.domain.com.ldap.1 2010-04-03 00:44:01 
ldap01.domain.com.snmp.1 2010-04-03 00:44:01 
ldap01.domain.com.stats.1 2010-04-03 00:44:01 
ldap02.domain.com.ldap.1 2010-04-03 00:44:01 
ldap02.domain.com.snmp.1 2010-04-03 00:44:01 

For every server logging to this logger, you should see a row with the servername.service.0/1 (0=not running, 1=running) and the date of last update


Now, you can use just about any monitoring system to do a GET on the /status.php page, and check for existence of (for example) "ldap01.domain.com.ldap.1" if it doesn't exist or is .0, your monitor wouldn't match, and hence the service is down, so set-off the appropriate alerts to notify your Support team.


Yes, all of this could be done using SNMP, but the OSS version doesn't include SNMP natively, and we didn't want to configure it on these servers, so this was a quick way to get the data we needed, without much of any overhead on the server itself, as it was just a sql query, that looked up data that was already being populated.

Steps to create an external monitor server to monitor Zimbra service status from any server in "cluster" or single server

This was created as an alternate method to the previous, and make it a bit better (as it includes a disk space monitor too) and can be utilized without having to make SQL calls, but it uses zmsoap, and caches the output for X# of seconds (so zmsoap is not called too often for monitoring each service). This can then be run from any server in a multi-server environment (or if you only have one, obviously from that server) and monitor all the services of all the servers.

Install and Configure Apache

First, install Apache (and php is needed for this example), and configure it to run on an alternate port (it is also suggested you firewall this to only be allowed from authorized hosts, but that is beyond this concept page).


Then add something like:

<Virtualhost 12.34.yourip.here:81>
        ServerAdmin     support@yourdomain.com
        DocumentRoot    /opt/zimbra/customstat/www
        ServerName      stat01.yourdomain.com
        ErrorLog        /opt/zimbra/customstat/logs/error_log
        CustomLog       /opt/zimbra/customstat/logs/access_log common
       DirectoryIndex index.php
</VirtualHost>

Then make the directory /opt/zimbra/customstat/ (owned by zimbra is OK, just be sure you have a chmod 744 on the directory and a chmod 755 on the files you are about to run)

Create 2 files

Create 2 files.

status.php

<?php
print "<html><body>";
$filename = '/tmp/status.txt';
$cache_life = '90'; //caching time, in seconds

$filemtime = @filemtime($filename);
if (!$filemtime or (time() - $filemtime >= $cache_life)){
        $soapgetr = `/opt/zimbra/bin/zmsoap -z GetServiceStatusRequest`;
        $soapgetr = str_replace("/>","<br/>",$soapgetr);
        $soapgetr = str_replace("</status>","<br/>",$soapgetr);
        $soapgetr = str_replace("<status","",$soapgetr);
        $soapgetr = str_replace("\">","\"",$soapgetr);
        $soapgetr = str_replace("urn:zimbraAdmin\"","urn:zimbraAdmin\"<br/>",$soapgetr);
        $soapgetr = str_replace("<GetServiceStatusResponse","GetServiceStatusResponse",$soapgetr);
        $soapgetr = str_replace("</GetServiceStatusResponse>","",$soapgetr);
        $soapgetr = str_replace("<timezone","timezone",$soapgetr);
        $soapgetr = str_replace("\"","",$soapgetr);
        $soapgetr = str_replace(" server=",".",$soapgetr);
        $fp = fopen($filename, 'w');
        fwrite($fp, $soapgetr);
        fclose($fp);
}else{
        // Do nothing, it is OK, as it exists, and is recent
}

$soapget = readfile($filename);

print $soapget;
print "<br/>";
print `./disk.sh`;
print "</body></html>"
?>

disk.sh

#!/bin/bash
# Shell script to monitor or watch the disk space
ALERT=90
df -P -H | grep -vE '^Filesystem|tmpfs|cdrom' | awk '{ print $5 " " $1 }' | while read output;
do

  usep=$(echo $output | awk '{ print $1}' | cut -d'%' -f1  )
  partition=$(echo $output | awk '{ print $2 }' )

if [ $usep -ge $ALERT ]; then
    echo "Disk space LOW for \"$partition ($usep%)\" on $(hostname) as on $(date)<br/>"

else
    echo "Disk space OK for \"$partition ($usep%)\" on $(hostname) as on $(date)<br/>"
fi
done

Start Monitoring Server

Now, just startup apache, and visit the new page you have created: You should get a table like this: (although it is actually in a real html table based on xsl above, but i couldn't figure that out in this wiki)

GetServiceStatusResponse xmlns=urn:zimbraAdmin
timezone id=America/Chicago displayName=Central Standard Time
t=1281825600 service=stats.ldap.yourdomain.com1
t=1281825600 service=ldap.ldap.yourdomain.com1
t=1281825630 service=convertd.mstore1.yourdomain.com1
t=1281825630 service=mta.mstore1.yourdomain.com1
t=1281825718 service=spell.mstore1.yourdomain.com1
t=1281825718 service=snmp.mstore1.yourdomain.com1
t=1281825718 service=stats.mstore1.yourdomain.com1
t=1281825718 service=memcached.mstore1.yourdomain.com1
t=1281825718 service=mailbox.mstore1.yourdomain.com1
t=1281825718 service=logger.mstore1.yourdomain.com1
t=1281825718 service=antispam.mstore1.yourdomain.com1
t=1281825718 service=antivirus.mstore1.yourdomain.com1
899
Disk space OK for "/dev/mapper/VolGroup00-LogVol00 (44%)" on mstore1 as on Sat Aug 14 17:47:56 CDT 2010
Disk space OK for "/dev/sda1 (13%)" on mstore1 as on Sat Aug 14 17:47:56 CDT 2010

(Yes, it only monitors disk space of the server actually running this... but hey, if you only have 1 server, PERFECT!)

For every server in the cluster, you should see a row with the service.servername0/1 (0=not running, 1=running) and the date of last update

Now, you can use just about any monitoring system to do a GET on the /status.php page, and check for existence of (for example) "logger.mstore1.yourdomain.com1" if it doesn't exist or is .0, your monitor wouldn't match, and hence the service is down, so set-off the appropriate alerts to notify your Support team.


Enjoy!

Back to Monitoring_Tools_Articles


Verified Against: unknown Date Created: 4/2/2010
Article ID: https://wiki.zimbra.com/index.php?title=Alternative_Way_to_monitor_Zimbra_service_status_from_an_external_%22monitor%22_server Date Modified: 2010-08-14



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »

Jump to: navigation, search