Centralized Logs - VMware Log Insight
|This article applies to the following ZCS versions.|
Centralized Logs - VMware Log Insight
The goal is install the VMware Log Insight appliance based in Open Suse to have a Centralized Log Server, and also a powerfull Dashboard to configure all the reports.
The VMware Log Insight Appliance will have all the Log Components, in the Zimbra Server, or servers, will be installed the Agent.
What is VMware vRealize Log Insight?
A while ago it was called vCenter Log Insight, now VMware vRealize Log Insight. It offers a Centralized and real-time log monitoring. Is based in VMware, and can be installed in appliance format. VMware vRealize Log Insight offer a high efficient search of our Logs and can help us to troubleshoot problems in our Zimbra environments using the Logs.
Zimbra with VMware vRealize Log Insight using pure Syslog
In this configuration, we will learn how to configure our Zimbra Server to send using Syslog the Zimbra Collaboration Logs. This is the simple step, really simple to configure it.
Configuring our Zimbra Collaboration Server
First, need to edit the Rsyslog configuration of our Zimbra, we need to edit the next file /etc/rsyslog.conf. Is important to add our own VMware vRealize Log Insight IP or hostname, and also the port 514, if we didn't change it in the Appliance, this is the default port.
# provides UDP syslog reception $ModLoad imudp $UDPServerRun 514 *.* @IPorHOSTNAMELOGINSIGHT:514
Next step is restart the Rsyslog service:
We can test if the VMware vRealize Log Insight can receive messages from our Zimbra server, we can run a UDP probe: nc -u IPorHOSTNAMELOGINSIGHT 514 Hi, from Zimbra Server
And press CTRL+C, we can see the test under the tab Interactive Analytics.
Monitoring the Logs from our Zimbra Server in VMware vRealize Log Insight
Now, login to the VMware vRealize Log Insight appliance. Go under the tab Interactive Analytics.
For example, we can sort the Logs per Host, App, etc.
Working with VMware vRealize Log Insight, instant help
Truthly, the best moment to have a Centralized Log system like this is when some problem happen, suffer an attack, etc. For example, with the next command we can see the Zimbra logged users: cat /var/log/zimbra.log | grep sasl_method
With a Centralized Log System, we can check all the Servers in the same time, or just the affected server, amazing stuff. So in the search field you can put the next text and have in real time the Logged Users, sasl_method. Also, have an auto-complete.
Here the result in my lab environment, I'm the only user right now:
We can also add this search into the Dashboard, to use it and have it always available:
Zimbra Collaboration using the VMware vRealize Log Insight Agent
Like the rest of the Log appliance, using an Agent, we can receive more and much better information. The previous step was a simpler one, but maybe we want more, just follow this steps.
Install the VMware vRealize Log Insight Agent in Zimbra Collaboration
We can find all the agents for the different OS in the next URL: https://YOURVREALIZELOGINSIGHTSERVER/admin/agents, we can see something like this:
If we had a previous agent installed, then we will see the next page:
In the next step, download the agent and install it, in this lab case, I will use the Debian one, because I'm using Ubuntu: sudo dpkg deb -i VMware-Log-Insight-Agent_2.5.0-2347850.deb
This is the installation process:
Preparing to unpack VMware-Log-Insight-Agent_2.5.0-2347850.deb ... Unpacking vmware-log-insight-agent (2.5.0-2347850) ... Setting up vmware-log-insight-agent (2.5.0-2347850) ... Adding system startup for /etc/init.d/liagentd ... /etc/rc0.d/K20liagentd -> ../init.d/liagentd /etc/rc1.d/K20liagentd -> ../init.d/liagentd /etc/rc6.d/K20liagentd -> ../init.d/liagentd /etc/rc2.d/S20liagentd -> ../init.d/liagentd /etc/rc3.d/S20liagentd -> ../init.d/liagentd /etc/rc4.d/S20liagentd -> ../init.d/liagentd /etc/rc5.d/S20liagentd -> ../init.d/liagentd Starting VMware Log Insight Agent: * Installation completed. Please edit /var/lib/loginsight-agent/liagent.ini to configure the agent. For online documentation please visit: http://pubs.vmware.com/log-insight-25/index.jsp?topic=%2Fcom.vmware.log-insight.administration.doc%2FGUID-DB4A27CF-BDA7-443F-94FB-AB9097AD8008.html Processing triggers for ureadahead (0.100.0-16) ... ureadahead will be reprofiled on next reboot
Configuration of the VMware vRealize Log Insight Agent
The Agent configuration is pretty easy, we need to edit the next file /var/lib/loginsight-agent/liagent.ini and add the Log Files that we want to monitor, the default values are the syslog and the messages.log, we can delete them if we have too much information.
This is an example of Zimbra Single Server:
[filelog|messages] directory=/var/log include=messages;messages.? [filelog|syslog] directory=/var/log include=syslog;syslog.? [filelog|Zimbra-Audit] directory=/opt/zimbra/log include=audit.log;audit.log [filelog|Zimbra-Access] directory=/opt/zimbra/log include=access.log;access.log* [filelog|Zimbra-Clamd] directory=/opt/zimbra/log include=clamd.log;clamd.log [filelog|Zimbra-EWS] directory=/opt/zimbra/log include=ews.log;ews.log [filelog|Zimbra-Mailbox] directory=/opt/zimbra/log include=mailbox.log;mailbox.log [filelog|Zimbra-MYSQL-Error] directory=/opt/zimbra/log include=mysql_error.log;mysql_error.log [filelog|Zimbra-Zmmailboxd] directory=/opt/zimbra/log include=zmmailboxd.out;zmmailboxd.out
Maybe you are asking yourself why I'm adding the ? symbol in the end sometimes. You can read more information about this in the next link.
After the Agent configuration, will start to send information to the Server, in case that doesn't, restart the VMware vRealize Agent service:
/etc/init.d/liagentd restart Stopping VMware Log Insight Agent: * Starting VMware Log Insight Agent: *
Working with VMware vRealize Log Insight, instant help II
Now is turn to return to the VMware vRealize Log Insight Dashboard to see if we are capable to see the Agent installed.
The default value is load the vSpheremenu, but click on the VMware vSphere and change to the General one.
And there, we can find much more information about the Environment, Hosts, etc.
But the best tab is the Agents, in this tab we can find the Servers with the Agent installed on them.
If we scroll, we can find the overview with all the Log files that the appliance are receiving, we can click to create our own Dashboard.
For example, I've created a Dashboard called Zimbra with Agent, and inside of it I've added the next Widgets:
If we do click inside the chart, for example on the Log names, we can see the Interactive Analytics, with all the info, amazing.
The final thoughts are that VMWare vRealize Log Insight is a powerful tool Business Ready, also with the VMware Support behind it.
- Better than Elasticsearch, Logstas and Kibana, is that this Log Insight come into an Appliance, pretty easy to install.
- Have the Business Support from VMware.
- Is a Business Ready solution
- Is a solution ready for a Large Support Company with different Support Levels
- I like the way to parse and present the Logs
- Content Packs coming from different Vendors
- Have a cost, not sure how much, but expensive.
- The partitioning in vSphere, like it comes in Appliance, we can't play with the partitioning, etc.
- OpenSuse based, difficult to change or edit values.