Enabling Core Files
Enabling Core Files
Sometimes a bug in a piece of software will cause it to crash, and, if enabled, create a core file. This core file can often be used by Zimbra Technical Support to examine what problem occurred and either fix it, or contact the proper project (OpenLDAP, MySQL, etc) with the details so it can be fixed in their code and incorporated into ZCS. This wiki discusses how to enable core files on Linux and OSX OSes.
Enabling core files on Linux varies slightly based on kernel revision and distributor. The primary parts are to enable core dumps via sysctl settings. The parameters are:
This appends the PID to the generated core file, allowing multiple core dumps.
Depending on the kernel release, one of these two must be set to two. Older 2.6 kernels use the kernel.suid_dumpable parameter (RHEL4, CentOS4, SLES10, OpenSuSE 10.2). Newer ones use fs.suid_dumpable (RHEL5, CentOS5, RHEL6, F7, DEBIAN4.0, UBUNTU6, UBUNTU8, UBUNTU10, UBUNTU12). What this parameter does is allow binaries that are setuid (like slapd) to drop core.
Optional (but useful):
kernel.core_pattern = /tmp/core-%e-%s-%u-%g-%p-%t
or for the GNR release and later:
kernel.core_pattern = /opt/zimbra/data/tmp/core-%e-%s-%u-%g-%p-%t
Filename variables: %e is the filename %g is the gid the process was running under %p is the pid of the process %s is the signal that caused the dump %t is the time the dump occurred %u is the uid the process was running under
This would cause all core files to get created in /tmp/ (or /opt/zimbra/data/tmp/). Obviously, this can be changed to any other directory as well. The default is kernel.core_pattern=core, which means it gets created in whatever the current working directory was at the time the process started, which is less useful, as that could be anywhere on the system and may not be writable by the executing process.
Note: This setting gets overridden on RHEL6/CentOS6 by default if ABRT is installed.
In general, these parameters should be set in /etc/sysctl.conf. After the modifications are finished, the command
/sbin/sysctl -p /etc/sysctl.conf
can be executed so that a reboot is not necessary (although some OSes require a reboot, see below).
See notes for Ubuntu14 and later below, however.
Ubuntu 14 and later
Unfortunately, generating usable core files on Ubuntu requires disabling the apport process. This is likely not a big deal unless you want various crashes reported upstream to Ubuntu. To do this, edit /etc/default/apport, and set enabled to 0, then stop the services:
service apport stop
Create /etc/sysctl.d/60-zimbra-core.conf with the following contents:
kernel.core_uses_pid=1 fs.suid_dumpable=2 kernel.core_pattern=/opt/zimbra/data/tmp/core-%e-%s-%u-%g-%p-%t
service procps start
REDHAT derivative OSes only
echo "DAEMON_COREFILE_LIMIT='unlimited'" >> /etc/sysconfig/init
then reboot the system.
On earlier RedHat derivative OSes (RHEL4, RHEL5, F7, CentOS) also need to have a modification made to the startup scripts. By default, those OSes hard set the coresize to zero in /etc/profile. To fix this, you need to find the following line in /etc/profile:
ulimit -S -c 0 > /dev/null 2>&1
and change it to
ulimit -S -c unlimited > /dev/null 2>&1
and then reboot the system.
Core dumps are disabled by default under Darwin/Mac OS X. To re-enable core dumps, a privileged user must edit /etc/hostconfig to contain the line: