OpenLDAP Performance Tuning 8.0
|This article applies to the following ZCS versions.|
Zimbra OpenLDAP Server
As of 8.0.0, multiple master LDAP servers can be set up; these servers are authoritative for user information, server configuration, etc. Zimbra recommends having no more than 4 master servers deployed. Replica LDAP servers can be defined to improve performance and to reduce the load on the master servers. All updates are made to the master servers and these updates are then replicated to the replica servers.
See also the documentation on configuring Multiple Masters.
You may also wish to read over the entire set of keys available on the OpenLDAP Tuning Keys Wiki.
OpenLDAP Server Tuning
For best performance, the following settings should be modified. These settings apply to both the master LDAP server and the replica servers.
- Set the threads count to an appropriate level. The ZCS default is 8, which is fine for 1 or 2 CPU servers. The general rule of thumb is 4 threads for every real core.
zmlocalconfig -e ldap_common_threads=8
- Set the toolthreads count to an appropriate level. The ZCS default is 2. It should be set to 2. The purpose of the toolthreads setting is to decrease the amount of time it takes to slapadd a database.
zmlocalconfig -e ldap_common_toolthreads=2
Notes on MDB
Starting with ZCS 8, Zimbra has moved to using the new MDB backend for OpenLDAP. More information on MDB in general can be found at Symas' MDB information page. ZCS previously used the HDB backend with BerkeleyDB as the underlying database software. Switching to MDB results in an increase in both read and write performance for the LDAP server over previous ZCS releases. In addition, there are no performance related tuning parameters for MDB. It simply works out of the box. You can find performance metric differences for MDB vs HDB with Zimbra here.
Kernel tuning options
If a very large database is in use (3 million users and more), it may be desirable to tune the kernel to better handle the MDB throughput.
Filesystem mount options
- If ext3 or ext4 is in use, make sure the partition is mounted with either noatime or relatime
- If ext3 or ext4 is in use, make sure to use writeback as the journal mode: data=writeback
- It is advised to set the dirty page ratio to 90%:
echo "vm.dirty_ratio = 90" >>/etc/sysctl.conf
- It is advised to change the flush rate for writing out data:
echo "vm.dirty_expire_centisecs = 60000" >>/etc/sysctl.conf
- Update sysctl:
/sbin/sysctl -p /etc/sysctl.conf
Maximum database size
There is one parameter for MDB that deployments may wish to adjust, which is the maximum database size. The default is 80GB on ZCS installations. When ZCS is installed, a sparse file will be created that will be appear to be the same size as the DB maxsize. I.e., there will be a "data.mdb" file on disk that appears to be 80GB in size. In fact, it will only be the size of your actual database. For example:
zimbra@zre-ldap002:~/data/ldap/mdb/db$ ls -l total 700 -rw------- 1 zimbra zimbra 85899345920 Nov 29 12:18 data.mdb -rw------- 1 zimbra zimbra 8192 Nov 29 13:03 lock.mdb zimbra@zre-ldap002:~/data/ldap/mdb/db$ du -c -h data.mdb 696K data.mdb 696K total
So the actual size of data.mdb is 696KB, although it appears to be 80GB. The maxsize setting for the database must always be greater than the total size of the LDAP DB or else writes will be rejected, slapd will stop, and data loss may occur. If it is desired to reduce the maxsize of the database to trim down the size of the sparse file, this can be controlled via the following two localconfig keys:
- The first key is for the primary LDAP DB (/opt/zimbra/data/ldap/mdb/db)
- The second key is for the accesslog DB for LDAP masters (/opt/zimbra/data/ldap/accesslog/db)
Keep in mind that in addition to the size of the data in the database it also needs room for page commits and expansion. If this value is changed to something less than the default of 80GB, careful monitoring of the size of "data.mdb" compared to the maximum size should be implemented.
automated database monitoring
With ZCS 8.0.4 and later, there is a cron job named zmldapmonitordb that runs on the LDAP servers to monitor the size of the MDB databases. By default, it will simply alert the admin account if the MDB database is close to running out of space vs its configured max size. It can be configured to take preemptive action if so desired. There is no need to restart services if one of the following localconfig keys are changed. The related localconfig keys which can be modified via zmlocalconfig are:
- ldap_monitor_mdb (default true): Whether or not to monitor the LDAP MDB databases.
- ldap_monitor_alert_only (default true): If true, only generate email alerts. If false, actively adjust the MDB database sizes if possible, and send out an email noting the actions taken.
- ldap_monitor_warning (default 80): The percentage of used space at which to issue a warning alert (and if ldap_monitor_alert_only is false, adjust the MDB maxsize upwards)
- ldap_monitor_critical (default 90): The percentage of used space at which to issue a critical alert (and if ldap_monitor_alert_only is false, adjust the MDB maxsize upwards)
- ldap_monitor_growth (default 25): The percentage of new space to allocate to the MDB database if an alert is triggered and ldap_monitor_alert_only is false
The mdb_stat utility ships with ZCS 8.0.2 and later. It allows the gathering of various information about the state of the MDB database via the command line. Available options are
- -e: prints database environment information
- -f: prints the freelist information
- -a: prints out the information for all sub-databases (indexes and entry database)
- -s sub database: prints out the information specific to the sub database specified
mdb_stat [-e] [-f] [-a|-s subdb]/path/to/database
mdb_stat -f /opt/zimbra/data/ldap/mdb/db mdb_stat -a -e -f /opt/zimbra/data/ldap/mdb mdb_stat -s id2e /opt/zimbra/data/ldap/mdb
The mdb_copy utility ships with ZCS 8.0.2 and later. It allows one to safely copy an MDB database from one location to another via the command line. This utility can be used while slapd is running.
mdb_copy /src/dir /destination/dir
mdb_copy /opt/zimbra/data/ldap/mdb/db /opt/zimbra/backups/ldap/mdb/db
Note: The copied MDB file will *not* be a sparse file. It will be the actual size of the database. I.e., if your databse is 25MB in size, the resulting copied database will be a full 25MB, and not an 80GB sparse file.
Mailbox store tuning with LDAP
If you have more than 100 domains, we suggest adjusting the following local config setting:
- ldap_cache_domain_maxsize. This sets the cache of the number of domains in the server. The default is 100. If more than 100 domains are configured, you should adjust this to the lower of the number of domains you have configured and 30,000. For example, with 45,000 domains, set as this to 30000.
# Apply this to all mailbox servers! zmlocalconfig -e ldap_cache_domain_maxsize=30000