View source for OpenLDAP Performance Tuning 8.0

{{ZC}}{{Article Infobox|{{admin}}|{{ZCS 8.0}}|||}}=Zimbra OpenLDAP Server=

As of 8.0.0, multiple master LDAP servers can be set up; these servers are authoritative for user information, server configuration, etc.  Zimbra recommends having no more than 4 master servers deployed. Replica LDAP servers can be defined to improve performance and to reduce the load on the master servers. All updates are made to the master servers and these updates are then replicated to the replica servers.

See also the documentation on configuring [http://wiki.zimbra.com/wiki/LDAP_Multi_Master_Replication Multiple Masters].

You may also wish to read over the entire set of keys available on the [http://wiki.zimbra.com/wiki/OpenLDAP_Tuning_Keys_8.0 OpenLDAP Tuning Keys Wiki].

== OpenLDAP Server Tuning ==
For best performance, the following settings should be modified.  These settings apply to both the master LDAP server and the replica servers.

* Set the '''threads''' count to an appropriate level. The ZCS default is 8, which is fine for 1 or 2 CPU servers.  The general rule of thumb is 4 threads for every real core.

 zmlocalconfig -e ldap_common_threads=8

* Set the '''toolthreads''' count to an appropriate level.  The ZCS default is 2.  It should be set to 2.  The purpose of the toolthreads setting is to decrease the amount of time it takes to slapadd a database.

 zmlocalconfig -e ldap_common_toolthreads=2

=== Notes on MDB ===
Starting with ZCS 8, Zimbra has moved to using the new MDB backend for OpenLDAP.  More information on MDB in general can be found at [http://www.symas.com/mdb/ Symas' MDB information page]. ZCS previously used the HDB backend with BerkeleyDB as the underlying database software.  Switching to MDB results in an increase in both read and write performance for the LDAP server over previous ZCS releases.  In addition, there are no performance related tuning parameters for MDB.  It simply works out of the box.  You can find performance metric differences for [http://wiki.zimbra.com/wiki/OpenLDAP_MDB_vs_HDB_performance MDB vs HDB with Zimbra here].

==== Maximum database size ====
There is one parameter for MDB that deployments may wish to adjust, which is the maximum database size.  The default is 80GB on ZCS installations.  When ZCS is installed, a [http://en.wikipedia.org/wiki/Sparse_file sparse file] will be created that will be appear to be the same size as the DB maxsize.  I.e., there will be a "data.mdb" file on disk that appears to be 80GB in size.  In fact, it will only be the size of your actual database.  For example:

 zimbra@zre-ldap002:~/data/ldap/mdb/db$ ls -l
 total 700
 -rw------- 1 zimbra zimbra 85899345920 Nov 29 12:18 data.mdb
 -rw------- 1 zimbra zimbra        8192 Nov 29 13:03 lock.mdb
 zimbra@zre-ldap002:~/data/ldap/mdb/db$ du -c -h data.mdb
 696K    data.mdb
 696K    total

So the actual size of data.mdb is 696KB, although it appears to be 80GB.  The maxsize setting for the database must always be greater than the total size of the LDAP DB or else writes will be rejected, slapd will stop, and data loss may occur.  If it is desired to reduce the maxsize of the database to trim down the size of the sparse file, this can be controlled via the following two localconfig keys:

 ldap_db_maxsize
 ldap_accesslog_maxsize

* The first key is for the primary LDAP DB (/opt/zimbra/data/ldap/mdb/db)
* The second key is for the accesslog DB for LDAP masters (/opt/zimbra/data/ldap/accesslog/db)

Keep in mind that in addition to the size of the data in the database it also needs room for page commits and expansion.  If this value is changed to something less than the default of 80GB, careful monitoring of the size of "data.mdb" compared to the maximum size should be implemented.

====automated database monitoring====
With ZCS 8.0.4 and later, there is a cron job named zmldapmonitordb that runs on the LDAP servers to monitor the size of the MDB databases.  By default, it will simply alert the admin account if the MDB database is close to running out of space vs its configured max size.  It can be configured to take preemptive action if so desired.  There is no need to restart services if one of the following localconfig keys are changed.  The related localconfig keys which can be modified via '''zmlocalconfig''' are:

* ldap_monitor_mdb (default true): Whether or not to monitor the LDAP MDB databases.
* ldap_monitor_alert_only (default true): If true, only generate email alerts.  If false, actively adjust the MDB database sizes if possible, and send out an email noting the actions taken.
* ldap_monitor_warning (default 80): The percentage of used space at which to issue a warning alert (and if ldap_monitor_alert_only is false, adjust the MDB maxsize upwards)
* ldap_monitor_critical (default 90): The percentage of used space at which to issue a critical alert (and if ldap_monitor_alert_only is false, adjust the MDB maxsize upwards)
* ldap_monitor_growth (default 25): The percentage of new space to allocate to the MDB database if an alert is triggered and ldap_monitor_alert_only is false

====mdb_stat utility====
The mdb_stat utility ships with ZCS 8.0.2 and later.  It allows the gathering of various information about the state of the MDB database via the command line.  Available options are

* -e: prints database environment information
* -f: prints the freelist information
* -a: prints out the information for all sub-databases (indexes and entry database)
* -s '''sub database''': prints out the information specific to the sub database specified

Format is:
 <code>
 mdb_stat [-e] [-f] [-a|-s subdb]/path/to/database
 </code>

Examples:
 <code>
 mdb_stat -f /opt/zimbra/data/ldap/mdb/db
 mdb_stat -a -e -f /opt/zimbra/data/ldap/mdb
 mdb_stat -s id2e /opt/zimbra/data/ldap/mdb
 </code>

====mdb_copy utility====
The mdb_copy utility ships with ZCS 8.0.2 and later.  It allows one to safely copy an MDB database from one location to another via the command line.  This utility can be used while slapd is running.

Format is:
 mdb_copy /src/dir /destination/dir

Example:
 <code>
 mdb_copy /opt/zimbra/data/ldap/mdb/db /opt/zimbra/backups/ldap/mdb/db
 </code>

Note: The copied MDB file will *not* be a sparse file.  It will be the actual size of the database.  I.e., if your databse is 25MB in size, the resulting copied database will be a full 25MB, and not an 80GB sparse file.

=== Mailbox store tuning with LDAP ===
If you have more than 100 domains, we suggest adjusting the following local config setting:

* '''ldap_cache_domain_maxsize'''. This sets the cache of the number of domains in the server. The default is 100. If more than 100 domains are configured, you should adjust this to the lower of the number of domains you have configured and 30,000. For example, with 45,000 domains, set this to 30000.

 # Apply this to all mailbox servers!
 zmlocalconfig -e ldap_cache_domain_maxsize=30000

{{Article_Footer|ZCS 8.0|7/20/2012}}

[[Category:LDAP]]
[[Category:Performance and Tuning]]
[[Category:ZCS 8.0]]