OpenLDAP Performance Tuning 8.0

From Zimbra :: Wiki

Jump to: navigation, search
   Article-check.png  - This is certified documentation and is protected for editing by Zimbra Employees & Moderators only.


Admin Article

Article Information

This article applies to the following ZCS versions.
  ZCS 8.0 Article  ZCS 8.0

Contents

Zimbra OpenLDAP Server

As of 8.0.0, multiple master LDAP servers can be set up; these servers are authoritative for user information, server configuration, etc. Zimbra recommends having no more than 4 master servers deployed. Replica LDAP servers can be defined to improve performance and to reduce the load on the master servers. All updates are made to the master servers and these updates are then replicated to the replica servers.

See also the documentation on configuring Multiple Masters.

You may also wish to read over the entire set of keys available on the OpenLDAP Tuning Keys Wiki.

OpenLDAP Server Tuning

For best performance, the following settings should be modified. These settings apply to both the master LDAP server and the replica servers.

  • Set the threads count to an appropriate level. The ZCS default is 8, which is fine for servers with up to 8 real cores. The general rule of thumb is one thread per real core.
zmlocalconfig -e ldap_common_threads=8
  • Set the toolthreads count to an appropriate level. The ZCS default is 2. It should be set to 2. The purpose of the toolthreads setting is to decrease the amount of time it takes to slapadd a database.
zmlocalconfig -e ldap_common_toolthreads=2

Notes on MDB

Starting with ZCS 8, Zimbra has moved to using the new MDB backend for OpenLDAP. More information on MDB in general can be found at Symas' MDB information page. ZCS previously used the HDB backend with BerkeleyDB as the underlying database software. Switching to MDB results in an increase in both read and write performance for the LDAP server over previous ZCS releases. In addition, there are no performance related tuning parameters for MDB. It simply works out of the box. You can find performance metric differences for MDB vs HDB with Zimbra here.

Maximum database size

There is one parameter for MDB that deployments may wish to adjust, which is the maximum database size. The default is 80GB on ZCS installations. When ZCS is installed, a sparse file will be created that will be appear to be the same size as the DB maxsize. I.e., there will be a "data.mdb" file on disk that appears to be 80GB in size. In fact, it will only be the size of your actual database. For example:

zimbra@zre-ldap002:~/data/ldap/mdb/db$ ls -l
total 700
-rw------- 1 zimbra zimbra 85899345920 Nov 29 12:18 data.mdb
-rw------- 1 zimbra zimbra        8192 Nov 29 13:03 lock.mdb
zimbra@zre-ldap002:~/data/ldap/mdb/db$ du -c -h data.mdb
696K    data.mdb
696K    total

So the actual size of data.mdb is 696KB, although it appears to be 80GB. The maxsize setting for the database must always be greater than the total size of the LDAP DB or else writes will be rejected, slapd will stop, and data loss may occur. If it is desired to reduce the maxsize of the database to trim down the size of the sparse file, this can be controlled via the following two localconfig keys:

ldap_db_maxsize
ldap_accesslog_maxsize
  • The first key is for the primary LDAP DB (/opt/zimbra/data/ldap/mdb/db)
  • The second key is for the accesslog DB for LDAP masters (/opt/zimbra/data/ldap/accesslog/db)

Keep in mind that in addition to the size of the data in the database it also needs room for page commits and expansion. If this value is changed to something less than the default of 80GB, careful monitoring of the size of "data.mdb" compared to the maximum size should be implemented.

automated database monitoring

With ZCS 8.0.4 and later, there is a cron job named zmldapmonitordb that runs on the LDAP servers to monitor the size of the MDB databases. By default, it will simply alert the admin account if the MDB database is close to running out of space vs its configured max size. It can be configured to take preemptive action if so desired. There is no need to restart services if one of the following localconfig keys are changed. The related localconfig keys which can be modified via zmlocalconfig are:

  • ldap_monitor_mdb (default true): Whether or not to monitor the LDAP MDB databases.
  • ldap_monitor_alert_only (default true): If true, only generate email alerts. If false, actively adjust the MDB database sizes if possible, and send out an email noting the actions taken.
  • ldap_monitor_warning (default 80): The percentage of used space at which to issue a warning alert (and if ldap_monitor_alert_only is false, adjust the MDB maxsize upwards)
  • ldap_monitor_critical (default 90): The percentage of used space at which to issue a critical alert (and if ldap_monitor_alert_only is false, adjust the MDB maxsize upwards)
  • ldap_monitor_growth (default 25): The percentage of new space to allocate to the MDB database if an alert is triggered and ldap_monitor_alert_only is false

mdb_stat utility

The mdb_stat utility ships with ZCS 8.0.2 and later. It allows the gathering of various information about the state of the MDB database via the command line. Available options are

  • -e: prints database environment information
  • -f: prints the freelist information
  • -a: prints out the information for all sub-databases (indexes and entry database)
  • -s sub database: prints out the information specific to the sub database specified

Format is:


mdb_stat [-e] [-f] [-a|-s subdb]/path/to/database

Examples:


mdb_stat -f /opt/zimbra/data/ldap/mdb/db
mdb_stat -a -e -f /opt/zimbra/data/ldap/mdb/db
mdb_stat -s id2e /opt/zimbra/data/ldap/mdb/db

mdb_copy utility

The mdb_copy utility ships with ZCS 8.0.2 and later. It allows one to safely copy an MDB database from one location to another via the command line. This utility can be used while slapd is running.

Format is:

mdb_copy /src/dir /destination/dir

Example:


mdb_copy /opt/zimbra/data/ldap/mdb/db /opt/zimbra/backups/ldap/mdb/db

Note: The copied MDB file will *not* be a sparse file. It will be the actual size of the database. I.e., if your databse is 25MB in size, the resulting copied database will be a full 25MB, and not an 80GB sparse file.

Mailbox store tuning with LDAP

If you have more than 100 domains, we suggest adjusting the following local config setting:

  • ldap_cache_domain_maxsize. This sets the cache of the number of domains in the server. The default is 100. If more than 100 domains are configured, you should adjust this to the lower of the number of domains you have configured and 30,000. For example, with 45,000 domains, set this to 30000.
# Apply this to all mailbox servers!
zmlocalconfig -e ldap_cache_domain_maxsize=30000
Verified Against: ZCS 8.0 Date Created: 7/20/2012
Article ID: http://wiki.zimbra.com/index.php?title=OpenLDAP_Performance_Tuning_8.0 Date Modified: 05/2/2013
Personal tools