OpenLDAP Performance Tuning

From Zimbra :: Wiki

Jump to: navigation, search
   Article-check.png  - This is certified documentation and is protected for editing by Zimbra Employees & Moderators only.


Admin Article

Article Information

This article applies to the following ZCS versions.
  ZCS 7.0 Article  ZCS 7.0
  ZCS 6.0 Article  ZCS 6.0

Contents

Zimbra OpenLDAP Server

Currently only one master LDAP server can be set up; this server is authoritative for user information, server configuration, etc. Replica LDAP servers can be defined to improve performance and to reduce the load on the master server. All updates are made to the master server and these updates are copied to the replica servers.

For ZCS 8.0 and later, use OpenLDAP performance tuning for ZCS 8 instead.

Upcoming on radar: RFE 27872 - Support multi-master LDAP deployment

You may also wish to read over the entire set of keys available on the OpenLDAP Tuning Keys Wiki.

OpenLDAP Server Tuning

For best performance, the following settings should be modified. These settings apply to both the master LDAP server and the replica servers.

  • Set the threads count to an appropriate level. The ZCS default is 8, which is fine for 1 or 2 CPU servers. The general rule of thumb is 4 threads for every real core.
zmlocalconfig -e ldap_common_threads=8
  • Set the toolthreads count to an appropriate level. The ZCS default is 1. It should be set to the number of real cores you have available. The purpose of the toolthreads setting is to decrease the amount of time it takes to slapadd a database. WARNING: If you are on ZCS 8, this can ONLY be set to 2 at this time. Please read the ZCS8 Performance Tuning Wiki.
zmlocalconfig -e ldap_common_toolthreads=4
  • Change the entry cachesize for the primary database. The number set should be the number of configured active accounts and the number of configured active domains. The default is 10000.
zmlocalconfig -e ldap_db_cachesize=50000
  • Set idlcachesize for the primary database. The number set should optimally be three times the cachesize setting.
zmlocalconfig -e ldap_db_idlcachesize=50000
  • Change the dn entry cachseize for the primary database. It is highly recommended to leave it at the default value of zero, which means an unlimited DN cache. If it is set to a non-zero value, it should be no less than two times the entry cache.
zmlocalconfig -e ldap_db_dncachesize=0

If you have more than 100 domains, we suggest adjusting the following local config setting:

  • ldap_cache_domain_maxsize. This sets the cache of the number of domains in the server. The default is 100. If more than 100 domains are configured, you should adjust this to the lower of the number of domains you have configured and 30,000. For example, with 45,000 domains, set as this to 30000.
# Apply this to all mailbox servers!
zmlocalconfig -e ldap_cache_domain_maxsize=30000

Configuring the Berkeley DB subsystem to increase LDAP server performance

Berkeley DB is the underlying high-performance transactional database used to store the LDAP data. Proper configuration of this database is essential to maintaining a performant LDAP service. There are several parameters involved in tuning the underlying Berkeley DB database.

Berkeley DB and Shared memory keys

Starting with ZCS 6.0 and later, it is possible to instruct slapd to allocate a shared memory key for usage with the Berkeley DB cache backing files. Prior to ZCS 6.0, a disk-based backing cache was always used. Using a shared memory key rather than using a disk-based cache shows a 3x improvement over the disk-based method IF and ONLY IF the size of your database is over 8GB, and results in the same amount of memory usage as the disk-based backing cache. To configure the LDAP server to use a shared memory key takes multiple steps. You will need to use the value of your BDB cachesize (See Berkeley DB DB_CONFIG section below) to set the correct values.

  • Get the current configured pagesize for the system
getconf PAGE_SIZE
  • Divide the amount of RAM you will need for your shared memory key by the value of PAGE_SIZE, and assign this to the kernel.shmall sysctl value. For example, if I have a 14GB BDB cachesize and the PAGE_SIZE result is 4096, the setting would be 3801088. Which is 14.5*1024*1024*1024/4096. The extra .5 GB is for other OS processes that will also need to use shared memory.
# For Linux add to /etc/sysctl.conf:
kernel.shmall=3801088
# Then update the currently used value:
/sbin/sysctl -e kernel.shmall=3801088
# For OSX 10.5 add to/etc/sysctl.conf:
kern.sysv.shmall=3801088
# Then update the currently used value:
/usr/sbin/sysctl -w kern.sysv.shmall=3801088
# For OSX 10.4, modify the current listed value for kernel.sysv.shmall in the /etc/rc file
# if necessary.  The line will look similar to:
#sysctl -w kern.sysv.shmmax=4194304 kern.sysv.shmmin=1 kern.sysv.shmmni=32 kern.sysv.shmseg=8 kern.sysv.shmall=3801088
  • Configure the maximum size allowed for an individual shared memory key. For example, if I have a 14GB BDB cachesize, the value would be 15032385536 (14*1024*1024*1024).
# For Linux add to /etc/sysctl.conf:
kernel.shmmax=15032385536
# Then update the currently used value:
/sbin/sysctl -e kernel.shmmax=15032385536
# For OSX 10.5 add to /etc/sysctl.conf:
kernel.sysv.shmmax=15032385536
# Then update the currently used value:
/usr/sbin/sysctl -w kern.sysv.shmmax=15032385536
# For OSX 10.4, modify the current listed value for kernel.sysv.shmmax in the /etc/rc file
# if necessary.  The line will look similar to:
#sysctl -w kern.sysv.shmmax=15032385536 kern.sysv.shmmin=1 kern.sysv.shmmni=32 kern.sysv.shmseg=8 kern.sysv.shmall=3801088
# Also on OSX 10.4, you will now want to run your fully modified sysctl line:
/usr/sbin/sysctl -w kern.sysv.shmmax=15032385536 kern.sysv.shmmin=1 kern.sysv.shmmni=32 kern.sysv.shmseg=8 kern.sysv.shmall=3801088
  • For all OSes, as the zimbra user, you will now need to tell slapd to use a shared memory key by setting it in localconfig. slapd and zmmtaconfig must both be running. On a master, it is also possible to tell the accesslog database to use a shared memory key. If a shared memory key is going to be used for both the accesslog and main db, then the shmmax value needs to be the larger of the two BDB cachesizes (this will most likely always be the main db value), and the shmall value needs to be the total of the two databases BDB cachesizes plus .5. So, if I had an 8 GB main DB and a 2GB accesslog DB cachesize, I would want to use 10.5 as the value for determining shmall size. Shared memory key values must be unique inside slapd, and they can be set to any number. For example, to set the main DB to use a shared memory key with the value 5:
zmlocalconfig -e ldap_db_shmkey=5
  • If it was then desired to set the accesslog DB to use a shared memory key, it would need a different unique value:
zmlocalconfig -e ldap_accesslog_shmkey=7
  • Wait several minutes (it can take up to at least 5 minutes for zmmtaconfig to react to the change) and verify that the modification has occurred in the LDAP configuration database. The value for olcDbShmKey should match what you set ldap_db_shmkey to be.
# On a replica or stand-alone server
ldapsearch -LLL -s base-x -H ldapi:/// -D cn=config -w `zmlocalconfig -s -m nokey ldap_root_password` -b olcDatabase={2}hdb,cn=config olcDbShmKey

# On a master with replication enabled
ldapsearch -LLL -s base-x -H ldapi:/// -D cn=config -w `zmlocalconfig -s -m nokey ldap_root_password` -b olcDatabase={3}hdb,cn=config olcDbShmKey
  • To check the SHM key value for the accesslog database on a master:
# On a master with replication enabled, to check the accesslog DB setting:
ldapsearch -LLL -s base-x -H ldapi:/// -D cn=config -w `zmlocalconfig -s -m nokey ldap_root_password` -b olcDatabase={2}hdb,cn=config olcDbShmKey
  • Once you have verified the settings have been applied, restart slapd as the zimbra user
 ldap stop
 ldap start

Berkeley DB DB_CONFIG tuning

Most of the tuning for BDB involves editing the DB_CONFIG file. Modifications to the DB_CONFIG file require a restart of the LDAP server before they are picked up, and should be made to both master and replica servers. The proper way to do this for Zimbra is documented at Managing_Customized_BDB_configurations

You can increase LDAP server performance by adjusting the BDB backend cache size to be at or near the size of your data set. This is subject to the limit of 4 GB for 32 bit and 10 TB for 64 bit, and the amount of RAM you have. The size of the data set is the sum of the Berkeley DataBase (BDB) files in /opt/zimbra/data/ldap/hdb/db. To increase the cache size, add (or replace) the following line to the DB_CONFIG file in /opt/zimbra/data/ldap/hdb/db. The following would set the database in-memory cachesize to 500MB:

set_cachesize 0 524288000 1

The format for the set_cachesize command is <gigabytes> <bytes> <segments>

It is possible to check that the cache setting has taken effect by using the /opt/zimbra/bdb/bin/db_stat -m -h /opt/zimbra/data/ldap/hdb/dbǀhead -n 16 command can be used to see the current cache setting, as well as to find other important information. Example output:

500MB Total cache size
1 Number of caches
1 Maximum number of caches
500MB Pool individual cache size
0 Maximum memory-mapped file size
0 Maximum open file descriptors
0 Maximum sequential buffer writes
0 Sleep after writing maximum sequential buffers
0 Requested pages mapped into the process' address space
3437M Requested pages found in the cache (100%)
526125 Requested pages not found in the cache
47822 Pages created in the cache
526125 Pages read into the cache
27M Pages written from the cache to the backing file
0 Clean pages forced from the cache
0 Dirty pages forced from the cache

The above output shows that there is a 500MB total cache in a single segment all of which is allocated to the cache pool. The other important data to evaluate are the Requested pages found in the cache, the Clean pages forced from the cache and the Dirty pages forced from the cache. For optimal performance, the Requested pages found in the cache should be above 95%, and the pages forced from the cache should be 0.

As part of the transaction interface, BDB uses a number of locks, lockers, and lock objects. The default value for each of these parameters is 1000. How many of each are being used depends on the number of entries and indices in the BDB database. The /opt/zimbra/bdb/bin/db_stat -c -h /opt/zimbra/data/ldap/hdb/dbǀhead -n 16 command can be used to determine current usage:

5634 Last allocated locker ID
0x7fffffff Current maximum unused locker ID
9 Number of lock modes
3000 Maximum number of locks possible
1500 Maximum number of lockers possible
1500 Maximum number of lock objects possible
93 Number of lock object partitions
93 Number of current locks
1921 Maximum number of locks at any one time
6 Maximum number of locks in any one bucket
0 Maximum number of locks stolen by for an empty partition
0 Maximum number of locks stolen for any one partition
483 Number of current lockers
485 Maximum number of lockers at any one time
93 Number of current lock objects
1011 Maximum number of lock objects at any one time

The above output shows that there are a maximum of 3000 locks, 1500 lockers, and 1500 lock objects available to the BDB database located in /opt/zimbra/data/ldap/hdb/db. Of those settings, there are currently 93 locks in use, 483 lockers in use, and 93 lock objects in use. Over the course of the lifetime of the database, the highest recorded values have been 1921 locks used, 485 lockers used, and 1011 lock objects used. As long as usage is within 85% of the configured value(s), it should not be necessary to modify the settings.

The following entries in DB_CONFIG would increase the number of locks to 3000, the lock objects to 1500, and the lockers to 1500:

set_lk_max_locks 3000
set_lk_max_objects 1500
set_lk_max_lockers 1500
Verified Against: ZCS 7.0, ZCS 6.0 Date Created: 9/3/2009
Article ID: http://wiki.zimbra.com/index.php?title=OpenLDAP_Performance_Tuning Date Modified: 01/10/2013
Personal tools