OpenLDAP Performance Tuning 6.0

Revision as of 17:23, 2 September 2009 by Quanah (talk | contribs)


Zimbra OpenLDAP Server

Currently only one master LDAP server can be set up; this server is authoritative for user information, server configuration, etc. Replica LDAP servers can be defined to improve performance and to reduce the load on the master server. All updates are made to the master server and these updates are copied to the replica servers.

Upcoming on radar: RFE 27872 - Support multi-master LDAP deployment

For best performance, the following settings should be modified. These settings apply to both the master LDAP server and the replica servers.

  • Set the threads count to an appropriate level. The ZCS default is 8, which is fine for 1 or 2 CPU servers. The general rule of thumb is 4 threads for every real core.
zmlocalconfig -e ldap_common_threads=8
  • Set the toolthreads count to an appropriate level. The ZCS default is 1. It should be set to the number of real cores you have available. The purpose of the toolthreads setting is to decrease the amount of time it takes to slapadd a database.
zmlocalconfig -e ldap_common_toolthreads=4
  • Change the entry cachesize for the primary database. The number set should be the number of configured active accounts and the number of configured active domains. The default is 10000.
zmlocalconfig -e ldap_db_cachesize=50000
  • Set idlcachesize for the primary database. The number set should be the same as the cachesize setting.
zmlocalconfig -e ldap_db_idlcachesize=50000
  • Change the dn entry cachseize for the primary database. It is highly recommended to leave it at the default value of zero, which means an unlimited DN cache. If it is set to a non-zero value, it should be no less than two times the entry cache.
zmlocalconfig -e ldap_db_dncachesize=0

If you have more than 100 domains, we suggest adjusting the following local config setting:

  • ldap_cache_domain_maxsize. This sets the cache of the number of domains in the server. The default is 100. If more than 100 domains are configured, you should adjust this to the lower of the number of domains you have configured and 30,000. For example, with 45,000 domains, set as this to 30000.
# Apply this to all mailbox servers!
zmlocalconfig -e ldap_cache_domain_maxsize=30000

Configuring the Berkeley DB subsystem to increase LDAP server performance

Berkeley DB is the underlying high-performance transactional database used to store the LDAP data. Proper configuration of this database is essential to maintaining a performant LDAP service. There are several parameters involved in tuning the underlying Berkeley DB database.

Berkeley DB and Shared memory keys

Starting with ZCS 6.0 and later, it is possible to instruct slapd to allocate a shared memory key for usage with the Berkeley DB cache backing files. Prior to ZCS 6.0, a disk-based backing cache was always used. Using a shared memory key rather than using a disk-based cache shows a 3x improvement over the disk-based method IF and ONLY IF the size of your database is over 8GB, and results in the same amount of memory usage as the disk-based backing cache. To configure the LDAP server to use a shared memory key takes multiple steps. You will need to use the value of your BDB cachesize (See Berkeley DB DB_CONFIG section below) to set the correct values.

  • Get the current configured pagesize for the system
getconf PAGE_SIZE
  • Divide the amount of RAM you will need for your shared memory key by the value of PAGE_SIZE, and assign this to the kernel.shmall sysctl value. For example, if I have a 14GB BDB cachesize and the PAGE_SIZE result is 4096, the setting would be 3801088. Which is 14.5*1024*1024*1024/4096. The extra .5 GB is for other OS processes that will also need to use shared memory.
# For Linux add to /etc/sysctl.conf:
kernel.shmall=3801088
# Then update the currently used value:
/sbin/sysctl -e kernel.shmall=3801088

# For OSX 10.5 add to/etc/sysctl.conf:
kern.sysv.shmall=3801088
# Then update the currently used value:
/usr/sbin/sysctl -w kern.sysv.shmall=3801088

# For OSX 10.4, modify the current listed value for kernel.sysv.shmall in the /etc/rc file
# if necessary.  The line will look similar to:
#sysctl -w kern.sysv.shmmax=4194304 kern.sysv.shmmin=1 kern.sysv.shmmni=32 kern.sysv.shmseg=8 kern.sysv.shmall=3801088
  • Configure the maximum size allowed for an individual shared memory key. For example, if I have a 14GB BDB cachesize, the value would be 15032385536 (14*1024*1024*1024).
# For Linux add to /etc/sysctl.conf:
kernel.shmmax=15032385536
# Then update the currently used value:
/sbin/sysctl -e kernel.shmmax=15032385536

# For OSX 10.5 add to /etc/sysctl.conf:
kernel.sysv.shmmax=15032385536
# Then update the currently used value:
/usr/sbin/sysctl -w kern.sysv.shmmax=15032385536

# For OSX 10.4, modify the current listed value for kernel.sysv.shmmax in the /etc/rc file
# if necessary.  The line will look similar to:
#sysctl -w kern.sysv.shmmax=15032385536 kern.sysv.shmmin=1 kern.sysv.shmmni=32 kern.sysv.shmseg=8 kern.sysv.shmall=3801088
# Also on OSX 10.4, you will now want to run your fully modified sysctl line:
/usr/sbin/sysctl -w kern.sysv.shmmax=15032385536 kern.sysv.shmmin=1 kern.sysv.shmmni=32 kern.sysv.shmseg=8 kern.sysv.shmall=3801088
  • For all OSes, as the zimbra user, you will now need to tell slapd to use a shared memory key by setting it in localconfig. slapd and zmmtaconfig must both be running. On a master, it is also possible to tell the accesslog database to use a shared memory key. If a shared memory key is going to be used for both the accesslog and main db, then the shmmax value needs to be the larger of the two BDB cachesizes (this will most likely always be the main db value), and the shmall value needs to be the total of the two databases BDB cachesizes plus .5. So, if I had an 8 GB main DB and a 2GB accesslog DB cachesize, I would want to use 10.5 as the value for determining shmall size. Shared memory key values must be unique inside slapd, and they can be set to any number. For example, to set the main DB to use a shared memory key with the value 5:
zmlocalconfig -e ldap_db_shmkey=5
  • If it was then desired to set the accesslog DB to use a shared memory key, it would need a different unique value:
zmlocalconfig -e ldap_accesslog_shmkey=7
  • Wait several minutes (it can take up to at least 5 minutes for zmmtaconfig to react to the change) and verify that the modification has occurred in the LDAP configuration database. The value for olcDbShmKey should match what you set ldap_db_shmkey to be.
# On a replica or stand-alone server
ldapsearch -LLL -s base-x -H ldapi:/// -D cn=config -w `zmlocalconfig -s -m nokey ldap_root_password` -b olcDatabase={2}hdb,cn=config olcDbShmKey

# On a master with replication enabled
ldapsearch -LLL -s base-x -H ldapi:/// -D cn=config -w `zmlocalconfig -s -m nokey ldap_root_password` -b olcDatabase={3}hdb,cn=config olcDbShmKey
  • To check the SHM key value for the accesslog database on a master:
# On a master with replication enabled, to check the accesslog DB setting:
ldapsearch -LLL -s base-x -H ldapi:/// -D cn=config -w `zmlocalconfig -s -m nokey ldap_root_password` -b olcDatabase={2}hdb,cn=config olcDbShmKey
  • Once you have verified the settings have been applied, restart slapd as the zimbra user
 ldap stop
 ldap start
Retrieved from "http://wiki.zimbra.com/index.php?title=OpenLDAP_Performance_Tuning_6.0&oldid=14777"
Jump to: navigation, search