How-to for cbpolicyd: Difference between revisions
No edit summary |
|||
Line 258: | Line 258: | ||
0 * * * * cat /opt/zimbra/log/scripts/clean_cbpolicyd_daily.sql | | 0 * * * * cat /opt/zimbra/log/scripts/clean_cbpolicyd_daily.sql | | ||
sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb | sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb | ||
=== VACUUM the cbpolicyd.sqlitedb === | |||
The sqlite3 "VACUUM" command rebuilds the database and can significantly decrease the amount of space consumed on the filesystem by repacking the DB. Read more about the VACUUM command here: https://sqlite.org/lang_vacuum.html | |||
The first time you run this may require quite a bit of time. Make a backup of the file to be cautious: | |||
# su - zimbra | |||
$ cp -pr /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb.bak # get backup | |||
$ ls -ld /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb # get current size | |||
$ /usr/bin/sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb 'vacuum;' | |||
$ ls -ld /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb # get new size | |||
One can then delete the backup if comfortable everything running ok. | |||
A once a week vacuum command can be run out of the "zimbra" user crontab, like the following: | |||
# su - zimbra | |||
$ crontab -e | |||
And enter a weekly crontab like the following: | |||
0 1 * * 0 /usr/bin/sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb 'vacuum;' | |||
[[Category:policyd]] | [[Category:policyd]] | ||
{{Article Footer|Zimbra Collaboration 8.0, 7.0|04/16/2014}} | {{Article Footer|Zimbra Collaboration 8.0, 7.0|04/16/2014}} |
Revision as of 19:03, 26 June 2017
How-to for CBPolicyd
See Also
Introduction
- Packaged as part of the zimbra-mta package
- Configured to run as a daemon on 127.0.0.1:10031
- Integrated with postfix as a check_policy_service
- Cbpolicyd database uses sqlite
- Default configuration enables quotas module
Enable/Disable
To enable :
zmprov ms <mta server> +zimbraServiceEnabled cbpolicyd
The following changes get written by zmconfigd to postconf :
smtpd_end_of_data_restrictions = check_policy_service inet:localhost:10031 smtpd_recipient_restrictions = check_policy_service inet:localhost:10031, reject_non_fqdn_recipient, reject_unlisted_recipient, reject_invalid_helo_hostname, reject_non_fqdn_sender, permit
When the service is enabled the following is logged in /opt/zimbra/log/cbpolicyd.log
[2013/04/02-02:15:39 - 8686] [CORE] NOTICE: 2013/04/02-02:15:39 cbp (type Net::Server::PreFork) starting! pid(8686) [2013/04/02-02:15:39 - 8686] [CORE] NOTICE: Resolved [localhost]:10031 to [::1]:10031, IPv6 [2013/04/02-02:15:39 - 8686] [CORE] NOTICE: Resolved [localhost]:10031 to [127.0.0.1]:10031, IPv4 [2013/04/02-02:15:39 - 8686] [CORE] NOTICE: Binding to TCP port 10031 on host ::1 with IPv6 [2013/04/02-02:15:39 - 8686] [CORE] NOTICE: Binding to TCP port 10031 on host 127.0.0.1 with IPv4 [2013/04/02-02:15:39 - 8686] [CORE] NOTICE: Setting gid to "501 501" [2013/04/02-02:15:39 - 8686] [CORE] INFO: Setting up serialization via flock [2013/04/02-02:15:39 - 8686] [CORE] INFO: Beginning prefork (4 processes) [2013/04/02-02:15:39 - 8686] [CORE] INFO: Starting "4" children
Process :
0:00 /usr/bin/perl /opt/zimbra/cbpolicyd/bin/cbpolicyd --config /opt/zimbra/conf/cbpolicyd.conf
To Disable :
zmprov ms <mta server> -zimbraServiceEnabled cbpolicyd
- Note: Ideally zmconfigd should be removing the postconf entry. If it does not then remove run zmcontrol stop;zmcontrol start. This will force a rewrite of the postfix configuration files.
Config file
Config file for the cbpolicyd process :
/opt/zimbra/conf/cbpolicyd.conf.in gets rewritten by configd to /opt/zimbra/conf/cbpolicyd.conf
Configuration of cbpolicyd daemon via localconfig :
# default log, pid and database files cbpolicyd_pid_file cbpolicyd_log_file cbpolicyd_db_file # policyd log_level default is 3 for info,notices,warnings and errors cbpolicyd_log_level=3 [0-4] # use cbpolicyd_log_file (main) or syslog (maillog) for logging cbpolicyd_log_mail=main [main|maillog] # components of policyd to log data cbpolicyd_log_detail=modules [modules,tracking,policies,protocols] # bind host/port default is 127.0.0.1:10031 cbpolicyd_bind_host=127.0.0.1 cbpolicyd_bind_port=10031
# timeout in communication with clients. ZCS7, ZCS8 < 8.0.4 cbpolicyd_timeout=120 # idle timeout with. ZCS 8.0.4 and higher cbpolicyd_timeout_idle=1020
# Busy timeout. ZCS 8.0.4 and higher cbpolicyd_timeout_busy=120
# how many seconds before retrying database connection cbpolicyd_bypass_timeout=30
# what to do when there is a database connection problem cbpolicyd_bypass_mode=tempfail [tempfail|pass] # enable/disable specific policyd modules cbpolicyd_module_accesscontrol=0 [0|1] cbpolicyd_module_accounting=0 [0|1] ZCS8.0.4 and higher cbpolicyd_module_amavis=0 [0|1] ZCS8.0.4 and higher cbpolicyd_module_greylisting=0 [0|1] cbpolicyd_module_greylisting_training=0 [0|1] ZCS8.0.4 and higher. If enabled, greylisting is in training mode only (does not actually do greylisting) cbpolicyd_module_greylisting_defer_msg ZCS8.0.4 and higher. Message to log for deferred mail cbpolicyd_module_greylisting_blacklist_msg ZCS8.0.4 and higher. Message to log for blacklisted mail cbpolicyd_module_checkhelo=0 [0|1] cbpolicyd_module_checkspf=0 [0|1] cbpolicyd_module_quotas=1 [0|1]
Default config values :
$ zmlocalconfig | grep -i cbpolicy cbpolicyd_bind_port = 10031 cbpolicyd_bypass_mode = tempfail cbpolicyd_bypass_timeout = 30 cbpolicyd_cache_file = ${zimbra_home}/data/cache cbpolicyd_db_file = ${zimbra_home}/data/cbpolicyd/db/cbpolicyd.sqlitedb cbpolicyd_log_detail = modules cbpolicyd_log_file = ${zimbra_log_directory}/cbpolicyd.log cbpolicyd_log_level = 4 cbpolicyd_log_mail = main cbpolicyd_module_accesscontrol = 0 cbpolicyd_module_checkhelo = 0 cbpolicyd_module_checkspf = 0 cbpolicyd_module_greylisting = 0 cbpolicyd_module_quotas = 1 cbpolicyd_pid_file = ${zimbra_log_directory}/cbpolicyd.pid cbpolicyd_timeout = 120
Logging
Log file for Cbpolicyd :
log_file=/opt/zimbra/log/cbpolicyd.log
Log levels can be controlled by LC - "cbpolicyd_log_level", the value can be set to one of the below :
# Log level: # 0 - Errors only # 1 - Warnings and errors # 2 - Notices, warnings, errors # 3 - Info, notices, warnings, errors # 4 - Debugging
To log in further detail the below can be set, controlling LC attribute is "cbpolicyd_log_detail"
# modules - Log detailed module running information # tracking - Log detailed tracking information # policies - Log policy resolution # protocols - Log general protocol info, but detailed # bizanga - Log the bizanga protocol # # There is no default for this configuration option. Options can be # separated by commas. ie. protocols,modules # log_detail=modules
Database Design
To access the policyd db :
sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb
List of tables :
sqlite> .tables access_control greylisting_whitelist checkhelo policies checkhelo_blacklist policy_group_members checkhelo_tracking policy_groups checkhelo_whitelist policy_members checkspf quotas greylisting quotas_limits greylisting_autoblacklist quotas_tracking greylisting_autowhitelist session_tracking greylisting_tracking
Master table, to keep count of the policies :
sqlite> select * from sqlite_sequence; policies|5 policy_members|5 policy_groups|2 policy_group_members|3 quotas|2 quotas_limits|3 checkhelo_blacklist|4
Default policy:
sqlite> select * from quotas; 1|5|Recipient quotas|Recipient:user@domain|3600|REJECT||0||0 2|5|Quota on all /24s|SenderIP:/24|3600|REJECT||0||0 * 1st Column is the identifier, sort of like "id" of the policy * 5th Column is the time interval for the policy. Explained in the example below.
Quota limits for the above policies :
sqlite> select * from quotas_limits; 1|1|MessageCount|10||0 2|1|MessageCumulativeSize|8000||0 3|2|MessageCount|12||0 * 2nd Columne is the same identifier as referenced in table "quotas". * 4th Column is the number of "rate limit". For example "12" is the number of hits coming from "SenderIP" in a period of 3600 seconds.
Example
The below example adds two policies :
* Rate limit any sender from sending more then 20 emails every 60 seconds. Messages beyond this limit are deferred. * Rate limit any @domain from receiving more then 50 emails in a 60 second period. Messages beyond this rate are rejected.
Example sql file, to be imported into "cbpolicyd.sqlitedb" database
BEGIN TRANSACTION; INSERT INTO "policies" VALUES(6, 'Zimbra', 0, 'Zimbra QA Test Policy', 0); DELETE FROM sqlite_sequence; INSERT INTO "sqlite_sequence" VALUES('policies', 6); INSERT INTO "sqlite_sequence" VALUES('policy_members', 6); INSERT INTO "sqlite_sequence" VALUES('policy_groups', 2); INSERT INTO "sqlite_sequence" VALUES('policy_group_members', 3); INSERT INTO "sqlite_sequence" VALUES('quotas', 4); INSERT INTO "sqlite_sequence" VALUES('quotas_limits', 5); INSERT INTO "sqlite_sequence" VALUES('checkhelo_blacklist', 4); INSERT INTO "policy_members" VALUES(6, 6, 'any', 'any', , 0); INSERT INTO "quotas" VALUES(3, 6, 'Sender:user@domain','Sender:user@domain', 60, 'DEFER', 'Deferring: Too many messages from sender in last 60', , 0); INSERT INTO "quotas" VALUES(4, 6, 'Recipient:@domain', 'Recipient:@domain', 60, 'REJECT', , , 0); INSERT INTO "quotas_limits" VALUES(4, 3, 'MessageCount', 20, , 0); INSERT INTO "quotas_limits" VALUES(5, 4, 'MessageCount', 50, , 0); COMMIT;
Performance tuning
Default cbpolicyd configuration is not optimized for high traffic environments.
For a starters in /opt/zimbra/conf/cbpolicyd.conf.in set :
# Large mailserver: 8, 8, 16, 64, 1000 min_servers=8 min_spare_servers=8 max_spare_servers=16 max_servers=64 max_requests=1000
Understanding the terms :
# Preforking configuration # # min_server - Minimum servers to keep around # min_spare_servers - Minimum spare servers to keep around ready to # handle requests # max_spare_servers - Maximum spare servers to have around doing nothing # max_servers - Maximum servers alltogether # max_requests - Maximum number of requests each child will serve # # One may want to use the following as a rough guideline... # Small mailserver: 2, 2, 4, 10, 1000 # Medium mailserver: 4, 4, 12, 25, 1000 # Large mailserver: 8, 8, 16, 64, 1000
Script to purge sesstion_tracking DB
If you are using very heavily session_tracking because you want to count how many sasl authentication you have have during last minute or during last 5 minutes, you might want to purge this DB every day, for example, as the result could be a DB with to many GB, if your server is highly used.
- Creating the SQL script:
vi /opt/zimbra/log/scripts/purge_cbpolicyd_daily.sql
And add inside:
delete from session_tracking where ( (strftime('%s','now') - UnixTimestamp) > 86400) ; vacuum;
- Add the SQL script to the zimbra crontab at the bottom
# ZIMBRAEND -- DO NOT EDIT ANYTHING BETWEEN THIS LINE AND ZIMBRASTART
0 * * * * cat /opt/zimbra/log/scripts/clean_cbpolicyd_daily.sql | sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb
VACUUM the cbpolicyd.sqlitedb
The sqlite3 "VACUUM" command rebuilds the database and can significantly decrease the amount of space consumed on the filesystem by repacking the DB. Read more about the VACUUM command here: https://sqlite.org/lang_vacuum.html
The first time you run this may require quite a bit of time. Make a backup of the file to be cautious:
# su - zimbra $ cp -pr /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb.bak # get backup $ ls -ld /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb # get current size $ /usr/bin/sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb 'vacuum;' $ ls -ld /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb # get new size
One can then delete the backup if comfortable everything running ok.
A once a week vacuum command can be run out of the "zimbra" user crontab, like the following:
# su - zimbra $ crontab -e
And enter a weekly crontab like the following:
0 1 * * 0 /usr/bin/sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb 'vacuum;'