How-to for cbpolicyd: Difference between revisions

No edit summary
Line 258: Line 258:
  0 * * * * cat /opt/zimbra/log/scripts/clean_cbpolicyd_daily.sql  |
  0 * * * * cat /opt/zimbra/log/scripts/clean_cbpolicyd_daily.sql  |
  sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb
  sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb
=== VACUUM the cbpolicyd.sqlitedb  ===
The sqlite3 "VACUUM" command rebuilds the database and can significantly decrease the amount of space consumed on the filesystem by repacking the DB. Read more about the VACUUM command here: https://sqlite.org/lang_vacuum.html
The first time you run this may require quite a bit of time. Make a backup of the file to be cautious:
# su - zimbra
$ cp -pr /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb.bak  # get backup
$ ls -ld /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb  # get current size
$ /usr/bin/sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb 'vacuum;'
$ ls -ld /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb  # get new size
One can then delete the backup if comfortable everything running ok.
A once a week vacuum command can be run out of the "zimbra" user crontab, like the following:
# su - zimbra
$ crontab -e
And enter a weekly crontab like the following:
0 1 * * 0 /usr/bin/sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb 'vacuum;'


[[Category:policyd]]
[[Category:policyd]]
{{Article Footer|Zimbra Collaboration 8.0, 7.0|04/16/2014}}
{{Article Footer|Zimbra Collaboration 8.0, 7.0|04/16/2014}}

Revision as of 19:03, 26 June 2017

How-to for CBPolicyd

   KB 20402        Last updated on 2017-06-26  




0.00
(0 votes)

See Also

Introduction

  • Packaged as part of the zimbra-mta package
  • Configured to run as a daemon on 127.0.0.1:10031
  • Integrated with postfix as a check_policy_service
  • Cbpolicyd database uses sqlite
  • Default configuration enables quotas module


Enable/Disable

To enable :

zmprov ms <mta server> +zimbraServiceEnabled cbpolicyd

The following changes get written by zmconfigd to postconf :

smtpd_end_of_data_restrictions = check_policy_service inet:localhost:10031
smtpd_recipient_restrictions = check_policy_service inet:localhost:10031, reject_non_fqdn_recipient, reject_unlisted_recipient, reject_invalid_helo_hostname, reject_non_fqdn_sender, permit

When the service is enabled the following is logged in /opt/zimbra/log/cbpolicyd.log

[2013/04/02-02:15:39 - 8686] [CORE] NOTICE: 2013/04/02-02:15:39 cbp (type Net::Server::PreFork) starting! pid(8686)
[2013/04/02-02:15:39 - 8686] [CORE] NOTICE: Resolved [localhost]:10031 to [::1]:10031, IPv6
[2013/04/02-02:15:39 - 8686] [CORE] NOTICE: Resolved [localhost]:10031 to [127.0.0.1]:10031,  IPv4
[2013/04/02-02:15:39 - 8686] [CORE] NOTICE: Binding to TCP port 10031 on host ::1 with IPv6
[2013/04/02-02:15:39 - 8686] [CORE] NOTICE: Binding to TCP port 10031 on host 127.0.0.1 with IPv4
[2013/04/02-02:15:39 - 8686] [CORE] NOTICE: Setting gid to "501 501"
[2013/04/02-02:15:39 - 8686] [CORE] INFO: Setting up serialization via flock
[2013/04/02-02:15:39 - 8686] [CORE] INFO: Beginning prefork (4 processes)
[2013/04/02-02:15:39 - 8686] [CORE] INFO: Starting "4" children

Process :

 0:00 /usr/bin/perl /opt/zimbra/cbpolicyd/bin/cbpolicyd --config /opt/zimbra/conf/cbpolicyd.conf


To Disable :

zmprov ms <mta server> -zimbraServiceEnabled cbpolicyd
  • Note: Ideally zmconfigd should be removing the postconf entry. If it does not then remove run zmcontrol stop;zmcontrol start. This will force a rewrite of the postfix configuration files.

Config file

Config file for the cbpolicyd process :

/opt/zimbra/conf/cbpolicyd.conf.in gets rewritten by configd to /opt/zimbra/conf/cbpolicyd.conf

Configuration of cbpolicyd daemon via localconfig :

 # default log, pid and database files
 cbpolicyd_pid_file
 cbpolicyd_log_file
 cbpolicyd_db_file
 
 # policyd log_level default is 3 for info,notices,warnings and errors
 cbpolicyd_log_level=3            [0-4]
 
 # use cbpolicyd_log_file (main) or syslog (maillog) for logging
 cbpolicyd_log_mail=main          [main|maillog]
 
 # components of policyd to log data
 cbpolicyd_log_detail=modules     [modules,tracking,policies,protocols]
 
 # bind host/port default is 127.0.0.1:10031
 cbpolicyd_bind_host=127.0.0.1   
 cbpolicyd_bind_port=10031
 # timeout in communication with clients. ZCS7, ZCS8 < 8.0.4
 cbpolicyd_timeout=120
 
 # idle timeout with.  ZCS 8.0.4 and higher
 cbpolicyd_timeout_idle=1020
 # Busy timeout.  ZCS 8.0.4 and higher
 cbpolicyd_timeout_busy=120
 # how many seconds before retrying database connection
 cbpolicyd_bypass_timeout=30
 # what to do when there is a database connection problem
 cbpolicyd_bypass_mode=tempfail   [tempfail|pass]
 
 # enable/disable specific policyd modules
 cbpolicyd_module_accesscontrol=0 [0|1]
 cbpolicyd_module_accounting=0    [0|1] ZCS8.0.4 and higher
 cbpolicyd_module_amavis=0        [0|1] ZCS8.0.4 and higher
 cbpolicyd_module_greylisting=0   [0|1]
 cbpolicyd_module_greylisting_training=0  [0|1] ZCS8.0.4 and higher.  If enabled, greylisting is in training mode only (does not actually do greylisting)
 cbpolicyd_module_greylisting_defer_msg  ZCS8.0.4 and higher.  Message to log for deferred mail
 cbpolicyd_module_greylisting_blacklist_msg  ZCS8.0.4 and higher.  Message to log for blacklisted mail
 cbpolicyd_module_checkhelo=0     [0|1]
 cbpolicyd_module_checkspf=0      [0|1]
 cbpolicyd_module_quotas=1        [0|1]

Default config values :

$ zmlocalconfig  | grep -i cbpolicy
cbpolicyd_bind_port = 10031
cbpolicyd_bypass_mode = tempfail
cbpolicyd_bypass_timeout = 30
cbpolicyd_cache_file = ${zimbra_home}/data/cache
cbpolicyd_db_file = ${zimbra_home}/data/cbpolicyd/db/cbpolicyd.sqlitedb
cbpolicyd_log_detail = modules
cbpolicyd_log_file = ${zimbra_log_directory}/cbpolicyd.log
cbpolicyd_log_level = 4
cbpolicyd_log_mail = main
cbpolicyd_module_accesscontrol = 0
cbpolicyd_module_checkhelo = 0
cbpolicyd_module_checkspf = 0
cbpolicyd_module_greylisting = 0
cbpolicyd_module_quotas = 1
cbpolicyd_pid_file = ${zimbra_log_directory}/cbpolicyd.pid
cbpolicyd_timeout = 120


Logging

Log file for Cbpolicyd :

log_file=/opt/zimbra/log/cbpolicyd.log

Log levels can be controlled by LC - "cbpolicyd_log_level", the value can be set to one of the below :

# Log level:
# 0 - Errors only
# 1 - Warnings and errors
# 2 - Notices, warnings, errors
# 3 - Info, notices, warnings, errors
# 4 - Debugging

To log in further detail the below can be set, controlling LC attribute is "cbpolicyd_log_detail"

# modules   - Log detailed module running information
# tracking  - Log detailed tracking information
# policies  - Log policy resolution
# protocols     - Log general protocol info, but detailed
# bizanga   - Log the bizanga protocol
#
# There is no default for this configuration option. Options can be
# separated by commas. ie. protocols,modules
#
log_detail=modules


Database Design

To access the policyd db :

sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb

List of tables :

sqlite> .tables
access_control             greylisting_whitelist
checkhelo                  policies
checkhelo_blacklist        policy_group_members
checkhelo_tracking         policy_groups
checkhelo_whitelist        policy_members
checkspf                   quotas
greylisting                quotas_limits
greylisting_autoblacklist  quotas_tracking
greylisting_autowhitelist  session_tracking
greylisting_tracking

Master table, to keep count of the policies :

sqlite> select * from sqlite_sequence;
policies|5
policy_members|5
policy_groups|2
policy_group_members|3
quotas|2
quotas_limits|3
checkhelo_blacklist|4

Default policy:

sqlite> select * from quotas;
1|5|Recipient quotas|Recipient:user@domain|3600|REJECT||0||0
2|5|Quota on all /24s|SenderIP:/24|3600|REJECT||0||0
* 1st Column is the identifier, sort of like "id" of the policy
* 5th Column is the time interval for the policy. Explained in the example below.

Quota limits for the above policies :

sqlite> select * from quotas_limits;
1|1|MessageCount|10||0
2|1|MessageCumulativeSize|8000||0
3|2|MessageCount|12||0
* 2nd Columne is the same identifier as referenced in table "quotas".
* 4th Column is the number of "rate limit". For example "12" is the number of hits coming from "SenderIP" in a period of 3600 seconds.

Example

The below example adds two policies :

* Rate limit any sender from sending more then 20 emails every 60 seconds.  Messages beyond this limit are deferred.

* Rate limit any @domain from receiving more then 50 emails in a 60 second period.  Messages beyond this rate are rejected.

Example sql file, to be imported into "cbpolicyd.sqlitedb" database

BEGIN TRANSACTION;
INSERT INTO "policies" VALUES(6, 'Zimbra', 0, 'Zimbra QA Test Policy', 0);
DELETE FROM sqlite_sequence;
INSERT INTO "sqlite_sequence" VALUES('policies', 6);
INSERT INTO "sqlite_sequence" VALUES('policy_members', 6);
INSERT INTO "sqlite_sequence" VALUES('policy_groups', 2);
INSERT INTO "sqlite_sequence" VALUES('policy_group_members', 3);
INSERT INTO "sqlite_sequence" VALUES('quotas', 4);
INSERT INTO "sqlite_sequence" VALUES('quotas_limits', 5);
INSERT INTO "sqlite_sequence" VALUES('checkhelo_blacklist', 4);
INSERT INTO "policy_members" VALUES(6, 6, 'any', 'any', , 0);
INSERT INTO "quotas" VALUES(3, 6, 'Sender:user@domain','Sender:user@domain', 60, 'DEFER', 'Deferring: Too many messages from sender in last 60', , 0);
INSERT INTO "quotas" VALUES(4, 6, 'Recipient:@domain', 'Recipient:@domain', 60, 'REJECT', , , 0);
INSERT INTO "quotas_limits" VALUES(4, 3, 'MessageCount', 20, , 0);
INSERT INTO "quotas_limits" VALUES(5, 4, 'MessageCount', 50, , 0);
COMMIT;


Performance tuning

Default cbpolicyd configuration is not optimized for high traffic environments.

For a starters in /opt/zimbra/conf/cbpolicyd.conf.in set :

# Large mailserver: 8, 8, 16, 64, 1000
min_servers=8
min_spare_servers=8
max_spare_servers=16
max_servers=64
max_requests=1000

Understanding the terms :

# Preforking configuration
#
# min_server        - Minimum servers to keep around
# min_spare_servers - Minimum spare servers to keep around ready to
#             handle requests
# max_spare_servers - Maximum spare servers to have around doing nothing
# max_servers       - Maximum servers alltogether
# max_requests      - Maximum number of requests each child will serve
#
# One may want to use the following as a rough guideline...
# Small mailserver:  2, 2, 4, 10, 1000
# Medium mailserver: 4, 4, 12, 25, 1000
# Large mailserver: 8, 8, 16, 64, 1000

Script to purge sesstion_tracking DB

If you are using very heavily session_tracking because you want to count how many sasl authentication you have have during last minute or during last 5 minutes, you might want to purge this DB every day, for example, as the result could be a DB with to many GB, if your server is highly used.

  • Creating the SQL script:
vi /opt/zimbra/log/scripts/purge_cbpolicyd_daily.sql

And add inside:

delete from session_tracking where ( (strftime('%s','now') -
UnixTimestamp) > 86400) ;
vacuum;
  • Add the SQL script to the zimbra crontab at the bottom
# ZIMBRAEND -- DO NOT EDIT ANYTHING BETWEEN THIS LINE AND ZIMBRASTART
0 * * * * cat /opt/zimbra/log/scripts/clean_cbpolicyd_daily.sql  |
sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb

VACUUM the cbpolicyd.sqlitedb

The sqlite3 "VACUUM" command rebuilds the database and can significantly decrease the amount of space consumed on the filesystem by repacking the DB. Read more about the VACUUM command here: https://sqlite.org/lang_vacuum.html

The first time you run this may require quite a bit of time. Make a backup of the file to be cautious:

# su - zimbra
$ cp -pr /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb.bak   # get backup
$ ls -ld /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb   # get current size
$ /usr/bin/sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb 'vacuum;'
$ ls -ld /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb   # get new size

One can then delete the backup if comfortable everything running ok.

A once a week vacuum command can be run out of the "zimbra" user crontab, like the following:

# su - zimbra
$ crontab -e

And enter a weekly crontab like the following:

0 1 * * 0 /usr/bin/sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb 'vacuum;'
Verified Against: Zimbra Collaboration 8.0, 7.0 Date Created: 04/16/2014
Article ID: https://wiki.zimbra.com/index.php?title=How-to_for_cbpolicyd Date Modified: 2017-06-26



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »


Jump to: navigation, search