Cluebringer Policy Daemon

(Redirected from Postfix Policyd)

Cluebringer Policy Daemon in Zimbra

   KB 22647        Last updated on 2023-01-18  




0.00
(0 votes)

Policyd is an anti-spam policy daemon for Postfix (written in C) that does Greylisting, Sender-(envelope, SASL or host / ip)-based throttling (on messages and/or volume per defined time unit), Recipient rate limiting, Spamtrap monitoring / blacklisting, HELO auto blacklisting and HELO randomization preventation.

It is included as a part of the Zimbra package, but it is not enabled by default.

Enabling policyd

See: https://wiki.zimbra.com/wiki/Bash_Scripts_for_cbpolicyd and https://wiki.zimbra.com/wiki/How-to_for_cbpolicyd some parts of this page still refer to SQLite. SQLite is no longer recommended and you should use MariaDB instead.

For all releases, to enable policyd, simply tell the Zimbra installation you want it enabled:

zmprov ms <mta server> +zimbraServiceEnabled cbpolicyd

After 1 to 2 minutes the zmconfigd process will detect that you want this service enabled, and automatically install the database and start the service.

Localconfig keys related to policyd

  • cbpolicyd_pid_file - location of PID file.
  • cbpolicyd_log_file - location of log file.
  • cbpolicyd_db_file - location of SQLite database
  • cbpolicyd_min_servers - Minimum number of servers to have running. ZCS8.0.5 and higher
  • cbpolicyd_min_spare_servers - Minimum number of spare servers to have on hand. ZCS8.0.5 and higher
  • cbpolicyd_max_spare_servers - Maximum number of spare servers to have on hand that are idle. ZCS8.0.5 and higher
  • cbpolicyd_max_servers - Maximum number of servers to allow. ZCS8.0.5 and higher
  • cbpolicyd_max_requests - Maximum number of requests to process per server. ZCS8.0.5 and higher
  • cbpolicyd_cache_file - location of cache
  • cbpolicyd_log_level - loglevel (default 3)
  • cbpolicyd_log_mail - Default is main. Log to policyd's main log mechanism, accepts NO args
  • cbpolicyd_log_detail - What detail of logging to provide. Default "modules"
  • cbpolicyd_bind_port - Bind port for cbpolicyd. default 10031
  • cbpolicyd_timeout - Timeout when talking to clients. Default 120 seconds. ZCS7 and ZCS 8 < ZCS8.0.4
  • cbpolicyd_timeout_idle - Idle timeout. Defaults to 1020 seconds. ZCS8.0.4 and higher
  • cbpolicyd_timeout_busy - Busy timeout. Defaults to 120 seconds. ZCS8.0.4 and higher
  • cbpolicyd_bypass_timeout - Bypass timeout. Defaults to 30 seconds.
  • cbpolicyd_bypass_mode - Bypass fail mode. Default is "tempfail"
  • cbpolicyd_module_accesscontrol - Enable Access Control module. Default 0 (disabled)
  • cbpolicyd_module_greylisting - Enable greylisting. Default 0 (disabled)
  • cbpolicyd_module_greylisting_training - If greylisting is enabled, whether or not to run in training mode. Default 0 (Do not run in training mode). ZCS8 only
  • cbpolicyd_module_greylisting_defer_msg - Message to give when deferring email. Default "Greylisting in effect, please come back later". ZCS8 only
  • cbpolicyd_module_greylisting_blacklist_msg - Message to give when client is on blacklist. Default "Greylisting in effect, sending server blacklisted". ZCS8 only
  • cbpolicyd_module_checkhelo - Enable checkhelo module. Default 0 (disabled)
  • cbpolicyd_module_checkspf - Enable CheckSPF module. Default 0 (disabled)
  • cbpolicyd_module_quotas - Enable Quotas module. Default 1 (enabled)
  • cbpolicyd_module_amavis - Enable amavis module. Default 0 (disabled) ZCS8.0.4 and higher.
  • cbpolicyd_module_accounting - Enable accounting module. Default 0 (disabled). ZCS8.0.4 and higher.

Table data for cbpolicyd

The ".tables" command will display the tables that currently exist in the sqlite DB for cbpolicyd policies. There is also a "master" table named sqlite_sequence that counts the policies:

  • Master table with no added policies. Default amavis_rules policy only exists in ZCS8.0.5 and higher:
# su - zimbra
$ cd /opt/zimbra/data/cbpolicyd/db
$ sqlite3 cbpolicyd.sqlitedb
sqlite> select * from sqlite_sequence;
policies|5
policy_members|5
policy_groups|2
policy_group_members|3
quotas|2
quotas_limits|3
checkhelo_blacklist|4
amavis_rules|1

The above shows that there are:

5 policies
5 policy members
2 policy groups
3 policy group members
2 quota policies
2 quota limit policies
4 checkhelo_blacklist policies
1 amavis rules policy
  • Policy tables available in ZCS8.0.5 and later:
sqlite> .tables
access_control             greylisting_autowhitelist
accounting                 greylisting_tracking
accounting_tracking        greylisting_whitelist
amavis_rules               policies
checkhelo                  policy_group_members
checkhelo_blacklist        policy_groups
checkhelo_tracking         policy_members
checkhelo_whitelist        quotas
checkspf                   quotas_limits
greylisting                quotas_tracking
greylisting_autoblacklist  session_tracking
  • Policy tables available in ZCS8.0.4 and previous:
sqlite> .tables
access_control             greylisting_whitelist
checkhelo                  policies
checkhelo_blacklist        policy_group_members
checkhelo_tracking         policy_groups
checkhelo_whitelist        policy_members
checkspf                   quotas
greylisting                quotas_limits
greylisting_autoblacklist  quotas_tracking
greylisting_autowhitelist  session_tracking
greylisting_tracking

Database Layout

Table Schemas

An important concept with policyd is that all tables are associated via a primary ID. Whether it is policy_group to policy_group_members, or associating quotas to quotas_limits, the ID column is the key association between these tables:

$ sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb
sqlite> .tables
access_control             greylisting_whitelist    
checkhelo                  policies                 
checkhelo_blacklist        policy_group_members     
checkhelo_tracking         policy_groups            
checkhelo_whitelist        policy_members           
checkspf                   quotas                   
greylisting                quotas_limits            
greylisting_autoblacklist  quotas_tracking          
greylisting_autowhitelist  session_tracking         
greylisting_tracking  

Policies

sqlite> PRAGMA table_info(policies);
0|ID|INTEGER|0||1
1|Name|VARCHAR(255)|1||0
2|Priority|SMALLINT|1||0
3|Description|TEXT|0||0
4|Disabled|SMALLINT|1|'0'|0

Policy Members

Using policy_members as an example, the ID column in the policy_members table is associated as a unique ID to the PolicyID column in the policies table:

sqlite> PRAGMA table_info(policy_members);
0|ID|INTEGER|0||1
1|PolicyID|INT8|0||0
2|Source|TEXT|0||0
3|Destination|TEXT|0||0
4|Comment|VARCHAR(1024)|0||0
5|Disabled|SMALLINT|1|'0'|0

Policy Groups

Using policy_groups as an example, the ID column in the policy_groups table is associated as a unique ID to the PolicyGroupID column in the policy_group_members table:

sqlite> PRAGMA table_info(policy_groups);
0|ID|INTEGER|0||1
1|Name|VARCHAR(255)|1||0
2|Disabled|SMALLINT|1|'0'|0
3|Comment|VARCHAR(1024)|0||0
sqlite> PRAGMA table_info(policy_group_members);
0|ID|INTEGER|0||1
1|PolicyGroupID|INT8|0||0
2|Member|VARCHAR(255)|1||0
3|Disabled|SMALLINT|1|'0'|0
4|Comment|VARCHAR(1024)|0||0

Quotas

sqlite> PRAGMA table_info(quotas);
0|ID|INTEGER|0||1
1|PolicyID|INT8|0||0
2|Name|VARCHAR(255)|1||0
3|Track|VARCHAR(255)|1||0
4|Period|UNSIGNED BIG INT|0||0
5|Verdict|VARCHAR(255)|0||0
6|Data|TEXT|0||0
7|LastQuota|SMALLINT|1|'0'|0
8|Comment|VARCHAR(1024)|0||0
9|Disabled|SMALLINT|1|'0'|0
sqlite> PRAGMA table_info(quotas_limits);
0|ID|INTEGER|0||1
1|QuotasID|INT8|0||0
2|Type|VARCHAR(255)|0||0
3|CounterLimit|UNSIGNED BIG INT|0||0
4|Comment|VARCHAR(1024)|0||0
5|Disabled|SMALLINT|1|'0'|0

Default Configurations

Including the default configurations here is important, because the Examples below build on the default configurations

Policies

sqlite> select * from policies;
1|Default|0|Default System Policy|0
2|Default Outbound|10|Default Outbound System Policy|0
3|Default Inbound|10|Default Inbound System Policy|0
4|Default Internal|20|Default Internal System Policy|0
5|Test|50|Test policy|0

Policy Members

sqlite> select * from policy_members;
1|1||||0
2|2|%internal_ips,%internal_domains|!%internal_domains||0
3|3|!%internal_ips,!%internal_domains|%internal_domains||0
4|4|%internal_ips,%internal_domains|%internal_domains||0
5|5|@example.net|||0

Policy Groups

While there are useful Policy Groups for "internal_ips" and "internal_domains", they are defined by default only with one IP range (10.0.0.0/8) and only example domains:

sqlite> select * from policy_groups;
1|internal_ips|0|
2|internal_domains|0|
sqlite> select * from policy_group_members;
1|1|10.0.0.0/8|0|
2|2|@example.org|0|
3|2|@example.com|0|

In order for these to be useful, you would need to add additional policy_group_members to those policy group IDs (1=internal_ips, 2=internal_domains).

Updating Policy Groups

sqlite> select * from policy_group_members;
1|1|10.0.0.0/8|0|
2|2|@example.org|0|
3|2|@example.com|0|
sqlite> update policy_group_members SET Member="192.168.0.0/16" WHERE PolicyGroupID=1;
sqlite> insert into policy_group_members(PolicyGroupID,member,Disabled) VALUES (1,"10.10.10.0/8",0);
sqlite> update policy_group_members SET Member="@mydomain.com" WHERE PolicyGroupID=2;
sqlite> .quit

Quotas

sqlite> select * from quotas;
1|5|Recipient quotas|Recipient:user@domain|3600|REJECT||0||0
2|5|Quota on all /24s|SenderIP:/24|3600|REJECT||0||0

sqlite> select * from quotas_limits;
1|1|MessageCount|10||0
2|1|MessageCumulativeSize|8000||0
3|2|MessageCount|12||0

Examining defined policies for a given table

You can use a select statement to view the defined policies for a given table.

For our example, we will examine the quota (rate limiting) policies that are installed by default. The Quota (rate-limiting) policy consists of two separate tables. The first table is named "quotas", the second table is named "quotas_limits".

sqlite> select * from quotas;
1|5|Recipient quotas|Recipient:user@domain|3600|REJECT||0||0
2|5|Quota on all /24s|SenderIP:/24|3600|REJECT||0||0
  • The first column is the identifier for the quota policy (used in quota_limits for reference back).
  • The fifth column is the time interval for the policy. In this case, they both default to 3600 seconds.
sqlite> select * from quotas_limits;
1|1|MessageCount|10||0
2|1|MessageCumulativeSize|8000||0
3|2|MessageCount|12||0
  • The second column is referring to the identifier in the quota table. More than one rule can be defined per quota identifier as seen above.
  • The fourth column is the "rate limit". For example, in the 3rd row, "12" is the max number of hits coming from a specific sender IP within 3600 seconds.

Example Configuration

Adding policy definitions via the command line

The following examples show how to implement various policies via the command line

Add a Zimbra Policy Group

First, we will add a Zimbra policy group to cbpolicyd and associate any newly added rules with this group. This should only be done once. As Zimbra:

  • Create a file called zimbra-group.sql with the following contents. Note: this policy will get added as PolicyID 6, because there are 5 entries already in the default policies table above:
BEGIN TRANSACTION;
INSERT INTO "policies" (Name,Priority,Description) VALUES('Zimbra CBPolicyd Policies', 0, 'Zimbra CBPolicyd Policies');
INSERT INTO "policy_members" (PolicyID,Source,Destination) VALUES(6, 'any', 'any');
COMMIT;
  • Import the policy group into cbpolicyd:
sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb < zimbra-group.sql

This will create a new "zimbra policy" group with priority 0 (highest priority) that gets applied to all email.

Defining a rate-limit (quota) with cbpolicyd

By default "quota" or "rate limiting" is enabled, but there is no policy defining how it should behave. The below example adds two policies :

  • Rate limit any sender from sending more then 100 emails every 60 seconds. Messages beyond this limit are deferred.
  • Rate limit any @domain from receiving more than 125 emails in a 60 second period. Messages beyond this rate are rejected.

As Zimbra:

  • Create a file called rate-limit.sql with the following contents. Note: these two quotas entries will get added as QuotasID 3 and 4, because there are two entries already in the default quotas table, as shown above:
BEGIN TRANSACTION;
INSERT INTO "quotas" (PolicyID,Name,Track,Period,Verdict,Data) VALUES (6, 'Sender:user@domain','Sender:user@domain', 60, 'DEFER', 'Deferring: Too many messages from sender in last 60');
INSERT INTO "quotas" (PolicyID,Name,Track,Period,Verdict) VALUES (6, 'Recipient:@domain', 'Recipient:@domain', 60, 'REJECT');
INSERT INTO "quotas_limits" (QuotasID,Type,CounterLimit) VALUES(3, 'MessageCount', 100);
INSERT INTO "quotas_limits" (QuotasID,Type,CounterLimit) VALUES(4, 'MessageCount', 125);
COMMIT;
  • Import the rules into cbpolicyd:
sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb < rate-limit.sql

Changing a Quota or Quotas Limit

Easiest method to change a quota is to delete your previously set quotas and quotas limits, and import a new rate-limit.sql:

Delete

sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb
sqlite> select * from quotas;
1|5|Recipient quotas|Recipient:user@domain|3600|REJECT||0||0
2|5|Quota on all /24s|SenderIP:/24|3600|REJECT||0||0
3|6|Sender:user@domain|Sender:user@domain|60|DEFER|Deferring: Too many messages sent|0||0
4|6|Recipient:@domain|Recipient:@domain|60|REJECT||0||0
sqlite> delete from quotas where ID=3;
sqlite> delete from quotas where ID=4;
sqlite> select * from quotas_limits;
sqlite> delete from quotas_limits where ID=4;
sqlite> delete from quotas_limits where ID=4;
sqlite> .quit

Update rate-limit.sql and Re-Import

vi rate-limit.sql
sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb < rate-limit.sql

Defining a greylisting policy with cbpolicyd

Note: See the official wiki for an explanation of the parameters By default, the greylisting module is not enabled, and there are no greylisting policies. To enable greylisting, do the following as zimbra:

For Zimbra 8.6+, do

  • zmprov ms `zmhostname` zimbraCBPolicydGreylistingEnabled TRUE

for older version of Zimbra, do the following instead:

  • zmlocalconfig -e cbpolicyd_module_greylisting=1

ZCS8 only: If you want to start building your greylisting table without enforcing policy (I.e., train your server before enabling greylisting):

In Zimbra 8.6+, do:

  • zmprov ms `zmhostname` zimbraCBPolicydGreylistingTrainingEnabled TRUE

for earlier 8.x versions, do:

  • zmlocalconfig -e cbpolicyd_module_greylisting_training=1

When you are ready to have greylisting applied, set the value for cbpolicyd_module_greylisting_training to zero.

Greylisting policy types

There are TWO types of greylisting policies -- Percentage and message total. Note that you will likely want to tweak the following values to match your particular MTA requirements.

  • Percentage policy example. This example will create a greylist policy that whitelists SenderIP/32 (Individual IP). You may want this to be /24 (Subnet). If a client IP retries 95 out of 100 messages (95%), then cbpolicyd will autowhitelist the sending MTA. If it fails 100/100 retries (100%), it will auto-blacklist the sending MTA. If you have numerous low volume MTAs sending to your site, you may wish to change the 100/95 ratio downwards. You may also wish to adjust the 100/100 downwards if you want to auto-blacklist more quickly.
BEGIN TRANSACTION;
INSERT INTO "greylisting" (PolicyID,Name,UseGreylisting,GreylistPeriod,Track,GreylistAuthValidity,
GreylistUnAuthValidity,UseAutoWhitelist,AutoWhitelistPeriod,AutoWhitelistCount,AutoWhitelistPercentage,
UseAutoBlacklist,AutoBlacklistPeriod,AutoBlacklistCount,AutoBlacklistPercentage)
 VALUES(6,"Zimbra Greylisting percentage",1,120,"SenderIP:/32",604800,86400,1,604800,100,95,1,604800,100,100);
COMMIT;
  • Message total policy example. This example will create a greylist policy that whitelist SenderIP/32 (Individual IP). You may want this to be /24 (Subnet). If a client IP retries the same message 50 times, then cbpolicyd will autowhitelist the sending MTA. If it fails 80 times, it will autoblacklist it. If you have numerous low volume MTAs sending to your site, you may wish to adjust these settings.
BEGIN TRANSACTION;
INSERT INTO "greylisting" (PolicyID,Name,UseGreylisting,GreylistPeriod,Track,GreylistAuthValidity,
GreylistUnAuthValidity,UseAutoWhitelist,AutoWhitelistPeriod,AutoWhitelistCount,UseAutoBlacklist,
AutoBlacklistPeriod,AutoBlacklistCount)
 Values(6,"Zimbra Greylisting Count",1,120,"SenderIP:/32",604800,86400,1,604800,50,1,604800,80);
COMMIT;
  • It is generally advised to whitelist all mailbox servers, so that outgoing mail from the web client does not end up triggering the greylist. Replace "1.2.3.4" with the IP address of the mailbox server. You can add as many INSERT INTO lines as necessary.
BEGIN TRANSACTION;
INSERT INTO "greylisting_whitelist" (Source,Comment,Disabled) VALUES ("SenderIP:1.2.3.4", "Whitelist mailbox",0);
COMMIT;


  • Import the policy:
sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb < zimbra-greylist.sql

Defining an SPF policy with cbpolicyd

By default, the SPF checking module is not enabled, and there are no SPF policies. To enable SPF checking, do the following as the zimbra user:

  • zmlocalconfig -e cbpolicyd_module_checkspf=1
  • Create /tmp/check-spf.sql and import it. This will enable SPF checking, and add a header on whether or not the SPF check was valid. It will not reject emails with failed SPF checks.
BEGIN TRANSACTION;
INSERT INTO "checkspf" (PolicyID,Name,UseSPF,RejectFailedSPF,AddSPFHeader,Comment,Disabled) VALUES (6,"SPF Policy",1,0,1,"Zimbra CheckSPF Policy",0);
COMMIT;

Database cleanup for cbpolicyd

Only necessary for ZCS 8.6 and prior. ZCS 8.7 and later include a cronjob to cleanup cluebringer.

Old and outdated entries are not automatically purged from CBPolicyd's database. Thanks to Cine @ Zextras for noting this. A tool called "cbpadmin" is provided to take care of this operation, so one may wish to schedule a cronjob on the MTAs to do this

35 3 * * * /opt/zimbra/cbpolicyd/bin/cbpadmin --config=/opt/zimbra/conf/cbpolicyd.conf  --cleanup >/dev/null

Performance tuning

In ZCS 8.0.5, you can easily tune the cbpolicyd daemon for high volume MTAs. By default it is tuned for medium volume servers.

For a large volume server:

zmlocalconfig -e cbpolicyd_min_servers=8
zmlocalconfig -e cbpolicyd_min_spare_servers=8
zmlocalconfig -e cbpolicyd_max_spare_servers=16
zmlocalconfig -e cbpolicyd_max_servers=64
zmlocalconfig -e cbpolicyd_max_requests=1000

Disabling cbpolicyd

Disabling cbpolicyd is as simple as removing it as an enabled service:

  • zmprov ms <mta server> -zimbraServiceEnabled cbpolicyd

This will trigger zmconfigd to rewrite the postfix configuration and shut down cbpolicyd.

See Also

Detailed cbpolicyd for zimbra information

Verified Against: ZCS 8.0, ZCS 7.0 Date Created: 2/26/2013
Article ID: https://wiki.zimbra.com/index.php?title=Cluebringer_Policy_Daemon Date Modified: 2023-01-18



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »

Jump to: navigation, search