SpamAssassin Customizations

From Zimbra :: Wiki

Jump to: navigation, search
Admin Article

Article Information

This article applies to the following ZCS versions.
  ZCS 8.0 Article  ZCS 8.0
  ZCS 7.0 Article  ZCS 7.0


Contents

For SpamAssassin and Anti-spam Updates

Customizing SpamAssassin

ZCS 8 and later

For ZCS 8 and later, SpamAssassin scans for all *.cf files in /opt/zimbra/conf/sa and loads them in alphabetical order. If you create a sausers.cf file, it will be loaded after salocal.cf is loaded. This is the supported method for doing customizations of SpamAssassin.

ZCS 6 and ZCS7

For ZCS 6 and ZCS7, SpamAssassin customizations go in /opt/zimbra/conf/sauser.cf. When upgrading to ZCS8 the file will be reloacted to /opt/zimbra/conf/sa

Customizing Postfix

In ZCS 7 and ZCS 8, customizing Postfix is a mix of zmlocalconfig and zmprov settings. In ZCS 8.5, virutally all settings are done via zmprov (zmlocalconfig settings will be migrated on upgrade if they do not match the default value).

zmprov/zmlocalconfig are both permissible and the recommended way to perform Postfix customizations for supported keys.

For example:

zmprov ms <server> +zimbraMtaRestriction reject_unknown_reverse_client_hostname

Specific Suggested Tweaks

Last update 14 April 2014 by L. Mark Stone, Reliable Networks

Our client base is very nervous about spam-delivered malware but even more concerned about "false-positives" i.e. legitimate email incorrectly identified as spam. Consequently, we've had to develop tweaks to improve Zimbra's default SpamAssassin configurations. The results have been that users with very public email addresses who typically receive several hundred to more than a thousand emails per day will see no more than ~3 spam emails per day in their Inbox. In our experience, anything less than that and you are likely to wind up with false positives.

If your end-user base is more tolerant of false positives, then you can tighten things up.

Keep in mind that Zimbra's Postfix takes a cut at filtering the email stream before Zimbra's SpamAssassin, and that SpamAssassin's processing of emails is much more resource intensive than Postfix's. Consequently, any filtering that you can do at the Postfix level to block emails outright will be helpful in both blocking spam and lowering resource utilization on your Zimbra server. Just be careful of inducing false positives!

Postfix Tweaks via the Admin Console

At the Postfix level we use just a few complementary and conservative RBLs, one DNS check and one Protocol check. All of these can be configured via the Admin Console: (Global Settings > MTA).

The RBLs we use are:

  • zen.spamhaus.org
  • psbl.surriel.com
  • b.barracudacentral.org

On the same Admin Console page we also enable (and leave the remaining Protocol and DNS checks disabled):

  • reject_non_fqdn_sender
  • reject_unknown_sender_domain (Note this setting will be updated in 8.0.5)

On that same page we also make sure disable "Add X-Originating-IP to messages" as this can block email from remote users with fat email clients like Outlook and Thunderbird on home and public networks like Internet cafes (ZWC clients are unaffected by this.)

SpamAssassin Tweaks via the Commandline

Our current recommended SpamAssassin customizations comprise three complementary methods:

  1. Increase the log level reported by Amavis to get clarity from SpamAssassin on why/how spam is being blocked and getting through.
  2. Put Amavis's temporary directory on a RAM disk to speed up processing.
  3. Tweak the scores for a few selected individual SpamAssassin tests after installing Pyzor and Razor2.
1. Increase Amavis's Log Level

We found that increasing the log level from 1 to 2 puts in /var/log/zimbra.log the specific SpamAssassin tests which each email has triggered.

Customizing the Amavis Loglevel is supported in ZCS 8.0.5 and later:

zmprov mcf zimbraAmavisLogLevel 2

If you are on an earlier release, this can be achieved by editing /opt/zimbra/conf/amavisd.conf.in. You will need to change the file's permissions to be writable, edit the file, then change the permissions back. Probably a good idea to make a backup copy of the file first... The final edit should should look like this:

$log_level = 2; # verbosity 0..5 - 1 is the minimum for msg tracing

Restart amavis for the change to take effect (zmavavisdctl restart). If you are on ZCS 8.0.5 or later, zmconfigd will automatically restart Amavis for you if you change the loglevel.

Now when an email is marked as spam and an end user asks you "Why?", you can grep /opt/zimbra/log and find out exactly why. Note the sender and recipient email addresses in the actual log file snippet below have been altered for privacy:

Nov 26 13:55:02 mail2 amavis[19107]: (19107-13) SPAM, <comsumer_health@spamsender.com> -> <masked_recipient@example.com>, Yes, score=17.071 tag=-10 tag2=3.8 kill=16 tests=[BAYES_99=4, HTML_MESSAGE=0.001, RAZOR2_CF_RANGE_51_100=0.5, RAZOR2_CF_RANGE_E8_51_100=1.886, RAZOR2_CHECK=2.75, RDNS_NONE=3.5, SPF_PASS=-0.001, T_HK_NAME_DR=0.01, URIBL_BLACK=2.725, URIBL_DBL_SPAM=1.7] autolearn=spam


In the above example you can see that the sending server has no PTR (Reverse DNS record) and has already been reported to Razor.

2. Put Amavis's Temp Dir on a RAM Disk

We have seen even with fast RAID10 arrays that Amavis's processing an email with large attachments through SpamAssassin can take as long as 10-20 seconds. Putting Amavis'd temp directory on a RAM disk cuts this down to 1-2 seconds. Ralf Hildebrandt's book on Postfix has a section describing how to size the RAM disk, and why this is entirely safe for mail flow even in the event of a server crash. After you've done the homework for sizing, all you need to do is:

  1. Determine how your operating system supports RAM disks (SLES and Ubuntu use /dev/shm built in).
  2. Determine the UID:GID of the zimbra user account.
  3. Stop amavis, mount the RAM disk, start amavis and then edit /etc/fstab to make the change permanent.

To determine the UID:GID for the zimbra user (on each MTA server; the UID:GID pair is unique to each server):

On a SLES system the UID (1004) will be different than the GID (1000) on an Ubuntu or RHEL system where they are both typically the same:

malbec:~ # cat /etc/passwd | grep "zimbra:"
zimbra:x:1004:1000::/opt/zimbra:/bin/bash
malbec:~ #
zimbra@mail2:~$ cat /etc/passwd | grep "zimbra:"
zimbra:x:1001:1001::/opt/zimbra:/bin/bash
zimbra@mail2:~$


Our /etc/fstab entry for a 1GB RAM disks on the server mail2 therefore looks like:

zimbra@mail2:~$ cat /etc/fstab | grep amavis
/dev/shm /opt/zimbra/data/amavisd/tmp tmpfs defaults,noexec,nodev,nosuid,size=1024m,mode=750,uid=1001,gid=1001 0 0
zimbra@mail2:~$
3. Tweak Selected SpamAssasin Scores After Installing Pyzor and Razor2
Installing Razor and Pyzor on Ubuntu
  • aptitude install razor pyzor
Installing Razor and Pyzor on RHEL6/CentOS6
  • Create /etc/yum.repos.d/epel.repo

[epel]
name=EPEL repository
baseurl=http://mirrors.kernel.org/fedora-epel/6/x86_64
enabled=1
gpgcheck=0

  • yum update
  • yum install pyzor perl-Razor-Agent
Configuring Pyzor

As the zimbra user

pyzor --homedir /opt/zimbra/data/amavisd/.pyzor discover

Update /opt/zimbra/conf/sa/sauser.cf

# pyzor
use_pyzor 1
pyzor_path /usr/bin/pyzor

Configuring Razor

As the zimbra user

razor-admin -home=/opt/zimbra/data/amavisd/.razor -create
razor-admin -home=/opt/zimbra/data/amavisd/.razor -discover
razor-admin -home=/opt/zimbra/data/amavisd/.razor -register -user postmaster@yourdomain.com

Update /opt/zimbra/conf/sa/sauser.cf

# razor
use_razor2 1

Update SpamAssassin scoring

After installing Pyzor and Razor2 and restarting Zimbra's Amavis to make sure these modules are loaded by SpamAssassin, we add the following to the appropriate custom SpamAssassin configuration file:

score URIBL_BLACK 2.725
score RAZOR2_CHECK 2.750
score PYZOR_CHECK 2.750
score BAYES_99 4.000
score RDNS_NONE 3.500
score BAYES_00 -0.8

Then run zmamavisdctl restart to load the new scores

4. Add custom rules from Kevin McGrail to your scores
5. Add the SOUGHT ruleset to your scores

TODO: Document configuring http://taint.org/2007/08/15/004348a.html for Zimbra

6. Enable DCC

TODO: Document http://www.rhyolite.com/dcc/

DNSWL registration

Register your MTAs with DNSWL: http://dnswl.org/request.pl

Other notes

We have found that increasing the scores of the above selected SpamAssassin scores blocks a lot of spam that would otherwise get through.

As we make updates to our own configurations, we will endeavor to keep this page updated as well.

Personal tools