Difference between revisions of "Anti-spam Strategies"

(ZCS 7.2.5 and later)
(Specific SpamAssassin tweaks from Reliable Networks)
Line 23: Line 23:
  
 
'''zmprov ms <server> +zimbraMtaRestriction reject_unknown_reverse_client_hostname'''
 
'''zmprov ms <server> +zimbraMtaRestriction reject_unknown_reverse_client_hostname'''
 +
 +
=== Specific Suggested Tweaks ===
 +
''Last update 1 September 2013 by L. Mark Stone, Reliable Networks
 +
 +
Our client base is very nervous about spam-delivered malware but even more concerned about "false-positives" i.e. legitimate email incorrectly identified as spam. Consequently, we've had to develop tweaks to improve Zimbra's default SpamAssassin configurations. The results have been that users with very public email addresses who typically receive several hundred to more than a thousand emails per day will see no more than ~3 spam emails per day in their Inbox.  In our experience, anything less than that and you are likely to wind up with false positives.
 +
 +
If your end-user base is more tolerant of false positives, then you can tighten things up. 
 +
 +
Keep in mind that Zimbra's Postfix takes a cut at filtering the email stream before Zimbra's SpamAssassin, and that SpamAssassin's processing of emails is much more resource intensive than Postfix's.  Consequently, any filtering that you can do at the Postfix level to block emails outright will be helpful in both blocking spam and lowering resource utilization on your Zimbra server.  Just be careful of inducing false positives!
 +
 +
==== Postfix Tweaks via the Admin Console ====
 +
At the Postfix level we use just a few complementary and conservative RBLs, one DNS check and one Protocol check.  All of these can be configured via the Admin Console: (Global Settings > MTA).
 +
 +
The RBLs we use are:
 +
* zen.spamhaus.org
 +
* psbl.surriel.com
 +
* b.barracudacentral.org
 +
 +
On the same Admin Console page we also enable (and leave the remaining Protocol and DNS checks disabled):
 +
* reject_non_fqdn_sender
 +
* reject_unknown_sender_domain (Note this setting will be updated in 8.0.5)
 +
 +
On that same page we also make sure disable "Add X-Originating-IP to messages" as this can block email from remote users with fat email clients like Outlook and Thunderbird on home and public networks like Internet cafes (ZWC clients are unaffected by this.)
 +
 +
==== SpamAssassin Tweaks via the Commandline ====
 +
Our current recommended SpamAssassin customizations comprise three complementary methods:
 +
# Increase the log level reported by Amavis to get clarity from SpamAssassin on why/how spam is being blocked and getting through.
 +
# Put Amavis's temporary directory on a RAM disk to speed up processing.
 +
# Tweak the scores for a few selected individual SpamAssassin tests after installing Pyzor and Razor2.
 +
 +
===== 1. Increase Amavis's Log Level =====
 +
We found that increasing the log level from 1 to 2 puts in /var/log/zimbra.log the specific SpamAssassin tests which each email has triggered. This tweak is not supported by Zimbra and will not survive Zimbra upgrades.
 +
 +
The file to edit is /opt/zimbra/conf/amavisd.conf.in.  You will need to change the file's permissions to be writable, edit the file, then change the permissions back. Probably a good idea to make a backup copy of the file first...  The final edit should should look like this:
 +
 +
: $log_level = 2;              # verbosity 0..5 - 1 is the minimum for msg tracing
 +
 +
Restart amavis for the change to take effect (zmavavisdctl restart).
 +
 +
===== 2. Put Amavis's Temp Dir on a RAM Disk =====
 +
We have seen even with fast RAID10 arrays that Amavis's processing an email with large attachments through SpamAssassin can take as long as 10-20 seconds.  Putting Amavis'd temp directory on a RAM disk cuts this down to 1-2 seconds. Ralf Hildebrandt's book on Postfix has a section describing how to size the RAM disk, and why this entirely safe for mail flow even in the event of a server crash.  After you've done the homework for sizing, all you need to do is:
 +
 +
# Determine how your operating system supports RAM disks (SLES and Ubuntu use /dev/shm built in).
 +
# Determine the UID:GID of the zimbra user account.
 +
# Stop amavis, mount the RAM disk, start amavis and then edit /etc/fstab to make the change permanent.
 +
 +
To determine the UID:GID for the zimbra user (on each MTA server; the UID:GID pair is unique to each server):
 +
 +
On a SLES system the UID (1004) will be different than the GID (1000) on an Ubuntu or RHEL system where they are both typically the same:
 +
: malbec:~ # cat /etc/passwd | grep "zimbra:"
 +
: zimbra:x:1004:1000::/opt/zimbra:/bin/bash
 +
: malbec:~ #
 +
 +
: zimbra@mail2:~$ cat /etc/passwd | grep "zimbra:"
 +
: zimbra:x:1001:1001::/opt/zimbra:/bin/bash
 +
: zimbra@mail2:~$
 +
 +
 +
Our /etc/fstab entry for a 1GB RAM disks on the server mail2 therefore looks like:
 +
 +
: zimbra@mail2:~$ cat /etc/fstab | grep amavis
 +
: /dev/shm        /opt/zimbra/data/amavisd/tmp              tmpfs  defaults,noexec,nodev,nosuid,size=1024m,mode=750,uid=1001,gid=1001 0 0
 +
: zimbra@mail2:~$
 +
 +
 +
===== 3. Tweak Selected SpamAssasin Scores After Installing Pyzor and Razor2 =====
 +
[Need to add section for installing and initializing  Pyzor and Razor2 on Ubuntu Server].
 +
 +
After installing Pyzor and Razor2 and restarting Zimbra's Amavis to make sure these modules are loaded by SpamAssassin, we add the following to the appropriate custom SpamAssassin configuration file:
 +
 +
: score URIBL_BLACK 2.725
 +
: score RAZOR2_CHECK 2.750
 +
: score PYZOR_CHECK 2.750
 +
: score BAYES_99 4.000
 +
: score RDNS_NONE 3.500
 +
 +
 +
We have found that increasing the scores of the above selected SpamAssassin scores blocks a lot of spam that would otherwise get through.
 +
 +
As we make updates to our own configurations, we will endeavor to keep this page updated as well.

Revision as of 16:08, 1 September 2013

Admin Article

Article Information

This article applies to the following ZCS versions.

ZCS 8.0 Article ZCS 8.0 ZCS 7.0 Article ZCS 7.0


For SpamAssassin and Anti-spam Updates

This page is currently a draft and in the process of being developed

Customizing SpamAssassin

ZCS 8 and later

For ZCS 8 and later, SpamAssassin scans for all *.cf files in /opt/zimbra/conf/sa and loads them in alphabetical order. If you create a sausers.cf file, it will be loaded after salocal.cf is loaded. This is the supported method for doing customizations of SpamAssassin.

ZCS 6 and later

For ZCS 6 and later, SpamAssassin customizations go in /opt/zimbra/conf/sauser.cf. On upgrade to ZCS8, that file gets moved to /opt/zimbra/conf/ca

Customizing Postfix

Customizing Postfix using the "zmprov ms ..." method survives upgrades. On plain (non-Zimbra) Postfix servers, it is helpful to use certain Postfix restrictions not configurable using the Zimbra administration console. It is both permissible and the recommended way now to perform such customizations, for example:

zmprov ms <server> +zimbraMtaRestriction reject_unknown_reverse_client_hostname

Specific Suggested Tweaks

Last update 1 September 2013 by L. Mark Stone, Reliable Networks

Our client base is very nervous about spam-delivered malware but even more concerned about "false-positives" i.e. legitimate email incorrectly identified as spam. Consequently, we've had to develop tweaks to improve Zimbra's default SpamAssassin configurations. The results have been that users with very public email addresses who typically receive several hundred to more than a thousand emails per day will see no more than ~3 spam emails per day in their Inbox. In our experience, anything less than that and you are likely to wind up with false positives.

If your end-user base is more tolerant of false positives, then you can tighten things up.

Keep in mind that Zimbra's Postfix takes a cut at filtering the email stream before Zimbra's SpamAssassin, and that SpamAssassin's processing of emails is much more resource intensive than Postfix's. Consequently, any filtering that you can do at the Postfix level to block emails outright will be helpful in both blocking spam and lowering resource utilization on your Zimbra server. Just be careful of inducing false positives!

Postfix Tweaks via the Admin Console

At the Postfix level we use just a few complementary and conservative RBLs, one DNS check and one Protocol check. All of these can be configured via the Admin Console: (Global Settings > MTA).

The RBLs we use are:

  • zen.spamhaus.org
  • psbl.surriel.com
  • b.barracudacentral.org

On the same Admin Console page we also enable (and leave the remaining Protocol and DNS checks disabled):

  • reject_non_fqdn_sender
  • reject_unknown_sender_domain (Note this setting will be updated in 8.0.5)

On that same page we also make sure disable "Add X-Originating-IP to messages" as this can block email from remote users with fat email clients like Outlook and Thunderbird on home and public networks like Internet cafes (ZWC clients are unaffected by this.)

SpamAssassin Tweaks via the Commandline

Our current recommended SpamAssassin customizations comprise three complementary methods:

  1. Increase the log level reported by Amavis to get clarity from SpamAssassin on why/how spam is being blocked and getting through.
  2. Put Amavis's temporary directory on a RAM disk to speed up processing.
  3. Tweak the scores for a few selected individual SpamAssassin tests after installing Pyzor and Razor2.
1. Increase Amavis's Log Level

We found that increasing the log level from 1 to 2 puts in /var/log/zimbra.log the specific SpamAssassin tests which each email has triggered. This tweak is not supported by Zimbra and will not survive Zimbra upgrades.

The file to edit is /opt/zimbra/conf/amavisd.conf.in. You will need to change the file's permissions to be writable, edit the file, then change the permissions back. Probably a good idea to make a backup copy of the file first... The final edit should should look like this:

$log_level = 2; # verbosity 0..5 - 1 is the minimum for msg tracing

Restart amavis for the change to take effect (zmavavisdctl restart).

2. Put Amavis's Temp Dir on a RAM Disk

We have seen even with fast RAID10 arrays that Amavis's processing an email with large attachments through SpamAssassin can take as long as 10-20 seconds. Putting Amavis'd temp directory on a RAM disk cuts this down to 1-2 seconds. Ralf Hildebrandt's book on Postfix has a section describing how to size the RAM disk, and why this entirely safe for mail flow even in the event of a server crash. After you've done the homework for sizing, all you need to do is:

  1. Determine how your operating system supports RAM disks (SLES and Ubuntu use /dev/shm built in).
  2. Determine the UID:GID of the zimbra user account.
  3. Stop amavis, mount the RAM disk, start amavis and then edit /etc/fstab to make the change permanent.

To determine the UID:GID for the zimbra user (on each MTA server; the UID:GID pair is unique to each server):

On a SLES system the UID (1004) will be different than the GID (1000) on an Ubuntu or RHEL system where they are both typically the same:

malbec:~ # cat /etc/passwd | grep "zimbra:"
zimbra:x:1004:1000::/opt/zimbra:/bin/bash
malbec:~ #
zimbra@mail2:~$ cat /etc/passwd | grep "zimbra:"
zimbra:x:1001:1001::/opt/zimbra:/bin/bash
zimbra@mail2:~$


Our /etc/fstab entry for a 1GB RAM disks on the server mail2 therefore looks like:

zimbra@mail2:~$ cat /etc/fstab | grep amavis
/dev/shm /opt/zimbra/data/amavisd/tmp tmpfs defaults,noexec,nodev,nosuid,size=1024m,mode=750,uid=1001,gid=1001 0 0
zimbra@mail2:~$


3. Tweak Selected SpamAssasin Scores After Installing Pyzor and Razor2

[Need to add section for installing and initializing Pyzor and Razor2 on Ubuntu Server].

After installing Pyzor and Razor2 and restarting Zimbra's Amavis to make sure these modules are loaded by SpamAssassin, we add the following to the appropriate custom SpamAssassin configuration file:

score URIBL_BLACK 2.725
score RAZOR2_CHECK 2.750
score PYZOR_CHECK 2.750
score BAYES_99 4.000
score RDNS_NONE 3.500


We have found that increasing the scores of the above selected SpamAssassin scores blocks a lot of spam that would otherwise get through.

As we make updates to our own configurations, we will endeavor to keep this page updated as well.

Jump to: navigation, search