Difference between revisions of "Improving Anti-spam system"

m (Meta Rules: grammar)
Line 362: Line 362:
  
 
I think we should prepare a script to save and restore this config changes upon zimbra updates...
 
I think we should prepare a script to save and restore this config changes upon zimbra updates...
 +
 +
 +
== Postfix Tweaks ==
 +
(Added by L. Mark Stone 12 May 2007)
 +
 +
Postfix itself features a number of anti-UCE capabilities.  Some of them are available via the admin console, but some are not. 
 +
 +
=== Simultaneous Connection Throttling ===
 +
If your Zimbra system gets targeted by spammers, you'll notice that a spammer's email server can open up a large number of simultaneous connections to Zimbra's Postfix.
 +
 +
Most of these connections will fail, often because the recipients don't actually exist on the system. But, these connections still use resources.
 +
 +
So, we have for years on our other Postfix mail servers been taking advantage of two Postfix configuration settings that have reduced this problem significantly. We have now updated our Zimbra installations with the same settings, so I thought I would pass them on.
 +
 +
The two settings we add to main.cf are:<br>
 +
<pre>
 +
smtpd_soft_error_limit = 2<br>
 +
smtpd_hard_error_limit = 3<br>
 +
</pre>
 +
 +
We do this by becoming the zimbra user and then running:
 +
<pre>
 +
postconf -e 'smtpd_hard_error_limit = 3'<br>
 +
postconf -e 'smtpd_soft_error_limit = 2'<br>
 +
</pre>
 +
 +
We then restart Postfix to implement the changes.
 +
 +
Documentation from Postfix is here:
 +
http://www.postfix.org/postconf.5.html#smtpd_soft_error_limit

Revision as of 17:14, 12 May 2007

These are some community suggested methods for improving Zimbra's anti-spam system. They have met with some success, but are not tested by Zimbra, and are not supported by Zimbra support. Additionally, changes made here will likely not persist across upgrades.

--ApolloDS 02:35, 10 February 2007 (CST) I setup today the following also on Zimbra 4.5.1 and it works without problems.

salocal.cf.in

The easiest way to "tweak" your spamassassin filtering setup is to edit your /opt/zimbra/conf/salocal.cf.in file, which is designed specifically for "local" (ie user) configuration. Modifying only this file makes for easier upgrades, and protects your spamassasin installation from inadvertant destruction. Keep in mind, however, that Zimbra replaces this file on every upgrade, so you'll want to create a cron job to copy it to the backup folder (or some other safe location) on a regular basis.

Blacklists and Whitelists

The simplest filtering methods for spamassasin are the blacklist and whitelist. Blacklist entries block all email from an address or domain, and whitelist entries bypass all filtering for an address or domain. To add blackist or whitelist entries to your salocal.cf.in file, simply add lines in the following format:

blacklist_from sales@traveloforange.com
whitelist_from bill@yahoo.net
blacklist_from *@emn-mysavingsnow.net

Note that * is a wildcard. In this example *@emn-mysavingsnow.net indicates all email from any user at emn-mysavingsnow.net.

When you are finished editing the salocal.cf.in file, restart Zimbra spamassassin by issuing the following command at the server prompt (as the zimbra user):

zmamavisdctl restart

Basic Rules

Spamassasin works by reading the headers and content of an email, and applying rules to that content. Rules can be in the form of a particular word or phrase, as well as a variety of built in functions. When a rule is "hit" while evaluating an email, a point score is added to that email's total score. When an emails total score exceeds a certain threshold (typically 5 on a Zimbra system,) the email is either marked as spam, or, if the score is high enough, deleted automatically.

Rules are in the form of a test followed by a score. The rule mechanism typically uses perl regular expressions to search for specific content within an email. Custom rules should be added to the salocal.cf.in file in the following format:

body LOCAL_RULE    /sale/
score LOCAL_RULE   0.5

The above text creates a rule called LOCAL_RULE that searches the body of the message for the word "sale" in lower case. If it finds the word "sale" anywhere in the body, it adds 0.5 to the total score of the email. Note that the score is only applied once - multiple instances of the word "sale" in the same email will not be scored separately. Also note that you should always precede the name of your own rules with the word LOCAL, as in the example above, to distinguish them from built in spamassasin rules, and prevent accidental duplicate names.

Perl regular expressions are quite a powerful mechanism for locating text. Some additional examples of perl regular expression based rules:

body LOCAL_SALE    /sale/i
performs a case-insensitive search for the word "sale"
body LOCAL_STOCK1    /^hot stock tip/i
searches for a line that starts with the words "hot stock tip" in any case
body LOCAL_4CAPS    /[A-Z][A-Z][A-Z][A-Z]/
searches for any 4 capital letters in a row (generally a stock symbol)
body LOCAL_MONEY    /\d?\d?\d?.\d\d\b/
searches for 3 digits, a decimal point, and 2 more digits, and treats as a word

Google for "perl regular expressions" for help constructing your spamassassin rules.

You can also search headers for values, and assign a score to them, using the following format:

header LOCAL_LOCALHOST reply-to =~ /@localhost/

where "LOCAL_LOCALHOST" is the rule name and "reply-to" is the header field name. The above rule would generate a "hit" if "@localhost" exists anywhere in the header field "reply-to." You can easily view several header options in Zimbra by right clicking on an email in the message list, and choosing "Show Original" from the context menu.

URIs can be detected as well in the content of an email. URI rules are in the following format:

uri LOCAL_SALES    /sales/

The above would generate a "hit" only in a URI that has the word "sales" in it, but would not hit on the word "sales" if it does not appear in a URI.

Meta Rules

You can also search for a combination of rules, and apply a score to that combination by creating a "meta" rule, in the following format:

body LOCAL_FOUR_CAPS    /[A-Z][A-Z][A-Z][A-Z]/ 
body LOCAL_MONEY        /\d?\d?\d?.\d\d\b/
meta LOCAL_STOCK        (LOCAL_MONEY && LOCAL_FOUR_CAPS)
score LOCAL_STOCK       1

The above rule would add 1 to an email's score only if both "LOCAL_FOUR_CAPS" AND "LOCAL_MONEY" were hits. Be careful when creating meta rules, as it is easy to "over-score" and email," such as in the case of the following:

body LOCAL_FOUR_CAPS    /[A-Z][A-Z][A-Z][A-Z]/
score LOCAL_FOUR_CAPS   1 
body LOCAL_MONEY        /\d?\d?\d?.\d\d\b/
score LOCAL_MONEY       1
meta LOCAL_STOCK        (LOCAL_MONEY && LOCAL_FOUR_CAPS)
score LOCAL_STOCK       1

The above could add 3 points to the email score, if the meta rule hits.

When you are finished editing the salocal.cf.in file, restart Zimbra spamassassin by issuing the following command at the server prompt (as the zimbra user):

zmamavisdctl restart

SPF

First of all, zimbra spamassassin has no SPF enabled. Since perl enviroment is system way integrated, adding SPF support is fair simple. If you use RedHat or CentOS, you can "yum-it" from dag.wieers.com repositories by executing:

yum install perl-Mail-SPF-Query

For every other platform you can install SPF by opening, and configuring in case you didn't, cpan command line utility and executing:

perl -MCPAN -eshell
install Mail::SPF::Query

Razor2

Second, we added Razor2 in order to improve score.

Installing Razor

CentOS

As root: Get razor-agents-sdk from razor.sourceforge.net, untar it and

perl Makefile.PL
make
make install

Get also razor-agents from razor.sourceforge.net, untar it and

perl Makefile.PL
make 
make install

Open your firewall ports for razor2 (TCP/2703 outgoing).

Fedora

Downloading Packages:
(1/2): perl-Razor-Agent-2 100% |=========================|  84 kB    00:07     
(2/2): razor-agents-2.81- 100% |=========================|  51 kB    00:06     
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing: perl-Razor-Agent             ######################### [1/2] 
  Installing: razor-agents                 ######################### [2/2] 

Installed: razor-agents.i386 0:2.81-2.fc5.rf
Dependency Installed: perl-Razor-Agent.i386 0:2.81-2.fc5.rf
Complete!

Configuring Razor

Create .razor folder in /opt/zimbra/amavisd and give zimbra user permissions

mkdir /opt/zimbra/amavisd/.razor; chown -Rf zimbra:zimbra /opt/zimbra/amavisd/.razor

As zimbra user, create your razor account:

razor-admin -home=/opt/zimbra/amavisd/.razor -create 
razor-admin -home=/opt/zimbra/amavisd/.razor -discover 
razor-admin -home=/opt/zimbra/amavisd/.razor -register


And finally enable razor. Edit /opt/zimbra/conf/spamassassin/v310.pre and uncomment line

loadplugin Mail::SpamAssassin::Plugin::Razor2

Pyzor

Now we are going to add pyzor support for increase (again) spam score

Installing Pyzor

CentOS

As root, install python support.

yum install python

Get pyzor package from pyzor.sourceforge.net, untar it and:

python setup.py build 
python setup.py install

Set perms according with pyzor readme.

chmod -R a+rX /usr/share/doc/pyzor /usr/lib/python2.3/site-packages/pyzor /usr/bin/pyzor /usr/bin/pyzord

Fedora

As root, install pyzor RPM. It's included in the extra Repository of Fedora.

yum install pyzor
.
.
Downloading Packages:
(1/1): pyzor-0.4.0-10.fc5 100% |=========================|  65 kB    00:01     
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing: pyzor                        ######################### [1/1] 

Installed: pyzor.noarch 0:0.4.0-10.fc5
Complete!

SUSE 10

As root, install python and python-devel via yast2 Software -> Software Management menu.

Get pyzor package from pyzor.sourceforge.net, untar it and:

python setup.py build 
python setup.py install

Set perms according with pyzor readme.

chmod -R a+rX /usr/local/share/doc/pyzor /usr/local/lib/python2.4/site-packages/pyzor /usr/local/bin/pyzor /usr/local/bin/pyzord

Configuring Pyzor

Create .pyzor folder into zimbra-amavisd home and set perms

mkdir /opt/zimbra/amavisd/.pyzor; chown zimbra:zimbra /opt/zimbra/amavisd/.pyzor

Open your firewall ports for pyzor (UDP/24441 outgoing)

And ready to go, as zimbra user, with:

pyzor --homedir /opt/zimbra/amavisd/.pyzor discover



Spamassassin Config

Now we have PYZOR + RAZOR + SPF. But it would be advisable to enable it and give SPF a higher score. Those admins with wrong SPF entries should be punished since it's not mandatory and so, if you enable it, do it well. So open your spamassassin config at /opt/zimbra/conf/spamassassin/local.cf and add this rules at the end (customize it at your own):

ok_languages en es 
ok_locales en es 
trusted_networks 127. 10.70. 192.168.
use_bayes 1
skip_rbl_checks 0
use_razor2 1
#use_dcc 1 <<< WORK IN PROGRESS
use_pyzor 1 
dns_available yes 

## Optional Score Increases 
## Choose your preferred values...
score DCC_CHECK 4.000
score SPF_FAIL 10.000  
score SPF_HELO_FAIL 10.000
score RAZOR2_CHECK 2.500
score PYZOR_CHECK 2.500
score BAYES_99 4.300
score BAYES_90 3.500
score BAYES_80 3.000
bayes_ignore_header Received: from mail3.example.com
bayes_ignore_header Received: from localhost
bayes_ignore_header Received: from mail1.example.com
bayes_ignore_header Received: from mail2.example.com

Amavisd Config

Some notes about this: In zimbra, by default, spam with 15 score of higher is discarded by amavisd. If you want your user receive these mails, you have to modify amavisd.conf settings (/opt/zimbra/conf/amavisd.conf) in order to pass this email.

$final_spam_destiny = D_PASS


Enabling DCC

To setup DCC: Download dcc from DCC Site

I compile on different system to build an rpm to install in production environment. Use this spec file (rename it to .spec) to build an rpm with the command:

rpmbuild -ba /usr/src/redhat/SPECS/dcc.spec

install it on the production server:

rpm -ivh dcc-x.y.z.rpm

Change /etc/dcc/dcc_conf to read:

DCCUID=zimbra 
DCCD_ENABLE=off

Change /opt/zimbra/conf/spamassassin/v310.pre to enable the DCC plugin:

 
loadplugin Mail::SpamAssassin::Plugin::DCC 

Enable DCC on firewall (UDP/6277 outgoing)

Have fun. I use sqlgrey as greylist server, so I don't need another one. As to me the standard value DCC == 2.5 Spamassassin point is ok, so I do not change it. With SA 3.xx you do not need to use enable_dcc in local.cf. That's the same for razor2 indeed...

Implementing Whitelist/Blacklist

Domain white/black list

This can be accomplished by modifying /opt/zimbra/conf/amavisd.conf.in and adding a score for the domain that you want to change.

When scoring the domain, remember that negative scores whitelist, positive scores blacklist

Here's a whitelisting example:

Edit the file /opt/zimbra/conf/amavisd.conf.in and look for this section:

  { # a hash-type lookup table (associative array)
    'nobody@cert.org'                        => -3.0,
    'cert-advisory@us-cert.gov'              => -3.0,
    'owner-alert@iss.net'                    => -3.0,
    'slashdot@slashdot.org'                  => -3.0,
    'bugtraq@securityfocus.com'              => -3.0,
    'ntbugtraq@listserv.ntbugtraq.com'       => -3.0,
    'security-alerts@linuxsecurity.com'      => -3.0,


At the top, add the domain you want to whitelist (eg, zimbra.com), with a strong negative score:

  { # a hash-type lookup table (associative array)
    'zimbra.com'                              => -10.0,
    'nobody@cert.org'                        => -3.0,
    'cert-advisory@us-cert.gov'              => -3.0,
    'owner-alert@iss.net'                    => -3.0,
    'slashdot@slashdot.org'                  => -3.0,
    'bugtraq@securityfocus.com'              => -3.0,
    'ntbugtraq@listserv.ntbugtraq.com'       => -3.0,
    'security-alerts@linuxsecurity.com'      => -3.0,
    'mailman-announce-admin@python.org'      => -3.0,

Remember, if you want to blacklist a domain, make the score positive

Then restart amavis:

zmamavisdctl stop
zmamavisdctl start 

Remember - you're trusting the sender's domain to be valid, so any email sent with an address in that domain will receive the score weighting - the address is not verified.

This can also be used with individual sender email addresses, as seen above.

User white/black list

It very simple changing amavis config:

put in /opt/zimbra/conf/amavis.conf*

read_hash(\%whitelist_sender, '/etc/zimbra/whitelist');
read_hash(\%blacklist_sender, '/etc/zimbra/blacklist');
read_hash(\%spam_lovers, '/etc/zimbra/spamlovers');

In /etc/zimbra/* put sender address or domain, one per line. Wildcards allowed. Example:

hotstuff@sexnzen.com
spammersites.net

A spamlovers list is for that accounts that always need to receive all messages, even if spam. According to rfc 2822 postmaster, abuse and other account of this kind should be spam lovers.


I think we should prepare a script to save and restore this config changes upon zimbra updates...


Postfix Tweaks

(Added by L. Mark Stone 12 May 2007)

Postfix itself features a number of anti-UCE capabilities. Some of them are available via the admin console, but some are not.

Simultaneous Connection Throttling

If your Zimbra system gets targeted by spammers, you'll notice that a spammer's email server can open up a large number of simultaneous connections to Zimbra's Postfix.

Most of these connections will fail, often because the recipients don't actually exist on the system. But, these connections still use resources.

So, we have for years on our other Postfix mail servers been taking advantage of two Postfix configuration settings that have reduced this problem significantly. We have now updated our Zimbra installations with the same settings, so I thought I would pass them on.

The two settings we add to main.cf are:

smtpd_soft_error_limit = 2<br>
smtpd_hard_error_limit = 3<br>

We do this by becoming the zimbra user and then running:

postconf -e 'smtpd_hard_error_limit = 3'<br>
postconf -e 'smtpd_soft_error_limit = 2'<br>

We then restart Postfix to implement the changes.

Documentation from Postfix is here: http://www.postfix.org/postconf.5.html#smtpd_soft_error_limit

Jump to: navigation, search