Improving Anti-spam system: Difference between revisions

m (→‎Discarding Emails Sent to Invalid Addresses: 5.0.12 policyd for alias domains)
m (Adding category)
Line 841: Line 841:
[[Category:RHEL]]
[[Category:RHEL]]
[[Category:Ubuntu]]
[[Category:Ubuntu]]
[[Category:ZCS 4.5]]

Revision as of 23:05, 7 December 2009


These are some community suggested methods for improving Zimbra's anti-spam system. They have met with some success, but are not tested/supported by Zimbra. Additionally, changes made here will likely not persist across upgrades.

--ApolloDS 02:35, 10 February 2007 (CST) I setup today the following also on Zimbra 4.5.1 and it works without problems.

--Centurion 16:40, 15 June 2007 (Aust EST) I succesfully added a custom rule set to /opt/zimbra/conf/spamassassin (on our Zimbra 4.5.5 server). The rules were a series of files ending in ".cf" and contained known-working SpamAssassin rules. Seeing as I couldn't find any way to "lint" the rules from any code under /opt/zimbra, I simply used a different machine with the same version of SpamAssassin. Storing all your rules in salocal.cf.in can get cumbersome as your custom rules grow - so create ".cf" files, drop the rules into /opt/zimbra/conf/spamassassin and you're done. Also there are plenty of pre-written rules you can use to extend the Zimbra SpamAssassin rules, such as, SpamAssassin's List and SARE.

salocal.cf.in

The easiest way to "tweak" your spamassassin filtering setup is to edit your /opt/zimbra/conf/salocal.cf.in file, which is designed specifically for "local" (ie user) configuration. Modifying only this file makes for easier upgrades, and protects your spamassasin installation from inadvertant destruction. Keep in mind, however, that Zimbra replaces this file on every upgrade, so you'll want to create a cron job to copy it to the backup folder (or some other safe location) on a regular basis.

Blacklists and Whitelists

The simplest filtering methods for spamassasin are the blacklist and whitelist. Blacklist entries block all email from an address or domain, and whitelist entries bypass all filtering for an address or domain. To add blackist or whitelist entries to your salocal.cf.in file, simply add lines in the following format:

blacklist_from sales@traveloforange.com
whitelist_from bill@yahoo.net
blacklist_from *@emn-mysavingsnow.net

Note that * is a wildcard. In this example *@emn-mysavingsnow.net indicates all email from any user at emn-mysavingsnow.net.

When you are finished editing the salocal.cf.in file, restart Zimbra spamassassin by issuing the following command at the server prompt (as the zimbra user):

zmamavisdctl restart

Basic Rules

Spamassasin works by reading the headers and content of an email, and applying rules to that content. Rules can be in the form of a particular word or phrase, as well as a variety of built in functions. When a rule is "hit" while evaluating an email, a point score is added to that email's total score. When an emails total score exceeds a certain threshold (typically 5 on a Zimbra system) the email is either marked as spam, or, if the score is high enough, deleted automatically.

Rules are in the form of a test followed by a score. The rule mechanism typically uses perl regular expressions to search for specific content within an email. Custom rules should be added to the salocal.cf.in file in the following format:

body LOCAL_RULE    /sale/
score LOCAL_RULE   0.5

The above text creates a rule called LOCAL_RULE that searches the body of the message for the word "sale" in lower case. If it finds the word "sale" anywhere in the body, it adds 0.5 to the total score of the email. Note that the score is only applied once - multiple instances of the word "sale" in the same email will not be scored separately. Also note that you should always precede the name of your own rules with the word LOCAL, as in the example above, to distinguish them from built in spamassasin rules, and prevent accidental duplicate names.

Perl regular expressions are quite a powerful mechanism for locating text. Some additional examples of perl regular expression based rules:

body LOCAL_SALE    /sale/i

performs a case-insensitive search for the word "sale"

body LOCAL_STOCK1    /^hot stock tip/i

searches for a line that starts with the words "hot stock tip" in any case

body LOCAL_4CAPS    /[A-Z][A-Z][A-Z][A-Z]/

searches for any 4 capital letters in a row (generally a stock symbol)

body LOCAL_MONEY    /\d?\d?\d?.\d\d\b/

searches for 3 digits, a decimal point, and 2 more digits, and treats as a word

Google for "perl regular expressions" for help constructing your spamassassin rules.

You can also search headers for values, and assign a score to them, using the following format:

header LOCAL_LOCALHOST reply-to =~ /@localhost/

where "LOCAL_LOCALHOST" is the rule name and "reply-to" is the header field name. The above rule would generate a "hit" if "@localhost" exists anywhere in the header field "reply-to." You can easily view several header options in Zimbra by right clicking on an email in the message list, and choosing "Show Original" from the context menu.

URIs can be detected as well in the content of an email. URI rules are in the following format:

uri LOCAL_SALES    /sales/

The above would generate a "hit" only in a URI that has the word "sales" in it, but would not hit on the word "sales" if it does not appear in a URI.

Meta Rules

You can also search for a combination of rules, and apply a score to that combination by creating a "meta" rule, in the following format:

body LOCAL_FOUR_CAPS    /[A-Z][A-Z][A-Z][A-Z]/ 
body LOCAL_MONEY        /\d?\d?\d?.\d\d\b/
meta LOCAL_STOCK        (LOCAL_MONEY && LOCAL_FOUR_CAPS)
score LOCAL_STOCK       1

The above rule would add 1 to an email's score only if both "LOCAL_FOUR_CAPS" AND "LOCAL_MONEY" were hits. Be careful when creating meta rules, as it is easy to "over-score" and email," such as in the case of the following:

body LOCAL_FOUR_CAPS    /[A-Z][A-Z][A-Z][A-Z]/
score LOCAL_FOUR_CAPS   1 
body LOCAL_MONEY        /\d?\d?\d?.\d\d\b/
score LOCAL_MONEY       1
meta LOCAL_STOCK        (LOCAL_MONEY && LOCAL_FOUR_CAPS)
score LOCAL_STOCK       1

The above could add 3 points to the email score, if the meta rule hits.

When you are finished editing the salocal.cf.in file, restart Zimbra spamassassin by issuing the following command at the server prompt (as the zimbra user):

zmamavisdctl restart

Class A IP Address Blocks

For mail servers in Unites States, below is a list of Class "A" blocks of IP's registered to non-ARIN entities. I also have US ISP's that have been bad in the past, so have added IP's using format examples below. Since it's one of those YMMV things, am only including the Non-ARIN Class "A" blocks below for starters. Arin's Website

As 'root' : vi /opt/zimbra/conf/salocal.cf.in

Copy and paste below in salocal.cf.in and save. Then,

su zimbra
zmamavisdctl restart
header    LOCAL_RULE_RIPE_1   Received =~ /\[25\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_1   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_1   4.123

header    LOCAL_RULE_RIPE_4   Received =~ /\[62\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_4   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_4   4.123

header    LOCAL_RULE_RIPE_5   Received =~ /\[80\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_5   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_5   4.123

header    LOCAL_RULE_RIPE_6   Received =~ /\[81\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_6   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_6   4.123

header    LOCAL_RULE_RIPE_7   Received =~ /\[82\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_7   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_7   4.123

header    LOCAL_RULE_RIPE_8   Received =~ /\[83\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_8   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_8   4.123

header    LOCAL_RULE_RIPE_9   Received =~ /\[84\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_9   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_9   4.123

header    LOCAL_RULE_RIPE_10   Received =~ /\[85\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_10   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_10   4.123

header    LOCAL_RULE_RIPE_11   Received =~ /\[86\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_11   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_11   4.123

header    LOCAL_RULE_RIPE_12   Received =~ /\[87\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_12   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_12   4.123

header    LOCAL_RULE_RIPE_13   Received =~ /\[88\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_13   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_13   4.123

header    LOCAL_RULE_RIPE_14   Received =~ /\[89\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_14   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_14   4.123

header    LOCAL_RULE_RIPE_15   Received =~ /\[80\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_15   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_15   4.123

header    LOCAL_RULE_RIPE_16   Received =~ /\[90\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_16   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_16   4.123

header    LOCAL_RULE_RIPE_17   Received =~ /\[91\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_17   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_17   4.123

header    LOCAL_RULE_RIPE_18   Received =~ /\[188\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_18   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_18   4.123

header    LOCAL_RULE_RIPE_19   Received =~ /\[193\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_19   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_19   4.123

header    LOCAL_RULE_RIPE_20   Received =~ /\[194\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_20   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_20   4.123

header    LOCAL_RULE_RIPE_21   Received =~ /\[195\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_21   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_21   4.123

header    LOCAL_RULE_RIPE_22   Received =~ /\[212\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_22   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_22   4.123

header    LOCAL_RULE_RIPE_23   Received =~ /\[213\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_23   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_23   4.123

header    LOCAL_RULE_RIPE_24   Received =~ /\[217\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_RIPE_24   Spam passed through possible spammer relay
score     LOCAL_RULE_RIPE_24   4.123

header    LOCAL_RULE_APNIC_1   Received =~ /\[58\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_1   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_1   4.123

header    LOCAL_RULE_APNIC_2   Received =~ /\[59\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_2   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_2   4.123

header    LOCAL_RULE_APNIC_3   Received =~ /\[60\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_3   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_3   4.123

header    LOCAL_RULE_APNIC_4   Received =~ /\[61\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_4   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_4   4.123

header    LOCAL_RULE_APNIC_5   Received =~ /\[121\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_5   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_5   4.123

header    LOCAL_RULE_APNIC_6   Received =~ /\[122\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_6   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_6   4.123

header    LOCAL_RULE_APNIC_7   Received =~ /\[123\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_7   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_7   4.123

header    LOCAL_RULE_APNIC_8   Received =~ /\[124\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_8   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_8   4.123

header    LOCAL_RULE_APNIC_9   Received =~ /\[125\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_9   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_9   4.123

header    LOCAL_RULE_APNIC_10   Received =~ /\[126\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_10   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_10  4.123

header    LOCAL_RULE_APNIC_11   Received =~ /\[202\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_11   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_11   4.123

header    LOCAL_RULE_APNIC_12   Received =~ /\[203\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_12   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_12   4.123

header    LOCAL_RULE_APNIC_13   Received =~ /\[210\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_13   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_13   4.123

header    LOCAL_RULE_APNIC_14   Received =~ /\[211\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_14   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_14   4.123

header    LOCAL_RULE_APNIC_15   Received =~ /\[218\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_15   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_15   4.123

header    LOCAL_RULE_APNIC_16   Received =~ /\[219\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_16   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_16   4.123

header    LOCAL_RULE_APNIC_17   Received =~ /\[220\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_17   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_17   4.123

header    LOCAL_RULE_APNIC_18   Received =~ /\[221\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_18   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_18   4.123

header    LOCAL_RULE_APNIC_19   Received =~ /\[222\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_APNIC_19   Spam passed through possible spammer relay
score     LOCAL_RULE_APNIC_19   4.123

header    LOCAL_RULE_JPNIC_1   Received =~ /\[43\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_JPNIC_1   Spam passed through possible spammer relay
score     LOCAL_RULE_JPNIC_1   4.123

header    LOCAL_RULE_JPNIC_2   Received =~ /\[133\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_JPNIC_2   Spam passed through possible spammer relay
score     LOCAL_RULE_JPNIC_2   4.123

header    LOCAL_RULE_AFRINIC_1   Received =~ /\[41\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_AFRINIC_1   Spam passed through possible spammer relay
score     LOCAL_RULE_AFRINIC_1   4.123

header    LOCAL_RULE_AFRINIC_2   Received =~ /\[196\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_AFRINIC_2   Spam passed through possible spammer relay
score     LOCAL_RULE_AFRINIC_2   4.123

header    LOCAL_RULE_LACNIC_1   Received =~ /\[189\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_LACNIC_1   Spam passed through possible spammer relay
score     LOCAL_RULE_LACNIC_1   4.123

header    LOCAL_RULE_LACNIC_2   Received =~ /\[190\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_LACNIC_2   Spam passed through possible spammer relay
score     LOCAL_RULE_LACNIC_2   4.123

header    LOCAL_RULE_LACNIC_3   Received =~ /\[200\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_LACNIC_3   Spam passed through possible spammer relay
score     LOCAL_RULE_LACNIC_3   4.123

header    LOCAL_RULE_LACNIC_4   Received =~ /\[201\.\d{1,3}\.\d{1,3}\.\d{1,3}\]/
describe  LOCAL_RULE_LACNIC_4   Spam passed through possible spammer relay
score     LOCAL_RULE_LACNIC_4   4.123

SPF

First of all, zimbra spamassassin has no SPF enabled. Since perl enviroment is system way integrated, adding SPF support is fair simple.

Read more about SPF and how to configure it for your domain: http://www.openspf.org/

Also, there are tools for testing your SPF settings: http://www.openspf.org/Tools

RedHat/CentOS

If you use RedHat or CentOS, you can "yum-it" from dag.wieers.com repositories by executing:

yum install perl-Mail-SPF

Important note: perl-Mail-SPF and perl-Mail-SPF-Query are mutually exclusive. If you install one, don't install the other. Per this thread (http://lists.rpmforge.net/pipermail/users/2007-October/thread.html#1044), it appears that perl-Mail-SPF-Query has not been updated since Feb 2006, while perl-Mail-SPF was updated as recently as May 2007 and is more RFC compliant. So, the recommendation is to use perl-Mail-SPF and not perl-Mail-SPF-Query.

You can add the dag.wieers.com repository to your server by installing an rpm for your system. See:

http://dag.wieers.com/rpm/FAQ.php#B2

Ubuntu 6.06LTS (Dapper)

sudo apt-get install libmail-spf-query-perl

Other

For every other platform you can install SPF by opening, and configuring in case you didn't, cpan command line utility and executing:

perl -MCPAN -eshell
install Mail::SPF::Query

Razor2

Second, we added Razor2 in order to improve score.

Installing Razor

CentOS

There are several ways to install Razor-Agent. Two common ways are listed below:

Yum / RPM package

The perl-Razor-Agent is available through Dag Wiers apt/yum repository:

You will need to configure yum to use Dag Wiers repository for your Release and Architecture which is outside the scope of this document (google rpmforge-release). Enable Dag's repository and append the following line to Dag's repository section:

includepkgs=perl-Razor-Agent perl-Digest-HMAC perl-Digest-SHA1 perl-Net-DNS perl-Net-IP razor-agents

Install the Razor-Agnet and its dependencies:

# yum install perl-Razor-Agent razor-agents

Alternatively you can download the specific packages directly from Dag's mirrors and install manually with the rpm command. The downside is you are not notified if there is a patch or update to these packages.

Open your firewall ports for razor2 (TCP/2703 outgoing).


Compile

As root: Get razor-agents-sdk from razor.sourceforge.net, untar it and

perl Makefile.PL
make
make install

Get also razor-agents from razor.sourceforge.net, untar it and

perl Makefile.PL
make 
make install

Open your firewall ports for razor2 (TCP/2703 outgoing).

Fedora

Downloading Packages:
(1/2): perl-Razor-Agent-2 100% |=========================|  84 kB    00:07     
(2/2): razor-agents-2.81- 100% |=========================|  51 kB    00:06     
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing: perl-Razor-Agent             ######################### [1/2] 
  Installing: razor-agents                 ######################### [2/2] 

Installed: razor-agents.i386 0:2.81-2.fc5.rf
Dependency Installed: perl-Razor-Agent.i386 0:2.81-2.fc5.rf
Complete!

Configuring Razor

Create .razor folder in /opt/zimbra/amavisd and give zimbra user permissions

mkdir /opt/zimbra/amavisd/.razor; chown -Rf zimbra:zimbra /opt/zimbra/amavisd/.razor

As zimbra user, create your razor account:

razor-admin -home=/opt/zimbra/amavisd/.razor -create 
razor-admin -home=/opt/zimbra/amavisd/.razor -discover 
razor-admin -home=/opt/zimbra/amavisd/.razor -register


And finally enable razor. Edit /opt/zimbra/conf/spamassassin/v310.pre and uncomment line

loadplugin Mail::SpamAssassin::Plugin::Razor2

Pyzor

Now we are going to add pyzor support for increase (again) spam score

Installing Pyzor

CentOS

As root, install python support.

yum install python

Get pyzor package from pyzor.sourceforge.net, untar it and:

python setup.py build 
python setup.py install

Set perms according with pyzor readme.

chmod -R a+rX /usr/share/doc/pyzor /usr/lib/python2.3/site-packages/pyzor /usr/bin/pyzor /usr/bin/pyzord

Set perms for RHEL 5 x86_64 slightly different than above

chmod -R a+rX /usr/share/doc/pyzor /usr/lib/python2.4/site-packages/pyzor/usr/local/bin/pyzor /usr/bin/pyzord

Fedora

As root, install pyzor RPM. It's included in the extra Repository of Fedora.

yum install pyzor
.
.
Downloading Packages:
(1/1): pyzor-0.4.0-10.fc5 100% |=========================|  65 kB    00:01     
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing: pyzor                        ######################### [1/1] 

Installed: pyzor.noarch 0:0.4.0-10.fc5
Complete!

SUSE 10

As root, install python and python-devel via yast2 Software -> Software Management menu.

Get pyzor package from pyzor.sourceforge.net, untar it and:

python setup.py build 
python setup.py install

Set perms according with pyzor readme.

chmod -R a+rX /usr/local/share/doc/pyzor /usr/local/lib/python2.4/site-packages/pyzor /usr/local/bin/pyzor /usr/local/bin/pyzord

Configuring Pyzor

Create .pyzor folder into zimbra-amavisd home and set perms

mkdir /opt/zimbra/amavisd/.pyzor; chown zimbra:zimbra /opt/zimbra/amavisd/.pyzor

Open your firewall ports for pyzor (UDP/24441 outgoing)

And ready to go, as zimbra user, with:

pyzor --homedir /opt/zimbra/amavisd/.pyzor discover



Spamassassin Config

Now we have PYZOR + RAZOR + SPF. But it would be advisable to enable it and give SPF a higher score. Those admins with wrong SPF entries should be punished since it's not mandatory and so, if you enable it, do it well. So open your spamassassin config at /opt/zimbra/conf/spamassassin/local.cf and add this rules at the end (customize it at your own):

ok_languages en es 
ok_locales en es 
trusted_networks 127. 10.70. 192.168.
use_bayes 1
skip_rbl_checks 0
use_razor2 1
#use_dcc 1 <<< WORK IN PROGRESS
use_pyzor 1 
dns_available yes 

## Optional Score Increases 
## Choose your preferred values...
score DCC_CHECK 4.000
score SPF_FAIL 10.000  
score SPF_HELO_FAIL 10.000
score RAZOR2_CHECK 2.500
score PYZOR_CHECK 2.500
score BAYES_99 4.300
score BAYES_90 3.500
score BAYES_80 3.000
bayes_ignore_header Received: from mail3.example.com
bayes_ignore_header Received: from localhost
bayes_ignore_header Received: from mail1.example.com
bayes_ignore_header Received: from mail2.example.com

Note that these numbers can be made even higher if you want the particular filter to have more weight. Check your headers and adjust as needed to achieve the desired result.

required_score

To tweak the required_score parameter in Zimbra you don't need to edit any config file. This value is calculated from a setting in Zimbra admin page. Enter administration, go to Global Settings >> AV/AS. The required_score is tag percent * 0,2. So a tag percent value of 25 will result in a required score of 5 (25*0,5=5).


Externally-Maintained Whitelists

Even with the Bayes configurations above, some messages with high Bayes scores get through due to the existence of several externally-maintained whitelists. Essentially these are programs whereby those who subscribe to the program--for a price and agreement to follow certain rules of conduct--get a pass to send unsolicited messages. Spamassassin uses these trusted lists to REDUCE your spam score by assigning a negative point score to the message, which offsets the positive (i.e. "spammy") scores that might result from other filters in your system.

Some of these lists, such as dnswl.org, are maintained by an all-volunteer group; others, such as the Bonded Sender Program (now known as SenderScoreCertified at www.senderscorecertified.com) and Habeas (www.habeas.com) are commercial enterprises. Each describes their standards on their website; one can, of course, find plenty of heated discussion as to the extent to which the commercial ones enforce their standards.

Without engaging in the debate as to the motives or purity of one list or another, the administrator needs to evaluate each list and determine whether he/she is comfortable having that list's maintainers influence the performance of local spam filters. This section is intended to help the administrator adjust the relative scoring influences of these whitelists if so desired.

As with any technology, the services change with time. It is probably a good discipline to review your SpamAssassin configuration files from time to time (after an update in particular) looking for anything that gives your messages a negative score, so you can evaluate if you want to accept that scoring for your local system.

Bonded Sender Program (BSP)

The Bonded Sender Program is described at www.senderscorecertified.com. Spamassassin gives BSP hits a -4.5 score, which pretty well overrides everything else you've done and makes the message come through anyhow (BSP's own website actually advocates a -100 score!). The following adjustment in your local.cf file can reduce, or if you wish, neutralize, the effect of BSP on your spam scores:

# Score to reduce the effect of Bonded Sender Program (BSP) whitelisting
score RCVD_IN_BSP_TRUSTED -0.500
score RCVD_IN_BSP_OTHER -0.500
score RCVD_IN_BONDEDSENDER -0.500

Change these values to zero and it goes away completely!

Habeas

Habeas, at www.habeas.com, is another such subscription-based whitelisting program. Habeas also recommends a -100 score for the most highly-rated senders in their list, although Spamassassin gives them the more conservative score of -8.0 for the highest-rated senders. A reduced impact score for Habeas (again in local.cf) might look like this:

# Score to reduce the effect of Habeas whitelisting
score HABEAS_ACCREDITED_COI 0 -0.5 0 -0.5
score HABEAS_ACCREDITED_SOI 0 -0.25 0 -0.25
score HABEAS_CHECKED 0 -0.1 0 -0.1

Again, all zeros would completely negate these scores

ISIPP's SuretyMail (IADB)

The Institute for Spam and Internet Public Policy (ISIPP) is another for-profit whitelister whose stated purpose in its marketing materials (www.suretymail.com) is to "Send Legitimate E-mail in a Spam-Filtered World." The ISIPP settings appear in SpamAssassin as IADB, and can be modified as follows:

# Score to reduce the effect of ISIPP/IADB SuretyMail whitelisting
score RCVD_IN_IADB_VOUCHED 0 -0.2 0 -0.2
score RCVD_IN_IADB_DOPTIN 0 -0.4 0 -0.4
score RCVD_IN_IADB_ML_DOPTIN 0 -0.6 0 -0.6

And of course zeros work as well.

dnswl.org

DNSWL is different from the lists described above, in that it is deliberately a noncommercial list, and its maintainers recognize the potential conflict of interest in having an economic incentive to let senders off the hook (see their "background" page to hear it in their own words. Nevertheless, it is conceivable that administrators will find DNSWL's judgment to be allowing messages through local filters in contravention of local policy. DNSWL's default scores in Spamassassin are -1, -4, and -8. Administrators wishing to reduce these could use the following settings:

# Score to reduce the effect of DNSWL whitelisting
score RCVD_IN_DNSWL_LOW 0 -0.1 0 -0.1
score RCVD_IN_DNSWL_MED 0 -0.4 0 -0.4
score RCVD_IN_DNSWL_HI 0 -0.8 0 -0.8

Amavisd Config

Some notes about this: In zimbra, by default, spam with 15 score of higher is discarded by amavisd. If you want your user receive these mails, you have to modify amavisd.conf settings (/opt/zimbra/conf/amavisd.conf) in order to pass this email.

$final_spam_destiny = D_PASS


Enabling DCC

To setup DCC: Download dcc from DCC Site

I compile on different system to build an rpm to install in production environment. Use this spec file (rename it to .spec) to build an rpm with the command:

rpmbuild -ba /usr/src/redhat/SPECS/dcc.spec

install it on the production server:

rpm -ivh dcc-x.y.z.rpm

Change /etc/dcc/dcc_conf to read:

DCCUID=zimbra 
DCCD_ENABLE=off

Change /opt/zimbra/conf/spamassassin/v310.pre to enable the DCC plugin:

 
loadplugin Mail::SpamAssassin::Plugin::DCC 

Enable DCC on firewall (UDP/6277 outgoing)

Have fun. I use sqlgrey as greylist server, so I don't need another one. As to me the standard value DCC == 2.5 Spamassassin point is ok, so I do not change it. With SA 3.xx you do not need to use enable_dcc in local.cf. That's the same for razor2 indeed...

Implementing Whitelist/Blacklist

Domain white/black list

This can be accomplished by modifying /opt/zimbra/conf/amavisd.conf.in and adding a score for the domain that you want to change.

When scoring the domain, remember that negative scores whitelist, positive scores blacklist

Here's a whitelisting example:

Edit the file /opt/zimbra/conf/amavisd.conf.in and look for this section:

  { # a hash-type lookup table (associative array)
    'nobody@cert.org'                        => -3.0,
    'cert-advisory@us-cert.gov'              => -3.0,
    'owner-alert@iss.net'                    => -3.0,
    'slashdot@slashdot.org'                  => -3.0,
    'bugtraq@securityfocus.com'              => -3.0,
    'ntbugtraq@listserv.ntbugtraq.com'       => -3.0,
    'security-alerts@linuxsecurity.com'      => -3.0,


At the top, add the domain you want to whitelist (eg, zimbra.com), with a strong negative score:

  { # a hash-type lookup table (associative array)
    'zimbra.com'                              => -10.0,
    'nobody@cert.org'                        => -3.0,
    'cert-advisory@us-cert.gov'              => -3.0,
    'owner-alert@iss.net'                    => -3.0,
    'slashdot@slashdot.org'                  => -3.0,
    'bugtraq@securityfocus.com'              => -3.0,
    'ntbugtraq@listserv.ntbugtraq.com'       => -3.0,
    'security-alerts@linuxsecurity.com'      => -3.0,
    'mailman-announce-admin@python.org'      => -3.0,

Remember, if you want to blacklist a domain, make the score positive

Then restart amavis:

zmamavisdctl stop
zmamavisdctl start 

Remember - you're trusting the sender's domain to be valid, so any email sent with an address in that domain will receive the score weighting - the address is not verified.

This can also be used with individual sender email addresses, as seen above.

User white/black list

It very simple changing amavis config:

put in /opt/zimbra/conf/amavis.conf*

read_hash(\%whitelist_sender, '/etc/zimbra/whitelist');
read_hash(\%blacklist_sender, '/etc/zimbra/blacklist');
read_hash(\%spam_lovers, '/etc/zimbra/spamlovers');

In /etc/zimbra/* put sender address or domain, one per line. Wildcards allowed. Example:

hotstuff@sexnzen.com
spammersites.net

A spamlovers list is for that accounts that always need to receive all messages, even if spam. According to rfc 2822 postmaster, abuse and other account of this kind should be spam lovers.


I think we should prepare a script to save and restore this config changes upon zimbra updates...


Postfix Tweaks

(Added by L. Mark Stone 12 May 2007)

Postfix itself features a number of anti-UCE capabilities. Some of them are available via the admin console, but some are not.

Simultaneous Connection Throttling

If your Zimbra system gets targeted by spammers, you'll notice that a spammer's email server can open up a large number of simultaneous connections to Zimbra's Postfix.

Most of these connections will fail, often because the recipients don't actually exist on the system. But, these connections still use resources.

So, we have for years on our other Postfix mail servers been taking advantage of two Postfix configuration settings that have reduced this problem significantly. We have now updated our Zimbra installations with the same settings, so I thought I would pass them on.

The two settings we add to main.cf are:

smtpd_soft_error_limit = 2
smtpd_hard_error_limit = 3

We do this by becoming the zimbra user and then running:

postconf -e 'smtpd_hard_error_limit = 3'
postconf -e 'smtpd_soft_error_limit = 2'

We then restart Postfix to implement the changes. To restart Postfix, you need to be root and to run the Zimbra-supplied Postfix binary:

viognier:~ # cd /opt/zimbra/postfix/sbin
viognier:/opt/zimbra/postfix/sbin # ./postfix stop
postfix/postfix-script: stopping the Postfix mail system
viognier:/opt/zimbra/postfix/sbin # postfix start
postfix/postfix-script: starting the Postfix mail system
viognier:/opt/zimbra/postfix/sbin #

Documentation from Postfix is here: http://www.postfix.org/postconf.5.html#smtpd_soft_error_limit

Greylisting

In the forums, you'll probably get the most support for postgrey (below) as it's the fastest to setup.

followed by http://wiki.zimbra.com/index.php?title=Connecting_with_SQLGrey

then http://wiki.zimbra.com/index.php?title=Postfix_Policyd

Google or see http://Greylisting.org for some examples & see all sorts of ideas. There are tons of different greylist programs, for example: Some can be configure so that you hold the mail for up to 30min, (unless they get a reattempt response sooner), and then deliver it anyway with an additional spam score tacked on etc.


Postgrey

Postfix Greylisting Policy Server-the original authors site http://postgrey.schweikert.ch/

When a request for delivery of a mail is received by Postfix via SMTP, the triplet CLIENT_IP / SENDER / RECIPIENT is built. If it is the first time that this triplet is seen, or if the triplet was first seen less than 5 minutes ago, then the mail gets rejected with a temporary 450 deffer error.

It auto-remembers valid senders for up to xdays (default 35days) who are auto-whitelisted to skip the delivery delay. You can also define permanent whitelist based on clients/email addresses.

Example install on Ubuntu 6.06LTS (Dapper) Install (Tested by K. Diebold 25 July 2007)

Installing Postgrey:

sudo apt-get install postgrey
.
.
Creating config file /etc/postgrey/whitelist_clients with new version (some big companies are put here-you can add your own)
Creating config file /etc/postgrey/whitelist_recipients with new version (as needed put in users who do not want greylisting)
Creating config file /etc/default/postgrey with new version
Starting postfix greylisting daemon: postgrey.

The package adds the appropriate init scripts (update-rc.d postgrey defaults) and is configured to answer on localhost:60000.

Configuring the Zimbra Postfix:

sudo vi /opt/zimbra/conf/postfix_recipient_restrictions.cf

...and add the following above any lines that contain '%%' at the beginning and end. The final line should contain only 'permit':

check_policy_service inet:127.0.0.1:60000

...then restart Postfix (which will re-create /opt/zimbra/postfix/main.cf)

postfix reload

Notes: See the stuff that get's added to smtpd_recipient_restrictions?

Changing the delay

-The default is 5/10 minutes depending on where you get your download so if you wanted it 10 minutes: /etc/default/postgrey Depending on your version/if you download the package and manually edit before you install:

POSTGREY_OPTS="--inet=127.0.0.1:60000 --delay=300"
OR before hand
delay            => $opt{delay}     || 300,
max_age          => $opt{'max-age'} || 35,

Whitelists allow you to specify client addresses or recipient address, for which no greylisting should be done. Per default postgrey will read the following files:

/etc/postfix/postgrey_whitelist_clients
/etc/postfix/postgrey_whitelist_clients.local
/etc/postfix/postgrey_whitelist_recipients

Add-ons: p0f - passive OS detection and white-listing based on detected OS

taRgrey (tarpit + greylist) - a patch that makes postgrey into a tarpitting policy server.

Discarding Emails Sent to Invalid Addresses

To reduce email to accounts that you don't even have: Change the entry in zmmta.cf for smtpd_reject_unlisted_recipients to 'yes', save the file and restart postfix. (postfix reload)

-This rejects the request when the RCPT TO address is not listed in the list of valid recipients for its domain class. (ie: there's no such user account on the server)

If 5.0.12+ using alias domains enable set postfix_enable_smtpd_policyd=yes instead. ManagingDomains#Email_to_non-existant_accounts


One email server I administered got 400,000 messages a day. 99.2% of them were sent to addresses that didn't exist on my domain. However, my server happily scanned all of them for spam, viruses, etc. You can configure Zimbra to reject such messages with 450, saying the address doesn't exist. In addition, once an RCPT TO: command is sent specifying an invalid address, Zimbra delays about 5 seconds before it accepts another command, slowing down the spammer.

Add the following lines to /opt/zimbra/conf/postfix_recipient_restrictions.cf:

reject_unknown_recipient_domain 

-Which rejects when:

a) the RCPT TO address has no DNS A or MX record

b) when Postfix is not final destination for the recipient address

c) or when it has a malformed MX record such as a record with a zero-length MX hostname

reject_unverified_recipient

-Rejects the request when mail to the RCPT TO address is known to bounce, or when the recipient address destination is not reachable.

I add these lines just after the first line, which should be reject_non_fqdn_recipient.

Restart Zimbra and enjoy.  :)

--BJ Quinn

Caveat: There is a possible downside to this. These mass e-mailings to non-existant addresses at your domain are often part of a directory harvesting attack. By enabling this feature you will reveal legitimate addresses at your domain (through process of elimination). These will then be sold to spammers, or worse used as sender addresses by spammers.

--CG

References

http://www.zimbra.com/forums/administrators/4933-improving-spam-filtering.html

Verified Against: ZCS 4.5.x Date Created: 12/14/2006
Article ID: https://wiki.zimbra.com/index.php?title=Improving_Anti-spam_system Date Modified: 2009-12-07



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »

Jump to: navigation, search