Difference between revisions of "Using DSPAM for Spam Filtering"

m
 
(25 intermediate revisions by 7 users not shown)
Line 1: Line 1:
 +
{{BC|Community Sandbox}}
 +
__FORCETOC__
 +
<div class="col-md-12 ibox-content">
 +
=Using DSPAM for SPAM Filtering=
 +
{{KB|{{Unsupported}}|{{ZCS 8.0}}|{{ZCS 7.0}}|}}
 +
{{WIP}}
 
==Enabling==
 
==Enabling==
  
 
===ZCS 5.0.11+===
 
===ZCS 5.0.11+===
 +
 
zmlocalconfig -e amavis_dspam_enabled=true
 
zmlocalconfig -e amavis_dspam_enabled=true
  
let zmmtaconfig restart amavis or zmamavisdctl reload etc
+
The zmmtaconfig will automatically reload amavis within 2 minutes or you can run "zmamavisdctl reload" to have it take effect immediately.
 +
 
 +
See also section "Anti-Virus and Anti-Spam Protection" in the Administrator's Guide.
  
 
===ZCS 3.1+===
 
===ZCS 3.1+===
Line 16: Line 25:
 
zmamavisdctl start
 
zmamavisdctl start
  
 +
==Adjusting the scoring==
 +
 +
With the default configuration amavis will use DSPAM to add a score of -1 for ham and positive 10 for spam. If you'd like to change that, here's how. Note that these changes may not persist across Zimbra upgrades, so be sure to back up any modifications you make.
 +
 +
===Method I===
 +
Edit /opt/zimbra/conf/amavisd.conf.in
 +
 +
Locate the line '''%%uncomment LOCAL:amavis_dspam_enabled%%        mail_body_size_limit => 64000, score_factor => 1'''
 +
 +
Change the value of score_factor from 1 to something else. Changing the score_factor to e.g. .1 will cause DSPAM to add a score of -0.1 (ham) or positive 1 (spam).
 +
 +
After you save the file, stop and start amavisd as above.
 +
 +
===Method II===
 +
Let's say you want to change the scoring so that it uses -2 for ham and positive 4 for spam.
 +
 +
Edit /opt/zimbra/amavisd/sbin/amavisd
  
==History==
+
Locate the line: '''$spam_score = $dspam_result eq 'Spam' ? 10 : -1;  # fabricated'''
  
dspam 3.9.0
+
and change to: '''$spam_score = $dspam_result eq 'Spam' ? 4 : -2;  # fabricated'''
in 6.0.4 it is DSPAM-3.9.0-BETA1 which is completely broken
 
in 6.0.5 it will be DPSAM-3.9.0-RC2
 
  
 +
You may need to stop & start amavisd for this to take effect.
  
[http://bugzilla.zimbra.com/show_bug.cgi?id=20555 Bug 20555] (fixed 5.0.11)
+
References: [http://www.zimbra.com/forums/administrators/40242-solved-change-dspam-value.html Change DSPAM value] and [http://www.zimbra.com/forums/administrators/15908-enabling-dspam-8.html#post184408 Enabling DSPAM]. See also the section on score_factor in the [http://www.ijs.si/software/amavisd/release-notes.txt amavisd release notes].
  
 +
==History==
 +
===ZCS 6.0.x===
 +
*ZCS 6.0.6 will be DPSAM 3.9.0 final [http://bugzilla.zimbra.com/show_bug.cgi?id=43978 Bug 43978]
  
Some stability issues with it we're found in ZCS4.0.3 So beginning with ZCS4.5.1, DSPAM was disabled when you install or upgrade ZCS because it was occasionally interrupting mail delivery.
+
*ZCS 6.0.5 it is DPSAM-3.9.0-RC2  [http://bugzilla.zimbra.com/show_bug.cgi?id=43550 Bug 43550]
  
[http://bugzilla.zimbra.com/show_bug.cgi?id=13962 Bug 13962] & [http://bugzilla.zimbra.com/show_bug.cgi?id=12185 Bug 12185] & [http://bugzilla.zimbra.com/show_bug.cgi?id=12079 Bug 12079]
+
*ZCS 6.0.4 it is DSPAM-3.9.0-BETA1 which is completely broken http://www.zimbra.com/forums/mod-stump/35789-solved-6-0-4-dspam-av-problems.html
  
http://www.zimbra.com/forums/administrators/6322-upgraded-4-5-os-edition-now-no-dspam.html
+
*ZCS 6.0.x may need to chown zimbra.zimbra the dspam folder
  
 +
===ZCS 5.0.x===
 +
*ZCS 5.0.11 fixed dspam stability, zmfixperms adjusted, added localconfig key  [http://bugzilla.zimbra.com/show_bug.cgi?id=20555 Bug 20555]
  
Dspam upgraded to v3.6.8 - June 7, 2006 
+
===ZCS 4.5.x===
 +
*ZCS 4.5.1 Dspam disabled on upgrades or fresh installs
  
Dspam v3.8.0 - March 19, 2007 
+
*ZCS 4.0.3 stability issues found interrupting mail delivery. [http://bugzilla.zimbra.com/show_bug.cgi?id=13962 Bug 13962] & [http://bugzilla.zimbra.com/show_bug.cgi?id=12185 Bug 12185] & [http://bugzilla.zimbra.com/show_bug.cgi?id=12079 Bug 12079]
  
 +
*http://www.zimbra.com/forums/administrators/6322-upgraded-4-5-os-edition-now-no-dspam.html
  
ZCS 3.1 GA (April 2006) DSPAM is integrated into the core product.
+
===ZCS 3.x===
 +
*Dspam upgraded to v3.6.8 - June 7, 2006 (4.5/5.0 betas?)  
  
 +
*Dspam uses v3.8.0 - March 19, 2007 
 +
 +
*ZCS 3.1 GA (April 2006) DSPAM is integrated into the core product.
  
 
==Original How-To ==
 
==Original How-To ==
Line 93: Line 129:
 
Inside of your dspam.conf add the following lines (they are commented out be default)
 
Inside of your dspam.conf add the following lines (they are commented out be default)
 
  MySQLServer        /opt/zimbra/db/mysql.sock
 
  MySQLServer        /opt/zimbra/db/mysql.sock
  MySQLPort 7306
+
  MySQLPort         7306
 
  MySQLUser          dspam
 
  MySQLUser          dspam
 
  MySQLPass          ThisIsMyPassword
 
  MySQLPass          ThisIsMyPassword
Line 159: Line 195:
 
Later once DSPAM's own database has been built up, I suggest that you add the following lines to one of SpamAssassin's rule files (salocal.cf?) to filter out mail that DSPAM thinks is bogus:
 
Later once DSPAM's own database has been built up, I suggest that you add the following lines to one of SpamAssassin's rule files (salocal.cf?) to filter out mail that DSPAM thinks is bogus:
  
     header DSPAM_SPAM X-DSPAM-Result =~ /^Spam$/
+
     header DSPAM_SPAM X-DSPAM-Result =~ /^(Spam|Virus|Bl[ao]cklisted)$/
 
     describe DSPAM_SPAM DSPAM claims it is spam
 
     describe DSPAM_SPAM DSPAM claims it is spam
 
     score DSPAM_SPAM 0.5
 
     score DSPAM_SPAM 0.5
 
   
 
   
     header DSPAM_HAM X-DSPAM-Result =~ /^Innocent$/
+
     header DSPAM_HAM X-DSPAM-Result =~ /^(Innocent|Whitelisted)$/
 
     describe DSPAM_HAM DSPAM claims it is ham
 
     describe DSPAM_HAM DSPAM claims it is ham
 
     score DSPAM_HAM -0.1
 
     score DSPAM_HAM -0.1
Line 170: Line 206:
  
 
This works for me so far, filtering seems to work, and seems to be effective. I will post about training as soon as I have finished up doing it.
 
This works for me so far, filtering seems to work, and seems to be effective. I will post about training as soon as I have finished up doing it.
 +
 +
 +
{{Article Footer|ZCS 5.0.11|3/6/2006}}
  
 
[[Category:Anti-spam]]
 
[[Category:Anti-spam]]
 +
[[Category:ZCS 6.0]]
 +
[[Category:ZCS 5.0]]
 +
[[Category:ZCS 4.5]]

Latest revision as of 11:30, 13 July 2015

Using DSPAM for SPAM Filtering

   KB 1338        Last updated on 2015-07-13  




0.00
(0 votes)

Enabling

ZCS 5.0.11+

zmlocalconfig -e amavis_dspam_enabled=true

The zmmtaconfig will automatically reload amavis within 2 minutes or you can run "zmamavisdctl reload" to have it take effect immediately.

See also section "Anti-Virus and Anti-Spam Protection" in the Administrator's Guide.

ZCS 3.1+

/opt/zimbra/conf/amavisd.conf.in

uncomment

#$dspam = '/opt/zimbra/dspam/bin/dspam';

zmamavisdctl stop

zmamavisdctl start

Adjusting the scoring

With the default configuration amavis will use DSPAM to add a score of -1 for ham and positive 10 for spam. If you'd like to change that, here's how. Note that these changes may not persist across Zimbra upgrades, so be sure to back up any modifications you make.

Method I

Edit /opt/zimbra/conf/amavisd.conf.in

Locate the line %%uncomment LOCAL:amavis_dspam_enabled%% mail_body_size_limit => 64000, score_factor => 1

Change the value of score_factor from 1 to something else. Changing the score_factor to e.g. .1 will cause DSPAM to add a score of -0.1 (ham) or positive 1 (spam).

After you save the file, stop and start amavisd as above.

Method II

Let's say you want to change the scoring so that it uses -2 for ham and positive 4 for spam.

Edit /opt/zimbra/amavisd/sbin/amavisd

Locate the line: $spam_score = $dspam_result eq 'Spam' ? 10 : -1; # fabricated

and change to: $spam_score = $dspam_result eq 'Spam' ? 4 : -2; # fabricated

You may need to stop & start amavisd for this to take effect.

References: Change DSPAM value and Enabling DSPAM. See also the section on score_factor in the amavisd release notes.

History

ZCS 6.0.x

  • ZCS 6.0.6 will be DPSAM 3.9.0 final Bug 43978
  • ZCS 6.0.x may need to chown zimbra.zimbra the dspam folder

ZCS 5.0.x

  • ZCS 5.0.11 fixed dspam stability, zmfixperms adjusted, added localconfig key Bug 20555

ZCS 4.5.x

  • ZCS 4.5.1 Dspam disabled on upgrades or fresh installs

ZCS 3.x

  • Dspam upgraded to v3.6.8 - June 7, 2006 (4.5/5.0 betas?)
  • Dspam uses v3.8.0 - March 19, 2007
  • ZCS 3.1 GA (April 2006) DSPAM is integrated into the core product.

Original How-To

Original How-To by Joshua Prismon before it was integrated into ZCS follows:


Download and decompress DSPAM

Note, everything I am doing here, I do as the user zimbra. If I need super user, I use sudo.

The configuration options I used were: .

./configure \
   --prefix=/opt/dspam \
   --sysconfdir=/opt/dspam/conf \
   --with-dspam-home=/var/dspam \
   --with-dspam-home-mode=2770 \
   --with-dspam-home-owner=zimbra \
   --with-dspam-home-group=zimbra \
   --with-storage-driver=mysql_drv \
   --enable-preferences-extension \
   --with-mysql-includes=/usr/include/mysql \
   --with-mysql-libraries=/usr/lib/mysql \
   --enable-signature-headers \
   --without-delivery-agent \
   --without-quarantine-agent \
   --with-dspam-owner=zimbra \
   --with-dspam-group=zimbra
make

(as root)

make install


After this login as the zimbra user. Make sure that MySQL is running. Then create a new table for DSPAM related info:

mysqladmin create dspam

After that load the schema for the dspam tables

mysql dspam < ~/dspam-3.6.1/src/tools.mysql_drv/mysql_objects-4.1.sql

We don't want to use the same user that we use for zimbra for dspam for the database. So create a new user:

mysql
grant all on dspam.* to dspam@localhost identified by 'ThisIsMyPassword';

(Obviously change your password)

Inside of your dspam.conf add the following lines (they are commented out be default)

MySQLServer        /opt/zimbra/db/mysql.sock
MySQLPort          7306
MySQLUser          dspam
MySQLPass          ThisIsMyPassword
MySQLDb            dspam


Now we have to tell amavisd-new (which serves as a pipeline for various steps proccessing email before zimbra's java code sees emails) to enable dspam Search for the $path and $dspam variables and replace them.

To make things a bit tricky, dspam uses auto-generated files, so modify the file amavisd.conf.in rather then amavisd.conf.

$path = '/opt/zimbra/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/sbin:/usr/bin:/bin:/opt/dspam/bin';
$dspam = 'dspam';

I suggest you run amavisd once by hand with the debug options to make sure everything works:

/opt/zimbra/amavisd/sbin/amavisd -c /opt/zimbra/conf/amavisd.conf debug

And if everything is happy you should see lines like:


Nov 16 18:57:32 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: starting.  /opt/zimbra/amavisd/sbin/amavisd at bldmail01 amavisd-new-2.3.1 (20050509), Unicode aware, LANG=en_US.UTF-8
Nov 16 18:57:32 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: user=515, EUID: 515 (515);  group=, EGID: 515 516 515 5 (515 516 515 5)
Nov 16 18:57:32 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: Perl version               5.008006
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: Net::Server: 2005/11/16-18:57:33 Amavis (type  Net::Server::PreForkSimple) starting! pid(31241)
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: Net::Server: Binding to TCP port 10024 on host 127.0.0.1
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: Net::Server: Group Not Defined.  Defaulting to EGID '515 516 515 5'
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: Net::Server: User Not Defined.  Defaulting to EUID '515'
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: Net::Server: Setting up serialization via flock
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: Module Amavis::Conf        2.038
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: Module Archive::Tar        1.26
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: Module Archive::Zip        1.16
[LINES DELETED]
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: No primary av scanner: Norman Virus Control v5 / Linux
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: No primary av scanner: Panda Antivirus for Linux
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: No primary av scanner: NAI McAfee AntiVirus (uvscan)
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: No primary av scanner: VirusBuster
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: No primary av scanner: CyberSoft VFind
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: No primary av scanner: Ikarus AntiVirus for Linux
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: No primary av scanner: BitDefender
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: Found secondary av scanner ClamAV-clamscan at /usr/local/bin/clamscan
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: No secondary av scanner: FRISK F-Prot Antivirus
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: No secondary av scanner: Trend Micro FileScanner
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: No secondary av scanner: drweb - DrWeb Antivirus
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: No secondary av scanner: KasperskyLab kavscanner
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: Creating db in /opt/zimbra/amavisd/db/; BerkeleyDB 0.26, libdb  4.3
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: SpamControl: initializing Mail::SpamAssassin
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: SpamControl: done
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: Net::Server: Beginning prefork (5 processes)
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: Net::Server: Starting "5" children
Nov 16 18:57:33 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31242]: Net::Server: Child Preforked (31242)
Nov 16 18:57:34 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31242]: TIMING [total 19 ms] - bdb-open: 19 (100%)100, rundown: 0 (0%)100
Nov 16 18:57:34 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31243]: Net::Server: Child Preforked (31243)\
Nov 16 18:57:34 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31243]: TIMING [total 10 ms] - bdb-open: 10 (100%)100, rundown: 0 (0%)100
Nov 16 18:57:34 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31244]: Net::Server: Child Preforked (31244)
Nov 16 18:57:34 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31244]: TIMING [total 10 ms] - bdb-open: 10 (100%)100, rundown: 0 (0%)100
Nov 16 18:57:34 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31245]: Net::Server: Child Preforked (31245)
Nov 16 18:57:34 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31245]: TIMING [total 9 ms] - bdb-open: 9 (100%)100, rundown: 0 (0%)100
Nov 16 18:57:34 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31246]: Net::Server: Child Preforked (31246)
Nov 16 18:57:34 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31241]: Net::Server: Parent ready for children.
Nov 16 18:57:34 bldmail01 /opt/zimbra/amavisd/sbin/amavisd[31246]: TIMING [total 12 ms] - bdb-open: 12 (100%)100, rundown: 0 (0%)100

That should get you to where both DSPAM and SpamAssassin check for spams.

Later once DSPAM's own database has been built up, I suggest that you add the following lines to one of SpamAssassin's rule files (salocal.cf?) to filter out mail that DSPAM thinks is bogus:

    header DSPAM_SPAM X-DSPAM-Result =~ /^(Spam|Virus|Bl[ao]cklisted)$/
    describe DSPAM_SPAM DSPAM claims it is spam
    score DSPAM_SPAM 0.5

    header DSPAM_HAM X-DSPAM-Result =~ /^(Innocent|Whitelisted)$/
    describe DSPAM_HAM DSPAM claims it is ham
    score DSPAM_HAM -0.1

Anyways, what I write here is my own, with the exceptions that I lifted off of the amavis-new list, esp this message: [url]http://sourceforge.net/mailarchive/message.php?msg_id=11025890[/url]

This works for me so far, filtering seems to work, and seems to be effective. I will post about training as soon as I have finished up doing it.


Verified Against: ZCS 5.0.11 Date Created: 3/6/2006
Article ID: https://wiki.zimbra.com/index.php?title=Using_DSPAM_for_Spam_Filtering Date Modified: 2015-07-13



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »

Jump to: navigation, search