Preventing Spamming
Preventing Spamming
Problem
- Huge and unresponsive mail queue due to Incoming Spams and NDRs
- Observing excessive spamming activity on the server.
- MTA IP blacklisting in the Global RBLs resulting huge impact on outgoing mails.
Solution
The solution divided into 2 parts.
- Cure
- Prevention
Cure
In this section let's check how we can identify the spammer, control the spamming and clear the queue.
- First of all hold the queue
su - zimbra
~/common/sbin/postsuper -h ALL
- To release the queue once done with the work.
~/common/sbin/postsuper -r ALL
- Additionally, we can hold all the incoming mails on MTA level by following below wiki.
There are 2 ways to check the mail queue A) From the admin panel B) Server CLI.
- We will first check the queue from the Admin Control Panel.
Open your Admin Panel and in Left pane go to > Monitor > Mail Queues and you will see the window something like this,
This Mail Queue section containing mainly 4 Tabs. i) Deferred ii) Incoming iii) Active iv) Held v) Corrupt.
And Each tab Containing a summary of accumulated mails, which are summarized between 6 columns,
i) Receiver Domain ii) Origin IP iii) Sender Domain iv) Receiver Address v) Sender Address vi) Error.
Because of load on the server sometimes you may experience difficulty accessing the Admin panel hence we will check the CLI method to find out the spammer
We are going to use the following commands which will give almost a similar result as the admin panel.
$sudo ~/libexec/zmqstat
Thus, by referring to the queue, we can easily identify which email Address has compromised and from which IP addresses spam mails are receiving and you can take the further action.
$qshape -s deferred | head
- For detailed analysis please do the following, check which user has compromised and used that email address for spamming.
$ grep sasl_user /var/log/zimbra.log | sed 's/.*sasl_username=//g' | sort | uniq -c | sort -nr | head
The Output will show something like this,
12289 user1@domain.tld
13 user2@domain.tld
10 user3@domain.tld
7 user4@domain.tld
7 user5@domain.tld
4 user6@domain.tld
1 user7@domain.tld
1 user8@domain.tld
Thus, by the above stats, we can easily guess user1@domain.tld has compromised since no other users authenticated as much as user1.
To cross-verify, we can grep the user1@doamin.tld in /var/log/zimbra.log and we may find thousands of entries like this
Sep 21 17:34:11 zmta1 postfix/smtps/smtpd\[12679\]: ACDBA60F68: filter: RCPT from unknown\[177.39.32.97\]: <user1@domain.tld>: Sender address triggers FILTER smtp-amavis:\[127.0.0.1\]:10026; from=<user1@domain.tld> to=<greenxyso333@aol.com> proto=ESMTP helo=<\[127.0.0.1\]> Sep 21 17:34:12 zmta1 postfix/smtps/smtpd\[14032\]: B6A3860C53: filter: RCPT from unknown\[191.242.246.133\]: <user1@domain.tld>: Sender address triggers FILTER smtp-amavis:\[127.0.0.1\]:10026; from=<user1@domain.tld> to=<ssrkettlecasdaorn@aol.com> proto=ESMTP helo=<\[127.0.0.1\]> Sep 21 17:34:12 zmta1 postfix/smtps/smtpd\[12700\]: 56C3960F69: filter: RCPT from 6-48-130-177.redewsp.com.br\[177.130.48.6\]: <user1@domain.tld>: Sender address triggers FILTER smtp-amavis:\[127.0.0.1\]:10026; from=<user1@domain.tld> to=<dtgadsadulf@yahoo.com> proto=ESMTP helo=<\[127.0.0.1\]>
By referring to a few mails we will notice
1) Mails are going to all junk recipient (those probably do not exist).
2) Originating IPs are all different.
By this we can confirm that all those are spam mails, hence change the password of the affected user and/or disable the user to prevent further authentication and spamming.
Since we found there are several authentications happened hence let’s find out the originating IPs
Just grep and find out from which IP address, authentication and mails are originated. here we can use 2 ways
i) By “from address”
ii) By Authentication
- Finding originating IP using “From Address”
grep "from=<user1@domain.tld>" /var/log/zimbra.log | awk '{print $10}' |sort -nr -k 1 | sed -rn 's/.*\[//;s/\].*//p' | uniq -c | sort -nr -k 1
The output will show something like this.
- Finding originating IP using Authentication.”
# grep sasl_user /var/log/zimbra.log | grep user1@domain.tld | awk '{print $7}' |sed -rn 's/.*\[//;s/\].*//p' | sort -nr -k 1 | uniq -c |sort -nr -k 1
The output will show something like this.
In above example we have found a bunch of IPs from which thousands of mails are originated hence it gives us a clear picture that all those are spam mails. Hence we will go ahead and ask the customer to block all these IPs in their firewall so that further authentication won’t happen from these IP.
We can also blacklist all these IPs on Zimbra as well by following the below mentioned steps.
==i) Create a file /opt/zimbra/conf/postfix_blacklist and add all the IPs in the following format.==
177.71.83.241 REJECT 191.37.158.10 REJECT 177.39.32.97 REJECT 177.53.74.33 REJECT 177.71.23.64 REJECT
ii) Now do postmap and it will create DB file.
postmap /opt/zimbra/conf/postfix_blacklist
iii) Now add this file in the zimbra mta restriction so that postfix will refer this file while accepting the connection and reject all the mentioned IPs.
zmprov mcf +zimbraMtaRestriction 'check_client_access lmdb:/opt/zimbra/conf/postfix_blacklist'
iv) Now restart the MTA services.
zmmtactl restart
v) We are done the with initial phase, by this we can rest assured that now no more new spam will hit nor any authentication will happen. Still just to cross-check whether still there is any brute force attack happening, just check the zimbra.log for auth failures by executing below command.
$ grep “authentication failure” /var/log/zimbra.log
And the output
If you find more such entries with new email addresses, please follow similar processes mentioned in point no. 4.
5) Since we have stopped the further authentication and held the queue, Let’s move further and delete all deferred and spam mails from the queue.
5-A) From the Admin panel – >> Monitor >> Mail Queue. Here right click on either originating IP or sender domain/email address and click on delete.
5-B) If Admin panel is not accessible we will have to do the same process from CLI.
Let’s first delete all the deferred mails from the queue, these mails are usually throttled by the recipient server due to spamming complained from the sender’s MTA
/opt/zimbra/common/sbin/postsuper -d ALL deferred
The safest method is to delete using from the email address that has compromised, and the email address from which NDR has received,
Thus, first, execute the below command and check which mails we are going to delete, here we are mentioning from email address that has compromised, and later you can change the NDR email addresses as well.
/opt/zimbra/common/sbin/postqueue -p | egrep -v '^ *\(|-Queue ID-' \ | awk 'BEGIN { RS = "" } { if ($7 == "user1@domain.tld") print $1} ' | tr -d '*!'
Here replace user1@domain.tld with your actual affected email address. After executing above command, it will give you list of the mails which we are going to delete, later you can check all the list just to verify we are on the right path.
Now execute the final command by adding -d at the end to delete all the mails.
/opt/zimbra/common/sbin/postqueue -p | egrep -v '^ *\(|-Queue ID-' \ | awk 'BEGIN { RS = "" } { if ($7 == "user1@domain.tld") print $1} ' \ | tr -d '*!' | /opt/zimbra/common/sbin/postsuper -d -
6) Once all the mails are deleted now we can check the queue again,
$ sudo ~/libexec/zmqstat
hold=125 corrupt=0 deferred=0 active=0 incoming=0
7) Since all the queue is clear and junk and deferred mail are already deleted, we can now release the hold queue.
~/common/sbin/postsuper -r ALL
Prevention -
| Submitted by: Amol Mistry |
