Ajcody-MTA-Postfix-Topics: Difference between revisions

mNo edit summary
 
(290 intermediate revisions by 6 users not shown)
Line 1: Line 1:
{|  width="100%" border="0"
{{BC|Zeta Alliance}}                        <!-- Note, this will also add [[Category: Zeta Alliance]] to bottom of wiki page. -->
| bgcolor="orange" | [[Image:Attention.png]] - This article is NOT official Zimbra documentation. It is a user contribution and may include unsupported customizations, references, suggestions, or information.
__FORCETOC__                              <!-- Will force a TOC regards of size of article. __NOTOC__ if no TOC is wanted. -->
|}
<div class="col-md-12 ibox-content">
= Ajcody MTA Postfix Topics=            <!-- Normally will reflect page title. Is listed at very top of page. -->
{{KB|{{ZETA}}|{{ZCS 8.5}}|{{ZCS 8.0}}|{{ZCS 7.0}}|}}            <!-- Can only handle 3 ZCS versions. -->
{{WIP}}                                                <!-- For pages that are "work in progress". -->
 


==Postfix - MTA==
==Postfix - MTA==
Line 11: Line 15:
===Missing main.cf Error===
===Missing main.cf Error===


If you get something like:
Moved to [[Missing_main.cf_Error_-_MTA]]
 
===MTA Mail Flow - Birds-eye Overview===
 
Moved to [[Postfix-Amavisd_Mail_Flow_-_Birds-eye_Overview_-_MTA]]
 
===Understanding /var/log/zimbra.log And Postfix Log Events===
 
Moved to: [[Understanding_zimbra.log_And_Postfix_Log_Events_-_MTA]]
 
====Postfix Queue ID vs. message-id====
 
Moved to: [[Postfix_Queue_ID_vs._message-id_-_MTA]]
 
====Authentication Log Events====
 
Moved to [[Understanding_And_Troubleshooting_Authentication_Log_Events]]
 
=====IMAP And Authenticated SMTP [SSL] Example=====


postfix/postqueue[8739]: fatal: open /opt/zimbra/postfix-2.4.7.5z/conf/main.cf: No such file or directory
Moved to [[Understanding_And_Troubleshooting_Authentication_Log_Events#IMAP_And_Authenticated_SMTP_.5BSSL.5D_Example]]


All you need to do is create an empty main.cf file and zimbra will rewrite it.
======IMAP Test Via Telnet And Logging Events Of It - Proxy Included======


touch main.cf
Moved to [[Understanding_And_Troubleshooting_Authentication_Log_Events#IMAP_Test_Via_Telnet_And_Logging_Events_Of_It_-_Proxy_Included]]


When you now do something to start the mta, it will generate the values for main.cf
======IMAP Login Via Openssl - LOGIN TLS - Proxy Included======


zmmtactl stop
Moved to [[Understanding_And_Troubleshooting_Authentication_Log_Events#IMAP_Login_Via_Openssl_-_LOGIN_TLS_-_Proxy_Included]]
zmmtactl start
 
===Network Tracing Between A Remote Host And A ZCS MTA===
 
Moved to [[Network_Tracing_Between_A_Remote_Host_And_A_ZCS_MTA]]
 
===Finding Messages - zmmsgtrace===
 
See the following for details [for 7.1.1+]:
* zmmsgtrace replacement
** http://bugzilla.zimbra.com/show_bug.cgi?id=41078
 
Documentation at [[CLI_zmmsgtrace]] . Note, if you get command not found as the zimbra user, try /opt/zimbra/libexec/zmmsgtrace instead.
 
===How To Increase SMTP Debug Logging - MTA===
 
Moved to [[How_To_Increase_SMTP_Debug_Logging_-_MTA]]
 
===Simple Troubleshooting For SMTP Via Telnet, Openssl===
 
Moved to [[Simple_Troubleshooting_For_SMTP_Via_Telnet_And_Openssl]]
 
====First - Understanding Your Authentication Requirements In ZCS====
 
Moved to [[Simple_Troubleshooting_For_SMTP_Via_Telnet_And_Openssl#First_-_Understanding_Your_Authentication_Requirements_In_ZCS]]
 
====Second - Encoding Username And Passwords For AUTH Sequence====
 
Moved to [[Simple_Troubleshooting_For_SMTP_Via_Telnet_And_Openssl#Second_-_Encoding_Username_And_Passwords_For_AUTH_Sequence]]
 
====For ESMTP Auth is LOGIN - Example====
 
Moved to [[Simple_Troubleshooting_For_SMTP_Via_Telnet_And_Openssl#For_ESMTP_Auth_is_LOGIN_-_Example]]
 
====For ESMTP Auth is Plain - Example====
 
Moved to [[Simple_Troubleshooting_For_SMTP_Via_Telnet_And_Openssl#For_ESMTP_Auth_is_Plain_-_Example]]
 
====For TLS/SSL - Example====
 
Moved to [[Simple_Troubleshooting_For_SMTP_Via_Telnet_And_Openssl#For_TLS.2FSSL_-_Example]]
 
=====Testing Against Port 465=====
 
Moved to [[Simple_Troubleshooting_For_SMTP_Via_Telnet_And_Openssl#Testing_Against_Port_465]]
 
====To Confirm An Auth User Can't Send With Another FROM Address====
 
Moved to [[Simple_Troubleshooting_For_SMTP_Via_Telnet_And_Openssl#To_Confirm_An_Auth_User_Can.27t_Send_With_Another_FROM_Address]]


===Adding A New MTA Server===
===Adding A New MTA Server===
Line 34: Line 103:
===Load Balancing For SMTP - Out Bound Mail===
===Load Balancing For SMTP - Out Bound Mail===


Currently, 5.x code, you have the following options:
Currently, '''5.x code''', you have the following options:


* Configure zimbraMtaRelayHost and zimbraSmtpHostname [[Ajcody-Server-Topics#Using_Different_SMTP_Server_For_Webclient_.28ZWC.29.2C_Mobiles.2C_And_ZCO|zimbraSmtpHostname Details]] to:
* Configure zimbraMtaRelayHost and zimbraSmtpHostname [[Ajcody-Server-Topics#Using_Different_SMTP_Server_For_Webclient_.28ZWC.29.2C_Mobiles.2C_And_ZCO|zimbraSmtpHostname Details]] to:
Line 41: Line 110:
** Setup a round-robin A record situation in your DNS for the external mta's you'll be using.
** Setup a round-robin A record situation in your DNS for the external mta's you'll be using.


In the future, GNR/6.x, you'll be able to add multiple targets to the variables and we'll have some degree of "balancing" between them.
In, '''GNR/6.x''', you are able to add multiple targets to the variables and we'll have some degree of "balancing" between them.


* "allow list for zimbraSmtpHostname"
* "allow list for zimbraSmtpHostname"
Line 52: Line 121:
See [[User_Alias_Mapping_and_Mail_Transport_with_Postfix_%26_LDAP]]
See [[User_Alias_Mapping_and_Mail_Transport_with_Postfix_%26_LDAP]]


===Traditional Aliases Use - /etc/aliases type lookups===
====Multiple LDAP Servers?====
 
Completed RFE:
 
* "mta should be able to take a list of LDAP servers to take advantage of replicas."
** http://bugzilla.zimbra.com/show_bug.cgi?id=9353
*** zmmtainit to allow for multiple command line options that will set the URL. Grab the contents of the ldap_url localconfig variable.
 
From :
 
* http://www.postfix.org/ldap_table.5.html
 
<pre>
        server_host (default: localhost)
              The name of the host running the LDAP server,  e.g.
 
                  server_host = ldap.example.com
 
              Depending  on the LDAP client library you're using,
              it should be possible to specify  multiple  servers
              here,  with the library trying them in order should
              the first one fail. It should also be  possible  to
              give  each  server  in  the  list  a different port
              (overriding server_port below), by naming them like
 
                  server_host = ldap.example.com:1444
 
              With OpenLDAP, a (list of) LDAP URLs can be used to
              specify both the hostname(s) and the port(s):


Filed this RFE:
                  server_host = ldap://ldap.example.com:1444
* "Support traditional email aliases via aliases file or ldap - admin console view"
                              ldap://ldap2.example.com:1444
** http://bugzilla.zimbra.com/show_bug.cgi?id=33642
 
*** Aliases use without being tied into authentication methods or consuming a license file.
              All LDAP URLs accepted by the OpenLDAP library  are
              supported,  including  connections over UNIX domain
              sockets, and LDAP SSL (the last one  provided  that
              OpenLDAP was compiled with support for SSL):
 
                  server_host = ldapi://%2Fsome%2Fpath
                              ldaps://ldap.example.com:636
 
**my note**
This thread - http://archives.neohapsis.com/archives/postfix/2004-09/1763.html
give me the impression they made a mistake in modifying the help file on this
and they dropped the use/need of the command:
 
  server_host = ldap://ldap.example.com:1444, ldap://ldap2.example.com:1444
</pre>
 
Just a small note on where var shows up:
 
<pre>
[root@mail3 conf]# pwd
/opt/zimbra/conf
[root@mail3 conf]# grep server_host *
amavisd.conf.in:$myhostname = '@@zimbra_server_hostname@@';  # must be a fully-qualified domain name!
ldap-scm.cf:server_host = ldap://mail3.zimbra.DOMAIN.com:389
ldap-transport.cf:server_host = ldap://mail3.zimbra.DOMAIN.com:389
ldap-vad.cf:server_host = ldap://mail3.zimbra.DOMAIN.com:389
ldap-vam.cf:server_host = ldap://mail3.zimbra.DOMAIN.com:389
ldap-vmd.cf:server_host = ldap://mail3.zimbra.DOMAIN.com:389
ldap-vmm.cf:server_host = ldap://mail3.zimbra.DOMAIN.com:389
localconfig.xml:  <key name="zimbra_server_hostname">
zmmta.cf: LOCAL zimbra_server_hostname
zmmta.cf: POSTCONF myhostname LOCAL zimbra_server_hostname
</pre>
 
References:
 
* http://archives.neohapsis.com/archives/postfix/2000-04/0200.html
 
===Traditional Aliases Use - /etc/aliases Type Lookups===
 
Moved to [[Traditional_Aliases_Use_-_/etc/aliases_Type_Lookups]]


===Allowing Accounts To Change The From Address===
===Allowing Accounts To Change The From Address===
Line 103: Line 240:


===Automatic BCC===
===Automatic BCC===
====Option 1 - Via Postfix Customization====
From the postfix website:
*always_bcc = address
** Deliver a copy of all mail to the specified address. In Postfix versions before 2.1, this feature is implemented by smtpd(8), qmqpd(8), or pickup(8).
* sender_bcc_maps = type:table
** Search the specified "type:table" lookup table with the envelope sender address for an automatic BCC address. This feature is available in Postfix 2.1 and later.
* recipient_bcc_maps = type:table
** Search the specified "type:table" lookup table with the envelope recipient address for an automatic BCC address. This feature is available in Postfix 2.1 and later.
* Note: automatic BCC recipients are produced only for new mail. To avoid mailer loops, automatic BCC recipients are not generated for mail that Postfix forwards internally, nor for mail that Postfix generates itself.


Please see the following:
Please see the following:
* A very nice forum post on the subject from our very own mmorse
** http://www.zimbra.com/forums/administrators/28606-master-incoming-outgoing-archive.html#post134490
* Postfix workaround
* Postfix workaround
** http://www.postfix.com/ADDRESS_REWRITING_README.html#auto_bcc
** http://www.postfix.com/ADDRESS_REWRITING_README.html#auto_bcc
Line 111: Line 263:
* "Next rev of (mail) identities preferences management (server side)"
* "Next rev of (mail) identities preferences management (server side)"
** http://bugzilla.zimbra.com/show_bug.cgi?id=17320
** http://bugzilla.zimbra.com/show_bug.cgi?id=17320
====Option 2 - Via ZCS Legal Intercept====
Generally used for [[Ajcody-User-Management-Topics#Managing_Legal_Requests_for_Information|Managing Legal Requests for Information]]
Description:
:: The ZCS legal intercept feature is used to obtain copies of email messages that are sent, received, or saved as drafts from targeted accounts and send these message to a designated “shadow” email address. Legal Intercept can be configured to send the complete content of the message or to send only the header information. When a targeted account sends, receives, or saves a draft message, an intercept message is automatically created to forward copies of the messages as attachments to the specified email address.
Please see:
* http://www.zimbra.com/docs/ne/latest/administration_guide/managing_other_zcs_features.8.1.html
* http://wiki.zimbra.com/index.php?title=Legal_Intercept
* http://bugzilla.zimbra.com/show_bug.cgi?id=17539
====Option 3 - Zimbra's Archiving And Discovery====
See [[Ajcody-Notes-Archive-Discovery]] concerning A&D setup and options.


===Limiting Or Increasing Number Of Recipents / Messages===
===Limiting Or Increasing Number Of Recipents / Messages===
Line 122: Line 290:
If you want to restrict messages per hour,  you can look into Policy Daemon:
If you want to restrict messages per hour,  you can look into Policy Daemon:


* http://wiki.zimbra.com/wiki/Postfix_Policyd
* http://wiki.zimbra.com/wiki/How-to_for_cbpolicyd
* http://www.policyd.org/features.html
* http://www.policyd.org/features.html
* http://wiki.zimbra.com/index.php?title=Postfix_Policyd
 
Beta release in ZCS 7 , see:
 
* "make support for postfix-policyd easier"
** http://bugzilla.zimbra.com/show_bug.cgi?id=8791


====Postfix====
====Postfix====
Line 173: Line 347:
* "Mail policies and access control for sending to distribution lists"
* "Mail policies and access control for sending to distribution lists"
** http://bugzilla.zimbra.com/show_bug.cgi?id=9620
** http://bugzilla.zimbra.com/show_bug.cgi?id=9620
*** Target Milestone currently for GunsNRoses release
*** Target Milestone currently for GunsNRoses


RFE 14645 is along the lines of what I'm thinking of as the end goal.
===Controlling SMTPD Client Connections===


===Controlling SMTPD Client Connections===
Mmorse did a good write up on these variables in the forum:
 
* http://www.zimbra.com/forums/administrators/13591-solved-limit-max-recipriants.html#post69582


Postfix Resources At Their Site (All Clients/Connections):
Postfix Resources At Their Site (All Clients/Connections):
Line 190: Line 366:
Postfix Resources At Their Site (Exceptions To Clients/Connections Or Single Source):
Postfix Resources At Their Site (Exceptions To Clients/Connections Or Single Source):
* [http://www.postfix.org/postconf.5.html#smtpd_client_event_limit_exceptions smtpd_client_event_limit_exceptions - Clients that are excluded from connection count, connection rate, or SMTP request rate restrictions.]
* [http://www.postfix.org/postconf.5.html#smtpd_client_event_limit_exceptions smtpd_client_event_limit_exceptions - Clients that are excluded from connection count, connection rate, or SMTP request rate restrictions.]
===Restrictions===
Besides using external mailing list software, [[Ajcody-MailingLists-And-Mailman#Mailman_-_Mailing_List_Manager|Mailman]] or [[Ajcody-MailingLists-And-Mailman#Sympa_-_Mailing_List_Manager|Sympa]], here's some other topical items in regards to restrictions.
Some user contributed articles:
* [[RestrictPostfixRecipients]]
* [[Restrict_sending_to_certain_domains]]
* [[Restrict_users_to_certain_domain]]
* [http://www.zimbra.com/forums/administrators/15041-guide-postifx-how-multiple-access-lists-protected-distribution-lists.html Forum Post: GUIDE: Postifx: HOW TO: Multiple access lists for protected Distribution-lists]
Some Postfix references:
* http://www.postfix.org/RESTRICTION_CLASS_README.html#internal
* http://www.postfix.org/SMTPD_POLICY_README.html
Some RFE's related to mta based restrictions [targets are based upon today - July 21, 2010]:
* "Dynamic distribution lists - Internal Directory"
** http://bugzilla.zimbra.com/show_bug.cgi?id=3884
* "per-domain send restriction" - Not Committed
** http://bugzilla.zimbra.com/show_bug.cgi?id=5595
*** These are marked as dup's of the above:
**** "disable outbound e-mail for one user"
***** http://bugzilla.zimbra.com/show_bug.cgi?id=34654
**** "Add an facility to detemine internal relay users in admin"
***** http://bugzilla.zimbra.com/show_bug.cgi?id=33255
* "policy for who can send to a distribution lists" - Helix
** http://bugzilla.zimbra.com/show_bug.cgi?id=9620
*** RFE 9620 is also a blocker for the following RFE:
**** "milter to check if sender can send to a distribution list"
***** http://bugzilla.zimbra.com/show_bug.cgi?id=46311
*** These are marked as dup's of the above:
**** "Ability to Specify Mail Policy"
***** http://bugzilla.zimbra.com/show_bug.cgi?id=5555
**** "domain level filters rules"
***** http://bugzilla.zimbra.com/show_bug.cgi?id=6128
**** "Distribution List Restrictions"
***** http://bugzilla.zimbra.com/show_bug.cgi?id=7104
**** "Feature request - Mail Policies"
***** http://bugzilla.zimbra.com/show_bug.cgi?id=9328
**** "limit  "send from"  to certain domains"
***** http://bugzilla.zimbra.com/show_bug.cgi?id=12038
**** "'Internal email only' options in admin control panel"
***** http://bugzilla.zimbra.com/show_bug.cgi?id=16671
**** "Access control for free busy and resources (ie permission to invite)"
***** http://bugzilla.zimbra.com/show_bug.cgi?id=22913
**** "RFE: Admin GUI: Restrict the use of Distribution List among users."
***** http://bugzilla.zimbra.com/show_bug.cgi?id=29305
* "Implement smtpd_sender_restrictions"
** http://bugzilla.zimbra.com/show_bug.cgi?id=15808
* "How to restrict a user to only send via zwc"
** http://bugzilla.zimbra.com/show_bug.cgi?id=16623
* "enable configuration of  "smtpd_sender_restriction""
** http://bugzilla.zimbra.com/show_bug.cgi?id=22363


===Spam Control And Related Issues===
===Spam Control And Related Issues===
====High Over View Steps Of What To Do====
* '''Step 1:''' Confirm your not an open relay and double check your postfix $mynetworks variable.
** [[ZimbraMtaMyNetworks|ZimbraMtaMyNetworks And Postfix mynetworks]]
** [[Ajcody-MTA-Postfix-Topics#Open_Relay_Check|Open Relay Check]]
* '''Step 2:''' Stop or put on-hold mail queue.
** Put all messages into HOLD queue:
*** Get a report of your current mailq [can be useful if you clean out the queue later but need to identify what external mail hosts are now denying you and who you'll need to contact about getting removed from their denial list]
**** Example: /opt/zimbra/postfix/sbin/mailq > /tmp/zimbra_mailq_report.txt
*** /opt/zimbra/postfix/sbin/postsuper -h ALL
** Or put all messages match compromised account into HOLD queue:
*** /opt/zimbra/postfix/sbin/mailq | grep user_compromised@domain | awk '{ print $1 }' | tr -d '!*' | /opt/zimbra/postfix/sbin/postsuper -h -
**** Note, this is an example - you might with the grep grab more than the compromised account with the match.
** See whole section - [[Ajcody-MTA-Postfix-Topics#Managing_Postfix_Queue| Managing The Postfix Queue]]
* '''Step 3:''' Check your mail log [On ZCS servers running MTA services] - /var/log/zimbra.log
** [[Ajcody-MTA-Postfix-Topics#Understanding_.2Fvar.2Flog.2Fzimbra.log_And_Postfix_Log_Events| Understanding the zimbra.log file and Postfix log events.]] , see subsection about queue ID and message ID also.
** [[Ajcody-MTA-Postfix-Topics#Who.27s_My_Spammer.3F| Who's My Spammer?]]
* '''Step 4:''' Identify compromised account authenticating SMTP AUTH connection or block ip address where emails are coming from at firewall.
** [[Ajcody-MTA-Postfix-Topics#Who.27s_My_Spammer.3F| Who's My Spammer?]]
** Continue to monitor compromised account and block ip addresses:
*** tail -f /var/log/zimbra.log | grep username | grep sasl
**** Jun  8 18:14:10 mail postfix/smtpd[15794]: 004358EEB16: client=unknown[XXXX.236.197.216], sasl_method=LOGIN, sasl_username=username@domain
* '''Step 5:''' Disable the exploited email account, expire auth session, etc.
** [[Ajcody-User-Management-Topics#Resetting_A_User.27s_Account_From_CLI| Resetting Or Expiring User Auth]]
** '''Note''' - Restarting the mta services will be important once you reset the password/s or lock the account. It's required to ensure the active connections will be closed and any existing auth tokens no longer are valid. See:
*** Force currently active SMTP authenticated sessions to be renegotiated when locking an account
**** https://bugzilla.zimbra.com/show_bug.cgi?id=80299
* '''Step 6:''' Move the mail queue or delete the spam email
** See whole section - [[Ajcody-MTA-Postfix-Topics#Managing_Postfix_Queue| Managing The Postfix Queue]]
* '''Step 7:''' Release Mail queue
====Who's My Spammer?====
=====Getting Some Initial Summary Data=====
======zmdialyreport======
First, some notable bug/RFE's in regards to the zmdailyreport:
* RFE - add explanations to Daily mail report / pflogsumm.pl output
** https://bugzilla.zimbra.com/show_bug.cgi?id=86630
* Daily mail report shows incorrect output because pflogsumm.pl doubles the result
** https://bugzilla.zimbra.com/show_bug.cgi?id=84444
* Descriptions of mta_counts numbers vs daily reports and other msg stats
** https://bugzilla.zimbra.com/show_bug.cgi?id=79632
You can first get some summary data by doing the following:
<pre>
[zimbra@zcs806 ~]$ /opt/zimbra/libexec/zmdailyreport
Grand Totals
------------
messages
      7  received
    11  delivered
      0  forwarded
      0  deferred
      0  bounced
      3  rejected (21%)
      0  reject warnings
      0  held
      0  discarded (0%)
  2780  bytes received
  10914  bytes delivered
      2  senders
      1  sending hosts/domains
      1  recipients
      1  recipient hosts/domains
Per-Hour Traffic Summary
    time          received  delivered  deferred    bounced    rejected
    --------------------------------------------------------------------
    0000-0100          0          0          0          0          0
    0100-0200          1          3          0          0          0
    0200-0300          0          0          0          0          0
    0300-0400          0          0          0          0          0
    0400-0500          0          0          0          0          0
    0500-0600          0          0          0          0          0
    0600-0700          0          0          0          0          0
    0700-0800          1          0          0          0          2
    0800-0900          1          0          0          0          0
    0900-1000          0          0          0          0          1
    1000-1100          0          0          0          0          0
    1100-1200          0          0          0          0          0
    1200-1300          4          8          0          0          0
    1300-1400          0          0          0          0          0
    1400-1500          0          0          0          0          0
    1500-1600          0          0          0          0          0
    1600-1700          0          0          0          0          0
    1700-1800          0          0          0          0          0
    1800-1900          0          0          0          0          0
    1900-2000          0          0          0          0          0
    2000-2100          0          0          0          0          0
    2100-2200          0          0          0          0          0
    2200-2300          0          0          0          0          0
    2300-2400          0          0          0          0          0
Host/Domain Summary: Message Delivery (top 50)
sent cnt  bytes  defers  avg dly max dly host/domain
-------- -------  -------  ------- ------- -----------
    11    10914        0    7.4 s  24.0 s  zcs806.DOMAIN.com
Host/Domain Summary: Messages Received (top 50)
msg cnt  bytes  host/domain
-------- -------  -----------
      5    2780  zcs806.DOMAIN.com
top 50 Senders by message count
-------------------------------
      4  zimbra@zcs806.DOMAIN.com
      1  admin@zcs806.DOMAIN.com
top 50 Recipients by message count
----------------------------------
    11  admin@zcs806.DOMAIN.com
top 50 Senders by message size
------------------------------
  1974  zimbra@zcs806.DOMAIN.com
    806  admin@zcs806.DOMAIN.com
top 50 Recipients by message size
---------------------------------
  10914  admin@zcs806.DOMAIN.com
message deferral detail: none
message bounce detail (by relay): none
message reject detail
---------------------
  MAIL
    5.3.4 Message size exceeds fixed limit (total: 3)
          3  domain-ext.com
message reject warning detail: none
message hold detail: none
message discard detail: none
smtp delivery failures: none
Warnings
--------
  sendmail (total: 3)
        1  or the command is run from a set-uid root process
        1  the Postfix sendmail command has set-uid root file permissions
        1  the Postfix sendmail command must be installed without set-uid ...
  smtpd (total: 1)
        1  7A735345A: queue file size limit exceeded
Fatal Errors: none
Panics: none
Master daemon messages: none
</pre>
======client_usage_report.py======
This will give some stats on your mail activity. Note, there are some issue with this script double reporting mail counts etc, but it's useful to identify the top 50 for activity.
<pre>
[zimbra@zcs806 ~]$ /opt/zimbra/libexec/client_usage_report.py
Reading /opt/zimbra/log/access_log.2014-04-17 ..
Reading /opt/zimbra/log/access_log.2014-04-18 ..
Reading /opt/zimbra/log/access_log.2014-04-19 ..
Reading /opt/zimbra/log/access_log.2014-04-20 ..
Reading /opt/zimbra/log/access_log.2014-04-21 ..
Reading /opt/zimbra/log/access_log.2014-04-22 ..
Reading /opt/zimbra/log/access_log.2014-04-23 ..
Writing /opt/zimbra/zmstat/client_usage_report_2014-04-24.csv ..
</pre>
Then review the file it will create , it will give :  "user_agent","client_IP","req_count"
<pre>
[zimbra@zcs806 ~]$ cat /opt/zimbra/zmstat/client_usage_report_2014-04-24.csv
"user_agent","client_IP","req_count"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:28.0) Gecko/20100101 Firefox/28.0","192.168.1.166","14"
"Mozilla/5.0 (Windows NT 6.2; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0","192.168.1.166","93"
"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0","192.168.1.174","6"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:28.0) Gecko/20100101 Firefox/28.0","192.168.1.166","71"
</pre>
======qshape======
You can also look at the results of [[http://www.postfix.org/qshape.1.html qshape]] - the default is the active queue. For more on qshape, see [[http://www.postfix.org/QSHAPE_README.html Postfix Qshape Readme]] .
<pre>
qshape deferred
              T  5 10 20 40 80 160 320 640 1280 1280+
        TOTAL 12  0  0  0  0  0  0  0  0    0    12
    gmail.com  9  0  0  0  0  0  0  0  0    0    9
    yahoo.com  3  0  0  0  0  0  0  0  0    0    3
</pre>
======3rd Party Log Reports - postfix-logwatch and amavis-logwatch======
Created RFE for us to include these in ZCS:
* Include postfix-logwatch_and_amavis-logwatch
** https://bugzilla.zimbra.com/show_bug.cgi?id=89450
You can download them from http://logreporters.sourceforge.net/ . It's a fairly simply install, download and then extract - cd into extracted directory and as root type :
make install-standalone
They will install to /usr/local/bin/amavis-logwatch & postfix-logwatch . The config files are in /usr/local/etc/amavis-logwatch.conf & postfix-logwatch.conf . Here's an example of the output.
/usr/local/bin/amavis-logwatch output example:
<pre>
[root@zcs806 amavis-logwatch-1.51.02]# /usr/local/bin/amavis-logwatch /var/log/zimbra.log
****** Summary *************************************************************************************
      4  Total messages scanned ------------------  100.00%
  1.926K  Total bytes scanned                          1,972
========  ==================================================
      4  Passed ----------------------------------  100.00%
      4    Clean passed                            100.00%
========  ==================================================
      4  Ham -------------------------------------  100.00%
      4    Clean passed                            100.00%
========  ==================================================
==================================================================================
Spam Score Percentiles        0%      50%      90%      95%      98%      100%
----------------------------------------------------------------------------------
Score Ham (4)            -1.900    -1.900    -1.900    -1.900    -1.900    -1.900
==================================================================================
======================================================================================================
Spam Score Frequency      <= -10    <= -5      <= 0      <= 5    <= 10    <= 20    <= 30      > 30
------------------------------------------------------------------------------------------------------
Hits (4)                      0        0        4        0        0        0        0        0
Percent of Hits            0.00%    0.00%  100.00%    0.00%    0.00%    0.00%    0.00%    0.00%
======================================================================================================
</pre>
/usr/local/bin/postfix-logwatch output example:
<pre>
[root@zcs806 amavis-logwatch-1.51.02]# /usr/local/bin/postfix-logwatch /var/log/zimbra.log
****** Summary *************************************************************************************
      1  *Warning: Queue file size limit exceeded
  6.512K  Bytes accepted                              6,668
  1.928K  Bytes sent via SMTP                          1,974
  4.584K  Bytes sent via LMTP                          4,694
========  ==================================================
      10  Accepted                                    76.92%
      3  Rejected                                    23.08%
--------  --------------------------------------------------
      13  Total                                      100.00%
========  ==================================================
      3  5xx Reject message size                    100.00%
--------  --------------------------------------------------
      3  Total 5xx Rejects                          100.00%
========  ==================================================
      10  Connections
      10  Disconnections
      8  Removed from queue
      4  Sent via SMTP
      4  Sent via LMTP
      4  Filtered
****** Detail (10) *********************************************************************************
      3  5xx Reject message size -----------------------------------------------------------------
      3      192.168.1.166    remote.domain.com
      3        *unavailable
      3            *unavailable
      4  Sent via SMTP ---------------------------------------------------------------------------
      4      zcs806.DOMAIN.com
      4  Sent via LMTP ---------------------------------------------------------------------------
      4      zcs806.DOMAIN.com
      4  Filtered --------------------------------------------------------------------------------
      2      smtp-amavis:[127.0.0.1]:10024
      2        Sender address
      1            admin@zcs806.DOMAIN.com
      1              admin@zcs806.DOMAIN.com
      1                  192.168.1.166    remote.domain.com
      1            user@DOMAIN.com
      1              admin@zcs806.DOMAIN.com
      1                  192.168.1.184    remote2.domain.com
      2      smtp-amavis:[127.0.0.1]:10026
      2        Sender address
      1            admin@zcs806.DOMAIN.com
      1              admin@zcs806.DOMAIN.com
      1                  192.168.1.166    remote.domain.com
      1            user@DOMAIN.com
      1              admin@zcs806.DOMAIN.com
      1                  192.168.1.184    remote2.domain.com
=== Delivery Delays Percentiles ============================================================
                    0%      25%      50%      75%      90%      95%      98%      100%
--------------------------------------------------------------------------------------------
Before qmgr      0.04      0.09      0.11      0.11      0.23      0.35      0.43      0.48
In qmgr          0.00      0.00      0.01      0.01      0.04      0.07      0.08      0.09
Conn setup        0.00      0.01      0.29      1.30      2.05      2.23      2.33      2.40
Transmission      0.10      2.81      4.85      9.60    21.00    21.00    21.00    21.00
Total            0.20      2.91      5.20    11.00    23.30    23.65    23.86    24.00
============================================================================================
</pre>
'''Note''' - First, look at the options each command has using the -h output. You might want to use --full when doing an investigation and also include a wildcard - /var/log/zimbra.lo* to take in all the log data. For example:
<pre>
[zimbra@zcs806 ~]$ /usr/local/bin/postfix-logwatch --full /var/log/zimbra.lo*
****** Summary *************************************************************************************
      9  *Fatal:  General fatal
      1  *Warning: Queue file size limit exceeded
      21  Miscellaneous warnings
710.888K  Bytes accepted                            727,949
193.036K  Bytes sent via SMTP                        197,669
520.114K  Bytes sent via LMTP                        532,597
========  ==================================================
    1041  Accepted                                    99.71%
      3  Rejected                                    0.29%
--------  --------------------------------------------------
    1044  Total                                      100.00%
========  ==================================================
      3  5xx Reject message size                    100.00%
--------  --------------------------------------------------
      3  Total 5xx Rejects                          100.00%
========  ==================================================
      65  Connections
      65  Disconnections
    1041  Removed from queue
    523  Sent via SMTP
    517  Sent via LMTP
      2  Bounced (local)
      1  Bounced (remote)
      9  Filtered
      2  Notifications sent
      4  Timeouts (inbound)
      1  PIX workaround enabled
****** Detail (10) *********************************************************************************
      9  *Fatal:  General fatal -----------------------------------------------------------------
      3      Queue report unavailable - mail system is down
      3      Usage: sendmail [options]
      2      The Postfix mail system is not running
      1      Usage: send-mail [options]
      21  Miscellaneous warnings ------------------------------------------------------------------
      7      or the command is run from a set-uid root process
      7      the Postfix sendmail command has set-uid root file permissions
      7      the Postfix sendmail command must be installed without set-uid root file permissions
      3  5xx Reject message size -----------------------------------------------------------------
      3      10.X.X.166    fence.DOMAIN.com
      3        *unavailable
      3            *unavailable
    523  Sent via SMTP ---------------------------------------------------------------------------
    507      86.lab
      15      zcs806.DOMAIN.com
      1      domaina.com
    517  Sent via LMTP ---------------------------------------------------------------------------
    507      86.lab
      10      zcs806.DOMAIN.com
      2  Bounced (local) -------------------------------------------------------------------------
      2      5.0.0: Permanent failure: Other/Undefined status: Other undefined status
      2        zcs806.DOMAIN.com
      2            Zcs806.DOMAIN.com
      1              subject:test
      1              zimbra
      1  Bounced (remote) ------------------------------------------------------------------------
      1      5.0.0: Permanent failure: Other/Undefined status: Other undefined status
      1        domain.com
      1            user
      1              64.X.X.28      sentry.DOMAIN.com
      1                  505 5.0.0 Unknown recipient: RCPT TO
      9  Filtered --------------------------------------------------------------------------------
      7      smtp-amavis:[127.0.0.1]:10026
      7        Sender address
      3            admin@zcs806.DOMAIN.com
      3              admin@zcs806.DOMAIN.com
      2                  10.X.X.36    zcs806.DOMAIN.com
      1                  10.X.X.166    gatewayXX.DOMAIN.com
      2            zimbra@zcs806.DOMAIN.com
      2              admin@zcs806.DOMAIN.com
      2                  10.X.X.36    zcs806.DOMAIN.com
      1            ajcody@DOMAIN.com
      1              admin@zcs806.DOMAIN.com
      1                  10.X.X.184    edgeXX.DOMAIN.com
      1            san5@zcs806.DOMAIN.com
      1              b@zcs806.DOMAIN.com
      1                  10.X.X.36    zcs806.DOMAIN.com
      2      smtp-amavis:[127.0.0.1]:10024
      2        Sender address
      1            admin@zcs806.DOMAIN.com
      1              admin@zcs806.DOMAIN.com
      1                  10.X.X.166    gatewayXX.DOMAIN.com
      1            ajcody@DOMAIN.com
      1              admin@zcs806.DOMAIN.com
      1                  10.X.X.184    edgeXX.DOMAIN.com
      2  Notifications sent ----------------------------------------------------------------------
      2      Non-delivery
      2        sender
      4  Timeouts (inbound) ----------------------------------------------------------------------
      4      After END-OF-MESSAGE
      1  PIX workaround enabled ------------------------------------------------------------------
      1      disable_esmtp delay_dotcrlf
      1        64.X.X.28      sentry.DOMAIN.com
=== Delivery Delays Percentiles ============================================================
                    0%      25%      50%      75%      90%      95%      98%      100%
--------------------------------------------------------------------------------------------
Before qmgr      0.01      0.03      0.06      0.14      0.27      0.34      0.48      2.60
In qmgr          0.00      0.00      0.08    117.50    193.00    216.00    231.48    246.00
Conn setup        0.00      0.00      0.00      0.00      0.02      0.04      0.22    20.00
Transmission      0.05      0.09      3.60      9.80    10.00    10.00    11.00    160.00
Total            0.07      0.13      3.80    129.00    203.00    226.00    241.64    259.00
============================================================================================
</pre>
======zmaccts======
One way to note accounts that are actively logging in vs. those that aren't, can help shrink the number of accounts you might want to investigate or monitor. [example below, I cut out a lot of the  accounts]
<pre>
          account                          status            created      last logon
------------------------------------  -----------    ---------------  ---------------
zcstest001@zcs806.DOMAIN.com                active      01/20/14 18:47  03/02/14 21:11
zcstest002@zcs806.DOMAIN.com                active      01/30/14 01:48  02/19/14 00:07
admin-20140415@zcs806.DOMAIN.com            active      04/15/14 14:42            never
archtest-prod-20140402@zcs806.DOMAIN        active      04/02/14 07:42            never
          account                          status            created      last logon
------------------------------------  -----------    ---------------  ---------------
bruce@test1.lab                            active      02/22/14 09:32            never
test.cal@test1.lab                          active      04/06/14 05:35  04/06/14 05:35
test200@test1.lab                          active      04/12/14 00:50            never
                                domain summary
    domain                  active    closed    locked    maintenance    total
-----------------------  --------  --------  --------  -------------  --------
test1.lab                        3        0        0              0        3
test2.com                        2        0        0              0        2
angad.com                        2        0        0              0        2
test.test                        3        0        0              0        3
test.DOMAIN.com                  6        0        0              0        6
zcs806.DOMAIN.com              58        0        0              0        58
zcs806.DOMAIN.com                2        0        0              0        2
</pre>
=====By Authentication Attempts=====
A fast way to see who is doing a lot of authentications, which normally happens when a spammer has compromised an account with a weak password, is to do:
<pre>
# cat /var/log/zimbra.log | sed -n 's/.*sasl_username=//p' | sort | uniq -c | sort -n
      1 Auser@domain.com
      3 Buser@domain.com
      4 Cuser@domain.com
      5 Duser@domain.com
    36 SPAMMER@domain.com
</pre>
'''Note - This might take a long time, if so - try pruning it down'''
Example:
<pre>
# cat /var/log/zimbra.log | grep sasl_username > /tmp/zimbra_sasl_username.txt
# cat /tmp/zimbra_sasl_username.txt | sed -n 's/.*sasl_username=//p' | sort | uniq -c | sort -n
      1 Auser@domain.com
      3 Buser@domain.com
      4 Cuser@domain.com
      5 Duser@domain.com
    36 SPAMMER@domain.com
</pre>
The full log event will look like this:
<pre>
zimbra1 postfix/smtpd[29431]: B28914D5978: client=xxxxx.server.com[w.x.y.z], sasl_method=LOGIN, sasl_username=user
zimbra1 postfix/cleanup[5522]: B28914D5978: message-id=<20090420154255.B28914D5978@zimbraserver.com>
zimbra1 postfix/qmgr[20690]: B28914D5978: from=<spam@spam.com>, size=6026, nrcpt=10 (queue active)
zimbra1 postfix/cleanup[3983]: 2BA56465D28: message-id=<20090420154255.B28914D5978@zimbraserver.com>
</pre>
Against your older logs, you could:
<pre>
# zcat /var/log/zimbra.log* | sed -n 's/.*sasl_username=//p' | sort | uniq -c | sort -n
</pre>
And you can look at the specific information for the user in question with:
<pre>
# grep -C2 "sasl_username=SPAMMER@domain.com" /var/log/zimbra.log
</pre>
Or if searching against the older logs:
<pre>
# zgrep -C2 "sasl_username=SPAMMER@domain.com" /var/log/zimbra.log*
</pre>
If you want to check on a specific message ID, do:
<pre>
grep 9DF7520804A /var/log/zimbra.log*
</pre>
For older message logs:
<pre>
zgrep 9DF7520804A /var/log/zimbra.log*
</pre>
To read/view the message in the queue:
/opt/zimbra/postfix/sbin/postcat -q 9DF7520804A
One would then normally lock/change password on the one account showing the most activity. Grep'ing the /var/log/zimbra.log with the username in question will also show the ip address being used, this can be blocked with your firewall.
To be alerted of a compromised account and have it lock automatically see below. Slightly modified from this reference : http://www.zimbra.com/forums/administrators/62613-identify-compromised-accounts.html#post278732 :
<pre>
#!/bin/bash
# checks log file and gets a count of authentications sent per minute, per user
# and if the count exceeds the maxmails value the user's account is locked.
logfile="/var/log/zimbra.log"
maxmails="10"
mydomain="example.com"
support="<postmaster-userid>@$mydomain"
accounts="/tmp/active_accounts"
su zimbra -c "/opt/zimbra/bin/zmaccts" | grep "@" | grep active | awk '{print $1}' > $accounts
zgrep -i "auth ok" $logfile | sed 's/  / /g' | awk -F"[ :]" '{print $3":"$4,$11;}' | uniq -c | sort -n | \
while read line
do
    count=`echo ${line} | cut -d' ' -f 1`
    userid=`echo ${line} | cut -d' ' -f 3`
    timestamp=`echo ${line} | cut -d' ' -f 2`
    active=`grep "$userid@$mydomain" $accounts`
    if [ "$count" -gt "$maxmails" ] && [ "$active" == "$userid@$mydomain" ]; then
        echo "Maximum email rate exceeded, $userid@$mydomain will be locked"
        su zimbra -c "/opt/zimbra/bin/zmprov ma $userid@$mydomain zimbraAccountStatus locked"
        subject="$userid account locked due to excessive connections"
        # Email text/message
        message="/tmp/emailmessage.txt"
        echo "$userid account has been locked as there were $count connections made at"> $message
        echo "$timestamp.  Please have the user change their password, and check for phishing" >>$message
        echo "emails if possible." >>$message
        # send an email using /bin/mail
        /usr/bin/mail -s "$subject" "$support" < $message
        rm -f $message
        #update list of active accounts
        su zimbra -c "/opt/zimbra/bin/zmaccts" | grep "@" | grep active | awk '{print $1}' > $accounts
    fi
done
rm -f $accounts
</pre>
Then run it as a cron job.  The frequency will depend on the number of accounts you're managing.
<pre>
* * * * * /opt/zimbra/find_spammer.sh
</pre>
=====By Connecting IP - Useful For Blocking IP At Firewall=====
See also the following:
* http://wiki.zimbra.com/wiki/Log_Files#Logging_the_Originating_IP
To find the originating IP address of where the emails are coming from:
grep 'connect from' /var/log/zimbra.log | sed 's/.*connect from.*\[\(.[^]]*\)\]/\1/g' | sort | uniq -c | sort -nr | head
To check your older logs [example output below]:
<pre>
zgrep 'connect from' /var/log/zimbra.log* | sed 's/.*connect from.*\[\(.[^]]*\)\]/\1/g' | sort | uniq -c | sort -nr | head
    36 10.137.xx.34
    34 127.0.0.1
</pre>
=====Open Relay Check=====
You should also confirm you aren't an open relay.
<pre>
$ host -t mx DOMAIN.com
DOMAIN.com mail is handled by 10 mail.DOMAIN.com.
$ telnet mail.DOMAIN.com 25
Trying 184.###.##.## ...
Connected to mail.DOMAIN.com.
Escape character is '^]'.
220 mail.DOMAIN.com ESMTP Postfix
helo support.test
250 mail.DOMAIN.com
mail from:<SPAMMER@domain.com>
250 2.1.0 Ok
rcpt to:<TEST@DOMAIN.COM>
554 5.7.1 <TEST@DOMAIN.COM>: Relay access denied
rcpt to:<SPAMMER@domain.com>
554 5.7.1 Service unavailable; Client host [71.XXX.XX.XX] blocked
using zen.spamhaus.org; http://www.spamhaus.org/query/bl?ip=71.202.XX.XX
quit
221 2.0.0 Bye
Connection closed by foreign host.
</pre>
=====Telnet Test To Confirm/Show Authentication Required For SMTP/Port 25=====
This is an example:
<pre>
esx2:~ ajcody$ telnet zcs723.EXAMPLE.com 25
Trying 10.137.27.32...
Connected to zcs723.EXAMPLE.com.
Escape character is '^]'.
220 zcs723.EXAMPLE.com ESMTP Postfix
helo zcs723.EXAMPLE.com  << I typed
250 zcs723.EXAMPLE.com
mail from:ajcody@zcs723.EXAMPLE.com  << I typed
250 2.1.0 Ok
rcpt to:ajcody2@zcs723.EXAMPLE.com  << I typed
553 5.7.1 <ajcody@zcs723.EXAMPLE.com>: Sender address rejected: not logged in
</pre>
But note - if you do this from the ZCS server or a server that is within the ip range or has it's specific ip listed in the mynetworks, you will not get this authentication requirement.
<pre>
[root@zcs723 ~]# telnet localhost 25
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 zcs723.EXAMPLE.com ESMTP Postfix
helo myworkstation
250 zcs723.EXAMPLE.com
ehlo myworkstation
250-zcs723.EXAMPLE.com
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-AUTH LOGIN PLAIN
250-AUTH=LOGIN PLAIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
mail from:<ajcody@zcs723.EXAMPLE.com>
250 2.1.0 Ok
rcpt to: <ajcody2@zcs723.EXAMPLE.com>
250 2.1.5 Ok
data
354 End data with <CR><LF>.<CR><LF>
From: Adam <ajcody@zcs723.EXAMPLE.com>
To: Adam 2 <ajcody2@zcs723.EXAMPLE.com>
Subject: From Localhost - NOT Auth
test
.
250 2.0.0 Ok: queued as 8B19E1E78D1
quit
221 2.0.0 Bye
Connection closed by foreign host.
</pre>


====Resources====
====Resources====
Line 197: Line 1,137:
A list of resources you'll find useful:
A list of resources you'll find useful:


http://wiki.zimbra.com/index.php?title=Zimbra_MTA#Anti-Spam_Training_Filters
* [[Zimbra_MTA#Anti-Spam_Training_Filters]]
* [[CLI_zmtrainsa]]
* [[Improving_Anti-spam_system]]
* [[Postfix_Policyd]]
* [[IP_Address_whitelisting]]
* [[Spam_training]]
*  Restrict by user
** [[RestrictPostfixRecipients]]
* Restrict by ip addresses and sender and other items:
** [http://www.postfix.org/RESTRICTION_CLASS_README.html Postfix - Restriction Class Readme]
*** Note, from the readme:
**** "What follows is based on the SMTP client IP address, and therefore is subject to IP spoofing."
**** "What follows is based on the sender SMTP envelope address, and therefore is subject to SMTP sender spoofing."
 
Wiki articles that have been assigned to the anit-spam category:
 
* [http://wiki.zimbra.com/index.php?title=Category:Anti-spam Category:Anti-spam]
 
Down to the end-user:
 
* [[Cool_User_Spam_Filters]]
* [http://www.zimbra.com/community/end_user_guide_and_how_to.html End-User Guide And How-To]
 
=====External Relay Test Pages=====
 
* http://www.checkor.com/
** Note - this test is in regards to the From spoofing spammers sometimes do for DL's.
** Also, for the test - make an account/DL on your system for test1@[your domain] . Otherwise you'll just error about account not existing.
<pre>
RSET
250 2.0.0 Ok
MAIL FROM: spam@mail59.DOMAIN.com
250 2.1.0 Ok
RCPT TO: test1@mail59.DOMAIN.com
Test Failed, 250 2.1.5 Ok
</pre>
 
* http://www.mailradar.com/openrelay/
 
====Blocking MAIL FROM - smtpd_sender_restrictions - Default Is Empty====
 
=====External References=====
 
* External Sources
** Postfix
*** [http://www.postfix.org/postconf.5.html#smtpd_sender_restrictions Postfix On smtpd_sender_restrictions]
** Milter
*** [http://www.postfix.org/MILTER_README.html#limitations Postfix's Milter Readme - Limitations]
**** [http://puszcza.gnu.org.ua/software/mailfromd/ Mailfromd]
***** [http://puszcza.gnu.org.ua/software/mailfromd/manual/html_section/SAV.html#SEC7 Mailfromd - Sender Address Verification]
** [http://www.symantec.com/connect/articles/anti-spam-solutions-and-security Anti-Spam Solutions and Security]
 
=====Zimbra References And Bugs & RFE's=====
 
* Zimbra Related Soures
** "policy for who can send to a distribution lists"
*** https://bugzilla.zimbra.com/show_bug.cgi?id=9620
**** '''Note - This will not stop spammers from mailing into your DL's by way of forged Mail From and guessing your DL address in the To'''
** "support smtpd_sender_login_maps for smtp auth"
*** http://bugzilla.zimbra.com/show_bug.cgi?id=11258
** "Implement smtpd_sender_restrictions"
*** http://bugzilla.zimbra.com/show_bug.cgi?id=15808
** "milter to check if sender can send to a distribution list"
*** https://bugzilla.zimbra.com/show_bug.cgi?id=46311
**** Dependent upon bug 9620
** Zimbra Forum Post on using smtpd_sender_restrictions options
*** [http://www.zimbra.com/forums/administrators/28770-how-enforce-sasl_username-address.html  How to enforce sasl_username=FROM ADDRESS"
** Another Zimbra Forum Post on using smtpd_sender_restrictions options
*** [http://www.zimbra.com/forums/administrators/39095-need-urgent-help-spamming-issue.html Need urgent help on spamming issue]
 
=====Protecting DL's From Spammers - Forging Mail From=====
 
======First Recommendation - As Given By Dev's From Critical Meeting Notes======
 
----
 
* Enabled SASL/SMTP Authentication
** Ref: http://wiki.zimbra.com/index.php?title=SMTP_Auth_Problems
* Implement how-to as described in :
** Permitted Senders: [[RestrictPostfixRecipients]]
*** '''Note: You'll see on the above page a reference to the spoof hole.'''
**** '''''"This method can be spoofed by forging the MAIL FROM: header (so mail appears to originate from within the domain), so it isn't foolproof, but it works for basic needs."'''''
*** '''Note: You'll also modify the instructions as above with addition details provided below.'''
* Force authentication for local-domain senders:
** modify the main.cf to have the following:
*** smtpd_sender_restrictions = check_sender_access hash:/path/to/file
** Then for the /path/to/file that you used in the mail.cf for smtpd_sender_restrictions, you'll have a line like:
*** example.com            permit_sasl_authenticated, reject
 
======Second Recommendation - Unpredictable DL name or Non-routing Domain======
 
----
 
Do not use predictable DL names. Instead of using everyone@company.com , use something like everyone-[random-string]@company.com .
 
Another option is to use a non-routing domain - company.local - and setup your DL's there. You'll want to configure your main domain to be able to query the GAL of this domain.
 
To see the existing setting:
 
zmprov gd [domainname] zimbraGalLdapSearchBase
 
To change the variable for the domain:
 
zmprov md [domainname] zimbraGalInternalSearchBase ROOT
 
======Third Recommendation - Using smtpd_sender_restrictions======


http://wiki.zimbra.com/index.php?title=CLI_zmtrainsa
----


http://wiki.zimbra.com/index.php?title=Improving_Anti-spam_system
:::'''Work In Progress. I'm testing this now. Please don't attempt until this line is removed.'''


http://wiki.zimbra.com/index.php?title=Postfix_Policyd
Update: See the following:
* "Enforcing a match between the FROM Address and sasl_username in Zimbra Collaboration Server (2011281)"
** http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2011281


http://wiki.zimbra.com/index.php?title=IP_Address_whitelisting
This should work if your "clients" are using ZWC, ZCO, or ActiveSync mobile devices. If you are using IMAP/POP + STMP thick clients, you'll most likely have to enable smtp authentication [sasl] and use the reject_authenticated_sender_login_mismatch variable instead.


http://wiki.zimbra.com/index.php?title=Spam_training
postconf -e smtpd_sender_restrictions=reject_sender_login_mismatch
postfix reload


Wiki articles that have been assigned to the anit-spam category:
This option is described as:


http://wiki.zimbra.com/index.php?title=Category:Anti-spam
:'''''reject_sender_login_mismatch'''''
:: ''Reject the request when $smtpd_sender_login_maps  specifies an owner for the MAIL FROM address, but the client is not (SASL) logged in as that MAIL FROM address owner; or when the client is (SASL) logged in, but the client login name doesn't own the MAIL FROM address according to $smtpd_sender_login_maps.''  [http://www.postfix.org/postconf.5.html#smtpd_sender_restrictions Man page]


Down to the end-user:
====Some Other SMTP Sending Restrictions====


http://wiki.zimbra.com/index.php?title=Cool_User_Spam_Filters
=====Blocking Incoming From Domain And By User=====


http://www.zimbra.com/community/end_user_guide_and_how_to.html
See the following:


====Some SMTP Sending Restrictions====
* [[Domain_level_blocking_of_users]]
* [[Improving_Anti-spam_system#Implementing_Whitelist.2FBlacklist]]


=====check_client_access=====
=====check_client_access=====
Line 298: Line 1,348:
===Using Different SMTP Server For Webclient (ZWC), Mobiles, And ZCO===
===Using Different SMTP Server For Webclient (ZWC), Mobiles, And ZCO===


There a variable called zimbraSmtpHostname that is in the global (zmprov gacf) and server (zmprov gs `hostname`) configs. It's defaulted to 'localhost' - at least on a single ZCS configuration.
====Note Of Caution About Using External MTAs====
 
Using non-zimbra MTA's can cause some options in zimbra to not function anymore - since it no longer has zimbra's mta services available.
 
=====Zimbra Mail Forwarding Possibly Will Not Work - Turn Off User Option To Set MailForwarding=====
 
Mail forwarding might no longer work depending on the configuration you setup regarding the use of your external mta's. When this happens you'll most likely want to disable the option for users to set a mail forwarding address in their preferences. This can be done via their COS or USER configuration.
 
* In the admin console, goto the COS configuration the user/s are using and the "Features" tab. Uncheck the option "Allow the user to specify a forwarding address" under Mail Features. It is in the same location under a USERs configuration panel in the admin console.
 
In the CLI, you will see these set as the defaults for the default COS. The admin gui option above only adjusts the zimbraFeatureMailForwardingEnabled variable :
 
$ zmprov gc default | grep zimbraFeatureMailForwarding
    zimbraFeatureMailForwardingEnabled: TRUE
    zimbraFeatureMailForwardingInFiltersEnabled: TRUE
 
$ zmprov ga ajcody@`zmhostname` | grep zimbraFeatureMailForwarding
    zimbraFeatureMailForwardingEnabled: TRUE
    zimbraFeatureMailForwardingInFiltersEnabled: TRUE
 
More details about them are in the /opt/zimbra/conf/attrs/zimbra-attrs.xml file.
 
<pre>
<attr id="342" name="zimbraFeatureMailForwardingEnabled" type="boolean" cardinality="single"
optionalIn="account,cos" flags="accountInfo,accountInherited,domainAdminModifiable">
  <defaultCOSValue>TRUE</defaultCOSValue>
  <desc>enable end-user mail forwarding features</desc>
</attr>


In cases where you need/want all mail to be processed by, for example, an external non-Zimbra spam filter box you could set this variable to the spam filter boxes hostname.
<attr id="704" name="zimbraFeatureMailForwardingInFiltersEnabled" type="boolean"
cardinality="single" optionalIn="account,cos" flags="accountInfo,accountInherited,
domainAdminModifiable" since="5.0.10">
  <defaultCOSValue>TRUE</defaultCOSValue>
  <desc>enable end-user mail forwarding defined in mail filters features</desc>
</attr>
</pre>


Global:
=====Configure External MTA To Use LDAP Virtual Alias Maps=====


zmprov mcf zimbraSmtpHostname hostname-of-ext-server
Here's the basic info in regards to how Zimbra's mta [postfix/etc] uses Zimbra's LDAP to get the forwarding information:


Per Server:
$ grep Forward conf/ldap-*
conf/ldap-vam.cf:result_attribute = 
zimbraMailDeliveryAddress,zimbraMailForwardingAddress,
zimbraPrefMailForwardingAddress,zimbraMailCatchAllForwardingAddress


  zmprov ms servername zimbraSmtpHostname hostname-of-ext-server
  $ postconf |grep vam
virtual_alias_maps = proxy:ldap:/opt/zimbra/conf/ldap-vam.cf


===Global Or System Wide Filters===
See http://www.postfix.org/postconf.5.html#virtual_alias_maps for more information.


There is no "supported" solution for this - depending on the exact circumstances. It's a complex issue because each request for "global filters" tends to be very specific on either what's to be filter, what actions are to be taken, and so forth.
====Confirming And Setting zimbraMtaRelayHost And zimbraMtaDnsLookupsEnabled====


====RFE's Related To Global Filters====
First we'll set '''''zimbraMtaRelayHost''''' and '''''zimbraMtaDnsLookupsEnabled''''' variables. These options are also shown in the admin console and can be configured there. These variable alone will not redirect ALL traffic to an external MTA first though. There's alao a variable called zimbraSmtpHostname that is in the global (zmprov gacf) and server (zmprov gs `hostname`) configs - addressed in the section below. It's defaulted value is 'localhost' - at least on a single ZCS configuration.


* "Define a default filter rule for spam that users can apply before custom filters"
In situations where you need/want all mail to be processed by, for example, an external non-Zimbra spam filter box you could set this variable to the spam filter servers hostname.
** http://bugzilla.zimbra.com/show_bug.cgi?id=12701
* "admin assignable mail filters"
** http://bugzilla.zimbra.com/show_bug.cgi?id=35452


====Use The Legal Intercept Method====
Normally, when zimbraMtaRelayHost is set to a non-zimbra external MTA would disable DNS lookups. If you disable DNS Lookups (under the MTA tab of the admin console, or with zmprov), Zimbra will end up using (according to the postconf man page) the "gethostbyname() system library routine which normally also looks in /etc/hosts" (based on the entries on the "hosts" line in /etc/nsswitch.conf). If you do this but don't also specify an SMTP relay host (typically your ISP's SMTP server), which will take care of checking DNS, you will reverse your ability to send mail: suddenly you can send mail to other users on the Zimbra server, but you can't send to the internet (though you can still receive mail from the internet either way).


Depending on the details of your request, using the Legal Intercept options might be useful. You could take the results that goto the LI mailbox and then perform your admin global searches against your mailbox accounts to take the appropriate action.
Query Global - `zmhostname` would use the value returned, you can manual type out the servername as well :


See [[Legal_Intercept]]
zmprov gacf zimbraMtaRelayHost
zmprov gacf zimbraMtaDnsLookupsEnabled


====Sieve Filter Set For Every Account====
Query Per Server - `zmhostname` would use the value returned, you can manual type out the servername as well :


One could setup a forloop for all of your users and setup a sieve filter. Downside here is the rules are editable by the users and you would have to manage the rules for new accounts going forward.
zmprov gs `zmhostname` zimbraMtaRelayHost
zmprov gs `zmhostname` zimbraMtaDnsLookupsEnabled


See [[User_Migration#Migrating_Sieve_Filter_Rules ]] for details.
Note - if you get errors about doing the query on your non-mailstores like, "ERROR: zclient.IO_ERROR (invoke Connection refused, server: localhost) (cause: java.net.ConnectException Connection refused)" then you might need to adjust this variable. First query it, it's most likely set to localhost


====Double Check The Current Anti-Spam Options====
zmlocalconfig zimbra_zmprov_default_soap_server


Make sure your request can't be solved by the current solutions described in [[Category:Anti-spam]] articles.
If you had the error and it was set to localhost, modify it to be one of your mailstores.


====Postfix , Amavis Customizations====
zmlocalconfig -e zimbra_zmprov_default_soap_server=mailstore.example.com


I've seen some posts on the forums that customers found their own workarounds by customizing postfix and amavis. This will most likely result in an unsupported situation. Unfortunately, those forum posts don't also include details that I can share here. Hopefully I can find them going forward and I'll post them here.
No restart of anything is needed, the zmprov query should now work.


===Global Disclaimer Options===
Modify Global - `zmhostname` would use the value returned, you can manual type out the servername as well :


Here's the url to review for a "current" possibility:
zmprov mcf zimbraMtaRelayHost hostname-of-ext-server:PORT
  ** ex : zmprov mcf zimbraMtaRelayHost primary.YYY.state.XX.us:25
  ** At the end, 25 is the port number for smtp on the targeted system.
  ** Adjust this number if you changed the smtp port.
zmprov mcf zimbraMtaDnsLookupsEnabled FALSE


* http://wiki.zimbra.com/index.php?title=Domain_Disclaimer_Extension_Admin_UI
Modify Per Server - `zmhostname` would use the value returned, you can manual type out the servername as well :


And in the notes section there's a comment about multi-servers:
zmprov ms `zmhostname` zimbraMtaRelayHost hostname-of-ext-server:PORT
  ** ex : zmprov ms `zmhostname` zimbraMtaRelayHost primary.YYY.state.XX.us:25
  ** At the end, 25 is the port number for smtp on the targeted system.
  ** Adjust this number if you changed the smtp port.
zmprov ms `zmhostname` zimbraMtaDnsLookupsEnabled FALSE


* http://wiki.zimbra.com/index.php?title=Talk:Domain_Disclaimer_Extension_Admin_UI
====Confirming And Setting zimbraSmtpHostname====


Please note though, "This article is a community contribution and may include unsupported customizations." Meaning, it's an unsupported customization, so please take the necessary precautions.
I'm assuming you already set, '''''zimbraMtaRelayHost''''' and '''''zimbraMtaDnsLookupsEnabled''''' for your needs - see above section. The variable called zimbraSmtpHostname is in the global (zmprov gacf) and server (zmprov gs `hostname`) configs. It's default value is set to 'localhost' - at least on a single ZCS configuration.


In regards to an official and supported way to do this, please review this RFE:
In cases where you need/want all mail to be processed by, for example, an external non-Zimbra spam filter box you could set this variable to the spam filter boxes hostname.


* http://bugzilla.zimbra.com/show_bug.cgi?id=4720
Query Global - `zmhostname` would use the value returned, you can manual type out the servername as well :


===Quota Issues===
zmprov gacf zimbraSmtpHostname


====Where To Adjust Message User Gets When They Are Over Quota====
Query Per Server - `zmhostname` would use the value returned, you can manual type out the servername as well :


* From the web admin console:
zmprov gs `zmhostname` zimbraSmtpHostname
** Configuration > Class of Service
** Select the COS in question
** Then goto the Advanced tab on the right
** There's a quota section. The sub-section you want is called:
*** "Quota warning message template:"


====Controlling Behavior For Messages Sent To Over Quota Mailbox====
Note - if you get errors about doing the query on your non-mailstores like, "ERROR: zclient.IO_ERROR (invoke Connection refused, server: localhost) (cause: java.net.ConnectException Connection refused)" then you might need to adjust this variable. First query it, it's most likely set to localhost


The variable to set for a 452 Temp/Try Again response verses a 552 Permanent Error.
zmlocalconfig zimbra_zmprov_default_soap_server


zmprov gacf zimbraLmtpPermanentFailureWhenOverQuota
If you had the error and it was set to localhost, modify it to be one of your mailstores.


Setting to TRUE will flag it for the 552 response.
zmlocalconfig -e zimbra_zmprov_default_soap_server=mailstore.example.com


====Message Senders Receive About Mailbox Over Quota====
No restart of anything is needed, the zmprov query should now work.


File that holds text of message:
Modify Global - `zmhostname` would use the value returned, you can manual type out the servername as well :


  /opt/zimbra/postfix/conf/bounce.cf.default
  zmprov mcf zimbraSmtpHostname hostname-of-ext-server
  ** ex : zmprov mcf zimbraSmtpHostname primary.YYY.state.XX.us


Note, please read the [http://www.postfix.org/bounce.5.html bounce MAN] page before you attempt to edit this file directly.
Modify Per Server - `zmhostname` would use the value returned, you can manual type out the servername as well :


Also, I haven't been able to confirm the relationship of this above file with the postconf default output:
zmprov ms `zmhostname` zimbraSmtpHostname hostname-of-ext-server
  ** ex : zmprov ms `zmhostname` zimbraSmtpHostname primary.YYY.state.XX.us


<pre>
====External Email Clients Setting A SMTP Server====
[root@mail3 ~]# postconf | grep -i bounce
2bounce_notice_recipient = postmaster
backwards_bounce_logfile_compatibility = yes
bounce_notice_recipient = postmaster
bounce_queue_lifetime = 5d
bounce_service_name = bounce
bounce_size_limit = 50000
bounce_template_file =  
disable_verp_bounces = no
double_bounce_sender = double-bounce
multi_recipient_bounce_reject_code = 550
soft_bounce = no
</pre>


To use a bounce.cf file, you'll want to add the file to variable and reload postfix via the zmmtactl script. It looks like zmlocalconfig doesn't currently handle this variable.
The above variable, zimbraSmtpHostname, will not alter your third party email clients that are setting the smtp server to your ZCS mta's. Here are your options if you also need to have that traffic to go through another device [mta, spam filter, etc.] prior to local delivery [lmtp] to an internal address.


cp /opt/zimbra/postfix/conf/bounce.cf.default /opt/zimbra/postfix/conf/bounce.cf
* Set your clients to use the another devices ip address or hostname that you set for zimbraSmtpHostname.
postconf -e bounce_template_file="/opt/zimbra/postfix/conf/bounce.cf"
zmmtactl reload


Note, this might get lost during upgrades so make a note to yourself about this change.
If you can't do the above, for whatever reason -- maybe security constraints or issues that might arise being a hosting provider, then see below.


===Managing Postfix Queue===
# You could investigate the alternation of postfix's content_filter option to place the external device/host [a barracuda for example] within that process. postfix.org has information on this - [http://www.postfix.org/FILTER_README.html Postfix After-Queue Content Filter]. This would be unsupported by Zimbra.
# Contact Zimbra's Professional Services [PS] team for help.
# Setup another server using a mta of your choice [postfix , sendmail] that the clients can use for the smtp server variable. This "new" mta would then simply relay to the device - a barracuda box for example. The barracuda would then do what it needs to and then forwards the messages to the appropriate servers for delivery. Your ZCS mta's in the case of local delivery that would of normally occurred over lmtp - userA@domainC.com sending to userB@domainC.com .


====Postfix, Amavis, Clamav Spool Directory Paths And Names====
===Global Or System Wide Filters===


ls /opt/zimbra/data
There is no "supported" solution for this - depending on the exact circumstances. It's a complex issue because each request for "global filters" tends to be very specific on either what's to be filter, what actions are to be taken, and so forth.
  amavisd  clamav  dspam  postfix


ls /opt/zimbra/data/postfix/spool/
====RFE's Related To Global Filters====
  active  active.old  bounce  corrupt  defer  deferred 
  flush  hold  incoming  incoming.old  maildrop  pid 
  private  public  saved  trace


====Stop And Starting Postfix And Mta====
* "Define a default filter rule for spam that users can apply before custom filters"
** http://bugzilla.zimbra.com/show_bug.cgi?id=12701
* "admin assignable mail filters"
** http://bugzilla.zimbra.com/show_bug.cgi?id=35452


To only stop and start postfix:
====Use The Legal Intercept Method====


postfix stop
Depending on the details of your request, using the Legal Intercept options might be useful. You could take the results that goto the LI mailbox and then perform your admin global searches against your mailbox accounts to take the appropriate action.
postfix start


To stop and start postfix, amavis, and clam:
See [[Legal_Intercept]]


zmmtactl stop
====Sieve Filter Set For Every Account====
zmmtactl start


====To See Postfix Queue====
One could setup a forloop for all of your users and setup a sieve filter. Downside here is the rules are editable by the users and you would have to manage the rules for new accounts going forward.


/opt/zimbra/postfix/sbin/postqueue -p
See [[User_Migration#Migrating_Sieve_Filter_Rules ]] for details.


or
====Double Check The Current Anti-Spam Options====


mailq
Make sure your request can't be solved by the current solutions described in [[Category:Anti-spam]] articles.


====To Flush Postfix Queue====
====Postfix , Amavis Customizations====


postqueue -f
I've seen some posts on the forums that customers found their own workarounds by customizing postfix and amavis. This will most likely result in an unsupported situation. Unfortunately, those forum posts don't also include details that I can share here. Hopefully I can find them going forward and I'll post them here.


====To Delete Messages From Queue====
===Global Disclaimer Options===


=====Cautionary Note=====
Here's the url to review for a "current" possibility:


::'''Warning, deleting messages from the queue can have a negative consequence of your users. You might need to account for the action and/or confirm your deletion was appropriate. Please try to save the postqueue -p information from the various messages prior to deleting them. This will at least you give you he information to  later justify your actions on why you delete msg#.'''
* http://wiki.zimbra.com/index.php?title=Domain_Disclaimer_Extension_Admin_UI


=====Relevant Sections Of Postsuper Man Page=====
And in the notes section there's a comment about multi-servers:


<pre>
* http://wiki.zimbra.com/index.php?title=Talk:Domain_Disclaimer_Extension_Admin_UI
By  default,  postsuper(1)  performs  the operations requested with the -s and -p
command-line options on all Postfix queue directories - this includes the incoming,
active and deferred directories with mail files and the bounce, defer, trace and flush
directories with log files.


  -d queue_id Delete one message with the named queue ID from the named mail queue(s)
Please note though, "This article is a community contribution and may include unsupported customizations." Meaning, it's an unsupported customization, so please take the necessary precautions.
    (default: hold, incoming, active and deferred).
    If a queue_id of - is specified, the program reads queue IDs from standard input.
    For example, to delete all mail with exactly one recipient user@example.com:


              mailq | tail +2 | grep -v '^ *(' | awk  'BEGIN { RS = "" }
In regards to an official and supported way to do this, please review this RFE:
                  # $7=sender, $8=recipient1, $9=recipient2
                  { if ($8 == "user@example.com" && $9 == "")
                        print $1 }
              ' | tr -d '*!' | postsuper -d -


      Specify "-d ALL" to remove all messages; for example, specify "-d ALL deferred"
* http://bugzilla.zimbra.com/show_bug.cgi?id=4720
      to delete all mail in the deferred queue. As a safety measure, the word ALL must
      be specified in upper case.


      Warning: Postfix queue IDs are reused.  There is a very small possibility that
===Quota Issues===
      postsuper deletes the wrong message file when it is executed while the Postfix mail
      system is delivering mail.


      The scenario is as follows:
====Where To Adjust Message User Gets When They Are Over Quota====
      1)  The Postfix queue manager deletes the message that postsuper(1) is asked to
        delete, because Postfix is finished with the message (it is delivered, or it is
        returned to the sender).
      2)  New mail arrives, and the new message is given the same queue ID as the message
        that postsuper(1) is supposed to delete.  The probability for reusing a deleted
        queue ID is about 1 in 2**15 (the number  of  different microsecond values that
        the system clock can distinguish within a second).
      3)  postsuper(1) deletes the new message, instead of the old message that it should
        have deleted.


  -h queue_id Put mail "on hold" so that no attempt is made to deliver it.  Move one
Moved to [[How_To_Adjust_The_Over_Quota_Message_The_User_Receives]]
    message with the named queue ID from the named mail queue(s) (default: incoming,
    active and deferred) to the hold queue.


    If a queue_id of - is specified, the program reads queue IDs from standard input.
====See Current User Qoutas====
    Specify "-h ALL" to hold all messages; for example, specify "-h ALL deferred" to hold
    all mail in the deferred queue.  As a safety measure, the word ALL must be specified
    in upper case.
    Note: while mail is "on hold" it will not expire when its time in the queue exceeds
    the maximal_queue_lifetime or bounce_queue_lifetime setting. It becomes subject to
    expiration after it is released from "hold".


  -H queue_id Release mail that was put "on hold".  Move one message with the named queue
Moved to [[Getting_All_Users_Quota_Data]]
    ID from the named mail queue(s) (default: hold) to the deferred queue.


    If a queue_id of - is specified, the program reads queue IDs from standard input.
====Controlling Behavior For Messages Sent To Over Quota Mailbox - LMTP====
    Note: specify "postsuper -r" to release mail that was kept on hold for a significant
    fraction of $maximal_queue_lifetime or $bounce_queue_lifetime, or longer.


    Specify "-H ALL" to release all mail that is "on hold".  As a safety measure, the
Moved to [[Controlling_Behavior_For_Messages_Sent_To_Over_Quota_Mailbox_-_LMTP]]
    word ALL must be specified in upper case.


  -p Purge old temporary files that are left over after system or software crashes.
====Controlling Behavior For Messages Sent To Over Quota Mailbox - SMTP====
</pre>


=====To Delete Single Message From Queue=====
Moved to [[Controlling_Behavior_For_Messages_Sent_To_Over_Quota_Mailbox_-_SMTP]]


/opt/zimbra/postfix/sbin/postsuper -d [MSGID From postqueue -p]
====Message Senders Receive About Mailbox Over Quota====


=====To Delete ALL Messages From Queue=====
Moved to [[Message_Senders_Receive_About_Mailbox_Over_Quota]]


/opt/zimbra/postfix/sbin/postsuper -d ALL
====Quota Not Showing In Admin Console - After ZCS Upgrade====


Another way to do this:
Moved to [[Quota_Not_Showing_In_Admin_Console_-_After_ZCS_Upgrade]]


mailq | awk '{print $1}' | postsuper -d -
===Managing Postfix Queue===


======To Delete ALL Messages From The Deferred Queue======
Moved to [[Managing_The_Postfix_Queues#Managing_The_Postfix_Queues]]


/opt/zimbra/postfix/sbin/postsuper -d ALL deferred
====Postfix, Amavis, Clamav Spool Directory Paths And Names====


=====To Delete Many Messages From Queue=====
Moved to [[Managing_The_Postfix_Queues#Postfix.2C_Amavis.2C_Clamav_Spool_Directory_Paths_And_Names]]


To delete a large number of files one would use:
====Stop And Starting Postfix And Mta====


/opt/zimbra/postfix/sbin/postsuper -d - < filename-with-queue-ids.txt
Moved to [[Managing_The_Postfix_Queues#Stop_And_Starting_Postfix_And_Mta]]


The filename, filename-with-queue-ids.txt example, would have a listing of id's like:
====To See Postfix Queue====


<pre>
Moved to [[Managing_The_Postfix_Queues#To_See_The_Postfix_Queues]]
3E1C6CAFFFE
6B862CC9D76
0BC38CC1BC9
90628CC6F3C
E26B9CC3C62
92A35CC943D
A84BDBCE15D
EA57CB1DF04
0F102CC74CB
386E8CC4DFF
92606CC0BDA
0799FC8149A
024CFCBD0DE
2D30FC47DA0
31D85CC6308
B8B3FC3DEBC
AA4C7C913D0
280F5CC8C6C
9F341CC8A26
93CD1B3B0EC
433D0BF3716
A1435CB4C38
2DB04CC911D
56A29CC8819
11881C8268C
5C050A79851
C6739CC4BA5
11D3FCC7D09
8CBC0B20E0A
</pre>


=====Delete From Queue By Email Address=====
=====Qshape - Print Postfix queue domain and age distribution=====


======From CLI======
Moved to [[Managing_The_Postfix_Queues#Qshape_-_Print_Postfix_queue_domain_and_age_distribution]]


Change the [ email@address.com ] variable below first.
====To View A Message In The Queue====


To first see what would be deleted:
Moved to [[Managing_The_Postfix_Queues#To_View_A_Message_In_The_Queue]]


mailq | tail +2 | grep -v '^ *(' | awk  'BEGIN { RS = "" } { if ($8 == "email@address.com" && $9 == "") print $1 } ' | tr -d '*!'
====To Flush Postfix Queue====


To now delete, just include the postsuper -d at end:
Moved to [[Managing_The_Postfix_Queues#To_Flush_Postfix_Queue]]


mailq | tail +2 | grep -v '^ *(' | awk  'BEGIN { RS = "" } { if ($8 == "email@address.com" && $9 == "") print $1 } ' | tr -d '*!' | postsuper -d -
====To Requeue Messages In Postfix====


======Script To Delete From Queue By Email Address======
Moved to [[Managing_The_Postfix_Queues#To_Requeue_Messages_In_Postfix]]


::'''Non-Zimbra Script and not QA'd or tested. Path adjusted though for /opt/zimbra/postfix/sbin/*'''
====To Put Messages On Hold====


From http://www.ustrem.org/en/articles/postfix-queue-delete-en/
Moved to [[Managing_The_Postfix_Queues#To_Put_Messages_On_Hold]]


Save on file system, calling it something like - delete-queue-by-email.sh . Give it execute permission. Example usage would be: ./delete-queue-by-email.sh  user-name@domain-test.com
====To Delete Messages From Queue====


<pre>
Moved to [[Managing_The_Postfix_Queues#To_Delete_Messages_From_Queue]]
#!/usr/bin/perl -w
#
# pfdel - deletes message containing specified address from
# Postfix queue. Matches either sender or recipient address.
#
# Usage: pfdel <email_address>
#


use strict;
=====Cautionary Note=====


# Change these paths if necessary.
Moved to [[Managing_The_Postfix_Queues#Cautionary_Note]]
my $LISTQ = "/opt/zimbra/postfix/sbin/postqueue -p";
my $POSTSUPER = "/opt/zimbra/postfix/sbin/postsuper";


my $email_addr = "";
=====Relevant Sections Of Postsuper Man Page=====
my $qid = "";
my $euid = $>;


if ( @ARGV !=  1 ) {
Moved to [[Managing_The_Postfix_Queues#Relevant_Sections_Of_Postsuper_Man_Page]]
die "Usage: pfdel <email_address>\n";
} else {
$email_addr = $ARGV[0];
}


if ( $euid != 0 ) {
=====To Delete Single Message From Queue=====
        die "You must be root to delete queue files.\n";
}


Moved to [[Managing_The_Postfix_Queues#To_Delete_Single_Message_From_Queue]]


open(QUEUE, "$LISTQ |") ||
=====To Delete ALL Messages From Queue=====
  die "Can't get pipe to $LISTQ: $!\n";


my $entry = <QUEUE>; # skip single header line
Moved to [[Managing_The_Postfix_Queues#To_Delete_ALL_Messages_From_Queue]]
$/ = ""; # Rest of queue entries print on
# multiple lines.
while ( $entry = <QUEUE> ) {
if ( $entry =~ / $email_addr$/m ) {
($qid) = split(/\s+/, $entry, 2);
$qid =~ s/[\*\!]//;
next unless ($qid);


#
======To Delete ALL Messages From The Deferred Queue======
# Execute postsuper -d with the queue id.
# postsuper provides feedback when it deletes
# messages. Let its output go through.
#
if ( system($POSTSUPER, "-d", $qid) != 0 ) {
# If postsuper has a problem, bail.
die "Error executing $POSTSUPER: error " .
  "code " .  ($?/256) . "\n";
}
}
}
close(QUEUE);


if (! $qid ) {
Moved to [[Managing_The_Postfix_Queues#To_Delete_ALL_Messages_From_The_Deferred_Queue]]
die "No messages with the address <$email_addr> " .
  "found in queue.\n";
}


exit 0;
======To Delete ALL Messages From The Hold Queue======
</pre>


======Script To Delete From Queue By Various Variable Targets======
Moved to [[Managing_The_Postfix_Queues#To_Delete_ALL_Messages_From_The_Hold_Queue]]


::'''Non-Zimbra Script and not QA'd or tested. Path adjusted though for /opt/zimbra/postfix/sbin/*'''
=====To Delete Many Messages From Queue=====


From http://jwcub.wordpress.com/2006/01/20/bulk-delete-from-postfix-queue/
Moved to [[Managing_The_Postfix_Queues#To_Delete_Many_Messages_From_Queue]]


Perl script called “delete-from-mailq”:
=====Delete From Queue By Email Address=====


<pre>
Moved to [[Managing_The_Postfix_Queues#Delete_From_Queue_By_Email_Address]]
#!/usr/bin/perl


$REGEXP = shift || die “no email-adress given (regexp-style, e.g. bl.*\@yahoo.com)!”;
======From CLI======


@data = qx;
Moved to [[Managing_The_Postfix_Queues#From_CLI]]
for (@data) {
if (/^(\w+)(\*|\!)?\s/) {
$queue_id = $1;
}
if($queue_id) {
if (/$REGEXP/i) {
$Q{$queue_id} = 1;
$queue_id = “”;
}
}
}


open(POSTSUPER,”|/opt/zimbra/postfix/sbin/postsuper -d -”) || die “couldn’t open postsuper” ;
======Script To Delete From Queue By Email Address======


foreach (keys %Q) {
Moved to [[Managing_The_Postfix_Queues#Script_To_Delete_From_Queue_By_Email_Address]]
print POSTSUPER “$_\n”;
};
close(POSTSUPER);
</pre>


Save the above script to a file say “delete-queue.pl” in your home directory, and make it excutable:
======Script To Delete From Queue By Various Variable Targets======


chmod 755 delete-queue
Moved to [[Managing_The_Postfix_Queues#Script_To_Delete_From_Queue_By_Various_Variable_Targets]]


Usage:
{{Article Footer|Zimbra Collaboration 8.0, 7.0|04/16/2014}}


*Delete all queued messages from or to the domain “iamspammer.com”
----
./delete-queue iamspammer.com
*Delete all queued messages to specific address “bogususer@mydomain.com”
./delete-queue bogususer@mydomain.com
*Delete all queued messages that begin with the word “bush” in the e-mail address:
./delete-queue bush*\@whateverdomain.com
*Delete all queued messages that contain the word “biz” in the e-mail address:
./delete-queue biz


[[Category: Community Sandbox]]
[[Category: Community Sandbox]]
[[Category: Author:Ajcody]]
[[Category: Zeta Alliance]]

Latest revision as of 19:28, 20 June 2016

Ajcody MTA Postfix Topics

   KB 2705        Last updated on 2016-06-20  




0.00
(0 votes)
24px ‎  - This is Zeta Alliance Certified Documentation. The content has been tested by the Community.


Postfix - MTA

Actual MTA & Postfix Topics Homepage

Please see Ajcody-MTA-Postfix-Topics

Missing main.cf Error

Moved to Missing_main.cf_Error_-_MTA

MTA Mail Flow - Birds-eye Overview

Moved to Postfix-Amavisd_Mail_Flow_-_Birds-eye_Overview_-_MTA

Understanding /var/log/zimbra.log And Postfix Log Events

Moved to: Understanding_zimbra.log_And_Postfix_Log_Events_-_MTA

Postfix Queue ID vs. message-id

Moved to: Postfix_Queue_ID_vs._message-id_-_MTA

Authentication Log Events

Moved to Understanding_And_Troubleshooting_Authentication_Log_Events

IMAP And Authenticated SMTP [SSL] Example

Moved to Understanding_And_Troubleshooting_Authentication_Log_Events#IMAP_And_Authenticated_SMTP_.5BSSL.5D_Example

IMAP Test Via Telnet And Logging Events Of It - Proxy Included

Moved to Understanding_And_Troubleshooting_Authentication_Log_Events#IMAP_Test_Via_Telnet_And_Logging_Events_Of_It_-_Proxy_Included

IMAP Login Via Openssl - LOGIN TLS - Proxy Included

Moved to Understanding_And_Troubleshooting_Authentication_Log_Events#IMAP_Login_Via_Openssl_-_LOGIN_TLS_-_Proxy_Included

Network Tracing Between A Remote Host And A ZCS MTA

Moved to Network_Tracing_Between_A_Remote_Host_And_A_ZCS_MTA

Finding Messages - zmmsgtrace

See the following for details [for 7.1.1+]:

Documentation at CLI_zmmsgtrace . Note, if you get command not found as the zimbra user, try /opt/zimbra/libexec/zmmsgtrace instead.

How To Increase SMTP Debug Logging - MTA

Moved to How_To_Increase_SMTP_Debug_Logging_-_MTA

Simple Troubleshooting For SMTP Via Telnet, Openssl

Moved to Simple_Troubleshooting_For_SMTP_Via_Telnet_And_Openssl

First - Understanding Your Authentication Requirements In ZCS

Moved to Simple_Troubleshooting_For_SMTP_Via_Telnet_And_Openssl#First_-_Understanding_Your_Authentication_Requirements_In_ZCS

Second - Encoding Username And Passwords For AUTH Sequence

Moved to Simple_Troubleshooting_For_SMTP_Via_Telnet_And_Openssl#Second_-_Encoding_Username_And_Passwords_For_AUTH_Sequence

For ESMTP Auth is LOGIN - Example

Moved to Simple_Troubleshooting_For_SMTP_Via_Telnet_And_Openssl#For_ESMTP_Auth_is_LOGIN_-_Example

For ESMTP Auth is Plain - Example

Moved to Simple_Troubleshooting_For_SMTP_Via_Telnet_And_Openssl#For_ESMTP_Auth_is_Plain_-_Example

For TLS/SSL - Example

Moved to Simple_Troubleshooting_For_SMTP_Via_Telnet_And_Openssl#For_TLS.2FSSL_-_Example

Testing Against Port 465

Moved to Simple_Troubleshooting_For_SMTP_Via_Telnet_And_Openssl#Testing_Against_Port_465

To Confirm An Auth User Can't Send With Another FROM Address

Moved to Simple_Troubleshooting_For_SMTP_Via_Telnet_And_Openssl#To_Confirm_An_Auth_User_Can.27t_Send_With_Another_FROM_Address

Adding A New MTA Server

Basic instructions can be found here:

Additional instructions needed beyond the above will follow as I hear about them.

Load Balancing For SMTP - Out Bound Mail

Currently, 5.x code, you have the following options:

    • An external load balancing device that will then split the traffic behind it
    • Setup a round-robin A record situation in your DNS for the external mta's you'll be using.

In, GNR/6.x, you are able to add multiple targets to the variables and we'll have some degree of "balancing" between them.

User Alias Mapping And Mail Transport with Postfix & LDAP

See User_Alias_Mapping_and_Mail_Transport_with_Postfix_&_LDAP

Multiple LDAP Servers?

Completed RFE:

  • "mta should be able to take a list of LDAP servers to take advantage of replicas."

From :

        server_host (default: localhost)
              The name of the host running the LDAP server,  e.g.

                  server_host = ldap.example.com

              Depending  on the LDAP client library you're using,
              it should be possible to specify  multiple  servers
              here,  with the library trying them in order should
              the first one fail. It should also be  possible  to
              give  each  server  in  the  list  a different port
              (overriding server_port below), by naming them like

                  server_host = ldap.example.com:1444

              With OpenLDAP, a (list of) LDAP URLs can be used to
              specify both the hostname(s) and the port(s):

                  server_host = ldap://ldap.example.com:1444
                              ldap://ldap2.example.com:1444

              All LDAP URLs accepted by the OpenLDAP library  are
              supported,  including  connections over UNIX domain
              sockets, and LDAP SSL (the last one  provided  that
              OpenLDAP was compiled with support for SSL):

                  server_host = ldapi://%2Fsome%2Fpath
                              ldaps://ldap.example.com:636

 **my note**
 This thread - http://archives.neohapsis.com/archives/postfix/2004-09/1763.html
 give me the impression they made a mistake in modifying the help file on this
 and they dropped the use/need of the command:

  server_host = ldap://ldap.example.com:1444, ldap://ldap2.example.com:1444

Just a small note on where var shows up:

[root@mail3 conf]# pwd
/opt/zimbra/conf
[root@mail3 conf]# grep server_host *
amavisd.conf.in:$myhostname = '@@zimbra_server_hostname@@';  # must be a fully-qualified domain name!
ldap-scm.cf:server_host = ldap://mail3.zimbra.DOMAIN.com:389
ldap-transport.cf:server_host = ldap://mail3.zimbra.DOMAIN.com:389
ldap-vad.cf:server_host = ldap://mail3.zimbra.DOMAIN.com:389
ldap-vam.cf:server_host = ldap://mail3.zimbra.DOMAIN.com:389
ldap-vmd.cf:server_host = ldap://mail3.zimbra.DOMAIN.com:389
ldap-vmm.cf:server_host = ldap://mail3.zimbra.DOMAIN.com:389
localconfig.xml:  <key name="zimbra_server_hostname">
zmmta.cf:	LOCAL zimbra_server_hostname
zmmta.cf:	POSTCONF myhostname		LOCAL zimbra_server_hostname

References:

Traditional Aliases Use - /etc/aliases Type Lookups

Moved to Traditional_Aliases_Use_-_/etc/aliases_Type_Lookups

Allowing Accounts To Change The From Address

Please see:

Related BUG/RFE's

Creating A Domain Alias

Please see ManagingDomains#Creating_a_Domain_Alias

Relay Domain Forwarding

Please see ManagingDomains#Relaying.2FDomain_Forwarding

Domain Catchall

Please see ManagingDomains#Domain_Catchall

Rewriting From Address For Outbound Email

Please see ManagingDomains#Domain_Masquerading

Rewrite Recipient Address For Incoming Email

There is a way to rewrite the incoming mail, but it's not a standard Zimbra feature. You can implement it as a configuration change in Postfix. Here's what you do:

  1. Create a file in /opt/zimbra/conf named 'postfix_recipientmap'.
    • The format is a single line that reads something like: @alias.domain.com @domain.com
  2. Run 'postmap postfix_recipientmap' in the conf directory.
  3. Run "postconf -e recipient_canonical_maps=hash:/opt/zimbra/conf/postfix_recipientmap".
  4. Run 'postfix reload'.

This will cause postfix to map any incoming mail with a recipient of '@alias.domain.com' to '@domain.com'. You will need to re-apply this postconf change after upgrades, though the postfix_recipientmap file should survive.

Automatic BCC

Option 1 - Via Postfix Customization

From the postfix website:

  • always_bcc = address
    • Deliver a copy of all mail to the specified address. In Postfix versions before 2.1, this feature is implemented by smtpd(8), qmqpd(8), or pickup(8).
  • sender_bcc_maps = type:table
    • Search the specified "type:table" lookup table with the envelope sender address for an automatic BCC address. This feature is available in Postfix 2.1 and later.
  • recipient_bcc_maps = type:table
    • Search the specified "type:table" lookup table with the envelope recipient address for an automatic BCC address. This feature is available in Postfix 2.1 and later.
  • Note: automatic BCC recipients are produced only for new mail. To avoid mailer loops, automatic BCC recipients are not generated for mail that Postfix forwards internally, nor for mail that Postfix generates itself.

Please see the following:

Option 2 - Via ZCS Legal Intercept

Generally used for Managing Legal Requests for Information

Description:

The ZCS legal intercept feature is used to obtain copies of email messages that are sent, received, or saved as drafts from targeted accounts and send these message to a designated “shadow” email address. Legal Intercept can be configured to send the complete content of the message or to send only the header information. When a targeted account sends, receives, or saves a draft message, an intercept message is automatically created to forward copies of the messages as attachments to the specified email address.

Please see:

Option 3 - Zimbra's Archiving And Discovery

See Ajcody-Notes-Archive-Discovery concerning A&D setup and options.

Limiting Or Increasing Number Of Recipents / Messages

Mailing Lists - Distribution Lists

Please see Ajcody-MailingLists-And-Mailman#Problems_Resolving_Virtual_Aliases_For_Members_Of_Large_Distribution_Lists

Policy Daemon

If you want to restrict messages per hour, you can look into Policy Daemon:

Beta release in ZCS 7 , see:

Postfix

Also, there are some default postfix parameters set to control sending a message to x amount recipients. The parameters you will need to look at are smtpd_recipient_limit & smtpd_recipient_overshoot_limit, these have a default value of 1000.

Postfix defines these parameters as:

  • smtpd_recipient_limit: The maximum number of recipients that the Postfix SMTP server accepts per message delivery request.
  • smtpd_recipient_overshoot_limit: The number of recipients that a remote SMTP client can send in excess of the limit specified with $smtpd_recipient_limit, before the Postfix SMTP server increments the per-session error count for each excess recipient.

From the command line you can change the default values.

su - zimbra
postconf -e smtpd_recipient_limit=<new value>
postconf -e smtpd_recipient_overshoot_limit=<new value>
postfix reload

Bugs RFE's For Customers To Get Behind

I'm wondering if policyd gives one the control everyone is looking for? I've not used it myself.

Policyd References:

There's other additions [add-on's] one can get for policyd.

We have this RFE in regards to policyd support:

Other related rfe/bugs, specially to push variables into admin web console:

Controlling SMTPD Client Connections

Mmorse did a good write up on these variables in the forum:

Postfix Resources At Their Site (All Clients/Connections):

Postfix Resources At Their Site (Exceptions To Clients/Connections Or Single Source):

Restrictions

Besides using external mailing list software, Mailman or Sympa, here's some other topical items in regards to restrictions.

Some user contributed articles:

Some Postfix references:

Some RFE's related to mta based restrictions [targets are based upon today - July 21, 2010]:

Spam Control And Related Issues

High Over View Steps Of What To Do

  • Step 1: Confirm your not an open relay and double check your postfix $mynetworks variable.
  • Step 2: Stop or put on-hold mail queue.
    • Put all messages into HOLD queue:
      • Get a report of your current mailq [can be useful if you clean out the queue later but need to identify what external mail hosts are now denying you and who you'll need to contact about getting removed from their denial list]
        • Example: /opt/zimbra/postfix/sbin/mailq > /tmp/zimbra_mailq_report.txt
      • /opt/zimbra/postfix/sbin/postsuper -h ALL
    • Or put all messages match compromised account into HOLD queue:
      • /opt/zimbra/postfix/sbin/mailq | grep user_compromised@domain | awk '{ print $1 }' | tr -d '!*' | /opt/zimbra/postfix/sbin/postsuper -h -
        • Note, this is an example - you might with the grep grab more than the compromised account with the match.
    • See whole section - Managing The Postfix Queue
  • Step 3: Check your mail log [On ZCS servers running MTA services] - /var/log/zimbra.log
  • Step 4: Identify compromised account authenticating SMTP AUTH connection or block ip address where emails are coming from at firewall.
    • Who's My Spammer?
    • Continue to monitor compromised account and block ip addresses:
      • tail -f /var/log/zimbra.log | grep username | grep sasl
        • Jun 8 18:14:10 mail postfix/smtpd[15794]: 004358EEB16: client=unknown[XXXX.236.197.216], sasl_method=LOGIN, sasl_username=username@domain
  • Step 5: Disable the exploited email account, expire auth session, etc.
  • Step 6: Move the mail queue or delete the spam email
  • Step 7: Release Mail queue

Who's My Spammer?

Getting Some Initial Summary Data
zmdialyreport

First, some notable bug/RFE's in regards to the zmdailyreport:

You can first get some summary data by doing the following:

[zimbra@zcs806 ~]$ /opt/zimbra/libexec/zmdailyreport

Grand Totals
------------
messages

      7   received
     11   delivered
      0   forwarded
      0   deferred
      0   bounced
      3   rejected (21%)
      0   reject warnings
      0   held
      0   discarded (0%)

   2780   bytes received
  10914   bytes delivered
      2   senders
      1   sending hosts/domains
      1   recipients
      1   recipient hosts/domains


Per-Hour Traffic Summary
    time          received  delivered   deferred    bounced     rejected
    --------------------------------------------------------------------
    0000-0100           0          0          0          0          0
    0100-0200           1          3          0          0          0
    0200-0300           0          0          0          0          0
    0300-0400           0          0          0          0          0
    0400-0500           0          0          0          0          0
    0500-0600           0          0          0          0          0
    0600-0700           0          0          0          0          0
    0700-0800           1          0          0          0          2
    0800-0900           1          0          0          0          0
    0900-1000           0          0          0          0          1
    1000-1100           0          0          0          0          0
    1100-1200           0          0          0          0          0
    1200-1300           4          8          0          0          0
    1300-1400           0          0          0          0          0
    1400-1500           0          0          0          0          0
    1500-1600           0          0          0          0          0
    1600-1700           0          0          0          0          0
    1700-1800           0          0          0          0          0
    1800-1900           0          0          0          0          0
    1900-2000           0          0          0          0          0
    2000-2100           0          0          0          0          0
    2100-2200           0          0          0          0          0
    2200-2300           0          0          0          0          0
    2300-2400           0          0          0          0          0

Host/Domain Summary: Message Delivery (top 50)
 sent cnt  bytes   defers   avg dly max dly host/domain
 -------- -------  -------  ------- ------- -----------
     11    10914        0     7.4 s   24.0 s  zcs806.DOMAIN.com

Host/Domain Summary: Messages Received (top 50)
 msg cnt   bytes   host/domain
 -------- -------  -----------
      5     2780   zcs806.DOMAIN.com

top 50 Senders by message count
-------------------------------
      4   zimbra@zcs806.DOMAIN.com
      1   admin@zcs806.DOMAIN.com

top 50 Recipients by message count
----------------------------------
     11   admin@zcs806.DOMAIN.com

top 50 Senders by message size
------------------------------
   1974   zimbra@zcs806.DOMAIN.com
    806   admin@zcs806.DOMAIN.com

top 50 Recipients by message size
---------------------------------
  10914   admin@zcs806.DOMAIN.com

message deferral detail: none

message bounce detail (by relay): none

message reject detail
---------------------
  MAIL
    5.3.4 Message size exceeds fixed limit (total: 3)
           3   domain-ext.com

message reject warning detail: none

message hold detail: none

message discard detail: none

smtp delivery failures: none

Warnings
--------
  sendmail (total: 3)
         1   or the command is run from a set-uid root process
         1   the Postfix sendmail command has set-uid root file permissions
         1   the Postfix sendmail command must be installed without set-uid ...
  smtpd (total: 1)
         1   7A735345A: queue file size limit exceeded

Fatal Errors: none

Panics: none

Master daemon messages: none
client_usage_report.py

This will give some stats on your mail activity. Note, there are some issue with this script double reporting mail counts etc, but it's useful to identify the top 50 for activity.

[zimbra@zcs806 ~]$ /opt/zimbra/libexec/client_usage_report.py
Reading /opt/zimbra/log/access_log.2014-04-17 ..
Reading /opt/zimbra/log/access_log.2014-04-18 ..
Reading /opt/zimbra/log/access_log.2014-04-19 ..
Reading /opt/zimbra/log/access_log.2014-04-20 ..
Reading /opt/zimbra/log/access_log.2014-04-21 ..
Reading /opt/zimbra/log/access_log.2014-04-22 ..
Reading /opt/zimbra/log/access_log.2014-04-23 ..
Writing /opt/zimbra/zmstat/client_usage_report_2014-04-24.csv ..

Then review the file it will create , it will give : "user_agent","client_IP","req_count"

[zimbra@zcs806 ~]$ cat /opt/zimbra/zmstat/client_usage_report_2014-04-24.csv
"user_agent","client_IP","req_count"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:28.0) Gecko/20100101 Firefox/28.0","192.168.1.166","14"
"Mozilla/5.0 (Windows NT 6.2; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0","192.168.1.166","93"
"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0","192.168.1.174","6"
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:28.0) Gecko/20100101 Firefox/28.0","192.168.1.166","71"
qshape

You can also look at the results of [qshape] - the default is the active queue. For more on qshape, see [Postfix Qshape Readme] .

qshape deferred
               T  5 10 20 40 80 160 320 640 1280 1280+
        TOTAL 12  0  0  0  0  0   0   0   0    0     12
    gmail.com  9  0  0  0  0  0   0   0   0    0     9
    yahoo.com  3  0  0  0  0  0   0   0   0    0     3
3rd Party Log Reports - postfix-logwatch and amavis-logwatch

Created RFE for us to include these in ZCS:

You can download them from http://logreporters.sourceforge.net/ . It's a fairly simply install, download and then extract - cd into extracted directory and as root type :

make install-standalone

They will install to /usr/local/bin/amavis-logwatch & postfix-logwatch . The config files are in /usr/local/etc/amavis-logwatch.conf & postfix-logwatch.conf . Here's an example of the output.

/usr/local/bin/amavis-logwatch output example:

[root@zcs806 amavis-logwatch-1.51.02]# /usr/local/bin/amavis-logwatch /var/log/zimbra.log
****** Summary *************************************************************************************

       4   Total messages scanned ------------------  100.00%
   1.926K  Total bytes scanned                          1,972
========   ==================================================

       4   Passed ----------------------------------  100.00%
       4     Clean passed                             100.00%
========   ==================================================

       4   Ham -------------------------------------  100.00%
       4     Clean passed                             100.00%
========   ==================================================


==================================================================================
Spam Score Percentiles        0%       50%       90%       95%       98%      100%
----------------------------------------------------------------------------------
Score Ham (4)             -1.900    -1.900    -1.900    -1.900    -1.900    -1.900
==================================================================================

======================================================================================================
Spam Score Frequency      <= -10     <= -5      <= 0      <= 5     <= 10     <= 20     <= 30      > 30
------------------------------------------------------------------------------------------------------
Hits (4)                       0         0         4         0         0         0         0         0
Percent of Hits            0.00%     0.00%   100.00%     0.00%     0.00%     0.00%     0.00%     0.00%
======================================================================================================

/usr/local/bin/postfix-logwatch output example:

[root@zcs806 amavis-logwatch-1.51.02]# /usr/local/bin/postfix-logwatch /var/log/zimbra.log

****** Summary *************************************************************************************

       1   *Warning: Queue file size limit exceeded

   6.512K  Bytes accepted                               6,668
   1.928K  Bytes sent via SMTP                          1,974
   4.584K  Bytes sent via LMTP                          4,694
========   ==================================================

      10   Accepted                                    76.92%
       3   Rejected                                    23.08%
--------   --------------------------------------------------
      13   Total                                      100.00%
========   ==================================================

       3   5xx Reject message size                    100.00%
--------   --------------------------------------------------
       3   Total 5xx Rejects                          100.00%
========   ==================================================

      10   Connections
      10   Disconnections
       8   Removed from queue
       4   Sent via SMTP
       4   Sent via LMTP
       4   Filtered

****** Detail (10) *********************************************************************************

       3   5xx Reject message size -----------------------------------------------------------------
       3      192.168.1.166     remote.domain.com
       3         *unavailable
       3            *unavailable

       4   Sent via SMTP ---------------------------------------------------------------------------
       4      zcs806.DOMAIN.com

       4   Sent via LMTP ---------------------------------------------------------------------------
       4      zcs806.DOMAIN.com

       4   Filtered --------------------------------------------------------------------------------
       2      smtp-amavis:[127.0.0.1]:10024
       2         Sender address
       1            admin@zcs806.DOMAIN.com
       1               admin@zcs806.DOMAIN.com
       1                  192.168.1.166     remote.domain.com
       1            user@DOMAIN.com
       1               admin@zcs806.DOMAIN.com
       1                  192.168.1.184     remote2.domain.com
       2      smtp-amavis:[127.0.0.1]:10026
       2         Sender address
       1            admin@zcs806.DOMAIN.com
       1               admin@zcs806.DOMAIN.com
       1                  192.168.1.166     remote.domain.com
       1            user@DOMAIN.com
       1               admin@zcs806.DOMAIN.com
       1                  192.168.1.184     remote2.domain.com

=== Delivery Delays Percentiles ============================================================
                    0%       25%       50%       75%       90%       95%       98%      100%
--------------------------------------------------------------------------------------------
Before qmgr       0.04      0.09      0.11      0.11      0.23      0.35      0.43      0.48
In qmgr           0.00      0.00      0.01      0.01      0.04      0.07      0.08      0.09
Conn setup        0.00      0.01      0.29      1.30      2.05      2.23      2.33      2.40
Transmission      0.10      2.81      4.85      9.60     21.00     21.00     21.00     21.00
Total             0.20      2.91      5.20     11.00     23.30     23.65     23.86     24.00
============================================================================================

Note - First, look at the options each command has using the -h output. You might want to use --full when doing an investigation and also include a wildcard - /var/log/zimbra.lo* to take in all the log data. For example:

[zimbra@zcs806 ~]$ /usr/local/bin/postfix-logwatch --full /var/log/zimbra.lo*
****** Summary *************************************************************************************

       9   *Fatal:   General fatal
       1   *Warning: Queue file size limit exceeded
      21   Miscellaneous warnings

 710.888K  Bytes accepted                             727,949
 193.036K  Bytes sent via SMTP                        197,669
 520.114K  Bytes sent via LMTP                        532,597
========   ==================================================

    1041   Accepted                                    99.71%
       3   Rejected                                     0.29%
--------   --------------------------------------------------
    1044   Total                                      100.00%
========   ==================================================

       3   5xx Reject message size                    100.00%
--------   --------------------------------------------------
       3   Total 5xx Rejects                          100.00%
========   ==================================================

      65   Connections
      65   Disconnections
    1041   Removed from queue
     523   Sent via SMTP
     517   Sent via LMTP
       2   Bounced (local)
       1   Bounced (remote)
       9   Filtered
       2   Notifications sent

       4   Timeouts (inbound)
       1   PIX workaround enabled

****** Detail (10) *********************************************************************************

       9   *Fatal:   General fatal -----------------------------------------------------------------
       3      Queue report unavailable - mail system is down
       3      Usage: sendmail [options]
       2      The Postfix mail system is not running
       1      Usage: send-mail [options]

      21   Miscellaneous warnings ------------------------------------------------------------------
       7      or the command is run from a set-uid root process
       7      the Postfix sendmail command has set-uid root file permissions
       7      the Postfix sendmail command must be installed without set-uid root file permissions

       3   5xx Reject message size -----------------------------------------------------------------
       3      10.X.X.166     fence.DOMAIN.com
       3         *unavailable
       3            *unavailable

     523   Sent via SMTP ---------------------------------------------------------------------------
     507      86.lab
      15      zcs806.DOMAIN.com
       1      domaina.com

     517   Sent via LMTP ---------------------------------------------------------------------------
     507      86.lab
      10      zcs806.DOMAIN.com

       2   Bounced (local) -------------------------------------------------------------------------
       2      5.0.0: Permanent failure: Other/Undefined status: Other undefined status
       2         zcs806.DOMAIN.com
       2            Zcs806.DOMAIN.com
       1               subject:test
       1               zimbra

       1   Bounced (remote) ------------------------------------------------------------------------
       1      5.0.0: Permanent failure: Other/Undefined status: Other undefined status
       1         domain.com
       1            user
       1               64.X.X.28      sentry.DOMAIN.com
       1                  505 5.0.0 Unknown recipient: RCPT TO

       9   Filtered --------------------------------------------------------------------------------
       7      smtp-amavis:[127.0.0.1]:10026
       7         Sender address
       3            admin@zcs806.DOMAIN.com
       3               admin@zcs806.DOMAIN.com
       2                  10.X.X.36     zcs806.DOMAIN.com
       1                  10.X.X.166     gatewayXX.DOMAIN.com
       2            zimbra@zcs806.DOMAIN.com
       2               admin@zcs806.DOMAIN.com
       2                  10.X.X.36     zcs806.DOMAIN.com
       1            ajcody@DOMAIN.com
       1               admin@zcs806.DOMAIN.com
       1                  10.X.X.184     edgeXX.DOMAIN.com
       1            san5@zcs806.DOMAIN.com
       1               b@zcs806.DOMAIN.com
       1                  10.X.X.36     zcs806.DOMAIN.com
       2      smtp-amavis:[127.0.0.1]:10024
       2         Sender address
       1            admin@zcs806.DOMAIN.com
       1               admin@zcs806.DOMAIN.com
       1                  10.X.X.166     gatewayXX.DOMAIN.com
       1            ajcody@DOMAIN.com
       1               admin@zcs806.DOMAIN.com
       1                  10.X.X.184     edgeXX.DOMAIN.com

       2   Notifications sent ----------------------------------------------------------------------
       2      Non-delivery
       2         sender

       4   Timeouts (inbound) ----------------------------------------------------------------------
       4      After END-OF-MESSAGE

       1   PIX workaround enabled ------------------------------------------------------------------
       1      disable_esmtp delay_dotcrlf
       1         64.X.X.28      sentry.DOMAIN.com

=== Delivery Delays Percentiles ============================================================
                    0%       25%       50%       75%       90%       95%       98%      100%
--------------------------------------------------------------------------------------------
Before qmgr       0.01      0.03      0.06      0.14      0.27      0.34      0.48      2.60
In qmgr           0.00      0.00      0.08    117.50    193.00    216.00    231.48    246.00
Conn setup        0.00      0.00      0.00      0.00      0.02      0.04      0.22     20.00
Transmission      0.05      0.09      3.60      9.80     10.00     10.00     11.00    160.00
Total             0.07      0.13      3.80    129.00    203.00    226.00    241.64    259.00
============================================================================================
zmaccts

One way to note accounts that are actively logging in vs. those that aren't, can help shrink the number of accounts you might want to investigate or monitor. [example below, I cut out a lot of the accounts]

           account                          status             created       last logon
------------------------------------   -----------     ---------------  ---------------
zcstest001@zcs806.DOMAIN.com                active      01/20/14 18:47   03/02/14 21:11
zcstest002@zcs806.DOMAIN.com                active      01/30/14 01:48   02/19/14 00:07
admin-20140415@zcs806.DOMAIN.com            active      04/15/14 14:42            never
archtest-prod-20140402@zcs806.DOMAIN        active      04/02/14 07:42            never

           account                          status             created       last logon
------------------------------------   -----------     ---------------  ---------------
bruce@test1.lab                             active      02/22/14 09:32            never
test.cal@test1.lab                          active      04/06/14 05:35   04/06/14 05:35
test200@test1.lab                           active      04/12/14 00:50            never

                                 domain summary

    domain                  active    closed    locked    maintenance     total
-----------------------   --------  --------  --------  -------------  --------
test1.lab                        3         0         0              0         3
test2.com                        2         0         0              0         2
angad.com                        2         0         0              0         2
test.test                        3         0         0              0         3
test.DOMAIN.com                  6         0         0              0         6
zcs806.DOMAIN.com               58         0         0              0        58
zcs806.DOMAIN.com                2         0         0              0         2
By Authentication Attempts

A fast way to see who is doing a lot of authentications, which normally happens when a spammer has compromised an account with a weak password, is to do:

# cat /var/log/zimbra.log | sed -n 's/.*sasl_username=//p' | sort | uniq -c | sort -n
      1 Auser@domain.com
      3 Buser@domain.com
      4 Cuser@domain.com
      5 Duser@domain.com
     36 SPAMMER@domain.com

Note - This might take a long time, if so - try pruning it down

Example:

# cat /var/log/zimbra.log | grep sasl_username > /tmp/zimbra_sasl_username.txt
# cat /tmp/zimbra_sasl_username.txt | sed -n 's/.*sasl_username=//p' | sort | uniq -c | sort -n
      1 Auser@domain.com
      3 Buser@domain.com
      4 Cuser@domain.com
      5 Duser@domain.com
     36 SPAMMER@domain.com

The full log event will look like this:

zimbra1 postfix/smtpd[29431]: B28914D5978: client=xxxxx.server.com[w.x.y.z], sasl_method=LOGIN, sasl_username=user
zimbra1 postfix/cleanup[5522]: B28914D5978: message-id=<20090420154255.B28914D5978@zimbraserver.com>
zimbra1 postfix/qmgr[20690]: B28914D5978: from=<spam@spam.com>, size=6026, nrcpt=10 (queue active)
zimbra1 postfix/cleanup[3983]: 2BA56465D28: message-id=<20090420154255.B28914D5978@zimbraserver.com>

Against your older logs, you could:

# zcat /var/log/zimbra.log* | sed -n 's/.*sasl_username=//p' | sort | uniq -c | sort -n

And you can look at the specific information for the user in question with:

# grep -C2 "sasl_username=SPAMMER@domain.com" /var/log/zimbra.log

Or if searching against the older logs:

# zgrep -C2 "sasl_username=SPAMMER@domain.com" /var/log/zimbra.log*

If you want to check on a specific message ID, do:

grep 9DF7520804A /var/log/zimbra.log*

For older message logs:

zgrep 9DF7520804A /var/log/zimbra.log*

To read/view the message in the queue:

/opt/zimbra/postfix/sbin/postcat -q 9DF7520804A 

One would then normally lock/change password on the one account showing the most activity. Grep'ing the /var/log/zimbra.log with the username in question will also show the ip address being used, this can be blocked with your firewall.

To be alerted of a compromised account and have it lock automatically see below. Slightly modified from this reference : http://www.zimbra.com/forums/administrators/62613-identify-compromised-accounts.html#post278732 :


#!/bin/bash
# checks log file and gets a count of authentications sent per minute, per user
# and if the count exceeds the maxmails value the user's account is locked.

logfile="/var/log/zimbra.log"
maxmails="10"
mydomain="example.com"
support="<postmaster-userid>@$mydomain"
accounts="/tmp/active_accounts"

su zimbra -c "/opt/zimbra/bin/zmaccts" | grep "@" | grep active | awk '{print $1}' > $accounts

zgrep -i "auth ok" $logfile | sed 's/  / /g' | awk -F"[ :]" '{print $3":"$4,$11;}' | uniq -c | sort -n | \
while read line
do
    count=`echo ${line} | cut -d' ' -f 1`
    userid=`echo ${line} | cut -d' ' -f 3`
    timestamp=`echo ${line} | cut -d' ' -f 2`
    active=`grep "$userid@$mydomain" $accounts`

    if [ "$count" -gt "$maxmails" ] && [ "$active" == "$userid@$mydomain" ]; then
        echo "Maximum email rate exceeded, $userid@$mydomain will be locked"
        su zimbra -c "/opt/zimbra/bin/zmprov ma $userid@$mydomain zimbraAccountStatus locked"
        subject="$userid account locked due to excessive connections"
        # Email text/message
        message="/tmp/emailmessage.txt"
        echo "$userid account has been locked as there were $count connections made at"> $message
        echo "$timestamp.  Please have the user change their password, and check for phishing" >>$message
        echo "emails if possible." >>$message
        # send an email using /bin/mail
        /usr/bin/mail -s "$subject" "$support" < $message
        rm -f $message

        #update list of active accounts
        su zimbra -c "/opt/zimbra/bin/zmaccts" | grep "@" | grep active | awk '{print $1}' > $accounts
    fi
done

rm -f $accounts

Then run it as a cron job. The frequency will depend on the number of accounts you're managing.

* * * * * /opt/zimbra/find_spammer.sh
By Connecting IP - Useful For Blocking IP At Firewall

See also the following:

To find the originating IP address of where the emails are coming from:

grep 'connect from' /var/log/zimbra.log | sed 's/.*connect from.*\[\(.[^]]*\)\]/\1/g' | sort | uniq -c | sort -nr | head

To check your older logs [example output below]:

zgrep 'connect from' /var/log/zimbra.log* | sed 's/.*connect from.*\[\(.[^]]*\)\]/\1/g' | sort | uniq -c | sort -nr | head
     36 10.137.xx.34
     34 127.0.0.1
Open Relay Check

You should also confirm you aren't an open relay.

$ host -t mx DOMAIN.com
DOMAIN.com mail is handled by 10 mail.DOMAIN.com.

$ telnet mail.DOMAIN.com 25
Trying 184.###.##.## ...
Connected to mail.DOMAIN.com.
Escape character is '^]'.
220 mail.DOMAIN.com ESMTP Postfix
helo support.test
250 mail.DOMAIN.com
mail from:<SPAMMER@domain.com>
250 2.1.0 Ok
rcpt to:<TEST@DOMAIN.COM>
554 5.7.1 <TEST@DOMAIN.COM>: Relay access denied

rcpt to:<SPAMMER@domain.com>
554 5.7.1 Service unavailable; Client host [71.XXX.XX.XX] blocked 
 using zen.spamhaus.org; http://www.spamhaus.org/query/bl?ip=71.202.XX.XX
quit
221 2.0.0 Bye
Connection closed by foreign host.
Telnet Test To Confirm/Show Authentication Required For SMTP/Port 25

This is an example:

esx2:~ ajcody$ telnet zcs723.EXAMPLE.com 25

Trying 10.137.27.32...
Connected to zcs723.EXAMPLE.com.
Escape character is '^]'.
220 zcs723.EXAMPLE.com ESMTP Postfix
helo zcs723.EXAMPLE.com   << I typed
250 zcs723.EXAMPLE.com
mail from:ajcody@zcs723.EXAMPLE.com   << I typed
250 2.1.0 Ok
rcpt to:ajcody2@zcs723.EXAMPLE.com   << I typed
553 5.7.1 <ajcody@zcs723.EXAMPLE.com>: Sender address rejected: not logged in

But note - if you do this from the ZCS server or a server that is within the ip range or has it's specific ip listed in the mynetworks, you will not get this authentication requirement.

[root@zcs723 ~]# telnet localhost 25
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 zcs723.EXAMPLE.com ESMTP Postfix
helo myworkstation
250 zcs723.EXAMPLE.com
ehlo myworkstation
250-zcs723.EXAMPLE.com
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-AUTH LOGIN PLAIN
250-AUTH=LOGIN PLAIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
mail from:<ajcody@zcs723.EXAMPLE.com>
250 2.1.0 Ok
rcpt to: <ajcody2@zcs723.EXAMPLE.com>
250 2.1.5 Ok
data
354 End data with <CR><LF>.<CR><LF>
From: Adam <ajcody@zcs723.EXAMPLE.com>
To: Adam 2 <ajcody2@zcs723.EXAMPLE.com>
Subject: From Localhost - NOT Auth
test
.
250 2.0.0 Ok: queued as 8B19E1E78D1
quit
221 2.0.0 Bye
Connection closed by foreign host.

Resources

A list of resources you'll find useful:

Wiki articles that have been assigned to the anit-spam category:

Down to the end-user:

External Relay Test Pages
  • http://www.checkor.com/
    • Note - this test is in regards to the From spoofing spammers sometimes do for DL's.
    • Also, for the test - make an account/DL on your system for test1@[your domain] . Otherwise you'll just error about account not existing.
RSET
250 2.0.0 Ok
MAIL FROM: spam@mail59.DOMAIN.com
250 2.1.0 Ok
RCPT TO: test1@mail59.DOMAIN.com
Test Failed, 250 2.1.5 Ok 

Blocking MAIL FROM - smtpd_sender_restrictions - Default Is Empty

External References
Zimbra References And Bugs & RFE's
Protecting DL's From Spammers - Forging Mail From
First Recommendation - As Given By Dev's From Critical Meeting Notes

  • Enabled SASL/SMTP Authentication
  • Implement how-to as described in :
    • Permitted Senders: RestrictPostfixRecipients
      • Note: You'll see on the above page a reference to the spoof hole.
        • "This method can be spoofed by forging the MAIL FROM: header (so mail appears to originate from within the domain), so it isn't foolproof, but it works for basic needs."
      • Note: You'll also modify the instructions as above with addition details provided below.
  • Force authentication for local-domain senders:
    • modify the main.cf to have the following:
      • smtpd_sender_restrictions = check_sender_access hash:/path/to/file
    • Then for the /path/to/file that you used in the mail.cf for smtpd_sender_restrictions, you'll have a line like:
      • example.com permit_sasl_authenticated, reject
Second Recommendation - Unpredictable DL name or Non-routing Domain

Do not use predictable DL names. Instead of using everyone@company.com , use something like everyone-[random-string]@company.com .

Another option is to use a non-routing domain - company.local - and setup your DL's there. You'll want to configure your main domain to be able to query the GAL of this domain.

To see the existing setting:

zmprov gd [domainname] zimbraGalLdapSearchBase

To change the variable for the domain:

zmprov md [domainname] zimbraGalInternalSearchBase ROOT
Third Recommendation - Using smtpd_sender_restrictions

Work In Progress. I'm testing this now. Please don't attempt until this line is removed.

Update: See the following:

This should work if your "clients" are using ZWC, ZCO, or ActiveSync mobile devices. If you are using IMAP/POP + STMP thick clients, you'll most likely have to enable smtp authentication [sasl] and use the reject_authenticated_sender_login_mismatch variable instead.

postconf -e smtpd_sender_restrictions=reject_sender_login_mismatch
postfix reload

This option is described as:

reject_sender_login_mismatch
Reject the request when $smtpd_sender_login_maps specifies an owner for the MAIL FROM address, but the client is not (SASL) logged in as that MAIL FROM address owner; or when the client is (SASL) logged in, but the client login name doesn't own the MAIL FROM address according to $smtpd_sender_login_maps. Man page

Some Other SMTP Sending Restrictions

Blocking Incoming From Domain And By User

See the following:

check_client_access

The smtpd_client_restrictions parameter restricts what clients this system accepts SMTP connections from. The default behavior is to allow SMTP connections from any client. This is discussed under Spam Controls on the Postfix site.

Example:

check_client_access regexp:/etc/postfix/access_sender_client_server,

Example:

check_sender_access regexp:/etc/postfix/access_sender_toplevel
smtpd_reject_unlisted_sender

Details can be found on the mail.cf Postfix page.

Example:

smtpd_reject_unlisted_sender = yes

Possible Bug:

http://bugzilla.zimbra.com/show_bug.cgi?id=24889

What's Your SPF Records Say, When Getting "does not designate 74.x.x.x as permitted sender Errors"

This is most likely related to the SPF records for your domain and what the header content of the sending email states as it's Mail From. For example, this is from the header of an email that was "received":

Received: from mail.XYZ-FAKE.com (mailhost.XYZ-FAKE.com [74.X.X.244]) by mta01.ABC-FAKE.com with ESMTP id 
Cft0mO3fjlFGQjTA for <support@ABC-FAKE.com>; Tue, 21 Apr 2009 05:14:13 -0700 (PDT)
X-Barracuda-Envelope-From: testuser@XYZ-FAKE.com
Received-SPF: pass (mta01.ABC-FAKE.com: domain of testuser@XYZ-FAKE.com designates 74.X.X.244 as permitted sender) 
receiver=mta01.ABC-FAKE.com; client_ip=74.X.X.244; envelope-from=testuser@XYZ-FAKE.com;

To see what this check was done against, do the following below. I'll trim the output and adjust the information used to protect the innocent. Also, notice how a DNS "alias" might cause an issue here? :


$ host 74.X.X.244
244.X.X.74.in-addr.arpa domain name pointer mailhost.XYZ-FAKE.com.

$ host mailhost.XYZ-FAKE.com
mailhost.XYZ-FAKE.com has address 74.X.X.244

$ host mail.XYZ-FAKE.com
mail.XYZ-FAKE.com is an alias for mailhost.XYZ-FAKE.com.
mailhost.XYZ-FAKE.com has address 74.X.X.244

$ dig XYZ-FAKE.com MX

;; QUESTION SECTION:
;XYZ-FAKE.com.			IN	MX

;; ANSWER SECTION:
XYZ-FAKE.com.		3600	IN	MX	22 serverA.DNS-FAKE.com.
XYZ-FAKE.com.		3600	IN	MX	11 serverB.DNS-FAKE.com.

$ dig XYZ-FAKE.com TXT

;; QUESTION SECTION:
;XYZ-FAKE.com.			IN	TXT

;; ANSWER SECTION:
XYZ-FAKE.com.		3600	IN	TXT	"v=spf1 a:mail.XYZ-FAKE.com ~all"

See the following for more information:

Using Different SMTP Server For Webclient (ZWC), Mobiles, And ZCO

Note Of Caution About Using External MTAs

Using non-zimbra MTA's can cause some options in zimbra to not function anymore - since it no longer has zimbra's mta services available.

Zimbra Mail Forwarding Possibly Will Not Work - Turn Off User Option To Set MailForwarding

Mail forwarding might no longer work depending on the configuration you setup regarding the use of your external mta's. When this happens you'll most likely want to disable the option for users to set a mail forwarding address in their preferences. This can be done via their COS or USER configuration.

  • In the admin console, goto the COS configuration the user/s are using and the "Features" tab. Uncheck the option "Allow the user to specify a forwarding address" under Mail Features. It is in the same location under a USERs configuration panel in the admin console.

In the CLI, you will see these set as the defaults for the default COS. The admin gui option above only adjusts the zimbraFeatureMailForwardingEnabled variable :

$ zmprov gc default | grep zimbraFeatureMailForwarding
    zimbraFeatureMailForwardingEnabled: TRUE
    zimbraFeatureMailForwardingInFiltersEnabled: TRUE
$ zmprov ga ajcody@`zmhostname` | grep zimbraFeatureMailForwarding
    zimbraFeatureMailForwardingEnabled: TRUE
    zimbraFeatureMailForwardingInFiltersEnabled: TRUE

More details about them are in the /opt/zimbra/conf/attrs/zimbra-attrs.xml file.

<attr id="342" name="zimbraFeatureMailForwardingEnabled" type="boolean" cardinality="single" 
 optionalIn="account,cos" flags="accountInfo,accountInherited,domainAdminModifiable">
  <defaultCOSValue>TRUE</defaultCOSValue>
  <desc>enable end-user mail forwarding features</desc>
</attr>

<attr id="704" name="zimbraFeatureMailForwardingInFiltersEnabled" type="boolean" 
 cardinality="single" optionalIn="account,cos" flags="accountInfo,accountInherited,
 domainAdminModifiable" since="5.0.10">
  <defaultCOSValue>TRUE</defaultCOSValue>
  <desc>enable end-user mail forwarding defined in mail filters features</desc>
</attr>
Configure External MTA To Use LDAP Virtual Alias Maps

Here's the basic info in regards to how Zimbra's mta [postfix/etc] uses Zimbra's LDAP to get the forwarding information:

$ grep Forward conf/ldap-*
conf/ldap-vam.cf:result_attribute =  
zimbraMailDeliveryAddress,zimbraMailForwardingAddress,
zimbraPrefMailForwardingAddress,zimbraMailCatchAllForwardingAddress
$ postconf |grep vam
virtual_alias_maps = proxy:ldap:/opt/zimbra/conf/ldap-vam.cf

See http://www.postfix.org/postconf.5.html#virtual_alias_maps for more information.

Confirming And Setting zimbraMtaRelayHost And zimbraMtaDnsLookupsEnabled

First we'll set zimbraMtaRelayHost and zimbraMtaDnsLookupsEnabled variables. These options are also shown in the admin console and can be configured there. These variable alone will not redirect ALL traffic to an external MTA first though. There's alao a variable called zimbraSmtpHostname that is in the global (zmprov gacf) and server (zmprov gs `hostname`) configs - addressed in the section below. It's defaulted value is 'localhost' - at least on a single ZCS configuration.

In situations where you need/want all mail to be processed by, for example, an external non-Zimbra spam filter box you could set this variable to the spam filter servers hostname.

Normally, when zimbraMtaRelayHost is set to a non-zimbra external MTA would disable DNS lookups. If you disable DNS Lookups (under the MTA tab of the admin console, or with zmprov), Zimbra will end up using (according to the postconf man page) the "gethostbyname() system library routine which normally also looks in /etc/hosts" (based on the entries on the "hosts" line in /etc/nsswitch.conf). If you do this but don't also specify an SMTP relay host (typically your ISP's SMTP server), which will take care of checking DNS, you will reverse your ability to send mail: suddenly you can send mail to other users on the Zimbra server, but you can't send to the internet (though you can still receive mail from the internet either way).

Query Global - `zmhostname` would use the value returned, you can manual type out the servername as well :

zmprov gacf zimbraMtaRelayHost
zmprov gacf zimbraMtaDnsLookupsEnabled

Query Per Server - `zmhostname` would use the value returned, you can manual type out the servername as well :

zmprov gs `zmhostname` zimbraMtaRelayHost
zmprov gs `zmhostname` zimbraMtaDnsLookupsEnabled

Note - if you get errors about doing the query on your non-mailstores like, "ERROR: zclient.IO_ERROR (invoke Connection refused, server: localhost) (cause: java.net.ConnectException Connection refused)" then you might need to adjust this variable. First query it, it's most likely set to localhost

zmlocalconfig zimbra_zmprov_default_soap_server

If you had the error and it was set to localhost, modify it to be one of your mailstores.

zmlocalconfig -e zimbra_zmprov_default_soap_server=mailstore.example.com

No restart of anything is needed, the zmprov query should now work.

Modify Global - `zmhostname` would use the value returned, you can manual type out the servername as well :

zmprov mcf zimbraMtaRelayHost hostname-of-ext-server:PORT
 ** ex : zmprov mcf zimbraMtaRelayHost primary.YYY.state.XX.us:25
 ** At the end, 25 is the port number for smtp on the targeted system. 
 ** Adjust this number if you changed the smtp port.
zmprov mcf zimbraMtaDnsLookupsEnabled FALSE

Modify Per Server - `zmhostname` would use the value returned, you can manual type out the servername as well :

zmprov ms `zmhostname` zimbraMtaRelayHost hostname-of-ext-server:PORT
 ** ex : zmprov ms `zmhostname` zimbraMtaRelayHost primary.YYY.state.XX.us:25
 ** At the end, 25 is the port number for smtp on the targeted system. 
 ** Adjust this number if you changed the smtp port.
zmprov ms `zmhostname` zimbraMtaDnsLookupsEnabled FALSE

Confirming And Setting zimbraSmtpHostname

I'm assuming you already set, zimbraMtaRelayHost and zimbraMtaDnsLookupsEnabled for your needs - see above section. The variable called zimbraSmtpHostname is in the global (zmprov gacf) and server (zmprov gs `hostname`) configs. It's default value is set to 'localhost' - at least on a single ZCS configuration.

In cases where you need/want all mail to be processed by, for example, an external non-Zimbra spam filter box you could set this variable to the spam filter boxes hostname.

Query Global - `zmhostname` would use the value returned, you can manual type out the servername as well :

zmprov gacf zimbraSmtpHostname

Query Per Server - `zmhostname` would use the value returned, you can manual type out the servername as well :

zmprov gs `zmhostname` zimbraSmtpHostname

Note - if you get errors about doing the query on your non-mailstores like, "ERROR: zclient.IO_ERROR (invoke Connection refused, server: localhost) (cause: java.net.ConnectException Connection refused)" then you might need to adjust this variable. First query it, it's most likely set to localhost

zmlocalconfig zimbra_zmprov_default_soap_server

If you had the error and it was set to localhost, modify it to be one of your mailstores.

zmlocalconfig -e zimbra_zmprov_default_soap_server=mailstore.example.com

No restart of anything is needed, the zmprov query should now work.

Modify Global - `zmhostname` would use the value returned, you can manual type out the servername as well :

zmprov mcf zimbraSmtpHostname hostname-of-ext-server
 ** ex : zmprov mcf zimbraSmtpHostname primary.YYY.state.XX.us

Modify Per Server - `zmhostname` would use the value returned, you can manual type out the servername as well :

zmprov ms `zmhostname` zimbraSmtpHostname hostname-of-ext-server
 ** ex : zmprov ms `zmhostname` zimbraSmtpHostname primary.YYY.state.XX.us

External Email Clients Setting A SMTP Server

The above variable, zimbraSmtpHostname, will not alter your third party email clients that are setting the smtp server to your ZCS mta's. Here are your options if you also need to have that traffic to go through another device [mta, spam filter, etc.] prior to local delivery [lmtp] to an internal address.

  • Set your clients to use the another devices ip address or hostname that you set for zimbraSmtpHostname.

If you can't do the above, for whatever reason -- maybe security constraints or issues that might arise being a hosting provider, then see below.

  1. You could investigate the alternation of postfix's content_filter option to place the external device/host [a barracuda for example] within that process. postfix.org has information on this - Postfix After-Queue Content Filter. This would be unsupported by Zimbra.
  2. Contact Zimbra's Professional Services [PS] team for help.
  3. Setup another server using a mta of your choice [postfix , sendmail] that the clients can use for the smtp server variable. This "new" mta would then simply relay to the device - a barracuda box for example. The barracuda would then do what it needs to and then forwards the messages to the appropriate servers for delivery. Your ZCS mta's in the case of local delivery that would of normally occurred over lmtp - userA@domainC.com sending to userB@domainC.com .

Global Or System Wide Filters

There is no "supported" solution for this - depending on the exact circumstances. It's a complex issue because each request for "global filters" tends to be very specific on either what's to be filter, what actions are to be taken, and so forth.

RFE's Related To Global Filters

Use The Legal Intercept Method

Depending on the details of your request, using the Legal Intercept options might be useful. You could take the results that goto the LI mailbox and then perform your admin global searches against your mailbox accounts to take the appropriate action.

See Legal_Intercept

Sieve Filter Set For Every Account

One could setup a forloop for all of your users and setup a sieve filter. Downside here is the rules are editable by the users and you would have to manage the rules for new accounts going forward.

See User_Migration#Migrating_Sieve_Filter_Rules for details.

Double Check The Current Anti-Spam Options

Make sure your request can't be solved by the current solutions described in articles.

Postfix , Amavis Customizations

I've seen some posts on the forums that customers found their own workarounds by customizing postfix and amavis. This will most likely result in an unsupported situation. Unfortunately, those forum posts don't also include details that I can share here. Hopefully I can find them going forward and I'll post them here.

Global Disclaimer Options

Here's the url to review for a "current" possibility:

And in the notes section there's a comment about multi-servers:

Please note though, "This article is a community contribution and may include unsupported customizations." Meaning, it's an unsupported customization, so please take the necessary precautions.

In regards to an official and supported way to do this, please review this RFE:

Quota Issues

Where To Adjust Message User Gets When They Are Over Quota

Moved to How_To_Adjust_The_Over_Quota_Message_The_User_Receives

See Current User Qoutas

Moved to Getting_All_Users_Quota_Data

Controlling Behavior For Messages Sent To Over Quota Mailbox - LMTP

Moved to Controlling_Behavior_For_Messages_Sent_To_Over_Quota_Mailbox_-_LMTP

Controlling Behavior For Messages Sent To Over Quota Mailbox - SMTP

Moved to Controlling_Behavior_For_Messages_Sent_To_Over_Quota_Mailbox_-_SMTP

Message Senders Receive About Mailbox Over Quota

Moved to Message_Senders_Receive_About_Mailbox_Over_Quota

Quota Not Showing In Admin Console - After ZCS Upgrade

Moved to Quota_Not_Showing_In_Admin_Console_-_After_ZCS_Upgrade

Managing Postfix Queue

Moved to Managing_The_Postfix_Queues#Managing_The_Postfix_Queues

Postfix, Amavis, Clamav Spool Directory Paths And Names

Moved to Managing_The_Postfix_Queues#Postfix.2C_Amavis.2C_Clamav_Spool_Directory_Paths_And_Names

Stop And Starting Postfix And Mta

Moved to Managing_The_Postfix_Queues#Stop_And_Starting_Postfix_And_Mta

To See Postfix Queue

Moved to Managing_The_Postfix_Queues#To_See_The_Postfix_Queues

Qshape - Print Postfix queue domain and age distribution

Moved to Managing_The_Postfix_Queues#Qshape_-_Print_Postfix_queue_domain_and_age_distribution

To View A Message In The Queue

Moved to Managing_The_Postfix_Queues#To_View_A_Message_In_The_Queue

To Flush Postfix Queue

Moved to Managing_The_Postfix_Queues#To_Flush_Postfix_Queue

To Requeue Messages In Postfix

Moved to Managing_The_Postfix_Queues#To_Requeue_Messages_In_Postfix

To Put Messages On Hold

Moved to Managing_The_Postfix_Queues#To_Put_Messages_On_Hold

To Delete Messages From Queue

Moved to Managing_The_Postfix_Queues#To_Delete_Messages_From_Queue

Cautionary Note

Moved to Managing_The_Postfix_Queues#Cautionary_Note

Relevant Sections Of Postsuper Man Page

Moved to Managing_The_Postfix_Queues#Relevant_Sections_Of_Postsuper_Man_Page

To Delete Single Message From Queue

Moved to Managing_The_Postfix_Queues#To_Delete_Single_Message_From_Queue

To Delete ALL Messages From Queue

Moved to Managing_The_Postfix_Queues#To_Delete_ALL_Messages_From_Queue

To Delete ALL Messages From The Deferred Queue

Moved to Managing_The_Postfix_Queues#To_Delete_ALL_Messages_From_The_Deferred_Queue

To Delete ALL Messages From The Hold Queue

Moved to Managing_The_Postfix_Queues#To_Delete_ALL_Messages_From_The_Hold_Queue

To Delete Many Messages From Queue

Moved to Managing_The_Postfix_Queues#To_Delete_Many_Messages_From_Queue

Delete From Queue By Email Address

Moved to Managing_The_Postfix_Queues#Delete_From_Queue_By_Email_Address

From CLI

Moved to Managing_The_Postfix_Queues#From_CLI

Script To Delete From Queue By Email Address

Moved to Managing_The_Postfix_Queues#Script_To_Delete_From_Queue_By_Email_Address

Script To Delete From Queue By Various Variable Targets

Moved to Managing_The_Postfix_Queues#Script_To_Delete_From_Queue_By_Various_Variable_Targets

Verified Against: Zimbra Collaboration 8.0, 7.0 Date Created: 04/16/2014
Article ID: https://wiki.zimbra.com/index.php?title=Ajcody-MTA-Postfix-Topics Date Modified: 2016-06-20



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »




Jump to: navigation, search