Ajcody-MTA-Postfix-Topics

Revision as of 19:50, 21 July 2009 by Ajcody (talk | contribs) (To Flush Postfix Queue)

Attention.png - This article is NOT official Zimbra documentation. It is a user contribution and may include unsupported customizations, references, suggestions, or information.

Contents

Postfix - MTA

Actual MTA & Postfix Topics Homepage

Please see Ajcody-MTA-Postfix-Topics

Missing main.cf Error

If you get something like:

postfix/postqueue[8739]: fatal: open /opt/zimbra/postfix-2.4.7.5z/conf/main.cf: No such file or directory 

All you need to do is create an empty main.cf file and zimbra will rewrite it.

touch main.cf

When you now do something to start the mta, it will generate the values for main.cf

zmmtactl stop
zmmtactl start

Adding A New MTA Server

Basic instructions can be found here:

Additional instructions needed beyond the above will follow as I hear about them.

Load Balancing For SMTP - Out Bound Mail

Currently, 5.x code, you have the following options:

    • An external load balancing device that will then split the traffic behind it
    • Setup a round-robin A record situation in your DNS for the external mta's you'll be using.

In the future, GNR/6.x, you'll be able to add multiple targets to the variables and we'll have some degree of "balancing" between them.

User Alias Mapping And Mail Transport with Postfix & LDAP

See User_Alias_Mapping_and_Mail_Transport_with_Postfix_&_LDAP

Traditional Aliases Use - /etc/aliases type lookups

Filed this RFE:

Allowing Accounts To Change The From Address

Please see:

Related BUG/RFE's

Creating A Domain Alias

Please see ManagingDomains#Creating_a_Domain_Alias

Relay Domain Forwarding

Please see ManagingDomains#Relaying.2FDomain_Forwarding

Domain Catchall

Please see ManagingDomains#Domain_Catchall

Rewriting From Address For Outbound Email

Please see ManagingDomains#Domain_Masquerading

Rewrite Recipient Address For Incoming Email

There is a way to rewrite the incoming mail, but it's not a standard Zimbra feature. You can implement it as a configuration change in Postfix. Here's what you do:

  1. Create a file in /opt/zimbra/conf named 'postfix_recipientmap'.
    • The format is a single line that reads something like: @alias.domain.com @domain.com
  2. Run 'postmap postfix_recipientmap' in the conf directory.
  3. Run "postconf -e recipient_canonical_maps=hash:/opt/zimbra/conf/postfix_recipientmap".
  4. Run 'postfix reload'.

This will cause postfix to map any incoming mail with a recipient of '@alias.domain.com' to '@domain.com'. You will need to re-apply this postconf change after upgrades, though the postfix_recipientmap file should survive.

Automatic BCC

Please see the following:

Limiting Or Increasing Number Of Recipents / Messages

Mailing Lists - Distribution Lists

Please see Ajcody-MailingLists-And-Mailman#Problems_Resolving_Virtual_Aliases_For_Members_Of_Large_Distribution_Lists

Policy Daemon

If you want to restrict messages per hour, you can look into Policy Daemon:

Postfix

Also, there are some default postfix parameters set to control sending a message to x amount recipients. The parameters you will need to look at are smtpd_recipient_limit & smtpd_recipient_overshoot_limit, these have a default value of 1000.

Postfix defines these parameters as:

  • smtpd_recipient_limit: The maximum number of recipients that the Postfix SMTP server accepts per message delivery request.
  • smtpd_recipient_overshoot_limit: The number of recipients that a remote SMTP client can send in excess of the limit specified with $smtpd_recipient_limit, before the Postfix SMTP server increments the per-session error count for each excess recipient.

From the command line you can change the default values.

su - zimbra
postconf -e smtpd_recipient_limit=<new value>
postconf -e smtpd_recipient_overshoot_limit=<new value>
postfix reload

Bugs RFE's For Customers To Get Behind

I'm wondering if policyd gives one the control everyone is looking for? I've not used it myself.

Policyd References:

There's other additions [add-on's] one can get for policyd.

We have this RFE in regards to policyd support:

Other related rfe/bugs, specially to push variables into admin web console:

RFE 14645 is along the lines of what I'm thinking of as the end goal.

Controlling SMTPD Client Connections

Postfix Resources At Their Site (All Clients/Connections):

Postfix Resources At Their Site (Exceptions To Clients/Connections Or Single Source):

Spam Control And Related Issues

Resources

A list of resources you'll find useful:

http://wiki.zimbra.com/index.php?title=Zimbra_MTA#Anti-Spam_Training_Filters

http://wiki.zimbra.com/index.php?title=CLI_zmtrainsa

http://wiki.zimbra.com/index.php?title=Improving_Anti-spam_system

http://wiki.zimbra.com/index.php?title=Postfix_Policyd

http://wiki.zimbra.com/index.php?title=IP_Address_whitelisting

http://wiki.zimbra.com/index.php?title=Spam_training

Wiki articles that have been assigned to the anit-spam category:

http://wiki.zimbra.com/index.php?title=Category:Anti-spam

Down to the end-user:

http://wiki.zimbra.com/index.php?title=Cool_User_Spam_Filters

http://www.zimbra.com/community/end_user_guide_and_how_to.html

Some SMTP Sending Restrictions

check_client_access

The smtpd_client_restrictions parameter restricts what clients this system accepts SMTP connections from. The default behavior is to allow SMTP connections from any client. This is discussed under Spam Controls on the Postfix site.

Example:

check_client_access regexp:/etc/postfix/access_sender_client_server,

Example:

check_sender_access regexp:/etc/postfix/access_sender_toplevel
smtpd_reject_unlisted_sender

Details can be found on the mail.cf Postfix page.

Example:

smtpd_reject_unlisted_sender = yes

Possible Bug:

http://bugzilla.zimbra.com/show_bug.cgi?id=24889

What's Your SPF Records Say, When Getting "does not designate 74.x.x.x as permitted sender Errors"

This is most likely related to the SPF records for your domain and what the header content of the sending email states as it's Mail From. For example, this is from the header of an email that was "received":

Received: from mail.XYZ-FAKE.com (mailhost.XYZ-FAKE.com [74.X.X.244]) by mta01.ABC-FAKE.com with ESMTP id 
Cft0mO3fjlFGQjTA for <support@ABC-FAKE.com>; Tue, 21 Apr 2009 05:14:13 -0700 (PDT)
X-Barracuda-Envelope-From: testuser@XYZ-FAKE.com
Received-SPF: pass (mta01.ABC-FAKE.com: domain of testuser@XYZ-FAKE.com designates 74.X.X.244 as permitted sender) 
receiver=mta01.ABC-FAKE.com; client_ip=74.X.X.244; envelope-from=testuser@XYZ-FAKE.com;

To see what this check was done against, do the following below. I'll trim the output and adjust the information used to protect the innocent. Also, notice how a DNS "alias" might cause an issue here? :


$ host 74.X.X.244
244.X.X.74.in-addr.arpa domain name pointer mailhost.XYZ-FAKE.com.

$ host mailhost.XYZ-FAKE.com
mailhost.XYZ-FAKE.com has address 74.X.X.244

$ host mail.XYZ-FAKE.com
mail.XYZ-FAKE.com is an alias for mailhost.XYZ-FAKE.com.
mailhost.XYZ-FAKE.com has address 74.X.X.244

$ dig XYZ-FAKE.com MX

;; QUESTION SECTION:
;XYZ-FAKE.com.			IN	MX

;; ANSWER SECTION:
XYZ-FAKE.com.		3600	IN	MX	22 serverA.DNS-FAKE.com.
XYZ-FAKE.com.		3600	IN	MX	11 serverB.DNS-FAKE.com.

$ dig XYZ-FAKE.com TXT

;; QUESTION SECTION:
;XYZ-FAKE.com.			IN	TXT

;; ANSWER SECTION:
XYZ-FAKE.com.		3600	IN	TXT	"v=spf1 a:mail.XYZ-FAKE.com ~all"

See the following for more information:

Using Different SMTP Server For Webclient (ZWC), Mobiles, And ZCO

There a variable called zimbraSmtpHostname that is in the global (zmprov gacf) and server (zmprov gs `hostname`) configs. It's defaulted to 'localhost' - at least on a single ZCS configuration.

In cases where you need/want all mail to be processed by, for example, an external non-Zimbra spam filter box you could set this variable to the spam filter boxes hostname.

Global:

zmprov mcf zimbraSmtpHostname hostname-of-ext-server

Per Server:

zmprov ms servername zimbraSmtpHostname hostname-of-ext-server

Global Or System Wide Filters

There is no "supported" solution for this - depending on the exact circumstances. It's a complex issue because each request for "global filters" tends to be very specific on either what's to be filter, what actions are to be taken, and so forth.

RFE's Related To Global Filters

Use The Legal Intercept Method

Depending on the details of your request, using the Legal Intercept options might be useful. You could take the results that goto the LI mailbox and then perform your admin global searches against your mailbox accounts to take the appropriate action.

See Legal_Intercept

Sieve Filter Set For Every Account

One could setup a forloop for all of your users and setup a sieve filter. Downside here is the rules are editable by the users and you would have to manage the rules for new accounts going forward.

See User_Migration#Migrating_Sieve_Filter_Rules for details.

Double Check The Current Anti-Spam Options

Make sure your request can't be solved by the current solutions described in articles.

Postfix , Amavis Customizations

I've seen some posts on the forums that customers found their own workarounds by customizing postfix and amavis. This will most likely result in an unsupported situation. Unfortunately, those forum posts don't also include details that I can share here. Hopefully I can find them going forward and I'll post them here.

Global Disclaimer Options

Here's the url to review for a "current" possibility:

And in the notes section there's a comment about multi-servers:

Please note though, "This article is a community contribution and may include unsupported customizations." Meaning, it's an unsupported customization, so please take the necessary precautions.

In regards to an official and supported way to do this, please review this RFE:

Quota Issues

Where To Adjust Message User Gets When They Are Over Quota

  • From the web admin console:
    • Configuration > Class of Service
    • Select the COS in question
    • Then goto the Advanced tab on the right
    • There's a quota section. The sub-section you want is called:
      • "Quota warning message template:"

Controlling Behavior For Messages Sent To Over Quota Mailbox

The variable to set for a 452 Temp/Try Again response verses a 552 Permanent Error.

zmprov gacf zimbraLmtpPermanentFailureWhenOverQuota

Setting to TRUE will flag it for the 552 response.

Message Senders Receive About Mailbox Over Quota

File that holds text of message:

/opt/zimbra/postfix/conf/bounce.cf.default

Note, please read the bounce MAN page before you attempt to edit this file directly.

Also, I haven't been able to confirm the relationship of this above file with the postconf default output:

[root@mail3 ~]# postconf | grep -i bounce
2bounce_notice_recipient = postmaster
backwards_bounce_logfile_compatibility = yes
bounce_notice_recipient = postmaster
bounce_queue_lifetime = 5d
bounce_service_name = bounce
bounce_size_limit = 50000
bounce_template_file = 
disable_verp_bounces = no
double_bounce_sender = double-bounce
multi_recipient_bounce_reject_code = 550
soft_bounce = no

To use a bounce.cf file, you'll want to add the file to variable and reload postfix via the zmmtactl script. It looks like zmlocalconfig doesn't currently handle this variable.

cp /opt/zimbra/postfix/conf/bounce.cf.default /opt/zimbra/postfix/conf/bounce.cf
postconf -e bounce_template_file="/opt/zimbra/postfix/conf/bounce.cf"
zmmtactl reload

Note, this might get lost during upgrades so make a note to yourself about this change.

Managing Postfix Queue

Postfix, Amavis, Clamav Spool Directory Paths And Names

ls /opt/zimbra/data
 amavisd  clamav  dspam  postfix
ls /opt/zimbra/data/postfix/spool/
 active  active.old  bounce  corrupt  defer  deferred  
 flush  hold  incoming  incoming.old  maildrop  pid  
 private  public  saved  trace

Stop And Starting Postfix And Mta

To only stop and start postfix:

postfix stop
postfix start

To stop and start postfix, amavis, and clam:

zmmtactl stop
zmmtactl start

To See Postfix Queue

/opt/zimbra/postfix/sbin/postqueue -p

or

mailq

To Flush Postfix Queue

postqueue -f

To Requeue Messages In Postfix

postsuper -r ALL

More explanation from the postsuper man page:

-r queue_id
              Requeue  the  message  with the named queue ID from
              the named mail queue(s) (default:  hold,  incoming,
              active  and  deferred).   To  requeue multiple mes-
              sages, specify multiple -r command-line options.

              Alternatively, if a queue_id of - is specified, the
              program reads queue IDs from standard input.

              Specify  "-r  ALL"  to  requeue  all messages. As a
              safety measure, the word ALL must be  specified  in
              upper case.

              A  requeued message is moved to the maildrop queue,
              from where  it  is  copied  by  the  pickup(8)  and
              cleanup(8)  daemons  to  a  new queue file. In many
              respects its handling differs from that  of  a  new
              local submission.

              o      The   message   is   not  subjected  to  the
                     smtpd_milters or non_smtpd_milters settings.
                     When  mail  has  passed  through an external
                     content filter, this would produce incorrect
                     results with Milter applications that depend
                     on original SMTP connection  state  informa-
                     tion.

              o      The  message  is  subjected  again  to  mail
                     address rewriting and substitution.  This is
                     useful  when rewriting rules or virtual map-
                     pings have changed.

                     The  address  rewriting  context  (local  or
                     remote)  is the same as when the message was
                     received.

              o      The message is subjected to  the  same  con-
                     tent_filter  settings  (if  any) as used for
                     new local mail submissions.  This is  useful
                     when content_filter settings have changed.

              Warning:  Postfix queue IDs are reused.  There is a
              very small possibility that  postsuper(1)  requeues
              the  wrong  message  file when it is executed while
              the Postfix mail system is  running,  but  no  harm
              should be done.

              This feature is available in Postfix 1.1 and later.

To Delete Messages From Queue

Cautionary Note
Warning, deleting messages from the queue can have a negative consequence of your users. You might need to account for the action and/or confirm your deletion was appropriate. Please try to save the postqueue -p information from the various messages prior to deleting them. This will at least you give you he information to later justify your actions on why you delete msg#.
Relevant Sections Of Postsuper Man Page
 By  default,  postsuper(1)  performs  the operations requested with the -s and -p 
 command-line options on all Postfix queue directories - this includes the incoming, 
 active and deferred directories with mail files and the bounce, defer, trace and flush 
 directories with log files.

  -d queue_id Delete one message with the named queue ID from the named mail queue(s) 
     (default: hold, incoming, active and deferred).
     If a queue_id of - is specified, the program reads queue IDs from standard input. 
     For example, to delete all mail with exactly one recipient user@example.com:

              mailq | tail +2 | grep -v '^ *(' | awk  'BEGIN { RS = "" }
                  # $7=sender, $8=recipient1, $9=recipient2
                  { if ($8 == "user@example.com" && $9 == "")
                        print $1 }
              ' | tr -d '*!' | postsuper -d -

      Specify "-d ALL" to remove all messages; for example, specify "-d ALL deferred" 
      to delete all mail in the deferred queue.  As a safety measure, the word ALL must 
      be specified in upper case.

      Warning: Postfix queue IDs are reused.  There is a very small possibility that 
      postsuper deletes the wrong message file when it is executed while the Postfix mail 
      system is delivering mail.

      The scenario is as follows:
      1)  The Postfix queue manager deletes the message that postsuper(1) is asked to 
         delete, because Postfix is finished with the message (it is delivered, or it is 
         returned to the sender).
      2)  New mail arrives, and the new message is given the same queue ID as the message 
         that postsuper(1) is supposed to delete.  The probability for reusing a deleted 
         queue ID is about 1 in 2**15 (the number  of  different microsecond values that 
         the system clock can distinguish within a second).
      3)  postsuper(1) deletes the new message, instead of the old message that it should 
         have deleted.

  -h queue_id Put mail "on hold" so that no attempt is made to deliver it.  Move one 
     message with the named queue ID from the named mail queue(s) (default: incoming, 
     active and deferred) to the hold queue.

     If a queue_id of - is specified, the program reads queue IDs from standard input.
     Specify "-h ALL" to hold all messages; for example, specify "-h ALL deferred" to hold
     all mail in the deferred queue.  As a safety measure, the word ALL must be specified 
     in upper case.
     Note: while mail is "on hold" it will not expire when its time in the queue exceeds 
     the maximal_queue_lifetime or bounce_queue_lifetime setting. It becomes subject to 
     expiration after it is released from "hold".

  -H queue_id Release mail that was put "on hold".  Move one message with the named queue 
     ID from the named mail queue(s) (default: hold) to the deferred queue.

     If a queue_id of - is specified, the program reads queue IDs from standard input.
     Note: specify "postsuper -r" to release mail that was kept on hold for a significant 
     fraction of $maximal_queue_lifetime or $bounce_queue_lifetime, or longer.

     Specify "-H ALL" to release all mail that is "on hold".  As a safety measure, the 
     word ALL must be specified in upper case.

   -p Purge old temporary files that are left over after system or software crashes.
To Delete Single Message From Queue
/opt/zimbra/postfix/sbin/postsuper -d [MSGID From postqueue -p]
To Delete ALL Messages From Queue
/opt/zimbra/postfix/sbin/postsuper -d ALL

Another way to do this:

mailq | awk '{print $1}' | postsuper -d -
To Delete ALL Messages From The Deferred Queue
/opt/zimbra/postfix/sbin/postsuper -d ALL deferred
To Delete Many Messages From Queue

To delete a large number of files one would use:

/opt/zimbra/postfix/sbin/postsuper -d - < filename-with-queue-ids.txt

The filename, filename-with-queue-ids.txt example, would have a listing of id's like:

3E1C6CAFFFE
6B862CC9D76
0BC38CC1BC9
90628CC6F3C
E26B9CC3C62
92A35CC943D
A84BDBCE15D
EA57CB1DF04
0F102CC74CB
386E8CC4DFF
92606CC0BDA
0799FC8149A
024CFCBD0DE
2D30FC47DA0
31D85CC6308
B8B3FC3DEBC
AA4C7C913D0
280F5CC8C6C
9F341CC8A26
93CD1B3B0EC
433D0BF3716
A1435CB4C38
2DB04CC911D
56A29CC8819
11881C8268C
5C050A79851
C6739CC4BA5
11D3FCC7D09
8CBC0B20E0A
Delete From Queue By Email Address
From CLI

Change the [ email@address.com ] variable below first.

To first see what would be deleted:

mailq | tail +2 | grep -v '^ *(' | awk  'BEGIN { RS = "" } { if ($8 == "email@address.com" && $9 == "") print $1 } ' | tr -d '*!'

To now delete, just include the postsuper -d at end:

mailq | tail +2 | grep -v '^ *(' | awk  'BEGIN { RS = "" } { if ($8 == "email@address.com" && $9 == "") print $1 } ' | tr -d '*!' | postsuper -d -
Script To Delete From Queue By Email Address
Non-Zimbra Script and not QA'd or tested. Path adjusted though for /opt/zimbra/postfix/sbin/*

From http://www.ustrem.org/en/articles/postfix-queue-delete-en/

Save on file system, calling it something like - delete-queue-by-email.sh . Give it execute permission. Example usage would be: ./delete-queue-by-email.sh user-name@domain-test.com

#!/usr/bin/perl -w
#
# pfdel - deletes message containing specified address from
# Postfix queue. Matches either sender or recipient address.
#
# Usage: pfdel <email_address>
#

use strict;

# Change these paths if necessary.
my $LISTQ = "/opt/zimbra/postfix/sbin/postqueue -p";
my $POSTSUPER = "/opt/zimbra/postfix/sbin/postsuper";

my $email_addr = "";
my $qid = "";
my $euid = $>;

if ( @ARGV !=  1 ) {
	die "Usage: pfdel <email_address>\n";
} else {
	$email_addr = $ARGV[0];
}

if ( $euid != 0 ) {
        die "You must be root to delete queue files.\n";
}


open(QUEUE, "$LISTQ |") || 
  die "Can't get pipe to $LISTQ: $!\n";

my $entry = <QUEUE>;	# skip single header line
$/ = "";		# Rest of queue entries print on
			# multiple lines.
while ( $entry = <QUEUE> ) {
	if ( $entry =~ / $email_addr$/m ) {
		($qid) = split(/\s+/, $entry, 2);
		$qid =~ s/[\*\!]//;
		next unless ($qid);

		#
		# Execute postsuper -d with the queue id.
		# postsuper provides feedback when it deletes
		# messages. Let its output go through.
		#
		if ( system($POSTSUPER, "-d", $qid) != 0 ) {
			# If postsuper has a problem, bail.
			die "Error executing $POSTSUPER: error " .
			   "code " .  ($?/256) . "\n";
		}
	}
}
close(QUEUE);

if (! $qid ) {
	die "No messages with the address <$email_addr> " .
	  "found in queue.\n";
}

exit 0;
Script To Delete From Queue By Various Variable Targets
Non-Zimbra Script and not QA'd or tested. Path adjusted though for /opt/zimbra/postfix/sbin/*

From http://jwcub.wordpress.com/2006/01/20/bulk-delete-from-postfix-queue/

Perl script called “delete-from-mailq”:

#!/usr/bin/perl

$REGEXP = shift || die “no email-adress given (regexp-style, e.g. bl.*\@yahoo.com)!”;

@data = qx;
for (@data) {
if (/^(\w+)(\*|\!)?\s/) {
$queue_id = $1;
}
if($queue_id) {
if (/$REGEXP/i) {
$Q{$queue_id} = 1;
$queue_id = “”;
}
}
}

open(POSTSUPER,”|/opt/zimbra/postfix/sbin/postsuper -d -”) || die “couldn’t open postsuper” ;

foreach (keys %Q) {
print POSTSUPER “$_\n”;
};
close(POSTSUPER);

Save the above script to a file say “delete-queue.pl” in your home directory, and make it excutable:

chmod 755 delete-queue

Usage:

  • Delete all queued messages from or to the domain “iamspammer.com”
./delete-queue iamspammer.com
  • Delete all queued messages to specific address “bogususer@mydomain.com”
./delete-queue bogususer@mydomain.com
  • Delete all queued messages that begin with the word “bush” in the e-mail address:
./delete-queue bush*\@whateverdomain.com
  • Delete all queued messages that contain the word “biz” in the e-mail address:
./delete-queue biz
Jump to: navigation, search