Postfix before-queue Milter support Postfix before-queue Milter support Postfix implements support for the Sendmail version 8 Milter (mail filter) protocol. This protocol is used by applications that run outside the MTA to inspect SMTP events (CONNECT, DISCONNECT), SMTP commands (HELO, MAIL FROM, etc.) as well as mail content (headers and body). All this happens before mail is queued.
The reason for adding Milter support to Postfix is that there exists a large collection of applications, not only to block unwanted mail, but also to verify authenticity (examples: OpenDKIM, DomainKeys Identified Mail (DKIM), SenderID+SPF and DomainKeys) or to digitally sign mail (examples: OpenDKIM, DomainKeys Identified Mail (DKIM), DomainKeys). Having yet another Postfix-specific version of all that software is a poor use of human and system resources.
The Milter protocol has evolved over time, and different Postfix versions implement different feature sets. See the workarounds and limitations sections at the end of this document for differences between Postfix and Sendmail implementations.
This document provides information on the following topics:
The Postfix Milter implementation uses two different lists of mail filters: one list of filters for SMTP mail only, and one list of filters for non-SMTP mail. The two lists have different capabilities, which is unfortunate. Avoiding this would require major restructuring of Postfix.
The SMTP-only filters handle mail that arrives via the Postfix smtpd(8) server. They are typically used to filter unwanted mail and to sign mail from authorized SMTP clients. You specify SMTP-only Milter applications with the smtpd_milters parameter as described in a later section. Mail that arrives via the Postfix smtpd(8) server is not filtered by the non-SMTP filters that are described next.
The non-SMTP filters handle mail that arrives via the Postfix sendmail(1) command-line or via the Postfix qmqpd(8) server. They are typically used to digitally sign mail only. Although non-SMTP filters can be used to filter unwanted mail, they have limitations compared to the SMTP-only filters. You specify non-SMTP Milter applications with the non_smtpd_milters parameter as described in a later section.
For those who are familiar with the Postfix architecture, the figure below shows how Milter applications plug into Postfix. Names followed by a number are Postfix commands or server programs, while unnumbered names inside shaded areas represent Postfix queues. To avoid clutter, the path for local submission is simplified (the OVERVIEW document has a more complete description of the Postfix architecture).
SMTP-only
filtersnon-SMTP
filters
^
||
v
^
|
|
||
|
|
vNetwork -> smtpd(8) \ Network -> qmqpd(8) -> cleanup(8) -> incoming / pickup(8) : Local -> sendmail(1)
Milter applications have been written in C, JAVA and Perl, but this document deals with C applications only. For these, you need an object library that implements the Sendmail 8 Milter protocol. Postfix currently does not provide such a library, but Sendmail does.
The first option is to use a pre-compiled library. Some systems install the Sendmail libmilter library by default. With other systems, libmilter may be provided by a package (called "sendmail-devel" on some Linux systems).
Once libmilter is installed, applications such as OpenDKIM, dkim-milter and sid-milter build out of the box without requiring any tinkering:
$ gzcat opendkim-x.y.z.tar.gz | tar xf - $ cd opendkim-x.y.z $ ./configure ...options... $ make [...lots of output omitted...] $ make install
$ gzcat dkim-milter-x.y.z.tar.gz | tar xf - $ cd dkim-milter-x.y.z $ make [...lots of output omitted...]
The other option is to build the libmilter library from Sendmail source code:
$ gzcat sendmail-x.y.z.tar.gz | tar xf - $ cd sendmail-x.y.z/libmilter $ make [...lots of output omitted...]
After building your own libmilter library, follow the installation instructions in the Milter application source distribution to specify the location of the libmilter include files and object library. Typically, these settings are configured in a file named sid-filter/Makefile.m4 or similar:
APPENDDEF(`confINCDIRS', `-I/some/where/sendmail-x.y.z/include') APPENDDEF(`confLIBDIRS', `-L/some/where/sendmail-x.y.z/obj.systemtype/libmilter')
Then build the Milter application.
To run a Milter application, see the documentation of the filter for options. A typical command looks like this:
# /some/where/dkim-filter -u userid -p inet:portnumber@localhost ...other options...
Please specify a userid value that isn't used for other applications (not "postfix", not "www", etc.).
Like Sendmail, Postfix has a lot of configuration options that control how it talks to Milter applications. With the initial Postfix Milter protocol implementation, many options are global, that is, they apply to all Milter applications. Future Postfix versions may support per-Milter timeouts, per-Milter error handling, etc.
Information in this section:
The SMTP-only Milter applications handle mail that arrives via the Postfix smtpd(8) server. They are typically used to filter unwanted mail, and to sign mail from authorized SMTP clients. Mail that arrives via the Postfix smtpd(8) server is not filtered by the non-SMTP filters that are described in the next section.
NOTE: Do not use the header_checks(5) IGNORE action to remove Postfix's own Received: message header. This causes problems with mail signing filters. Instead, keep Postfix's own Received: message header and use the header_checks(5) REPLACE action to sanitize information.
You specify SMTP-only Milter applications (there can be more than one) with the smtpd_milters parameter. Each Milter application is identified by the name of its listening socket; other Milter configuration options will be discussed in later sections. Milter applications are applied in the order as specified, and the first Milter application that rejects a command will override the responses from other Milter applications.
/etc/postfix/main.cf:
# Milters for mail that arrives via the smtpd(8) server.
# See below for socket address syntax.
smtpd_milters = inet:localhost:portnumber ...other filters...
The general syntax for listening sockets is as follows:
- unix:pathname
Connect to the local UNIX-domain server that is bound to the specified pathname. If the smtpd(8) or cleanup(8) process runs chrooted, an absolute pathname is interpreted relative to the Postfix queue directory.
- inet:host:port
Connect to the specified TCP port on the specified local or remote host. The host and port can be specified in numeric or symbolic form.
NOTE: Postfix syntax differs from Milter syntax which has the form inet:port@host.
The non-SMTP Milter applications handle mail that arrives via the Postfix sendmail(1) command-line or via the Postfix qmqpd(8) server. They are typically used to digitally sign mail. Although non-SMTP filters can be used to filter unwanted mail, there are limitations as discussed later in this section. Mail that arrives via the Postfix smtpd(8) server is not filtered by the non-SMTP filters.
NOTE: Do not use the header_checks(5) IGNORE action to remove Postfix's own Received: message header. This causes problems with mail signing filters. Instead, keep Postfix's own Received: message header and use the header_checks(5) REPLACE action to sanitize information.
You specify non-SMTP Milter applications with the non_smtpd_milters parameter. This parameter uses the same syntax as the smtpd_milters parameter in the previous section. As with the SMTP-only filters, you can specify more than one Milter application; they are applied in the order as specified, and the first Milter application that rejects a command will override the responses from the other applications.
/etc/postfix/main.cf:
# Milters for non-SMTP mail.
# See below for socket address syntax.
non_smtpd_milters = inet:localhost:portnumber ...other filters...
There's one small complication when using Milter applications for non-SMTP mail: there is no SMTP session. To keep Milter applications happy, the Postfix cleanup(8) server actually has to simulate the SMTP client CONNECT and DISCONNECT events, and the SMTP client EHLO, MAIL FROM, RCPT TO and DATA commands.
When new mail arrives via the sendmail(1) command line, the Postfix cleanup(8) server pretends that the mail arrives with ESMTP from "localhost" with IP address "127.0.0.1". The result is very similar to what happens with command line submissions in Sendmail version 8.12 and later, although Sendmail uses a different mechanism to achieve this result.
When new mail arrives via the qmqpd(8) server, the Postfix cleanup(8) server pretends that the mail arrives with ESMTP, and uses the QMQPD client hostname and IP address.
When old mail is re-injected into the queue with "postsuper -r", the Postfix cleanup(8) server uses the same client information that was used when the mail arrived as new mail.
This generally works as expected, with only one exception: non-SMTP filters must not REJECT or TEMPFAIL simulated RCPT TO commands. When a non_smtpd_milters application REJECTs or TEMPFAILs a recipient, Postfix will report a configuration error, and mail will stay in the queue.
None of this is a problem for mail filters that digitally sign mail.
The milter_default_action parameter specifies how Postfix handles Milter application errors. The default action is to respond with a temporary error status, so that the client will try again later. Specify "accept" if you want to receive mail as if the filter does not exist, and "reject" to reject mail with a permanent status. The "quarantine" action is like "accept" but freezes the message in the "hold" queue, and is available with Postfix 2.6 or later.
/etc/postfix/main.cf:
# What to do in case of errors? Specify accept, reject, tempfail,
# or quarantine (Postfix 2.6 or later).
milter_default_action = tempfail
As Postfix is not built with the Sendmail libmilter library, you may need to configure the Milter protocol version that Postfix should use. The default version is 6 (before Postfix 2.6 the default version is 2).
/etc/postfix/main.cf:
# Postfix ≥ 2.6
milter_protocol = 6
# 2.3 ≤ Postfix ≤ 2.5
milter_protocol = 2
If the Postfix milter_protocol setting specifies a too low version, the libmilter library will log an error message like this:
application name: st_optionneg[xxxxx]: 0xyy does not fulfill action requirements 0xzz
The remedy is to increase the Postfix milter_protocol version number. See, however, the limitations section below for features that aren't supported by Postfix.
With Postfix 2.7 and earlier, if the Postfix milter_protocol setting specifies a too high version, the libmilter library simply hangs up without logging a warning, and you see a Postfix warning message like one of the following:
warning: milter inet:host:port: can't read packet header: Unknown error : 0 warning: milter inet:host:port: can't read packet header: Success warning: milter inet:host:port: can't read SMFIC_DATA reply packet header: No such file or directory
The remedy is to lower the Postfix milter_protocol version number. Postfix 2.8 and later will automatically turn off protocol features that the application's libmilter library does not expect.
Postfix uses different time limits at different Milter protocol stages. The table shows the timeout settings and the corresponding protocol stages (EOH = end of headers; EOM = end of message).
Postfix parameter Time limit Milter protocol stage milter_connect_timeout 30s CONNECT milter_command_timeout 30s HELO, MAIL, RCPT, DATA, UNKNOWN milter_content_timeout 300s HEADER, EOH, BODY, EOM
Beware: 30s may be too short for Milter applications that do lots of DNS lookups. However, if you increase the above timeouts too much, remote SMTP clients may hang up and mail may be delivered multiple times. This is an inherent problem with before-queue filtering.
Postfix emulates a limited number of Sendmail macros, as shown in the table. Some macro values depend on whether a recipient is rejected (rejected recipients are available on request by the Milter application). Different macros are available at different Milter protocol stages (EOH = end-of-header, EOM = end-of-message); their availability is not always the same as in Sendmail. See the workarounds section below for solutions.
Sendmail macro Milter protocol stage Description i DATA, EOH, EOM Queue ID, also Postfix queue file name j Always Value of myhostname _ Always The validated client name and address {auth_authen} MAIL, DATA, EOH, EOM SASL login name {auth_author} MAIL, DATA, EOH, EOM SASL sender {auth_type} MAIL, DATA, EOH, EOM SASL login method {client_addr} Always Client IP address {client_connections} CONNECT Connection concurrency for this client (zero if the client is excluded from all smtpd_client_* limits). {client_name} Always Client hostname
When address → name lookup or name → address verification fails: "unknown"{client_port} Always (Postfix ≥2.5) Client TCP port {client_ptr} CONNECT, HELO, MAIL, DATA Client name from address → name lookup
When address → name lookup fails: "unknown"{cert_issuer} HELO, MAIL, DATA, EOH, EOM TLS client certificate issuer {cert_subject} HELO, MAIL, DATA, EOH, EOM TLS client certificate subject {cipher_bits} HELO, MAIL, DATA, EOH, EOM TLS session key size {cipher} HELO, MAIL, DATA, EOH, EOM TLS cipher {daemon_name} Always value of milter_macro_daemon_name {mail_addr} Sender address {mail_host} MAIL (Postfix ≥ 2.6, only with smtpd_milters) Sender next-hop destination {mail_mailer} MAIL (Postfix ≥ 2.6, only with smtpd_milters) Sender mail delivery transport {rcpt_addr} RCPT Recipient address
With rejected recipient: descriptive text{rcpt_host} RCPT (Postfix ≥ 2.6, only with smtpd_milters) Recipient next-hop destination
With rejected recipient: enhanced status code{rcpt_mailer} RCPT (Postfix ≥ 2.6, only with smtpd_milters) Recipient mail delivery transport
With rejected recipient: "error"{tls_version} HELO, MAIL, DATA, EOH, EOM TLS protocol version v Always value of milter_macro_v
Postfix sends specific sets of macros at different Milter protocol stages. The sets are configured with the parameters as described in the table (EOH = end of headers; EOM = end of message). The protocol version is a number that Postfix sends at the beginning of the Milter protocol handshake.
As of Sendmail 8.14.0, Milter applications can specify what macros they want to receive at different Milter protocol stages. An application-specified list takes precedence over a Postfix-specified list.
Postfix parameter Milter protocol version Milter protocol stage milter_connect_macros 2 or higher CONNECT milter_helo_macros 2 or higher HELO/EHLO milter_mail_macros 2 or higher MAIL FROM milter_rcpt_macros 2 or higher RCPT TO milter_data_macros 4 or higher DATA milter_end_of_header_macros 6 or higher EOH milter_end_of_data_macros 2 or higher EOM milter_unknown_command_macros 3 or higher unknown command
To avoid breaking DKIM etc. signatures with an SMTP-based content filter, update the before-filter SMTP client in master.cf, and add a line with "-o disable_mime_output_conversion=yes" (note: no spaces around the "="). For details, see the advanced content filter example.
/etc/postfix/master.cf: # ============================================================= # service type private unpriv chroot wakeup maxproc command # (yes) (yes) (yes) (never) (100) # ============================================================= scan unix - - n - 10 smtp -o smtp_send_xforward_command=yes -o disable_mime_output_conversion=yes -o smtp_generic_maps=
Some Milter applications use the "{if_addr}" macro to recognize local mail; this macro does not exist in Postfix. Workaround: use the "{client_addr}" macro instead.
Some Milter applications log a warning that looks like this:
sid-filter[36540]: WARNING: sendmail symbol 'i' not available
And they may insert an ugly message header with "unknown-msgid" like this:
X-SenderID: Sendmail Sender-ID Filter vx.y.z host.example.com <unknown-msgid>
The problem is that Milter applications expect that the queue ID is known before the MTA accepts the MAIL FROM (sender) command. Postfix does not choose a queue ID, which is used as the queue file name, until after it accepts the first valid RCPT TO (recipient) command.
If you experience the ugly header problem, see if a recent version of the Milter application fixes it. For example, current versions of dkim-filter and dk-filter already have code that looks up the Postfix queue ID at a later protocol stage, and sid-filter version 1.0.0 no longer includes the queue ID in the message header.
To fix the ugly message header, you will need to add code that looks up the Postfix queue ID at some later point in time. The example below adds the lookup after the end-of-message.
Edit the filter source file (typically named xxx-filter/xxx-filter.c or similar).
Look up the mlfi_eom() function and add code near the top shown as bold text below:
dfc = cc->cctx_msg;
assert(dfc != NULL);
/* Determine the job ID for logging. */
if (dfc->mctx_jobid == 0 || strcmp(dfc->mctx_jobid, JOBIDUNKNOWN) == 0) {
char *jobid = smfi_getsymval(ctx, "i");
if (jobid != 0)
dfc->mctx_jobid = jobid;
}
NOTES:
Different mail filters use slightly different names for variables. If the above code does not compile, look elsewhere in the mail filter source file for code that looks up the "i" macro value, and copy that code.
This change fixes only the ugly message header, but not the WARNING message. Fortunately, many Milters log that message only once.
This section lists limitations of the Postfix Milter implementation. Some limitations will be removed as the implementation is extended over time. Of course the usual limitations of before-queue filtering will always apply. See the CONTENT_INSPECTION_README document for a discussion.
The Milter protocol has evolved over time. Therefore, different Postfix versions implement different feature sets.
| Postfix | Supported Milter requests |
|---|---|
| 2.6 | All Milter requests of Sendmail 8.14.0 (see notes below). |
| 2.5 | All Milter requests of
Sendmail 8.14.0, except: SMFIP_RCPT_REJ (report rejected recipients to the mail filter), SMFIR_CHGFROM (replace sender, with optional ESMTP parameters), SMFIR_ADDRCPT_PAR (add recipient, with optional ESMTP parameters). |
| 2.4 | All Milter requests of Sendmail 8.13.0. |
| 2.3 | All Milter requests of
Sendmail 8.13.0, except: SMFIR_REPLBODY (replace message body). |
For Milter applications that are written in C, you need to use the Sendmail libmilter library.
Postfix has TWO sets of mail filters: filters that are used for SMTP mail only (specified with the smtpd_milters parameter), and filters for non-SMTP mail (specified with the non_smtpd_milters parameter). The non-SMTP filters are primarily for local submissions.
When mail is filtered by non_smtpd_milters, the Postfix cleanup(8) server has to simulate SMTP client requests. This works as expected, with only one exception: non_smtpd_milters must not REJECT or TEMPFAIL simulated RCPT TO commands. When this rule is violated, Postfix will report a configuration error, and mail will stay in the queue.
Postfix currently does not apply content filters to mail that is forwarded or aliased internally, or to mail that is generated internally such as bounces or Postmaster notifications. This may be a problem when you want to apply a signing Milter to such mail.
When you use the before-queue content filter for incoming SMTP mail (see SMTPD_PROXY_README), Milter applications have access only to the SMTP command information; they have no access to the message header or body, and cannot make modifications to the message or to the envelope.
Postfix 2.6 ignores the optional ESMTP parameters in requests to replace the sender (SMFIR_CHGFROM) or to append a recipient (SMFIR_ADDRCPT_PAR). Postfix logs a warning message when a Milter application supplies such ESMTP parameters:
warning: queue-id: cleanup_chg_from: ignoring ESMTP arguments "whatever" warning: queue-id: cleanup_add_rcpt: ignoring ESMTP arguments "whatever"
Postfix 2.3 does not implement requests to replace the message body. Milter applications log a warning message when they need this unsupported operation:
st_optionneg[134563840]: 0x3d does not fulfill action requirements 0x1e
The solution is to use Postfix version 2.4 or later.
Most Milter configuration options are global. Future Postfix versions may support per-Milter timeouts, per-Milter error handling, etc.