Postfix before-queue Milter support


Introduction

Postfix version 2.3 introduces support for Sendmail 8 Milter (mail filter) applications. These applications run outside the MTA to inspect SMTP events (CONNECT, HELO, MAIL FROM, etc.) as well as mail content. A Milter application can instruct the MTA to accept, reject, discard or quarantine an entire connection, command, or message content; to delete or add a recipient or message header; and to replace a message header or entire message body. All this happens before mail is queued.

The reason for adding Milter support to Postfix is that there exists a large collection of applications, not only to block unwanted mail, but also to verify authenticity (examples: SenderID+SPF and Domain keys) or to sign mail (example: Domain keys). Having yet another MTA-specific version of all that software is a poor use of human and system resources.

Postfix 2.3 implements all the requests of Sendmail 8 Milter protocols up to version 4, except one: message body replacement. See, however, the limitations section at the end of this document.

This document provides information on the following topics:

Building Milter applications

Although Milter applications can be written in C, JAVA or Perl, this text deals with C applications only. For these, you need a library that implements the Milter protocol. Postfix currently does not provide such a library.

On some recent Linux and *BSD distributions, the Sendmail libmilter library is installed by default. Applications such as dk-milter and and sid-milter build out of the box:

$ gzcat dk-milter-x.y.z.tar.gz | tar xf -
$ cd dk-milter-x.y.z
$ make
[...lots of output omitted...]

On other platforms you have two options:

Running Milter applications

To run a Milter application, see the documentation of the filter for options. A typical command looks like this:

$ /some/where/dk-filter -p inet:portnumber@localhost ...other options...

Configuring Postfix

Like Sendmail, Postfix has a lot of configuration options that control how it talks to Milter applications. Initially, many options are global, that is, they apply to all Milter applications. Future Postfix versions may support per-Milter timeouts, per-Milter error handling, etc.

Information in this section:

Milter applications

The Milter protocol was originally developed to block unwanted mail that arrives from the network. For this, Postfix uses the Milter applications that are listed with the smtpd_milters parameter. You specify Milter applications by the name of their listening socket (other Milter options will be discussed below). The example below shows only one Milter application, but Postfix allows multiple milters. They will be applied in the order as specified.

/etc/postfix/main.cf:
    # Milters for mail that arrives via the smtpd(8) server.
    # See below for socket address syntax.
    smtpd_milters = inet:localhost:portnumber ...other filters...

Nowadays, Milter applications are also developed to sign mail so that other systems can detect spam, phishing, etc. Using such signing Milters is easy enough for mail that arrives via SMTP, but that does not handle submissions via the Postfix sendmail command line.

To make Milter processing of non-SMTP mail possible, Postfix uses the Milter applications that are listed with the cleanup_milters parameter. This allows you to sign submissions via the Postfix sendmail command.

/etc/postfix/main.cf:
    # Milters for non-SMTP mail.
    # See below for socket address syntax.
    cleanup_milters = inet:localhost:portnumber ...other filters...

To keep the Milter applications happy in the case of non-SMTP mail, the cleanup(8) server simulates SMTP events for CONNECT, EHLO, MAIL FROM, RCPT TO, and DATA, pretending that the mail arrives with ESMTP from "localhost" and IP address "127.0.0.1". However, there is no real ESMTP client involved. For this reason, the Milter application must not reject simulated RCPT TO commands. When it does, Postfix will report a configuration error, and mail will stay in the queue.

The general syntax for listening sockets is as follows:

unix:pathname

Connect to the local UNIX-domain server that is bound to the specified pathname. If the smtpd(8) or cleanup(8) process runs chrooted, an absolute pathname is interpreted relative to the Postfix queue directory.

inet:host:port

Connect to the specified TCP port on the specified local or remote host. The host and port can be specified in numeric or symbolic form.

Note: Postfix syntax differs from Milter syntax which has the form inet:port@host.

Milter error handling

The milter_default_action parameter specifies how Postfix handles errors. The default is to respond with a temporary error status, so that the client will try again later. Specify "accept" if you want to receive mail as if the filter does not exist, and "reject" to reject mail with a permanent status.

    # What to do in case of errors? Specify accept, reject, or tempfail.
    milter_default_action = tempfail

Milter protocol version

As Postfix is not built with the Sendmail libmilter library, you may also need to configure the Milter protocol version that Postfix should use. The default version is 2.

milter_protocol = 2

If the Postfix milter_protocol setting specifies a too low version, the libmilter library will log an error message like this:

application name: st_optionneg[xxxxx]: 0xyy does not fulfill action requirements 0xzz

The remedy is to increase the Postfix milter_protocol version number. See, however, the limitations section below for features that aren't supported by Postfix.

If the Postfix milter_protocol setting specifies a too high version, the libmilter library simply hangs up without logging a warning, and you see a Postfix warning message like one of the following:

postfix/smtpd[21045]: warning: milter inet:host:port: can't read packet header: Unknown error : 0
postfix/cleanup[15190]: warning: milter inet:host:port: can't read packet header: Success

The remedy is to lower the Postfix milter_protocol version number.

Milter protocol timeouts

Postfix uses different time limits at different Milter protocol stages. The table shows wich timeouts are used and when (EOH = end of headers; EOM = end of message).

Parameter Time limit Protocol stage
milter_connect_timeout 30s CONNECT
milter_command_timeout 30s HELO, MAIL, RCPT, DATA, UNKNOWN
milter_content_timeout 300s HEADER, EOH, BODY, EOM

Beware: 30s is not a lot for applications that do a lot of DNS lookups. However, if you increase the above timeouts too much, remote SMTP clients may hang up and mail may be delivered multiple times. This is an inherent problem with before-queue filtering.

Sendmail macro emulation

Postfix emulates a limited number of Sendmail macros, as shown in the table. Different macros are available at different SMTP protocol stages (EOM = end-of-message); their availability is not always the same as in Sendmail. See the workarounds section below for solutions.

Name Availability Description
i DATA, EOM Queue ID
j Always value of myhostname
{auth_authen} MAIL, DATA, EOM SASL login name
{auth_author} MAIL, DATA, EOM SASL sender
{auth_type} MAIL, DATA, EOM SASL login method
{client_addr} Always Client IP address
{client_connections} CONNECT Connection concurrency for this client
{client_name} Always Client hostname, "unknown" when lookup or verification fails
{client_ptr} CONNECT, HELO, MAIL, DATA Client name from reverse lookup, "unknown" when lookup fails
{cert_issuer} HELO, MAIL, DATA, EOM TLS client certificate issuer
{cert_subject} HELO, MAIL, DATA, EOM TLS client certificate subject
{cipher_bits} HELO, MAIL, DATA, EOM TLS session key size
{cipher} HELO, MAIL, DATA, EOM TLS cipher
{daemon_name} Always value of milter_macro_daemon_name
{mail_addr} MAIL Sender address
{rcpt_addr} RCPT Recipient address
{tls_version} HELO, MAIL, DATA, EOM TLS protocol version
v Always value of milter_macro_v

Postfix sends specific sets of macros at different SMTP protocol stages. The sets are configured with the parameters as described in the table (EOM = end of message).

Parameter name Protocol version Protocol stage
milter_connect_macros 2 or higher CONNECT
milter_helo_macros 2 or higher HELO/EHLO
milter_mail_macros 2 or higher MAIL FROM
milter_rcpt_macros 2 or higher RCPT TO
milter_data_macros 4 or higher DATA
milter_end_of_data_macros 2 or higher EOM
milter_unknown_command_macros 3 or higher unknown command

Workarounds

Sendmail Milter applications were originally developed for the Sendmail version 8 MTA, which has a different architecture than Postfix. The result is that some Milter applications make assumptions that aren't true in a Postfix environment.

Limitations

This section lists limitations of the Postfix Milter implementation. Some limitations will be removed disappear as support is extended over time. Of course the usual limitations of before-queue filtering will always apply. See the CONTENT_INSPECTION_README document for a discussion.