Postfix Postscreen HowtoPostfix Postscreen HowtoThe Postfix postscreen(8) server performs triage on multiple inbound SMTP connections at the same time. While a single postscreen(8) process keeps zombies away from Postfix SMTP server processes, more Postfix SMTP server processes remain available for legitimate clients.
postscreen(8) should not be used on SMTP ports that receive mail from end-user clients (MUAs). In a typical deployment, postscreen(8) is used on the "port 25" service, while MUA clients submit mail via the submission service.
postscreen(8) is part of a multi-layer defense.
As the first layer, postscreen(8) blocks connections from zombies and other spambots that are responsible for about 90% of all spam. It is implemented as a single process to make this defense as cheap as possible.
The second layer implements more complex SMTP-level access checks with Postfix SMTP servers, policy daemons, and Milter applications.
The third layer performs light-weight content inspection with the Postfix built-in header_checks and body_checks. This can block unacceptable attachments such as executable programs, and worms or viruses with easy-to-recognize signatures.
The fourth layer provides heavy-weight content inspection with external content filters. Typical examples are Amavisd-new, SpamAssassin, and Milter applications.
Each layer reduces the spam volume. The general strategy is to use the less expensive defenses first, and to use the more expensive defenses for the spam that remains.
Topics in this document:
Most email is spam, and most spam is sent out by zombies (malware on compromised end-user computers). Wietse expects that the zombie problem will get worse before things improve, if ever. Without a tool like postscreen(8) that keeps the zombies away, Postfix would be spending most of its resources not receiving email.
The main challenge for postscreen(8) is to make an is-it-a-zombie decision based on a single measurement. This is necessary because many zombies try to fly under the radar and avoid spamming the same site repeatedly. Once postscreen(8) decides that a client is not-a-zombie, it whitelists the client temporarily to avoid further delays for legitimate mail.
Zombies have challenges too: they have only a limited amount of time to deliver spam before their IP address becomes blacklisted. To speed up spam deliveries, zombies make compromises in their SMTP protocol implementation. For example, they speak before their turn, or they ignore responses from SMTP servers and continue sending mail even when the server tells them to go away.
postscreen(8) uses a variety of measurements to recognize zombies. First, postscreen(8) determines if the remote SMTP client IP address is blacklisted. Second, postscreen(8) looks for protocol compromises that are made to speed up delivery. These are good indicators for making is-it-a-zombie decisions based on single measurements.
postscreen(8) does not inspect message content. Message content can vary from one delivery to the next, especially with clients that (also) send legitimate email. Content is not a good indicator for making is-it-a-zombie decisions based on single measurements, and that is the problem that postscreen(8) is focused on.
The postscreen(8) triage process involves a number of tests, in the order as described below. Some tests introduce a delay of a few seconds. Once a client passes a test, its IP address is whitelisted from 24 hours for simple tests, to 1 week for complex tests. Whitelisting minimizes the impact of postscreen(8)'s tests on legitimate mail clients.
By default, postscreen(8) hands off all connections to a Postfix SMTP server process after logging its findings. This mode is useful for non-destructive testing.
In a typical production setting, postscreen(8) is configured to reject mail from clients that fail one or more tests, after logging the helo, sender and recipient information.
Note: postscreen(8) is not an SMTP proxy; this is intentional. The purpose is to keep zombies away from Postfix, with minimal overhead for legitimate clients.
Before engaging in SMTP-level tests. postscreen(8) queries a number of local black and whitelists. These tests speed up the handling of known clients.
The postscreen_access_list parameter (default: permit_mynetworks) specifies a permanent access list for SMTP client IP addresses. Typically one would specify something that whitelists local networks, followed by a CIDR table for selective white- and blacklisting.
Example:
/etc/postfix/main.cf:
postscreen_access_list = permit_mynetworks,
cidr:/etc/postfix/postscreen_access.cidr
/etc/postfix/postscreen_access.cidr:
# Rules are evaluated in the order as specified.
# Blacklist 192.168.* except 192.168.0.1.
192.168.0.1 permit
192.168.0.0/16 reject
See the postscreen_access_list manpage documentation for more details.
When the SMTP client address matches a "permit" action, postscreen(8) logs this with the client address and port number as:
WHITELISTED [address]:port
The whitelist action is not configurable: immediately hand off the connection to a Postfix SMTP server process.
When the SMTP client address matches a "reject" action, postscreen(8) logs this with the client address and port number as:
BLACKLISTED [address]:port
The postscreen_blacklist_action parameter specifies the action that is taken next. See "When tests fail before the 220 SMTP server greeting" below.
The postscreen(8) daemon maintains a temporary whitelist for SMTP client IP addresses that have passed all the tests described below. The postscreen_cache_map parameter specifies the location of the temporary whitelist. The temporary whitelist is not used for SMTP client addresses that appear on the permanent access list.
When the SMTP client address appears on the temporary whitelist, postscreen(8) logs this with the client address and port number as:
PASS OLD [address]:port
The action is not configurable: immediately hand off the connection to a Postfix SMTP server process. The client is excluded from further tests until its temporary whitelist entry expires, as controlled with the postscreen_*_ttl parameters. Expired entries are silently renewed if possible.
When the remote SMTP client is not on the static access list or temporary whitelist, postscreen(8) can implement a number of whitelist tests before it grants the client a temporary whitelist status to talk to a Postfix SMTP server process.
By listening on both primary and backup MX addresses, postscreen(8) can deny the temporary whitelist status to clients that connect only to backup MX hosts (an old trick to take advantage of backup MX hosts with weaker anti-spam policies).
Note 1: The status of this feature is still experimental, and implementation details are likely to change.
Note 2: MX policy enforcement is currently supported only for domains with one Postfix MTA. Support for domains with multiple Postfix MTAs will have to wait until Postfix has a database client that can update a shared postscreen(8) database.
First, configure the host to listen on both primary and backup MX addresses. Use the appropriate ifconfig command for the local operating system, or update the appropriate configuration files and "refresh" the network protocol stack.
Then, configure postscreen(8) to deny the temporary whitelist status on the backup MX address(es). An example for Wietse's server is:
/etc/postfix/main.cf: postscreen_whitelist_interfaces = !168.100.189.8 static:all
Translation: allow clients to obtain the temporary whitelist status on all server IP addresses except 168.100.189.8, which is a backup MX address.
When a non-whitelisted client connects the backup MX address, postscreen(8) logs this with the client address and port number as:
CONNECT from [address]:port to [168.100.189.8]:25
WHITELIST VETO [address]:port
Translation: the client at [address]:port connected to the backup MX address 168.100.189.8 while it was not whitelisted. The client will not be granted the temporary whitelist status, even if passes all the whitelist tests described below.
The postscreen_greet_wait parameter specifies a short time interval before the "220 text..." server greeting, where postscreen(8) can run a number of tests in parallel.
When a good client passes these tests, and no "deep protocol tests" are configured, postscreen(8) adds the client to the temporary whitelist and hands off the "live" connection to a Postfix SMTP server process. The client can then continue as if postscreen(8) never even existed (except of course for the short postscreen_greet_wait delay).
The SMTP protocol is a classic example of a protocol where the server speaks before the client. postscreen(8) detects zombies that are in a hurry and that speak before their turn. This test is enabled by default.
The postscreen_greet_banner parameter specifies the text portion of a "220-text..." teaser banner (default: $smtpd_banner). Note that this becomes the first part of a multi-line server greeting. The postscreen(8) daemon sends this before the postscreen_greet_wait timer is started. The purpose of the teaser banner is to confuse zombies so that they speak before their turn. It has no effect on SMTP clients that correctly implement the protocol.
To avoid problems with poorly-implemented SMTP engines in network appliances or network testing tools, either exclude them from all tests with the postscreen_access_list feature or else specify an empty teaser banner:
/etc/postfix/main.cf:
# Exclude broken clients by whitelisting. Clients in mynetworks
# should always be whitelisted.
postscreen_access_list = permit_mynetworks,
cidr:/etc/postfix/postscreen_access.cidr
/etc/postfix/postscreen_access.cidr:
192.168.254.0/24 permit
/etc/postfix/main.cf:
# Disable the teaser banner (try whitelisting first if you can).
postscreen_greet_banner =
When an SMTP client sends a command before the postscreen_greet_wait time has elapsed, postscreen(8) logs this as:
PREGREET count after time from [address]:port text...
Translation: the client at [address]:port sent count bytes before its turn to speak. This happened time seconds after the postscreen_greet_wait timer was started. The text is what the client sent (truncated to 100 bytes, and with non-printable characters replaced with C-style escapes such as \r for carriage-return and \n for newline).
The postscreen_greet_action parameter specifies the action that is taken next. See "When tests fail before the 220 SMTP server greeting" below.
The postscreen_dnsbl_sites parameter (default: empty) specifies a list of DNS blocklist servers with optional filters and weight factors (positive weights for blacklisting, negative for whitelisting). These servers will be queried in parallel with the reverse client IP address. This test is disabled by default.
CAUTION: when postscreen rejects mail, its SMTP reply contains the DNSBL domain name. Use the postscreen_dnsbl_reply_map feature to hide "password" information in DNSBL domain names.
When the postscreen_greet_wait time has elapsed, and the combined DNSBL score is equal to or greater than the postscreen_dnsbl_threshold parameter value, postscreen(8) logs this as:
DNSBL rank count for [address]:port
Translation: the SMTP client at [address]:port has a combined DNSBL score of count.
The postscreen_dnsbl_action parameter specifies the action that is taken when the combined DNSBL score is equal to or greater than the threshold. See "When tests fail before the 220 SMTP server greeting" below.
When the client address matches the permanent blacklist, or when the client fails the pregreet or DNSBL tests, the action is specified with postscreen_blacklist_action, postscreen_greet_action, or postscreen_dnsbl_action, respectively.
In this phase of the protocol, postscreen(8) implements a number of "deep protocol" tests. These tests use an SMTP protocol engine that is built into the postscreen(8) server.
Important note: deep protocol tests are disabled by default. They are more intrusive than the pregreet and DNSBL tests, and they have limitations as discussed next.
When a good client passes the deep protocol tests, postscreen(8) adds the client to the temporary whitelist but it cannot hand off the "live" connection to a Postfix SMTP server process in the middle of the session. Instead, postscreen(8) defers mail delivery attempts with a 4XX status, logs the helo/sender/recipient information, and waits for the client to disconnect.
The next time the client connects it will be allowed to talk to a Postfix SMTP server process to deliver its mail. To minimize the impact of this limitation, postscreen(8) gives deep protocol tests a relatively long expiration time.
postscreen(8)'s built-in SMTP engine does not implement the AUTH, XCLIENT, and XFORWARD features. AUTH support may be added in a future version. In the mean time, if you need to make these services available on port 25, then do not enable the tests after the 220 server greeting.
End-user clients should connect directly to the submission service, so that they never have to deal with postscreen(8)'s tests.
By default, SMTP is a half-duplex protocol: the sender and receiver send one command and one response at a time. Unlike the Postfix SMTP server, postscreen(8) does not announce support for ESMTP command pipelining. Therefore, clients are not allowed to send multiple commands. postscreen(8)'s deep protocol test for this is disabled by default.
With "postscreen_pipelining_enable = yes", postscreen(8) detects zombies that send multiple commands, instead of sending one command and waiting for the server to reply.
This test is opportunistically enabled when postscreen(8) has to use the built-in SMTP engine anyway. This is to make postscreen(8) logging more informative.
When a client sends multiple commands, postscreen(8) logs this as:
COMMAND PIPELINING from [address]:port after command: text
Translation: the SMTP client at [address]:port sent multiple SMTP commands, instead of sending one command and then waiting for the server to reply. This happened after the client sent command. The text shows part of the input that was sent too early; it is not logged with Postfix 2.8.
The postscreen_pipelining_action parameter specifies the action that is taken next. See "When tests fail after the 220 SMTP server greeting" below.
Some spambots send their mail through open proxies. A symptom of this is the usage of commands such as CONNECT and other non-SMTP commands. Just like the Postfix SMTP server's smtpd_forbidden_commands feature, postscreen(8) has an equivalent postscreen_forbidden_commands feature to block these clients. postscreen(8)'s deep protocol test for this is disabled by default.
With "postscreen_non_smtp_command_enable = yes", postscreen(8) detects zombies that send commands specified with the postscreen_forbidden_commands parameter. This also detects commands with the syntax of a message header label. The latter is a symptom that the client is sending message content after ignoring all the responses from postscreen(8) that reject mail.
This test is opportunistically enabled when postscreen(8) has to use the built-in SMTP engine anyway. This is to make postscreen(8) logging more informative.
When a client sends non-SMTP commands, postscreen(8) logs this as:
NON-SMTP COMMAND from [address]:port command
Translation: the SMTP client at [address]:port sent a command that matches the postscreen_forbidden_commands parameter, or that has the syntax of a message header label.
The postscreen_non_smtp_command_action parameter specifies the action that is taken next. See "When tests fail after the 220 SMTP server greeting" below.
SMTP is a line-oriented protocol: lines have a limited length, and are terminated with <CR><LF>. Lines ending in a "bare" <LF>, that is newline not preceded by carriage return, are not allowed in SMTP. postscreen(8)'s deep protocol test for this is disabled by default.
With "postscreen_bare_newline_enable = yes", postscreen(8) detects clients that send lines ending in bare newline characters.
This test is opportunistically enabled when postscreen(8) has to use the built-in SMTP engine anyway. This is to make postscreen(8) logging more informative.
When a client sends bare newline characters, postscreen(8) logs this as:
BARE NEWLINE from [address]:port
Translation: the SMTP client at [address]:port sent a bare newline character, that is newline not preceded by carriage return.
The postscreen_bare_newline_action parameter specifies the action that is taken next. See "When tests fail after the 220 SMTP server greeting" below.
When the client fails the pipelining, non-SMTP command or bare newline tests, the action is specified with postscreen_pipelining_action, postscreen_non_smtp_command_action or postscreen_bare_newline_action, respectively.
When an SMTP client hangs up unexpectedly during any tests, postscreen(8) logs this as:
HANGUP after time from [address]:port in test name
Translation: the SMTP client at [address]:port disconnected unexpectedly, time seconds after the start of the test named test name.
The following errors are reported by the built-in SMTP engine. This engine never accepts mail, therefore it has per-session limits on the number of commands and on the session length.
COMMAND TIME LIMIT from [address]:port
Translation: the SMTP client at [address]:port reached the per-command time limit as specified with the postscreen_command_time_limit parameter. The session is terminated immediately.
COMMAND COUNT LIMIT from [address]:port
Translation: the SMTP client at [address]:port reached the per-session command count limit as specified with the postscreen_command_count_limit parameter. The session is terminated immediately.
COMMAND LENGTH LIMIT from [address]:port
Translation: the SMTP client at [address]:port reached the per-command length limit, as specified with the line_length_limit parameter. The session is terminated immediately.
When an SMTP client makes too many connections at the same time, or when all postscreen(8) ports are busy, postscreen(8) rejects the connection with a 421 status code and logs:
NOQUEUE: reject: CONNECT from [address]:port: too many connections
NOQUEUE: reject: CONNECT from [address]:port: all server ports busy
The postscreen_client_connection_count_limit and postscreen_pre_queue_limit parameters control these limits.
When a new SMTP client passes all tests (i.e. it is not whitelisted via some mechanism), postscreen(8) logs this as:
PASS NEW [address]:port
Where [address]:port are the client IP address and port. Then, postscreen(8) creates a temporary whitelist entry that excludes the client IP address from further tests until the temporary whitelist entry expires, as controlled with the postscreen_*_ttl parameters.
When no "deep protocol tests" are configured, postscreen(8) hands off the "live" connection to a Postfix SMTP server process. The client can then continue as if postscreen(8) never even existed (except for the short postscreen_greet_wait delay).
When any "deep protocol tests" are configured, postscreen(8) cannot hand off the "live" connection to a Postfix SMTP server process in the middle of the session. Instead, postscreen(8) defers mail delivery attempts with a 4XX status, logs the helo/sender/recipient information, and waits for the client to disconnect. The next time the client connects it will be allowed to talk to a Postfix SMTP server process to deliver its mail. postscreen(8) mitigates the impact of this limitation by giving deep protocol tests a long expiration time.
postscreen(8) has been tested on FreeBSD [4-8], Linux 2.[4-6] and Solaris 9 systems.
To enable the postscreen(8) service and log client information without blocking mail:
Make sure that local clients and systems with non-standard SMTP implementations are excluded from any postscreen(8) tests. The default is to exclude all clients in mynetworks. To exclude additional clients, for example, third-party performance monitoring tools (these tend to have broken SMTP implementations):
/etc/postfix/main.cf: # Exclude broken clients by whitelisting. Clients in mynetworks # should always be whitelisted. postscreen_access_list = permit_mynetworks, cidr:/etc/postfix/postscreen_access.cidr /etc/postfix/postscreen_access.cidr: 192.168.254.0/24 permit
Comment out the "smtp inet ... smtpd" service in master.cf, including any "-o parameter=value" entries that follow.
/etc/postfix/master.cf: #smtp inet n - n - - smtpd # -o parameter=value ...
Uncomment the new "smtpd pass ... smtpd" service in master.cf, and duplicate any "-o parameter=value" entries from the smtpd service that was commented out in the previous step.
/etc/postfix/master.cf: smtpd pass - - n - - smtpd -o parameter=value ...
Uncomment the new "smtp inet ... postscreen" service in master.cf.
/etc/postfix/master.cf: smtp inet n - n - 1 postscreen
Uncomment the new "tlsproxy unix ... tlsproxy" service in master.cf. This service implements STARTTLS support for postscreen(8).
/etc/postfix/master.cf: tlsproxy unix - - n - 0 tlsproxy
Uncomment the new "dnsblog unix ... dnsblog" service in master.cf. This service does DNSBL lookups for postscreen(8) and logs results.
/etc/postfix/master.cf: dnsblog unix - - n - 0 dnsblog
To enable DNSBL lookups, list some DNS blocklist sites in main.cf, separated by whitespace. Different sites can have different weights. For example:
/etc/postfix/main.cf: postscreen_dnsbl_threshold = 2 postscreen_dnsbl_sites = zen.spamhaus.org*2 bl.spamcop.net*1 b.barracudacentral.org*1
Note: if your DNSBL queries have a "secret" in the domain name, you must censor this information from the postscreen(8) SMTP replies. For example:
/etc/postfix/main.cf: postscreen_dnsbl_reply_map = texthash:/etc/postfix/dnsbl_reply
/etc/postfix/dnsbl_reply:
# Secret DNSBL name Name in postscreen(8) replies
secret.zen.spamhaus.org zen.spamhaus.org
The texthash: format is similar to hash: except that there is no need to run postmap(1) before the file can be used, and that it does not detect changes after the file is read. It is new with Postfix version 2.8.
Read the new configuration with "postfix reload".
Notes:
Some postscreen(8) configuration parameters implement stress-dependent behavior. This is supported only when the default value is stress-dependent (that is, "postconf -d parametername" output shows "parametername = ${stress?something}${stress:something}"). Other parameters always evaluate as if the stress value is the empty string.
See "Tests before the 220 SMTP server greeting" for details about the logging from these postscreen(8) tests.
If you run Postfix 2.6 or earlier you must stop and start the master daemon ("postfix stop; postfix start"). This is needed because the Postfix "pass" master service type did not work reliably on all systems.
postscreen(8) TLS support is available for remote SMTP clients that aren't whitelisted, including clients that need to renew their temporary whitelist status. When a remote SMTP client requests TLS service, postscreen(8) invisibly hands off the connection to a tlsproxy(8) process. Then, tlsproxy(8) encrypts and decrypts the traffic between postscreen(8) and the remote SMTP client. One tlsproxy(8) process can handle multiple SMTP sessions. The number of tlsproxy(8) processes slowly increases with server load, but it should always be much smaller than the number of postscreen(8) TLS sessions.
TLS support for postscreen(8) and tlsproxy(8) uses the same parameters as with smtpd(8). We recommend that you keep the relevant configuration parameters in main.cf. If you must specify "-o smtpd_mumble=value" parameter overrides in master.cf for a postscreen-protected smtpd(8) service, then you should specify those same parameter overrides for the postscreen(8) and tlsproxy(8) services.
For compatibility with smtpd(8), postscreen(8) implements the soft_bounce safety feature. This causes Postfix to reject mail with a "try again" reply code.
To turn this on for all of Postfix, specify "soft_bounce = yes" in main.cf.
To turn this on for postscreen(8) only, append "-o soft_bounce=yes" (note: NO SPACES around '=') to the postscreen entry in master.cf.
Execute "postfix reload" to make the change effective.
After testing, do not forget to remove the soft_bounce feature, otherwise senders won't receive their non-delivery notification until many days later.
To use the postscreen(8) service to block mail, edit main.cf and specify one or more of:
"postscreen_dnsbl_action = enforce", to reject clients that are on DNS blocklists, and to log the helo/sender/recipient information. With good DNSBLs this reduces the amount of load on Postfix SMTP servers dramatically.
"postscreen_greet_action = enforce", to reject clients that talk before their turn, and to log the helo/sender/recipient information. This stops over half of all known-to-be illegitimate connections to Wietse's mail server. It is backup protection for zombies that haven't yet been blacklisted.
You can also enable "deep protocol tests", but these are more intrusive than the pregreet or DNSBL tests.
When a good client passes the "deep protocol tests", postscreen(8) adds the client to the temporary whitelist but it cannot hand off the "live" connection to a Postfix SMTP server process in the middle of the session. Instead, postscreen(8) defers mail delivery attempts with a 4XX status, logs the helo/sender/recipient information, and waits for the client to disconnect.
When the good client comes back in a later session, it is allowed to talk directly to a Postfix SMTP server. See "after_220 Tests after the 220 SMTP server greeting above for limitations with AUTH and other features that clients may need.
An unexpected benefit from "deep protocol tests" is that some "good" clients don't return after the 4XX reply; these clients were not so good after all. Wietse enables "deep protocol tests" on his own internet-facing mail server.
There is also support for permanent blacklisting and whitelisting; see the description of the postscreen_access_list parameter for details.
To turn off postscreen(8) and handle mail directly with Postfix SMTP server processes:
Comment out the "smtp inet ... postscreen" service in master.cf, including any "-o parameter=value" entries that follow.
/etc/postfix/master.cf: #smtp inet n - n - 1 postscreen # -o parameter=value ...
Comment out the "dnsblog unix ... dnsblog" service in master.cf.
/etc/postfix/master.cf: #dnsblog unix - - n - 0 dnsblog
Comment out the "smtpd pass ... smtpd" service in master.cf, including any "-o parameter=value" entries that follow.
/etc/postfix/master.cf: #smtpd pass - - n - - smtpd # -o parameter=value ...
Comment out the "tlsproxy unix ... tlsproxy" service in master.cf, including any "-o parameter=value" entries that follow.
/etc/postfix/master.cf: #tlsproxy unix - - n - 0 tlsproxy # -o parameter=value ...
Uncomment the "smtp inet ... smtpd" service in master.cf, including any "-o parameter=value" entries that may follow.
/etc/postfix/master.cf: smtp inet n - n - - smtpd -o parameter=value ...
Read the new configuration with "postfix reload".
Many ideas in postscreen(8) were explored in earlier work by Michael Tokarev, in OpenBSD spamd, and in MailChannels Traffic Control.
Wietse threw together a crude prototype with pregreet and dnsbl support in June 2009, because he needed something new for a Mailserver conference presentation in July. Ralf Hildebrandt ran this code on several servers to collect real-world statistics. This version used the dnsblog(8) ad-hoc DNS client program.
Wietse needed new material for a LISA conference presentation in November 2010, so he added support for DNSBL weights and filters in August, followed by a major code rewrite, deep protocol tests, helo/sender/recipient logging, and stress-adaptive behavior in September. Ralf Hildebrandt ran this code on several servers to collect real-world statistics. This version still used the embarrassing dnsblog(8) ad-hoc DNS client program.
Wietse added STARTTLS support in December 2010. This makes postscreen(8) usable for sites that require TLS support. The implementation introduces the tlsproxy(8) event-driven TLS proxy that decrypts/encrypts the sessions for multiple SMTP clients.