Table of Contents
Filtering out spam with Postfix
The following article is the result of many years of tinkering with the Postfix mail server; I first started in 2007 with a mini-ITX that I ran at home and which I had to shut down, when there were strong thunderstorms. Now, since 2011 I have a virtual server with Rackspace and I have to say that, for my humble needs, it runs quite well.
Hopefully I wrote some useful tips that can help others who think of implementing it or want to improve their spam blocking system configuration. I'd like to stress that this is not an introductory guide to Postfix and that these are just notes that I found useful to write down in order to better understand some configuration bits and have a place where I can easily recall them. Usually spam guides in Postfix teach you how to implement Amavis and Spamassassin; this is not the purpose of this document.
I found that Amavis/Spamassassin are very resource intensive, so I studied how to implement a spam blocking system by simple fine tuning the Postfix configuration and by the implementation of SMTP RFC's and standards.
My mail server is authoritative for two domains, which here I'll name example.com and example.net.
Comparing a regular letter to an email
The following is an image taken from “The book of Postfix” (R.Hildebrandt and P.Koetter - No Starch Press), which I highly recommend:
It explains a bit of the terminology that is used in MTA's (Mail Transport Agents); in particular it divides a message (a mail) into its constituent parts: envelope, headers, body and attachments, each of which is dealt with by different directives in the configuration files.
High-level overview of an incoming SMTP connection
The following are some logs that describe how an incoming connection is treated by my mail server.
Incoming SMTP Connection
This one is the first step in my realm!
Aug 20 10:35:26 robhost2 postfix/smtpd[22980]: connect from mx164.201.mndsender.com[46.29.201.164]
SPF
SPF checks if what is specified in MAIL FROM: complies with the server that the mail is coming from.
Aug 20 10:35:26 robhost2 postfix/policy-spf[22986]: Policy action=PREPEND Received-SPF: pass (mn1.mndsender.com: Sender is authorized to use 'bounce-back.mnidm.3103.38454.51.689828439._@mn1.mndsender.com' in 'mfrom' identity (mechanism 'include:spf.mndsender.com' matched)) receiver=example.com; identity=mailfrom; envelope-from="bounce-back.mnidm.3103.38454.51.689828439._@mn1.mndsender.com"; helo=mx164.201.mndsender.com; client-ip=46.29.201.164
Whitelisting
Sometimes I need to whitelist a sender address or an entire domain. In this case the following checks are skipped.
Sanity Checks
All incoming mails must be RFC compliant.
Aug 21 01:04:50 robhost2 postfix/smtpd[25569]: NOQUEUE: reject: RCPT from unknown[186.116.8.2]: 450 4.7.1 Client host rejected: cannot find your reverse hostname, [186.116.8.2]; from=<pijman@admcanada.com> to=<last.fm@example.net> proto=ESMTP helo=<admcanada-com.mail.protection.outlook.com> ... Aug 21 09:58:39 robhost2 postfix/smtpd[15192]: NOQUEUE: reject: RCPT from cloud.rexmedia.nl[144.76.173.201]: 450 4.1.8 <noreplay@legambientemonza.org>: Sender address rejected: Domain not found; from=<noreplay@legambientemonza.org> to=<dropbox@example.net> proto=SMTP helo=<cloud.rexmedia.nl> ... Aug 23 07:30:33 robhost2 postfix/smtpd[5964]: NOQUEUE: reject: RCPT from xvm70328.vps.cloud.tagadab.com[185.77.82.130]: 504 5.5.2 <xvm70328>: Helo command rejected: need fully-qualified hostname; from=<whate@messaggi.com> to=<tuttoferramenta@example.net> proto=ESMTP helo=<xvm70328> ...
DNSBL
Stands for DNS Blocking Lists; they are free databases that are populated with IP's of known spammers and that are checked against in order to look for the reputation of a sender.
Aug 21 11:07:45 robhost2 postfix/smtpd[18147]: NOQUEUE: reject: RCPT from unknown[139.193.133.244]: 554 5.7.1 Service unavailable; Client host [139.193.133.244] blocked using zen.spamhaus.org; https://www.spamhaus.org/sbl/query/SBLCSS / https://www.spamhaus.org/query/ip/139.193.133.244; from=<Helga5555@example.net> to=<lav@example.net> proto=ESMTP helo=<fm-dyn-139-193-133-244.fast.net.id>
Postgrey
Temporary delay for new messages, in order to block hasty spam server.
Aug 20 10:35:26 robhost2 postgrey[1303]: action=pass, reason=triplet found, delay=301, client_name=mx164.201.mndsender.com, client_address=46.29.201.164, sender=bounce-back.mnidm.3103.38454.51.689828439._@mn1.mndsender.com, recipient=info@example.com Aug 20 10:35:27 robhost2 postfix/smtpd[22980]: 06383AE90B: client=mx164.201.mndsender.com[46.29.201.164] Aug 20 10:35:27 robhost2 postfix/cleanup[22987]: 06383AE90B: message-id=<mnidm.3103.38454.51.689828439._@mn1.mndsender.com>
DKIM
Additional header that can help in marking incoming (and outgoing) mails as legitimate.
Aug 20 10:35:27 robhost2 opendkim[1328]: 06383AE90B: mx164.201.mndsender.com [46.29.201.164] not internal Aug 20 10:35:27 robhost2 opendkim[1328]: 06383AE90B: not authenticated Aug 20 10:35:27 robhost2 opendkim[1328]: 06383AE90B: s=20150717dmd d=mndsender.com SSL
Delivery
Well, after all the incoming mail seems to be legit: do the local delivery.
Aug 20 10:35:27 robhost2 postfix/qmgr[9868]: 06383AE90B: from=<bounce-back.mnidm.3103.38454.51.689828439._@mn1.mndsender.com>, size=38558, nrcpt=1 (queue active) Aug 20 10:35:27 robhost2 postfix/virtual[22988]: 06383AE90B: to=<rob@example.net>, orig_to=<info@example.com>, relay=virtual, delay=1.6, delays=1.5/0.02/0/0.01, dsn=2.0.0, status=sent (delivered to maildir) Aug 20 10:35:27 robhost2 postfix/qmgr[9868]: 06383AE90B: removed
End of SMTP connection
And terminate the SMTP connection.
Aug 20 10:35:27 robhost2 postfix/smtpd[22980]: disconnect from mx164.201.mndsender.com[46.29.201.164]
main.cf configuration
The connection above is a consequence of how I configured
/etc/postfix/main.cf
on my mail server:
HELO command is required at the beginning of the SMTP connection:
smtpd_helo_required = yes
Then comes the smtpd_recipient_restrictions:
smtpd_recipient_restrictions =
permit_mynetworks reject_sender_login_mismatch <---- SASL login permit_sasl_authenticated <---- SASL login reject_unauth_destination <---- the server is not an Open Relay: important! check_policy_service unix:private/policy-spf <---- SPF reject_non_fqdn_recipient <---- sanity check reject_non_fqdn_sender <---- sanity check reject_unknown_sender_domain <---- sanity check reject_unknown_recipient_domain <---- sanity check check_recipient_access hash:/etc/postfix/roleaccount_exceptions <---- postmaster@, abuse@: always accept check_client_access hash:/etc/postfix/client_access <---- useless? check_sender_access hash:/etc/postfix/sender_access <---- whitelisting reject_non_fqdn_hostname <---- sanity check reject_invalid_hostname <---- sanity check check_helo_access pcre:/etc/postfix/helo_checks <---- don't use my hostname or IP check_sender_mx_access cidr:/etc/postfix/bogus_mx <---- exclude messages coming from RFC1918 Internal Lans: ex. 192.168..., 10.... reject_unknown_reverse_client_hostname <---- MX hostname not configured in reverse lookup zone reject_rbl_client zen.spamhaus.org <---- DNSBL and RHSBL check_policy_service inet:127.0.0.1:10023 <---- Postgrey permit <---- end of smtpd_recipient_restrictions: finally!
Now, there are various stages where smtpd * restrictions could take place and these stages correspond to the different moments in a SMTP connection:
While it could make sense to insert restrictions only in the corresponding stage, that is restrictions regarding HELO stage in stmpd_helo_restrictions, restrictions regarding sender stage in smtpd_sender_restrictions and so on, the best practice is to wait until all the stages have occurred, during the reception of a message, and place all restrictions in smtpd_recipient_restrictions.
In this way the complexity of the configuration is reduced and we will collect more data about the incoming message. Furthermore, some mail clients keep on sending the message if the server rejects it too early, because their implementation mandates that they be able to send the command of inputing at least one recipient.
So I placed all the restrictions under smtpd_recipient_restrictions.
Incoming SMTPD connection
Let's start from the beginning!
The incoming SMTP connection enters my mail server via SMTP daemon (TCP port 25):
postfix/smtpd[7956]: connect from mx1.phx.paypal.com[66.211.168.231]
smtpd_recipient_restrictions = permit_mynetworks
With the above settings local networks aren't subject to smtpd restrictions.
Then comes the first SPF settings, which requires further explanation:
check_policy_service unix:private/policy-spf:
SPF - Sender Policy Framework
The first real check is against SPF (Sender Policy Framework); in the following case the mail is accepted.
postfix/policy-spf[7961]: Policy action=PREPEND Received-SPF: pass (paypal.it: Sender is authorized to use 'assistenza@paypal.it' in 'mfrom' identity (mechanism 'include:pp._spf.paypal.com' matched)) receiver=example.com;
The example above is explained below.
What SPF does
“The Sender Policy Framework (SPF) is an open standard specifying a technical method to prevent sender address forgery.
More precisely, the current version of SPF — called SPFv1 or SPF Classic — protects the envelope sender address, which is used for the delivery of messages.
The technology requires two sides to play together:
(1) the domain owner publishes this information in an SPF record in the domain's DNS zone, and when someone else's mail server receives a message claiming to come from that domain, then
(2) the receiving server can check whether the message complies with the domain's stated policy.
If, e.g., the message comes from an unknown server, it can be considered a fake.”
SPF configuration in DNS
$ cat /etc/bind/db.example.com ... ; SPF record @ IN TXT "v=spf1 a mx ~all"
The SPF record mechanisms explained:
v=spf1 SPF version 1 a Allow current IP address of the domain to send email for this domain mx Allow servers listed as MX to send email for this domain ~all SoftFail (not compliant will be accepted but marked)
Mechanisms can be prefixed with one of four qualifiers:
"+" Pass "-" Fail "~" SoftFail <----------- that's the mechanism I chose "?" Neutral
When in the description it says it protects the envelope sender address it means that SPF acts during the SMTP connection between the sender and receiving mail servers. In the case above an SMPT connection was initiated by a host with a Mail From: that specified paypal.it as sender SMTP server.
SPF on my mail server did a DNS query to look for a SPF record in the paypal.it zone; the query is similar to this one, executed with the program dig:
$ dig TXT paypal.it
;; ANSWER SECTION: paypal.it. 3600 IN TXT "v=spf1 mx include:... ~all"
As it can be seen, the zone paypal.it has got a TXT record which includes a MX, which, as we saw above, means that only servers listed as MX can send email for this domain. So with another DNS query, this time of MX type, SPF can check whether the transmitting mail server is in the list of MX servers allowed to send mail for paypal.it.
The point here is that the TXT record in the paypal.it is set to SoftFail (mechanism '~', before 'all'), so even if a sender fraudulently uses paypal.it in its MAIL FROM: envelope, SPF on my logs will mark it, but the message will be let passed nonetheless. like in the next example.
SPF softfail example
Here the sender SMTP server 212.237.35.109 sent me an email setting bidoo.com in MAIL FROM:, but it wasn't listed in the bidoo.com DNS zone as authorized. But, as the message states, the domain is using mechanism '~all', so it's simply telling the receiving SPF clients that the sender could be unreliable.
Jan 3 00:42:32 robhost2 postfix/policy-spf[21043]: Policy action=PREPEND Received-SPF: softfail (bidoo.com: Sender is not authorized by default to use 'newsletter@bidoo.com' in 'mfrom' identity, however domain is not currently prepared for false failures (mechanism '~all' matched)) receiver=example.com; identity=mailfrom; envelope-from="newsletter@bidoo.com"; helo=bidoo.com; client-ip=212.237.35.109
SPF reject example
Here we can see an example of a SPF reject; I received a mail from someone claiming to have a mailbox in @skyware.pl domain.
Oct 5 10:22:50 robhost2 postfix/smtpd[13970]: NOQUEUE: reject: RCPT from ip-91.246.67.91.skyware.pl[91.246.67.91]: 550 5.7.1 <dropbox@example.net>: Recipient address rejected: Please see http://www.openspf.net/Why ?s=mfrom;id=Adams.512%40skyware.pl;ip=91.246.67.91;r=example.com; from=<Adams.512@skyware.pl> to=<dropbox@example.net> proto=ESMTP helo=<ip-91.246.67.91.skyware.pl> Oct 5 10:22:50 robhost2 postfix/smtpd[13970]: disconnect from ip-91.246.67.91.skyware.pl[91.246.67.91]
As it can be seen, the mail has been rejected.
When clicking on the link http://www.openspf.net/Why?…, it gives the following explanation:
What SPF on my server did was:
- looking for a TXT/SPF record in that domain
- finding a SPF record
- telling my mail server that noone could send mail for @skyware.pl from the host ip-91.246.67.91.skyware.pl (91.246.67.91).
In fact, SPF in skyware.pl is configured to fail (prefix “-” before “all” means “Fail”), not simply soft fail as in paypal.it:
skyware.pl. 38400 IN TXT "v=spf1 mx ip4:91.189.223.154 a:mx1.skyware.pl -all"
So any SPF client will discard any message claiming to come from skyware.pl if the sending mail server is not one the allowed ones.
Whitelisting with sender_access
smtpd_recipient_restrictions = ... check_sender_access hash:/etc/postfix/sender_access
If a sender is specified as OK in
/etc/postfix/sender_access
it is effectively whitelisted and so the connection doesn't go through further checks (i.e. greylisting); we can whitelist specific addresses or entire domains:
$ cat /etc/postfix/sender_access # Restricts sender addresses this system accepts in MAIL FROM commands.
noreply@openpolis.it OK open.ac.uk OK
Example of a connection from a whitelisted domain (open.ac.uk):
Sep 28 12:20:30 robhost2 postfix/smtpd[17396]: connect from mail-ve1eur01on0054.outbound.protection.outlook.com[104.47.1.54]
SPF is checked, because it's higher in the smtpd_recipient_restrictions settings:
Sep 28 12:20:31 robhost2 postfix/policy-spf[17439]: Policy action=PREPEND Received-SPF: pass (open.ac.uk: Sender is authorized to use 'tutor@open.ac.uk' in 'mfrom' identity (mechanism 'include:spf.protection.outlook.com' matched)) receiver=example.com; identity=mailfrom; envelope-from="tutor@open.ac.uk"; helo=EUR01-VE1-obe.outbound.protection.outlook.com; client-ip=104.47.1.54
Just after SPF, Postgrey should be checked against next here, but since the domain is whitelisted, this check is skipped
Sep 28 12:20:31 robhost2 postfix/smtpd[17396]: 611EF11216: client=mail-ve1eur01on0054.outbound.protection.outlook.com[104.47.1.54] Sep 28 12:20:31 robhost2 postfix/cleanup[17440]: 611EF11216: message-id=<E1bpDqe-0004JF-0s@ariel.open.ac.uk>
DKIM is checked as usual:
Sep 28 12:20:31 robhost2 opendkim[1182]: 611EF11216: mail-ve1eur01on0054.outbound.protection.outlook.com [104.47.1.54] not internal Sep 28 12:20:31 robhost2 opendkim[1182]: 611EF11216: not authenticated Sep 28 12:20:31 robhost2 opendkim[1182]: 611EF11216: failed to parse Authentication-Results: header field Sep 28 12:20:31 robhost2 opendkim[1182]: 611EF11216: s=selector1-open-ac-uk d=openuniv.onmicrosoft.com SSL
The message is eventually delivered:
Sep 28 12:20:32 robhost2 postfix/qmgr[1318]: 611EF11216: from=<tutor@open.ac.uk>, size=12411, nrcpt=1 (queue active) Sep 28 12:20:32 robhost2 postfix/virtual[17441]: 611EF11216: to=<rob@example.net>, orig_to=<info@example.com>, relay=virtual, delay=1.1, delays=1/0.02/0/0.03, dsn=2.0.0, status=sent (delivered to maildir) Sep 28 12:20:32 robhost2 postfix/smtpd[17396]: disconnect from mail-ve1eur01on0054.outbound.protection.outlook.com[104.47.1.54] Sep 28 12:20:32 robhost2 postfix/qmgr[1318]: 611EF11216: removed
The following checks are skipped, thanks to whitelisting:
reject_non_fqdn_hostname reject_invalid_hostname check_helo_access pcre:/etc/postfix/helo_checks check_sender_mx_access cidr:/etc/postfix/bogus_mx reject_unknown_reverse_client_hostname # DNSBL e RHSBL reject_rbl_client zen.spamhaus.org # Postgrey check_policy_service inet:127.0.0.1:10023 permit
Some sanity checks
RFC requirement, HELO is required:
# RFC 821 and 2821: hostname required smtpd_helo_required = yes
Sender and recipient must also be FQDN, as in RFC's:
smtpd_recipient_restrictions = ... reject_non_fqdn_recipient reject_non_fqdn_sender
Also advertised hostname after HELO command must be FQDN:
... reject_non_fqdn_hostname
Issue with telecomitalia.it
Telecom Italia is the main ISP here in Italy and, unfortunately, sometimes its SMTP servers are misconfigured.
In fact, some of their mail servers don't advertise themselves with a FQDN, and they would be blocked, like in this case:
Sep 3 05:42:33 robhost2 postfix/smtpd[23786]: connect from host204-78-static.94-94-b.business.telecomitalia.it[94.94.78.204] Sep 3 05:42:34 robhost2 postfix/policy-spf[23791]: Policy action=PREPEND Received-SPF: none (xxx2000.com: No applicable sender policy available) receiver=example.com; identity=mailfrom; envelope-from="biglietteria@xxx2000.com"; helo=SRV-W1; client-ip=94.94.78.204 Sep 3 05:42:35 robhost2 postfix/smtpd[23786]: NOQUEUE: reject: RCPT from host204-78-static.94-94-b.business.telecomitalia.it[94.94.78.204]: 504 5.5.2 <SRV-W1>: Helo command rejected: need fully-qualified hostname; from=<biglietteria@xxx2000.com> to=<rob@example.net> proto=ESMTP helo=<SRV-W1>
If these mails need to pass, I suggest that the above setting reject_non_fqdn_hostname be removed.
Mail from: sanity check
reject_unknown_sender_domain
The MAIL FROM domain has no DNS MX or malformed MX and no DNS A record.
Rcpt to: sanity check
reject_unknown_recipient_domain
When Postfix is not final destination for the recipient domain or the RCPT TO domain has no DNS MX or malformed MX and no DNS A record.
Exceptions:
smtpd_recipient_restrictions = ... check_recipient_access hash:/etc/postfix/roleaccount_exceptions
$ cat /etc/postfix/roleaccount_exceptions
# Always accept mail to these recipients, # independently from recipient restrictions postmaster@ OK abuse@ OK
Client access. (Useless??)
This setting seems not to be working. From what I've read, I just need to type IP's here:
smtpd_recipient_restrictions = ... check_client_access hash:/etc/postfix/client_access
$ cat /etc/postfix/client_access # Restricts which clients this system accepts SMTP connections from.
125.89.61.122 OK 156.54.132.89 OK
check_helo_access
check_helo_access pcre:/etc/postfix/helo_checks
$ cat /etc/postfix/helo_checks /^mail\.robertocarraro\.com$/ 550 Don't use my hostname /^31\.222\.165\.32$/ 550 Don't use my IP address /^\[31\.222\.165\.32\]$/ 550 Don't use my IP address
reject_unknown_reverse_client_hostname
This setting turned out to be super important in my configuration to block tons of spam (see graph at the bottom of this page).
Discard mail from unknown hostnames; more Precisely (from Wikipedia):
Forward-confirmed reverse DNS
(1) First a reverse DNS lookup (PTR query) is performed on the IP address, which returns a list of zero or more PTR records.
(2) For each domain name returned in the (1) PTR query results, a regular 'forward' DNS lookup (type A or AAAA query) is then performed on that domain name.
(3) Any A or AAAA record returned by the second query (2) is then compared against the original IP address, and if there is a match, then the FCrDNS (Forward-confirmed reverse DNS) check passes.
Example:
(1) DNS query type PTR on 192.0.2.4 –> returns PTR-record=“hostname.example.com” (1 result)
(2) DNS query type A on “hostname.example.com” –> returns A-record=192.0.2.4 (1 result)
(3) Matches original IP address, therefore check passes
I commented this configuration, because many legitimate mails were blocked:
# reject_unknown_client_hostname
And I used this configuration, which only does the first step (1: reverse DNS lookup query):
reject_unknown_reverse_client_hostname
For a not correctly configured SMTP server I see in the logs:
Aug 20 09:50:48 robhost2 postfix/smtpd[21144]: NOQUEUE: reject: RCPT from unknown[221.203.169.50]: 450 4.7.1 Client host rejected: cannot find your reverse hostname, [221.203.169.50]; from=<avmbdiev@rofe.com> to=<last.fm@example.net> proto=ESMTP helo=<rofe.com>
Infact, if I do a manual reverse lookup for the IP of the SMTP, by using dig, I don't get any answer (ANSWER: 0):
$ dig -x 221.203.169.50 ; <<>> DiG 9.9.5-3ubuntu0.15-Ubuntu <<>> -x 221.203.169.50 ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 58381 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1 ...
On Logwatch logs appears here:
2 4xx Reject unknown reverse client host -------------------------------------------------- ... 1 221.203.169.50
As said, this setting alone truncates a lot of spam!
DKIM
DomainKeys Identified Mail (DKIM) lets an organization take responsibility for a message while it is in transit.
With DKIM enabled, the sender's MTA signs every outgoing message with a private key. The recipient retrieves the public key from the sender's DNS records and verifies if the message body and some of the header fields were not altered since the message signing took place.
DKIM configuration in main.cf
... # DKIM milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 ...
From the Postfix documentation:
Milter error handling
The milter_default_action parameter specifies how Postfix handles Milter application errors.
The default action is to respond with a temporary error status, so that the client will try again later.
Specify accept if you want to receive mail as if the filter does not exist, and reject to reject mail with a permanent status.
The quarantine action is like accept but freezes the message in the hold queue.
So, in my configuration above, I decided to accept mails with DKIM Milter errors and not to block them. More on that below.
DKIM configuration in DNS
In the DNS zone I have placed the public key:
... ; DKIM record example.com._domainkey IN TXT "v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCsntpU8hiRIw815EkliPtaa2Jqe2NFQasMxtXsqdtT5xC8oqKwwP6PlZe2oePQKSWcumHxEZ+GHxKbzFhG0wsEx9thqQ9TrgWc7WRUeSMe43if2Y2hI2A8XxMK8oh7t4kyivoD6mHubpubQUmGxeMJBkB+M9LVVVCuAai+9eS1vwIDAQAB" ; ----- DKIM key example.com for example.com
OpenDKIM milter (mail filter) seems to insert after the Postfix cleanup process and before the incoming queue:
It therefore appears as if OpenDKIM is a non-SMTP filter:
DKIM headers in mails
Whenever a recipient receives mails signed with DKIM this is what appears in the mails' headers:
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; s=mt; d=pmta.sailthru.com; h=Date:From:Reply-To:To:Message-ID:Subject:MIME-Version:Content-Type; bh=zgnGhasPyWqkPy1BdhF9TRe4Pls=; b=bB91gEBgTZMGVcaBHQcaW0qYHj1F5OJtdl11uJLqRv0r8c+n/P1Cy90u2/14NgIgimQaX5kIgDM6 HpDvWqcYXAkVGvl6IaQwmIa4LeJDAXsmwQPh0tYsbPDaLH50B4CbJl4iZMYAy5LKZUMcFEH3UMRQ BydZlp4dDthWslO+llM=
And in mail.log:
Aug 22 10:48:53 robhost2 opendkim[1328]: 0E1B9AE90B: mx-indiegogo-a.sailthru.com [192.64.236.98] not internal Aug 22 10:48:53 robhost2 opendkim[1328]: 0E1B9AE90B: not authenticated Aug 22 10:48:53 robhost2 opendkim[1328]: 0E1B9AE90B: message has signatures from pmta.sailthru.com, indiegogo.com Aug 22 10:48:53 robhost2 opendkim[1328]: 0E1B9AE90B: s=mt d=pmta.sailthru.com SSL
DKIM is always checked; if the originating server is not using it, there won't be any DKIM headers in the received mail and this is what would appear in the mail logs:
Aug 22 10:51:47 robhost2 opendkim[1328]: 532C0AE90B: smtp5.ymlpsvr.com [185.83.51.14] not internal Aug 22 10:51:47 robhost2 opendkim[1328]: 532C0AE90B: not authenticated Aug 22 10:51:47 robhost2 opendkim[1328]: 532C0AE90B: no signature data
Advantages of DKIM
DKIM can be used for incoming mails to increase or decrease the spam score that is evaluated by systems like Amavis/SpamAssassin, which I don't use because I saw that I are very resource intensive.
As for outgoing DKIM-signed mails, for the same reasons they are less likely to be treated as spam by external systems.
ZEN SpamHaus
As written before, DNSBL (DNS Blocking Lists) are free databases that are populated with IP's of known spammers and that are checked against in order to look for the reputation of a sender.
I found ZEN SpamHaus DNSBL to be one of the most effective blocking lists available.
From Wikipedia:
DNSBL queries
When a mail server receives a connection from a client, and wishes to check that client against a DNSBL (let's say, dnsbl.example.net), it does more or less the following:
- Take the client's IP address—say, 192.168.42.23—and reverse the order of octets, yielding 23.42.168.192.
- Append the DNSBL's domain name: 23.42.168.192.dnsbl.example.net.
- Look up this name in the DNS as a domain name (“A” record). This will return either an address, indicating that the client is listed; or an “NXDOMAIN” (“No such domain”) code, indicating that the client is not.
- Optionally, if the client is listed, look up the name as a text record (“TXT” record). Most DNSBLs publish information about why a client is listed as TXT records.
Looking up an address in a DNSBL is thus similar to looking it up in reverse-DNS. The differences are that a DNSBL lookup uses the “A” rather than “PTR” record type, and uses a forward domain (such as dnsbl.example.net above) rather than the special reverse domain in-addr.arpa.
It is configured in Postfix by simply adding this line in main.cf:
smtpd_recipient_restrictions = .... reject_rbl_client zen.spamhaus.org
Here's how it works, from the log:
Aug 21 11:07:45 robhost2 postfix/smtpd[18147]: NOQUEUE: reject: RCPT from unknown[139.193.133.244]: 554 5.7.1 Service unavailable; Client host [139.193.133.244] blocked using zen.spamhaus.org; https://www.spamhaus.org/sbl/query/SBLCSS / https://www.spamhaus.org/query/ip/139.193.133.244; from=<Helga5555@example.net> to=<lav@example.net> proto=ESMTP helo=<fm-dyn-139-193-133-244.fast.net.id>
And here is what appears when clicking on the indicated link:
Postgrey
As explained here:
Greylisting is a spam reduction technique that works by temporarily rejecting client mail servers that are unknown to the server's greylisting service.
If the client is standards-compliant, it will attempt to re-send its message after its initial failed smtp session, and your receiving mail server will accept it. The client is then added to a list of known clients, and will not be delayed in the future. This means that the first e-mail from an unknown client will be delayed, but subsequent ones will be processed right away.
Most spam mailers, on the other hand, do not re-send messages after failed smtp sessions. Thus, in theory, greylisting effectively blocks the majority of spammers.
Here I attach an example from mail.log.
After the smtpd connection and SPF check (not shown), Postgrey intervenes:
Sep 26 18:09:22 robhost2 postfix/smtpd[7956]: connect from mx1.phx.paypal.com[66.211.168.231] ... Sep 26 18:09:24 robhost2 postgrey[1135]: action=greylist, reason=new, client_name=mx1.phx.paypal.com, client_address=66.211.168.231, sender=assistenza@paypal.it, recipient=info@example.com Sep 26 18:09:24 robhost2 postfix/smtpd[7956]: NOQUEUE: reject: RCPT from mx1.phx.paypal.com[66.211.168.231]: 450 4.2.0 <info@example.com>: Recipient address rejected: Greylisted, see http://postgrey.schweikert.ch/help/example.com.html; from=<assistenza@paypal.it> to=<info@example.com> proto=ESMTP helo=<mx1.phx.paypal.com> Sep 26 18:09:30 robhost2 postfix/smtpd[7956]: disconnect from mx1.phx.paypal.com[66.211.168.231]
Here the new message was rejected (greylisted) and the sender MTA was warned with a non-critical error (“450”).
After 10 minutes a new connection attempt from the sender comes; Postgrey controls that:
- the message comes from the same triplet: client_name/IP (mx1.phx.paypal.com, 66.211.168.231), sender (assistenza@paypal.it) and recipient (info@example.com)
- sufficient time (default 5 minutes) has passed since the first connection attempt; in the example 1024 seconds have passed.
Postgrey let then the message pass:
Sep 26 18:29:25 robhost2 postfix/smtpd[8991]: connect from mx1.phx.paypal.com[66.211.168.231] ... Sep 26 18:29:28 robhost2 postgrey[1135]: action=pass, reason=triplet found, delay=1204, client_name=mx1.phx.paypal.com, client_address=66.211.168.231, sender=assistenza@paypal.it, recipient=info@example.com
The message is then treated as usual by the Postfix processes/queues:
Sep 26 18:29:28 robhost2 postfix/smtpd[8991]: CCF1111216: client=mx1.phx.paypal.com[66.211.168.231] Sep 26 18:29:29 robhost2 postfix/cleanup[8997]: CCF1111216: message-id=<1474913360.8035@paypal.com> ...
Postgrey whitelist
There is a whitelist also for Postgrey, and it is updated by the O.. package mantainer. It is here:
/etc/postgrey/whitelist_clients
It shouldn't be edited; if clients have to be whitelisted follow the previous instructions.
Header Checks
So far we have dealt with the STMPD connection step of the message.
After evaluating all the smtpd_*_restrictions I have some header checks in place in main.cf, which come after the SMTP connection take care of parameters that are listed in the visible headers of mails.
# *_checks header_checks = regexp:/etc/postfix/header_checks
This is the only header check that I've implemented so far:
$ cat /etc/postfix/header_checks
# Intercept senders that I don't manage to block otherwise /^Reply-To: clickandgotowebsite@gmail.com/ DISCARD SPAM
/^From: "OneTwoSlim"/ DISCARD SPAM
Outgoing Mail: SMTP Authentication
All the configuration bits above referred to an incoming message; this is how I configured Postfix in order to also send messages.
Takern from http://www.postfix.org/SASL_README.html:
Envelope sender address authorization.
By default an SMTP client may specify any envelope sender address in the MAIL FROM command. That is because the Postfix SMTP server only knows the remote SMTP client hostname and IP address that is the sender SMTP server, but not the user who controls the remote SMTP client.
This changes the moment an SMTP client uses SASL authentication. Now, the Postfix SMTP server knows who the sender is. Given a table of envelope sender addresses and SASL login names, the Postfix SMTP server can decide if the SASL authenticated client is allowed to use a particular envelope sender address (MAIL FROM).
SASL, via Dovecot, is then needed to perform SMTP authentication, in order to be able to send mail.
smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth # and the common settings to enable SASL: smtpd_sasl_auth_enable = yes
# accept only these senders in MAIL FROM smtpd_sender_login_maps = hash:/etc/postfix/controlled_envelope_senders
smtpd_recipient_restrictions = ... reject_sender_login_mismatch permit_sasl_authenticated
$ cat /etc/postfix/controlled_envelope_senders
# envelope sender owners (SASL login names) info@example.com rob@example.net
From http://www.postfix.org/SASL_README.html:
The controlled_envelope_senders table specifies the binding between a sender envelope address (MAIL FROM) and the SASL login names that own that address.
With this, the reject_sender_login_mismatch restriction above will reject the sender address in the MAIL FROM command if smtpd_sender_login_maps does not specify the SMTP client's login name as an owner of that address.
Configuration in master.cf for SASL submission port (outogoing messages)
The idea is to use TCP port 25 for transporting email (MTA) from server to server and port 587 for submitting (MSA) email from a user to a mail server, which gives the benefit of a secure authentication.
# Submission port 587 inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_reject_unlisted_recipient=no # -o smtpd_client_restrictions=$mua_client_restrictions # -o smtpd_helo_restrictions=$mua_helo_restrictions # -o smtpd_sender_restrictions=$mua_sender_restrictions -o smtpd_recipient_restrictions= -o smtpd_relay_restrictions=permit_sasl_authenticated,reject -o milter_macro_daemon_name=ORIGINATING
Example of an outgoing mail via submission port
The following is an example of an outgoing mail via submission port, made from a smartphone mail client:
Aug 23 15:20:51 robhost2 postfix/submission/smtpd[28176]: connect from host2-111-static.96-5-b.business.telecomitalia.it[5.96.111.2] Aug 23 15:20:52 robhost2 dovecot: imap-login: Login: user=<rob>, method=PLAIN, rip=5.96.111.2, lip=31.222.165.32, mpid=28177, TLS, session=<HH/kQW1X+QAFYG8C> Aug 23 15:20:52 robhost2 postfix/submission/smtpd[28176]: A2F96AE90B: client=host2-111-static.96-5-b.business.telecomitalia.it[5.96.111.2], sasl_method=PLAIN, sasl_username=rob Aug 23 15:20:52 robhost2 postfix/cleanup[28180]: A2F96AE90B: message-id=<> Aug 23 15:20:53 robhost2 postfix/qmgr[9868]: A2F96AE90B: from=<rob@example.com>, size=572, nrcpt=1 (queue active)
The user is successfully authenticated; since the recipient is not local (r.carraro@test.it) Postfix uses the _smtp_ client command, which transports outbound messages to remote destinations:
Aug 23 15:20:54 robhost2 postfix/smtp[28181]: A2F96AE90B: to=<r.carraro@test.it>, relay=prefilter.emailsecurity.trendmicro.eu[150.70.226.147]:25, delay=2.3, delays=0.82/0.13/0.29/1, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as BB0C868002E) Aug 23 15:20:54 robhost2 postfix/qmgr[9868]: A2F96AE90B: removed
SASL LOGIN authentication failed in logs
In mail.log I see tons of warning like these:
Aug 24 08:16:34 robhost2 postfix/smtpd[11564]: warning: hostname default-rdns.vocus.co.nz does not resolve to address 101.98.214.248: Name or service not known Aug 24 08:16:34 robhost2 postfix/smtpd[11564]: connect from unknown[101.98.214.248] Aug 24 08:16:37 robhost2 postfix/smtpd[11564]: warning: unknown[101.98.214.248]: SASL LOGIN authentication failed: UGFzc3dvcmQ6 Aug 24 08:16:37 robhost2 postfix/smtpd[11564]: disconnect from unknown[101.98.214.248]
They are due to goddamn spammers who try to relay on my server in order to send outbound mails (i.e. not addressed to my internal domains).
They connect to port TCP 25, but then are blocked by SASL auth, as can be seen by netstat:
# netstat -tunp Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 16 31.222.165.32:25 101.98.214.248:60324 FIN_WAIT1 - tcp 0 0 127.0.0.1:60199 127.0.0.1:8891 TIME_WAIT -
Fail2ban
I take care of this continuous connections with fail2ban, which I plan to discuss in a future article. But for the sake of completeness I can anticipate that I have modified the file:
/etc/fail2ban/jail.local
like this:
... [sasl]
enabled = true port = smtp,ssmtp,imap2,imap3,imaps,pop3,pop3s filter = sasl maxretry = 1 bantime = 3600
That means that I ban every SMTP connection, that generated a single error, for one hour (3600 seconds). That prevents my server from being continuosly hammered by spammers, who try to relay on it.
Graph of the reasons for mail rejection on my server
The following graph summarizes the results of my struggle to fight spam; I looked at the logs of all mail received during a month and here are the percentages of blocked mails generated with the above restrictions in place:
Conclusions
As it can be seen, around 70% of all the rejections are due to the missing reverse hostname by senders, that is a misconfiguration in the reverse lookup zone, which can be taken care of by the setting:
smtpd_recipient_restrictions = ... reject_unknown_reverse_client_hostname
The other consistent block is the one with more than 22% mails being rejected by checking against ZEN Spamhaus DNS blocking list, and it is configured like this:
smtpd_recipient_restrictions = ... reject_rbl_client zen.spamhaus.org
Those two restrictions alone constitute the bulk (more than 90%) of the unwanted mails blocked by my mail system.
So the take-home message for this too long article is: if you want to fight spam (who doesn't want?) make sure you have in place at least those two settings!
And be to sure to implement fail2ban too.