From 3b7fc5930ce0b9f00f27600fbb72c3cfa3d97571 Mon Sep 17 00:00:00 2001 From: 17Halbe Date: Sun, 4 Feb 2018 21:31:08 +0100 Subject: [PATCH] Introducing Postscreen (#799) * Introduced Postscreen cheaper, earlier and simpler blocking of zombies/spambots. From http://postfix.cs.utah.edu/POSTSCREEN_README.html : As a first layer, postscreen(8) blocks connections from zombies and other spambots that are responsible for about 90% of all spam. It is implemented as a single process to make this defense as cheap as possible. Things we need to consider: - Do we need a whitelist/backlist file? (http://postfix.cs.utah.edu/postconf.5.html#postscreen_access_list) - Via introducing an optional config/postfix-access.cidr - The only permanent whitelisting I could imagine are monitoring services(which might (still?) behave weird/hastely) or blacklisting backup servers(since no traffic should be coming from them anyway) - Do we need deep inspections? They are desireable, but these tests are expensive: a good client must disconnect after it passes the test, before it can talk to a real Postfix SMTP server. Considered tests are: - postscreen_bare_newline_enable (http://postfix.cs.utah.edu/postconf.5.html#postscreen_bare_newline_action) - postscreen_non_smtp_command_enable (http://postfix.cs.utah.edu/postconf.5.html#postscreen_non_smtp_command_action) - postscreen_pipelining_enable (http://postfix.cs.utah.edu/postconf.5.html#postscreen_pipelining_action) - Do we need to make the blacklisting via dnsblocking configurable? It's currently set and weighted as follows, where a score of 3 results in blocking, a score of -1 results in whitelisting: (*: adds the specified weight to the SMTP client's DNSBL score. Specify a negative number for whitelisting.) (http://postfix.cs.utah.edu/postconf.5.html#postscreen_dnsbl_sites) - zen.spamhaus.org*3 - bl.mailspike.net - b.barracudacentral.org*2 - bl.spameatingmonkey.net - bl.spamcop.net - dnsbl.sorbs.net - psbl.surriel.com - list.dnswl.org=127.0.[0..255].0*-2 - list.dnswl.org=127.0.[0..255].1*-3 - list.dnswl.org=127.0.[0..255].[2..3]*-4 - What to do when blacklisting? I currently set it to drop. We could - ignore: Ignore the failure of this test. Allow other tests to complete. Repeat this test the next time the client connects. This option is useful for testing and collecting statistics without blocking mail. - enforce: Allow other tests to complete. Reject attempts to deliver mail with a 550 SMTP reply, and log the helo/sender/recipient information. Repeat this test the next time the client connects. - drop: Drop the connection immediately with a 521 SMTP reply. Repeat this test the next time the client connects. In the end I think we could drop postgrey support. Postscreen replaces postgrey in its entirety, while being more selective and not delaying mail. Especially if we consider using the deep inspection options of postscreen. Hope that wasn't too much to read! ;) * main.cf got misformatted.. Don't know how, should be ok now. * fixed malformatted main.cf & repaired master.cf * reenabled rbl stuff.. It's cached, therefore doesn't hurt * fixed tests * added tests, repaired tests, added info, introduced new Variable POSTSCREEN_ACTION, fixes --- Makefile | 10 ++++++++ README.md | 22 +++++++++++------ target/postfix/main.cf | 20 ++++++++++++++-- target/postfix/master.cf | 5 +++- target/start-mailserver.sh | 8 +++++++ test/tests.bats | 49 ++++++++++++++++++++++++++++++++++---- 6 files changed, 100 insertions(+), 14 deletions(-) diff --git a/Makefile b/Makefile index e1525d32..001f4f4b 100644 --- a/Makefile +++ b/Makefile @@ -91,6 +91,7 @@ run: -v "`pwd`/test/config":/tmp/docker-mailserver \ -v "`pwd`/test":/tmp/docker-mailserver-test \ -e ENABLE_FAIL2BAN=1 \ + -e POSTSCREEN_ACTION=ignore \ --cap-add=NET_ADMIN \ -h mail.my-domain.com -t $(NAME) sleep 15 @@ -160,6 +161,13 @@ run: -e DMS_DEBUG=0 \ -h mail.my-domain.com -t $(NAME) sleep 15 + docker run -d --name mail_postscreen \ + -v "`pwd`/test/config":/tmp/docker-mailserver \ + -v "`pwd`/test":/tmp/docker-mailserver-test \ + -e POSTSCREEN_ACTION=enforce \ + --cap-add=NET_ADMIN \ + -h mail.my-domain.com -t $(NAME) + sleep 15 docker run -d --name mail_lmtp_ip \ -v "`pwd`/test/config":/tmp/docker-mailserver \ -v "`pwd`/test/config/dovecot-lmtp":/etc/dovecot \ @@ -180,6 +188,7 @@ run: -h mail.my-domain.com -t $(NAME) sleep 20 + generate-accounts-after-run: docker run --rm -e MAIL_USER=added@localhost.localdomain -e MAIL_PASS=mypassword -t $(NAME) /bin/sh -c 'echo "$$MAIL_USER|$$(doveadm pw -s SHA512-CRYPT -u $$MAIL_USER -p $$MAIL_PASS)"' >> test/config/postfix-accounts.cf sleep 10 @@ -237,6 +246,7 @@ clean: mail_with_imap \ mail_lmtp_ip \ mail_with_postgrey \ + mail_postscreen \ mail_override_hostname @if [ -f config/postfix-accounts.cf.bak ]; then\ diff --git a/README.md b/README.md index 3138138f..5e31e41e 100644 --- a/README.md +++ b/README.md @@ -12,16 +12,18 @@ Includes: - postfix with smtp or ldap auth - dovecot for sasl, imap (and optional pop3) with ssl support, with ldap auth - saslauthd with ldap auth -- amavis -- spamassasin supporting custom rules -- clamav with automatic updates +- [amavis](https://www.amavis.org/) +- [spamassasin](http://spamassassin.apache.org/) supporting custom rules +- [clamav](https://www.clamav.net/) with automatic updates - opendkim - opendmarc -- fail2ban -- fetchmail -- postgrey +- [fail2ban](https://www.fail2ban.org/wiki/index.php/Main_Page) +- [fetchmail](http://www.fetchmail.info/fetchmail-man.html) +- [postscreen](http://www.postfix.org/POSTSCREEN_README.html) +- [postgrey](https://postgrey.schweikert.ch/) - basic [sieve support](https://github.com/tomav/docker-mailserver/wiki/Configure-Sieve-filters) using dovecot - [LetsEncrypt](https://letsencrypt.org/) and self-signed certificates +- [setup script](https://github.com/tomav/docker-mailserver/wiki/Setup-docker-mailserver-using-the-script-setup.sh) to easily configure and maintain your mailserver - persistent data and state (but think about backups!) - [integration tests](https://travis-ci.org/tomav/docker-mailserver) - [automated builds on docker hub](https://hub.docker.com/r/tvial/docker-mailserver/) @@ -253,7 +255,7 @@ Otherwise, `iptables` won't be able to ban IPs. - **empty** => Managesieve service disabled - 1 => Enables Managesieve on port 4190 -##### ENABLE_FETCHMAIL +#### ENABLE_FETCHMAIL - **0** => `fetchmail` disabled - 1 => `fetchmail` enabled @@ -332,6 +334,12 @@ Otherwise, `iptables` won't be able to ban IPs. - **empty** => postmaster@domain.com - => Specify the postmaster address +##### POSTSCREEN_ACTION + + - **enforce** => Allow other tests to complete. Reject attempts to deliver mail with a 550 SMTP reply, and log the helo/sender/recipient information. Repeat this test the next time the client connects. + - drop => Drop the connection immediately with a 521 SMTP reply. Repeat this test the next time the client connects. + - ignore => Ignore the failure of this test. Allow other tests to complete. Repeat this test the next time the client connects. This option is useful for testing and collecting statistics without blocking mail. + #### ENABLE_POSTGREY - **0** => `postgrey` is disabled diff --git a/target/postfix/main.cf b/target/postfix/main.cf index 20c33f59..41cd6f30 100644 --- a/target/postfix/main.cf +++ b/target/postfix/main.cf @@ -45,12 +45,28 @@ smtpd_delay_reject = yes smtpd_helo_restrictions = permit_mynetworks, reject_invalid_helo_hostname, permit smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination, check_policy_service unix:private/policyd-spf, - reject_unauth_pipelining, reject_invalid_helo_hostname, reject_non_fqdn_helo_hostname, reject_unknown_recipient_domain, - reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net + reject_unauth_pipelining, reject_invalid_helo_hostname, reject_non_fqdn_helo_hostname, reject_unknown_recipient_domain, reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net smtpd_client_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, reject_unauth_pipelining smtpd_sender_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unknown_sender_domain disable_vrfy_command = yes +# Postscreen settings to drop zombies/open relays/spam early +postscreen_dnsbl_action = enforce +postscreen_dnsbl_sites = zen.spamhaus.org*3 + bl.mailspike.net + b.barracudacentral.org*2 + bl.spameatingmonkey.net + bl.spamcop.net + dnsbl.sorbs.net + psbl.surriel.com + list.dnswl.org=127.0.[0..255].0*-2 + list.dnswl.org=127.0.[0..255].1*-3 + list.dnswl.org=127.0.[0..255].[2..3]*-4 +postscreen_dnsbl_threshold = 3 +postscreen_dnsbl_whitelist_threshold = -1 +postscreen_greet_action = enforce +postscreen_bare_newline_action = enforce + # SASL smtpd_sasl_auth_enable = yes smtpd_sasl_path = /var/spool/postfix/private/auth diff --git a/target/postfix/master.cf b/target/postfix/master.cf index 57a474f8..6b4e80bf 100644 --- a/target/postfix/master.cf +++ b/target/postfix/master.cf @@ -10,7 +10,10 @@ # (yes) (yes) (no) (never) (100) # ========================================================================== -smtp inet n - n - - smtpd +smtp inet n - n - 1 postscreen +smtpd pass - - n - - smtpd +tlsproxy unix - - n - 0 tlsproxy +dnsblog unix - - n - 0 dnsblog submission inet n - n - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt diff --git a/target/start-mailserver.sh b/target/start-mailserver.sh index a3b2405f..442a73c0 100644 --- a/target/start-mailserver.sh +++ b/target/start-mailserver.sh @@ -23,6 +23,7 @@ DEFAULT_VARS["ENABLE_SASLAUTHD"]="${ENABLE_SASLAUTHD:="0"}" DEFAULT_VARS["SMTP_ONLY"]="${SMTP_ONLY:="0"}" DEFAULT_VARS["DMS_DEBUG"]="${DMS_DEBUG:="0"}" DEFAULT_VARS["OVERRIDE_HOSTNAME"]="${OVERRIDE_HOSTNAME}" +DEFAULT_VARS["POSTSCREEN_ACTION"]="${POSTSCREEN_ACTION:="enforce"}" ########################################################################## # << DEFAULT VARS ########################################################################## @@ -114,6 +115,7 @@ function register_functions() { _register_setup_function "_setup_postfix_aliases" _register_setup_function "_setup_postfix_vhost" _register_setup_function "_setup_postfix_dhparam" + _register_setup_function "_setup_postfix_postscreen" if [ ! -z "$AWS_SES_HOST" -a ! -z "$AWS_SES_USERPASS" ]; then _register_setup_function "_setup_postfix_relay_amazon_ses" @@ -602,6 +604,12 @@ function _setup_postgrey() { fi } +function _setup_postfix_postscreen() { + notify 'inf' "Configuring postscreen" + sed -i -e "s/postscreen_dnsbl_action = enforce/postscreen_dnsbl_action = $POSTSCREEN_ACTION/" \ + -e "s/postscreen_greet_action = enforce/postscreen_greet_action = $POSTSCREEN_ACTION/" \ + -e "s/postscreen_bare_newline_action = enforce/postscreen_bare_newline_action = $POSTSCREEN_ACTION/" /etc/postfix/main.cf +} function _setup_postfix_sasl() { if [[ ${ENABLE_SASLAUTHD} == 1 ]];then diff --git a/test/tests.bats b/test/tests.bats index ddd5ac04..5ddb488f 100644 --- a/test/tests.bats +++ b/test/tests.bats @@ -755,11 +755,19 @@ load 'test_helper/bats-assert/load' # Getting mail_fail2ban container IP MAIL_FAIL2BAN_IP=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' mail_fail2ban) - # Create a container which will send wrong authentications and should banned + # Create a container which will send wrong authentications and should get banned docker run --name fail-auth-mailer -e MAIL_FAIL2BAN_IP=$MAIL_FAIL2BAN_IP -v "$(pwd)/test":/tmp/docker-mailserver-test -d $(docker inspect --format '{{ .Config.Image }}' mail) tail -f /var/log/faillog - - docker exec fail-auth-mailer /bin/sh -c "nc $MAIL_FAIL2BAN_IP 25 < /tmp/docker-mailserver-test/auth/smtp-auth-login-wrong.txt" - docker exec fail-auth-mailer /bin/sh -c "nc $MAIL_FAIL2BAN_IP 25 < /tmp/docker-mailserver-test/auth/smtp-auth-login-wrong.txt" + + # can't pipe the file as usual due to postscreen. (respecting postscreen_greet_wait time and talking in turn): + for i in {1,2}; do + docker exec fail-auth-mailer /bin/bash -c \ + 'exec 3<>/dev/tcp/$MAIL_FAIL2BAN_IP/25 && \ + while IFS= read -r cmd; do \ + head -1 <&3; \ + [[ "$cmd" == "EHLO"* ]] && sleep 6; \ + echo $cmd >&3; \ + done < "/tmp/docker-mailserver-test/auth/smtp-auth-login-wrong.txt"' + done sleep 5 @@ -789,6 +797,39 @@ load 'test_helper/bats-assert/load' assert_failure } +# +# postscreen +# + +@test "checking postscreen" { + # Getting mail container IP + MAIL_POSTSCREEN_IP=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' mail_postscreen) + + # talk too fast: + + docker exec fail-auth-mailer /bin/sh -c "nc $MAIL_POSTSCREEN_IP 25 < /tmp/docker-mailserver-test/auth/smtp-auth-login.txt" + sleep 5 + + run docker exec mail_postscreen grep 'COMMAND PIPELINING' /var/log/mail/mail.log + assert_success + + # positive test. (respecting postscreen_greet_wait time and talking in turn): + for i in {1,2}; do + docker exec fail-auth-mailer /bin/bash -c \ + 'exec 3<>/dev/tcp/'$MAIL_POSTSCREEN_IP'/25 && \ + while IFS= read -r cmd; do \ + head -1 <&3; \ + [[ "$cmd" == "EHLO"* ]] && sleep 6; \ + echo $cmd >&3; \ + done < "/tmp/docker-mailserver-test/auth/smtp-auth-login.txt"' + done + + sleep 5 + + run docker exec mail_postscreen grep 'PASS NEW ' /var/log/mail/mail.log + assert_success +} + # # fetchmail #