docker-mailserver/test/tests/parallel/set1/spam_virus/fail2ban.bats

191 lines
6.7 KiB
Plaintext
Raw Normal View History

load "${REPOSITORY_ROOT}/test/helper/setup"
load "${REPOSITORY_ROOT}/test/helper/common"
BATS_TEST_NAME_PREFIX='[Fail2Ban] '
CONTAINER1_NAME='dms-test_fail2ban'
CONTAINER2_NAME='dms-test_fail2ban_fail-auth-mailer'
function setup_file() {
export CONTAINER_NAME
CONTAINER_NAME=${CONTAINER1_NAME}
local CUSTOM_SETUP_ARGUMENTS=(
--env ENABLE_FAIL2BAN=1
--env POSTSCREEN_ACTION=ignore
--cap-add=NET_ADMIN
refactor: Parallel Tests - `disabled_clamav_spamassassin`: - Just shuffling the test order around, and removing the restart test at the end which doesn't make sense. - `postscreen`: - Now uses common helper for getting container IP - Does not appear to need the `NET_ADMIN` capability? - Reduced startup time for the 2nd container + additional context about it's relevance. - Test cases are largely the same, but refactored the `nc` alternative that properly waits it's turn. This only needs to run once. Added additional commentary and made into a generic method if needed in other tests. - `fail2ban`: - Use the common container IP helper method. - Postscreen isn't affecting this test, it's not required to do the much slower exchange with the mail server when sending a login failure. - IP being passed into ENV is no longer necessary. - `sleep 5` in the related test cases doesn't seem necessary, can better rely on polling with timeout. - `sleep 10` for `setup.sh` also doesn't appear to be necessary. - `postgrey`: - Reduced POSTGREY_DELAY to 3, which shaves a fair amount of wasted time while still verifying the delay works. - One of the checks in `main.cf` doesn't seem to need to know about the earlier spamhaus portion of the line to work, removed. - Better test case descriptions. - Improved log matching via standard method that better documents the expected triplet under test. - Removed a redundant whitelist file and test that didn't seem to have any relevance. Added a TODO with additional notes about a concern with these tests. - Reduced test time as 8 second timeouts from `-w 8` don't appear to be required, better to poll with grep instead. - Replaced `wc -l` commands with a new method to assert expected line count, better enabling assertions on the actual output. - `undef_spam_subject`: - Split to two separate test cases, and initialize each container in their case instead of `setup_file()`, allowing for using the default `teardown()` method (and slight benefit if running in parallel). - `permit_docker`: - Not a parallel test, but I realized that the repeat helper methods don't necessarily play well with `run` as the command (can cause false positive of what was successful).
2023-01-03 06:11:36 +00:00
# NOTE: May no longer be needed with newer F2B:
--ulimit "nofile=$(ulimit -Sn):$(ulimit -Hn)"
)
_init_with_defaults
_common_container_setup 'CUSTOM_SETUP_ARGUMENTS'
_wait_for_smtp_port_in_container
tests(fix): Adjust for local testing conditions (#2606) * tests(fix): Increase some timeouts Running tests locally via a VM these tests would fail sometimes due to the time from being queued and Amavis actually processing being roughly around 30 seconds. There should be no harm in raising this to 60 seconds, other than delaying a failure case which will ripple through other time sensitive tests. It's better to pass when functionality is actually correct but just needs a bit longer to complete. * tests(fix): Don't setup an invalid hostname During container startup `helpers/dns.sh` would panic with `hostname -f` failing. Dropping `--domainname` for this container is fine and does not affect the point of it's test. --- It's unclear why this does not occur in CI. Possibly changes within the docker daemon since as CI runs docker on Ubuntu 20.04? (2020). For clarity, this may be equivalent to setting a hostname of `domain.com.domain.com`, or `--hostname` value truncated the NIS domain (`--domainname`) of the same value. IIRC, it would still fail with both options using different values if `--hostname` was multi-label. I believe I've documented how non-deterministic these options can be across different environments. `--hostname` should be preferred. There doesn't seem to be any reason to actually need `--domainname` (which is NIS domain name, unrelated to the DNS domain name). We still need to properly investigate reworking our ENV support that `dns.sh` manages. --- Containers were also not removing themselves after failures either (missing teardown). Which would cause problems when running tests again. * chore: Normalize white-space Sets a consistent indent size of 2 spaces. Previously this varied a fair bit, sometimes with tabs or mixed tabs and spaces. Some formatting with blank lines. Easier to review with white-space in diff ignored. Some minor edits besides blank lines, but no change in functionality. * fix: `setup.sh` target container under test Some of the `setup.sh` commands did not specify the container which was problematic if another `docker-mailserver` container was running, causing test failures. This probably doesn't help with `test/no_container.bats`, but at least prevents `test/tests.bats` failing at this point.
2022-05-30 00:53:30 +00:00
# Create a container which will send wrong authentications and should get banned
CONTAINER_NAME=${CONTAINER2_NAME}
_init_with_defaults
_common_container_setup 'CUSTOM_SETUP_ARGUMENTS'
tests(fix): Adjust for local testing conditions (#2606) * tests(fix): Increase some timeouts Running tests locally via a VM these tests would fail sometimes due to the time from being queued and Amavis actually processing being roughly around 30 seconds. There should be no harm in raising this to 60 seconds, other than delaying a failure case which will ripple through other time sensitive tests. It's better to pass when functionality is actually correct but just needs a bit longer to complete. * tests(fix): Don't setup an invalid hostname During container startup `helpers/dns.sh` would panic with `hostname -f` failing. Dropping `--domainname` for this container is fine and does not affect the point of it's test. --- It's unclear why this does not occur in CI. Possibly changes within the docker daemon since as CI runs docker on Ubuntu 20.04? (2020). For clarity, this may be equivalent to setting a hostname of `domain.com.domain.com`, or `--hostname` value truncated the NIS domain (`--domainname`) of the same value. IIRC, it would still fail with both options using different values if `--hostname` was multi-label. I believe I've documented how non-deterministic these options can be across different environments. `--hostname` should be preferred. There doesn't seem to be any reason to actually need `--domainname` (which is NIS domain name, unrelated to the DNS domain name). We still need to properly investigate reworking our ENV support that `dns.sh` manages. --- Containers were also not removing themselves after failures either (missing teardown). Which would cause problems when running tests again. * chore: Normalize white-space Sets a consistent indent size of 2 spaces. Previously this varied a fair bit, sometimes with tabs or mixed tabs and spaces. Some formatting with blank lines. Easier to review with white-space in diff ignored. Some minor edits besides blank lines, but no change in functionality. * fix: `setup.sh` target container under test Some of the `setup.sh` commands did not specify the container which was problematic if another `docker-mailserver` container was running, causing test failures. This probably doesn't help with `test/no_container.bats`, but at least prevents `test/tests.bats` failing at this point.
2022-05-30 00:53:30 +00:00
# Set default implicit container fallback for helpers:
CONTAINER_NAME=${CONTAINER1_NAME}
}
function teardown_file() {
docker rm -f "${CONTAINER1_NAME}" "${CONTAINER2_NAME}"
}
@test "localhost is not banned because ignored" {
_run_in_container fail2ban-client status postfix-sasl
assert_success
refute_output --regexp '.*IP list:.*127\.0\.0\.1.*'
_run_in_container grep 'ignoreip = 127.0.0.1/8' /etc/fail2ban/jail.conf
assert_success
}
@test "fail2ban-fail2ban.cf overrides" {
_run_in_container fail2ban-client get loglevel
assert_success
assert_output --partial 'DEBUG'
}
@test "fail2ban-jail.cf overrides" {
for FILTER in 'dovecot' 'postfix' 'postfix-sasl'
do
_run_in_container fail2ban-client get "${FILTER}" bantime
assert_output 1234
_run_in_container fail2ban-client get "${FILTER}" findtime
assert_output 321
_run_in_container fail2ban-client get "${FILTER}" maxretry
assert_output 2
_run_in_container fail2ban-client -d
assert_output --partial "['set', 'dovecot', 'addaction', 'nftables-multiport']"
assert_output --partial "['set', 'postfix', 'addaction', 'nftables-multiport']"
assert_output --partial "['set', 'postfix-sasl', 'addaction', 'nftables-multiport']"
done
}
refactor: Parallel Tests - `disabled_clamav_spamassassin`: - Just shuffling the test order around, and removing the restart test at the end which doesn't make sense. - `postscreen`: - Now uses common helper for getting container IP - Does not appear to need the `NET_ADMIN` capability? - Reduced startup time for the 2nd container + additional context about it's relevance. - Test cases are largely the same, but refactored the `nc` alternative that properly waits it's turn. This only needs to run once. Added additional commentary and made into a generic method if needed in other tests. - `fail2ban`: - Use the common container IP helper method. - Postscreen isn't affecting this test, it's not required to do the much slower exchange with the mail server when sending a login failure. - IP being passed into ENV is no longer necessary. - `sleep 5` in the related test cases doesn't seem necessary, can better rely on polling with timeout. - `sleep 10` for `setup.sh` also doesn't appear to be necessary. - `postgrey`: - Reduced POSTGREY_DELAY to 3, which shaves a fair amount of wasted time while still verifying the delay works. - One of the checks in `main.cf` doesn't seem to need to know about the earlier spamhaus portion of the line to work, removed. - Better test case descriptions. - Improved log matching via standard method that better documents the expected triplet under test. - Removed a redundant whitelist file and test that didn't seem to have any relevance. Added a TODO with additional notes about a concern with these tests. - Reduced test time as 8 second timeouts from `-w 8` don't appear to be required, better to poll with grep instead. - Replaced `wc -l` commands with a new method to assert expected line count, better enabling assertions on the actual output. - `undef_spam_subject`: - Split to two separate test cases, and initialize each container in their case instead of `setup_file()`, allowing for using the default `teardown()` method (and slight benefit if running in parallel). - `permit_docker`: - Not a parallel test, but I realized that the repeat helper methods don't necessarily play well with `run` as the command (can cause false positive of what was successful).
2023-01-03 06:11:36 +00:00
# NOTE: This test case is fragile if other test cases were to be run concurrently.
# - After multiple login fails and a slight delay, f2b will ban that IP.
# - You could hard-code `sleep 5` on both cases to avoid the alternative assertions,
# but the polling + piping into grep approach here reliably minimizes the delay.
@test "ban ip on multiple failed login" {
CONTAINER1_IP=$(_get_container_ip "${CONTAINER1_NAME}")
refactor: Parallel Tests - `disabled_clamav_spamassassin`: - Just shuffling the test order around, and removing the restart test at the end which doesn't make sense. - `postscreen`: - Now uses common helper for getting container IP - Does not appear to need the `NET_ADMIN` capability? - Reduced startup time for the 2nd container + additional context about it's relevance. - Test cases are largely the same, but refactored the `nc` alternative that properly waits it's turn. This only needs to run once. Added additional commentary and made into a generic method if needed in other tests. - `fail2ban`: - Use the common container IP helper method. - Postscreen isn't affecting this test, it's not required to do the much slower exchange with the mail server when sending a login failure. - IP being passed into ENV is no longer necessary. - `sleep 5` in the related test cases doesn't seem necessary, can better rely on polling with timeout. - `sleep 10` for `setup.sh` also doesn't appear to be necessary. - `postgrey`: - Reduced POSTGREY_DELAY to 3, which shaves a fair amount of wasted time while still verifying the delay works. - One of the checks in `main.cf` doesn't seem to need to know about the earlier spamhaus portion of the line to work, removed. - Better test case descriptions. - Improved log matching via standard method that better documents the expected triplet under test. - Removed a redundant whitelist file and test that didn't seem to have any relevance. Added a TODO with additional notes about a concern with these tests. - Reduced test time as 8 second timeouts from `-w 8` don't appear to be required, better to poll with grep instead. - Replaced `wc -l` commands with a new method to assert expected line count, better enabling assertions on the actual output. - `undef_spam_subject`: - Split to two separate test cases, and initialize each container in their case instead of `setup_file()`, allowing for using the default `teardown()` method (and slight benefit if running in parallel). - `permit_docker`: - Not a parallel test, but I realized that the repeat helper methods don't necessarily play well with `run` as the command (can cause false positive of what was successful).
2023-01-03 06:11:36 +00:00
# Trigger a ban by failing to login twice:
2023-02-24 09:44:18 +00:00
CONTAINER_NAME=${CONTAINER2_NAME} _send_email 'auth/smtp-auth-login-wrong' "${CONTAINER1_IP} 465"
CONTAINER_NAME=${CONTAINER2_NAME} _send_email 'auth/smtp-auth-login-wrong' "${CONTAINER1_IP} 465"
# Checking that CONTAINER2_IP is banned in "${CONTAINER1_NAME}"
CONTAINER2_IP=$(_get_container_ip "${CONTAINER2_NAME}")
run _repeat_in_container_until_success_or_timeout 10 "${CONTAINER_NAME}" /bin/bash -c "fail2ban-client status postfix-sasl | grep -F '${CONTAINER2_IP}'"
assert_success
refactor: Parallel Tests - `disabled_clamav_spamassassin`: - Just shuffling the test order around, and removing the restart test at the end which doesn't make sense. - `postscreen`: - Now uses common helper for getting container IP - Does not appear to need the `NET_ADMIN` capability? - Reduced startup time for the 2nd container + additional context about it's relevance. - Test cases are largely the same, but refactored the `nc` alternative that properly waits it's turn. This only needs to run once. Added additional commentary and made into a generic method if needed in other tests. - `fail2ban`: - Use the common container IP helper method. - Postscreen isn't affecting this test, it's not required to do the much slower exchange with the mail server when sending a login failure. - IP being passed into ENV is no longer necessary. - `sleep 5` in the related test cases doesn't seem necessary, can better rely on polling with timeout. - `sleep 10` for `setup.sh` also doesn't appear to be necessary. - `postgrey`: - Reduced POSTGREY_DELAY to 3, which shaves a fair amount of wasted time while still verifying the delay works. - One of the checks in `main.cf` doesn't seem to need to know about the earlier spamhaus portion of the line to work, removed. - Better test case descriptions. - Improved log matching via standard method that better documents the expected triplet under test. - Removed a redundant whitelist file and test that didn't seem to have any relevance. Added a TODO with additional notes about a concern with these tests. - Reduced test time as 8 second timeouts from `-w 8` don't appear to be required, better to poll with grep instead. - Replaced `wc -l` commands with a new method to assert expected line count, better enabling assertions on the actual output. - `undef_spam_subject`: - Split to two separate test cases, and initialize each container in their case instead of `setup_file()`, allowing for using the default `teardown()` method (and slight benefit if running in parallel). - `permit_docker`: - Not a parallel test, but I realized that the repeat helper methods don't necessarily play well with `run` as the command (can cause false positive of what was successful).
2023-01-03 06:11:36 +00:00
assert_output --partial 'Banned IP list:'
# Checking that CONTAINER2_IP is banned by nftables
_run_in_container_bash 'nft list set inet f2b-table addr-set-postfix-sasl'
assert_success
assert_output --partial "elements = { ${CONTAINER2_IP} }"
}
refactor: Parallel Tests - `disabled_clamav_spamassassin`: - Just shuffling the test order around, and removing the restart test at the end which doesn't make sense. - `postscreen`: - Now uses common helper for getting container IP - Does not appear to need the `NET_ADMIN` capability? - Reduced startup time for the 2nd container + additional context about it's relevance. - Test cases are largely the same, but refactored the `nc` alternative that properly waits it's turn. This only needs to run once. Added additional commentary and made into a generic method if needed in other tests. - `fail2ban`: - Use the common container IP helper method. - Postscreen isn't affecting this test, it's not required to do the much slower exchange with the mail server when sending a login failure. - IP being passed into ENV is no longer necessary. - `sleep 5` in the related test cases doesn't seem necessary, can better rely on polling with timeout. - `sleep 10` for `setup.sh` also doesn't appear to be necessary. - `postgrey`: - Reduced POSTGREY_DELAY to 3, which shaves a fair amount of wasted time while still verifying the delay works. - One of the checks in `main.cf` doesn't seem to need to know about the earlier spamhaus portion of the line to work, removed. - Better test case descriptions. - Improved log matching via standard method that better documents the expected triplet under test. - Removed a redundant whitelist file and test that didn't seem to have any relevance. Added a TODO with additional notes about a concern with these tests. - Reduced test time as 8 second timeouts from `-w 8` don't appear to be required, better to poll with grep instead. - Replaced `wc -l` commands with a new method to assert expected line count, better enabling assertions on the actual output. - `undef_spam_subject`: - Split to two separate test cases, and initialize each container in their case instead of `setup_file()`, allowing for using the default `teardown()` method (and slight benefit if running in parallel). - `permit_docker`: - Not a parallel test, but I realized that the repeat helper methods don't necessarily play well with `run` as the command (can cause false positive of what was successful).
2023-01-03 06:11:36 +00:00
# NOTE: Depends on previous test case, if no IP was banned at this point, it passes regardless..
@test "unban ip works" {
CONTAINER2_IP=$(_get_container_ip "${CONTAINER2_NAME}")
_run_in_container fail2ban-client set postfix-sasl unbanip "${CONTAINER2_IP}"
assert_success
# Checking that CONTAINER2_IP is unbanned in "${CONTAINER1_NAME}"
_run_in_container fail2ban-client status postfix-sasl
assert_success
refute_output --partial "${CONTAINER2_IP}"
# Checking that CONTAINER2_IP is unbanned by nftables
_run_in_container_bash 'nft list set inet f2b-table addr-set-postfix-sasl'
refute_output --partial "${CONTAINER2_IP}"
}
@test "bans work properly (single IP)" {
_run_in_container fail2ban ban 192.0.66.7
assert_success
assert_output 'Banned custom IP: 1'
_run_in_container fail2ban
assert_success
assert_output --regexp 'Banned in custom:.*192\.0\.66\.7'
_run_in_container nft list set inet f2b-table addr-set-custom
2022-10-05 10:19:49 +00:00
assert_success
assert_output --partial 'elements = { 192.0.66.7 }'
2022-10-05 10:19:49 +00:00
_run_in_container fail2ban unban 192.0.66.7
assert_success
assert_output --partial 'Unbanned IP from custom: 1'
2022-10-05 10:19:49 +00:00
_run_in_container nft list set inet f2b-table addr-set-custom
refute_output --partial '192.0.66.7'
}
2022-10-15 10:01:59 +00:00
@test "bans work properly (subnet)" {
_run_in_container fail2ban ban 192.0.66.0/24
2022-10-15 10:01:59 +00:00
assert_success
assert_output 'Banned custom IP: 1'
2022-10-15 10:01:59 +00:00
_run_in_container fail2ban
2022-10-15 10:01:59 +00:00
assert_success
assert_output --regexp 'Banned in custom:.*192\.0\.66\.0/24'
2022-10-15 10:01:59 +00:00
_run_in_container nft list set inet f2b-table addr-set-custom
2022-10-15 10:01:59 +00:00
assert_success
assert_output --partial 'elements = { 192.0.66.0/24 }'
2022-10-15 10:01:59 +00:00
_run_in_container fail2ban unban 192.0.66.0/24
2022-10-15 10:01:59 +00:00
assert_success
assert_output --partial 'Unbanned IP from custom: 1'
2022-10-15 10:01:59 +00:00
_run_in_container nft list set inet f2b-table addr-set-custom
refute_output --partial '192.0.66.0/24'
2022-10-05 10:19:49 +00:00
}
@test "FAIL2BAN_BLOCKTYPE is really set to drop" {
# ban IPs here manually so we can be sure something is inside the jails
for JAIL in dovecot postfix-sasl custom; do
_run_in_container fail2ban-client set "${JAIL}" banip 192.33.44.55
assert_success
done
_run_in_container nft list table inet f2b-table
2022-10-05 10:19:49 +00:00
assert_success
assert_output --partial 'tcp dport { 110, 143, 465, 587, 993, 995, 4190 } ip saddr @addr-set-dovecot drop'
assert_output --partial 'tcp dport { 25, 110, 143, 465, 587, 993, 995 } ip saddr @addr-set-postfix-sasl drop'
assert_output --partial 'tcp dport { 25, 110, 143, 465, 587, 993, 995, 4190 } ip saddr @addr-set-custom drop'
# unban the IPs previously banned to get a clean state again
for JAIL in dovecot postfix-sasl custom; do
_run_in_container fail2ban-client set "${JAIL}" unbanip 192.33.44.55
assert_success
done
}
@test "setup.sh fail2ban" {
_run_in_container fail2ban-client set dovecot banip 192.0.66.4
_run_in_container fail2ban-client set dovecot banip 192.0.66.5
2021-05-15 09:11:10 +00:00
# Originally: run ./setup.sh -c "${CONTAINER1_NAME}" fail2ban
_run_in_container setup fail2ban
assert_output --regexp '^Banned in dovecot:.*192\.0\.66\.4'
assert_output --regexp '^Banned in dovecot:.*192\.0\.66\.5'
2021-05-15 09:11:10 +00:00
_run_in_container setup fail2ban unban 192.0.66.4
2021-05-15 09:11:10 +00:00
assert_output --partial "Unbanned IP from dovecot: 1"
_run_in_container setup fail2ban
assert_output --regexp '^Banned in dovecot:.*192\.0\.66\.5'
2021-05-15 09:11:10 +00:00
_run_in_container setup fail2ban unban 192.0.66.5
assert_output --partial 'Unbanned IP from dovecot: 1'
2021-05-15 09:11:10 +00:00
_run_in_container setup fail2ban unban
assert_output --partial 'You need to specify an IP address: Run'
}