docker-mailserver/test/tests/serial/mail_changedetector.bats

88 lines
3.9 KiB
Plaintext
Raw Normal View History

load "${REPOSITORY_ROOT}/test/test_helper/common"
fix: `check-for-changes.sh` should not fall out of sync with shared logic (#2260) Removes duplicate logic from `check-for-changes.sh` that is used/maintained elsewhere to avoid risk of problems, as this code is already starting to diverge / rot. --- Previously the change detection support has had code added for rebuilding config upon change detection which is the same as code run during startup scripts. Unfortunately over time this has fallen out of sync. Mostly the startup scripts would get maintenance and the contributor and reviewers may not have been aware of the duplicate code handled by `check-for-changes.sh`. That code was starting to diverge in addition to some changes in structure (_eg: relay host logic seems interleaved here vs separated out in startup scripts_). I wanted to address this before it risks becoming a much bigger headache. Rather than bloat `helper-functions.sh` further, I've added a `helpers/` folder extracting relevant common logic between startup scripts and `changedetector`. If you want to follow that process I've kept scoped commits to make those diffs easier. Some minor changes/improvements were added but nothing significant. --- - chore: Extract relay host logic to new `relay.sh` helper - chore: Extract `/etc/postfix/sasl_passwd` logic to new `sasl.sh` helper - chore: Extract `postfix-accounts.cf` logic to new `accounts.sh` helper - chore: Extract `/etc/aliases` logic to new `aliases.sh` helper - chore: Extract `/etc/postfix/vhost` logic to new `postfix.sh` helper - chore: Add inline docs for Postfix configs > These are possibly more verbose than needed and can be reduced at a later stage. > They are helpful during this refactor process while investigating that everything is handled correctly. `accounts.sh`: - Add note regarding potential bug for bare domain setups with `/etc/postfix/vhost` and `mydestination` sharing same domain value. `relay.sh`: - Remove the tabs for a single space delimiter, revised associated comment. - Add PR reference for original `_populate_relayhost_map` implementation which has some useful details. Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com> Co-authored-by: Casper <casperklein@users.noreply.github.com>
2021-11-20 20:33:49 +00:00
# Note if tests fail asserting against `supervisorctl tail changedetector` output,
# use `supervisorctl tail -<num bytes> changedetector` instead to increase log output.
# Default `<num bytes>` appears to be around 1500.
function setup_file() {
local PRIVATE_CONFIG
PRIVATE_CONFIG=$(duplicate_config_for_container . mail_changedetector_one)
docker run -d --name mail_changedetector_one \
tests(fix): Adjust for local testing conditions (#2606) * tests(fix): Increase some timeouts Running tests locally via a VM these tests would fail sometimes due to the time from being queued and Amavis actually processing being roughly around 30 seconds. There should be no harm in raising this to 60 seconds, other than delaying a failure case which will ripple through other time sensitive tests. It's better to pass when functionality is actually correct but just needs a bit longer to complete. * tests(fix): Don't setup an invalid hostname During container startup `helpers/dns.sh` would panic with `hostname -f` failing. Dropping `--domainname` for this container is fine and does not affect the point of it's test. --- It's unclear why this does not occur in CI. Possibly changes within the docker daemon since as CI runs docker on Ubuntu 20.04? (2020). For clarity, this may be equivalent to setting a hostname of `domain.com.domain.com`, or `--hostname` value truncated the NIS domain (`--domainname`) of the same value. IIRC, it would still fail with both options using different values if `--hostname` was multi-label. I believe I've documented how non-deterministic these options can be across different environments. `--hostname` should be preferred. There doesn't seem to be any reason to actually need `--domainname` (which is NIS domain name, unrelated to the DNS domain name). We still need to properly investigate reworking our ENV support that `dns.sh` manages. --- Containers were also not removing themselves after failures either (missing teardown). Which would cause problems when running tests again. * chore: Normalize white-space Sets a consistent indent size of 2 spaces. Previously this varied a fair bit, sometimes with tabs or mixed tabs and spaces. Some formatting with blank lines. Easier to review with white-space in diff ignored. Some minor edits besides blank lines, but no change in functionality. * fix: `setup.sh` target container under test Some of the `setup.sh` commands did not specify the container which was problematic if another `docker-mailserver` container was running, causing test failures. This probably doesn't help with `test/no_container.bats`, but at least prevents `test/tests.bats` failing at this point.
2022-05-30 00:53:30 +00:00
-v "${PRIVATE_CONFIG}":/tmp/docker-mailserver \
-v "$(pwd)/test/test-files":/tmp/docker-mailserver-test:ro \
-e LOG_LEVEL=trace \
-h mail.my-domain.com -t "${NAME}"
docker run -d --name mail_changedetector_two \
tests(fix): Adjust for local testing conditions (#2606) * tests(fix): Increase some timeouts Running tests locally via a VM these tests would fail sometimes due to the time from being queued and Amavis actually processing being roughly around 30 seconds. There should be no harm in raising this to 60 seconds, other than delaying a failure case which will ripple through other time sensitive tests. It's better to pass when functionality is actually correct but just needs a bit longer to complete. * tests(fix): Don't setup an invalid hostname During container startup `helpers/dns.sh` would panic with `hostname -f` failing. Dropping `--domainname` for this container is fine and does not affect the point of it's test. --- It's unclear why this does not occur in CI. Possibly changes within the docker daemon since as CI runs docker on Ubuntu 20.04? (2020). For clarity, this may be equivalent to setting a hostname of `domain.com.domain.com`, or `--hostname` value truncated the NIS domain (`--domainname`) of the same value. IIRC, it would still fail with both options using different values if `--hostname` was multi-label. I believe I've documented how non-deterministic these options can be across different environments. `--hostname` should be preferred. There doesn't seem to be any reason to actually need `--domainname` (which is NIS domain name, unrelated to the DNS domain name). We still need to properly investigate reworking our ENV support that `dns.sh` manages. --- Containers were also not removing themselves after failures either (missing teardown). Which would cause problems when running tests again. * chore: Normalize white-space Sets a consistent indent size of 2 spaces. Previously this varied a fair bit, sometimes with tabs or mixed tabs and spaces. Some formatting with blank lines. Easier to review with white-space in diff ignored. Some minor edits besides blank lines, but no change in functionality. * fix: `setup.sh` target container under test Some of the `setup.sh` commands did not specify the container which was problematic if another `docker-mailserver` container was running, causing test failures. This probably doesn't help with `test/no_container.bats`, but at least prevents `test/tests.bats` failing at this point.
2022-05-30 00:53:30 +00:00
-v "${PRIVATE_CONFIG}":/tmp/docker-mailserver \
-v "$(pwd)/test/test-files":/tmp/docker-mailserver-test:ro \
-e LOG_LEVEL=trace \
-h mail.my-domain.com -t "${NAME}"
wait_for_finished_setup_in_container mail_changedetector_one
wait_for_finished_setup_in_container mail_changedetector_two
}
function teardown_file() {
docker rm -f mail_changedetector_one
docker rm -f mail_changedetector_two
}
@test "checking changedetector: servers are ready" {
wait_for_service mail_changedetector_one changedetector
wait_for_service mail_changedetector_two changedetector
}
@test "checking changedetector: can detect changes & between two containers using same config" {
echo "" >> "$(private_config_path mail_changedetector_one)/postfix-accounts.cf"
sleep 25
tests(fix): Adjust for local testing conditions (#2606) * tests(fix): Increase some timeouts Running tests locally via a VM these tests would fail sometimes due to the time from being queued and Amavis actually processing being roughly around 30 seconds. There should be no harm in raising this to 60 seconds, other than delaying a failure case which will ripple through other time sensitive tests. It's better to pass when functionality is actually correct but just needs a bit longer to complete. * tests(fix): Don't setup an invalid hostname During container startup `helpers/dns.sh` would panic with `hostname -f` failing. Dropping `--domainname` for this container is fine and does not affect the point of it's test. --- It's unclear why this does not occur in CI. Possibly changes within the docker daemon since as CI runs docker on Ubuntu 20.04? (2020). For clarity, this may be equivalent to setting a hostname of `domain.com.domain.com`, or `--hostname` value truncated the NIS domain (`--domainname`) of the same value. IIRC, it would still fail with both options using different values if `--hostname` was multi-label. I believe I've documented how non-deterministic these options can be across different environments. `--hostname` should be preferred. There doesn't seem to be any reason to actually need `--domainname` (which is NIS domain name, unrelated to the DNS domain name). We still need to properly investigate reworking our ENV support that `dns.sh` manages. --- Containers were also not removing themselves after failures either (missing teardown). Which would cause problems when running tests again. * chore: Normalize white-space Sets a consistent indent size of 2 spaces. Previously this varied a fair bit, sometimes with tabs or mixed tabs and spaces. Some formatting with blank lines. Easier to review with white-space in diff ignored. Some minor edits besides blank lines, but no change in functionality. * fix: `setup.sh` target container under test Some of the `setup.sh` commands did not specify the container which was problematic if another `docker-mailserver` container was running, causing test failures. This probably doesn't help with `test/no_container.bats`, but at least prevents `test/tests.bats` failing at this point.
2022-05-30 00:53:30 +00:00
run docker exec mail_changedetector_one /bin/bash -c "supervisorctl tail -3000 changedetector"
assert_output --partial "postfix: stopped"
assert_output --partial "postfix: started"
assert_output --partial "Change detected"
assert_output --partial "Removed lock"
tests(fix): Adjust for local testing conditions (#2606) * tests(fix): Increase some timeouts Running tests locally via a VM these tests would fail sometimes due to the time from being queued and Amavis actually processing being roughly around 30 seconds. There should be no harm in raising this to 60 seconds, other than delaying a failure case which will ripple through other time sensitive tests. It's better to pass when functionality is actually correct but just needs a bit longer to complete. * tests(fix): Don't setup an invalid hostname During container startup `helpers/dns.sh` would panic with `hostname -f` failing. Dropping `--domainname` for this container is fine and does not affect the point of it's test. --- It's unclear why this does not occur in CI. Possibly changes within the docker daemon since as CI runs docker on Ubuntu 20.04? (2020). For clarity, this may be equivalent to setting a hostname of `domain.com.domain.com`, or `--hostname` value truncated the NIS domain (`--domainname`) of the same value. IIRC, it would still fail with both options using different values if `--hostname` was multi-label. I believe I've documented how non-deterministic these options can be across different environments. `--hostname` should be preferred. There doesn't seem to be any reason to actually need `--domainname` (which is NIS domain name, unrelated to the DNS domain name). We still need to properly investigate reworking our ENV support that `dns.sh` manages. --- Containers were also not removing themselves after failures either (missing teardown). Which would cause problems when running tests again. * chore: Normalize white-space Sets a consistent indent size of 2 spaces. Previously this varied a fair bit, sometimes with tabs or mixed tabs and spaces. Some formatting with blank lines. Easier to review with white-space in diff ignored. Some minor edits besides blank lines, but no change in functionality. * fix: `setup.sh` target container under test Some of the `setup.sh` commands did not specify the container which was problematic if another `docker-mailserver` container was running, causing test failures. This probably doesn't help with `test/no_container.bats`, but at least prevents `test/tests.bats` failing at this point.
2022-05-30 00:53:30 +00:00
run docker exec mail_changedetector_two /bin/bash -c "supervisorctl tail -3000 changedetector"
assert_output --partial "postfix: stopped"
assert_output --partial "postfix: started"
assert_output --partial "Change detected"
assert_output --partial "Removed lock"
}
@test "checking changedetector: lock file found, blocks, and doesn't get prematurely removed" {
run docker exec mail_changedetector_two /bin/bash -c "supervisorctl stop changedetector"
docker exec mail_changedetector_one /bin/bash -c "touch /tmp/docker-mailserver/check-for-changes.sh.lock"
echo "" >> "$(private_config_path mail_changedetector_one)/postfix-accounts.cf"
run docker exec mail_changedetector_two /bin/bash -c "supervisorctl start changedetector"
sleep 15
tests(fix): Adjust for local testing conditions (#2606) * tests(fix): Increase some timeouts Running tests locally via a VM these tests would fail sometimes due to the time from being queued and Amavis actually processing being roughly around 30 seconds. There should be no harm in raising this to 60 seconds, other than delaying a failure case which will ripple through other time sensitive tests. It's better to pass when functionality is actually correct but just needs a bit longer to complete. * tests(fix): Don't setup an invalid hostname During container startup `helpers/dns.sh` would panic with `hostname -f` failing. Dropping `--domainname` for this container is fine and does not affect the point of it's test. --- It's unclear why this does not occur in CI. Possibly changes within the docker daemon since as CI runs docker on Ubuntu 20.04? (2020). For clarity, this may be equivalent to setting a hostname of `domain.com.domain.com`, or `--hostname` value truncated the NIS domain (`--domainname`) of the same value. IIRC, it would still fail with both options using different values if `--hostname` was multi-label. I believe I've documented how non-deterministic these options can be across different environments. `--hostname` should be preferred. There doesn't seem to be any reason to actually need `--domainname` (which is NIS domain name, unrelated to the DNS domain name). We still need to properly investigate reworking our ENV support that `dns.sh` manages. --- Containers were also not removing themselves after failures either (missing teardown). Which would cause problems when running tests again. * chore: Normalize white-space Sets a consistent indent size of 2 spaces. Previously this varied a fair bit, sometimes with tabs or mixed tabs and spaces. Some formatting with blank lines. Easier to review with white-space in diff ignored. Some minor edits besides blank lines, but no change in functionality. * fix: `setup.sh` target container under test Some of the `setup.sh` commands did not specify the container which was problematic if another `docker-mailserver` container was running, causing test failures. This probably doesn't help with `test/no_container.bats`, but at least prevents `test/tests.bats` failing at this point.
2022-05-30 00:53:30 +00:00
run docker exec mail_changedetector_one /bin/bash -c "supervisorctl tail changedetector"
assert_output --partial "another execution of 'check-for-changes.sh' is happening"
run docker exec mail_changedetector_two /bin/bash -c "supervisorctl tail changedetector"
assert_output --partial "another execution of 'check-for-changes.sh' is happening"
tests(fix): Adjust for local testing conditions (#2606) * tests(fix): Increase some timeouts Running tests locally via a VM these tests would fail sometimes due to the time from being queued and Amavis actually processing being roughly around 30 seconds. There should be no harm in raising this to 60 seconds, other than delaying a failure case which will ripple through other time sensitive tests. It's better to pass when functionality is actually correct but just needs a bit longer to complete. * tests(fix): Don't setup an invalid hostname During container startup `helpers/dns.sh` would panic with `hostname -f` failing. Dropping `--domainname` for this container is fine and does not affect the point of it's test. --- It's unclear why this does not occur in CI. Possibly changes within the docker daemon since as CI runs docker on Ubuntu 20.04? (2020). For clarity, this may be equivalent to setting a hostname of `domain.com.domain.com`, or `--hostname` value truncated the NIS domain (`--domainname`) of the same value. IIRC, it would still fail with both options using different values if `--hostname` was multi-label. I believe I've documented how non-deterministic these options can be across different environments. `--hostname` should be preferred. There doesn't seem to be any reason to actually need `--domainname` (which is NIS domain name, unrelated to the DNS domain name). We still need to properly investigate reworking our ENV support that `dns.sh` manages. --- Containers were also not removing themselves after failures either (missing teardown). Which would cause problems when running tests again. * chore: Normalize white-space Sets a consistent indent size of 2 spaces. Previously this varied a fair bit, sometimes with tabs or mixed tabs and spaces. Some formatting with blank lines. Easier to review with white-space in diff ignored. Some minor edits besides blank lines, but no change in functionality. * fix: `setup.sh` target container under test Some of the `setup.sh` commands did not specify the container which was problematic if another `docker-mailserver` container was running, causing test failures. This probably doesn't help with `test/no_container.bats`, but at least prevents `test/tests.bats` failing at this point.
2022-05-30 00:53:30 +00:00
# Ensure starting a new check-for-changes.sh instance (restarting here) doesn't delete the lock
docker exec mail_changedetector_two /bin/bash -c "rm -f /var/log/supervisor/changedetector.log"
run docker exec mail_changedetector_two /bin/bash -c "supervisorctl restart changedetector"
sleep 5
run docker exec mail_changedetector_two /bin/bash -c "supervisorctl tail changedetector"
refute_output --partial "another execution of 'check-for-changes.sh' is happening"
refute_output --partial "Removed lock"
}
@test "checking changedetector: lock stale and cleaned up" {
docker rm -f mail_changedetector_two
docker exec mail_changedetector_one /bin/bash -c "touch /tmp/docker-mailserver/check-for-changes.sh.lock"
echo "" >> "$(private_config_path mail_changedetector_one)/postfix-accounts.cf"
sleep 15
tests(fix): Adjust for local testing conditions (#2606) * tests(fix): Increase some timeouts Running tests locally via a VM these tests would fail sometimes due to the time from being queued and Amavis actually processing being roughly around 30 seconds. There should be no harm in raising this to 60 seconds, other than delaying a failure case which will ripple through other time sensitive tests. It's better to pass when functionality is actually correct but just needs a bit longer to complete. * tests(fix): Don't setup an invalid hostname During container startup `helpers/dns.sh` would panic with `hostname -f` failing. Dropping `--domainname` for this container is fine and does not affect the point of it's test. --- It's unclear why this does not occur in CI. Possibly changes within the docker daemon since as CI runs docker on Ubuntu 20.04? (2020). For clarity, this may be equivalent to setting a hostname of `domain.com.domain.com`, or `--hostname` value truncated the NIS domain (`--domainname`) of the same value. IIRC, it would still fail with both options using different values if `--hostname` was multi-label. I believe I've documented how non-deterministic these options can be across different environments. `--hostname` should be preferred. There doesn't seem to be any reason to actually need `--domainname` (which is NIS domain name, unrelated to the DNS domain name). We still need to properly investigate reworking our ENV support that `dns.sh` manages. --- Containers were also not removing themselves after failures either (missing teardown). Which would cause problems when running tests again. * chore: Normalize white-space Sets a consistent indent size of 2 spaces. Previously this varied a fair bit, sometimes with tabs or mixed tabs and spaces. Some formatting with blank lines. Easier to review with white-space in diff ignored. Some minor edits besides blank lines, but no change in functionality. * fix: `setup.sh` target container under test Some of the `setup.sh` commands did not specify the container which was problematic if another `docker-mailserver` container was running, causing test failures. This probably doesn't help with `test/no_container.bats`, but at least prevents `test/tests.bats` failing at this point.
2022-05-30 00:53:30 +00:00
run docker exec mail_changedetector_one /bin/bash -c "supervisorctl tail changedetector"
assert_output --partial "another execution of 'check-for-changes.sh' is happening"
sleep 65
tests(fix): Adjust for local testing conditions (#2606) * tests(fix): Increase some timeouts Running tests locally via a VM these tests would fail sometimes due to the time from being queued and Amavis actually processing being roughly around 30 seconds. There should be no harm in raising this to 60 seconds, other than delaying a failure case which will ripple through other time sensitive tests. It's better to pass when functionality is actually correct but just needs a bit longer to complete. * tests(fix): Don't setup an invalid hostname During container startup `helpers/dns.sh` would panic with `hostname -f` failing. Dropping `--domainname` for this container is fine and does not affect the point of it's test. --- It's unclear why this does not occur in CI. Possibly changes within the docker daemon since as CI runs docker on Ubuntu 20.04? (2020). For clarity, this may be equivalent to setting a hostname of `domain.com.domain.com`, or `--hostname` value truncated the NIS domain (`--domainname`) of the same value. IIRC, it would still fail with both options using different values if `--hostname` was multi-label. I believe I've documented how non-deterministic these options can be across different environments. `--hostname` should be preferred. There doesn't seem to be any reason to actually need `--domainname` (which is NIS domain name, unrelated to the DNS domain name). We still need to properly investigate reworking our ENV support that `dns.sh` manages. --- Containers were also not removing themselves after failures either (missing teardown). Which would cause problems when running tests again. * chore: Normalize white-space Sets a consistent indent size of 2 spaces. Previously this varied a fair bit, sometimes with tabs or mixed tabs and spaces. Some formatting with blank lines. Easier to review with white-space in diff ignored. Some minor edits besides blank lines, but no change in functionality. * fix: `setup.sh` target container under test Some of the `setup.sh` commands did not specify the container which was problematic if another `docker-mailserver` container was running, causing test failures. This probably doesn't help with `test/no_container.bats`, but at least prevents `test/tests.bats` failing at this point.
2022-05-30 00:53:30 +00:00
fix: `check-for-changes.sh` should not fall out of sync with shared logic (#2260) Removes duplicate logic from `check-for-changes.sh` that is used/maintained elsewhere to avoid risk of problems, as this code is already starting to diverge / rot. --- Previously the change detection support has had code added for rebuilding config upon change detection which is the same as code run during startup scripts. Unfortunately over time this has fallen out of sync. Mostly the startup scripts would get maintenance and the contributor and reviewers may not have been aware of the duplicate code handled by `check-for-changes.sh`. That code was starting to diverge in addition to some changes in structure (_eg: relay host logic seems interleaved here vs separated out in startup scripts_). I wanted to address this before it risks becoming a much bigger headache. Rather than bloat `helper-functions.sh` further, I've added a `helpers/` folder extracting relevant common logic between startup scripts and `changedetector`. If you want to follow that process I've kept scoped commits to make those diffs easier. Some minor changes/improvements were added but nothing significant. --- - chore: Extract relay host logic to new `relay.sh` helper - chore: Extract `/etc/postfix/sasl_passwd` logic to new `sasl.sh` helper - chore: Extract `postfix-accounts.cf` logic to new `accounts.sh` helper - chore: Extract `/etc/aliases` logic to new `aliases.sh` helper - chore: Extract `/etc/postfix/vhost` logic to new `postfix.sh` helper - chore: Add inline docs for Postfix configs > These are possibly more verbose than needed and can be reduced at a later stage. > They are helpful during this refactor process while investigating that everything is handled correctly. `accounts.sh`: - Add note regarding potential bug for bare domain setups with `/etc/postfix/vhost` and `mydestination` sharing same domain value. `relay.sh`: - Remove the tabs for a single space delimiter, revised associated comment. - Add PR reference for original `_populate_relayhost_map` implementation which has some useful details. Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com> Co-authored-by: Casper <casperklein@users.noreply.github.com>
2021-11-20 20:33:49 +00:00
run docker exec mail_changedetector_one /bin/bash -c "supervisorctl tail -3000 changedetector"
assert_output --partial "removing stale lock file"
}