docker-mailserver/test/tests/serial/test_helper.bats
Brennan Kinney 8d80c6317f
tests(refactor): Adjust mail_changedetector + change detection helpers (#2997)
* tests(refactor): `mail_changedetector.bats` - Leverage DRY methods

`supervisorctl tail` is not the most reliably way to get logs for the latest change detection and has been known to be fragile in the past.

We can instead read the full log for the service directly with `tac` and `sed` to extract all log content since the last change detection.

Common asserts have also been extracted out into separate methods.

* tests(chore): Remove sleep and redundant change event

Container 1 is still blocked at this point from an existing lock and change event.

Make the lock stale immediately and no extra sleep is required when paired with the helper method to wait until the event is processed (which should remove the stale lock).

* tests(refactor): Add more DRY methods

- Simplify the test case so it's easier to grok.
- 2nd test case (blocking) extracts out initial setup into a separate method and merges the later service restart logic which is redundant.
- Additional comments for improved context of what is going on / expected.

* tests(chore): Revise the change detection helper method

- Add explicit counting arg to change detection support.
- Extract revised logic into it's own generic helper method.
- Utilize this for a separate method that monitors for a change event having started, but not waiting for completion.

This allows dropping the 40 sec of remaining `sleep` in `mail_changedetector` test. It was also required due to potentially missing the timing of a change event completing concurrently in a 2nd container that needed to be waited on and then checked.

* tests(chore): Migrate to current test conventions

- Switch to common container setup helpers
- Update container name and change usage to variables instead.
- Adopt the new convention of prefix variable for test cases (revised test case descriptions).

* tests(chore): Remove legacy change detection

This has since been replaced with the new helper watches the `changedetector` service logs directly instead of only detecting a change has occurred via checksum comparison.

No tests use this method anymore, it was originally for `tests.bats`. Thus the tests in `test_helper.bats` are being dropped too. The new helper has test coverage in `changedetector` tests.

* chore: Lock removal should not incur `sleep 5` afterwards

- A new lock should be created by this script after removal. The sleep doesn't help avoid a race condition with lock file creation after removal.
- Reduces test time as a bonus.
- Added some additional comments to test.

* tests(chore): `tls_letsencrypt.bats` leverage improved change detection

- No need to wait on the change detection service anymore during container startup.
- No need to count change events processed either as waiting a fixed duration is no longer relied on.
- This makes the reload count method redundant, dropped.

* tests(chore): Convert `setup-cli.bats` to new test conventions

This test file was already adapted to the original common setup helpers.

- `TEST_NAME` replaced with `CONTAINER_NAME`.
- Prefix var added, test case descriptions drop explicit prefix.
- No other changes.

* tests(chore): Extract out helpers related to change-detection

- New helper file for sharing these helpers to tests.
- Includes the helpful log method from changedetector tests.
- No longer need to maintain duplicate copies of these methods during the test migration. All tests that use them are now importing the separate helper file.
- `tls_letsencrypt.bats` has switched to using the log helper.
- Generic log count helper is removed from `test_helper/common.bash` as any test that needs it in future can adapt to `helper/common.bash`.

* tests(refactor): `tls_letsencrypt.bats` remove `_get_service_logs()`

This helper does not seem useful as moving away from `supervisorctl tail` and no other tests had a need for it.

* tests(chore): Remove common setup methods from `test_helper/common.bash`

No other tests depend on this. Future tests will adopt the revised versions from `helper/setup.bash`.

Additionally updates `helper/setup.bash` comments that are no longer applicable to `TEST_TMP_CONFIG` and `CONTAINER_NAME`.

* chore: Use `|| true` to simplify setting `EXPECTED_COUNT` correctly
2023-01-16 20:39:46 +13:00

221 lines
7 KiB
Bash

load "${REPOSITORY_ROOT}/test/test_helper/common"
BATS_TEST_NAME_PREFIX='test helper functions:'
@test "repeat_until_success_or_timeout returns instantly on success" {
SECONDS=0
repeat_until_success_or_timeout 1 true
[[ ${SECONDS} -le 1 ]]
}
@test "repeat_until_success_or_timeout waits for timeout on persistent failure" {
SECONDS=0
run repeat_until_success_or_timeout 2 false
[[ ${SECONDS} -ge 2 ]]
assert_failure
assert_output --partial "Timed out on command"
}
@test "repeat_until_success_or_timeout aborts immediately on fatal failure" {
SECONDS=0
run repeat_until_success_or_timeout --fatal-test false 2 false
[[ ${SECONDS} -le 1 ]]
assert_failure
assert_output --partial "early aborting"
}
@test "repeat_until_success_or_timeout expects integer timeout" {
run repeat_until_success_or_timeout 1 true
assert_success
run repeat_until_success_or_timeout timeout true
assert_failure
run repeat_until_success_or_timeout --fatal-test true timeout true
assert_failure
}
@test "run_until_success_or_timeout returns instantly on success" {
SECONDS=0
run_until_success_or_timeout 2 true
[[ ${SECONDS} -le 1 ]]
assert_success
}
@test "run_until_success_or_timeout waits for timeout on persistent failure" {
SECONDS=0
! run_until_success_or_timeout 2 false
[[ ${SECONDS} -ge 2 ]]
assert_failure
}
@test "repeat_in_container_until_success_or_timeout fails immediately for non-running container" {
SECONDS=0
! repeat_in_container_until_success_or_timeout 10 name-of-non-existing-container true
[[ ${SECONDS} -le 1 ]]
}
@test "repeat_in_container_until_success_or_timeout run command in container" {
local CONTAINER_NAME
CONTAINER_NAME=$(docker run --rm -d alpine sleep 100)
SECONDS=0
! repeat_in_container_until_success_or_timeout 10 "${CONTAINER_NAME}" sh -c "echo '${CONTAINER_NAME}' > /tmp/marker"
[[ ${SECONDS} -le 1 ]]
run docker exec "${CONTAINER_NAME}" cat /tmp/marker
assert_output "${CONTAINER_NAME}"
}
@test "container_is_running" {
local CONTAINER_NAME
CONTAINER_NAME=$(docker run --rm -d alpine sleep 100)
container_is_running "${CONTAINER_NAME}"
docker rm -f "${CONTAINER_NAME}"
! container_is_running "${CONTAINER_NAME}"
}
@test "wait_for_smtp_port_in_container aborts wait after timeout" {
local CONTAINER_NAME
CONTAINER_NAME=$(docker run --rm -d alpine sleep 100)
SECONDS=0
TEST_TIMEOUT_IN_SECONDS=2 run wait_for_smtp_port_in_container "${CONTAINER_NAME}"
[[ ${SECONDS} -ge 2 ]]
assert_failure
assert_output --partial "Timed out on command"
}
# NOTE: Test requires external network access available
@test "wait_for_smtp_port_in_container returns immediately when port found" {
local CONTAINER_NAME
CONTAINER_NAME=$(docker run --rm -d alpine sh -c "sleep 100")
docker exec "${CONTAINER_NAME}" apk add netcat-openbsd
docker exec "${CONTAINER_NAME}" nc -l 25 &
SECONDS=0
TEST_TIMEOUT_IN_SECONDS=5 run wait_for_smtp_port_in_container "${CONTAINER_NAME}"
[[ ${SECONDS} -lt 5 ]]
assert_success
}
@test "wait_for_finished_setup_in_container" {
# variable not local to make visible to teardown
local PRIVATE_CONFIG
PRIVATE_CONFIG=$(duplicate_config_for_container .)
CONTAINER_NAME=$(docker run -d --rm \
-v "${PRIVATE_CONFIG}":/tmp/docker-mailserver \
-h mail.my-domain.com \
-t "${NAME}")
teardown() { docker rm -f "${CONTAINER_NAME}"; }
# the setup should not be finished immediately after starting
! TEST_TIMEOUT_IN_SECONDS=0 wait_for_finished_setup_in_container "${CONTAINER_NAME}"
# but it will finish eventually
SECONDS=1
wait_for_finished_setup_in_container "${CONTAINER_NAME}"
[[ ${SECONDS} -gt 0 ]]
}
@test "duplicate_config_for_container" {
local path
path=$(duplicate_config_for_container duplicate_config_test)
run cat "${path}/marker"
assert_line "This marker file is there to identify the correct config being copied"
run duplicate_config_for_container non-existent-source-folder "${BATS_TEST_NAME}2"
assert_failure
}
@test "container_has_service_running/wait_for_service" {
local PRIVATE_CONFIG
PRIVATE_CONFIG=$(duplicate_config_for_container .)
# variable not local to make visible to teardown
CONTAINER_NAME=$(docker run -d --rm \
-v "${PRIVATE_CONFIG}":/tmp/docker-mailserver \
-h mail.my-domain.com \
-t "${NAME}")
teardown() { docker rm -f "${CONTAINER_NAME}"; }
# pick a service that was not started
! container_has_service_running "${CONTAINER_NAME}" clamav
# wait for a service that should be started
wait_for_service "${CONTAINER_NAME}" postfix
# shut down the service
docker exec "${CONTAINER_NAME}" supervisorctl stop postfix
# now it should be off
SECONDS=0
TEST_TIMEOUT_IN_SECONDS=5 run wait_for_service "${CONTAINER_NAME}" postfix
[[ ${SECONDS} -ge 5 ]]
assert_failure
}
# TODO investigate why this test fails
@test "wait_for_empty_mail_queue_in_container fails when timeout reached" {
skip 'disabled as it fails randomly: https://github.com/docker-mailserver/docker-mailserver/pull/2177'
local PRIVATE_CONFIG
PRIVATE_CONFIG=$(duplicate_config_for_container .)
# variable not local to make visible to teardown
# enable ClamAV to make message delivery slower, so we can detect it
CONTAINER_NAME=$(docker run -d --rm \
-v "${PRIVATE_CONFIG}":/tmp/docker-mailserver \
-v "$(pwd)/test/test-files":/tmp/docker-mailserver-test:ro \
-e ENABLE_CLAMAV=1 \
-h mail.my-domain.com \
-t "${NAME}")
teardown() { docker rm -f "${CONTAINER_NAME}"; }
wait_for_smtp_port_in_container "${CONTAINER_NAME}" || docker logs "${CONTAINER_NAME}"
SECONDS=0
# no mails -> should return immediately
TEST_TIMEOUT_IN_SECONDS=5 wait_for_empty_mail_queue_in_container "${CONTAINER_NAME}"
[[ ${SECONDS} -lt 5 ]]
# fill the queue with a message
docker exec "${CONTAINER_NAME}" /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/amavis-virus.txt"
# that should still be stuck in the queue
! TEST_TIMEOUT_IN_SECONDS=0 wait_for_empty_mail_queue_in_container "${CONTAINER_NAME}"
}
# TODO investigate why this test fails
@test "wait_for_empty_mail_queue_in_container succeeds within timeout" {
skip 'disabled as it fails randomly: https://github.com/docker-mailserver/docker-mailserver/pull/2177'
local PRIVATE_CONFIG
PRIVATE_CONFIG=$(duplicate_config_for_container .)
# variable not local to make visible to teardown
# enable ClamAV to make message delivery slower, so we can detect it
CONTAINER_NAME=$(docker run -d --rm \
-v "${PRIVATE_CONFIG}":/tmp/docker-mailserver \
-v "$(pwd)/test/test-files":/tmp/docker-mailserver-test:ro \
-e ENABLE_CLAMAV=1 \
-h mail.my-domain.com \
-t "${NAME}")
teardown() { docker rm -f "${CONTAINER_NAME}"; }
wait_for_smtp_port_in_container "${CONTAINER_NAME}" || docker logs "${CONTAINER_NAME}"
# fill the queue with a message
docker exec "${CONTAINER_NAME}" /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/amavis-virus.txt"
# give it some time to clear the queue
SECONDS=0
wait_for_empty_mail_queue_in_container "${CONTAINER_NAME}"
[[ ${SECONDS} -gt 0 ]]
}