docker-mailserver/test/tests.bats

1023 lines
37 KiB
Plaintext
Raw Normal View History

load 'test_helper/common'
2020-10-18 00:24:26 +00:00
export IMAGE_NAME
IMAGE_NAME="${NAME}"
setup_file() {
export START_TIME
2020-10-19 13:19:34 +00:00
local PRIVATE_CONFIG
PRIVATE_CONFIG=$(duplicate_config_for_container . mail)
mv "${PRIVATE_CONFIG}/user-patches/user-patches.sh" "${PRIVATE_CONFIG}/user-patches.sh"
tests(fix): Adjust for local testing conditions (#2606) * tests(fix): Increase some timeouts Running tests locally via a VM these tests would fail sometimes due to the time from being queued and Amavis actually processing being roughly around 30 seconds. There should be no harm in raising this to 60 seconds, other than delaying a failure case which will ripple through other time sensitive tests. It's better to pass when functionality is actually correct but just needs a bit longer to complete. * tests(fix): Don't setup an invalid hostname During container startup `helpers/dns.sh` would panic with `hostname -f` failing. Dropping `--domainname` for this container is fine and does not affect the point of it's test. --- It's unclear why this does not occur in CI. Possibly changes within the docker daemon since as CI runs docker on Ubuntu 20.04? (2020). For clarity, this may be equivalent to setting a hostname of `domain.com.domain.com`, or `--hostname` value truncated the NIS domain (`--domainname`) of the same value. IIRC, it would still fail with both options using different values if `--hostname` was multi-label. I believe I've documented how non-deterministic these options can be across different environments. `--hostname` should be preferred. There doesn't seem to be any reason to actually need `--domainname` (which is NIS domain name, unrelated to the DNS domain name). We still need to properly investigate reworking our ENV support that `dns.sh` manages. --- Containers were also not removing themselves after failures either (missing teardown). Which would cause problems when running tests again. * chore: Normalize white-space Sets a consistent indent size of 2 spaces. Previously this varied a fair bit, sometimes with tabs or mixed tabs and spaces. Some formatting with blank lines. Easier to review with white-space in diff ignored. Some minor edits besides blank lines, but no change in functionality. * fix: `setup.sh` target container under test Some of the `setup.sh` commands did not specify the container which was problematic if another `docker-mailserver` container was running, causing test failures. This probably doesn't help with `test/no_container.bats`, but at least prevents `test/tests.bats` failing at this point.
2022-05-30 00:53:30 +00:00
chore: `addmailuser` - Remove delaying completion until `/var/mail` is ready (#2729) ## Quick Summary Resolves a `TODO` task with `addmailuser`. ## Overview The main change is adding three new methods in `common.bash`, which replace the completion delay in `addmailuser` / `setup email add` command. Other than that: - I swapped `sh -c 'addmailuser ...'` to `setup email add ...`. - Improved three tests in `setup-cli.bats` for `setup email add|update|del` (_logic remains effectively the same still_). - Rewrote the `TODO` comment for `setup-cli.bats` test on `setup email del` to better clarify the concern, but the test itself was no longer affected due to changes prior to this PR, so I enabled the commented out assertion. - Removed unnecessary waits. The two `skip` tests in `test/tests.bats` could be enabled again after this PR. - Additional fixes to tests were made during the PR (see discussion comments for details), resolving race conditions. Individual commit messages of the PR provide additional details if helpful. --- ## Relevant commit messages * chore: Remove creation delay in `addmailuser` This was apparently only for supporting tests that need to wait on account creation being ready to test against. As per the removed inline docs, it should be fine to remove once tests are updated to work correctly without it. * tests(feat): Add two new common helper methods `wait_until_account_maildir_exists()` provides the same logic `addmailuser` command was carrying, to wait upon the account dir creation in `/var/mail`. As this was specifically to support tests, it makes more sense as a test method. `add_mail_account_then_wait_until_ready()` was added to handle the common pattern of creating account and waiting on it. An internal assert will ensure the account was successfully created first during the test before attempting to wait. * tests(feat): Add common helper for waiting on change event to be processed The current helper is more complicated for no real benefit, it only detects when a change is made that would trigger a change event in the `changedetector` service. Our usage of this in tests however is only interested in waiting out the completion of the change event. Remove unnecessary change event waits. These waits should not be necessary if handled correctly. * tests: `addmailuser` to `add_mail_account_then_wait_until_ready mail()` This helper method is used where appropriate. - A password is not relevant (optional). - We need to wait on the creation on the account (Dovecot and `/var/mail` directory). * tests: `setup-cli` revise `add`, `update`, `del` tests The delete test was failing as the `/var/mail` directory did not yet exist. There is now a proper delay imposed in the `add` test now shares the same account for both `update` and `del` tests resolving that failure. Additionally tests use better asserts where appropriate and the wait + sleep logic in `add` has been improved (now takes 10 seconds to complete, approx half the time than before). The `del` test TODO while not technically addressed is no longer relevant due to the tests being switched to `-c` option (there is a separate `no container` test file, but it doesn't provide a `del` test). * tests(fix): Ensure Postfix is reachable after waiting on ClamAV There is not much reason to check before waiting on ClamAV. It is more helpful to debug failures from `nc` mail send commands if we know that nothing went wrong inbetween the ClamAV wait time. Additionally added an assertion which should provide more information if this part of the test setup fails again. * tests(fix): Move health check to the top This test is a bit fragile. It relies on defaults for the healthcheck with intervals of 30 seconds. If the check occurs while Postfix is down due a change event from earlier tests and the healthcheck kicks in at that point, then if there is not enough time to refresh the health status from `unhealthy`, the test will fail with a false-positive as Postfix is actually working and up again.. * tests(fix): Wait on directory to be removed Workaround that tries not to introduce heavier delays by waiting on a full change event to complete in the previous `email update` if possible. There is a chance that the account has the folder deleted, but restored from an active change event (for password update, then the account delete).
2022-08-22 22:15:06 +00:00
# `LOG_LEVEL=debug` required for using `wait_until_change_detection_event_completes()`
docker run --rm -d --name mail \
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
-v "${PRIVATE_CONFIG}":/tmp/docker-mailserver \
-v "$(pwd)/test/test-files":/tmp/docker-mailserver-test:ro \
-v "$(pwd)/test/onedir":/var/mail-state \
-e AMAVIS_LOGLEVEL=2 \
-e CLAMAV_MESSAGE_SIZE_LIMIT=30M \
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
-e ENABLE_CLAMAV=1 \
-e ENABLE_MANAGESIEVE=1 \
-e ENABLE_QUOTAS=1 \
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
-e ENABLE_SPAMASSASSIN=1 \
-e ENABLE_SRS=1 \
-e ENABLE_UPDATE_CHECK=0 \
chore: `addmailuser` - Remove delaying completion until `/var/mail` is ready (#2729) ## Quick Summary Resolves a `TODO` task with `addmailuser`. ## Overview The main change is adding three new methods in `common.bash`, which replace the completion delay in `addmailuser` / `setup email add` command. Other than that: - I swapped `sh -c 'addmailuser ...'` to `setup email add ...`. - Improved three tests in `setup-cli.bats` for `setup email add|update|del` (_logic remains effectively the same still_). - Rewrote the `TODO` comment for `setup-cli.bats` test on `setup email del` to better clarify the concern, but the test itself was no longer affected due to changes prior to this PR, so I enabled the commented out assertion. - Removed unnecessary waits. The two `skip` tests in `test/tests.bats` could be enabled again after this PR. - Additional fixes to tests were made during the PR (see discussion comments for details), resolving race conditions. Individual commit messages of the PR provide additional details if helpful. --- ## Relevant commit messages * chore: Remove creation delay in `addmailuser` This was apparently only for supporting tests that need to wait on account creation being ready to test against. As per the removed inline docs, it should be fine to remove once tests are updated to work correctly without it. * tests(feat): Add two new common helper methods `wait_until_account_maildir_exists()` provides the same logic `addmailuser` command was carrying, to wait upon the account dir creation in `/var/mail`. As this was specifically to support tests, it makes more sense as a test method. `add_mail_account_then_wait_until_ready()` was added to handle the common pattern of creating account and waiting on it. An internal assert will ensure the account was successfully created first during the test before attempting to wait. * tests(feat): Add common helper for waiting on change event to be processed The current helper is more complicated for no real benefit, it only detects when a change is made that would trigger a change event in the `changedetector` service. Our usage of this in tests however is only interested in waiting out the completion of the change event. Remove unnecessary change event waits. These waits should not be necessary if handled correctly. * tests: `addmailuser` to `add_mail_account_then_wait_until_ready mail()` This helper method is used where appropriate. - A password is not relevant (optional). - We need to wait on the creation on the account (Dovecot and `/var/mail` directory). * tests: `setup-cli` revise `add`, `update`, `del` tests The delete test was failing as the `/var/mail` directory did not yet exist. There is now a proper delay imposed in the `add` test now shares the same account for both `update` and `del` tests resolving that failure. Additionally tests use better asserts where appropriate and the wait + sleep logic in `add` has been improved (now takes 10 seconds to complete, approx half the time than before). The `del` test TODO while not technically addressed is no longer relevant due to the tests being switched to `-c` option (there is a separate `no container` test file, but it doesn't provide a `del` test). * tests(fix): Ensure Postfix is reachable after waiting on ClamAV There is not much reason to check before waiting on ClamAV. It is more helpful to debug failures from `nc` mail send commands if we know that nothing went wrong inbetween the ClamAV wait time. Additionally added an assertion which should provide more information if this part of the test setup fails again. * tests(fix): Move health check to the top This test is a bit fragile. It relies on defaults for the healthcheck with intervals of 30 seconds. If the check occurs while Postfix is down due a change event from earlier tests and the healthcheck kicks in at that point, then if there is not enough time to refresh the health status from `unhealthy`, the test will fail with a false-positive as Postfix is actually working and up again.. * tests(fix): Wait on directory to be removed Workaround that tries not to introduce heavier delays by waiting on a full change event to complete in the previous `email update` if possible. There is a chance that the account has the folder deleted, but restored from an active change event (for password update, then the account delete).
2022-08-22 22:15:06 +00:00
-e LOG_LEVEL='debug' \
-e PERMIT_DOCKER=container \
-e PERMIT_DOCKER=host \
-e PFLOGSUMM_TRIGGER=logrotate \
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
-e REPORT_RECIPIENT=user1@localhost.localdomain \
-e REPORT_SENDER=report1@mail.my-domain.com \
-e SA_KILL=3.0 \
-e SA_SPAM_SUBJECT="SPAM: " \
-e SA_TAG=-5.0 \
-e SA_TAG2=2.0 \
-e SPAMASSASSIN_SPAM_TO_INBOX=0 \
-e SPOOF_PROTECTION=1 \
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
-e SSL_TYPE='snakeoil' \
-e VIRUSMAILS_DELETE_DELAY=7 \
tests: Ensure excessive FD limits are avoided (#2730) * tests: Ensure excessive FD limits are avoided Processes that run as daemons (`postsrsd` and `fail2ban-server`) initialize by closing all FDs (File Descriptors). This behaviour queries that maximum limit and iterates through the entire range even if only a few FDs are open. In some environments (Docker, limit configured by distro) this can be a range exceeding 1 billion (from kernel default of 1024 soft, 4096 hard), causing an 8 minute delay with heavy CPU activity. `postsrsd` has since been updated to use `close_range()` syscall, and `fail2ban` will now iterate through `/proc/self/fd` (open FDs) which should resolve the performance hit. Until those updates reach our Docker image, we need to workaround it with `--ulimit` option. NOTE: If `docker.service` on a distro sets `LimitNOFILE=` to approx 1 million or lower, it should not be an issue. On distros such as Fedora 36, it is `LimitNOFILE=infinity` (approx 1 billion) that causes excessive delays. * chore: Use Docker host limits instead Typically on modern distros with systemd, this should equate to 1024 (soft) and 512K (hard) limits. A distro may override the built-in global defaults systemd sets via setting `DefaultLimitNOFILE=` in `/etc/systemd/user.conf` and `/etc/systemd/system.conf`. * tests(fix): Better prevent non-deterministic failures - `no_containers.bats` tests the external script `setup.sh` (without `-c`). It's expected that no existing DMS container is running - otherwise it may attempt to use that container and fail. Detect this and fail early via `setup_file()` step. - `mail_hostname.bats` had a odd timing failure with teardown due to the last tests bringing the containers down earlier (`docker stop` paired with the `docker run --rm`). Adding a moment of delay via `sleep` helps avoid that false positive scenario.
2022-08-22 23:24:23 +00:00
--hostname mail.my-domain.com \
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
--tty \
tests: Ensure excessive FD limits are avoided (#2730) * tests: Ensure excessive FD limits are avoided Processes that run as daemons (`postsrsd` and `fail2ban-server`) initialize by closing all FDs (File Descriptors). This behaviour queries that maximum limit and iterates through the entire range even if only a few FDs are open. In some environments (Docker, limit configured by distro) this can be a range exceeding 1 billion (from kernel default of 1024 soft, 4096 hard), causing an 8 minute delay with heavy CPU activity. `postsrsd` has since been updated to use `close_range()` syscall, and `fail2ban` will now iterate through `/proc/self/fd` (open FDs) which should resolve the performance hit. Until those updates reach our Docker image, we need to workaround it with `--ulimit` option. NOTE: If `docker.service` on a distro sets `LimitNOFILE=` to approx 1 million or lower, it should not be an issue. On distros such as Fedora 36, it is `LimitNOFILE=infinity` (approx 1 billion) that causes excessive delays. * chore: Use Docker host limits instead Typically on modern distros with systemd, this should equate to 1024 (soft) and 512K (hard) limits. A distro may override the built-in global defaults systemd sets via setting `DefaultLimitNOFILE=` in `/etc/systemd/user.conf` and `/etc/systemd/system.conf`. * tests(fix): Better prevent non-deterministic failures - `no_containers.bats` tests the external script `setup.sh` (without `-c`). It's expected that no existing DMS container is running - otherwise it may attempt to use that container and fail. Detect this and fail early via `setup_file()` step. - `mail_hostname.bats` had a odd timing failure with teardown due to the last tests bringing the containers down earlier (`docker stop` paired with the `docker run --rm`). Adding a moment of delay via `sleep` helps avoid that false positive scenario.
2022-08-22 23:24:23 +00:00
--ulimit "nofile=$(ulimit -Sn):$(ulimit -Hn)" \
--health-cmd "ss --listening --tcp | grep -P 'LISTEN.+:smtp' || exit 1" \
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
"${NAME}"
START_TIME=$(date +%s)
wait_for_finished_setup_in_container mail
# generate accounts after container has been started
docker run --rm -e MAIL_USER=added@localhost.localdomain -e MAIL_PASS=mypassword -t "${NAME}" /bin/sh -c 'echo "${MAIL_USER}|$(doveadm pw -s SHA512-CRYPT -u ${MAIL_USER} -p ${MAIL_PASS})"' >> "${PRIVATE_CONFIG}/postfix-accounts.cf"
docker exec mail addmailuser pass@localhost.localdomain 'may be \a `p^a.*ssword'
# setup sieve
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
docker cp "${PRIVATE_CONFIG}/sieve/dovecot.sieve" mail:/var/mail/localhost.localdomain/user1/.dovecot.sieve
chore: `addmailuser` - Remove delaying completion until `/var/mail` is ready (#2729) ## Quick Summary Resolves a `TODO` task with `addmailuser`. ## Overview The main change is adding three new methods in `common.bash`, which replace the completion delay in `addmailuser` / `setup email add` command. Other than that: - I swapped `sh -c 'addmailuser ...'` to `setup email add ...`. - Improved three tests in `setup-cli.bats` for `setup email add|update|del` (_logic remains effectively the same still_). - Rewrote the `TODO` comment for `setup-cli.bats` test on `setup email del` to better clarify the concern, but the test itself was no longer affected due to changes prior to this PR, so I enabled the commented out assertion. - Removed unnecessary waits. The two `skip` tests in `test/tests.bats` could be enabled again after this PR. - Additional fixes to tests were made during the PR (see discussion comments for details), resolving race conditions. Individual commit messages of the PR provide additional details if helpful. --- ## Relevant commit messages * chore: Remove creation delay in `addmailuser` This was apparently only for supporting tests that need to wait on account creation being ready to test against. As per the removed inline docs, it should be fine to remove once tests are updated to work correctly without it. * tests(feat): Add two new common helper methods `wait_until_account_maildir_exists()` provides the same logic `addmailuser` command was carrying, to wait upon the account dir creation in `/var/mail`. As this was specifically to support tests, it makes more sense as a test method. `add_mail_account_then_wait_until_ready()` was added to handle the common pattern of creating account and waiting on it. An internal assert will ensure the account was successfully created first during the test before attempting to wait. * tests(feat): Add common helper for waiting on change event to be processed The current helper is more complicated for no real benefit, it only detects when a change is made that would trigger a change event in the `changedetector` service. Our usage of this in tests however is only interested in waiting out the completion of the change event. Remove unnecessary change event waits. These waits should not be necessary if handled correctly. * tests: `addmailuser` to `add_mail_account_then_wait_until_ready mail()` This helper method is used where appropriate. - A password is not relevant (optional). - We need to wait on the creation on the account (Dovecot and `/var/mail` directory). * tests: `setup-cli` revise `add`, `update`, `del` tests The delete test was failing as the `/var/mail` directory did not yet exist. There is now a proper delay imposed in the `add` test now shares the same account for both `update` and `del` tests resolving that failure. Additionally tests use better asserts where appropriate and the wait + sleep logic in `add` has been improved (now takes 10 seconds to complete, approx half the time than before). The `del` test TODO while not technically addressed is no longer relevant due to the tests being switched to `-c` option (there is a separate `no container` test file, but it doesn't provide a `del` test). * tests(fix): Ensure Postfix is reachable after waiting on ClamAV There is not much reason to check before waiting on ClamAV. It is more helpful to debug failures from `nc` mail send commands if we know that nothing went wrong inbetween the ClamAV wait time. Additionally added an assertion which should provide more information if this part of the test setup fails again. * tests(fix): Move health check to the top This test is a bit fragile. It relies on defaults for the healthcheck with intervals of 30 seconds. If the check occurs while Postfix is down due a change event from earlier tests and the healthcheck kicks in at that point, then if there is not enough time to refresh the health status from `unhealthy`, the test will fail with a false-positive as Postfix is actually working and up again.. * tests(fix): Wait on directory to be removed Workaround that tries not to introduce heavier delays by waiting on a full change event to complete in the previous `email update` if possible. There is a chance that the account has the folder deleted, but restored from an active change event (for password update, then the account delete).
2022-08-22 22:15:06 +00:00
# this relies on the checksum file being updated after all changes have been applied
wait_until_change_detection_event_completes mail
# wait for ClamAV to be fully setup or we will get errors on the log
repeat_in_container_until_success_or_timeout 60 mail test -e /var/run/clamav/clamd.ctl
chore: `addmailuser` - Remove delaying completion until `/var/mail` is ready (#2729) ## Quick Summary Resolves a `TODO` task with `addmailuser`. ## Overview The main change is adding three new methods in `common.bash`, which replace the completion delay in `addmailuser` / `setup email add` command. Other than that: - I swapped `sh -c 'addmailuser ...'` to `setup email add ...`. - Improved three tests in `setup-cli.bats` for `setup email add|update|del` (_logic remains effectively the same still_). - Rewrote the `TODO` comment for `setup-cli.bats` test on `setup email del` to better clarify the concern, but the test itself was no longer affected due to changes prior to this PR, so I enabled the commented out assertion. - Removed unnecessary waits. The two `skip` tests in `test/tests.bats` could be enabled again after this PR. - Additional fixes to tests were made during the PR (see discussion comments for details), resolving race conditions. Individual commit messages of the PR provide additional details if helpful. --- ## Relevant commit messages * chore: Remove creation delay in `addmailuser` This was apparently only for supporting tests that need to wait on account creation being ready to test against. As per the removed inline docs, it should be fine to remove once tests are updated to work correctly without it. * tests(feat): Add two new common helper methods `wait_until_account_maildir_exists()` provides the same logic `addmailuser` command was carrying, to wait upon the account dir creation in `/var/mail`. As this was specifically to support tests, it makes more sense as a test method. `add_mail_account_then_wait_until_ready()` was added to handle the common pattern of creating account and waiting on it. An internal assert will ensure the account was successfully created first during the test before attempting to wait. * tests(feat): Add common helper for waiting on change event to be processed The current helper is more complicated for no real benefit, it only detects when a change is made that would trigger a change event in the `changedetector` service. Our usage of this in tests however is only interested in waiting out the completion of the change event. Remove unnecessary change event waits. These waits should not be necessary if handled correctly. * tests: `addmailuser` to `add_mail_account_then_wait_until_ready mail()` This helper method is used where appropriate. - A password is not relevant (optional). - We need to wait on the creation on the account (Dovecot and `/var/mail` directory). * tests: `setup-cli` revise `add`, `update`, `del` tests The delete test was failing as the `/var/mail` directory did not yet exist. There is now a proper delay imposed in the `add` test now shares the same account for both `update` and `del` tests resolving that failure. Additionally tests use better asserts where appropriate and the wait + sleep logic in `add` has been improved (now takes 10 seconds to complete, approx half the time than before). The `del` test TODO while not technically addressed is no longer relevant due to the tests being switched to `-c` option (there is a separate `no container` test file, but it doesn't provide a `del` test). * tests(fix): Ensure Postfix is reachable after waiting on ClamAV There is not much reason to check before waiting on ClamAV. It is more helpful to debug failures from `nc` mail send commands if we know that nothing went wrong inbetween the ClamAV wait time. Additionally added an assertion which should provide more information if this part of the test setup fails again. * tests(fix): Move health check to the top This test is a bit fragile. It relies on defaults for the healthcheck with intervals of 30 seconds. If the check occurs while Postfix is down due a change event from earlier tests and the healthcheck kicks in at that point, then if there is not enough time to refresh the health status from `unhealthy`, the test will fail with a false-positive as Postfix is actually working and up again.. * tests(fix): Wait on directory to be removed Workaround that tries not to introduce heavier delays by waiting on a full change event to complete in the previous `email update` if possible. There is a chance that the account has the folder deleted, but restored from an active change event (for password update, then the account delete).
2022-08-22 22:15:06 +00:00
wait_for_smtp_port_in_container mail
# The first mail sent leverages an assert for better error output if a failure occurs:
run docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/amavis-spam.txt"
assert_success
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/amavis-virus.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-alias-external.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-alias-local.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-alias-recipient-delimiter.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-user1.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-user2.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-user3.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-added.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-user-and-cc-local-alias.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-regexp-alias-external.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-regexp-alias-local.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/existing-catchall-local.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/sieve-spam-folder.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/sieve-pipe.txt"
docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/non-existing-user.txt"
docker exec mail /bin/sh -c "sendmail root < /tmp/docker-mailserver-test/email-templates/root-email.txt"
wait_for_empty_mail_queue_in_container mail
}
teardown_file() {
docker rm -f mail
}
#
# configuration checks
#
@test "checking configuration: user-patches.sh executed" {
2020-10-18 00:24:26 +00:00
run docker logs mail
2019-11-08 01:22:33 +00:00
assert_output --partial "Default user-patches.sh successfully executed"
}
@test "checking configuration: hostname/domainname" {
2020-10-18 00:24:26 +00:00
run docker run "${IMAGE_NAME:?}"
assert_success
}
chore: `addmailuser` - Remove delaying completion until `/var/mail` is ready (#2729) ## Quick Summary Resolves a `TODO` task with `addmailuser`. ## Overview The main change is adding three new methods in `common.bash`, which replace the completion delay in `addmailuser` / `setup email add` command. Other than that: - I swapped `sh -c 'addmailuser ...'` to `setup email add ...`. - Improved three tests in `setup-cli.bats` for `setup email add|update|del` (_logic remains effectively the same still_). - Rewrote the `TODO` comment for `setup-cli.bats` test on `setup email del` to better clarify the concern, but the test itself was no longer affected due to changes prior to this PR, so I enabled the commented out assertion. - Removed unnecessary waits. The two `skip` tests in `test/tests.bats` could be enabled again after this PR. - Additional fixes to tests were made during the PR (see discussion comments for details), resolving race conditions. Individual commit messages of the PR provide additional details if helpful. --- ## Relevant commit messages * chore: Remove creation delay in `addmailuser` This was apparently only for supporting tests that need to wait on account creation being ready to test against. As per the removed inline docs, it should be fine to remove once tests are updated to work correctly without it. * tests(feat): Add two new common helper methods `wait_until_account_maildir_exists()` provides the same logic `addmailuser` command was carrying, to wait upon the account dir creation in `/var/mail`. As this was specifically to support tests, it makes more sense as a test method. `add_mail_account_then_wait_until_ready()` was added to handle the common pattern of creating account and waiting on it. An internal assert will ensure the account was successfully created first during the test before attempting to wait. * tests(feat): Add common helper for waiting on change event to be processed The current helper is more complicated for no real benefit, it only detects when a change is made that would trigger a change event in the `changedetector` service. Our usage of this in tests however is only interested in waiting out the completion of the change event. Remove unnecessary change event waits. These waits should not be necessary if handled correctly. * tests: `addmailuser` to `add_mail_account_then_wait_until_ready mail()` This helper method is used where appropriate. - A password is not relevant (optional). - We need to wait on the creation on the account (Dovecot and `/var/mail` directory). * tests: `setup-cli` revise `add`, `update`, `del` tests The delete test was failing as the `/var/mail` directory did not yet exist. There is now a proper delay imposed in the `add` test now shares the same account for both `update` and `del` tests resolving that failure. Additionally tests use better asserts where appropriate and the wait + sleep logic in `add` has been improved (now takes 10 seconds to complete, approx half the time than before). The `del` test TODO while not technically addressed is no longer relevant due to the tests being switched to `-c` option (there is a separate `no container` test file, but it doesn't provide a `del` test). * tests(fix): Ensure Postfix is reachable after waiting on ClamAV There is not much reason to check before waiting on ClamAV. It is more helpful to debug failures from `nc` mail send commands if we know that nothing went wrong inbetween the ClamAV wait time. Additionally added an assertion which should provide more information if this part of the test setup fails again. * tests(fix): Move health check to the top This test is a bit fragile. It relies on defaults for the healthcheck with intervals of 30 seconds. If the check occurs while Postfix is down due a change event from earlier tests and the healthcheck kicks in at that point, then if there is not enough time to refresh the health status from `unhealthy`, the test will fail with a false-positive as Postfix is actually working and up again.. * tests(fix): Wait on directory to be removed Workaround that tries not to introduce heavier delays by waiting on a full change event to complete in the previous `email update` if possible. There is a chance that the account has the folder deleted, but restored from an active change event (for password update, then the account delete).
2022-08-22 22:15:06 +00:00
#
# healthcheck
#
# NOTE: Healthcheck defaults an interval of 30 seconds
# If Postfix is temporarily down (eg: restart triggered by `check-for-changes.sh`),
# it may result in a false-positive `unhealthy` state.
# Be careful with re-locating this test if earlier tests could potentially fail it by
# triggering the `changedetector` service.
@test "checking container healthcheck" {
local NOW
NOW=$(date +%s)
# ensure, that at least 30 seconds have passed since container start
while (( NOW - START_TIME < 31 )); do
sleep 1
NOW=$(date +%s)
done
chore: `addmailuser` - Remove delaying completion until `/var/mail` is ready (#2729) ## Quick Summary Resolves a `TODO` task with `addmailuser`. ## Overview The main change is adding three new methods in `common.bash`, which replace the completion delay in `addmailuser` / `setup email add` command. Other than that: - I swapped `sh -c 'addmailuser ...'` to `setup email add ...`. - Improved three tests in `setup-cli.bats` for `setup email add|update|del` (_logic remains effectively the same still_). - Rewrote the `TODO` comment for `setup-cli.bats` test on `setup email del` to better clarify the concern, but the test itself was no longer affected due to changes prior to this PR, so I enabled the commented out assertion. - Removed unnecessary waits. The two `skip` tests in `test/tests.bats` could be enabled again after this PR. - Additional fixes to tests were made during the PR (see discussion comments for details), resolving race conditions. Individual commit messages of the PR provide additional details if helpful. --- ## Relevant commit messages * chore: Remove creation delay in `addmailuser` This was apparently only for supporting tests that need to wait on account creation being ready to test against. As per the removed inline docs, it should be fine to remove once tests are updated to work correctly without it. * tests(feat): Add two new common helper methods `wait_until_account_maildir_exists()` provides the same logic `addmailuser` command was carrying, to wait upon the account dir creation in `/var/mail`. As this was specifically to support tests, it makes more sense as a test method. `add_mail_account_then_wait_until_ready()` was added to handle the common pattern of creating account and waiting on it. An internal assert will ensure the account was successfully created first during the test before attempting to wait. * tests(feat): Add common helper for waiting on change event to be processed The current helper is more complicated for no real benefit, it only detects when a change is made that would trigger a change event in the `changedetector` service. Our usage of this in tests however is only interested in waiting out the completion of the change event. Remove unnecessary change event waits. These waits should not be necessary if handled correctly. * tests: `addmailuser` to `add_mail_account_then_wait_until_ready mail()` This helper method is used where appropriate. - A password is not relevant (optional). - We need to wait on the creation on the account (Dovecot and `/var/mail` directory). * tests: `setup-cli` revise `add`, `update`, `del` tests The delete test was failing as the `/var/mail` directory did not yet exist. There is now a proper delay imposed in the `add` test now shares the same account for both `update` and `del` tests resolving that failure. Additionally tests use better asserts where appropriate and the wait + sleep logic in `add` has been improved (now takes 10 seconds to complete, approx half the time than before). The `del` test TODO while not technically addressed is no longer relevant due to the tests being switched to `-c` option (there is a separate `no container` test file, but it doesn't provide a `del` test). * tests(fix): Ensure Postfix is reachable after waiting on ClamAV There is not much reason to check before waiting on ClamAV. It is more helpful to debug failures from `nc` mail send commands if we know that nothing went wrong inbetween the ClamAV wait time. Additionally added an assertion which should provide more information if this part of the test setup fails again. * tests(fix): Move health check to the top This test is a bit fragile. It relies on defaults for the healthcheck with intervals of 30 seconds. If the check occurs while Postfix is down due a change event from earlier tests and the healthcheck kicks in at that point, then if there is not enough time to refresh the health status from `unhealthy`, the test will fail with a false-positive as Postfix is actually working and up again.. * tests(fix): Wait on directory to be removed Workaround that tries not to introduce heavier delays by waiting on a full change event to complete in the previous `email update` if possible. There is a chance that the account has the folder deleted, but restored from an active change event (for password update, then the account delete).
2022-08-22 22:15:06 +00:00
run bash -c "docker inspect mail | jq -r '.[].State.Health.Status'"
assert_output "healthy"
assert_success
}
#
# processes
#
@test "checking process: postfix" {
run docker exec mail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/lib/postfix/sbin/master'"
assert_success
}
@test "checking process: clamd" {
run docker exec mail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/sbin/clamd'"
assert_success
}
@test "checking process: new" {
run docker exec mail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/sbin/amavisd-new'"
assert_success
}
@test "checking process: opendkim" {
run docker exec mail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/sbin/opendkim'"
assert_success
}
@test "checking process: opendmarc" {
run docker exec mail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/sbin/opendmarc'"
assert_success
}
@test "checking process: fail2ban (disabled in default configuration)" {
run docker exec mail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/bin/python3 /usr/bin/fail2ban-server'"
assert_failure
}
@test "checking process: fetchmail (disabled in default configuration)" {
run docker exec mail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/bin/fetchmail'"
assert_failure
}
#
# imap
#
2016-04-13 21:16:46 +00:00
@test "checking process: dovecot imaplogin (enabled in default configuration)" {
run docker exec mail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/sbin/dovecot'"
assert_success
}
@test "checking imap: server is ready with STARTTLS" {
2016-04-22 15:51:14 +00:00
run docker exec mail /bin/bash -c "nc -w 2 0.0.0.0 143 | grep '* OK' | grep 'STARTTLS' | grep 'ready'"
assert_success
}
@test "checking imap: authentication works" {
2016-04-20 23:08:14 +00:00
run docker exec mail /bin/sh -c "nc -w 1 0.0.0.0 143 < /tmp/docker-mailserver-test/auth/imap-auth.txt"
assert_success
}
@test "checking imap: added user authentication works" {
run docker exec mail /bin/sh -c "nc -w 1 0.0.0.0 143 < /tmp/docker-mailserver-test/auth/added-imap-auth.txt"
assert_success
}
#
# sasl
#
@test "checking sasl: doveadm auth test works with good password" {
run docker exec mail /bin/sh -c "doveadm auth test -x service=smtp user2@otherdomain.tld mypassword | grep 'auth succeeded'"
assert_success
}
@test "checking sasl: doveadm auth test fails with bad password" {
run docker exec mail /bin/sh -c "doveadm auth test -x service=smtp user2@otherdomain.tld BADPASSWORD | grep 'auth failed'"
assert_success
}
#
# logs
#
@test "checking logs: mail related logs should be located in a subdirectory" {
debian stretch slim (#784) * Switch to stretch-slim as base image. - first step correct the testdata, as newer packages are more strict about the mail-structure. * Switch to stretch-slim: correcting the test-environment and the build - add missing build-step to make - clean the userdb aswell - use timeout of netcat, as postgrey would not close the connection - there is 2 extra mail-logs -> assert_output 5 - cosmetic: use "" instead of '' * Switch to stretch-slim: new image: - smaller size - 0 CVEs compared to 11 CVEs in ubuntu 16.04 Image better backport situation - postfix 3.1.6 vs 3.1.0 - fail2ban 0.9.6 vs 0.9.3 ... changes needed because of stretch-slim: - add missing gnupg and iproute2 package - remove non-free rar, unrar-free should do - rsyslog does not add syslog user and has different conf-structure - pyzor command discover was deprecated and is missing in the new stretch package - dovecot does not know SSLv2 anymore. removed because of warnings in log - iptables does not know imap3, IMAP working group chose imap2 in favor of imap3 * Switch to debian stretch slim: SSLv2 seems to be a not known protocol anymore - good! * switch to debian stretch slim: make this test more stable. there might be more than only one mail.log (mail.info, mail.warn, ...) * switch to debian stretc slim: new openssl 1.1.0 needs stronger ciphers, removed some weekers ones. Please, look through the new list of cipher! this needs to be done in another commit for all other SSL/TLS-Endpoints aswell. * Switch to debian stretch slim: let our server pre-empt the cipher list. Did a read through, wwwDOTpostfixDOTorg/FORWARD_SECRECY_READMEDOThtml and wwwDOTpostfixDOTorg/TLS_READMEDOThtml * Switch to debian stretch slim: lets give this openssl-based test a new and independent but identical container. many other test on the main 'mail' container might interfere here. * Switch to debian stretch slim: remove unused lines
2017-12-31 11:33:48 +00:00
run docker exec mail /bin/sh -c "ls -1 /var/log/mail/ | grep -E 'clamav|freshclam|mail.log'|wc -l"
assert_success
assert_output 3
}
#
# smtp
#
@test "checking smtp: authentication works with good password (plain)" {
2016-04-20 23:08:14 +00:00
run docker exec mail /bin/sh -c "nc -w 5 0.0.0.0 25 < /tmp/docker-mailserver-test/auth/smtp-auth-plain.txt | grep 'Authentication successful'"
assert_success
}
@test "checking smtp: authentication fails with wrong password (plain)" {
2020-09-21 23:09:47 +00:00
run docker exec mail /bin/sh -c "nc -w 20 0.0.0.0 25 < /tmp/docker-mailserver-test/auth/smtp-auth-plain-wrong.txt"
assert_output --partial 'authentication failed'
assert_success
}
@test "checking smtp: authentication works with good password (login)" {
2016-04-20 23:08:14 +00:00
run docker exec mail /bin/sh -c "nc -w 5 0.0.0.0 25 < /tmp/docker-mailserver-test/auth/smtp-auth-login.txt | grep 'Authentication successful'"
assert_success
}
@test "checking smtp: authentication fails with wrong password (login)" {
2020-09-21 23:09:47 +00:00
run docker exec mail /bin/sh -c "nc -w 20 0.0.0.0 25 < /tmp/docker-mailserver-test/auth/smtp-auth-login-wrong.txt"
assert_output --partial 'authentication failed'
assert_success
}
@test "checking smtp: added user authentication works with good password (plain)" {
run docker exec mail /bin/sh -c "nc -w 5 0.0.0.0 25 < /tmp/docker-mailserver-test/auth/added-smtp-auth-plain.txt | grep 'Authentication successful'"
assert_success
}
@test "checking smtp: added user authentication fails with wrong password (plain)" {
run docker exec mail /bin/sh -c "nc -w 20 0.0.0.0 25 < /tmp/docker-mailserver-test/auth/added-smtp-auth-plain-wrong.txt | grep 'authentication failed'"
assert_success
}
@test "checking smtp: added user authentication works with good password (login)" {
run docker exec mail /bin/sh -c "nc -w 5 0.0.0.0 25 < /tmp/docker-mailserver-test/auth/added-smtp-auth-login.txt | grep 'Authentication successful'"
assert_success
}
@test "checking smtp: added user authentication fails with wrong password (login)" {
run docker exec mail /bin/sh -c "nc -w 20 0.0.0.0 25 < /tmp/docker-mailserver-test/auth/added-smtp-auth-login-wrong.txt | grep 'authentication failed'"
assert_success
}
# TODO add a test covering case SPAMASSASSIN_SPAM_TO_INBOX=1 (default)
@test "checking smtp: delivers mail to existing account" {
run docker exec mail /bin/sh -c "grep 'postfix/lmtp' /var/log/mail/mail.log | grep 'status=sent' | grep ' Saved)' | sed 's/.* to=</</g' | sed 's/, relay.*//g' | sort | uniq -c | tr -s \" \""
assert_success
2020-09-19 23:09:10 +00:00
assert_output <<'EOF'
1 <added@localhost.localdomain>
6 <user1@localhost.localdomain>
1 <user1@localhost.localdomain>, orig_to=<postmaster@my-domain.com>
1 <user1@localhost.localdomain>, orig_to=<root>
1 <user1~test@localhost.localdomain>
2 <user2@otherdomain.tld>
1 <user3@localhost.localdomain>
EOF
}
@test "checking smtp: delivers mail to existing alias" {
run docker exec mail /bin/sh -c "grep 'to=<user1@localhost.localdomain>, orig_to=<alias1@localhost.localdomain>' /var/log/mail/mail.log | grep 'status=sent' | wc -l"
assert_success
assert_output 1
}
@test "checking smtp: delivers mail to existing alias with recipient delimiter" {
run docker exec mail /bin/sh -c "grep 'to=<user1~test@localhost.localdomain>, orig_to=<alias1~test@localhost.localdomain>' /var/log/mail/mail.log | grep 'status=sent' | wc -l"
assert_success
assert_output 1
run docker exec mail /bin/sh -c "grep 'to=<user1~test@localhost.localdomain>' /var/log/mail/mail.log | grep 'status=bounced'"
assert_failure
}
2016-07-23 21:42:18 +00:00
@test "checking smtp: delivers mail to existing catchall" {
run docker exec mail /bin/sh -c "grep 'to=<user1@localhost.localdomain>, orig_to=<wildcard@localdomain2.com>' /var/log/mail/mail.log | grep 'status=sent' | wc -l"
assert_success
assert_output 1
2016-07-23 21:42:18 +00:00
}
2016-05-24 04:30:22 +00:00
@test "checking smtp: delivers mail to regexp alias" {
run docker exec mail /bin/sh -c "grep 'to=<user1@localhost.localdomain>, orig_to=<test123@localhost.localdomain>' /var/log/mail/mail.log | grep 'status=sent' | wc -l"
assert_success
assert_output 1
2016-05-24 04:30:22 +00:00
}
@test "checking smtp: user1 should have received 9 mails" {
run docker exec mail /bin/sh -c "grep Subject /var/mail/localhost.localdomain/user1/new/* | sed 's/.*Subject: //g' | sed 's/\.txt.*//g' | sed 's/VIRUS.*/VIRUS/g' | sort"
assert_success
# 9 messages, the virus mail has three subject lines
cat <<'EOF' | assert_output
Root Test Message
Test Message amavis-virus
Test Message amavis-virus
Test Message existing-alias-external
Test Message existing-alias-recipient-delimiter
Test Message existing-catchall-local
Test Message existing-regexp-alias-local
Test Message existing-user-and-cc-local-alias
Test Message existing-user1
Test Message sieve-spam-folder
VIRUS
EOF
}
@test "checking smtp: rejects mail to unknown user" {
run docker exec mail /bin/sh -c "grep '<nouser@localhost.localdomain>: Recipient address rejected: User unknown in virtual mailbox table' /var/log/mail/mail.log | wc -l"
assert_success
assert_output 1
}
2016-05-24 04:30:22 +00:00
@test "checking smtp: redirects mail to external aliases" {
run docker exec mail /bin/sh -c "grep -- '-> <external1@otherdomain.tld>' /var/log/mail/mail.log* | grep RelayedInbound | wc -l"
assert_success
assert_output 2
}
# TODO add a test covering case SPAMASSASSIN_SPAM_TO_INBOX=1 (default)
@test "checking smtp: rejects spam" {
run docker exec mail /bin/sh -c "grep 'Blocked SPAM' /var/log/mail/mail.log | grep external.tld=spam@my-domain.com | wc -l"
assert_success
assert_output 1
}
@test "checking smtp: rejects virus" {
run docker exec mail /bin/sh -c "grep 'Blocked INFECTED' /var/log/mail/mail.log | grep external.tld=virus@my-domain.com | wc -l"
assert_success
assert_output 1
}
@test "checking smtp: not advertising smtputf8" {
# Dovecot does not support SMTPUTF8, so while we can send we cannot receive
# Better disable SMTPUTF8 support entirely if we can't handle it correctly
run docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/smtp-ehlo.txt | grep SMTPUTF8 | wc -l"
assert_success
assert_output 0
}
#
# accounts
#
@test "checking accounts: user accounts" {
run docker exec mail doveadm user '*'
assert_success
2020-09-21 23:09:47 +00:00
assert_line --index 0 "user1@localhost.localdomain"
assert_line --index 1 "user2@otherdomain.tld"
assert_line --index 2 "user3@localhost.localdomain"
assert_line --index 3 "added@localhost.localdomain"
}
@test "checking accounts: user mail folder for user1" {
run docker exec mail /bin/bash -c "ls -d /var/mail/localhost.localdomain/user1"
assert_success
}
@test "checking accounts: user mail folder for user2" {
run docker exec mail /bin/bash -c "ls -d /var/mail/otherdomain.tld/user2"
assert_success
}
@test "checking accounts: user mail folder for user3" {
run docker exec mail /bin/bash -c "ls -d /var/mail/localhost.localdomain/user3/mail"
assert_success
}
@test "checking accounts: user mail folder for added user" {
run docker exec mail /bin/bash -c "ls -d /var/mail/localhost.localdomain/added"
assert_success
}
@test "checking accounts: comments are not parsed" {
run docker exec mail /bin/bash -c "ls /var/mail | grep 'comment'"
assert_failure
}
#
# postfix
#
@test "checking postfix: vhost file is correct" {
run docker exec mail cat /etc/postfix/vhost
assert_success
assert_line --index 0 "localdomain2.com"
2020-10-18 00:24:26 +00:00
assert_line --index 1 "localhost.localdomain"
assert_line --index 2 "otherdomain.tld"
}
2016-03-18 19:07:58 +00:00
@test "checking postfix: main.cf overrides" {
run docker exec mail grep -q 'max_idle = 600s' /tmp/docker-mailserver/postfix-main.cf
assert_success
run docker exec mail grep -q 'readme_directory = /tmp' /tmp/docker-mailserver/postfix-main.cf
assert_success
2016-03-18 19:07:58 +00:00
}
@test "checking postfix: master.cf overrides" {
run docker exec mail grep -q 'submission/inet/smtpd_sasl_security_options=noanonymous' /tmp/docker-mailserver/postfix-master.cf
assert_success
}
#
# dovecot
#
@test "checking dovecot: config additions" {
run docker exec mail grep -q 'mail_max_userip_connections = 69' /tmp/docker-mailserver/dovecot.cf
assert_success
run docker exec mail /bin/sh -c "doveconf | grep 'mail_max_userip_connections = 69'"
assert_success
assert_output 'mail_max_userip_connections = 69'
}
#
# spamassassin
#
@test "checking spamassassin: should be listed in amavis when enabled" {
run docker exec mail /bin/sh -c "grep -i 'ANTI-SPAM-SA code' /var/log/mail/mail.log | grep 'NOT loaded'"
assert_failure
}
@test "checking spamassassin: all registered domains should see spam headers" {
run docker exec mail /bin/sh -c "grep -ir 'X-Spam-' /var/mail/localhost.localdomain/user1/new"
assert_success
run docker exec mail /bin/sh -c "grep -ir 'X-Spam-' /var/mail/otherdomain.tld/user2/new"
assert_success
}
#
# ClamAV
#
@test "checking ClamAV: should be listed in amavis when enabled" {
run docker exec mail grep -i 'Found secondary av scanner ClamAV-clamscan' /var/log/mail/mail.log
assert_success
}
@test "checking ClamAV: CLAMAV_MESSAGE_SIZE_LIMIT" {
run docker exec mail grep -q '^MaxFileSize 30M$' /etc/clamav/clamd.conf
assert_success
}
#
# postsrsd
#
@test "checking SRS: main.cf entries" {
run docker exec mail grep "sender_canonical_maps = tcp:localhost:10001" /etc/postfix/main.cf
assert_success
run docker exec mail grep "sender_canonical_classes = envelope_sender" /etc/postfix/main.cf
assert_success
run docker exec mail grep "recipient_canonical_maps = tcp:localhost:10002" /etc/postfix/main.cf
assert_success
run docker exec mail grep "recipient_canonical_classes = envelope_recipient,header_recipient" /etc/postfix/main.cf
assert_success
}
@test "checking SRS: postsrsd running" {
run docker exec mail /bin/sh -c "ps aux | grep ^postsrsd"
assert_success
}
@test "checking SRS: fallback to hostname is handled correctly" {
run docker exec mail grep "SRS_DOMAIN=my-domain.com" /etc/default/postsrsd
assert_success
}
#
# system
#
@test "checking system: freshclam cron is enabled" {
run docker exec mail bash -c "grep '/usr/bin/freshclam' -r /etc/cron.d"
assert_success
}
@test "checking amavis: virusmail wiper cron exists" {
run docker exec mail bash -c "crontab -l | grep '/usr/local/bin/virus-wiper'"
assert_success
}
@test "checking amavis: VIRUSMAILS_DELETE_DELAY override works as expected" {
# shellcheck disable=SC2016
2020-10-18 00:24:26 +00:00
run docker run --rm -e VIRUSMAILS_DELETE_DELAY=2 "${IMAGE_NAME:?}" /bin/bash -c 'echo "${VIRUSMAILS_DELETE_DELAY}"'
assert_output 2
}
@test "checking amavis: old virusmail is wipped by cron" {
docker exec mail bash -c 'touch -d "`date --date=2000-01-01`" /var/lib/amavis/virusmails/should-be-deleted'
run docker exec mail bash -c '/usr/local/bin/virus-wiper'
assert_success
run docker exec mail bash -c 'ls -la /var/lib/amavis/virusmails/ | grep should-be-deleted'
assert_failure
}
@test "checking amavis: recent virusmail is not wipped by cron" {
docker exec mail bash -c 'touch -d "`date`" /var/lib/amavis/virusmails/should-not-be-deleted'
run docker exec mail bash -c '/usr/local/bin/virus-wiper'
assert_success
run docker exec mail bash -c 'ls -la /var/lib/amavis/virusmails/ | grep should-not-be-deleted'
assert_success
}
@test "checking system: /var/log/mail/mail.log is error free" {
run docker exec mail grep 'non-null host address bits in' /var/log/mail/mail.log
assert_failure
run docker exec mail grep 'mail system configuration error' /var/log/mail/mail.log
assert_failure
run docker exec mail grep ': error:' /var/log/mail/mail.log
assert_failure
run docker exec mail grep -i 'is not writable' /var/log/mail/mail.log
assert_failure
run docker exec mail grep -i 'permission denied' /var/log/mail/mail.log
assert_failure
run docker exec mail grep -i '(!)connect' /var/log/mail/mail.log
assert_failure
run docker exec mail grep -i 'using backwards-compatible default setting' /var/log/mail/mail.log
assert_failure
run docker exec mail grep -i 'connect to 127.0.0.1:10023: Connection refused' /var/log/mail/mail.log
assert_failure
}
2016-12-17 20:53:09 +00:00
@test "checking system: /var/log/auth.log is error free" {
run docker exec mail grep 'Unable to open env file: /etc/default/locale' /var/log/auth.log
assert_failure
2016-12-17 20:53:09 +00:00
}
@test "checking system: sets the server fqdn" {
run docker exec mail hostname
assert_success
assert_output "mail.my-domain.com"
}
@test "checking system: sets the server domain name in /etc/mailname" {
run docker exec mail cat /etc/mailname
assert_success
assert_output "my-domain.com"
}
2016-09-23 06:22:57 +00:00
@test "checking system: postfix should not log to syslog" {
run docker exec mail grep 'postfix' /var/log/syslog
assert_failure
2016-09-23 06:22:57 +00:00
}
@test "checking system: amavis decoders installed and available" {
run docker exec mail /bin/sh -c "grep -E '.*(Internal decoder|Found decoder) for\s+\..*' /var/log/mail/mail.log*|grep -Eo '(mail|Z|gz|bz2|xz|lzma|lrz|lzo|lz4|rpm|cpio|tar|deb|rar|arj|arc|zoo|doc|cab|tnef|zip|kmz|7z|jar|swf|lha|iso|exe)' | sort | uniq"
assert_success
# Support for doc and zoo removed in buster
cat <<'EOF' | assert_output
7z
Z
arc
arj
bz2
cab
cpio
deb
exe
gz
iso
jar
kmz
lha
lrz
lz4
lzma
lzo
mail
rar
rpm
swf
tar
tnef
xz
zip
EOF
}
#
# sieve
#
2016-04-28 08:25:11 +00:00
@test "checking sieve: user1 should have received 1 email in folder INBOX.spam" {
run docker exec mail /bin/sh -c "ls -A /var/mail/localhost.localdomain/user1/.INBOX.spam/new | wc -l"
assert_success
assert_output 1
}
@test "checking manage sieve: server is ready when ENABLE_MANAGESIEVE has been set" {
run docker exec mail /bin/bash -c "nc -z 0.0.0.0 4190"
assert_success
}
@test "checking sieve: user2 should have piped 1 email to /tmp/" {
run docker exec mail /bin/sh -c "ls -A /tmp/pipe-test.out | wc -l"
assert_success
assert_output 1
}
@test "checking sieve global: user1 should have gotten a copy of his spam mail" {
run docker exec mail /bin/sh -c "grep 'Spambot <spam@spam.com>' -R /var/mail/localhost.localdomain/user1/new/"
assert_success
}
#
# accounts
#
@test "checking accounts: user_without_domain creation should be rejected since user@domain format is required" {
run docker exec mail /bin/sh -c "addmailuser user_without_domain mypassword"
assert_failure
refactor: CLI commands for database management (#2654) See the associated PR for more detailed commentary on specific changes. ### Commands refactored: - User (**All:** add / list / update / del + _dovecot-master variants_) - Quota (**All:** set / del) - Virtual Alias (**All:** add / list /del) - Relay (**All:** add-relayhost / add-sasl / exclude-domain) ### Overall changes involve: - **Fairly common structure:** - `_main` method at the top provides an overview of logical steps: - After all methods are declared beneath it (_and imported from the new `helpers/database/db.sh`_), the `_main` is called at the bottom of the file. - `delmailuser` additionally processes option support for `-y` prior to calling `_main`. - `__usage` is now consistent with each of these commands, along with the `help` command. - Most logic delegated to new helper scripts. Some duplicate content remains on the basis that it's low-risk to maintenance and avoids less hassle to jump between files to check a single line, usually this is arg validation. - Error handling should be more consistent, along with var names (_no more `USER`/`EMAIL`/`FULL_EMAIL` to refer to the same expected value_). - **Three new management scripts** (in `helpers/database/manage/`) using a common structure for managing changes to their respective "Database" config file. - `postfix-accounts.sh` unified not only add and update commands, but also all the dovecot-master versions, a single password call for all 4 of them, with a 5th consumer of the password prompt from the relay command `addsaslpassword`. - These scripts delegate actual writes to `helpers/database/db.sh` which provides a common API to support the changes made. - This is more verbose/complex vs the current inline operations each command currently has, as it provides generic support instead of slightly different variations being maintained, along with handling some edge cases that existed and would lead to bugs (notably substring matches). - Centralizing changes here seems wiser than scattered about. I've tried to make it easy to grok, hopefully it's not worse than the current situation. - List operations were kept in their respective commands, `db.sh` is only really managing writes. I didn't see a nice way for removing the code duplication for list commands as the duplication was fairly minimal, especially for `listalias` and `listdovecotmasteruser` which were quite simple in their differences in the loop body. - `listmailuser` and `delmailuser` also retain methods exclusive to respective commands, I wasn't sure if there was any benefit to move those, but they were refactored.
2022-07-29 00:10:23 +00:00
assert_output --partial 'should include the domain (eg: user@example.com)'
}
@test "checking accounts: user3 should have been added to /tmp/docker-mailserver/postfix-accounts.cf" {
docker exec mail /bin/sh -c "addmailuser user3@domain.tld mypassword"
run docker exec mail /bin/sh -c "grep '^user3@domain\.tld|' -i /tmp/docker-mailserver/postfix-accounts.cf"
assert_success
[[ -n ${output} ]]
}
@test "checking accounts: auser3 should have been added to /tmp/docker-mailserver/postfix-accounts.cf" {
docker exec mail /bin/sh -c "addmailuser auser3@domain.tld mypassword"
run docker exec mail /bin/sh -c "grep '^auser3@domain\.tld|' -i /tmp/docker-mailserver/postfix-accounts.cf"
assert_success
[[ -n ${output} ]]
}
@test "checking accounts: a.ser3 should have been added to /tmp/docker-mailserver/postfix-accounts.cf" {
docker exec mail /bin/sh -c "addmailuser a.ser3@domain.tld mypassword"
run docker exec mail /bin/sh -c "grep '^a\.ser3@domain\.tld|' -i /tmp/docker-mailserver/postfix-accounts.cf"
assert_success
[[ -n ${output} ]]
}
@test "checking accounts: user3 should have been removed from /tmp/docker-mailserver/postfix-accounts.cf but not auser3" {
chore: `addmailuser` - Remove delaying completion until `/var/mail` is ready (#2729) ## Quick Summary Resolves a `TODO` task with `addmailuser`. ## Overview The main change is adding three new methods in `common.bash`, which replace the completion delay in `addmailuser` / `setup email add` command. Other than that: - I swapped `sh -c 'addmailuser ...'` to `setup email add ...`. - Improved three tests in `setup-cli.bats` for `setup email add|update|del` (_logic remains effectively the same still_). - Rewrote the `TODO` comment for `setup-cli.bats` test on `setup email del` to better clarify the concern, but the test itself was no longer affected due to changes prior to this PR, so I enabled the commented out assertion. - Removed unnecessary waits. The two `skip` tests in `test/tests.bats` could be enabled again after this PR. - Additional fixes to tests were made during the PR (see discussion comments for details), resolving race conditions. Individual commit messages of the PR provide additional details if helpful. --- ## Relevant commit messages * chore: Remove creation delay in `addmailuser` This was apparently only for supporting tests that need to wait on account creation being ready to test against. As per the removed inline docs, it should be fine to remove once tests are updated to work correctly without it. * tests(feat): Add two new common helper methods `wait_until_account_maildir_exists()` provides the same logic `addmailuser` command was carrying, to wait upon the account dir creation in `/var/mail`. As this was specifically to support tests, it makes more sense as a test method. `add_mail_account_then_wait_until_ready()` was added to handle the common pattern of creating account and waiting on it. An internal assert will ensure the account was successfully created first during the test before attempting to wait. * tests(feat): Add common helper for waiting on change event to be processed The current helper is more complicated for no real benefit, it only detects when a change is made that would trigger a change event in the `changedetector` service. Our usage of this in tests however is only interested in waiting out the completion of the change event. Remove unnecessary change event waits. These waits should not be necessary if handled correctly. * tests: `addmailuser` to `add_mail_account_then_wait_until_ready mail()` This helper method is used where appropriate. - A password is not relevant (optional). - We need to wait on the creation on the account (Dovecot and `/var/mail` directory). * tests: `setup-cli` revise `add`, `update`, `del` tests The delete test was failing as the `/var/mail` directory did not yet exist. There is now a proper delay imposed in the `add` test now shares the same account for both `update` and `del` tests resolving that failure. Additionally tests use better asserts where appropriate and the wait + sleep logic in `add` has been improved (now takes 10 seconds to complete, approx half the time than before). The `del` test TODO while not technically addressed is no longer relevant due to the tests being switched to `-c` option (there is a separate `no container` test file, but it doesn't provide a `del` test). * tests(fix): Ensure Postfix is reachable after waiting on ClamAV There is not much reason to check before waiting on ClamAV. It is more helpful to debug failures from `nc` mail send commands if we know that nothing went wrong inbetween the ClamAV wait time. Additionally added an assertion which should provide more information if this part of the test setup fails again. * tests(fix): Move health check to the top This test is a bit fragile. It relies on defaults for the healthcheck with intervals of 30 seconds. If the check occurs while Postfix is down due a change event from earlier tests and the healthcheck kicks in at that point, then if there is not enough time to refresh the health status from `unhealthy`, the test will fail with a false-positive as Postfix is actually working and up again.. * tests(fix): Wait on directory to be removed Workaround that tries not to introduce heavier delays by waiting on a full change event to complete in the previous `email update` if possible. There is a chance that the account has the folder deleted, but restored from an active change event (for password update, then the account delete).
2022-08-22 22:15:06 +00:00
wait_until_account_maildir_exists mail 'user3@domain.tld'
docker exec mail /bin/sh -c "delmailuser -y user3@domain.tld"
run docker exec mail /bin/sh -c "grep '^user3@domain\.tld' -i /tmp/docker-mailserver/postfix-accounts.cf"
assert_failure
[[ -z ${output} ]]
run docker exec mail /bin/sh -c "grep '^auser3@domain\.tld' -i /tmp/docker-mailserver/postfix-accounts.cf"
assert_success
[[ -n ${output} ]]
}
@test "checking user updating password for user in /tmp/docker-mailserver/postfix-accounts.cf" {
chore: `addmailuser` - Remove delaying completion until `/var/mail` is ready (#2729) ## Quick Summary Resolves a `TODO` task with `addmailuser`. ## Overview The main change is adding three new methods in `common.bash`, which replace the completion delay in `addmailuser` / `setup email add` command. Other than that: - I swapped `sh -c 'addmailuser ...'` to `setup email add ...`. - Improved three tests in `setup-cli.bats` for `setup email add|update|del` (_logic remains effectively the same still_). - Rewrote the `TODO` comment for `setup-cli.bats` test on `setup email del` to better clarify the concern, but the test itself was no longer affected due to changes prior to this PR, so I enabled the commented out assertion. - Removed unnecessary waits. The two `skip` tests in `test/tests.bats` could be enabled again after this PR. - Additional fixes to tests were made during the PR (see discussion comments for details), resolving race conditions. Individual commit messages of the PR provide additional details if helpful. --- ## Relevant commit messages * chore: Remove creation delay in `addmailuser` This was apparently only for supporting tests that need to wait on account creation being ready to test against. As per the removed inline docs, it should be fine to remove once tests are updated to work correctly without it. * tests(feat): Add two new common helper methods `wait_until_account_maildir_exists()` provides the same logic `addmailuser` command was carrying, to wait upon the account dir creation in `/var/mail`. As this was specifically to support tests, it makes more sense as a test method. `add_mail_account_then_wait_until_ready()` was added to handle the common pattern of creating account and waiting on it. An internal assert will ensure the account was successfully created first during the test before attempting to wait. * tests(feat): Add common helper for waiting on change event to be processed The current helper is more complicated for no real benefit, it only detects when a change is made that would trigger a change event in the `changedetector` service. Our usage of this in tests however is only interested in waiting out the completion of the change event. Remove unnecessary change event waits. These waits should not be necessary if handled correctly. * tests: `addmailuser` to `add_mail_account_then_wait_until_ready mail()` This helper method is used where appropriate. - A password is not relevant (optional). - We need to wait on the creation on the account (Dovecot and `/var/mail` directory). * tests: `setup-cli` revise `add`, `update`, `del` tests The delete test was failing as the `/var/mail` directory did not yet exist. There is now a proper delay imposed in the `add` test now shares the same account for both `update` and `del` tests resolving that failure. Additionally tests use better asserts where appropriate and the wait + sleep logic in `add` has been improved (now takes 10 seconds to complete, approx half the time than before). The `del` test TODO while not technically addressed is no longer relevant due to the tests being switched to `-c` option (there is a separate `no container` test file, but it doesn't provide a `del` test). * tests(fix): Ensure Postfix is reachable after waiting on ClamAV There is not much reason to check before waiting on ClamAV. It is more helpful to debug failures from `nc` mail send commands if we know that nothing went wrong inbetween the ClamAV wait time. Additionally added an assertion which should provide more information if this part of the test setup fails again. * tests(fix): Move health check to the top This test is a bit fragile. It relies on defaults for the healthcheck with intervals of 30 seconds. If the check occurs while Postfix is down due a change event from earlier tests and the healthcheck kicks in at that point, then if there is not enough time to refresh the health status from `unhealthy`, the test will fail with a false-positive as Postfix is actually working and up again.. * tests(fix): Wait on directory to be removed Workaround that tries not to introduce heavier delays by waiting on a full change event to complete in the previous `email update` if possible. There is a chance that the account has the folder deleted, but restored from an active change event (for password update, then the account delete).
2022-08-22 22:15:06 +00:00
add_mail_account_then_wait_until_ready mail 'user4@domain.tld'
2020-10-18 00:24:26 +00:00
initialpass=$(docker exec mail /bin/sh -c "grep '^user4@domain\.tld' -i /tmp/docker-mailserver/postfix-accounts.cf")
sleep 2
docker exec mail /bin/sh -c "updatemailuser user4@domain.tld mynewpassword"
sleep 2
2020-10-18 00:24:26 +00:00
changepass=$(docker exec mail /bin/sh -c "grep '^user4@domain\.tld' -i /tmp/docker-mailserver/postfix-accounts.cf")
[[ ${initialpass} != "${changepass}" ]]
run docker exec mail /bin/sh -c "delmailuser -y auser3@domain.tld"
assert_success
}
@test "checking accounts: listmailuser (quotas disabled)" {
run docker exec mail /bin/sh -c "echo 'ENABLE_QUOTAS=0' >> /etc/dms-settings && listmailuser | head -n 1"
assert_success
assert_output '* user1@localhost.localdomain'
}
@test "checking accounts: listmailuser (quotas enabled)" {
run docker exec mail /bin/sh -c "sed -i '/ENABLE_QUOTAS=0/d' /etc/dms-settings; listmailuser | head -n 1"
assert_success
assert_output '* user1@localhost.localdomain ( 12K / ~ ) [0%]'
}
@test "checking accounts: no error is generated when deleting a user if /tmp/docker-mailserver/postfix-accounts.cf is missing" {
run docker run --rm \
-v "$(duplicate_config_for_container without-accounts/ without-accounts-deleting-user)":/tmp/docker-mailserver/ \
2020-10-18 00:24:26 +00:00
"${IMAGE_NAME:?}" /bin/sh -c 'delmailuser -y user3@domain.tld'
assert_success
[[ -z ${output} ]]
}
@test "checking accounts: user3 should have been added to /tmp/docker-mailserver/postfix-accounts.cf even when that file does not exist" {
2020-10-19 13:19:34 +00:00
local PRIVATE_CONFIG
PRIVATE_CONFIG=$(duplicate_config_for_container without-accounts/ without-accounts_file_does_not_exist)
run docker run --rm \
-v "${PRIVATE_CONFIG}/without-accounts/":/tmp/docker-mailserver/ \
2020-10-18 00:24:26 +00:00
"${IMAGE_NAME:?}" /bin/sh -c 'addmailuser user3@domain.tld mypassword'
assert_success
run docker run --rm \
-v "${PRIVATE_CONFIG}/without-accounts/":/tmp/docker-mailserver/ \
2020-10-18 00:24:26 +00:00
"${IMAGE_NAME:?}" /bin/sh -c 'grep user3@domain.tld -i /tmp/docker-mailserver/postfix-accounts.cf'
assert_success
[[ -n ${output} ]]
}
@test "checking quota: setquota user must be existing" {
chore: `addmailuser` - Remove delaying completion until `/var/mail` is ready (#2729) ## Quick Summary Resolves a `TODO` task with `addmailuser`. ## Overview The main change is adding three new methods in `common.bash`, which replace the completion delay in `addmailuser` / `setup email add` command. Other than that: - I swapped `sh -c 'addmailuser ...'` to `setup email add ...`. - Improved three tests in `setup-cli.bats` for `setup email add|update|del` (_logic remains effectively the same still_). - Rewrote the `TODO` comment for `setup-cli.bats` test on `setup email del` to better clarify the concern, but the test itself was no longer affected due to changes prior to this PR, so I enabled the commented out assertion. - Removed unnecessary waits. The two `skip` tests in `test/tests.bats` could be enabled again after this PR. - Additional fixes to tests were made during the PR (see discussion comments for details), resolving race conditions. Individual commit messages of the PR provide additional details if helpful. --- ## Relevant commit messages * chore: Remove creation delay in `addmailuser` This was apparently only for supporting tests that need to wait on account creation being ready to test against. As per the removed inline docs, it should be fine to remove once tests are updated to work correctly without it. * tests(feat): Add two new common helper methods `wait_until_account_maildir_exists()` provides the same logic `addmailuser` command was carrying, to wait upon the account dir creation in `/var/mail`. As this was specifically to support tests, it makes more sense as a test method. `add_mail_account_then_wait_until_ready()` was added to handle the common pattern of creating account and waiting on it. An internal assert will ensure the account was successfully created first during the test before attempting to wait. * tests(feat): Add common helper for waiting on change event to be processed The current helper is more complicated for no real benefit, it only detects when a change is made that would trigger a change event in the `changedetector` service. Our usage of this in tests however is only interested in waiting out the completion of the change event. Remove unnecessary change event waits. These waits should not be necessary if handled correctly. * tests: `addmailuser` to `add_mail_account_then_wait_until_ready mail()` This helper method is used where appropriate. - A password is not relevant (optional). - We need to wait on the creation on the account (Dovecot and `/var/mail` directory). * tests: `setup-cli` revise `add`, `update`, `del` tests The delete test was failing as the `/var/mail` directory did not yet exist. There is now a proper delay imposed in the `add` test now shares the same account for both `update` and `del` tests resolving that failure. Additionally tests use better asserts where appropriate and the wait + sleep logic in `add` has been improved (now takes 10 seconds to complete, approx half the time than before). The `del` test TODO while not technically addressed is no longer relevant due to the tests being switched to `-c` option (there is a separate `no container` test file, but it doesn't provide a `del` test). * tests(fix): Ensure Postfix is reachable after waiting on ClamAV There is not much reason to check before waiting on ClamAV. It is more helpful to debug failures from `nc` mail send commands if we know that nothing went wrong inbetween the ClamAV wait time. Additionally added an assertion which should provide more information if this part of the test setup fails again. * tests(fix): Move health check to the top This test is a bit fragile. It relies on defaults for the healthcheck with intervals of 30 seconds. If the check occurs while Postfix is down due a change event from earlier tests and the healthcheck kicks in at that point, then if there is not enough time to refresh the health status from `unhealthy`, the test will fail with a false-positive as Postfix is actually working and up again.. * tests(fix): Wait on directory to be removed Workaround that tries not to introduce heavier delays by waiting on a full change event to complete in the previous `email update` if possible. There is a chance that the account has the folder deleted, but restored from an active change event (for password update, then the account delete).
2022-08-22 22:15:06 +00:00
add_mail_account_then_wait_until_ready mail 'quota_user@domain.tld'
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
run docker exec mail /bin/sh -c "setquota quota_user 50M"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 50M"
assert_success
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
run docker exec mail /bin/sh -c "setquota username@fulldomain 50M"
assert_failure
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
run docker exec mail /bin/sh -c "delmailuser -y quota_user@domain.tld"
assert_success
}
chore: `addmailuser` - Remove delaying completion until `/var/mail` is ready (#2729) ## Quick Summary Resolves a `TODO` task with `addmailuser`. ## Overview The main change is adding three new methods in `common.bash`, which replace the completion delay in `addmailuser` / `setup email add` command. Other than that: - I swapped `sh -c 'addmailuser ...'` to `setup email add ...`. - Improved three tests in `setup-cli.bats` for `setup email add|update|del` (_logic remains effectively the same still_). - Rewrote the `TODO` comment for `setup-cli.bats` test on `setup email del` to better clarify the concern, but the test itself was no longer affected due to changes prior to this PR, so I enabled the commented out assertion. - Removed unnecessary waits. The two `skip` tests in `test/tests.bats` could be enabled again after this PR. - Additional fixes to tests were made during the PR (see discussion comments for details), resolving race conditions. Individual commit messages of the PR provide additional details if helpful. --- ## Relevant commit messages * chore: Remove creation delay in `addmailuser` This was apparently only for supporting tests that need to wait on account creation being ready to test against. As per the removed inline docs, it should be fine to remove once tests are updated to work correctly without it. * tests(feat): Add two new common helper methods `wait_until_account_maildir_exists()` provides the same logic `addmailuser` command was carrying, to wait upon the account dir creation in `/var/mail`. As this was specifically to support tests, it makes more sense as a test method. `add_mail_account_then_wait_until_ready()` was added to handle the common pattern of creating account and waiting on it. An internal assert will ensure the account was successfully created first during the test before attempting to wait. * tests(feat): Add common helper for waiting on change event to be processed The current helper is more complicated for no real benefit, it only detects when a change is made that would trigger a change event in the `changedetector` service. Our usage of this in tests however is only interested in waiting out the completion of the change event. Remove unnecessary change event waits. These waits should not be necessary if handled correctly. * tests: `addmailuser` to `add_mail_account_then_wait_until_ready mail()` This helper method is used where appropriate. - A password is not relevant (optional). - We need to wait on the creation on the account (Dovecot and `/var/mail` directory). * tests: `setup-cli` revise `add`, `update`, `del` tests The delete test was failing as the `/var/mail` directory did not yet exist. There is now a proper delay imposed in the `add` test now shares the same account for both `update` and `del` tests resolving that failure. Additionally tests use better asserts where appropriate and the wait + sleep logic in `add` has been improved (now takes 10 seconds to complete, approx half the time than before). The `del` test TODO while not technically addressed is no longer relevant due to the tests being switched to `-c` option (there is a separate `no container` test file, but it doesn't provide a `del` test). * tests(fix): Ensure Postfix is reachable after waiting on ClamAV There is not much reason to check before waiting on ClamAV. It is more helpful to debug failures from `nc` mail send commands if we know that nothing went wrong inbetween the ClamAV wait time. Additionally added an assertion which should provide more information if this part of the test setup fails again. * tests(fix): Move health check to the top This test is a bit fragile. It relies on defaults for the healthcheck with intervals of 30 seconds. If the check occurs while Postfix is down due a change event from earlier tests and the healthcheck kicks in at that point, then if there is not enough time to refresh the health status from `unhealthy`, the test will fail with a false-positive as Postfix is actually working and up again.. * tests(fix): Wait on directory to be removed Workaround that tries not to introduce heavier delays by waiting on a full change event to complete in the previous `email update` if possible. There is a chance that the account has the folder deleted, but restored from an active change event (for password update, then the account delete).
2022-08-22 22:15:06 +00:00
@test "checking quota: setquota <quota> must be well formatted" {
chore: `addmailuser` - Remove delaying completion until `/var/mail` is ready (#2729) ## Quick Summary Resolves a `TODO` task with `addmailuser`. ## Overview The main change is adding three new methods in `common.bash`, which replace the completion delay in `addmailuser` / `setup email add` command. Other than that: - I swapped `sh -c 'addmailuser ...'` to `setup email add ...`. - Improved three tests in `setup-cli.bats` for `setup email add|update|del` (_logic remains effectively the same still_). - Rewrote the `TODO` comment for `setup-cli.bats` test on `setup email del` to better clarify the concern, but the test itself was no longer affected due to changes prior to this PR, so I enabled the commented out assertion. - Removed unnecessary waits. The two `skip` tests in `test/tests.bats` could be enabled again after this PR. - Additional fixes to tests were made during the PR (see discussion comments for details), resolving race conditions. Individual commit messages of the PR provide additional details if helpful. --- ## Relevant commit messages * chore: Remove creation delay in `addmailuser` This was apparently only for supporting tests that need to wait on account creation being ready to test against. As per the removed inline docs, it should be fine to remove once tests are updated to work correctly without it. * tests(feat): Add two new common helper methods `wait_until_account_maildir_exists()` provides the same logic `addmailuser` command was carrying, to wait upon the account dir creation in `/var/mail`. As this was specifically to support tests, it makes more sense as a test method. `add_mail_account_then_wait_until_ready()` was added to handle the common pattern of creating account and waiting on it. An internal assert will ensure the account was successfully created first during the test before attempting to wait. * tests(feat): Add common helper for waiting on change event to be processed The current helper is more complicated for no real benefit, it only detects when a change is made that would trigger a change event in the `changedetector` service. Our usage of this in tests however is only interested in waiting out the completion of the change event. Remove unnecessary change event waits. These waits should not be necessary if handled correctly. * tests: `addmailuser` to `add_mail_account_then_wait_until_ready mail()` This helper method is used where appropriate. - A password is not relevant (optional). - We need to wait on the creation on the account (Dovecot and `/var/mail` directory). * tests: `setup-cli` revise `add`, `update`, `del` tests The delete test was failing as the `/var/mail` directory did not yet exist. There is now a proper delay imposed in the `add` test now shares the same account for both `update` and `del` tests resolving that failure. Additionally tests use better asserts where appropriate and the wait + sleep logic in `add` has been improved (now takes 10 seconds to complete, approx half the time than before). The `del` test TODO while not technically addressed is no longer relevant due to the tests being switched to `-c` option (there is a separate `no container` test file, but it doesn't provide a `del` test). * tests(fix): Ensure Postfix is reachable after waiting on ClamAV There is not much reason to check before waiting on ClamAV. It is more helpful to debug failures from `nc` mail send commands if we know that nothing went wrong inbetween the ClamAV wait time. Additionally added an assertion which should provide more information if this part of the test setup fails again. * tests(fix): Move health check to the top This test is a bit fragile. It relies on defaults for the healthcheck with intervals of 30 seconds. If the check occurs while Postfix is down due a change event from earlier tests and the healthcheck kicks in at that point, then if there is not enough time to refresh the health status from `unhealthy`, the test will fail with a false-positive as Postfix is actually working and up again.. * tests(fix): Wait on directory to be removed Workaround that tries not to introduce heavier delays by waiting on a full change event to complete in the previous `email update` if possible. There is a chance that the account has the folder deleted, but restored from an active change event (for password update, then the account delete).
2022-08-22 22:15:06 +00:00
add_mail_account_then_wait_until_ready mail 'quota_user@domain.tld'
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 26GIGOTS"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 123"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld M"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld -60M"
assert_failure
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10B"
assert_success
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10k"
assert_success
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10M"
assert_success
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10G"
assert_success
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10T"
assert_success
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
run docker exec mail /bin/sh -c "delmailuser -y quota_user@domain.tld"
assert_success
}
@test "checking quota: delquota user must be existing" {
chore: `addmailuser` - Remove delaying completion until `/var/mail` is ready (#2729) ## Quick Summary Resolves a `TODO` task with `addmailuser`. ## Overview The main change is adding three new methods in `common.bash`, which replace the completion delay in `addmailuser` / `setup email add` command. Other than that: - I swapped `sh -c 'addmailuser ...'` to `setup email add ...`. - Improved three tests in `setup-cli.bats` for `setup email add|update|del` (_logic remains effectively the same still_). - Rewrote the `TODO` comment for `setup-cli.bats` test on `setup email del` to better clarify the concern, but the test itself was no longer affected due to changes prior to this PR, so I enabled the commented out assertion. - Removed unnecessary waits. The two `skip` tests in `test/tests.bats` could be enabled again after this PR. - Additional fixes to tests were made during the PR (see discussion comments for details), resolving race conditions. Individual commit messages of the PR provide additional details if helpful. --- ## Relevant commit messages * chore: Remove creation delay in `addmailuser` This was apparently only for supporting tests that need to wait on account creation being ready to test against. As per the removed inline docs, it should be fine to remove once tests are updated to work correctly without it. * tests(feat): Add two new common helper methods `wait_until_account_maildir_exists()` provides the same logic `addmailuser` command was carrying, to wait upon the account dir creation in `/var/mail`. As this was specifically to support tests, it makes more sense as a test method. `add_mail_account_then_wait_until_ready()` was added to handle the common pattern of creating account and waiting on it. An internal assert will ensure the account was successfully created first during the test before attempting to wait. * tests(feat): Add common helper for waiting on change event to be processed The current helper is more complicated for no real benefit, it only detects when a change is made that would trigger a change event in the `changedetector` service. Our usage of this in tests however is only interested in waiting out the completion of the change event. Remove unnecessary change event waits. These waits should not be necessary if handled correctly. * tests: `addmailuser` to `add_mail_account_then_wait_until_ready mail()` This helper method is used where appropriate. - A password is not relevant (optional). - We need to wait on the creation on the account (Dovecot and `/var/mail` directory). * tests: `setup-cli` revise `add`, `update`, `del` tests The delete test was failing as the `/var/mail` directory did not yet exist. There is now a proper delay imposed in the `add` test now shares the same account for both `update` and `del` tests resolving that failure. Additionally tests use better asserts where appropriate and the wait + sleep logic in `add` has been improved (now takes 10 seconds to complete, approx half the time than before). The `del` test TODO while not technically addressed is no longer relevant due to the tests being switched to `-c` option (there is a separate `no container` test file, but it doesn't provide a `del` test). * tests(fix): Ensure Postfix is reachable after waiting on ClamAV There is not much reason to check before waiting on ClamAV. It is more helpful to debug failures from `nc` mail send commands if we know that nothing went wrong inbetween the ClamAV wait time. Additionally added an assertion which should provide more information if this part of the test setup fails again. * tests(fix): Move health check to the top This test is a bit fragile. It relies on defaults for the healthcheck with intervals of 30 seconds. If the check occurs while Postfix is down due a change event from earlier tests and the healthcheck kicks in at that point, then if there is not enough time to refresh the health status from `unhealthy`, the test will fail with a false-positive as Postfix is actually working and up again.. * tests(fix): Wait on directory to be removed Workaround that tries not to introduce heavier delays by waiting on a full change event to complete in the previous `email update` if possible. There is a chance that the account has the folder deleted, but restored from an active change event (for password update, then the account delete).
2022-08-22 22:15:06 +00:00
add_mail_account_then_wait_until_ready mail 'quota_user@domain.tld'
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
run docker exec mail /bin/sh -c "delquota uota_user@domain.tld"
assert_failure
run docker exec mail /bin/sh -c "delquota quota_user"
assert_failure
run docker exec mail /bin/sh -c "delquota dontknowyou@domain.tld"
assert_failure
run docker exec mail /bin/sh -c "setquota quota_user@domain.tld 10T"
assert_success
run docker exec mail /bin/sh -c "delquota quota_user@domain.tld"
assert_success
run docker exec mail /bin/sh -c "grep -i 'quota_user@domain.tld' /tmp/docker-mailserver/dovecot-quotas.cf"
assert_failure
run docker exec mail /bin/sh -c "delmailuser -y quota_user@domain.tld"
assert_success
}
chore: `addmailuser` - Remove delaying completion until `/var/mail` is ready (#2729) ## Quick Summary Resolves a `TODO` task with `addmailuser`. ## Overview The main change is adding three new methods in `common.bash`, which replace the completion delay in `addmailuser` / `setup email add` command. Other than that: - I swapped `sh -c 'addmailuser ...'` to `setup email add ...`. - Improved three tests in `setup-cli.bats` for `setup email add|update|del` (_logic remains effectively the same still_). - Rewrote the `TODO` comment for `setup-cli.bats` test on `setup email del` to better clarify the concern, but the test itself was no longer affected due to changes prior to this PR, so I enabled the commented out assertion. - Removed unnecessary waits. The two `skip` tests in `test/tests.bats` could be enabled again after this PR. - Additional fixes to tests were made during the PR (see discussion comments for details), resolving race conditions. Individual commit messages of the PR provide additional details if helpful. --- ## Relevant commit messages * chore: Remove creation delay in `addmailuser` This was apparently only for supporting tests that need to wait on account creation being ready to test against. As per the removed inline docs, it should be fine to remove once tests are updated to work correctly without it. * tests(feat): Add two new common helper methods `wait_until_account_maildir_exists()` provides the same logic `addmailuser` command was carrying, to wait upon the account dir creation in `/var/mail`. As this was specifically to support tests, it makes more sense as a test method. `add_mail_account_then_wait_until_ready()` was added to handle the common pattern of creating account and waiting on it. An internal assert will ensure the account was successfully created first during the test before attempting to wait. * tests(feat): Add common helper for waiting on change event to be processed The current helper is more complicated for no real benefit, it only detects when a change is made that would trigger a change event in the `changedetector` service. Our usage of this in tests however is only interested in waiting out the completion of the change event. Remove unnecessary change event waits. These waits should not be necessary if handled correctly. * tests: `addmailuser` to `add_mail_account_then_wait_until_ready mail()` This helper method is used where appropriate. - A password is not relevant (optional). - We need to wait on the creation on the account (Dovecot and `/var/mail` directory). * tests: `setup-cli` revise `add`, `update`, `del` tests The delete test was failing as the `/var/mail` directory did not yet exist. There is now a proper delay imposed in the `add` test now shares the same account for both `update` and `del` tests resolving that failure. Additionally tests use better asserts where appropriate and the wait + sleep logic in `add` has been improved (now takes 10 seconds to complete, approx half the time than before). The `del` test TODO while not technically addressed is no longer relevant due to the tests being switched to `-c` option (there is a separate `no container` test file, but it doesn't provide a `del` test). * tests(fix): Ensure Postfix is reachable after waiting on ClamAV There is not much reason to check before waiting on ClamAV. It is more helpful to debug failures from `nc` mail send commands if we know that nothing went wrong inbetween the ClamAV wait time. Additionally added an assertion which should provide more information if this part of the test setup fails again. * tests(fix): Move health check to the top This test is a bit fragile. It relies on defaults for the healthcheck with intervals of 30 seconds. If the check occurs while Postfix is down due a change event from earlier tests and the healthcheck kicks in at that point, then if there is not enough time to refresh the health status from `unhealthy`, the test will fail with a false-positive as Postfix is actually working and up again.. * tests(fix): Wait on directory to be removed Workaround that tries not to introduce heavier delays by waiting on a full change event to complete in the previous `email update` if possible. There is a chance that the account has the folder deleted, but restored from an active change event (for password update, then the account delete).
2022-08-22 22:15:06 +00:00
@test "checking quota: delquota allow when no quota for existing user" {
chore: `addmailuser` - Remove delaying completion until `/var/mail` is ready (#2729) ## Quick Summary Resolves a `TODO` task with `addmailuser`. ## Overview The main change is adding three new methods in `common.bash`, which replace the completion delay in `addmailuser` / `setup email add` command. Other than that: - I swapped `sh -c 'addmailuser ...'` to `setup email add ...`. - Improved three tests in `setup-cli.bats` for `setup email add|update|del` (_logic remains effectively the same still_). - Rewrote the `TODO` comment for `setup-cli.bats` test on `setup email del` to better clarify the concern, but the test itself was no longer affected due to changes prior to this PR, so I enabled the commented out assertion. - Removed unnecessary waits. The two `skip` tests in `test/tests.bats` could be enabled again after this PR. - Additional fixes to tests were made during the PR (see discussion comments for details), resolving race conditions. Individual commit messages of the PR provide additional details if helpful. --- ## Relevant commit messages * chore: Remove creation delay in `addmailuser` This was apparently only for supporting tests that need to wait on account creation being ready to test against. As per the removed inline docs, it should be fine to remove once tests are updated to work correctly without it. * tests(feat): Add two new common helper methods `wait_until_account_maildir_exists()` provides the same logic `addmailuser` command was carrying, to wait upon the account dir creation in `/var/mail`. As this was specifically to support tests, it makes more sense as a test method. `add_mail_account_then_wait_until_ready()` was added to handle the common pattern of creating account and waiting on it. An internal assert will ensure the account was successfully created first during the test before attempting to wait. * tests(feat): Add common helper for waiting on change event to be processed The current helper is more complicated for no real benefit, it only detects when a change is made that would trigger a change event in the `changedetector` service. Our usage of this in tests however is only interested in waiting out the completion of the change event. Remove unnecessary change event waits. These waits should not be necessary if handled correctly. * tests: `addmailuser` to `add_mail_account_then_wait_until_ready mail()` This helper method is used where appropriate. - A password is not relevant (optional). - We need to wait on the creation on the account (Dovecot and `/var/mail` directory). * tests: `setup-cli` revise `add`, `update`, `del` tests The delete test was failing as the `/var/mail` directory did not yet exist. There is now a proper delay imposed in the `add` test now shares the same account for both `update` and `del` tests resolving that failure. Additionally tests use better asserts where appropriate and the wait + sleep logic in `add` has been improved (now takes 10 seconds to complete, approx half the time than before). The `del` test TODO while not technically addressed is no longer relevant due to the tests being switched to `-c` option (there is a separate `no container` test file, but it doesn't provide a `del` test). * tests(fix): Ensure Postfix is reachable after waiting on ClamAV There is not much reason to check before waiting on ClamAV. It is more helpful to debug failures from `nc` mail send commands if we know that nothing went wrong inbetween the ClamAV wait time. Additionally added an assertion which should provide more information if this part of the test setup fails again. * tests(fix): Move health check to the top This test is a bit fragile. It relies on defaults for the healthcheck with intervals of 30 seconds. If the check occurs while Postfix is down due a change event from earlier tests and the healthcheck kicks in at that point, then if there is not enough time to refresh the health status from `unhealthy`, the test will fail with a false-positive as Postfix is actually working and up again.. * tests(fix): Wait on directory to be removed Workaround that tries not to introduce heavier delays by waiting on a full change event to complete in the previous `email update` if possible. There is a chance that the account has the folder deleted, but restored from an active change event (for password update, then the account delete).
2022-08-22 22:15:06 +00:00
add_mail_account_then_wait_until_ready mail 'quota_user@domain.tld'
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
run docker exec mail /bin/sh -c "grep -i 'quota_user@domain.tld' /tmp/docker-mailserver/dovecot-quotas.cf"
assert_failure
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
run docker exec mail /bin/sh -c "delquota quota_user@domain.tld"
assert_success
run docker exec mail /bin/sh -c "delquota quota_user@domain.tld"
assert_success
fix: Remove `mkcert.sh` usage + `_setup_ssl` refactor. (#2196) * chore(refactor): DRY up the `_setup_ssl` method - `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`. - The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead. - This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result. - Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths. - Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor. - `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key. - That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files. - I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion). --- * chore: Remove mkcert logic and dovecot cert - No longer serving a purpose. - Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`. - To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image. - Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition. --- * Error handling for SSL_TYPE - Added a panic utility to exit early when SSL_TYPE conditions are misconfigured. - Some info text had order of key/cert occurrence swapped to be consistent with key then cert. - Some existing comments moved and rephrased. - Additional comments added. - `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks). - _notify messages lifted out of conditionals so that they always output when the case is hit. - ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~ --- * Use sedfile + improve sed expressions + update case style - Uses sedfile when appropriate (file change intentional, not optional match/check). - sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)). - sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement. - case statements adopt the `(` prefix, adopting recent shell style for consistency. --- * Refactor SSL_TYPE=disabled - Postfix is also disabled now. - Included heavy inline documentation reference for maintainers. - Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'. --- * I'm a friggin' sed wizard now - The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`. - I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines). - Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better. --- * tests(fix): Tests that require SSL/TLS to pass - SSL_TYPE=snakeoil added as temporary workaround. - nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests. - Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past. - The removed SSL test is already handled in mail_ssl_manual.bats ldap test: - Replace `--link` alias option with `--network` and alias assignment. - Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure. privacy test: - Also needed `SSL_TYPE` to pass the starttls test. `tests.bats` had another starttls test for imap: - Workaround for now is to give the main test container `SSL_TYPE=snakeoil`. --- * Remove the expired lets-encrypt cert This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores. No longer the case today. --- * chore: Housekeeping Not required for this PR branch, little bit of tidying up while working on these two test files. - privacy test copied over content when extracted from `tests.bats` that isn't relevant. - ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find. - Additionally converted the two localhost FQDN to vars. --- * Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case) - This is to prevent other tests from failing by hitting the panic catchall case. - More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value. --- * Add inline documentation for `dms_panic` - This could later be better formatted and placed into contributor docs. Panic with kill (shutdown) not exit (errex): - `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process). - `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't. The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-19 12:31:11 +00:00
run docker exec mail /bin/sh -c "delmailuser -y quota_user@domain.tld"
assert_success
}
@test "checking quota: dovecot quota present in postconf" {
run docker exec mail /bin/bash -c "postconf | grep 'check_policy_service inet:localhost:65265'"
assert_success
}
@test "checking quota: dovecot mailbox max size must be equal to postfix mailbox max size" {
postfix_mailbox_size=$(docker exec mail sh -c "postconf | grep -Po '(?<=mailbox_size_limit = )[0-9]+'")
2020-10-18 00:08:11 +00:00
run echo "${postfix_mailbox_size}"
refute_output ""
# dovecot relies on virtual_mailbox_size by default
postfix_virtual_mailbox_size=$(docker exec mail sh -c "postconf | grep -Po '(?<=virtual_mailbox_limit = )[0-9]+'")
2020-10-18 00:08:11 +00:00
assert_equal "${postfix_virtual_mailbox_size}" "${postfix_mailbox_size}"
2020-10-18 00:08:11 +00:00
postfix_mailbox_size_mb=$(( postfix_mailbox_size / 1000000))
dovecot_mailbox_size_mb=$(docker exec mail sh -c "doveconf | grep -oP '(?<=quota_rule \= \*\:storage=)[0-9]+'")
2020-10-18 00:08:11 +00:00
run echo "${dovecot_mailbox_size_mb}"
refute_output ""
2020-10-18 00:08:11 +00:00
assert_equal "${postfix_mailbox_size_mb}" "${dovecot_mailbox_size_mb}"
}
@test "checking quota: dovecot message max size must be equal to postfix messsage max size" {
postfix_message_size=$(docker exec mail sh -c "postconf | grep -Po '(?<=message_size_limit = )[0-9]+'")
2020-10-18 00:08:11 +00:00
run echo "${postfix_message_size}"
refute_output ""
2020-10-18 00:24:26 +00:00
postfix_message_size_mb=$(( postfix_message_size / 1000000))
dovecot_message_size_mb=$(docker exec mail sh -c "doveconf | grep -oP '(?<=quota_max_mail_size = )[0-9]+'")
2020-10-18 00:08:11 +00:00
run echo "${dovecot_message_size_mb}"
refute_output ""
2020-10-18 00:08:11 +00:00
assert_equal "${postfix_message_size_mb}" "${dovecot_message_size_mb}"
}
@test "checking quota: quota directive is removed when mailbox is removed" {
chore: `addmailuser` - Remove delaying completion until `/var/mail` is ready (#2729) ## Quick Summary Resolves a `TODO` task with `addmailuser`. ## Overview The main change is adding three new methods in `common.bash`, which replace the completion delay in `addmailuser` / `setup email add` command. Other than that: - I swapped `sh -c 'addmailuser ...'` to `setup email add ...`. - Improved three tests in `setup-cli.bats` for `setup email add|update|del` (_logic remains effectively the same still_). - Rewrote the `TODO` comment for `setup-cli.bats` test on `setup email del` to better clarify the concern, but the test itself was no longer affected due to changes prior to this PR, so I enabled the commented out assertion. - Removed unnecessary waits. The two `skip` tests in `test/tests.bats` could be enabled again after this PR. - Additional fixes to tests were made during the PR (see discussion comments for details), resolving race conditions. Individual commit messages of the PR provide additional details if helpful. --- ## Relevant commit messages * chore: Remove creation delay in `addmailuser` This was apparently only for supporting tests that need to wait on account creation being ready to test against. As per the removed inline docs, it should be fine to remove once tests are updated to work correctly without it. * tests(feat): Add two new common helper methods `wait_until_account_maildir_exists()` provides the same logic `addmailuser` command was carrying, to wait upon the account dir creation in `/var/mail`. As this was specifically to support tests, it makes more sense as a test method. `add_mail_account_then_wait_until_ready()` was added to handle the common pattern of creating account and waiting on it. An internal assert will ensure the account was successfully created first during the test before attempting to wait. * tests(feat): Add common helper for waiting on change event to be processed The current helper is more complicated for no real benefit, it only detects when a change is made that would trigger a change event in the `changedetector` service. Our usage of this in tests however is only interested in waiting out the completion of the change event. Remove unnecessary change event waits. These waits should not be necessary if handled correctly. * tests: `addmailuser` to `add_mail_account_then_wait_until_ready mail()` This helper method is used where appropriate. - A password is not relevant (optional). - We need to wait on the creation on the account (Dovecot and `/var/mail` directory). * tests: `setup-cli` revise `add`, `update`, `del` tests The delete test was failing as the `/var/mail` directory did not yet exist. There is now a proper delay imposed in the `add` test now shares the same account for both `update` and `del` tests resolving that failure. Additionally tests use better asserts where appropriate and the wait + sleep logic in `add` has been improved (now takes 10 seconds to complete, approx half the time than before). The `del` test TODO while not technically addressed is no longer relevant due to the tests being switched to `-c` option (there is a separate `no container` test file, but it doesn't provide a `del` test). * tests(fix): Ensure Postfix is reachable after waiting on ClamAV There is not much reason to check before waiting on ClamAV. It is more helpful to debug failures from `nc` mail send commands if we know that nothing went wrong inbetween the ClamAV wait time. Additionally added an assertion which should provide more information if this part of the test setup fails again. * tests(fix): Move health check to the top This test is a bit fragile. It relies on defaults for the healthcheck with intervals of 30 seconds. If the check occurs while Postfix is down due a change event from earlier tests and the healthcheck kicks in at that point, then if there is not enough time to refresh the health status from `unhealthy`, the test will fail with a false-positive as Postfix is actually working and up again.. * tests(fix): Wait on directory to be removed Workaround that tries not to introduce heavier delays by waiting on a full change event to complete in the previous `email update` if possible. There is a chance that the account has the folder deleted, but restored from an active change event (for password update, then the account delete).
2022-08-22 22:15:06 +00:00
add_mail_account_then_wait_until_ready mail 'quserremoved@domain.tld'
run docker exec mail /bin/sh -c "setquota quserremoved@domain.tld 12M"
assert_success
run docker exec mail /bin/sh -c 'cat /tmp/docker-mailserver/dovecot-quotas.cf | grep -E "^quserremoved@domain.tld\:12M\$" | wc -l | grep 1'
assert_success
run docker exec mail /bin/sh -c "delmailuser -y quserremoved@domain.tld"
assert_success
run docker exec mail /bin/sh -c 'cat /tmp/docker-mailserver/dovecot-quotas.cf | grep -E "^quserremoved@domain.tld\:12M\$"'
assert_failure
}
@test "checking quota: dovecot applies user quota" {
run docker exec mail /bin/sh -c "doveadm quota get -u 'user1@localhost.localdomain' | grep 'User quota STORAGE'"
assert_output --partial "- 0"
run docker exec mail /bin/sh -c "setquota user1@localhost.localdomain 50M"
assert_success
2020-04-24 15:03:09 +00:00
# wait until quota has been updated
run repeat_until_success_or_timeout 20 sh -c "docker exec mail sh -c 'doveadm quota get -u user1@localhost.localdomain | grep -oP \"(User quota STORAGE\s+[0-9]+\s+)51200(.*)\"'"
assert_success
run docker exec mail /bin/sh -c "delquota user1@localhost.localdomain"
assert_success
2020-04-24 15:03:09 +00:00
# wait until quota has been updated
run repeat_until_success_or_timeout 20 sh -c "docker exec mail sh -c 'doveadm quota get -u user1@localhost.localdomain | grep -oP \"(User quota STORAGE\s+[0-9]+\s+)-(.*)\"'"
assert_success
}
@test "checking quota: warn message received when quota exceeded" {
skip 'disabled as it fails randomly: https://github.com/docker-mailserver/docker-mailserver/pull/2511'
# create user
chore: `addmailuser` - Remove delaying completion until `/var/mail` is ready (#2729) ## Quick Summary Resolves a `TODO` task with `addmailuser`. ## Overview The main change is adding three new methods in `common.bash`, which replace the completion delay in `addmailuser` / `setup email add` command. Other than that: - I swapped `sh -c 'addmailuser ...'` to `setup email add ...`. - Improved three tests in `setup-cli.bats` for `setup email add|update|del` (_logic remains effectively the same still_). - Rewrote the `TODO` comment for `setup-cli.bats` test on `setup email del` to better clarify the concern, but the test itself was no longer affected due to changes prior to this PR, so I enabled the commented out assertion. - Removed unnecessary waits. The two `skip` tests in `test/tests.bats` could be enabled again after this PR. - Additional fixes to tests were made during the PR (see discussion comments for details), resolving race conditions. Individual commit messages of the PR provide additional details if helpful. --- ## Relevant commit messages * chore: Remove creation delay in `addmailuser` This was apparently only for supporting tests that need to wait on account creation being ready to test against. As per the removed inline docs, it should be fine to remove once tests are updated to work correctly without it. * tests(feat): Add two new common helper methods `wait_until_account_maildir_exists()` provides the same logic `addmailuser` command was carrying, to wait upon the account dir creation in `/var/mail`. As this was specifically to support tests, it makes more sense as a test method. `add_mail_account_then_wait_until_ready()` was added to handle the common pattern of creating account and waiting on it. An internal assert will ensure the account was successfully created first during the test before attempting to wait. * tests(feat): Add common helper for waiting on change event to be processed The current helper is more complicated for no real benefit, it only detects when a change is made that would trigger a change event in the `changedetector` service. Our usage of this in tests however is only interested in waiting out the completion of the change event. Remove unnecessary change event waits. These waits should not be necessary if handled correctly. * tests: `addmailuser` to `add_mail_account_then_wait_until_ready mail()` This helper method is used where appropriate. - A password is not relevant (optional). - We need to wait on the creation on the account (Dovecot and `/var/mail` directory). * tests: `setup-cli` revise `add`, `update`, `del` tests The delete test was failing as the `/var/mail` directory did not yet exist. There is now a proper delay imposed in the `add` test now shares the same account for both `update` and `del` tests resolving that failure. Additionally tests use better asserts where appropriate and the wait + sleep logic in `add` has been improved (now takes 10 seconds to complete, approx half the time than before). The `del` test TODO while not technically addressed is no longer relevant due to the tests being switched to `-c` option (there is a separate `no container` test file, but it doesn't provide a `del` test). * tests(fix): Ensure Postfix is reachable after waiting on ClamAV There is not much reason to check before waiting on ClamAV. It is more helpful to debug failures from `nc` mail send commands if we know that nothing went wrong inbetween the ClamAV wait time. Additionally added an assertion which should provide more information if this part of the test setup fails again. * tests(fix): Move health check to the top This test is a bit fragile. It relies on defaults for the healthcheck with intervals of 30 seconds. If the check occurs while Postfix is down due a change event from earlier tests and the healthcheck kicks in at that point, then if there is not enough time to refresh the health status from `unhealthy`, the test will fail with a false-positive as Postfix is actually working and up again.. * tests(fix): Wait on directory to be removed Workaround that tries not to introduce heavier delays by waiting on a full change event to complete in the previous `email update` if possible. There is a chance that the account has the folder deleted, but restored from an active change event (for password update, then the account delete).
2022-08-22 22:15:06 +00:00
add_mail_account_then_wait_until_ready mail 'quotauser@otherdomain.tld'
run docker exec mail /bin/sh -c 'setquota quotauser@otherdomain.tld 10k'
assert_success
2020-04-24 15:03:09 +00:00
# wait until quota has been updated
run repeat_until_success_or_timeout 20 sh -c "docker exec mail sh -c 'doveadm quota get -u quotauser@otherdomain.tld | grep -oP \"(User quota STORAGE\s+[0-9]+\s+)10(.*)\"'"
assert_success
# dovecot and postfix has been restarted
wait_for_service mail postfix
wait_for_service mail dovecot
Final Migration Step (#6) * first migration steps * altered issue templates * altered README * removed .travis.yml * adjusting registry & repository, Dockerfile and compose.env * Close stale issues automatically * Integrated CI with Github Actions (#3) * feat: integrated ci with github actions * fix: use secrets for docker org and update image * docs: clarify why we use -t if no tty exists * fix: correct remaining references to old repo chore: prettier automatically updated markdown as well * fix: hardcode docker org * change testing image to just testing * ci: add armv7 as a supported platform * finished migration steps * corrected linting in build-push action * corrected linting in build-push action (2) * minor preps for PR * correcting push on pull request and minor details * adjusted workflows to adhere closer to @wernerfred's diagram * minor patches * adjusting Dockerfile's installation of base packages * adjusting schedule for stale issue action * reverting license text * improving CONTRIBUTING.md PR text * Update CONTRIBUTING.md * a bigger patch at the end * moved all scripts into one directory under target/scripts/ * moved the quota-warning.sh script into target/scripts/ and removed empty directory /target/dovecot/scripts * minor fixes here and there * adjusted workflows for use a fully qualified name (i.e. docker.io/...) * improved on the Dockerfile layer count * corrected local tests - now they (actually) work (fine)! * corrected start-mailserver.sh to make use of defaults consistently * removed very old, deprecated variables (actually only one) * various smaller improvements in the end * last commit before merging #6 * rearranging variables to use alphabetic order Co-authored-by: casperklein <casperklein@users.noreply.github.com> Co-authored-by: Nick Pappas <radicand@users.noreply.github.com> Co-authored-by: William Desportes <williamdes@wdes.fr>
2021-01-16 09:16:05 +00:00
sleep 10
# send some big emails
run docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/quota-exceeded.txt"
assert_success
run docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/quota-exceeded.txt"
assert_success
run docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/email-templates/quota-exceeded.txt"
assert_success
# check for quota warn message existence
run repeat_until_success_or_timeout 20 sh -c "docker exec mail sh -c 'grep \"Subject: quota warning\" /var/mail/otherdomain.tld/quotauser/new/ -R'"
assert_success
Final Migration Step (#6) * first migration steps * altered issue templates * altered README * removed .travis.yml * adjusting registry & repository, Dockerfile and compose.env * Close stale issues automatically * Integrated CI with Github Actions (#3) * feat: integrated ci with github actions * fix: use secrets for docker org and update image * docs: clarify why we use -t if no tty exists * fix: correct remaining references to old repo chore: prettier automatically updated markdown as well * fix: hardcode docker org * change testing image to just testing * ci: add armv7 as a supported platform * finished migration steps * corrected linting in build-push action * corrected linting in build-push action (2) * minor preps for PR * correcting push on pull request and minor details * adjusted workflows to adhere closer to @wernerfred's diagram * minor patches * adjusting Dockerfile's installation of base packages * adjusting schedule for stale issue action * reverting license text * improving CONTRIBUTING.md PR text * Update CONTRIBUTING.md * a bigger patch at the end * moved all scripts into one directory under target/scripts/ * moved the quota-warning.sh script into target/scripts/ and removed empty directory /target/dovecot/scripts * minor fixes here and there * adjusted workflows for use a fully qualified name (i.e. docker.io/...) * improved on the Dockerfile layer count * corrected local tests - now they (actually) work (fine)! * corrected start-mailserver.sh to make use of defaults consistently * removed very old, deprecated variables (actually only one) * various smaller improvements in the end * last commit before merging #6 * rearranging variables to use alphabetic order Co-authored-by: casperklein <casperklein@users.noreply.github.com> Co-authored-by: Nick Pappas <radicand@users.noreply.github.com> Co-authored-by: William Desportes <williamdes@wdes.fr>
2021-01-16 09:16:05 +00:00
2020-04-24 13:53:44 +00:00
run repeat_until_success_or_timeout 20 sh -c "docker logs mail | grep 'Quota exceeded (mailbox for user is full)'"
assert_success
# ensure only the first big message and the warn message are present (other messages are rejected: mailbox is full)
run docker exec mail sh -c 'ls /var/mail/otherdomain.tld/quotauser/new/ | wc -l'
assert_success
assert_output "2"
run docker exec mail /bin/sh -c "delmailuser -y quotauser@otherdomain.tld"
assert_success
}
#
# PERMIT_DOCKER mynetworks
#
@test "checking PERMIT_DOCKER: can get container ip" {
run docker exec mail /bin/sh -c "ip addr show eth0 | grep 'inet ' | sed 's/[^0-9\.\/]*//g' | cut -d '/' -f 1 | egrep '[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.[[:digit:]]{1,3}'"
assert_success
}
@test "checking PERMIT_DOCKER: my network value" {
run docker exec mail /bin/sh -c "postconf | grep '^mynetworks =' | egrep '[[:digit:]]{1,3}\.[[:digit:]]{1,3}\.0\.0/16'"
assert_success
}
#
# amavis
#
@test "checking amavis: config overrides" {
run docker exec mail /bin/sh -c "grep 'Test Verification' /etc/amavis/conf.d/50-user | wc -l"
assert_success
assert_output 1
}
# TODO investigate why this test fails
@test "checking user login: predefined user can login" {
skip 'disabled as it fails randomly: https://github.com/docker-mailserver/docker-mailserver/pull/2177'
run docker exec mail /bin/bash -c "doveadm auth test -x service=smtp pass@localhost.localdomain 'may be \\a \`p^a.*ssword' | grep 'passdb'"
assert_output "passdb: pass@localhost.localdomain auth succeeded"
}
#
# LDAP
#
# postfix
@test "checking dovecot: postmaster address" {
run docker exec mail /bin/sh -c "grep 'postmaster_address = postmaster@my-domain.com' /etc/dovecot/conf.d/15-lda.conf"
assert_success
}
@test "checking spoofing: rejects sender forging" {
# checking rejection of spoofed sender
wait_for_smtp_port_in_container_to_respond mail
2020-09-21 23:09:47 +00:00
run docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/auth/added-smtp-auth-spoofed.txt"
assert_output --partial 'Sender address rejected: not owned by user'
}
@test "checking spoofing: accepts sending as alias" {
run docker exec mail /bin/sh -c "nc 0.0.0.0 25 < /tmp/docker-mailserver-test/auth/added-smtp-auth-spoofed-alias.txt | grep 'End data with'"
assert_success
}
#
# Pflogsumm delivery check
#
@test "checking pflogsum delivery" {
# checking logrotation working and report being sent
docker exec mail logrotate --force /etc/logrotate.d/maillog
sleep 10
run docker exec mail grep "Subject: Postfix Summary for " /var/mail/localhost.localdomain/user1/new/ -R
assert_success
# check sender is the one specified in REPORT_SENDER
run docker exec mail grep "From: report1@mail.my-domain.com" /var/mail/localhost.localdomain/user1/new/ -R
assert_success
# check sender is not the default one.
run docker exec mail grep "From: mailserver-report@mail.my-domain.com" /var/mail/localhost.localdomain/user1/new/ -R
assert_failure
}
#
# supervisor
#
@test "checking restart of process: postfix" {
run docker exec mail /bin/bash -c "pkill master && sleep 10 && ps aux --forest | grep -v grep | grep '/usr/lib/postfix/sbin/master'"
assert_success
}
@test "checking restart of process: clamd" {
run docker exec mail /bin/bash -c "pkill clamd && sleep 10 && ps aux --forest | grep -v grep | grep '/usr/sbin/clamd'"
assert_success
}
@test "checking restart of process: amavisd-new" {
run docker exec mail /bin/bash -c "pkill amavi && sleep 12 && ps aux --forest | grep -v grep | grep '/usr/sbin/amavisd-new (master)'"
assert_success
}
@test "checking restart of process: opendkim" {
run docker exec mail /bin/bash -c "pkill opendkim && sleep 10 && ps aux --forest | grep -v grep | grep '/usr/sbin/opendkim'"
assert_success
}
@test "checking restart of process: opendmarc" {
run docker exec mail /bin/bash -c "pkill opendmarc && sleep 10 && ps aux --forest | grep -v grep | grep '/usr/sbin/opendmarc'"
assert_success
}
#
# root mail delivery
#
@test "checking that mail for root was delivered" {
run docker exec mail grep "Subject: Root Test Message" /var/mail/localhost.localdomain/user1/new/ -R
assert_success
}