Commit graph

2410 commits

Author SHA1 Message Date
Brennan Kinney 94a9d2af44
added ALWAYS_RUN target Makefile recipies (#3013)
This ensures the recipies are always run.

Co-authored-by: georglauterbach <44545919+georglauterbach@users.noreply.github.com>
2023-01-22 00:15:12 +01:00
Georg Lauterbach e3c4ef76c6
tests(refactor): Improve consistency and documentation for test helpers (#3012) 2023-01-22 00:05:28 +01:00
Brennan Kinney fb82082cf1
tests(refactor): mail_fetchmail.bats + co-locate test cases for processes (#3010)
* chore: Co-locate process checking and process restart verification

Extract the test cases for checking a process is running and properly restarts from various test files into a single one:

Core (always running):
opendkim, opendmarc, master (postfix)

ENV dependent:
amavi (amavisd-new), clamd, dovecot, fail2ban-server (fail2ban), fetchmail, postgrey, postsrsd, saslauthd

These now run off a single container with the required ENV and call a common function (the revised version in parallel test cases).

* fix(saslauthd): Quote wrap supervisor config vars

`saslauth.conf` calls `-O` option for most commands defined with an ENV that may be empty/null. This would cause the process to silently fail / die.

This doesn't happen if quote wrapping the ENV, which calls `-O` with an empty string.

Not necessary, but since one of `postgrey` ENV were quote wrapped in `supervisor-app.conf`, I've also done the same there.

* fix(postsrsd): Change supervisor `autorestart` policy to `true`

The PR that introduced the config switched from `true` to `unexpected` without any context. That prevents restart working when the process is killed. Setting to `true` instead will correctly restart the service.

* chore: Remove disabled postgrey test file

`mail_with_postgrey_disabled_by_default.bats` only checked the migrated test cases, removed as no longer serving a purpose.

* tests(refactor): Make `_should_restart_when_killed()` more reliable

The previous version did not ensure that the last checks process was actually restarted, only that it was running.

It turns out that `pkill` is only sending the signal, there can be some delay before the original process is actually killed and restarted.

This can be identified with `pgrep --older <seconds>`. First ensure the process is at a specified age, then after killing check that the process is not running that is at least that old, finally check that there is a younger process actually running.. (_could fail if a process doesn't restart, or there is a delay such as imposed by `sleep` in wrapper scripts for postfix and fail2ban_)

The helper method is not used anywhere else now, move it into this test instead. It has been refactored to accomodate the needs for `--older`, and `--list-full` provides some output that can be matched (similar for `pkill --echo`).

* test(docs): Add inline notes about processes

* chore: Compress test cases into single case with loop

Moves the list of processes into array vars to iterate through instead.

If a failure occurs, the process name is visible along with line number in `_should_restart_when_killed()` to identify what went wrong.

* chore: Handle `FETCHMAIL_PARALLEL=1` process checks as well

* tests: Add test case for disabled ENV

Additional coverage to match what other test files were doing before, ensuring that these ENV can prevent their respective service from running.

* chore: Move `clamd` enabled check to it's own test case

Not sure about this.

It reduces the time of CPU activity (sustained full load on a thread) and increase in memory usage (1GB+ loading signatures database), but as a separate test case it also adds 10 seconds without reducing the time of the test case it was extracted from.

* chore: Make `disabled` variant the 1st test case

* fix: Adjust test cases to pass when using slower wrapper scripts

* tests(refactor): `mail_fetchmail.bats` updated to new format

Additionally merges in the parallel test file.

* chore: Move `config/fetchmail.cf` into separate sub-directory

Keep out of the default base config for tests.

* chore: Change `fetchmail.cf` FQDNs to `.test` TLD

Changed the first configs remote and local user values to more clearly document what their values should represent (_and that they don't need to be a full mail address, that's just what our Dovecot is configured with for login_).

Shifted the `here` to the end of the `is` line. It's optional syntax, only intended to contrast with the remote `there` for readability.

Additionally configured imap protocol. Not tested or verified if that's correct configuration for usage with imap protocol instead. The fetchmail feature tests are currently lacking.

Added an inline doc into the fetchmail test to reference a PR about the importance of the trailing `.` in the config. Updated the partial matching to ensure it matches for that in the value as well.

* chore: Finalize `process-check-restart.bats`

Few minor adjustments. The other ENV for clamd doesn't seem to provide any benefit, trim out the noise. Added a note about why it's been split out.

Fetchmail parallel configs are matching the config file path in the process command that is returned. The `.rc` suffix is just to add further clarity to that.
2023-01-18 14:42:55 +13:00
dependabot[bot] e64827e4b2
chore(deps): Bump docker/build-push-action from 3.2.0 to 3.3.0 (#3008)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-16 17:33:09 +00:00
dependabot[bot] dbe0d8c14f
chore(deps): Bump docker/metadata-action from 4.1.1 to 4.3.0 (#3009)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-16 18:31:40 +01:00
Brennan Kinney 8d80c6317f
tests(refactor): Adjust mail_changedetector + change detection helpers (#2997)
* tests(refactor): `mail_changedetector.bats` - Leverage DRY methods

`supervisorctl tail` is not the most reliably way to get logs for the latest change detection and has been known to be fragile in the past.

We can instead read the full log for the service directly with `tac` and `sed` to extract all log content since the last change detection.

Common asserts have also been extracted out into separate methods.

* tests(chore): Remove sleep and redundant change event

Container 1 is still blocked at this point from an existing lock and change event.

Make the lock stale immediately and no extra sleep is required when paired with the helper method to wait until the event is processed (which should remove the stale lock).

* tests(refactor): Add more DRY methods

- Simplify the test case so it's easier to grok.
- 2nd test case (blocking) extracts out initial setup into a separate method and merges the later service restart logic which is redundant.
- Additional comments for improved context of what is going on / expected.

* tests(chore): Revise the change detection helper method

- Add explicit counting arg to change detection support.
- Extract revised logic into it's own generic helper method.
- Utilize this for a separate method that monitors for a change event having started, but not waiting for completion.

This allows dropping the 40 sec of remaining `sleep` in `mail_changedetector` test. It was also required due to potentially missing the timing of a change event completing concurrently in a 2nd container that needed to be waited on and then checked.

* tests(chore): Migrate to current test conventions

- Switch to common container setup helpers
- Update container name and change usage to variables instead.
- Adopt the new convention of prefix variable for test cases (revised test case descriptions).

* tests(chore): Remove legacy change detection

This has since been replaced with the new helper watches the `changedetector` service logs directly instead of only detecting a change has occurred via checksum comparison.

No tests use this method anymore, it was originally for `tests.bats`. Thus the tests in `test_helper.bats` are being dropped too. The new helper has test coverage in `changedetector` tests.

* chore: Lock removal should not incur `sleep 5` afterwards

- A new lock should be created by this script after removal. The sleep doesn't help avoid a race condition with lock file creation after removal.
- Reduces test time as a bonus.
- Added some additional comments to test.

* tests(chore): `tls_letsencrypt.bats` leverage improved change detection

- No need to wait on the change detection service anymore during container startup.
- No need to count change events processed either as waiting a fixed duration is no longer relied on.
- This makes the reload count method redundant, dropped.

* tests(chore): Convert `setup-cli.bats` to new test conventions

This test file was already adapted to the original common setup helpers.

- `TEST_NAME` replaced with `CONTAINER_NAME`.
- Prefix var added, test case descriptions drop explicit prefix.
- No other changes.

* tests(chore): Extract out helpers related to change-detection

- New helper file for sharing these helpers to tests.
- Includes the helpful log method from changedetector tests.
- No longer need to maintain duplicate copies of these methods during the test migration. All tests that use them are now importing the separate helper file.
- `tls_letsencrypt.bats` has switched to using the log helper.
- Generic log count helper is removed from `test_helper/common.bash` as any test that needs it in future can adapt to `helper/common.bash`.

* tests(refactor): `tls_letsencrypt.bats` remove `_get_service_logs()`

This helper does not seem useful as moving away from `supervisorctl tail` and no other tests had a need for it.

* tests(chore): Remove common setup methods from `test_helper/common.bash`

No other tests depend on this. Future tests will adopt the revised versions from `helper/setup.bash`.

Additionally updates `helper/setup.bash` comments that are no longer applicable to `TEST_TMP_CONFIG` and `CONTAINER_NAME`.

* chore: Use `|| true` to simplify setting `EXPECTED_COUNT` correctly
2023-01-16 20:39:46 +13:00
Jeidnx 8b36e903a2
Fix SRS link in README.md (#3005) 2023-01-15 17:23:06 +01:00
Brennan Kinney 133eb9bc2e
tests(refactor): mail_lmtp_ip.bats (#3004)
* chore: Drop ENV `ENABLE_POSTFIX_VIRTUAL_TRANSPORT`

* tests(chore): Remove redundant `dovecot-lmtp` config

None of this is needed. Only relevant change is changing the LMTP service listener for Dovecot and that can be delegated to `user-patches.sh`.

* tests(refactor): Use `user-patches.sh` instead of replacing config file

The only relevant changes in `test/config/dovecot-lmtp` regarding LMTP was:
- `/etc/dovecot/dovecot.conf` (`protocols = imap lmtp`) and `/etc/dovecot/protocols.d/` (`protocols = $protocols lmtp`).
- `conf.d/10-master.conf` only changed the LMTP service listener from a unix socket to TCP on port 24 (_this was the only change required for the test to pass_).

None of those configs are required as:
- `protocols = imap pop3 lmtp` [is the upstream default](https://doc.dovecot.org/settings/core/#core_setting-protocols), no need to add `lmtp`.
- The LMTP service listener is now configured for the test with `user-patches.sh`.

* tests(refactor): `mail_lmtp_ip.bats`

- Converted to new testing conventions and common container helpers.
- `ENABLE_POSTFIX_VIRTUAL_TRANSPORT` was not relevant, dropped.
- Revised test cases, logic remains the same.
- Large custom config used was not documented and doesn't appear to serve any purpose. Simplified by replacing with a single modification with `user-patches.sh`.
- Added some additional comments for context of test and improvements that could be made.

* tests(chore): Adjust comments

The comment from `mail_hostname` provides no valid context, it was likely copied over from `tests.bats` in Oct 2020 by accident.

The email sent is just for testing, nothing relevant to LMTP.

---

Added additional comment for test to reference extra information from.

* tests(chore): Update similar log line matching

Extracts out the match pattern and formatting commands into separate vars (reduces horizontal scrolling), and includes extra docs about what the matched line should be expected to look like.
2023-01-15 18:33:31 +13:00
Brennan Kinney 1650cdf76f
chore: Remove the Makefile backup target (#3000)
* chore: Remove `backup` target from Makefile

- The `backup` target is no longer serving any value to us. It was made redundant with changes added in Oct 2020.
- `clean` target inline docs revised.
- `.gitignore` remove test lines that are no longer valid.

* chore: Parallel test target split to multi-line

* tests(fix): Test `setup.sh` with temporary config dir

The `no_containers.bats` test has many redundant test cases already covered by `setup-cli`. They're basically identical. Removed all but one.

This removes some config dirs that were being explicitly created instead of using the test helper to generate a directory that can be used to test the `-p` option instead.

* ci: Ensure tests are run when `Makefile` is modified
2023-01-13 10:13:42 +13:00
Brennan Kinney a7e6439a39
fix: Workaround postconf write settling logic (#2998)
* fix: Workaround `postconf` write settle logic

After updating `main.cf`, to avoid an enforced delay from reading the config by postfix tools, we can ensure the modified time is at least 2 seconds in the past as a workaround. This should be ok with our usage AFAIK.

Shaves off 2+ seconds roughly off each container startup, reduces roughly 2+ minutes off tests.

* chore: Only modify `mtime` if less than 2 seconds ago

- Slight improvement by avoiding unnecessary writes with a conditional check on the util method.
- Can more comfortably call this during `postfix reload` in the change detection cycle now.
- Identified other tests that'd benefit from this, created a helper method to call instead of copy/paste.
- The `setup email restrict` command also did a modification and reload. Added util method here too.

* tests(fix): `mail_smtponly.bats` should wait for Postfix

- `postfix reload` fails if the service is not ready yet.
- `service postfix reload` and `/etc/init.d/postfix reload` presumably wait until it is ready? (as these work regardless)

* chore: Review feedback - Move reload method into utilities
2023-01-13 10:10:58 +13:00
Brennan Kinney 0ecb647ae2
tests(refactor): Adjust mail_tls_dhparams.bats (#2994)
* tests(chore): `tls-dh-params.bats` - Drop `ONE_DIR` ENV variants

There is no longer special handling for this ENV with this feature, these variant test cases serve no value.

* tests(refactor): `tls-dh-params.bats`

Converted to new common setup helper methods and testing structure.

No `setup_file` needed. Only two test cases used now, the Mozilla check is bundled into the default params test case where it's relevant.

Refactored some logic into common functions. Should be easier to grok intention.

* chore: Apply review feedback

Co-authored-by: Casper <casperklein@users.noreply.github.com>

* chore: Inline functions into test cases

As per review feedback
2023-01-12 10:04:50 +13:00
worldworm f5bcfa2e22
docs: FAQ - Add note for devnull alias gotcha when using a catchall rule (#2949)
* updates docs faq devnull with sub-catch-all
2023-01-11 13:57:11 +13:00
Casper dcf34fd63b
Fix several typos (#2993) 2023-01-11 13:31:21 +13:00
Casper 6ac59ef871
Fix several typos (#2990) 2023-01-10 14:13:50 +01:00
dependabot[bot] 7a6c2d375a
chore(deps): Bump peaceiris/actions-gh-pages from 3.9.0 to 3.9.1 (#2992) 2023-01-09 13:32:05 +01:00
Casper eeb6b72b3e
Add tools (ping & dig) to the image (#2989)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2023-01-09 13:13:36 +01:00
Georg Lauterbach 41c44cb91d
update BATS & helper + minor updates to BATS variables (#2988) 2023-01-09 08:54:04 +01:00
Brennan Kinney 2b4105ef0a
chore(housekeeping): Cleaning up broken links (#2667)
These two links have remained broken for over 6 months. Removing them. 

* chore(housekeeping): Broken links

* chore: Remove broken links from `mailserver.env`
2023-01-09 12:22:37 +13:00
Y.C.Huang 88715974eb
docs: Provision a cert with the ACME DNS-01 challenge via Certbot + Cloudflare (#2968)
* docs: Certbot cloudflare
Add docs for implement certbot-dns-cloudflare to generate certificate for mail server

* Apply suggestions from code review

* fix: certbot-cloudflare docs

Fix the docker-compose command according to the advice

* feat: DNS-Cloudflare certificate renew
Add docs for implementing renewing certificate with crontab

* Apply suggestions from code review

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2023-01-07 11:58:50 +13:00
Brennan Kinney 1024e0ccf2
tests: Extract some test cases out from tests.bats (#2980)
While working on tests, I noticed that some of the configs being mounted were adding a few seconds to the start-up time of each container. Notably `postfix-*` and `dovecot.conf` config files, which have been extracted out into their own tests with those files moved into a separate config folder.

`tests.bats` has been adapted to the common setup helper, and removed ENV no longer required to run those tests. Future PRs will extract out more tests.

Review may be easier via individual commit diffs and their associated commit messages describing relevant changes.

<details>
<summary>Commit message history for reference</summary>

```markdown
tests(chore): `tests.bats` - Remove redundant config
===
- ONEDIR volume support no longer relevant, this should have been dropped.
- ClamAV ENV no longer relevant as related tests have been extracted already.
- Same with the some of the SpamAssassin ENV config.
- `VIRUSMAILS_DELETE_DELAY` is tested in the file, but doesn't use this ENV at all? (runs a separate instance to test the ENV instead)
- Hostname updated in preparation for migrating to new test helpers. Relevant test lines referencing the hostname have likewise been updated.
```

```markdown
tests(chore): `tests.bats` - Convert to common setup
===
ENV remains the same, but required adding `ENABLE_AMAVIS=1` to bring that back, while the following became redundant as they're now defaulting to explicitly disabled in the helper method:

- `ENABLE_CLAMAV=0`
- `LOG_LEVEL=debug`
- `ENABLE_UPDATE_CHECK=0`
- `--hostname` + `--tty` + standard `--volume` lines
- `-e` option expanded to long-name `--env`, and all `\` dropped as no longer necessary.

`wait_for_finished_setup_in_container` is now redundant thanks to `common_container_setup`.
```

```markdown
tests(refactor): `tests.bats` - Extract out Dovecot Sieve tests
===
Sieve test files relocated into `test/config/dovecot-sieve/` for better isolation.

`dovecot.sieve` was not using the `reject` import, and we should not encourage it? (docs still do):
https://support.tigertech.net/sieve#the-sieve-reject-jmp
```

```markdown
tests: `tests.bats` - Extract out `checking smtp` tests
===
Migrated to the standard template and copied over the original test cases with `_run_in_container` adjustment only.

Identified minimum required ENV along with which mail is required for each test case.
```

```markdown
tests(refactor): `smtp-delivery.bats`
===
- Disabled `ENABLE_SRS=1`, not necessary for these tests.
- Added a SpamAssassin related test (X-SPAM headers) which requires `SA_TAG` to properly pass (or `ENABLE_SRS=1` to deliver into inbox).
- Many lines with double quotes changed to single quote wrapping, and moving out `grep` filters into `assert_output --partial` lines instead.
- Instead of `wc -l` making failures less helpful, switch to the helper method `_should_output_number_of_lines`
- x2 `assert_output` with different EOF style of usage was not actually failing on tests when it should. Changed to assert partial output of each expected line, and count the number of lines instead.
- Added additional comments related to the test cases with a `TODO` note about `SPAMASSASSIN_SPAM_TO_INBOX=1`.
- Revised test case names, including using the common prefix var.
- `tests.bats` no longer needs to send all these emails, no other test cases require them. This affects a test checking a `/mail` folder exists which has been corrected, and a quotas test case adjusted to expect an empty quota size output.
```

```markdown
tests: `tests.bats` - Extract out test cases for config overrides
===
Slight improvement by additionally matching `postconf` output to verify the setting is properly applied.
```

```markdown
tests: `tests.bats` - Extract out Amavis SpamAssassin test case
===
Removes the need for SpamAssassin ENV in `tests.bats`.
```

</details>
2023-01-07 11:36:20 +13:00
Casper 89352ce363
Add docker-data/ (#2982) 2023-01-06 09:09:58 +01:00
Brennan Kinney 623d53bea8
Merge pull request #2938 from docker-mailserver/ci/more-parallel-tests
tests: Convert more serial tests into parallel ready

<details>
<summary>Commit message history for reference</summary>

```markdown
tests(chore): Move some serial tests into parallel sets
===
Additionally with the `tls.bash` helper for the `letsencrypt` tests.
```

```markdown
tests: Adjusted files not directly related to tests
===
`tls.bash` helper was adapted to the new helper scripts location. The `setup.bash` helper saw a bugfix (expanding the array properly) and updates the container default config to configure for IPv4 explicitly.

The IPv4 default was added after recent Docker pushes and I saw weird IPv6 related errors in the logs.. now we're sure IPv4 is the default during tests.

Added functionality to check if a process is running:
- This change adds a helper function to check whether a program is running inside a container or not.
- This added the need for a function like `_run_in_container` but allowing for providing an explicit container name.
- Future PRs can use this helper function now to check whether a process is running or not. This was done for the tests of Fail2Ban, but can be used for other tests in the future as well.

---

chore: Restructured BATS flags in `Makefile`

The `Makefile` has seen a bit of a restructuring when it comes to flags:

1. The `MAKEFLAGS` variables is used by `make`, and allows for adding additional flags that can be used within in recursive calls (via `$(MAKE)`) too,  thus DRY approach.
2. The flags for calling BATS were adjusted. `--no-parallelize-within-files` has been added as well to ensure tests  _inside_ a single file are run sequentially.

`dms-test` prefix matching changed to expect a `_` suffix as a delimiter.

---

docs: Add a note regarding output from running tests in parallel
```

```markdown
tests: Adjust parallel tests
===
- The usual serial to parallel test conversion to utilize the `setup.bash` common setup structure, and adding a `TEST_PREFIX` var for each test case to leverage.
- Standardize on parallel test naming conventions for variables / values.
- More consistent use of `bash -c` instead of `/bin/bash -c` or `/bin/sh -c`.
- Using the `_run_in_container` helper instead of `run docker exec ${CONTAINER_NAME}`.
- Updates tests to use the `check_if_process_is_running` helper.

---

chore: Revise inline docs for the `ssl_letsencrypt` test

- Moves the override to be in closer proximity to the `initial_setup` call, and better communicates the intent to override.
- Removes top comment block that is no longer providing value or correct information to maintainers.
- Revised `acme.json` test case inline doc comments.
```

```markdown
refactor: Parallel Tests
===
- `disabled_clamav_spamassassin`:
  - Just shuffling the test order around, and removing the restart test at the end which doesn't make sense.

- `postscreen`:
  - Now uses common helper for getting container IP
  - Does not appear to need the `NET_ADMIN` capability?
  - Reduced startup time for the 2nd container + additional context about it's relevance.
  - Test cases are largely the same, but refactored the `nc` alternative that properly waits it's turn. This only needs to run once. Added additional commentary and made into a generic method if needed in other tests.

- `fail2ban`:
  - Use the common container IP helper method.
  - Postscreen isn't affecting this test, it's not required to do the much slower exchange with the mail server when sending a login failure.
  - IP being passed into ENV is no longer necessary.
  - `sleep 5` in the related test cases doesn't seem necessary, can better rely on polling with timeout.
  - `sleep 10` for `setup.sh` also doesn't appear to be necessary.

- `postgrey`:
  - Reduced POSTGREY_DELAY to 3, which shaves a fair amount of wasted time while still verifying the delay works.
  - One of the checks in `main.cf` doesn't seem to need to know about the earlier spamhaus portion of the line to work, removed.
  - Better test case descriptions.
  - Improved log matching via standard method that better documents the expected triplet under test.
  - Removed a redundant whitelist file and test that didn't seem to have any relevance. Added a TODO with additional notes about a concern with these tests.
  - Reduced test time as 8 second timeouts from `-w 8` don't appear to be required, better to poll with grep instead.
  - Replaced `wc -l` commands with a new method to assert expected line count, better enabling assertions on the actual output.

- `undef_spam_subject`:
  - Split to two separate test cases, and initialize each container in their case instead of `setup_file()`, allowing for using the default `teardown()` method (and slight benefit if running in parallel).

- `permit_docker`:
  - Not a parallel test, but I realized that the repeat helper methods don't necessarily play well with `run` as the command (can cause false positive of what was successful).
```

```markdown
docs: Revise contributing advice for tests
```

</details>
2023-01-06 16:53:20 +13:00
Brennan Kinney 52987e32e7 docs: Revise contributing advice for tests 2023-01-06 16:50:09 +13:00
Brennan Kinney 0bbec09529 refactor: Parallel Tests
- `disabled_clamav_spamassassin`:
  - Just shuffling the test order around, and removing the restart test at the end which doesn't make sense.

- `postscreen`:
  - Now uses common helper for getting container IP
  - Does not appear to need the `NET_ADMIN` capability?
  - Reduced startup time for the 2nd container + additional context about it's relevance.
  - Test cases are largely the same, but refactored the `nc` alternative that properly waits it's turn. This only needs to run once. Added additional commentary and made into a generic method if needed in other tests.

- `fail2ban`:
  - Use the common container IP helper method.
  - Postscreen isn't affecting this test, it's not required to do the much slower exchange with the mail server when sending a login failure.
  - IP being passed into ENV is no longer necessary.
  - `sleep 5` in the related test cases doesn't seem necessary, can better rely on polling with timeout.
  - `sleep 10` for `setup.sh` also doesn't appear to be necessary.

- `postgrey`:
  - Reduced POSTGREY_DELAY to 3, which shaves a fair amount of wasted time while still verifying the delay works.
  - One of the checks in `main.cf` doesn't seem to need to know about the earlier spamhaus portion of the line to work, removed.
  - Better test case descriptions.
  - Improved log matching via standard method that better documents the expected triplet under test.
  - Removed a redundant whitelist file and test that didn't seem to have any relevance. Added a TODO with additional notes about a concern with these tests.
  - Reduced test time as 8 second timeouts from `-w 8` don't appear to be required, better to poll with grep instead.
  - Replaced `wc -l` commands with a new method to assert expected line count, better enabling assertions on the actual output.

- `undef_spam_subject`:
  - Split to two separate test cases, and initialize each container in their case instead of `setup_file()`, allowing for using the default `teardown()` method (and slight benefit if running in parallel).

- `permit_docker`:
  - Not a parallel test, but I realized that the repeat helper methods don't necessarily play well with `run` as the command (can cause false positive of what was successful).
2023-01-06 16:50:09 +13:00
Brennan Kinney 2ec6c4abc0 tests: Adjust parallel tests
- The usual serial to parallel test conversion to utilize the `setup.bash` common setup structure, and adding a `TEST_PREFIX` var for each test case to leverage.
- Standardize on parallel test naming conventions for variables / values.
- More consistent use of `bash -c` instead of `/bin/bash -c` or `/bin/sh -c`.
- Using the `_run_in_container` helper instead of `run docker exec ${CONTAINER_NAME}`.
- Updates tests to use the `check_if_process_is_running` helper.

---

chore: Revise inline docs for the `ssl_letsencrypt` test

- Moves the override to be in closer proximity to the `initial_setup` call, and better communicates the intent to override.
- Removes top comment block that is no longer providing value or correct information to maintainers.
- Revised `acme.json` test case inline doc comments.
2023-01-06 16:50:09 +13:00
Brennan Kinney 306592fcad tests: Adjusted files not directly related to tests
`tls.bash` helper was adapted to the new helper scripts location. The `setup.bash` helper saw a bugfix (expanding the array properly) and updates the container default config to configure for IPv4 explicitly.

The IPv4 default was added after recent Docker pushes and I saw weird IPv6 related errors in the logs.. now we're sure IPv4 is the default during tests.

Added functionality to check if a process is running:
- This change adds a helper function to check whether a program is running inside a container or not.
- This added the need for a function like `_run_in_container` but allowing for providing an explicit container name.
- Future PRs can use this helper function now to check whether a process is running or not. This was done for the tests of Fail2Ban, but can be used for other tests in the future as well.

---

chore: Restructured BATS flags in `Makefile`

The `Makefile` has seen a bit of a restructuring when it comes to flags:

1. The `MAKEFLAGS` variables is used by `make`, and allows for adding additional flags that can be used within in recursive calls (via `$(MAKE)`) too,  thus DRY approach.
2. The flags for calling BATS were adjusted. `--no-parallelize-within-files` has been added as well to ensure tests  _inside_ a single file are run sequentially.

`dms-test` prefix matching changed to expect a `_` suffix as a delimiter.

---

docs: Add a note regarding output from running tests in parallel
2023-01-06 16:50:09 +13:00
Brennan Kinney a81de22819 tests(chore): Move some serial tests into parallel sets
Additionally with the `tls.bash` helper for the `letsencrypt` tests.
2023-01-06 16:50:09 +13:00
Georg Lauterbach 26ac48f34a
feature: provide initial Rspamd support (#2902) 2023-01-05 08:39:00 +01:00
Gabriel Euzet a00cdcdee9
fix regex in quota activation code (#2958) 2023-01-04 18:37:00 +01:00
Georg Lauterbach 3a8f6b74ad
update: bump Fail2Ban version to v1.0.2 (#2959) 2023-01-04 17:57:08 +01:00
Brennan Kinney 304747fad9
tests: Use mail.example.test as common container hostname (#2975) 2023-01-04 16:24:08 +01:00
Brennan Kinney b7bad82e07
tests(fix): wait_until_change_detection_event_completes to count (#2974) 2023-01-04 13:29:10 +01:00
Georg Lauterbach 1fd407b3d0
change default of DEVCOT_COMMUNITY_REPO to 1 (#2901) 2023-01-02 13:25:14 +01:00
github-actions[bot] 89efafe00c
docs(CONTRIBUTORS): update contributors (#2969)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-01-01 15:17:35 +01:00
René Plötz 55f6260bad
fix: Ensure relay host properly handles credentials check (#2965)
A recent change broke the conditional check. Reverting.
2022-12-28 08:50:03 +13:00
dependabot[bot] a430116e63
chore(deps): Bump actions/stale from 6 to 7 (#2960)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-26 23:01:08 +01:00
Brennan Kinney 2013d10bc5
docs(fix): README - Update CI status badge URL (#2951) 2022-12-24 02:32:06 +13:00
Brennan Kinney 4dda5f8b1f
chore: Drop support for deprecated TLS versions (#2945)
* chore: Set `TLS_INTERMEDIATE_SUITE` to only use TLS 1.2 ciphersuites

Removes support of the following cipher suites that are only valid for TLS 1.0 + 1.1:

- `ECDHE-ECDSA-AES128-SHA`
- `ECDHE-RSA-AES128-SHA`
- `ECDHE-ECDSA-AES256-SHA`
- `ECDHE-RSA-AES256-SHA`
- `DHE-RSA-AES128-SHA`
- `DHE-RSA-AES256-SHA`

* chore: Update TLS version min and ignore settings

These are now the same as modern settings.

* fix: Remove min TLS support workaround

No longer required now that outdated TLS versions have been dropped.

* tests: Remove support for TLS 1.0 and 1.1 ciphersuites

* tests: Remove support for TLS 1.0 and 1.1 ciphersuites (Port 25)

The removed SHA1 cipher suites are still supported in TLS 1.2, thus they've been excluded for port 25 via the `SHA1` exclusion pattern in `main.cf`.
2022-12-24 02:30:43 +13:00
Brennan Kinney b58165762a
fix(changedetector): Use service reload commands instead of supervisorctl restart <service> (#2947)
With `reload` a change detection event during local testing can be processed in less than a second according to logs. Previously this was 5+ seconds (_plus additional downtime for Postfix/Dovecot to become available again_).

In the past it was apparently an issue to use `<service> reload` due to a concern with the PID for wrapper scripts that `supervisorctl` managed, thus `supervisorctl <service> restart` had been used. Past discussions with maintainers suggest this is not likely an issue anymore, and `reload` should be fine to switch to now 👍 

---

**NOTE:** It may not be an issue in the CI, but on _**local systems running tests may risk failure in `setup-cli.bats` from a false positive**_ due to 1 second polling window of the test helper method, and a change event being possible to occur entirely between the two checks undetected by the current approach.

If this is a problem, we may need to think of a better way to catch the change. The `letsencrypt` test counts how many change events are expected to have been processed, and this could technically be leveraged by the test helper too.

---

**NOTE:** These two lines (_with regex pattern for postfix_) are output in the terminal when using the services respective `reload` commands:

```
postfix/master.*: reload -- version .*, configuration /etc/postfix
dovecot: master: Warning: SIGHUP received - reloading configuration
```

I wasn't sure how to match them as they did not appear in the `changedetector` log (_**EDIT:** they appear in the main log output, eg `docker logs <container name>`_).

Instead I've just monitored the `changedetector` log messages, which should be ok for logic that previously needed to ensure Dovecot / Postfix was back up after the `restart` was issued.

---

Commit history:

* chore: Change events `reload` Dovecot and Postfix instead of `restart`

Reloading is faster than restarting the processes.

Restarting is a bit heavy handed here and may no longer be necessary for general usage?

* tests: Adapt tests to support service `reload` instead of `restart`

* chore: Additional logging for debugging change event logs

* fix: Wait on change detection, then verify directory created

Change detection is too fast now (0-1 seconds vs 5+).

Directory being waited on here was created near the end of a change event, reducing that time to detect a change by the utility method further.

We can instead check that the directory exists after the change detection event is completed.

* chore: Keep using the maildir polling check

We don't presently use remote storage in tests, but it might be relevant in future when testing NFS.

This at least avoids any confusing failure happening when that scenario is tested.
2022-12-24 01:57:24 +13:00
Brennan Kinney fe21fe78e2
chore: Remove legacy ENV SASL_PASSWD (#2946)
As per deprecation notice from v11.3 release notes, and a related prior PR; this ENV is to be removed.

It's no longer considered useful, and none of the tests that configured it were actually using it for relaying anything.
2022-12-23 15:30:40 +13:00
Brennan Kinney edaeb89c9b
ci: Drop support for ARM v7 platform (#2943)
This platform has had a deprecation notice for 6 months. It is not worth continuing to maintain support for.
2022-12-23 14:56:36 +13:00
Casper bbe3640864
chore: Update changelog and version (#2944) 2022-12-22 23:27:40 +01:00
Casper b75fc448ea
fix: dovecot-fts-xapian dependency when using dovecot community repository (#2937) 2022-12-21 11:26:03 +01:00
georglauterbach 6108b28c1d Revert "update: update to latest version of Fail2Ban (#2903)"
This reverts commit 295d7f0a05.
2022-12-19 23:02:16 +01:00
Georg Lauterbach 295d7f0a05
update: update to latest version of Fail2Ban (#2903) 2022-12-19 08:40:49 +01:00
Casper 50257b46a7
release: v11.3.0 (#2882) 2022-12-19 07:39:16 +01:00
Vasiliy Gokoyev 7ec75ec0eb
fix: Change the default OpenDMARC policy to reject (#2933) 2022-12-19 08:54:38 +13:00
dependabot[bot] a4dee343d6
chore(deps): Bump myrotvorets/set-commit-status-action (#2931)
Bumps [myrotvorets/set-commit-status-action](https://github.com/myrotvorets/set-commit-status-action) from 1.1.5 to 1.1.6.
- [Release notes](https://github.com/myrotvorets/set-commit-status-action/releases)
- [Commits](https://github.com/myrotvorets/set-commit-status-action/compare/1.1.5...1.1.6)

---
updated-dependencies:
- dependency-name: myrotvorets/set-commit-status-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2022-12-13 08:18:35 +13:00
dependabot[bot] 7772e8ec3e
chore(deps): Bump nwtgck/actions-netlify from 1.2 to 2.0 (#2930)
Bumps [nwtgck/actions-netlify](https://github.com/nwtgck/actions-netlify) from 1.2 to 2.0.
- [Release notes](https://github.com/nwtgck/actions-netlify/releases)
- [Changelog](https://github.com/nwtgck/actions-netlify/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/nwtgck/actions-netlify/compare/v1.2...v2.0)

---
updated-dependencies:
- dependency-name: nwtgck/actions-netlify
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-13 08:16:57 +13:00
pravynandas 0eef7789a2
docs: Improve setup CLI password example (#2926)
* Update setup.sh.md

Example to include password
2022-12-11 09:29:41 +13:00