Commit graph

40 commits

Author SHA1 Message Date
Brennan Kinney 7ca056852f
tests(feat): Complete rewrite of letsencrypt tests (#2286)
* chore: Normalize container setup

Easier to grok what is different between configurations.

- Container name usage replaced with variable
- Volumes defined earlier and redeclared when relevant (only real difference is `VOLUME_LETSENCRYPT`)
- Contextual comment about the `acme.json` copy.
- Quoting `SSL_TYPE`, `SSL_DOMAIN` and `-h` values for syntax highlighting.
- Moved `-t` and `${NAME}` to separate line.
- Consistent indentation.

* chore: DRY test logic

Extracts out repeated test logic into methods

* chore: Scope configs to individual test cases (1/3)

- Preparation step for shifting out the container configs to their own scoped test cases. Split into multiple commits to ease reviewing by diffs for this change.
- Re-arrange the hostname and domain configs to match the expected order of the new test cases.
- Shuffle the hostname and domainname grouped tests into tests per container config scope.
- Collapse the `acme.json` test cases into single test case.

* chore: Scope configs to individual test cases (2/3)

- Shifts the hostname and domainname container configs into their respective scoped test cases.
- Moving the `acme.json` container config produces a less favorable diff, so is deferred to a follow-up commit.
- Test cases updated to refer to their `${CONTAINER_NAME}` var instead of the hard-coded string name.

* chore: Scope configs to individual test cases (3/3)

Final commit to shift out the container configs.

- Common vars are exported in `setup_file()` for the test cases to use without needing to repeat the declaration in each test case.
- `teardown_file()` shifts container removal at end of scoped test case.

* chore: Adapt to `common_container_setup` template

- `CONTAINER_NAME` becomes `TEST_NAME` (`common.bash` helper via `init_with_defaults`).
- `docker run ...` and related configuration is now outsourced to the `common.bash` helper, only extra args that the default template does not cover are defined in the test case.
- `TARGET_DOMAIN`establishes the domain folder name for `/etc/letsencrypt/live`.
- `_should*` methods no longer manage a `CONTAINER_NAME` arg, instead using the `TEST_NAME` global that should be valid as test is run as a sequence of test cases.
- `PRIVATE_CONFIG` and the `private_config_path ...` are now using the global `TEST_TMP_CONFIG` initialized at the start of each test case, slightly different as not locally defined/scoped like `PRIVATE_CONFIG` would be within the test case, hence the explicit choice of a different name for context.

* chore: Minor tweaks

- Test case comment descriptions.
- DRY: `docker rm -f` lines moved to `teardown()`
- Use `wait_for_service` helper instead of checking the `changedetector` script itself is running.
- There is a startup delay before the `changedetector` begins monitoring, wait until it ready event is logged.
- Added a helper to query logs for a service (useful later).
- `/bin/sh` commands reduced to `sh`.
- Change the config check to match and compare output, not number of lines returned. Provides better failure output by bats to debug against.

* chore: Add more test functions for `acme.json`

This just extracts out existing logic from the test case to functions to make the test case itself more readable/terse.

* chore: Housekeeping

No changes, just moving logic around and grouping into inline functions, with some added comments.

* chore: Switch to `example.test` certs

This also required copying the source files to match the expected letsencrypt file structure expected in the test/container usage.

* chore: Delete `test/config/letsencrypt/`

No longer necessary, using the `example.test/` certs instead.

These letsencrypt certs weren't for the domains they were used for, and of course long expired.

* chore: Housekeeping

Add more maintainer comments, rename some functions.

* tests: Expand `acme.json` extraction coverage

Finally able to add more test coverage! :)

- Two new methods to validate expected success/failure of extraction for a given FQDN.
- Added an RSA test prior to the wildcard to test a renewal simulation (just with different cert type).
- Added extra method to make sure we're detecting multiple successful change events, not just a previous logged success (false positive).

* tests: Refactor the negotiate_tls functionality

Covers all ports (except POP) and correctly tests against expected verification status with new `example.test` certs.

The `FQDN` var will be put to use in a follow-up commit.

* tests: Verify the certs contain the expected FQDNs

* chore: Extract TLS test methods into a separate helper script

Can be useful for other TLS tests to utilize.

* chore: Housekeeping

* chore: Fix test typo

There was a mismatch between the output and expected output between these two files "find key for" and "find key & cert for". Changed to "find key and/or cert for" to make the warning more clear that it's issued for either or both failure conditions.

Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2021-11-16 16:00:16 +01:00
Brennan Kinney b1a74bd47a
tests(feat): Refactor test_helper/common.bash common_container methods (#2275)
These are improvements for better supporting the requirements of other tests.

- Opted for passing an array reference instead of an ENV file. This seems to be a better approach and supports more than just ENV changes.
- Likewise, shifted to a `create` + `start` approach, instead of `docker run` for added flexibility.
- Using `TEST_TMP_CONFIG` instead of `PRIVATE_CONFIG` to make the difference in usage with config volume in tests more clear.
- Changed the config volume from read-only volume mount to be read-write instead, which seems required for other tests.
- Added notes about logged failures from a read-only config volume during container startup.
- Added `TEST_CA_CERT` as a default CA cert path for the test files volume. This can be used by default by openssl methods.
2021-11-05 09:35:01 +13:00
Brennan Kinney f4f0e4ef61
tests: Refactored bounced spam test + Introduce common container setup template (#2198)
* fix: Spam bounced test copy/paste typo
* tests(docs): Expand inline documentation

Should assist maintainers like myself that are not yet familiar with this functionality, saving some time :)

* Refactor bounced test + Introduce initial container template

DRY'd up the test and extracted a common init pattern for other tests to adopt in future.

The test does not need to run distinct containers at once, so a common name is fine, although the `init_with_defaults()` method could be given an arg to add a suffix: `init_with_defaults "_${BATS_TEST_NUMBER}"` which could be called in `setup()` for tests that can benefit from being run in parallel.

Often it seems the containers only need the bare minimum config such as accounts provided to actually make the container happy to perform a test, so sharing a `:ro` config mount is fine, or in future this could be better addressed.

---

The test would fail if the test cases requiring smtp access ran before postfix was ready (_only a few seconds after setup scripts announce being done_). Added the wait condition for smtp, took a while to track that failure down.
2021-09-20 19:35:03 +12:00
Nathan Pierce 0da66ccb34
setup.sh: docker_container first, then fall back to docker_image (#2134)
* docker_container first, then fall back to docker_image
+ test changes to support
+ test change to wait for smtp port to fix flakey tests since https://github.com/docker-mailserver/docker-mailserver/pull/2104

* quick fix

* Update setup.sh

Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2021-09-06 10:13:12 +00:00
Georg Lauterbach dba291dc43
Introducing the repository secret (#18)
* fixes #17
* corrections and adjustments
2021-01-18 20:51:56 +01:00
Georg Lauterbach 189e5376cc
Final Migration Step (#6)
* first migration steps
  * altered issue templates
  * altered README
  * removed .travis.yml
* adjusting registry & repository, Dockerfile and compose.env
* Close stale issues automatically
* Integrated CI with Github Actions (#3)
* feat: integrated ci with github actions
* fix: use secrets for docker org and update image
* docs: clarify why we use -t if no tty exists
* fix: correct remaining references to old repo
chore: prettier automatically updated markdown as well
* fix: hardcode docker org
* change testing image to just testing
* ci: add armv7 as a supported platform
* finished migration steps
* corrected linting in build-push action
* corrected linting in build-push action (2)
* minor preps for PR
* correcting push on pull request and minor details
* adjusted workflows to adhere closer to @wernerfred's diagram
* minor patches
* adjusting Dockerfile's installation of base packages
* adjusting schedule for stale issue action
* reverting license text
* improving CONTRIBUTING.md PR text
* Update CONTRIBUTING.md
* a bigger patch at the end
  * moved all scripts into one directory under target/scripts/
  * moved the quota-warning.sh script into target/scripts/ and removed empty directory /target/dovecot/scripts
  * minor fixes here and there
  * adjusted workflows for use a fully qualified name (i.e. docker.io/...)
  * improved on the Dockerfile layer count
  * corrected local tests - now they (actually) work (fine)!
  * corrected start-mailserver.sh to make use of defaults consistently
  * removed very old, deprecated variables (actually only one)
* various smaller improvements in the end
* last commit before merging #6
* rearranging variables to use alphabetic order

Co-authored-by: casperklein <casperklein@users.noreply.github.com>
Co-authored-by: Nick Pappas <radicand@users.noreply.github.com>
Co-authored-by: William Desportes <williamdes@wdes.fr>
2021-01-16 10:16:05 +01:00
Martin Schulze 6a9e5c0b62 Fix parallel development error: Use correct filename 2020-10-28 14:39:42 +01:00
Martin Schulze 7bdac2a910 Add tests for remaining test_helper functions 2020-10-20 00:50:19 +02:00
Martin Schulze 7ae4a73d3b Fix tests failing when emails are not yet delivered 2020-10-19 23:33:54 +02:00
Georg Lauterbach b948e01d39
re-trigger
wget through exception in the last test
2020-10-19 15:23:42 +02:00
Georg Lauterbach ab1da1e3f3
small changes in common.bash 2020-10-19 14:10:32 +02:00
Georg Lauterbach 3f5c518950
mostly pwd replacements and introduction of local keywords 2020-10-19 13:13:42 +02:00
Georg Lauterbach a3acedaf19
fixing NAME 2020-10-19 12:12:13 +02:00
Georg Lauterbach 0da7ee0c1d
{} and uppercase 2020-10-18 15:44:01 +02:00
Martin Schulze 5030e8278c Add tests for test/common.bash 2020-10-18 03:11:10 +02:00
Martin Schulze e2bca5f850 Always use ${} 2020-10-18 02:08:11 +02:00
Martin Schulze 13963747b8 Fix missing newline at the end of file 2020-10-17 23:24:23 +02:00
Martin Schulze 96b3fd726a Fix wait_for_service not working/hanging 2020-10-17 02:04:33 +02:00
Martin Schulze c6c6fb708c Avoid running tests while the services restart due to changes 2020-10-17 02:04:33 +02:00
Martin Schulze 05deaa3196 Move common functions into common.bash 2020-10-17 02:03:11 +02:00
Martin Schulze 53b930448d Fix shellcheck issues 2020-10-17 02:03:11 +02:00
Martin Schulze ca3ebc6c40 Improve error messages on problems 2020-10-17 02:03:11 +02:00
Martin Schulze 40dd3ae985 Implement and use run_until_success_or_timeout 2020-10-17 02:03:11 +02:00
Martin Schulze a477040abb Allow for seamless switching between old and new bats versions 2020-10-17 02:03:11 +02:00
Martin Schulze 0eb5bd0db9 Add --fatal-tests for early out in container waits 2020-10-17 02:02:46 +02:00
Martin Schulze 97806859b6 Fix private_config_path creating the wrong folder 2020-10-17 02:02:46 +02:00
Martin Schulze 5fef8a5b78 Give each container their own private config folder 2020-10-17 02:02:46 +02:00
Martin Schulze 06b137e5a8 Extract container mail_smtponly_without_config into test 2020-10-17 02:02:46 +02:00
Martin Schulze d38ff4c122 Extrract disabled clamav tests into own file 2020-10-17 02:02:46 +02:00
Martin Schulze 604a65024e Tests: extract mail_privacy into own .bats file 2019-10-08 23:11:27 +02:00
Martin Schulze d9a48878c9 Print last 10 log lines on container setup timeout 2019-09-12 00:43:44 +02:00
Martin Schulze 3f9dd4180a Fix setup_file marker logic 2019-09-12 00:42:04 +02:00
Erik Wramner ca821fa51a
Merge pull request #1259 from martin-schulze-vireso/bugfix/no_error_printed_on_timeout
Fix missing error output from timeout
2019-09-11 14:04:04 +02:00
Martin Schulze 3513944d0a Skip all tests of file after failed setup_file 2019-09-11 01:01:51 +02:00
Martin Schulze 8e6152ff2f Fix missing error output from timeout
The exits would prevent the printing code from running
2019-09-11 00:02:16 +02:00
Erik Wramner 8f23cb7b35 Adjust wait for tests as they often fail 2019-09-05 21:35:23 +02:00
Martin Schulze 9d90df7064 Fix integer check 2019-08-17 20:58:33 +02:00
Martin Schulze 85fa5d242f Fix setup/teardown_file to work in full suite run
The previous mechanism would only run them once for the whole suite
2019-08-17 20:58:33 +02:00
Martin Schulze db160bd977 Extract default relay host test
centralize common test variables and functions
2019-08-17 20:58:33 +02:00
Thomas VIAL 69d3bac45e Add bats asserts (#463)
* Added `bats-assert` for more detailed error, very useful locally and on Travis
* Moved `bats` to git submodule

Example:

```
not ok 52 checking opendkim: generator creates keys, tables and TrustedHosts
# (from function `assert_output' in file test/test_helper/bats-assert/src/assert.bash, line 239,
#  in test file test/tests.bats, line 368)
#   `assert_output 2' failed
# 
# -- output differs --
# expected : 2
# actual   : 4
# --
# 
```
2017-01-10 14:15:41 +01:00