Commit graph

380 commits

Author SHA1 Message Date
Casper 4ae04f0d06
Add tests for sedfile wrapper (#2363) 2022-01-10 01:12:07 +01:00
Casper 0c31f71358
Introduce DOVECOT_INET_PROTOCOLS env (#2358) 2022-01-06 00:53:18 +01:00
Casper 9d5a9a16a0
Introduce ENABLE_DNSBL env (#2342)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2022-01-03 22:03:46 +01:00
Casper 9bcc3df74a
Make TLS tests more reliable (#2354)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2022-01-01 16:20:17 +01:00
Brennan Kinney dfd649262a
tests(fix): Align with upstream testssl field name change (#2353)
The field name to check in JSON output was recently changed for the cipherlist results, causing tests to fail.
2022-01-02 02:12:24 +13:00
Casper d46e094280
Remove quotes to meet style guidelines (#2330)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2021-12-21 17:01:40 +01:00
Georg Lauterbach 99cc9fec2a
Updated ShellCheck to 0.8.0 and Hadolint to 2.8.0 (#2329)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2021-12-19 11:56:22 +01:00
Brennan Kinney 6d06149581
fix: Restore detection of letsencrypt certificate file changes (#2326)
The `DYNAMIC_FILES` var was quote wrapped, treating all filepaths to create checksums for as a single string that would be ignored instead of processed individually.

Removed the quotes, and changed the for loop to an array which accomplishes the same goal.


* fix: Prevent unnecessary change detection event

`acme.json` change would extract new cert files, which would then be hashed after restarting services and considered a change event, running through the logic again and restarting services once more when that was not required.

The checksum entries for those cert files are now replaced with new entries containing updated checksum hashes, after `acme.json` extraction.
2021-12-19 11:25:15 +13:00
Brennan Kinney 5254f7c658
fix: check-for-changes.sh should not fall out of sync with shared logic (#2260)
Removes duplicate logic from `check-for-changes.sh` that is used/maintained elsewhere to avoid risk of problems, as this code is already starting to diverge / rot.

---

Previously the change detection support has had code added for rebuilding config upon change detection which is the same as code run during startup scripts. Unfortunately over time this has fallen out of sync. Mostly the startup scripts would get maintenance and the contributor and reviewers may not have been aware of the duplicate code handled by `check-for-changes.sh`.

That code was starting to diverge in addition to some changes in structure (_eg: relay host logic seems interleaved here vs separated out in startup scripts_). I wanted to address this before it risks becoming a much bigger headache.

Rather than bloat `helper-functions.sh` further, I've added a `helpers/` folder extracting relevant common logic between startup scripts and `changedetector`. If you want to follow that process I've kept scoped commits to make those diffs easier. Some minor changes/improvements were added but nothing significant.

---

- chore: Extract relay host logic to new `relay.sh` helper
- chore: Extract `/etc/postfix/sasl_passwd` logic to new `sasl.sh` helper
- chore: Extract `postfix-accounts.cf` logic to new `accounts.sh` helper
- chore: Extract `/etc/aliases` logic to new `aliases.sh` helper
- chore: Extract `/etc/postfix/vhost` logic to new `postfix.sh` helper

- chore: Add inline docs for Postfix configs
> These are possibly more verbose than needed and can be reduced at a later stage.
> They are helpful during this refactor process while investigating that everything is handled correctly.

`accounts.sh`: 
- Add note regarding potential bug for bare domain setups with `/etc/postfix/vhost` and `mydestination` sharing same domain value.

`relay.sh`: 
- Remove the tabs for a single space delimiter, revised associated comment.
- Add PR reference for original `_populate_relayhost_map` implementation which has some useful details.


Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2021-11-21 09:33:49 +13:00
Brennan Kinney 7ca056852f
tests(feat): Complete rewrite of letsencrypt tests (#2286)
* chore: Normalize container setup

Easier to grok what is different between configurations.

- Container name usage replaced with variable
- Volumes defined earlier and redeclared when relevant (only real difference is `VOLUME_LETSENCRYPT`)
- Contextual comment about the `acme.json` copy.
- Quoting `SSL_TYPE`, `SSL_DOMAIN` and `-h` values for syntax highlighting.
- Moved `-t` and `${NAME}` to separate line.
- Consistent indentation.

* chore: DRY test logic

Extracts out repeated test logic into methods

* chore: Scope configs to individual test cases (1/3)

- Preparation step for shifting out the container configs to their own scoped test cases. Split into multiple commits to ease reviewing by diffs for this change.
- Re-arrange the hostname and domain configs to match the expected order of the new test cases.
- Shuffle the hostname and domainname grouped tests into tests per container config scope.
- Collapse the `acme.json` test cases into single test case.

* chore: Scope configs to individual test cases (2/3)

- Shifts the hostname and domainname container configs into their respective scoped test cases.
- Moving the `acme.json` container config produces a less favorable diff, so is deferred to a follow-up commit.
- Test cases updated to refer to their `${CONTAINER_NAME}` var instead of the hard-coded string name.

* chore: Scope configs to individual test cases (3/3)

Final commit to shift out the container configs.

- Common vars are exported in `setup_file()` for the test cases to use without needing to repeat the declaration in each test case.
- `teardown_file()` shifts container removal at end of scoped test case.

* chore: Adapt to `common_container_setup` template

- `CONTAINER_NAME` becomes `TEST_NAME` (`common.bash` helper via `init_with_defaults`).
- `docker run ...` and related configuration is now outsourced to the `common.bash` helper, only extra args that the default template does not cover are defined in the test case.
- `TARGET_DOMAIN`establishes the domain folder name for `/etc/letsencrypt/live`.
- `_should*` methods no longer manage a `CONTAINER_NAME` arg, instead using the `TEST_NAME` global that should be valid as test is run as a sequence of test cases.
- `PRIVATE_CONFIG` and the `private_config_path ...` are now using the global `TEST_TMP_CONFIG` initialized at the start of each test case, slightly different as not locally defined/scoped like `PRIVATE_CONFIG` would be within the test case, hence the explicit choice of a different name for context.

* chore: Minor tweaks

- Test case comment descriptions.
- DRY: `docker rm -f` lines moved to `teardown()`
- Use `wait_for_service` helper instead of checking the `changedetector` script itself is running.
- There is a startup delay before the `changedetector` begins monitoring, wait until it ready event is logged.
- Added a helper to query logs for a service (useful later).
- `/bin/sh` commands reduced to `sh`.
- Change the config check to match and compare output, not number of lines returned. Provides better failure output by bats to debug against.

* chore: Add more test functions for `acme.json`

This just extracts out existing logic from the test case to functions to make the test case itself more readable/terse.

* chore: Housekeeping

No changes, just moving logic around and grouping into inline functions, with some added comments.

* chore: Switch to `example.test` certs

This also required copying the source files to match the expected letsencrypt file structure expected in the test/container usage.

* chore: Delete `test/config/letsencrypt/`

No longer necessary, using the `example.test/` certs instead.

These letsencrypt certs weren't for the domains they were used for, and of course long expired.

* chore: Housekeeping

Add more maintainer comments, rename some functions.

* tests: Expand `acme.json` extraction coverage

Finally able to add more test coverage! :)

- Two new methods to validate expected success/failure of extraction for a given FQDN.
- Added an RSA test prior to the wildcard to test a renewal simulation (just with different cert type).
- Added extra method to make sure we're detecting multiple successful change events, not just a previous logged success (false positive).

* tests: Refactor the negotiate_tls functionality

Covers all ports (except POP) and correctly tests against expected verification status with new `example.test` certs.

The `FQDN` var will be put to use in a follow-up commit.

* tests: Verify the certs contain the expected FQDNs

* chore: Extract TLS test methods into a separate helper script

Can be useful for other TLS tests to utilize.

* chore: Housekeeping

* chore: Fix test typo

There was a mismatch between the output and expected output between these two files "find key for" and "find key & cert for". Changed to "find key and/or cert for" to make the warning more clear that it's issued for either or both failure conditions.

Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2021-11-16 16:00:16 +01:00
Brennan Kinney 584577787a
refactor: Internal HOSTNAME and DOMAINNAME configuration (#2280)
Better logical flow, handling and inline documentation.

Despite the verbosity, it's better to make this visible here for maintenance and debugging purposes than trying to dig through issue/PR or commit history for it.

* fix: Panic when HOSTNAME is misconfigured
* chore: Add more comment docs for maintainers
* tests(fix): Use `--domainname` not ENV `DOMAINNAME`

Co-authored-by: Casper <casperklein@users.noreply.github.com>
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2021-11-16 10:50:45 +13:00
Brennan Kinney b1a74bd47a
tests(feat): Refactor test_helper/common.bash common_container methods (#2275)
These are improvements for better supporting the requirements of other tests.

- Opted for passing an array reference instead of an ENV file. This seems to be a better approach and supports more than just ENV changes.
- Likewise, shifted to a `create` + `start` approach, instead of `docker run` for added flexibility.
- Using `TEST_TMP_CONFIG` instead of `PRIVATE_CONFIG` to make the difference in usage with config volume in tests more clear.
- Changed the config volume from read-only volume mount to be read-write instead, which seems required for other tests.
- Added notes about logged failures from a read-only config volume during container startup.
- Added `TEST_CA_CERT` as a default CA cert path for the test files volume. This can be used by default by openssl methods.
2021-11-05 09:35:01 +13:00
Brennan Kinney e807631a76
refactor: acme.json extraction (#2274)
Split into scoped commits with messages if further details are needed, view those via the associated PR :)

**Commit Summary:**

**`check-for-changes.sh`**

- Prevent `SSL_DOMAIN` silently skipping when value has wildcard prefix `*.` (_at least this was known as a bugfix when originally committed in linked PR_).
- Improved inlined docs for maintainers.
- Additional logging for debugging.

**`helper-functions.sh:_extract_certs_from_acme`**:

- Fail if the input arg (_`$CERT_DOMAIN`, aka the FQDN_) provided for extraction is empty.
- Use `$CERT_DOMAIN` in place of `$HOSTNAME` and `$1` for a consistent value (_previously could mismatch, eg with `SSL_DOMAIN` defined_).
- The conditional is now only for handling extraction failure (_key or cert value is missing from extraction_).
- Log an actual warning or success (debug) based on outcome.
- Don't use `SSL_DOMAIN` with wildcard value for the `mkdir` letsencrypt directory name (_wildcard prefix `*.` is first stripped instead_).

**`acme_extract`** (_new python utility for `acme.json` handling_):

- Extracted out into a python script that can be treated as a utility in the `$PATH` like other helper scripts. It can now be used and optionally tested directly instead of via `helper-functions.sh`.
-Made compatible with Python 3, as Python 2 is EOL and no longer in newer versions of Debian.
2021-11-04 09:28:40 +13:00
Brennan Kinney 936e5d2416
tests(chore): Adjust supported FQDNs in test certs + add wildcard and acme.json configs (#2284)
These files will replace the existing `test/config/letsencrypt` content which has some random provisioned FQDN for letsencrypt that doesn't match the FQDN tested, `acme.json` files with FQDNs that don't match those certs FQDNs and changes to certs that won't expire until 2031. `test/config/letsencrypt` will be removed with the associated test update PR.

The changes amount to:

- Re-configuring the FQDN values that some certs were created for (_needed for flexibility in testing_).
- Adding an `*.example.test` wildcard (_both RSA and ECDSA_).
- Adding `acme.json` encoded versions (_traefik extraction support will use these instead_).
- Updated / new internal docs for maintainers of this content.

For more detailed information on those changes, please see the associated commit messages via the PR.
2021-11-03 20:25:25 +00:00
Casper c7dec1e330
fix(listmailuser): Don't query quota, if ENABLE_QUOTAS is not 1
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2021-11-01 12:09:47 +01:00
Georg Lauterbach 537247031f
fix: Make Dovecot aware of basic aliases in userdb for quota support + Use correct hash scheme in passdb configuration (#2248)
Dovecot quota support would log auth failures when Postfix validated incoming mail to accept/reject and the `check_policy_service` for `quota-status` was queried with a recipient that was an account alias.

When Dovecot is not aware of the user account, it will not be able to check a quota and inform Postfix that everything is fine, Postfix will accept the mail and send it to Dovecot, where if the quota is exceeded will result in a bounce back to the sender. This is considered "backscatter" and can be abused by spammers forging the sender address which can get your server blacklisted.

The solution is to either disable quota support `ENABLE_QUOTAS=0`, or as a workaround, add dummy accounts to Dovecot userdb for aliases in `postfix-virtual.cf` (not `postfix-aliases.cf`), these dummy accounts will map to the real user account mailbox (real users are defined in `postfix-accounts.cf`).

The workaround is naive, in that we only check for basic 1-to-1 alias mapping to real accounts. This will still be an issue for aliases that map to another alias or multiple addresses (real or alias). Unfortunately Postfix will not expand aliases until accepting mail where this would be too late.

A better solution is to proxy the `check_policy_service` from Dovecot `quota-status` that Postfix queries in `main.cf:smtpd_recipient_restrictions`, however this requires a fair amount more of additional work and still requires an implementation to recursively query aliases for nested or multiple address mappings, which can then be forwarded to the `quota-status` service configured by Dovecot in `/etc/dovecot/conf.d/90-quota.conf`.

LDAP users are unaffected as quota support is not supported/implemented with `docker-mailserver` at this time, it is always considered disabled when using LDAP.

---

Additionally Dovecot configuration for `passdb` has been fixed to use the correct password hash scheme of `SHA512-CRYPT`. 

Co-authored-by: Casper <casperklein@users.noreply.github.com>
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2021-11-01 14:20:22 +13:00
Brennan Kinney d9e8accf5f
chore: Housekeeping on the 'open-dkim' script (#2267) 2021-10-30 10:10:32 +00:00
Georg Lauterbach 3b8059f2da
make setup.sh completely non-interactive (#2201)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2021-09-21 08:51:59 +02:00
Brennan Kinney f4f0e4ef61
tests: Refactored bounced spam test + Introduce common container setup template (#2198)
* fix: Spam bounced test copy/paste typo
* tests(docs): Expand inline documentation

Should assist maintainers like myself that are not yet familiar with this functionality, saving some time :)

* Refactor bounced test + Introduce initial container template

DRY'd up the test and extracted a common init pattern for other tests to adopt in future.

The test does not need to run distinct containers at once, so a common name is fine, although the `init_with_defaults()` method could be given an arg to add a suffix: `init_with_defaults "_${BATS_TEST_NUMBER}"` which could be called in `setup()` for tests that can benefit from being run in parallel.

Often it seems the containers only need the bare minimum config such as accounts provided to actually make the container happy to perform a test, so sharing a `:ro` config mount is fine, or in future this could be better addressed.

---

The test would fail if the test cases requiring smtp access ran before postfix was ready (_only a few seconds after setup scripts announce being done_). Added the wait condition for smtp, took a while to track that failure down.
2021-09-20 19:35:03 +12:00
Brennan Kinney c851f5b6aa
fix: Remove mkcert.sh usage + _setup_ssl refactor. (#2196)
* chore(refactor): DRY up the `_setup_ssl` method

- `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`.
- The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead.
- This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result.

- Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths.

- Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor.

- `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key.
- That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files.

- I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion).

---

* chore: Remove mkcert logic and dovecot cert

- No longer serving a purpose.
- Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`.
- To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image.
- Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition.

---

* Error handling for SSL_TYPE

- Added a panic utility to exit early when SSL_TYPE conditions are misconfigured.
- Some info text had order of key/cert occurrence swapped to be consistent with key then cert.
- Some existing comments moved and rephrased.
- Additional comments added.
- `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks).
- _notify messages lifted out of conditionals so that they always output when the case is hit.
- ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~

---

* Use sedfile + improve sed expressions + update case style

- Uses sedfile when appropriate (file change intentional, not optional match/check).
- sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)).
- sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement.

- case statements adopt the `(` prefix, adopting recent shell style for consistency.

---

* Refactor SSL_TYPE=disabled

- Postfix is also disabled now.
- Included heavy inline documentation reference for maintainers.
- Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'.

---

* I'm a friggin' sed wizard now

- The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`.

- I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines).

- Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better.

---

* tests(fix): Tests that require SSL/TLS to pass

- SSL_TYPE=snakeoil added as temporary workaround.

- nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests.
- Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past.
- The removed SSL test is already handled in mail_ssl_manual.bats

ldap test:
- Replace `--link` alias option with `--network` and alias assignment.
- Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure.

privacy test:
- Also needed `SSL_TYPE` to pass the starttls test.

`tests.bats` had another starttls test for imap:
- Workaround for now is to give the main test container `SSL_TYPE=snakeoil`.

---

* Remove the expired lets-encrypt cert

This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores.

No longer the case today.

---

* chore: Housekeeping

Not required for this PR branch, little bit of tidying up while working on these two test files.

- privacy test copied over content when extracted from `tests.bats` that isn't relevant.
- ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find.
- Additionally converted the two localhost FQDN to vars.

---

* Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case)

- This is to prevent other tests from failing by hitting the panic catchall case.
- More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value.

---

* Add inline documentation for `dms_panic`

- This could later be better formatted and placed into contributor docs.

Panic with kill (shutdown) not exit (errex):
- `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process).
- `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't.

The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-20 00:31:11 +12:00
Brennan Kinney 08cd4d3371
fix: Enable DH parameters (ffdhe4096) by default (#2192)
This feature was originally introduced by the PR: https://github.com/docker-mailserver/docker-mailserver/pull/1463

- Assign default DH params to use via Dockerfile build instead of copy and update at runtime.
- Parameterized service names and paths.
- Refactor postfix and dovecot dh methods to wrap shared dh logic
- I don't see any value in checking the alternative service for dh params file to copy over, so that's now dropped too.
- Another conditional check is dropped and the default fallback message for existing DH params file is no longer relevant.
- Improved the remaining `_notify` messages. Collapsing the warning into a single logged message also seemed relevant.
- There is no apparent need for special handling with `ONE_DIR=1`. Dropped it.

- Refactor DH params  tests
- Combine custom and default DH param tests into single test file
- docs: Add instructions to use custom DH params

There is no official documented support for custom DH parameters. As no guarantee is provided, this is considered an internal change, not a breaking one.
2021-09-15 20:28:04 +12:00
Nathan Pierce be35d9bef1
Lock file create and remove improvements (#2183)
* changed the locking function to better support multiple servers running at once and sharing the same config

* helper function testing now runs inside of container

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2021-09-13 20:09:01 +12:00
Georg Lauterbach c7e4981945
Move setup process via script into container (#2174)
Decoupling setup process from `setup.sh` script by introducing a setup script _inside_ the container that coordinates the setup process.

**This is not a breaking change**. This way, we do not have to keep track of versions of `setup.sh`.

This change brings the additional benefit for Kubernetes users to be able to make use of `setup` now, without the need for `setup.sh`.

---

* move setup process into container; setup.sh versioning not needed anymore

* add tilde functionality to docs

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2021-09-12 11:29:02 +12:00
Nathan Pierce c267d8a990
HOSTNAME and DOMAINNAME setting improvements (#2175)
Centralize the collection of the HOSTNAME and DOMAINAME so that it's predictable and uniform across the various scripts (using the helper). Ensure it supports the various configurations users can have (both subdomain and without subdomain, override and no override).

---

* using _obtain_hostname_and_domainname helper + covers when not a subdomain
doc: OVERRIDE_HOSTNAME takes priority

* added tests for non-subdomain hostname + further improvements

* moved SRS DOMAINANME tests into hostname test file + Allowing DOMAINNAME ENV to override what would be automatically set

---

Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2021-09-12 02:20:16 +12:00
Georg Lauterbach 8e91251d8c
follow up on #2177 with @polarathene's suggestions (#2180) 2021-09-12 00:43:42 +12:00
Georg Lauterbach 88773262d8
Skip non-deterministic tests until they've been debugged (#2177) 2021-09-11 11:30:01 +02:00
Nathan Pierce 0da66ccb34
setup.sh: docker_container first, then fall back to docker_image (#2134)
* docker_container first, then fall back to docker_image
+ test changes to support
+ test change to wait for smtp port to fix flakey tests since https://github.com/docker-mailserver/docker-mailserver/pull/2104

* quick fix

* Update setup.sh

Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2021-09-06 10:13:12 +00:00
Nathan Pierce dff7e428c0 Revert "check-for-changes: performance improvements + wait for settle (#2104)"
This reverts commit 232d463b81.
2021-08-28 19:16:34 -04:00
Nathan Pierce 232d463b81
check-for-changes: performance improvements + wait for settle (#2104) 2021-08-16 09:21:29 +02:00
Nathan Pierce 0e40f733f3
upgrading bats to 1.4.1 (#2095) 2021-07-25 18:58:33 +02:00
Nathan Pierce 5becce8064
chore(scripts): Removing flock so NFS works (#1980)
Co-authored-by: Casper <casperklein@users.noreply.github.com>
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2021-06-15 14:03:41 +02:00
Georg Lauterbach e7b88d865b
cleaned up >/dev/nulls in Dockerfile and replaced em dashes with normal dashes (#2024) 2021-06-08 13:20:20 +12:00
Nathan Pierce 543bd8b16b
MacOS linting & testing support + docs (#2001) 2021-06-07 14:58:34 +02:00
Georg Lauterbach abdf681d02
chore(ci): Linting Improved (#2000) 2021-06-01 18:12:17 +02:00
Moritz Marquardt 66bc157c1d
fix!: Allow protocol in SASLAUTHD_LDAP_SERVER & adjust SASLAUTHD_LDAP_ default values (#1989)
* fix: make sure the SASLAUTHD_LDAP_HOST/PROTO logic makes sense and use LDAP_SERVER_HOST as a fallback (#1983)

* chore(docs): document changes to LDAP/SASLAUTHD as of #1983

* fix!: apply default value modifications suggested in #1983

https://github.com/docker-mailserver/docker-mailserver/issues/1983#issuecomment-844848224

* chore(test): Test SASLAUTHD_LDAP_SERVER with protocol and ..._SSL=0, as well as with default bind credentials

Note that there are currently no regression tests for this as there's only one setup_file, so that would require big changes to the testing methodology.

* refactor!: completely remove SASLAUTHD_LDAP_SSL and SASLAUTHD_LDAP_PROTO

Co-authored-by: Georg Lauterbach <44545919+aendeavor@users.noreply.github.com>
Co-authored-by: Frederic Werner <20406381+wernerfred@users.noreply.github.com>
2021-05-22 22:52:56 +02:00
Georg Lauterbach 75e74e4a19
introduce F2B v0.11 (#1965) 2021-05-15 11:11:10 +02:00
Georg Lauterbach 04e98dc49f
introduce variable to control Amavis' loglevel (#1947) 2021-05-06 23:51:45 +02:00
Moritz Marquardt 94b5ac49c1
fix!: use dovecot's LDAP uris option instead of hosts (#1901)
* Use dovecot's LDAP uris option instead of hosts (fixes #1510)

* Clean up variables & environment documentation for #1901

Co-authored-by: Frederic Werner <20406381+wernerfred@users.noreply.github.com>
Co-authored-by: Georg Lauterbach <44545919+aendeavor@users.noreply.github.com>
2021-04-19 09:02:03 +02:00
Casper f7836c8b1a
Fail2Ban block behaviour (#1914)
* new default: block IP on all ports

* introduce FAIL2BAN_BLOCKTYPE

* fix test

* tests added

* test added

* test blocktype drop

* merged two tests
2021-04-18 12:55:43 +02:00
Moritz Marquardt 271d94a37e
Add LDAP_QUERY_FILTER_SENDERS setting for spoof protection with LDAP (#1902) 2021-04-17 22:40:19 +02:00
Casper dea9bca900
Enhance setup.sh email list (#1898)
* add quota and aliases to output

* shellcheck fixes

* fix test

Co-authored-by: Georg Lauterbach <44545919+aendeavor@users.noreply.github.com>
2021-04-12 15:18:15 +02:00
Casper ba0f9199b7
fail2ban cleanup (#1895) 2021-04-11 15:33:39 +00:00
Georg Lauterbach 0fa5c1ef9d
revamping the notify function (#1836) 2021-02-24 17:28:59 +01:00
Georg Lauterbach c881facbd2
start-mailserver.sh split (#1820)
* splitting start-mailserver.sh

* refactoring part 2

* refactored setup-stack.sh
* stzarted adjusting target/bin/*.sh to use new usage format

* corrected lowercase-uppercase test error

* better handling of .bashrc variable export

* linting tests and fix for default assignements

* last stylistic changes and rebase
2021-02-23 20:03:01 +01:00
Andreas Perhab 9df60f62ce linting: use local scripts to remove sudo need
install linting tools locally to remove the need for
sudo curling scripts and installing them for all users
2021-02-23 14:45:06 +01:00
Brennan Kinney d02ebc922c
Dual certificate support (eg ECDSA with RSA fallback) (#1801)
* feat: Change Postfix smtpd_tls key and cert files to chain_files

Since Postfix 3.4, `smtpd_tls_cert_file` and `smtpd_tls_key_file` have been deprecated in favor of `smtpd_tls_chain_files` which supports a list of values where a single or sequence of file paths provide a private key followed by it's certificate chain.

* feat: Dual certificate support

`smtpd_tls_chain_files` allows for multiple key+cert bundles so that you can provide different key types, such as ECDSA and RSA.

To maintain compatibility with the current CERT/KEY ENV vars only a 2nd certificate is supported.

Since Dovecot 2.2.31 a related feature is also available, but it is limited to only providing one alternative certificate via separate cert and key settings.

---

This feature enables support for multiple certificates, eg for serving modern ECDSA certs with RSA as fallback.

* chore: Refactor variable names to meet style guide

Improved some comments too.

* chore: Have function definitions respect style guide

* chore: Minor edits to comments

* chore: Expand on comments for maintenance, alert of insecure config

When `SSL_TYPE` isn't properly setup, we're still offering SSL connections but not warning in logs about the insecurity of such, or why a misconfiguration may have occurred.

This commit more clearly communicates to the user that they should look into the issue before considering deploying to production.

The `TODO` comments communicate to any future maintainer to consider treating these improper configs as disabling TLS instead.

* fix: Use `snakeoil` cert

I mistakenly thought this was placeholder text, which broke some tests. This adds the two files in the correct order (private key followed by cert/chain), to fix that issue.

* fix: Disable alt cert for Dovecot if necessary

Certain scenarios may persist state of previously configured alt cert via ENV vars that are removed from a future run. If the config is not reset to original immutable state, this will correctly disable the config from using alt cert unintentionally.

* fix: Satisfy ShellCheck lint

By switching from string var to array / list expansion, this better stores the extracted result and applies it in a manner that ShellCheck linting approves, removing the need to disable the rule.

* feat: Support dual cert test

Few tweaks to the test script allows re-purposing it for covering dual cert support as well.

* chore: Rearranged cert and key lines

A little reorganization, mostly placing private key ahead of related cert lines.

* chore: Refactor `_set_certificate`

This should make the parameters a little less confusing.

Previously was 3 parameters, but the Postfix parameter (1st) may look like two variables if you don't pay attention to the surrounding quotes; while the Dovecot parameters (2nd + 3rd) would have an opposing order. There was also a variant where the `FULLKEYCHAIN` var was passed in three times.

Now it's two params, with the 2nd param as an optional one. If the 2nd param is provided, then the two params are in the order of private key then certificate, otherwise if only a single parameter it's a single PEM file with the full cert chain and private key bundled.

This avoids implying that Postfix and Dovecot might use different files.

* chore: Document current state of `SSL_TYPE` logic better

Inlined for the benefit of anyone else maintaining this section if I'm unable to address the concerns within my own time.

* docs: ENV vars

`TLS_LEVEL=old` isn't in the codebase anymore, not likely to be relevant to retain.

No point in documenting what is considered invalid / unsupported config value in the first place for `SSL_TYPE`.

`SSL_TYPE=manual` was missing documentation for both related file path ENV vars, they've been added along with their alt fallback variants.

* chore: Update Dovecot LMTP SSL test config

Not sure how relevant this is, the file isn't complete sync with the main dovecot `10-ssl.conf` config, adding the support just in case.

* chore: Rename `FULLKEYCHAIN` to avoid confusion

There doesn't appear to be a standardized name for this type of file bundle, and `keychain` may be misleading (fullkeychain often provides macOS keychain  results on search engines).

Opting for a more explicit `KEY_WITH_FULLCHAIN` name instead.

* fix: Invalid var name

`_set_certificate` refactor commit accidentally changed a var name and committed that breaking the dual cert support (thanks tests!).

* test: Refactor `mail_ssl_manual.bats`

Proper test return values instead of `wc -l` based checking.

Tests with dual cert support active, tests that feature (to better detect failure case.

Third test case was unable to verify new self-signed certificate, added new certs signed with self-signed root CA.

Adjusted openssl `CApath` parameter to use `CAfile` instead as `letsencrypt` cert was replaced thus CA cert is missing from the system trust store.

* test: Properly check for files in `mail_ssl_manual.bats`

Fixes lint error.

Also realized I was accidentally asserting a file exists in the test environment, not within the container.

Resolved that and also added an additional test case to ensure the ENV var files are valid when passed in, in the event a change misconfigures them and that the issue is identified earlier.

* chore: Apply PR review feedback

Better format some strings that had mixed quotes when they weren't necessary.

Additionally DRYed up the config path for Postfix and Dovecot within the `_setup_ssl` method.

Co-authored-by: Georg Lauterbach <infrastructure@itbsd.com>
2021-02-21 23:43:41 +01:00
Astro a7ecb0ea8b
feat/enable custom dkim selector (#1811)
* let dkim generator accept selector as parameter

* test dkim-generator with selector parameter

* fix: correct name of domain argument in usage

* fix: adapt command to new syntax

* tests: use different quotes

* tests: use different quotes

* tests: remove domains that were never added

* style: change test name

* refactor: dkim setup

* style: remove trailing whitespace

* tests: remove test of removed dummy file

Co-authored-by: Frederic Werner <20406381+wernerfred@users.noreply.github.com>
2021-02-21 22:05:35 +01:00
Frederic Werner d182d65377
fix: dkim help message test (#1817) 2021-02-18 22:55:17 +01:00
Georg Lauterbach d221c585c2
inform about proper setup.sh usage between versions 2021-02-18 14:01:01 +01:00
Georg Lauterbach f3f38db0f9
adjust test to use new script output from openDKIM 2021-02-18 13:11:45 +01:00