Commit graph

178 commits

Author SHA1 Message Date
Brennan Kinney 40e2d88482
chore: Merge helpers/sasl.sh into helpers/relay.sh (#2605)
This helper was to support an earlier ENV for SASL auth support. When extracting logic into individual helpers, it was assumed this was separate from relay support, which it appears was not the case.

---

The `SASL_PASSWD` ENV is specified in tests but no longer used. There is no `external-domain.com` relay configured or tested against anywhere in the project.

The ENV was likely used in tests prior to improved relay support that allowed for adding more than a single set of relay credentials.

---

It likewise has no real relevance anywhere else outside of `relay.sh` as it's the only portion of code to operate with it.

It's only relevant for SASL auth as an SMTP client, not the SMTP server (`smtpd`) SASL support that is delegated to Dovecot. Functionality has been completely migrated into `relay.sh` as a result.

Documentation is poor for this ENV, it is unlikely in wide use? Should consider for removal.

---

The ENV has been dependent upon `RELAY_HOST` to actually enable postfix to use `/etc/postfix/sasl_passwd`, thus not likely relevant in existing setups?

---

Migrate `/etc/postfix/sasl_passwd` check from `tests.bats` as it belongs to relay tests.
2022-06-06 10:59:42 +12:00
Brennan Kinney 3d6e7a7bb8
service(postfix): Better handling of the compatibility_level setting (#2597)
* chore: Fix typo

* chore: Apply explicit chroot default for `sender-cleanup`

The implicit default is set to `y` as a compatibility fallback, but otherwise it is [advised to set to `n` going forward](http://www.postfix.org/COMPATIBILITY_README.html#chroot).

Test was changed to catch any backwards-compatibility logs, not just those for `chroot=y`. `using` added as a prefix to avoid catching log message whenever a setting is changed that the default compatibility level is active.

* chore: Set `compatibility_level` in `main.cf`

We retain the level`2` value previously set via scripts. This avoids log noise that isn't helpful.

Applied review feedback to give maintainers some context with this setting and why we have it presently set to `2`.
2022-06-05 12:10:20 +12:00
Brennan Kinney d04aceccae
fix: Only add state of enabled services for volume mount (#2608)
* fix: Conditionally add service state

These services will no longer copy over state unless they are enabled.

The biggest offender here was ClamAV as it's database that is baked into the docker image is over 200MB and would copy over to every container instance with a volume mounted state directory.

* chore: Add Dovecot to conditional support
2022-06-05 12:02:25 +12:00
Brennan Kinney c862e1451d
chore(housekeeping): Create helpers/change-detection.sh (#2610)
* chore: Extract change-detection method to it's own helper

This doesn't really belong in `helpers/ssl.sh`. Moving to it's own helper script.

* chore: Co-locate related change-detection method from container startup

It seems relevant to migrate the related support during startup for the change detection feature into this helper.

I opted to move the call from `start-mailserver.sh` into the `_setup` call at the end for a more explicit/visible location.

* chore: Move `CHKSUM_FILE` into `helpers/change-detection.sh`

It belongs there, not in `helpers/index.sh`.

* chore: Revise inline documentation

* tests(fix): Ensure correct functionality

Presently `test/test_helper.bats` is using it's own  `CHKSUM_FILE` instead of sourcing the var for the filepath.

`test_helper/common.bash` was calling a method to check for changes, but this helper may not correctly detect letsencrypt related changes as these are not ENV rely on, but global vars handled by `helpers/dns.sh`, so that should be run first like it is for `check-for-changes.sh`.

* tests(chore): Use `CHKSUM_FILE` var from helper

* chore: `addmailuser` should use `CHKSUM_FILE` var

* chore: Update `check-for-changes.sh` log message with correct path
2022-06-05 11:59:54 +12:00
Casper 9a739113f4
chore: Migrate SASLAUTHD_* vars into start-mailserver.sh (#2562) 2022-06-04 19:28:13 +00:00
Georg Lauterbach 05e45c349a
scripts: fix error message caused by grep on first starts (#2591)
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2022-05-28 20:59:31 +02:00
Georg Lauterbach b7e5d42e09
scripts: more consistency in start-mailserver.sh (#2587)
* consistently name functions (starting with `_`) in `start-mailserver.sh`

Most of the functions that execute the different stacks during startup
were not prefixed with `_`, but all our other functions are. This has
now been fixed.

* cleanup in `start-mailserver.sh`

I adjusted the comments for all sections in the start script so they are
properly displayed again.
2022-05-15 23:37:21 +02:00
Maximilian Hippler 777d4a8cb7
scripts: loglevel warning message fix (#2588)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2022-05-13 10:50:45 +02:00
Casper 2977cb6962
Streamline 'printf' usage and consequently use a format string (#2564) 2022-05-05 17:48:21 +02:00
Casper 628e902233
Remove unnecessary quotes from command substitutions (#2561) 2022-05-05 10:28:38 +02:00
Casper 32fba30bbc
Don't issue warning when DKIM is not used (#2563) 2022-04-29 19:03:16 +02:00
Casper de61d42e68
Add ban feature to fail2ban script (#2538) 2022-04-19 08:44:51 +00:00
Brennan Kinney 9aaf15b38f
chore: (setup-stack.sh) Fix a small typo (#2552) 2022-04-19 12:53:45 +12:00
Georg Lauterbach 412f675bfe
setup: added grace period for account creation (#2531) 2022-04-18 10:48:48 +02:00
Georg Lauterbach a1726dc45a
scripts: added TZ environment variable to set timezone (#2530) 2022-04-06 16:48:41 +02:00
Georg Lauterbach c7b16a599c
log: adjust level and message(s) slightly for four messages (#2532)
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2022-04-05 16:21:07 +02:00
Georg Lauterbach a9305a073f
firewall: replace iptables with nftables (#2505)
* first adjustments to use Fail2Ban with nftables

* replace `iptables` -> `nftables` and adjust tests

nftables lists IPs a bit differently , so the order was adjusted for the
tests to be more flexible.

* line correction in mailserver.env

* change from `.conf` -> `.local` and remove redundant config

* revert HEREDOC to `echo`

Co-authored-by: Casper <casperklein@users.noreply.github.com>
2022-04-05 15:13:59 +02:00
Ikko Ashimine 05f680b472
fix: typo in setup-stack.sh (#2521) 2022-04-02 10:41:46 +02:00
Georg Lauterbach a54d774587
scripts: refactored daemon-stack.sh (#2496)
* refactored `daemon-stack.sh`

A new method was introduced to uniformaly start daemons and log output
accordingly. The methods for daemon start were renamed (plural ->
singular), therefore the adjustments in `start-mailserver.sh`.

* cleaned Fetchmail setup from `daemon-stack.sh`

Not sure why, but the Fetchmail setup was somehow happening in
`daemon-stack.sh` - this is not supposed to be the case. I relocated the
setup into `setup-stack.sh`, where it belong.

* delete old, unnecessary script in `target/bin/`

These are unused leftovers from the last commit, that relocated the
setup of Fetchmail into `setup.stack.sh`.

* corrected changedetector function name

* Apply suggestions from code review

* adjusted `debug-fetchmail` script

It is absolutely fine to source `setup-stack.sh` because sourcing the
script does not execute a single function (by desing of the script).
This way, we retain functionality.

* praise be ShellCheck

* added `log.sh` to `debug-fetchmail` as a dependency

* final cleanup

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2022-03-27 09:43:39 +02:00
Georg Lauterbach 7721a48b9b
scripts: refactoring & miscellaneous small changes (#2499)
* `update-check.sh` now uses the new log

* refactored `setup-stack.sh`

The changes are:

1. Replaced `""` wiht `''` where possible (reasoning: Bash is very
   implicit and I'd like to use `''` where possible to indicate no
   variables are expanded here)
2. `> /file` -> `>/file` according to our style guide
3. Some log adjustments for messages where I deemed it appropriate
4. Then, an error message from a Dovecot setup was also prevented (by
   adding a check whether the directory is present before a `: >...`
   command would create a file in this directory).

These are all small, miscellaneous changes that I wanted to combine into
one commit and ultimately one PR because I see no point in opening a PR
for every small change here. I hope this is fine.

* added a small `sleep` to the `_shutdown` function

This ensure the last log message is actually logged before Supervisor
logs the message that it received a SIGTERM. This makes reading the log
easier because now the causal relationship is shown (we are terminating
Supervisor, and not someone else and we're just logging it).

I forgot to replace `""` with `''` in `update-check.sh`, so I included
it here because this is the last commit before PR review.

* re-add exit on successful update (only)

* re-added date information to update-check log messages

* added `_log_with_date` function

The new function will log a message with a proper timestamp. This is all
handled in `log.sh`, we therefore not need to source other files too.

This will be used in the future by `check-for-changes.sh` as well :)

Co-authored-by: Casper <casperklein@users.noreply.github.com>
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2022-03-26 10:17:08 +01:00
Georg Lauterbach 24031ae365
scripts: new log (#2493)
* added new `_log` function

With `_log`, the `_notify` method wa rendered obsolete. `_notify` was
not completely removed due to test failures in `check-for-changes.sh`.

The new `_log` function properly uses log levels such as `trace`,
`debug`, `info`, `warn` and `error`. It provides a cleaner solution
and renders `DMS_DEBUG` obsolete too (as only `_notify` depends on it).

* converted all helper script to new `_log` function

* converted all startup stacks to new `log` function

* `start-mailserver.sh` now uses new `_log` function

* final test and misc small script adjustments

* updated documentation
2022-03-21 07:07:52 +01:00
Georg Lauterbach 321ae744fa
scripts: small refactorings (#2485)
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2022-03-17 11:24:30 +01:00
Georg Lauterbach d101d0aa0a
fix: remove SA reload for KAM (#2456)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2022-03-04 23:38:29 +01:00
Casper b9fdb1e7b5
Introduce CLAMAV_MESSAGE_SIZE_LIMIT env (#2453) 2022-03-03 16:17:01 +01:00
Georg Lauterbach e6af5a118f
Adjust envrionment variables - more sensible defaults (#2428)
The new setup will now set env variables on one place and on one place
only. The old setup used two separate places wich is not DRY and
confusing.

Some default values changed:

1. PFLOGSUMM_TRIGGER: logrotate => none
2. REPORT_SENDER: mailserver-report@HOSTNAME => mailserver-report@DOMAIN
3. REPORT_RECIPIENT: "0" => POSTMASTER_ADDRESS

One env variable was renamed: REPORT_INTERVAL => LOGROTATE_INTERVAL

I believe these defaults to be more sensible, especially the REPORT_RECIPIENT
address. The PFLOGSUMM_TRIGGER value was changed to `none` because otherwise
people would start getting daily Postfix log summary reports automatically.
Now, this is opt-in, and reports are sent only when enabled properly.

Some of the variables changed were marked as deprecated. I removed the note,
as the variables now bear some (sane) defaults again for other variables
(i.e.) REPORT_RECIPIENT is now default for other recipient addresses.

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2022-03-03 10:22:17 +13:00
Casper 57c52d7b5b
PERMIT_DOCKER=none as new default value (#2424)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2022-03-02 16:03:40 +01:00
Georg Lauterbach 399284c092
fix: do not add accounts that already exists to account files (#2419)
Co-authored-by: Casper <casperklein@users.noreply.github.com>
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2022-02-21 13:31:03 +01:00
Georg Lauterbach b61dfe1e24
refactoring: split helper functions into smaller scripts (#2420) 2022-02-21 11:56:57 +01:00
Georg Lauterbach 2927cc47c7
Add SpamAssassin KAM (#2418) 2022-02-21 10:48:28 +01:00
Georg Lauterbach ec8b99335e
Add changedetector functionality for ${SSL_TYPE} == manual (#2404)
Now, setups that use `SSL_TYPE=manual` will profit from the changedetector as well. Certificate changes are picked up and properly propagated.
2022-02-18 11:29:51 +01:00
Christian Raue 908e95fa74
consistently make 1 the default value for SPAMASSASSIN_SPAM_TO_INBOX (#2361)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2022-02-17 09:31:45 +01:00
Georg Lauterbach ce9e86c462
fix: double occurence of /etc/postfix/regexp (#2397) 2022-02-12 09:20:45 +13:00
Philipp Fruck 4c3af32692
Rootless Podman security update (#2393)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2022-02-09 10:25:09 +01:00
Christian Raue da17e8bf0b
add env var LOGWATCH_SENDER (#2362)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2022-01-10 12:29:11 +01:00
Casper 29c2d975ec
Fix harmless startup errors (#2357) 2022-01-08 00:25:09 +01:00
Casper 0c31f71358
Introduce DOVECOT_INET_PROTOCOLS env (#2358) 2022-01-06 00:53:18 +01:00
Casper 9d5a9a16a0
Introduce ENABLE_DNSBL env (#2342)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2022-01-03 22:03:46 +01:00
Casper f7465a5088
Prevent race condition on supervisorctl reload (#2343)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2021-12-29 17:36:29 +01:00
Casper d46e094280
Remove quotes to meet style guidelines (#2330)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2021-12-21 17:01:40 +01:00
Georg Lauterbach a4095a7d48
Disable service stats for Dovecot explicitly (#2292)
Co-authored-by: Casper <casperklein@users.noreply.github.com>

Co-authored-by: Casper <casperklein@users.noreply.github.com>
2021-12-12 12:08:34 +01:00
Brennan Kinney 5254f7c658
fix: check-for-changes.sh should not fall out of sync with shared logic (#2260)
Removes duplicate logic from `check-for-changes.sh` that is used/maintained elsewhere to avoid risk of problems, as this code is already starting to diverge / rot.

---

Previously the change detection support has had code added for rebuilding config upon change detection which is the same as code run during startup scripts. Unfortunately over time this has fallen out of sync. Mostly the startup scripts would get maintenance and the contributor and reviewers may not have been aware of the duplicate code handled by `check-for-changes.sh`.

That code was starting to diverge in addition to some changes in structure (_eg: relay host logic seems interleaved here vs separated out in startup scripts_). I wanted to address this before it risks becoming a much bigger headache.

Rather than bloat `helper-functions.sh` further, I've added a `helpers/` folder extracting relevant common logic between startup scripts and `changedetector`. If you want to follow that process I've kept scoped commits to make those diffs easier. Some minor changes/improvements were added but nothing significant.

---

- chore: Extract relay host logic to new `relay.sh` helper
- chore: Extract `/etc/postfix/sasl_passwd` logic to new `sasl.sh` helper
- chore: Extract `postfix-accounts.cf` logic to new `accounts.sh` helper
- chore: Extract `/etc/aliases` logic to new `aliases.sh` helper
- chore: Extract `/etc/postfix/vhost` logic to new `postfix.sh` helper

- chore: Add inline docs for Postfix configs
> These are possibly more verbose than needed and can be reduced at a later stage.
> They are helpful during this refactor process while investigating that everything is handled correctly.

`accounts.sh`: 
- Add note regarding potential bug for bare domain setups with `/etc/postfix/vhost` and `mydestination` sharing same domain value.

`relay.sh`: 
- Remove the tabs for a single space delimiter, revised associated comment.
- Add PR reference for original `_populate_relayhost_map` implementation which has some useful details.


Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2021-11-21 09:33:49 +13:00
Georg Lauterbach ae70142d8f
Housekeeping (#2302)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2021-11-20 10:38:40 +01:00
Casper cac0647250
improve: sort environment variables (#2298) 2021-11-14 09:11:44 +13:00
Georg Lauterbach 6c1c669619
make Bash start user-patches.sh (#2295) 2021-11-13 10:19:56 +01:00
Georg Lauterbach cf4682fe5e
remove unnecessary ls from debug log (#2294) 2021-11-12 14:33:04 +01:00
Andreas Egli e436968655
openDMARC: remove quotation marks from setup script (#2291) 2021-11-10 12:21:26 +01:00
Brennan Kinney bdb35dd19a
chore: Refactor setup-stack.sh case SSL_TYPE=letsencrypt (#2278)
Mostly cleans up the code and documents it better, although there are some minor fixes for handling `SSL_DOMAIN` ENV and additional logging added for spotting issues related to it in future when troubleshooting.

Commits are scoped with context messages for easing review if necessary. Overview of changes:

Traefik specific:

- Logic extracted out into it's own function.
- Conditional reworked to assist with debugging.
- `SSL_DOMAIN` must not be empty when attempting to extract.
- Added additional notes.

`SSL_TYPE=letsencrypt` case:

- Revised top note block.
- Correct handling for `SSL_DOMAIN`.
- Removed some unnecessary nesting.
- Less repetitive error message for `LETSENCRYPT_DOMAIN`.
- Added use of panics where appropriate (kept `return 1` so failures still exit functionality early).
- Improved inline docs.
2021-11-03 09:12:33 +13:00
Casper ebb081c80f
Add missing SA variables (#2283) 2021-11-02 11:51:36 +01:00
Casper b117cd1423
Improve VIRUSMAILS_DELETE_DELAY usage (#2281) 2021-11-01 22:13:14 +01:00
Casper c7dec1e330
fix(listmailuser): Don't query quota, if ENABLE_QUOTAS is not 1
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2021-11-01 12:09:47 +01:00
Georg Lauterbach 537247031f
fix: Make Dovecot aware of basic aliases in userdb for quota support + Use correct hash scheme in passdb configuration (#2248)
Dovecot quota support would log auth failures when Postfix validated incoming mail to accept/reject and the `check_policy_service` for `quota-status` was queried with a recipient that was an account alias.

When Dovecot is not aware of the user account, it will not be able to check a quota and inform Postfix that everything is fine, Postfix will accept the mail and send it to Dovecot, where if the quota is exceeded will result in a bounce back to the sender. This is considered "backscatter" and can be abused by spammers forging the sender address which can get your server blacklisted.

The solution is to either disable quota support `ENABLE_QUOTAS=0`, or as a workaround, add dummy accounts to Dovecot userdb for aliases in `postfix-virtual.cf` (not `postfix-aliases.cf`), these dummy accounts will map to the real user account mailbox (real users are defined in `postfix-accounts.cf`).

The workaround is naive, in that we only check for basic 1-to-1 alias mapping to real accounts. This will still be an issue for aliases that map to another alias or multiple addresses (real or alias). Unfortunately Postfix will not expand aliases until accepting mail where this would be too late.

A better solution is to proxy the `check_policy_service` from Dovecot `quota-status` that Postfix queries in `main.cf:smtpd_recipient_restrictions`, however this requires a fair amount more of additional work and still requires an implementation to recursively query aliases for nested or multiple address mappings, which can then be forwarded to the `quota-status` service configured by Dovecot in `/etc/dovecot/conf.d/90-quota.conf`.

LDAP users are unaffected as quota support is not supported/implemented with `docker-mailserver` at this time, it is always considered disabled when using LDAP.

---

Additionally Dovecot configuration for `passdb` has been fixed to use the correct password hash scheme of `SHA512-CRYPT`. 

Co-authored-by: Casper <casperklein@users.noreply.github.com>
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2021-11-01 14:20:22 +13:00
Brennan Kinney 79617dcc40
fix: ONE_DIR=1 should keep correct permissions for spool-postfix (#2273)
GID of postfix(102) should not be set here. only GID is root(0) and postdrop(103). Ensure root(0) UID matches source location ownership too.
2021-11-01 10:23:00 +13:00
Casper 34ba3c2a5b
Add warnings when Amavis is disabled. (#2251)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2021-10-17 16:18:56 +00:00
Casper 61973f731c
Relaxed error handling (#2233) 2021-10-03 12:48:43 +00:00
Casper c7e9dd2bad
function _defunc removed (#2199)
* function _defunc removed

* _shutdown is better than just notify in that cases

* PANIC_TYPE 'fail-init' introduced

Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2021-09-23 19:49:07 +02:00
Brennan Kinney a0ee472501
docs(chore): Normalize for consistency (#2206)
"Brief" summary/overview of changes. See the PR discussion or individual commits from the PR for more details.

---

Only applies to the `docs/content/**` content (_and `setup` command_). `target/` and `test/` can be normalized at a later date.

* Normalize to `example.com`

- Domains normalized to `example.com`: `mywebserver.com`, `myserver.tld`, `domain.com`, `domain.tld`, `mydomain.net`, `my-domain.tld`, `my-domain.com`, `example.org`, `whoami.com`.
- Alternative domains normalized to `not-example.com`: `otherdomain.com`, `otherdomain.tld`, `domain2.tld`, `mybackupmx.com`, `whoareyou.org`.
- Email addresses normalized to `admin@example.com` (in `ssl.md`): `foo@bar.com`, `yourcurrentemail@gmail.com`, `email@email.com`, `admin@domain.tld`.
- Email addresses normalized to `external-account@gmail.com`: `bill@gates321boom.com`, `external@gmail.com`, `myemail@gmail.com`, `real-email-address@external-domain.com`.
- **`faq.md`:** A FAQ entry title with `sample.domain.com` changed to `subdomain.example.com`.
- **`mail-fetchmail.md`:** Config examples with FQDNs for `imap`/`pop3` used `example.com` domain for a third-party, changed to `gmail.com` as more familiar third-party/external MTA.

* Normalize config volume path

- Normalizing local config path references to `./docker-data/dms/config/`: `./config/`, `config/`, \``config`\`, `/etc/` (_volume mount src path prefix_).
- Normalize DMS volume paths to `docker-data/dms/mail-{data,state,log}`: `./mail`, `./mail-state` `./data/mail`, `./data/state`, `./data/logs`, `./data/maildata`, `./data/mailstate`, `./data/maillogs`, (_dropped/converted data volumes: `maildata`, `mailstate`_).
- Other docker images also adopt the `docker-data/{service name}/` prefix.

* `ssl.md` - Use `dms/custom-certs` where appropriate.

* Apply normalizations to README and example `docker-compose.yml`

---

Common terms, sometimes interchangeably used or now invalid depending on context: `mail`, `mail container`, `mail server`, `mail-server`, `mailserver`,`docker-mailserver`, `Docker Mailserver`.

Rough transformations applied to most matches (_conditionally, depending on context_):

- 'Docker Mailserver' => '`docker-mailserver`'
- 'mail container' => '`docker-mailserver`' (_optionally retaining ' container'_)
- 'mail server' => 'mail-server' / '`docker-mailserver`'
- 'mail-server' => '`docker-mailserver`'
- 'mailserver' => 'mail-server' / '`docker-mailserver`'

Additionally I checked `docker run` (_plus `exec`, `logs`, etc, sub-commands_) and `docker-compose` commands. Often finding usage of `mail` instead of the expected `mailserver`

Additionally changes `mailserver` hostname in k8s to `mail` to align with other non-k8s examples.

---

* drive-by revisions

Mostly minor revisions or improvements to docs that aren't related to normalization effort.
2021-09-23 11:29:37 +12:00
Brennan Kinney f4f0e4ef61
tests: Refactored bounced spam test + Introduce common container setup template (#2198)
* fix: Spam bounced test copy/paste typo
* tests(docs): Expand inline documentation

Should assist maintainers like myself that are not yet familiar with this functionality, saving some time :)

* Refactor bounced test + Introduce initial container template

DRY'd up the test and extracted a common init pattern for other tests to adopt in future.

The test does not need to run distinct containers at once, so a common name is fine, although the `init_with_defaults()` method could be given an arg to add a suffix: `init_with_defaults "_${BATS_TEST_NUMBER}"` which could be called in `setup()` for tests that can benefit from being run in parallel.

Often it seems the containers only need the bare minimum config such as accounts provided to actually make the container happy to perform a test, so sharing a `:ro` config mount is fine, or in future this could be better addressed.

---

The test would fail if the test cases requiring smtp access ran before postfix was ready (_only a few seconds after setup scripts announce being done_). Added the wait condition for smtp, took a while to track that failure down.
2021-09-20 19:35:03 +12:00
Brennan Kinney c851f5b6aa
fix: Remove mkcert.sh usage + _setup_ssl refactor. (#2196)
* chore(refactor): DRY up the `_setup_ssl` method

- `/etc/postfix/ssl` was a bit misleading in usage here. As a maintainer (of my own contribution!) I was confused why only `/etc/postfix/ssl` was referenced and not `/etc/dovecot/ssl`.
- The postfix specific path is unnecessary, dovecot was referencing it via it's config, the same can be done from postfix to a generic DMS specific config location instead.
- This location is defined and created early as `/etc/dms/tls` (with var `DMS_TLS_PATH`). All usage of `/etc/postfix/ssl` has been replaced, making it easier to grok. Several `mkdir` commands related to this have been dropped as a result.

- Likewise, a related `TMP_DMS_TLS_PATH` var provides a reference to the config volume path `/tmp/docker-mailserver` which is used for conditions on presently hard-coded paths.

- Other values that benefit from being DRY have been lifted up into vars. Definitely easier to follow now and makes some further opportunities clearer to tackle in a future refactor.

- `chmod` has been updated where appropriate. Public key/cert is acceptable to have as readable by non-root users (644). The custom type with single fullchain file was not root accessible only, but should as it contains a private key.
- That said, the security benefit can be a bit moot due to source files that were copied remain present, the user would be responsible to ensure similar permissions on their source files.

- I've not touched LetsEncrypt section as I don't have time to investigate into that yet (not familiar with that portion).

---

* chore: Remove mkcert logic and dovecot cert

- No longer serving a purpose.
- Our own TLS startup script handles a variety of cert scenarios, while the dropped code was always generating a self-signed cert and persisting an unused cert regardless with `ONE_DIR=1`.
- To avoid similar issues that DH params had with doveadm validating filepath values in the SSL config, the default dummy values match postfix pointing to "snakeoil" cert. That serves the same purpose as mkcert was covering in the image.
- Bonus, no more hassle with differing mkcert target paths for users replacing our supplied Dovecot with the latest community edition.

---

* Error handling for SSL_TYPE

- Added a panic utility to exit early when SSL_TYPE conditions are misconfigured.
- Some info text had order of key/cert occurrence swapped to be consistent with key then cert.
- Some existing comments moved and rephrased.
- Additional comments added.
- `-f` test for cert files instead of `-e` (true also for directories/devices/symlinks).
- _notify messages lifted out of conditionals so that they always output when the case is hit.
- ~~Empty SSL_TYPE collapsed into catch all panic, while it's contents is now mapped to a new 'disabled' value.~~

---

* Use sedfile + improve sed expressions + update case style

- Uses sedfile when appropriate (file change intentional, not optional match/check).
- sed expressions modified to be DRY and reduce escaping via `-r` flag (acceptable if actual text content contains no `?`,`+`,`()` or `{}` characters, [otherwise they must be escaped](https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html)).
- sed captures anything matched between the parenthesis`()` and inserts it via `\1` as part of the replacement.

- case statements adopt the `(` prefix, adopting recent shell style for consistency.

---

* Refactor SSL_TYPE=disabled

- Postfix is also disabled now.
- Included heavy inline documentation reference for maintainers.
- Dropped an obsolete postfix config option 'use_tls' on the relayhost function, it was replaced by 'security_level'.

---

* I'm a friggin' sed wizard now

- The `modern` TLS_LEVEL is the default values for the configs they modify. As such, `sedfile` outputs an "Error" which isn't an actual concern, back to regular `sed`.

- I realized that multiple edits for the same file can all be done at once via `-e` (assuming other sed options are the same for each operation), and that `g` suffix is global scope for single line match, not whole file (default as sed iterates through individual lines).

- Some postfix replacements have `smtp` and `smtpd` lines, collapsed into a single `smtpd?` instead now that I know sed better.

---

* tests(fix): Tests that require SSL/TLS to pass

- SSL_TYPE=snakeoil added as temporary workaround.

- nmap tests are being dropped. These were added about 4-5 years ago, I have since made these redundant with the `testssl.sh` tests.
- Additionally the `--link` option is deprecated and IIRC these grades were a bit misleading when I initially used nmap in my own TLS cipher suite update PRs in the past.
- The removed SSL test is already handled in mail_ssl_manual.bats

ldap test:
- Replace `--link` alias option with `--network` and alias assignment.
- Parameterized some values and added the `SSL_TYPE` to resolve the starttls test failure.

privacy test:
- Also needed `SSL_TYPE` to pass the starttls test.

`tests.bats` had another starttls test for imap:
- Workaround for now is to give the main test container `SSL_TYPE=snakeoil`.

---

* Remove the expired lets-encrypt cert

This expired in March 2021. It was originally required when first added back in 2016 as LetsEncrypt was fairly new and not as broadly accepted into OS trust stores.

No longer the case today.

---

* chore: Housekeeping

Not required for this PR branch, little bit of tidying up while working on these two test files.

- privacy test copied over content when extracted from `tests.bats` that isn't relevant.
- ldap test was not as easy to identify the source of DOVECOT_TLS. Added comment to make the prefix connection to `configomat.sh` and `.ext` files more easier to find.
- Additionally converted the two localhost FQDN to vars.

---

* Default SSL_TYPE becomes `''` (aka equivalent to desired `disabled` case)

- This is to prevent other tests from failing by hitting the panic catchall case.
- More ideal would be adjusting tests to default to `disabled`, rather than treating `disabled` as an empty / unset SSL_TYPE value.

---

* Add inline documentation for `dms_panic`

- This could later be better formatted and placed into contributor docs.

Panic with kill (shutdown) not exit (errex):
- `kill 1` from `_shutdown` will send SIGTERM signal to PID 1 (init process).
- `exit 1` within the `start-mailserver.sh` init scripts context, will just exit the initialization script leaving the container running when it shouldn't.

The two previous `_shutdown` methods can benefit from using `dms_panic` wrapper instead to standardize on panic messages.
2021-09-20 00:31:11 +12:00
Brennan Kinney 2bf24e4c08
fix: Relax openssl security level for intermediate TLS_LEVEL (#2193)
Although these two config lines have not changed since `debian:buster-slim` image, Dovecot seems to now be affected by it which results in rejecting cipher suites below TLS v1.2.

To continue supporting the `intermediate` TLS_LEVEL, we now need to relax the global config. Dovecot could alternatively be given a modified openssl config to only affect it's interaction with openssl.

Postfix is unaffected and continues to support TLS <1.2 cipher suites when configured to.
2021-09-16 01:28:42 +12:00
Brennan Kinney 08cd4d3371
fix: Enable DH parameters (ffdhe4096) by default (#2192)
This feature was originally introduced by the PR: https://github.com/docker-mailserver/docker-mailserver/pull/1463

- Assign default DH params to use via Dockerfile build instead of copy and update at runtime.
- Parameterized service names and paths.
- Refactor postfix and dovecot dh methods to wrap shared dh logic
- I don't see any value in checking the alternative service for dh params file to copy over, so that's now dropped too.
- Another conditional check is dropped and the default fallback message for existing DH params file is no longer relevant.
- Improved the remaining `_notify` messages. Collapsing the warning into a single logged message also seemed relevant.
- There is no apparent need for special handling with `ONE_DIR=1`. Dropped it.

- Refactor DH params  tests
- Combine custom and default DH param tests into single test file
- docs: Add instructions to use custom DH params

There is no official documented support for custom DH parameters. As no guarantee is provided, this is considered an internal change, not a breaking one.
2021-09-15 20:28:04 +12:00
Nathan Pierce c267d8a990
HOSTNAME and DOMAINNAME setting improvements (#2175)
Centralize the collection of the HOSTNAME and DOMAINAME so that it's predictable and uniform across the various scripts (using the helper). Ensure it supports the various configurations users can have (both subdomain and without subdomain, override and no override).

---

* using _obtain_hostname_and_domainname helper + covers when not a subdomain
doc: OVERRIDE_HOSTNAME takes priority

* added tests for non-subdomain hostname + further improvements

* moved SRS DOMAINANME tests into hostname test file + Allowing DOMAINNAME ENV to override what would be automatically set

---

Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2021-09-12 02:20:16 +12:00
William Desportes 392ee076ec
Fix #2122 - only chmod when needed (#2127) 2021-08-13 00:09:44 +02:00
Casper 7293e3c9e8
Check if CONTAINER_IP could be determined (#2046)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2021-06-19 22:24:06 +02:00
Casper 4822709000
do not delete supervisord.pid (#2044) 2021-06-19 14:01:38 +02:00
Georg Lauterbach abdf681d02
chore(ci): Linting Improved (#2000) 2021-06-01 18:12:17 +02:00
Moritz Marquardt 66bc157c1d
fix!: Allow protocol in SASLAUTHD_LDAP_SERVER & adjust SASLAUTHD_LDAP_ default values (#1989)
* fix: make sure the SASLAUTHD_LDAP_HOST/PROTO logic makes sense and use LDAP_SERVER_HOST as a fallback (#1983)

* chore(docs): document changes to LDAP/SASLAUTHD as of #1983

* fix!: apply default value modifications suggested in #1983

https://github.com/docker-mailserver/docker-mailserver/issues/1983#issuecomment-844848224

* chore(test): Test SASLAUTHD_LDAP_SERVER with protocol and ..._SSL=0, as well as with default bind credentials

Note that there are currently no regression tests for this as there's only one setup_file, so that would require big changes to the testing methodology.

* refactor!: completely remove SASLAUTHD_LDAP_SSL and SASLAUTHD_LDAP_PROTO

Co-authored-by: Georg Lauterbach <44545919+aendeavor@users.noreply.github.com>
Co-authored-by: Frederic Werner <20406381+wernerfred@users.noreply.github.com>
2021-05-22 22:52:56 +02:00
Casper bab0277723
Update check (#1951)
* mail binary

* initial work

* make env vars available

* typo

* some fixes

* make script ugly, to satisfy linter..

* mailserver.env updated

* Version to welcome message added

* remove VERSION file references

* VERSION --> DMS_VERSION

* fetch remote version

* variable usage

* Quoting added

* edge test & docu

* dash removed

* subject changed

* re-add VERSION

* VERSION added

* new file:   VERSION

* rewrite

* unnecessary additions from fail2ban PR removed

* UPDATE_CHECK_INTERVAL added

* syntax check & _log function

* comment added

* final commit
2021-05-19 21:18:06 +02:00
Georg Lauterbach 04e98dc49f
introduce variable to control Amavis' loglevel (#1947) 2021-05-06 23:51:45 +02:00
Moritz Marquardt 94b5ac49c1
fix!: use dovecot's LDAP uris option instead of hosts (#1901)
* Use dovecot's LDAP uris option instead of hosts (fixes #1510)

* Clean up variables & environment documentation for #1901

Co-authored-by: Frederic Werner <20406381+wernerfred@users.noreply.github.com>
Co-authored-by: Georg Lauterbach <44545919+aendeavor@users.noreply.github.com>
2021-04-19 09:02:03 +02:00
Georg Lauterbach 8313d9753b
Adjusted documentation for service name and Traefik certificate issuance (#1918)
Co-authored-by: Casper <casperklein@users.noreply.github.com>
2021-04-18 15:21:08 +02:00
Casper f7836c8b1a
Fail2Ban block behaviour (#1914)
* new default: block IP on all ports

* introduce FAIL2BAN_BLOCKTYPE

* fix test

* tests added

* test added

* test blocktype drop

* merged two tests
2021-04-18 12:55:43 +02:00
Moritz Marquardt 271d94a37e
Add LDAP_QUERY_FILTER_SENDERS setting for spoof protection with LDAP (#1902) 2021-04-17 22:40:19 +02:00
Casper ba0f9199b7
fail2ban cleanup (#1895) 2021-04-11 15:33:39 +00:00
Casper 22321c308c
fix SpamAssassin spelling/typos (#1869) 2021-03-28 22:07:52 +02:00
Casper dd0b399f33
feat: Introduce ENABLE_AMAVIS env (#1866)
* Introduce ENABLE_AMAVIS env

* missing 'fi' added

* documentation added

* add condition for amavis fix function

* Fix spelling

Co-authored-by: William Desportes <williamdes@wdes.fr>

* Fix spelling

Co-authored-by: William Desportes <williamdes@wdes.fr>

Co-authored-by: William Desportes <williamdes@wdes.fr>
Co-authored-by: Frederic Werner <20406381+wernerfred@users.noreply.github.com>
2021-03-28 15:37:48 +02:00
Georg Lauterbach 0d9fb096b7
adjusting _seup_supervisor to not restart when wrong log-level given 2021-02-25 10:57:20 +01:00
Georg Lauterbach 1ef66fd5c5
first (may-be) fixes for v9.0.0 startup problems (#1835)
* first (may-be) fixes for v9.0.0 startup problems
* adjust grep showing an error when it shouldn't
2021-02-24 10:12:20 +01:00
Georg Lauterbach c881facbd2
start-mailserver.sh split (#1820)
* splitting start-mailserver.sh

* refactoring part 2

* refactored setup-stack.sh
* stzarted adjusting target/bin/*.sh to use new usage format

* corrected lowercase-uppercase test error

* better handling of .bashrc variable export

* linting tests and fix for default assignements

* last stylistic changes and rebase
2021-02-23 20:03:01 +01:00