Commit graph

2377 commits

Author SHA1 Message Date
Georg Lauterbach 2234a53b60
docs: improve Rspamd docs (#3147) 2023-03-05 06:23:11 +00:00
Georg Lauterbach 8ec5dbec74
scripts: touchups for v12.0.0 (#3144) 2023-03-04 10:57:42 +01:00
Georg Lauterbach 5e9849d94f
rspamd: rename ENABLE_REDIS & add persistence for Redis (#3143) 2023-03-04 10:45:43 +01:00
Georg Lauterbach f0edcc28d9
config: ensure SASL socket file is not inside a volume mount (#3131) 2023-03-03 23:42:55 +01:00
Brennan Kinney 9a9380150e
fix: Avoid creating an unnecessary syslog socket for Postfix (#3134)
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2023-03-03 23:08:51 +01:00
Georg Lauterbach 5ec6845c96
config/ENV: improve Postfix config for spoof protection (#3127) 2023-03-03 15:55:32 +01:00
Brennan Kinney aa4d4fe315
ci(fix): Only apply permissions at the job level (#3142)
If permissions are specified at the workflow level, any that are not explicitly set became `none` and jobs cannot request that to change.

Permissions are therefore scoped to the job itself.
2023-03-03 15:32:00 +01:00
Georg Lauterbach 9e2f96448a
scripts: remove PostSRSD wrapper (#3128)
* remove PostSRSD wrapper

The setup is now completely done during _actual_ setup phase. The
wrapper did not even catch signals (SIGINT, etc.), which I think is
strange.

I also added all the ENVs the wrapper relied on (which previously could
have been unset) to the variables script.

* forgot adjusting the `Dockerfile`
2023-03-02 17:44:01 +01:00
github-actions[bot] b451742f0a
docs(CONTRIBUTORS): update contributors (#3135)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2023-03-01 16:36:54 +01:00
Georg Lauterbach f3edcf9cd5
issue warning in case of improper restart (#3129) 2023-03-01 16:33:54 +01:00
Georg Lauterbach 0949f16344
scripts/ENV: make disabling Redis possible (#3132)
* make disabling Redis possible

* add documentation

* Apply suggestions from code review

* Update docs/content/config/environment.md

---------

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2023-03-01 16:24:59 +01:00
Georg Lauterbach dfc2b39308
scripts: housekeeping & cleanup setup (2/2) (#3123) 2023-02-28 10:25:23 +01:00
Georg Lauterbach 9ead9a54ac
scripts: follow up of #3115 (feedback) (#3124) 2023-02-27 23:37:35 +01:00
Georg Lauterbach 4b04c3e31c
scripts: housekeeping & cleanup setup (1/2) (#3121) 2023-02-27 20:21:45 +01:00
Georg Lauterbach f35b60042f
scripts: split setup-stack.sh (#3115) 2023-02-26 11:42:14 +01:00
Brennan Kinney 1592698637
fix: Postfix service should proxy signals received (#3118)
`postfix start-fg` was not properly responding to signals received to stop. This caused `supervisorctl restart postfix` and `supervisor stop postfix` to not work as expected (_stopping the Postfix master process, before attempting to start the service again_).

Supervisor does not support custom commands for restarting or stopping a service, relying only on managing the process via  a signal. In the past we used a wrapper script to TRAP the signals and trigger commands that way.

However there is a feature which allows us to proxy signals to a different process by referencing a PID file. As Postfix master process creates a pid file when started, we can avoid a wrapper script and the `supervisorctl` functionality works as intended 👍
2023-02-26 19:32:53 +13:00
Georg Lauterbach ae05e6a7c3
tests: improve _send_email (#3105) 2023-02-24 10:44:18 +01:00
Dmitry R 199e3c7721
config: disable SMTP authentication on port 25 (#3006)
* postfix: remove smtpd_sasl_auth_enable global setting

* tests: disable auth on 25 port

* tests: revert ldap-smtp-auth-spoofed-sender-with-filter-exception.txt

* Skip failing test

The test seems to have been broken from the beginning.

Sadly, no LDAP maintainers can verify. Added a TODO item if ever a LDAP maintainer comes around.

* Apply PR feedback

---------

Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2023-02-23 15:19:39 +01:00
Georg Lauterbach 4e82d4de54
ci/docs: add vulnerability scanning workflow & security policy (#3106) 2023-02-23 08:53:12 +01:00
Georg Lauterbach 972406099e
docs: add docs about Abusix integration into Rspamd (#3104) 2023-02-22 10:46:24 +01:00
Georg Lauterbach 35692a9111
ci: refactored sedfile & used _send_mail where possible (#3103)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2023-02-22 10:26:04 +01:00
Georg Lauterbach 853301338c
completely refactor README & parts of docs (#3097)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2023-02-19 13:25:14 +01:00
Georg Lauterbach bee9e3627d
rspamd: add feature for adjusting options with a file parsed by DMS (#3059)
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2023-02-19 12:36:43 +01:00
Georg Lauterbach 40e10d755d
setup: improve Amavis setup routine (#3079)
* improve Amavis setup routine

see https://github.com/docker-mailserver/docker-mailserver/pull/3046#issuecomment-1423718811
2023-02-18 19:22:20 +01:00
Georg Lauterbach f5d325bdc1
fix restrict-access (#3067) 2023-02-18 16:52:42 +01:00
Georg Lauterbach 632012aead
add information so Envelope From is properly set (#3081) 2023-02-18 16:21:51 +01:00
Brennan Kinney 1c8a160621
chore: Remove delay starting the change detection service (#3064)
* chore: Only replace `CHKSUM_FILE` when a change has been processed

* chore: Change Detection service should be the last daemon started

* chore: Remove 10 second startup delay for change detector

There should be no concern with conflicts as any writes should have already been done by the time this daemon service is started.

* tests(fix): `smtp_delivery.bats` must wait for Amavis

The change event for adding a user can be processed much sooner now, which means Amavis may not yet be ready.

Added extra condition to wait on at least the Amavis port being reachable, and some failure asserts with the mail queue to better catch / debug when this problem occurs.

* chore: Add some minor delay to avoid Amavis failing to connect
2023-02-18 15:51:28 +01:00
dependabot[bot] 16f46c1e14
chore(deps): Bump docker/setup-buildx-action from 2.4.0 to 2.4.1 (#3087)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 2.4.0 to 2.4.1.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v2.4.0...v2.4.1)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-15 11:52:40 +13:00
Casper 26861ddf92
fix order (#3082) 2023-02-13 00:05:52 +01:00
Georg Lauterbach ac1df91181
chore(Postfix): disable DNSBLs (#3069) 2023-02-11 00:24:09 +01:00
Georg Lauterbach 29d8dcafb0
improve bug report template (#3080) 2023-02-10 00:33:58 +01:00
Casper 021a4a3c09
Fix: Make logrotate state persistant (#3077)
Fixes https://github.com/docker-mailserver/docker-mailserver/issues/3075
2023-02-09 22:34:26 +00:00
Brennan Kinney 88767f7cc8
tests(refactor): open_dkim.bats (#3060)
* tests(refactor): Make test cases for opendkim keysizes DRY

- These all do roughly the same logic that can be split into two separate methods.
- `_should_generate_dkim_key()` covers a bit more logic as it can be leveraged to handle other test cases that also perform the same logic.
- The `config/opendkim/` doesn't seem necessary for tests. Only the first few test cases here are testing against it, so we can conditionally make that available. `process_check_restart.bats` also depended on it to run OpenDKIM successfully, but this was due to the `setup-stack.sh` config defaults failing to find an "empty" file forcing `supervisord` to constantly restart the process..
- With this, there we inverse the default opendkim config, so we don't have to mount unique / empty subfolders for each test case, followed by copying over the two extra configs.

* tests(refactor): DRY up more test cases

All the remaining test cases but the last one were refactored here for a clean commit diff. The last test case will be refactored in the following commit.

Plenty of repeated logic spread across these test cases, now condensed into shared methods.

* tests(refactor): Make final test case DRY

* chore: Migrate to new testing helpers

* chore: Revise test case descriptions

* tests(refactor): Improve and simplify assertions

* tests(refactor): Use common container setup instead of `docker run`

- As the majority of test cases are only running `open-dkim` helper, we don't actually have to wait for a full container setup. So an alternative container start is called.
- Also improves assertions a bit more instead of just counting lines.
- Some test cases don't bind mount all of `/tmp/docker-mailserver` contents, thus don't raise permission errors on subsequent test runs.
- Instead of `rm -f` on some config files, have opted to mount them read-only instead, or alternatively mount an anonymous empty volume instead.
- Collapsed the first three test cases into one, thus no `setup_file()` necessary.
- Shift the `_wait_for_finished_setup_in_container()` method into `_common_container_setup()` instead since nothing else is using `_common_container_start()` yet, this allows for avoiding the wait.

* tests(refactor): Collapse dkim key size test cases into single test case

This makes these tests a bit more DRY, and enhances the raised quality issue with these tests. Now not only is the domain checked in the generated DNS dkim record, but we also verify the key size is corrected in the public and private keys via openssl.

* chore: Revise container names

* chore: Swap order of test case 1 and 2

* tests(refactor): Assert generated log output

- `__should_have_tables_trustedhosts_for_domain` shifted in each test case to just after generating the domains keys.
- Asserts `open-dkim` logs instead of just counting them.
- Added checks for domains that should not be present in a test case.
- Additional coverage and notes about the alias from vhost `@localdomain.com`
- Single assert statement with switch statement as all are using common args.

* chore: Minor changes

* tests(refactor):  Share `find` logic in helpers and tests

* tests(fix): Listing file content does not need to match line order

The order printed from local system vs CI differed causing the CI to fail. The order of lines is irrelevant so `--index` is not required.

Additionally correct the prefix of the called method to be only one `_` now that it's a `common.bash` helper method.

* chore: Collapse custom DKIM selector test into custom DKIM domain test

These cover the same test logic for the most part, the first domain could also be testing the custom selector.

`special_use_folders.bats` + `mailbox_format_dbox` can assert lines instead, removing the need for `--partial`.

* Apply suggestions from code review

Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>

* chore: Split switch statement method into wrapper methods

---------

Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2023-02-10 00:18:06 +13:00
Brennan Kinney 646e010cb7
ci(fix): Temporarily avoid specifying provenance (#3072)
* ci(fix): Temporarily avoid specifying `provenance`

As the test workflow does not use the `docker-container` buildx driver, it uses the Docker Engine bundled BuildKit version which until v23 release does not support attestations.

Likewise the current buildx version in CI is `0.10.0` which does not respect `--provenance false`, the presence of the option appears to trigger a BuildKit version compatibility check and fail early before it considers the value of the option.

* chore: Use buildx `docker-container` driver instead

An alternative solution to omitting `provenance: false` (_not supported by buildx 0.10.0 with default `docker` driver when Docker Engine bundles BuildKit less than 0.10.0, which is the case prior to the Docker Engine v23 release_). 

This approach provides more consistency with the build and publish workflows by using the same buildx `docker-container` driver (_and thus newer BuildKit, enabling support for  `provenance: false`_).

* chore: Revise test workflow inline docs

Buildx `docker-container` driver is not needed here, but it does seem like it improves cache-hit ratio when building from the retrieved build cache (from the earlier build workflow). Possibly due to building with the same BuildKit version.
2023-02-09 22:33:50 +13:00
dependabot[bot] 9e9f8026d9
chore(deps): Bump docker/build-push-action from 3.3.0 to 4.0.0 (#3066)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: casperklein <casperklein@users.noreply.github.com>
2023-02-08 00:07:56 +01:00
Casper e7790ce272
Better default value for SA_KILL variable (#3058) 2023-02-07 19:26:21 +00:00
Georg Lauterbach 2caec2cdf6
tests: refactor POP3, IMAP (actually SASLauthd + RIMAP) & relay (#3062)
* refactor `mail_pop3.bats`
* refactor `mail_with_imap.bats`
* refactor `mail_with_relays.bats`
* moved test that that did not belong into POP3 test
* slightly clean up `no_container.bats`

Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
2023-02-06 23:59:01 +01:00
Georg Lauterbach 4efbbbe571
remove CI ENV & disable fail-fast (#3065) 2023-02-06 23:49:21 +01:00
Georg Lauterbach 00b1d88ed7
removal: configomat (submodule) (#3045) 2023-02-05 12:39:05 +01:00
Leon Busch-George 9df71c27a0
fix: Only listen on 127.0.0.1 for the Dovecot quota-status service (#3057) 2023-02-04 09:27:10 +00:00
Brennan Kinney 05db27f817
tests(refactor): Extract mail account management tests from tests.bats (#3055)
* chore: Extract out accounts test cases from `tests.bats`

Standard test file format, the test cases have been copied over unmodified.

* chore: Revise test case descriptions

* tests(refactor): `accounts.bats`

Revised test cases:
- Some common test case logic extracted to test methods.
- Update direct user management commands to use the `setup email ...` variants.
- Improved assertions.
- Removed `sleep 2` lines as the need for that is ambiguous (may no longer be relevant?)
- Additional commentary for maintaining
- Two test cases for missing `postfix-accounts.cf` opted to just run the image without any volumes instead, as the `without-accounts/` folder  was empty anyway. 2nd test case can instead use a single `docker run` to check  the newly created`postfix-accounts.cf` content.
- `test/config/without-accounts/` remains as `open_dkim.bats` presently uses it.

* chore: Remove unnecessary account removal assert

Traced this back to the original PR where it appears to have been a typo and was probably intended as a cleanup on the `user4` account. Not necessary, removing.

* chore: Rename `accounts.bat` -> `account_management.bats`

---------

* feedback: Avoid `ls` for detecting directories

Replace `ls -d` approach from original test cases

Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>

* feedback: Remove asserting empty output on failure

Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
2023-02-03 22:52:30 +00:00
Georg Lauterbach 774a340d76
chore(Amavis): only add configuration to Postfix when enabled (#3046)
* only add Amavis configuration to Postfix when enabled

Since I am running Rspamd nowadays, I noticed there still are ports open
that belong to Amavis. This is because the Amavis configuration is a
fixed part of Postfix's `master.cf`. I changed that. Now, the Amavis
section is added when Amavis really is enabled.

I took the chance and added proper indentation to `master.cf`; hence the
diff is a bit fuzzy. **But**, only the Amavis part was adjusted, the
rest is just styling.
2023-02-03 17:32:07 +01:00
Brennan Kinney 7a61214aa5
tests: Migrate and combine ENV tests for *_INET_PROTOCOLS (#3052)
* tests: Migrate and combine ENV tests for `*_INET_PROTOCOLS`

These two features + tests were introduced years apart but serve the same purpose for both Postfix and Dovecot.
2023-02-02 15:31:37 +13:00
Brennan Kinney ed6917ff22
tests: Migrate Dovecot DBox tests (sdbox + mdbox) (#3051)
* tests: Migrate Dovecot DBox tests to new testing format
* tests: Group Dovecot related tests into subfolder
2023-02-02 02:30:16 +00:00
Brennan Kinney adb30178ae
tests: Reduce cipher suite test time by a third (#3050)
Using `--openssl` uses the native `openssl` package within the image instead of the older `1.0.2` bundled from `testssl.sh`.

The test is only testing cipher suite compatibility is what we expect it to be, thus we do not need to run `testssl.sh` with a broader range of ciphers.
2023-02-02 02:29:06 +00:00
github-actions[bot] 40053138db
docs(CONTRIBUTORS): update contributors (#3049)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2023-02-02 01:36:40 +01:00
dependabot[bot] 6a5bc44b73
chore(deps): Bump docker/setup-buildx-action from 2.2.1 to 2.4.0 (#3042)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-30 21:12:16 +01:00
Casper fc25cd5f6b
Linting: bump shellcheck version to 0.9.0 (#3041) 2023-01-30 11:30:27 +00:00
Georg Lauterbach e6dee0f2f1
ci: move tests than can be run in parallel now (#3038) 2023-01-30 09:19:47 +01:00
Georg Lauterbach 66f3bbc062
correct the casing for Mime vs MIME (#3040) 2023-01-30 08:58:35 +01:00