Previously we only monitored for `$HOSTNAME` in `/etc/letsencrypt/live` and only for hard-coded `.pem` filenames.
This ensures we check the locations of other locations that may not match `$HOSTNAME`, which we also support. Ideally in future at least the directory to look in would be better known in advance..
These are improvements for better supporting the requirements of other tests.
- Opted for passing an array reference instead of an ENV file. This seems to be a better approach and supports more than just ENV changes.
- Likewise, shifted to a `create` + `start` approach, instead of `docker run` for added flexibility.
- Using `TEST_TMP_CONFIG` instead of `PRIVATE_CONFIG` to make the difference in usage with config volume in tests more clear.
- Changed the config volume from read-only volume mount to be read-write instead, which seems required for other tests.
- Added notes about logged failures from a read-only config volume during container startup.
- Added `TEST_CA_CERT` as a default CA cert path for the test files volume. This can be used by default by openssl methods.
Split into scoped commits with messages if further details are needed, view those via the associated PR :)
**Commit Summary:**
**`check-for-changes.sh`**
- Prevent `SSL_DOMAIN` silently skipping when value has wildcard prefix `*.` (_at least this was known as a bugfix when originally committed in linked PR_).
- Improved inlined docs for maintainers.
- Additional logging for debugging.
**`helper-functions.sh:_extract_certs_from_acme`**:
- Fail if the input arg (_`$CERT_DOMAIN`, aka the FQDN_) provided for extraction is empty.
- Use `$CERT_DOMAIN` in place of `$HOSTNAME` and `$1` for a consistent value (_previously could mismatch, eg with `SSL_DOMAIN` defined_).
- The conditional is now only for handling extraction failure (_key or cert value is missing from extraction_).
- Log an actual warning or success (debug) based on outcome.
- Don't use `SSL_DOMAIN` with wildcard value for the `mkdir` letsencrypt directory name (_wildcard prefix `*.` is first stripped instead_).
**`acme_extract`** (_new python utility for `acme.json` handling_):
- Extracted out into a python script that can be treated as a utility in the `$PATH` like other helper scripts. It can now be used and optionally tested directly instead of via `helper-functions.sh`.
-Made compatible with Python 3, as Python 2 is EOL and no longer in newer versions of Debian.
These files will replace the existing `test/config/letsencrypt` content which has some random provisioned FQDN for letsencrypt that doesn't match the FQDN tested, `acme.json` files with FQDNs that don't match those certs FQDNs and changes to certs that won't expire until 2031. `test/config/letsencrypt` will be removed with the associated test update PR.
The changes amount to:
- Re-configuring the FQDN values that some certs were created for (_needed for flexibility in testing_).
- Adding an `*.example.test` wildcard (_both RSA and ECDSA_).
- Adding `acme.json` encoded versions (_traefik extraction support will use these instead_).
- Updated / new internal docs for maintainers of this content.
For more detailed information on those changes, please see the associated commit messages via the PR.
Mostly cleans up the code and documents it better, although there are some minor fixes for handling `SSL_DOMAIN` ENV and additional logging added for spotting issues related to it in future when troubleshooting.
Commits are scoped with context messages for easing review if necessary. Overview of changes:
Traefik specific:
- Logic extracted out into it's own function.
- Conditional reworked to assist with debugging.
- `SSL_DOMAIN` must not be empty when attempting to extract.
- Added additional notes.
`SSL_TYPE=letsencrypt` case:
- Revised top note block.
- Correct handling for `SSL_DOMAIN`.
- Removed some unnecessary nesting.
- Less repetitive error message for `LETSENCRYPT_DOMAIN`.
- Added use of panics where appropriate (kept `return 1` so failures still exit functionality early).
- Improved inline docs.
Dovecot quota support would log auth failures when Postfix validated incoming mail to accept/reject and the `check_policy_service` for `quota-status` was queried with a recipient that was an account alias.
When Dovecot is not aware of the user account, it will not be able to check a quota and inform Postfix that everything is fine, Postfix will accept the mail and send it to Dovecot, where if the quota is exceeded will result in a bounce back to the sender. This is considered "backscatter" and can be abused by spammers forging the sender address which can get your server blacklisted.
The solution is to either disable quota support `ENABLE_QUOTAS=0`, or as a workaround, add dummy accounts to Dovecot userdb for aliases in `postfix-virtual.cf` (not `postfix-aliases.cf`), these dummy accounts will map to the real user account mailbox (real users are defined in `postfix-accounts.cf`).
The workaround is naive, in that we only check for basic 1-to-1 alias mapping to real accounts. This will still be an issue for aliases that map to another alias or multiple addresses (real or alias). Unfortunately Postfix will not expand aliases until accepting mail where this would be too late.
A better solution is to proxy the `check_policy_service` from Dovecot `quota-status` that Postfix queries in `main.cf:smtpd_recipient_restrictions`, however this requires a fair amount more of additional work and still requires an implementation to recursively query aliases for nested or multiple address mappings, which can then be forwarded to the `quota-status` service configured by Dovecot in `/etc/dovecot/conf.d/90-quota.conf`.
LDAP users are unaffected as quota support is not supported/implemented with `docker-mailserver` at this time, it is always considered disabled when using LDAP.
---
Additionally Dovecot configuration for `passdb` has been fixed to use the correct password hash scheme of `SHA512-CRYPT`.
Co-authored-by: Casper <casperklein@users.noreply.github.com>
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
This was all introduced by the original project author early on, no explanation for it.
- None of the paths they use as `argv` values exist.
- `uucp` doesn't seem relevant to the project. No justification for it, no issues or PRs in project history or codebase.
See associated PR for further details and linked resources.
Using `set -ex` will exit the script as soon as a non-zero exit code is returned, such as when the docker image fails building the docs due to `build --strict` catching broken links. This also removes the need for `|| exit` when changing directory.
This seems fine for a small script, but AFAIK an alternative fix is just adding `|| exit` to the end of the `docker run` command too? There appears to be advice [against adopting `-e` carelessly](http://mywiki.wooledge.org/BashFAQ/105), while others [encourage `-e`](http://redsymbol.net/articles/unofficial-bash-strict-mode/). I know that several maintainers here have preference towards `set -e` so I've kept the original PR solution.
Additionally:
- `-x` is used to improve command visibility when reviewing the workflow log output.
- `--name` isn't necessary, but was part of the original PR.
- I've chosen not to include `-o pipefail`, only because no pipes are used in this script.
* docs(fix): Fix broken links
* ci(docs): Added inline docs
Extra documentation context for maintainers to quickly grok what's going on.
* chore(docs): Minor typo fix by wernerfred
Added from their related PR by request.
* docs(ssl): Adjust heading levels for provisioning sections
- Group provisioning sections under one heading level.
- Use `attr_list` syntax for headings to make the ToC sidebar entry less verbose.
* docs(ssl): Minor fixes
Typos, formatting.
* docs(ssl): Rephrase Traefik wildcard support
Split the line out into multiple with better phrasing.
* docs(ssl): Add FQDN section
We briefly mention the same info twice on the docs page, but as it applies to all provisioners in general, it's been given it's own detailed section with examples.
Single section to inform users about an FQDN, how it's configured and understood by `docker-mailserver` for both Docker CLI and `docker-compose.yml` variations.
Adds note about wildcard support and bare domains to clear up any confusion configuring FQDN for these two.
Additional note about Certbot using symlinks for it's cert storage.
* chore: Add FQDN comment for `docker-compose.yml` example config
* ci: Fix lint check status update
The lint workflow is not important for this PR, but a fixed requirement to pass for merging.
As this workflow is triggered by `schedule` or `workflow_dispatch`, it will not trigger other events such as `pull_request` for other workflows to respond to.
Since the linting workflow is not important for this type of PR, we can pretend it was "skipped" and set the check status to "success". This is simpler than running the actual Lint workflow redundantly.
* ci: Remove workflow_run approach
This didn't work out, reverting.
This should resolve the issue of the lint workflow not being triggered by PRs opened via another workflow (`contributors.yml`).
This workflow will be triggered after the dependent workflow completes (regardless of status).
This was a community contributed guide from the Github Wiki prior to docs migration. I've rewritten it by restructuring the content, introducing numerical steps and revising some of the content, while removing third-party software that was unnecessary (the original authors and content related to their use-case, Moodle).
See the PR for further details and reference links regarding the original documents history.
The PR provides improved diff via separate commits scoping changes at the correct change bounds, unfortunately the full diff doesn't align to those boundaries well making it more difficult to review vs individual commits.
Below commit messages are roughly equivalent to what is listed on the PR. The PR provides additional linked resources for reference to support commit message statements.
---
* docs: Add CT log warning
- Added a warning to make users aware that using a public CA like _Let's Encrypt_ will publicly log information that may be somewhat sensitive, or undesirable to have historic records made public which cannot be redacted.
* docs: Revise the manual `certbot` guide
- The `letsencrypt` repo that was linked early in this guide now redirects to the [Certbot repo](https://github.com/certbot/certbot).
- More explicit volume mount instruction for CertBot; the local location was a tad vague.
- Better clarified `/etc/letsencrypt/live` contents structure, as well as FQDN info. Removed the misleading `fqdn:` from `docker-compose.yml` example snippet.
* docs: Revise certbot with Docker guide
- General rewrite of the Docker Certbot section with additional tips (_renewals with automation, and using a alternative CA_).
- Generalized tone and paths in content.
- Update volume mount paths to be consistent with recent normalization effort.
- Moved some instructions into inline-comments for script examples instead.
* docs: Revise Docker with `nginx-proxy` and `acme-companion`
- Break apart into individual steps, indenting content into the step as appropriate.
- Use normalized volume paths (`docker-data/<service>/` prefix).
- `letsencrypt-nginx-proxy-companion` has _changed project name to `acme-companion`_, and _transferred to new maintainers and the `nginx-proxy` organization_. This also affects the DockerHub image references.
- `acme-companion` has _switched from using `simp_le` to `acme.sh`_ for provisioning certificates. This requires mounting an additional volume for persisting provisioner state.
- The dummy container (_webmail_) is no longer `library/nginx`, just [`nginx`](https://hub.docker.com/_/nginx). This container also doesn't appear to be required. I've verified that the ENV can be given to the `mailserver` service container directly. Retained for now.
* docs: Revise Docker Compose with `nginx-proxy` and `acme-companion`
Heavy rewrite of this section. Like the previous commit mentions, this content was outdated. It has been simplified with improved documentation and reference links.
It also looks like there was a mistake in the existing config example as it uses the regular `nginx` image instead of `nginx-proxy`.
- The bulk of the `mailserver` service has been removed, users are advised to have an existing `docker-compose.yml` config for `docker-mailserver` and update only what is relevant to integrate with the cert provisioner.
- `DEBUG` is _false_ by default.
- The `networks:` portion of the example appears to be taken from upstream, _which that has since dropped it_. While we could continue to document this, I consider it more of an advanced config detail that we don't need to touch on in our docs.
- The `htpasswd` volume is unnecessary, only relevant if using _"Basic Authentication"_ to protect access to web service endpoints. `conf.d/` is also not required by default, it can be useful for the `standalone` mode (_documented as a `tip`_). Remaining volumes have inline-comments to document their purpose.
- `volumes_from:` is _not supported in v3 Compose format_, _only v2_ and the Docker CLI. I did not want to advise v2, so I've duplicated the volumes between the two containers instead. Internally `acme-companion` would rely on `volumes_from:` to identify the `nginx-proxy` container, it _provides alternative discovery methods_, the label is outdated and refers the legacy label (_their script logic is the same_); using the ENV `NGINX_PROXY_CONTAINER` seemed most appropriate and has been added.
- Upstream `acme-companion` docs only cover support for v2 Compose format. _There is a note regarding `nginx-proxy`_ having _volumes configured in it's Dockerfile_. Providing a volume for `/etc/nginx/dhparam` is required to avoid creating anonymous volumes each run of `nginx-proxy`. I've used a named data volume here to make it stick out more, it's not desirable and upstream should fix this, then we can drop it.
- I've also opted to only demonstrate the _Two Container (Basic) setup_ that upstream documents. Previously our docs have been showing _`docker-gen` with the Three Container (Advanced) setup_, which allows for not having the Docker API socket attached as a volume to a container exposed to the web. This reduces the security a bit, and I have not mentioned that on our docs. I could caution the reader with a link to upstream about the risk, but I don't think we should maintain the `docker-gen` setup.
* docs(fix): Update anchor links
These mismatched the current section headers they were meant to link to.
* function _defunc removed
* _shutdown is better than just notify in that cases
* PANIC_TYPE 'fail-init' introduced
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
