Go to file
Brennan Kinney 623d53bea8
Merge pull request #2938 from docker-mailserver/ci/more-parallel-tests
tests: Convert more serial tests into parallel ready

<details>
<summary>Commit message history for reference</summary>

```markdown
tests(chore): Move some serial tests into parallel sets
===
Additionally with the `tls.bash` helper for the `letsencrypt` tests.
```

```markdown
tests: Adjusted files not directly related to tests
===
`tls.bash` helper was adapted to the new helper scripts location. The `setup.bash` helper saw a bugfix (expanding the array properly) and updates the container default config to configure for IPv4 explicitly.

The IPv4 default was added after recent Docker pushes and I saw weird IPv6 related errors in the logs.. now we're sure IPv4 is the default during tests.

Added functionality to check if a process is running:
- This change adds a helper function to check whether a program is running inside a container or not.
- This added the need for a function like `_run_in_container` but allowing for providing an explicit container name.
- Future PRs can use this helper function now to check whether a process is running or not. This was done for the tests of Fail2Ban, but can be used for other tests in the future as well.

---

chore: Restructured BATS flags in `Makefile`

The `Makefile` has seen a bit of a restructuring when it comes to flags:

1. The `MAKEFLAGS` variables is used by `make`, and allows for adding additional flags that can be used within in recursive calls (via `$(MAKE)`) too,  thus DRY approach.
2. The flags for calling BATS were adjusted. `--no-parallelize-within-files` has been added as well to ensure tests  _inside_ a single file are run sequentially.

`dms-test` prefix matching changed to expect a `_` suffix as a delimiter.

---

docs: Add a note regarding output from running tests in parallel
```

```markdown
tests: Adjust parallel tests
===
- The usual serial to parallel test conversion to utilize the `setup.bash` common setup structure, and adding a `TEST_PREFIX` var for each test case to leverage.
- Standardize on parallel test naming conventions for variables / values.
- More consistent use of `bash -c` instead of `/bin/bash -c` or `/bin/sh -c`.
- Using the `_run_in_container` helper instead of `run docker exec ${CONTAINER_NAME}`.
- Updates tests to use the `check_if_process_is_running` helper.

---

chore: Revise inline docs for the `ssl_letsencrypt` test

- Moves the override to be in closer proximity to the `initial_setup` call, and better communicates the intent to override.
- Removes top comment block that is no longer providing value or correct information to maintainers.
- Revised `acme.json` test case inline doc comments.
```

```markdown
refactor: Parallel Tests
===
- `disabled_clamav_spamassassin`:
  - Just shuffling the test order around, and removing the restart test at the end which doesn't make sense.

- `postscreen`:
  - Now uses common helper for getting container IP
  - Does not appear to need the `NET_ADMIN` capability?
  - Reduced startup time for the 2nd container + additional context about it's relevance.
  - Test cases are largely the same, but refactored the `nc` alternative that properly waits it's turn. This only needs to run once. Added additional commentary and made into a generic method if needed in other tests.

- `fail2ban`:
  - Use the common container IP helper method.
  - Postscreen isn't affecting this test, it's not required to do the much slower exchange with the mail server when sending a login failure.
  - IP being passed into ENV is no longer necessary.
  - `sleep 5` in the related test cases doesn't seem necessary, can better rely on polling with timeout.
  - `sleep 10` for `setup.sh` also doesn't appear to be necessary.

- `postgrey`:
  - Reduced POSTGREY_DELAY to 3, which shaves a fair amount of wasted time while still verifying the delay works.
  - One of the checks in `main.cf` doesn't seem to need to know about the earlier spamhaus portion of the line to work, removed.
  - Better test case descriptions.
  - Improved log matching via standard method that better documents the expected triplet under test.
  - Removed a redundant whitelist file and test that didn't seem to have any relevance. Added a TODO with additional notes about a concern with these tests.
  - Reduced test time as 8 second timeouts from `-w 8` don't appear to be required, better to poll with grep instead.
  - Replaced `wc -l` commands with a new method to assert expected line count, better enabling assertions on the actual output.

- `undef_spam_subject`:
  - Split to two separate test cases, and initialize each container in their case instead of `setup_file()`, allowing for using the default `teardown()` method (and slight benefit if running in parallel).

- `permit_docker`:
  - Not a parallel test, but I realized that the repeat helper methods don't necessarily play well with `run` as the command (can cause false positive of what was successful).
```

```markdown
docs: Revise contributing advice for tests
```

</details>
2023-01-06 16:53:20 +13:00
.github chore(deps): Bump actions/stale from 6 to 7 (#2960) 2022-12-26 23:01:08 +01:00
config-examples Remove unusual space from shebang line (#2834) 2022-10-17 10:40:09 +02:00
docs docs: Revise contributing advice for tests 2023-01-06 16:50:09 +13:00
target feature: provide initial Rspamd support (#2902) 2023-01-05 08:39:00 +01:00
test refactor: Parallel Tests 2023-01-06 16:50:09 +13:00
.all-contributorsrc Update contributors (#2143) 2021-08-28 15:23:11 +02:00
.dockerignore Update check (#1951) 2021-05-19 21:18:06 +02:00
.editorconfig cleaned up >/dev/nulls in Dockerfile and replaced em dashes with normal dashes (#2024) 2021-06-08 13:20:20 +12:00
.gitignore build: cleaned up Makefile (#2833) 2022-10-17 08:08:04 +13:00
.gitmodules tests: Update submodules for bats (#2715) 2022-08-12 11:09:17 +12:00
CHANGELOG.md chore: Update changelog and version (#2944) 2022-12-22 23:27:40 +01:00
CODE_OF_CONDUCT.md docs(fix): Update wiki references to the new docs url 2021-03-25 11:49:24 +13:00
CONTRIBUTORS.md docs(CONTRIBUTORS): update contributors (#2969) 2023-01-01 15:17:35 +01:00
docker-compose.yml Add basic container healthcheck (#2625) 2022-06-07 11:54:58 +02:00
Dockerfile feature: provide initial Rspamd support (#2902) 2023-01-05 08:39:00 +01:00
LICENSE Final Migration Step (#6) 2021-01-16 10:16:05 +01:00
mailserver.env feature: provide initial Rspamd support (#2902) 2023-01-05 08:39:00 +01:00
Makefile tests: Adjusted files not directly related to tests 2023-01-06 16:50:09 +13:00
README.md docs(fix): README - Update CI status badge URL (#2951) 2022-12-24 02:32:06 +13:00
setup.sh setup.sh: Remove __err function (#2876) 2022-10-31 10:46:00 +01:00
VERSION chore: Update changelog and version (#2944) 2022-12-22 23:27:40 +01:00

Docker Mailserver

ci::status docker::pulls documentation::badge

A production-ready fullstack but simple mail server (SMTP, IMAP, LDAP, Antispam, Antivirus, etc.). Only configuration files, no SQL database. Keep it simple and versioned. Easy to deploy and upgrade. Documentation via MkDocs.

Originally created by @tomav, docker-mailserver is now maintained by volunteers since January 2021.

If you have issues, read the full README and the documentation for your version (default is edge) first before opening an issue. The issue tracker is for issues, not for personal support.

  1. Included Services
  2. Issues and Contributing
  3. Requirements
  4. Usage
  5. Examples
  6. Environment Variables
  7. Documentation
  8. Release Notes

Included Services

Requirements

Recommended:

  • 1 Core
  • 2GB RAM
  • Swap enabled for the container

Minimum:

  • 1 vCore
  • 512MB RAM
  • You'll need to deactivate some services like ClamAV to be able to run on a host with 512MB of RAM. Even with 1G RAM you may run into problems without swap, see FAQ.

Usage

Available Images / Tags - Tagging Convention

CI/CD will automatically build, test and push new images to container registries. Currently, the following registries are supported:

  1. DockerHub
  2. GitHub Container Registry

All workflows are using the tagging convention listed below. It is subsequently applied to all images.

Event Image Tags
push on master edge
push tag 1.2.3, 1.2, 1, latest

Get the Tools

Since Docker Mailserver v10.2.0, setup.sh functionality is included within the container image. The external convenience script is no longer required if you prefer using docker exec <CONTAINER NAME> setup <COMMAND> instead. If you're new to docker-mailserver, it is recommended to use the script setup.sh for convenience.

DMS_GITHUB_URL='https://raw.githubusercontent.com/docker-mailserver/docker-mailserver/master'
wget "${DMS_GITHUB_URL}/docker-compose.yml"
wget "${DMS_GITHUB_URL}/mailserver.env"
wget "${DMS_GITHUB_URL}/setup.sh"
chmod a+x ./setup.sh

Create a docker-compose Environment

  1. Install the latest docker-compose
  2. Edit docker-compose.yml to your liking
    • substitute mail (hostname) and example.com (domainname) according to your FQDN
    • if you want to use SELinux for the ./docker-data/dms/config/:/tmp/docker-mailserver/ mount, append -z or -Z
  3. Configure the mailserver container to your liking by editing mailserver.env (Documentation), but keep in mind this .env file:
    • only basic VAR=VAL is supported (do not quote your values!)
    • variable substitution is not supported (e.g. 🚫 OVERRIDE_HOSTNAME=$HOSTNAME.$DOMAINNAME 🚫)

Note: If you're using podman, make sure to read the related documentation

Get up and running

First Things First

Use docker-compose up / down, not docker-compose start / stop. Otherwise, the container is not properly destroyed and you may experience problems during startup because of inconsistent state.

You are able to get a full overview of how the configuration works by either running:

  1. ./setup.sh help which includes the options of setup.sh.
  2. docker run --rm docker.io/mailserver/docker-mailserver:latest setup help which provides you with all the information on configuration provided "inside" the container itself.

If no docker-mailserver container is running, any ./setup.sh command will check online for the :latest image tag (the current stable release), performing a docker pull ... if necessary followed by running the command in a temporary container.

$ ./setup.sh help
Image 'docker.io/mailserver/docker-mailserver:latest' not found. Pulling ...
SETUP(1)

NAME
    setup - 'docker-mailserver' Administration & Configuration script
...

$ docker run --rm docker.io/mailserver/docker-mailserver:latest setup help
SETUP(1)

NAME
    setup - 'docker-mailserver' Administration & Configuration script
...

Starting for the first time

On first start, you will need to add at least one email account (unless you're using LDAP). You have two minutes to do so, otherwise DMS will shutdown and restart. You can add accounts with the following two methods:

  1. Use setup.sh: ./setup.sh email add <user@domain>
  2. Run the command directly in the container: docker exec -ti <CONTAINER NAME> setup email add <user@domain>

You can then proceed by creating the postmaster alias and by creating DKIM keys.

docker-compose up -d mailserver

# you may add some more users
# for SELinux, use -Z
./setup.sh [-Z] email add <user@domain> [<password>]

# and configure aliases, DKIM and more
./setup.sh [-Z] alias add postmaster@<domain> <user@domain>

Miscellaneous

DNS - DKIM

You can (and you should) generate DKIM keys by running

./setup.sh [-Z] config dkim

If you want to see detailed usage information, run

./setup.sh config dkim help

In case you're using LDAP, the setup looks a bit different as you do not add user accounts directly. Postfix doesn't know your domain(s) and you need to provide it when configuring DKIM:

./setup.sh config dkim domain '<domain.tld>[,<domain2.tld>]'

When keys are generated, you can configure your DNS server by just pasting the content of config/opendkim/keys/domain.tld/mail.txt to set up DKIM. See the documentation for more details.

Custom User Changes & Patches

If you'd like to change, patch or alter files or behavior of docker-mailserver, you can use a script. See the documentation for a detailed explanation.

Updating docker-mailserver

Make sure to read the CHANGELOG before updating to new versions, to be prepared for possible breaking changes.

docker-compose pull
docker-compose down
docker-compose up -d mailserver

You should see the new version number on startup, for example: [ TASKLOG ] Welcome to docker-mailserver 10.1.2.

You're done! And don't forget to have a look at the remaining functions of the setup.sh script with ./setup.sh help.

Supported Operating Systems

We are currently providing support for Linux. Windows is not supported and is known to cause problems. Similarly, macOS is not officially supported - but you may get it to work there. In the end, Linux should be your preferred operating system for this image, especially when using this mail-server in production.

Bare Domains

If you want to use a bare domain (hostname == domainname), see FAQ.

Support for Multiple Domains

docker-mailserver supports multiple domains out of the box, so you can do this:

./setup.sh email add user1@docker.example.com
./setup.sh email add user1@mail.example.de
./setup.sh email add user1@server.example.org

SPF/Forwarding Problems

If you got any problems with SPF and/or forwarding mails, give SRS a try. You enable SRS by setting ENABLE_SRS=1. See the variable description for further information.

Ports

See the documentation for further details and best practice advice, especially regarding security concerns.

Mailboxes (aka IMAP Folders)

INBOX is setup by default with the special IMAP folders Drafts, Sent, Junk and Trash. You can learn how to modify or add your own folders (including additional special folders like Archive) by visiting our docs page Customizing IMAP Folders for more information.

Examples

With Relevant Environmental Variables

This example provides you only with a basic example of what a minimal setup could look like. We strongly recommend that you go through the configuration file yourself and adjust everything to your needs. The default docker-compose.yml can be used for the purpose out-of-the-box, see the usage section.

version: '3.8'

services:
  mailserver:
    image: docker.io/mailserver/docker-mailserver:latest
    container_name: mailserver
    hostname: mail
    domainname: example.com
    ports:
      - "25:25"
      - "143:143"
      - "587:587"
      - "993:993"
    volumes:
      - ./docker-data/dms/mail-data/:/var/mail/
      - ./docker-data/dms/mail-state/:/var/mail-state/
      - ./docker-data/dms/mail-logs/:/var/log/mail/
      - ./docker-data/dms/config/:/tmp/docker-mailserver/
      - /etc/localtime:/etc/localtime:ro
    environment:
      - ENABLE_SPAMASSASSIN=1
      - SPAMASSASSIN_SPAM_TO_INBOX=1
      - ENABLE_CLAMAV=1
      - ENABLE_FAIL2BAN=1
      - ENABLE_POSTGREY=1
      - ENABLE_SASLAUTHD=0
      - ONE_DIR=1
    cap_add:
      - NET_ADMIN
    restart: always

LDAP Setup

Note There are currently no LDAP maintainers. If you encounter issues, please raise them in the issue tracker, but be aware that the core maintainers team will most likely not be able to help you. We would appreciate and we encourage everyone to actively participate in maintaining LDAP-related code by becoming a maintainer!

version: '3.8'

services:
  mailserver:
    image: docker.io/mailserver/docker-mailserver:latest
    container_name: mailserver
    hostname: mail
    domainname: example.com
    ports:
      - "25:25"
      - "143:143"
      - "587:587"
      - "993:993"
    volumes:
      - ./docker-data/dms/mail-data/:/var/mail/
      - ./docker-data/dms/mail-state/:/var/mail-state/
      - ./docker-data/dms/mail-logs/:/var/log/mail/
      - ./docker-data/dms/config/:/tmp/docker-mailserver/
      - /etc/localtime:/etc/localtime:ro
    environment:
      - ENABLE_SPAMASSASSIN=1
      - SPAMASSASSIN_SPAM_TO_INBOX=1
      - ENABLE_CLAMAV=1
      - ENABLE_FAIL2BAN=1
      - ENABLE_POSTGREY=1
      - ONE_DIR=1
      - ENABLE_LDAP=1 # with the :edge tag, use ACCOUNT_PROVISIONER
      - ACCOUNT_PROVISIONER=LDAP
      - LDAP_SERVER_HOST=ldap # your ldap container/IP/ServerName
      - LDAP_SEARCH_BASE=ou=people,dc=localhost,dc=localdomain
      - LDAP_BIND_DN=cn=admin,dc=localhost,dc=localdomain
      - LDAP_BIND_PW=admin
      - LDAP_QUERY_FILTER_USER=(&(mail=%s)(mailEnabled=TRUE))
      - LDAP_QUERY_FILTER_GROUP=(&(mailGroupMember=%s)(mailEnabled=TRUE))
      - LDAP_QUERY_FILTER_ALIAS=(|(&(mailAlias=%s)(objectClass=PostfixBookMailForward))(&(mailAlias=%s)(objectClass=PostfixBookMailAccount)(mailEnabled=TRUE)))
      - LDAP_QUERY_FILTER_DOMAIN=(|(&(mail=*@%s)(objectClass=PostfixBookMailAccount)(mailEnabled=TRUE))(&(mailGroupMember=*@%s)(objectClass=PostfixBookMailAccount)(mailEnabled=TRUE))(&(mailalias=*@%s)(objectClass=PostfixBookMailForward)))
      - DOVECOT_PASS_FILTER=(&(objectClass=PostfixBookMailAccount)(uniqueIdentifier=%n))
      - DOVECOT_USER_FILTER=(&(objectClass=PostfixBookMailAccount)(uniqueIdentifier=%n))
      - ENABLE_SASLAUTHD=1
      - SASLAUTHD_MECHANISMS=ldap
      - SASLAUTHD_LDAP_SERVER=ldap
      - SASLAUTHD_LDAP_BIND_DN=cn=admin,dc=localhost,dc=localdomain
      - SASLAUTHD_LDAP_PASSWORD=admin
      - SASLAUTHD_LDAP_SEARCH_BASE=ou=people,dc=localhost,dc=localdomain
      - SASLAUTHD_LDAP_FILTER=(&(objectClass=PostfixBookMailAccount)(uniqueIdentifier=%U))
      - POSTMASTER_ADDRESS=postmaster@localhost.localdomain
      - POSTFIX_MESSAGE_SIZE_LIMIT=100000000
    cap_add:
      - NET_ADMIN
    restart: always