From fb82082cf17b9721c1749304e24fcef6630ca034 Mon Sep 17 00:00:00 2001
From: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
Date: Wed, 18 Jan 2023 14:42:55 +1300
Subject: [PATCH] tests(refactor): `mail_fetchmail.bats` + co-locate test cases
 for processes (#3010)

* chore: Co-locate process checking and process restart verification

Extract the test cases for checking a process is running and properly restarts from various test files into a single one:

Core (always running):
opendkim, opendmarc, master (postfix)

ENV dependent:
amavi (amavisd-new), clamd, dovecot, fail2ban-server (fail2ban), fetchmail, postgrey, postsrsd, saslauthd

These now run off a single container with the required ENV and call a common function (the revised version in parallel test cases).

* fix(saslauthd): Quote wrap supervisor config vars

`saslauth.conf` calls `-O` option for most commands defined with an ENV that may be empty/null. This would cause the process to silently fail / die.

This doesn't happen if quote wrapping the ENV, which calls `-O` with an empty string.

Not necessary, but since one of `postgrey` ENV were quote wrapped in `supervisor-app.conf`, I've also done the same there.

* fix(postsrsd): Change supervisor `autorestart` policy to `true`

The PR that introduced the config switched from `true` to `unexpected` without any context. That prevents restart working when the process is killed. Setting to `true` instead will correctly restart the service.

* chore: Remove disabled postgrey test file

`mail_with_postgrey_disabled_by_default.bats` only checked the migrated test cases, removed as no longer serving a purpose.

* tests(refactor): Make `_should_restart_when_killed()` more reliable

The previous version did not ensure that the last checks process was actually restarted, only that it was running.

It turns out that `pkill` is only sending the signal, there can be some delay before the original process is actually killed and restarted.

This can be identified with `pgrep --older <seconds>`. First ensure the process is at a specified age, then after killing check that the process is not running that is at least that old, finally check that there is a younger process actually running.. (_could fail if a process doesn't restart, or there is a delay such as imposed by `sleep` in wrapper scripts for postfix and fail2ban_)

The helper method is not used anywhere else now, move it into this test instead. It has been refactored to accomodate the needs for `--older`, and `--list-full` provides some output that can be matched (similar for `pkill --echo`).

* test(docs): Add inline notes about processes

* chore: Compress test cases into single case with loop

Moves the list of processes into array vars to iterate through instead.

If a failure occurs, the process name is visible along with line number in `_should_restart_when_killed()` to identify what went wrong.

* chore: Handle `FETCHMAIL_PARALLEL=1` process checks as well

* tests: Add test case for disabled ENV

Additional coverage to match what other test files were doing before, ensuring that these ENV can prevent their respective service from running.

* chore: Move `clamd` enabled check to it's own test case

Not sure about this.

It reduces the time of CPU activity (sustained full load on a thread) and increase in memory usage (1GB+ loading signatures database), but as a separate test case it also adds 10 seconds without reducing the time of the test case it was extracted from.

* chore: Make `disabled` variant the 1st test case

* fix: Adjust test cases to pass when using slower wrapper scripts

* tests(refactor): `mail_fetchmail.bats` updated to new format

Additionally merges in the parallel test file.

* chore: Move `config/fetchmail.cf` into separate sub-directory

Keep out of the default base config for tests.

* chore: Change `fetchmail.cf` FQDNs to `.test` TLD

Changed the first configs remote and local user values to more clearly document what their values should represent (_and that they don't need to be a full mail address, that's just what our Dovecot is configured with for login_).

Shifted the `here` to the end of the `is` line. It's optional syntax, only intended to contrast with the remote `there` for readability.

Additionally configured imap protocol. Not tested or verified if that's correct configuration for usage with imap protocol instead. The fetchmail feature tests are currently lacking.

Added an inline doc into the fetchmail test to reference a PR about the importance of the trailing `.` in the config. Updated the partial matching to ensure it matches for that in the value as well.

* chore: Finalize `process-check-restart.bats`

Few minor adjustments. The other ENV for clamd doesn't seem to provide any benefit, trim out the noise. Added a note about why it's been split out.

Fetchmail parallel configs are matching the config file path in the process command that is returned. The `.rc` suffix is just to add further clarity to that.
---
 target/supervisor/conf.d/saslauth.conf        |   8 +-
 target/supervisor/conf.d/supervisor-app.conf  |   6 +-
 test/config/fetchmail.cf                      |  11 -
 test/config/fetchmail/fetchmail.cf            |  11 +
 test/helper/common.bash                       |   8 -
 .../parallel/set1/spam_virus/clamav.bats      |  12 --
 .../disabled_clamav_spamassassin.bats         |   5 -
 .../parallel/set1/spam_virus/fail2ban.bats    |  12 --
 .../set1/spam_virus/postgrey_enabled.bats     |   5 -
 .../parallel/set3/process-check-restart.bats  | 188 ++++++++++++++++++
 test/tests/serial/mail_fetchmail.bats         |  98 +++++----
 .../tests/serial/mail_fetchmail_parallel.bats |  82 --------
 test/tests/serial/mail_smtponly.bats          |   5 -
 test/tests/serial/mail_with_ldap.bats         |  16 --
 ...ail_with_postgrey_disabled_by_default.bats |  23 ---
 test/tests/serial/setup-cli.bats              |   1 +
 test/tests/serial/tests.bats                  |  68 -------
 17 files changed, 269 insertions(+), 290 deletions(-)
 delete mode 100644 test/config/fetchmail.cf
 create mode 100644 test/config/fetchmail/fetchmail.cf
 create mode 100644 test/tests/parallel/set3/process-check-restart.bats
 delete mode 100644 test/tests/serial/mail_fetchmail_parallel.bats
 delete mode 100644 test/tests/serial/mail_with_postgrey_disabled_by_default.bats

diff --git a/target/supervisor/conf.d/saslauth.conf b/target/supervisor/conf.d/saslauth.conf
index 264fff93..508ff83c 100644
--- a/target/supervisor/conf.d/saslauth.conf
+++ b/target/supervisor/conf.d/saslauth.conf
@@ -13,7 +13,7 @@ autostart=false
 autorestart=true
 stdout_logfile=/var/log/supervisor/%(program_name)s.log
 stderr_logfile=/var/log/supervisor/%(program_name)s.log
-command=/usr/sbin/saslauthd -d -a mysql -O %(ENV_SASLAUTHD_MECH_OPTIONS)s
+command=/usr/sbin/saslauthd -d -a mysql -O "%(ENV_SASLAUTHD_MECH_OPTIONS)s"
 pidfile=/var/run/saslauthd/saslauthd.pid
 
 [program:saslauthd_pam]
@@ -22,7 +22,7 @@ autostart=false
 autorestart=true
 stdout_logfile=/var/log/supervisor/%(program_name)s.log
 stderr_logfile=/var/log/supervisor/%(program_name)s.log
-command=/usr/sbin/saslauthd -d -a pam -O %(ENV_SASLAUTHD_MECH_OPTIONS)s
+command=/usr/sbin/saslauthd -d -a pam -O "%(ENV_SASLAUTHD_MECH_OPTIONS)s"
 pidfile=/var/run/saslauthd/saslauthd.pid
 
 [program:saslauthd_rimap]
@@ -31,7 +31,7 @@ autostart=false
 autorestart=true
 stdout_logfile=/var/log/supervisor/%(program_name)s.log
 stderr_logfile=/var/log/supervisor/%(program_name)s.log
-command=/usr/sbin/saslauthd -d -a rimap -r -O %(ENV_SASLAUTHD_MECH_OPTIONS)s
+command=/usr/sbin/saslauthd -d -a rimap -r -O "%(ENV_SASLAUTHD_MECH_OPTIONS)s"
 pidfile=/var/run/saslauthd/saslauthd.pid
 
 [program:saslauthd_shadow]
@@ -40,6 +40,6 @@ autostart=false
 autorestart=true
 stdout_logfile=/var/log/supervisor/%(program_name)s.log
 stderr_logfile=/var/log/supervisor/%(program_name)s.log
-command=/usr/sbin/saslauthd -d -a shadow -O %(ENV_SASLAUTHD_MECH_OPTIONS)s
+command=/usr/sbin/saslauthd -d -a shadow -O "%(ENV_SASLAUTHD_MECH_OPTIONS)s"
 pidfile=/var/run/saslauthd/saslauthd.pid
 
diff --git a/target/supervisor/conf.d/supervisor-app.conf b/target/supervisor/conf.d/supervisor-app.conf
index c0f9c700..0ffe3e1f 100644
--- a/target/supervisor/conf.d/supervisor-app.conf
+++ b/target/supervisor/conf.d/supervisor-app.conf
@@ -85,7 +85,7 @@ autostart=false
 autorestart=true
 stdout_logfile=/var/log/mail/mail.log
 stderr_logfile=/var/log/mail/mail.log
-command=/usr/sbin/postgrey --inet=127.0.0.1:10023 --syslog-facility=mail --delay=%(ENV_POSTGREY_DELAY)s --max-age=%(ENV_POSTGREY_MAX_AGE)s --auto-whitelist-clients=%(ENV_POSTGREY_AUTO_WHITELIST_CLIENTS)s --greylist-text="%(ENV_POSTGREY_TEXT)s"
+command=/usr/sbin/postgrey --inet=127.0.0.1:10023 --syslog-facility=mail --delay="%(ENV_POSTGREY_DELAY)s" --max-age="%(ENV_POSTGREY_MAX_AGE)s" --auto-whitelist-clients="%(ENV_POSTGREY_AUTO_WHITELIST_CLIENTS)s" --greylist-text="%(ENV_POSTGREY_TEXT)s"
 
 [program:amavis]
 startsecs=0
@@ -122,7 +122,7 @@ autorestart=true
 stdout_logfile=/var/log/supervisor/%(program_name)s.log
 stderr_logfile=/var/log/supervisor/%(program_name)s.log
 user=fetchmail
-command=/usr/bin/fetchmail -f /etc/fetchmailrc --nodetach --daemon %(ENV_FETCHMAIL_POLL)s -i /var/lib/fetchmail/.fetchmail-UIDL-cache --pidfile /var/run/fetchmail/fetchmail.pid
+command=/usr/bin/fetchmail -f /etc/fetchmailrc --nodetach --daemon "%(ENV_FETCHMAIL_POLL)s" -i /var/lib/fetchmail/.fetchmail-UIDL-cache --pidfile /var/run/fetchmail/fetchmail.pid
 
 [program:postfix]
 startsecs=0
@@ -145,7 +145,7 @@ command=/usr/local/bin/check-for-changes.sh
 startsecs=0
 stopwaitsecs=55
 autostart=false
-autorestart=unexpected
+autorestart=true
 stdout_logfile=/var/log/supervisor/%(program_name)s.log
 stderr_logfile=/var/log/supervisor/%(program_name)s.log
 command=/usr/local/bin/postsrsd-wrapper.sh
diff --git a/test/config/fetchmail.cf b/test/config/fetchmail.cf
deleted file mode 100644
index 0e6bfc40..00000000
--- a/test/config/fetchmail.cf
+++ /dev/null
@@ -1,11 +0,0 @@
-poll pop3.example.com. with proto POP3
-	user 'username' there with
-	password 'secret'
-	is 'user2@domain.tld'
-	here options keep ssl
-	
-poll pop3-2.example.com. with proto POP3
-	user 'username' there with
-	password 'secret'
-	is 'user3@domain.tld'
-	here options keep ssl
diff --git a/test/config/fetchmail/fetchmail.cf b/test/config/fetchmail/fetchmail.cf
new file mode 100644
index 00000000..aead698c
--- /dev/null
+++ b/test/config/fetchmail/fetchmail.cf
@@ -0,0 +1,11 @@
+poll pop3.third-party.test. with proto POP3
+	user 'remote_username' there with
+	password 'secret'
+	is 'local_username' here
+	options keep ssl
+
+poll imap.remote-service.test. with proto IMAP
+	user 'user3' there with
+	password 'secret'
+	is 'user3@example.test' here
+	options keep ssl
diff --git a/test/helper/common.bash b/test/helper/common.bash
index b9fcd41f..0dee96f8 100644
--- a/test/helper/common.bash
+++ b/test/helper/common.bash
@@ -38,14 +38,6 @@ function _reload_postfix() {
 
 # -------------------------------------------------------------------
 
-# @param ${1} program name [REQUIRED]
-# @param ${2} container name [IF UNSET: ${CONTAINER_NAME}]
-function check_if_process_is_running() {
-  local PROGRAM_NAME=${1:?Program name must be provided explicitly}
-  local CONTAINER_NAME=${2:-${CONTAINER_NAME}}
-  docker exec "${CONTAINER_NAME}" pgrep "${PROGRAM_NAME}"
-}
-
 # @param ${1} target container name [IF UNSET: ${CONTAINER_NAME}]
 function get_container_ip() {
   local TARGET_CONTAINER_NAME=${1:-${CONTAINER_NAME}}
diff --git a/test/tests/parallel/set1/spam_virus/clamav.bats b/test/tests/parallel/set1/spam_virus/clamav.bats
index 3f0d0cff..e9f40f9d 100644
--- a/test/tests/parallel/set1/spam_virus/clamav.bats
+++ b/test/tests/parallel/set1/spam_virus/clamav.bats
@@ -34,11 +34,6 @@ function setup_file() {
 
 function teardown_file() { _default_teardown ; }
 
-@test "process clamd is running" {
-  run check_if_process_is_running 'clamd'
-  assert_success
-}
-
 @test "log files exist at /var/log/mail directory" {
   _run_in_container bash -c "ls -1 /var/log/mail/ | grep -E 'clamav|freshclam|mail.log' | wc -l"
   assert_success
@@ -64,10 +59,3 @@ function teardown_file() { _default_teardown ; }
   _run_in_container bash -c "grep 'Blocked INFECTED' /var/log/mail/mail.log | grep '<virus@external.tld> -> <user1@localhost.localdomain>'"
   assert_success
 }
-
-@test "process clamd restarts when killed" {
-  _run_in_container pkill 'clamd'
-  assert_success
-
-  run_until_success_or_timeout 10 check_if_process_is_running 'clamd'
-}
diff --git a/test/tests/parallel/set1/spam_virus/disabled_clamav_spamassassin.bats b/test/tests/parallel/set1/spam_virus/disabled_clamav_spamassassin.bats
index e3e72635..cd5947b9 100644
--- a/test/tests/parallel/set1/spam_virus/disabled_clamav_spamassassin.bats
+++ b/test/tests/parallel/set1/spam_virus/disabled_clamav_spamassassin.bats
@@ -24,11 +24,6 @@ function setup_file() {
 
 function teardown_file() { _default_teardown ; }
 
-@test "ClamAV - should be disabled by ENV 'ENABLED_CLAMAV=0'" {
-  run check_if_process_is_running 'clamd'
-  assert_failure
-}
-
 @test "ClamAV - Amavis integration should not be active" {
   _run_in_container grep -i 'Found secondary av scanner ClamAV-clamscan' /var/log/mail/mail.log
   assert_failure
diff --git a/test/tests/parallel/set1/spam_virus/fail2ban.bats b/test/tests/parallel/set1/spam_virus/fail2ban.bats
index 6dc79f6c..16355b3c 100644
--- a/test/tests/parallel/set1/spam_virus/fail2ban.bats
+++ b/test/tests/parallel/set1/spam_virus/fail2ban.bats
@@ -33,11 +33,6 @@ function teardown_file() {
   docker rm -f "${CONTAINER1_NAME}" "${CONTAINER2_NAME}"
 }
 
-@test "Fail2Ban is running" {
-  run check_if_process_is_running 'fail2ban-server'
-  assert_success
-}
-
 @test "localhost is not banned because ignored" {
   _run_in_container fail2ban-client status postfix-sasl
   assert_success
@@ -193,10 +188,3 @@ function teardown_file() {
   _run_in_container setup fail2ban unban
   assert_output --partial 'You need to specify an IP address: Run'
 }
-
-@test "restart of Fail2Ban" {
-  _run_in_container pkill fail2ban
-  assert_success
-
-  run_until_success_or_timeout 10 check_if_process_is_running 'fail2ban-server'
-}
diff --git a/test/tests/parallel/set1/spam_virus/postgrey_enabled.bats b/test/tests/parallel/set1/spam_virus/postgrey_enabled.bats
index b396ca3f..fafec481 100644
--- a/test/tests/parallel/set1/spam_virus/postgrey_enabled.bats
+++ b/test/tests/parallel/set1/spam_virus/postgrey_enabled.bats
@@ -40,11 +40,6 @@ function teardown_file() { _default_teardown ; }
   _should_output_number_of_lines 1
 }
 
-@test "Postgrey is running" {
-  run check_if_process_is_running 'postgrey'
-  assert_success
-}
-
 @test "should initially reject (greylist) mail from 'user@external.tld'" {
   # Modify the postfix config in order to ensure that postgrey handles the test e-mail.
   # The other spam checks in `main.cf:smtpd_recipient_restrictions` would interfere with testing postgrey.
diff --git a/test/tests/parallel/set3/process-check-restart.bats b/test/tests/parallel/set3/process-check-restart.bats
new file mode 100644
index 00000000..74022c0b
--- /dev/null
+++ b/test/tests/parallel/set3/process-check-restart.bats
@@ -0,0 +1,188 @@
+load "${REPOSITORY_ROOT}/test/helper/common"
+load "${REPOSITORY_ROOT}/test/helper/setup"
+
+BATS_TEST_NAME_PREFIX='[Process Management] '
+CONTAINER1_NAME='dms-test_process-check-restart_disabled'
+CONTAINER2_NAME='dms-test_process-check-restart_enabled'
+CONTAINER3_NAME='dms-test_process-check-restart_clamav'
+
+function teardown() { _default_teardown ; }
+
+# Process matching notes:
+# opendkim (/usr/sbin/opendkim) - x2 of the same process are found running (1 is the parent)
+# opendmarc (/usr/sbin/opendmarc)
+# master (/usr/lib/postfix/sbin/master) - Postfix main process (Can take a few seconds running to be ready)
+# NOTE: pgrep or pkill used with `--full` would also match `/usr/sbin/amavisd-new (master)`
+#
+# amavi (/usr/sbin/amavi) - Matches three processes, the main process is `/usr/sbin/amavisd-new (master)`
+# NOTE: `amavisd-new` can only be matched with `--full`, regardless pkill would return `/usr/sbin/amavi`
+#
+# clamd (/usr/sbin/clamd)
+# dovecot (/usr/sbin/dovecot)
+# fetchmail (/usr/bin/fetchmail)
+# fail2ban-server (/usr/bin/python3 /usr/bin/fail2ban-server) - Started by fail2ban-wrapper.sh
+# postgrey (postgrey) - NOTE: This process lacks path information to match with `--full` in pgrep / pkill
+# postsrsd (/usr/sbin/postsrsd) - NOTE: Also matches the wrapper: `/bin/bash /usr/local/bin/postsrsd-wrapper.sh`
+# saslauthd (/usr/sbin/saslauthd) - x5 of the same process are found running (1 is a parent of 4)
+
+# Delays:
+# (An old process may still be running: `pkill -e opendkim && sleep 3 && pgrep -a --older 5 opendkim`)
+# dovecot + fail2ban, take approx 1 sec to kill properly
+# opendkim + opendmarc can take up to 6 sec to kill properly
+# clamd + postsrsd sometimes take 1-3 sec to restart after old process is killed.
+# postfix + fail2ban (due to Wrapper scripts) can delay a restart by up to 5 seconds from usage of sleep.
+
+# These processes should always be running:
+CORE_PROCESS_LIST=(
+  opendkim
+  opendmarc
+  master
+)
+
+# These processes can be toggled via ENV:
+# NOTE: clamd handled in separate test case
+ENV_PROCESS_LIST=(
+  amavi
+  dovecot
+  fail2ban-server
+  fetchmail
+  postgrey
+  postsrsd
+  saslauthd
+)
+
+@test "(disabled ENV) should only run expected processes" {
+  export CONTAINER_NAME=${CONTAINER1_NAME}
+  local CONTAINER_ARGS_ENV_CUSTOM=(
+    --env ENABLE_AMAVIS=0
+    --env ENABLE_CLAMAV=0
+    --env ENABLE_FAIL2BAN=0
+    --env ENABLE_FETCHMAIL=0
+    --env ENABLE_POSTGREY=0
+    --env ENABLE_SASLAUTHD=0
+    --env ENABLE_SRS=0
+    # Disable Dovecot:
+    --env SMTP_ONLY=1
+  )
+  init_with_defaults
+  common_container_setup 'CONTAINER_ARGS_ENV_CUSTOM'
+
+  # Required for Postfix (when launched by wrapper script which is slow to start)
+  wait_for_smtp_port_in_container "${CONTAINER_NAME}"
+
+  for PROCESS in "${CORE_PROCESS_LIST[@]}"
+  do
+    run _check_if_process_is_running "${PROCESS}"
+    assert_success
+    assert_output --partial "${PROCESS}"
+    refute_output --partial "is not running"
+  done
+
+  for PROCESS in "${ENV_PROCESS_LIST[@]}" clamd
+  do
+    run _check_if_process_is_running "${PROCESS}"
+    assert_failure
+    assert_output --partial "'${PROCESS}' is not running"
+  done
+}
+
+# Average time: 23 seconds (29 with wrapper scripts)
+@test "(enabled ENV) should restart processes when killed" {
+  export CONTAINER_NAME=${CONTAINER2_NAME}
+  local CONTAINER_ARGS_ENV_CUSTOM=(
+    --env ENABLE_AMAVIS=1
+    --env ENABLE_FAIL2BAN=1
+    --env ENABLE_FETCHMAIL=1
+    --env FETCHMAIL_PARALLEL=1
+    --env ENABLE_POSTGREY=1
+    --env ENABLE_SASLAUTHD=1
+    --env ENABLE_SRS=1
+    --env SMTP_ONLY=0
+    # Required workaround for some environments when using ENABLE_SRS=1:
+    # PR 2730: https://github.com/docker-mailserver/docker-mailserver/commit/672e9cf19a3bb1da309e8cea6ee728e58f905366
+    --ulimit "nofile=$(ulimit -Sn):$(ulimit -Hn)"
+  )
+  init_with_defaults
+  mv "${TEST_TMP_CONFIG}/fetchmail/fetchmail.cf" "${TEST_TMP_CONFIG}/fetchmail.cf"
+  # Average time: 6 seconds
+  common_container_setup 'CONTAINER_ARGS_ENV_CUSTOM'
+
+  local ENABLED_PROCESS_LIST=(
+    "${CORE_PROCESS_LIST[@]}"
+    "${ENV_PROCESS_LIST[@]}"
+  )
+
+  for PROCESS in "${ENABLED_PROCESS_LIST[@]}"
+  do
+    _should_restart_when_killed "${PROCESS}"
+  done
+
+  # By this point the fetchmail processes have been verified to exist and restart,
+  # For FETCHMAIL_PARALLEL=1 coverage, match full commandline for COUNTER values:
+  pgrep --full 'fetchmail-1.rc'
+  assert_success
+  pgrep --full 'fetchmail-2.rc'
+  assert_success
+}
+
+# Split into separate test case for the benefit of minimizing CPU + RAM overhead of clamd.
+# NOTE: Does not reduce test time of previous test case. Adds 10 seconds to test time.
+@test "(enabled ENV) should restart clamd when killed" {
+  export CONTAINER_NAME=${CONTAINER3_NAME}
+  local CONTAINER_ARGS_ENV_CUSTOM=(
+    --env ENABLE_CLAMAV=1
+  )
+  init_with_defaults
+  common_container_setup 'CONTAINER_ARGS_ENV_CUSTOM'
+
+  _should_restart_when_killed 'clamd'
+}
+
+function _should_restart_when_killed() {
+  local PROCESS=${1}
+  local MIN_PROCESS_AGE=4
+
+  # Wait until process has been running for at least MIN_PROCESS_AGE:
+  # (this allows us to more confidently check the process was restarted)
+  run_until_success_or_timeout 30 _check_if_process_is_running "${PROCESS}" "${MIN_PROCESS_AGE}"
+  # NOTE: refute_output doesn't have output to compare to when a run failure is due to a timeout
+  assert_success
+  assert_output --partial "${PROCESS}"
+
+  # Should kill the process successfully:
+  # (which should then get restarted by supervisord)
+  _run_in_container pkill --echo "${PROCESS}"
+  assert_output --partial "${PROCESS}"
+  assert_success
+
+  # Wait until original process is not running:
+  # (Ignore restarted process by filtering with MIN_PROCESS_AGE, --fatal-test with `false` stops polling on error):
+  run repeat_until_success_or_timeout --fatal-test "_check_if_process_is_running ${PROCESS} ${MIN_PROCESS_AGE}" 30 false
+  assert_output --partial "'${PROCESS}' is not running"
+  assert_failure
+
+  # Should be running:
+  # (poll as some processes a slower to restart, such as those run by wrapper scripts adding delay via sleep)
+  run_until_success_or_timeout 30 _check_if_process_is_running "${PROCESS}"
+  assert_success
+  assert_output --partial "${PROCESS}"
+}
+
+# NOTE: CONTAINER_NAME is implicit; it should have be set prior to calling.
+function _check_if_process_is_running() {
+  local PROCESS=${1}
+  local MIN_SECS_RUNNING
+  [[ -n ${2} ]] && MIN_SECS_RUNNING="--older ${2}"
+
+  local IS_RUNNING=$(docker exec "${CONTAINER_NAME}" pgrep --list-full ${MIN_SECS_RUNNING} "${PROCESS}")
+
+  # When no matches are found, nothing is returned. Provide something we can assert on (helpful for debugging):
+  if [[ ! ${IS_RUNNING} =~ "${PROCESS}" ]]
+  then
+    echo "'${PROCESS}' is not running"
+    return 1
+  fi
+
+  # Original output (if any) for assertions
+  echo "${IS_RUNNING}"
+}
diff --git a/test/tests/serial/mail_fetchmail.bats b/test/tests/serial/mail_fetchmail.bats
index 77c468f2..d75a3113 100644
--- a/test/tests/serial/mail_fetchmail.bats
+++ b/test/tests/serial/mail_fetchmail.bats
@@ -1,51 +1,77 @@
-load "${REPOSITORY_ROOT}/test/test_helper/common"
+load "${REPOSITORY_ROOT}/test/helper/common"
+load "${REPOSITORY_ROOT}/test/helper/setup"
+
+BATS_TEST_NAME_PREFIX='[Fetchmail] '
+CONTAINER1_NAME='dms-test_fetchmail'
+CONTAINER2_NAME='dms-test_fetchmail_parallel'
 
 function setup_file() {
-  local PRIVATE_CONFIG
-  PRIVATE_CONFIG=$(duplicate_config_for_container .)
+  export CONTAINER_NAME
 
-  docker run -d --name mail_fetchmail \
-    -v "${PRIVATE_CONFIG}":/tmp/docker-mailserver \
-    -v "$(pwd)/test/test-files":/tmp/docker-mailserver-test:ro \
-    -e ENABLE_FETCHMAIL=1 \
-    --cap-add=NET_ADMIN \
-    -h mail.my-domain.com -t "${NAME}"
+  CONTAINER_NAME=${CONTAINER1_NAME}
+  local CUSTOM_SETUP_ARGUMENTS=(
+    --env ENABLE_FETCHMAIL=1
+  )
+  init_with_defaults
+  mv "${TEST_TMP_CONFIG}/fetchmail/fetchmail.cf" "${TEST_TMP_CONFIG}/fetchmail.cf"
+  common_container_setup 'CUSTOM_SETUP_ARGUMENTS'
 
-  wait_for_finished_setup_in_container mail_fetchmail
+  CONTAINER_NAME=${CONTAINER2_NAME}
+  local CUSTOM_SETUP_ARGUMENTS=(
+    --env ENABLE_FETCHMAIL=1
+    --env FETCHMAIL_PARALLEL=1
+  )
+  init_with_defaults
+  mv "${TEST_TMP_CONFIG}/fetchmail/fetchmail.cf" "${TEST_TMP_CONFIG}/fetchmail.cf"
+  common_container_setup 'CUSTOM_SETUP_ARGUMENTS'
 }
 
 function teardown_file() {
-  docker rm -f mail_fetchmail
+  docker rm -f "${CONTAINER1_NAME}" "${CONTAINER2_NAME}"
 }
 
-#
-# processes
-#
+# ENV `FETCHMAIL=1` runs `setup-stack.sh:_setup_fetchmail()`:
+@test "(ENV ENABLE_FETCHMAIL=1) should configure /etc/fetchmailrc with fetchmail.cf contents" {
+  export CONTAINER_NAME=${CONTAINER1_NAME}
+  # /etc/fetchmailrc was created with general options copied from /etc/fetchmailrc_general:
+  _should_have_in_config 'set syslog' /etc/fetchmailrc
+  # fetchmail.cf content is appended into /etc/fetchmailrc:
+  # NOTE: FQDN value ends with a dot intentionally to avoid misleading DNS response:
+  # https://github.com/docker-mailserver/docker-mailserver/pull/1324
+  _should_have_in_config 'pop3.third-party.test.' /etc/fetchmailrc
+}
 
-@test "checking process: fetchmail (fetchmail server enabled)" {
-  run docker exec mail_fetchmail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/bin/fetchmail'"
+# ENV `FETCHMAIL=1` runs `setup-stack.sh:_setup_fetchmail_parallel()`:
+# fetchmail.cf should be parsed and split into multiple separate fetchmail configs:
+# NOTE: Parallel fetchmail instances are checked in the `process-check-restart.bats` test.
+@test "(ENV FETCHMAIL_PARALLEL=1) should create config fetchmail-1.rc" {
+  export CONTAINER_NAME=${CONTAINER2_NAME}
+
+  _should_have_in_config     'set syslog'                /etc/fetchmailrc.d/fetchmail-1.rc
+  _should_have_in_config     'pop3.third-party.test.'    /etc/fetchmailrc.d/fetchmail-1.rc
+  _should_not_have_in_config 'imap.remote-service.test.' /etc/fetchmailrc.d/fetchmail-1.rc
+}
+
+@test "(ENV FETCHMAIL_PARALLEL=1) should create config fetchmail-2.rc" {
+  export CONTAINER_NAME=${CONTAINER2_NAME}
+
+  _should_have_in_config     'set syslog'                /etc/fetchmailrc.d/fetchmail-2.rc
+  _should_have_in_config     'imap.remote-service.test.' /etc/fetchmailrc.d/fetchmail-2.rc
+  _should_not_have_in_config 'pop3.third-party.test. '   /etc/fetchmailrc.d/fetchmail-2.rc
+}
+
+function _should_have_in_config() {
+  local MATCH_CONTENT=$1
+  local MATCH_IN_FILE=$2
+
+  _run_in_container grep -F "${MATCH_CONTENT}" "${MATCH_IN_FILE}"
   assert_success
 }
 
-#
-# fetchmail
-#
+function _should_not_have_in_config() {
+  local MATCH_CONTENT=$1
+  local MATCH_IN_FILE=$2
 
-@test "checking fetchmail: gerneral options in fetchmailrc are loaded" {
-  run docker exec mail_fetchmail grep 'set syslog' /etc/fetchmailrc
-  assert_success
-}
-
-@test "checking fetchmail: fetchmail.cf is loaded" {
-  run docker exec mail_fetchmail grep 'pop3.example.com' /etc/fetchmailrc
-  assert_success
-}
-
-#
-# supervisor
-#
-
-@test "checking restart of process: fetchmail" {
-  run docker exec mail_fetchmail /bin/bash -c "pkill fetchmail && sleep 10 && ps aux --forest | grep -v grep | grep '/usr/bin/fetchmail'"
-  assert_success
+  _run_in_container grep -F "${MATCH_CONTENT}" "${MATCH_IN_FILE}"
+  assert_failure
 }
diff --git a/test/tests/serial/mail_fetchmail_parallel.bats b/test/tests/serial/mail_fetchmail_parallel.bats
deleted file mode 100644
index b1792de2..00000000
--- a/test/tests/serial/mail_fetchmail_parallel.bats
+++ /dev/null
@@ -1,82 +0,0 @@
-load "${REPOSITORY_ROOT}/test/test_helper/common"
-
-function setup_file() {
-  local PRIVATE_CONFIG
-  PRIVATE_CONFIG=$(duplicate_config_for_container .)
-
-  docker run -d --name mail_fetchmail_parallel \
-    -v "${PRIVATE_CONFIG}":/tmp/docker-mailserver \
-    -v "$(pwd)/test/test-files":/tmp/docker-mailserver-test:ro \
-    -e ENABLE_FETCHMAIL=1 \
-    -e FETCHMAIL_PARALLEL=1 \
-    --cap-add=NET_ADMIN \
-    -h mail.my-domain.com -t "${NAME}"
-
-  wait_for_finished_setup_in_container mail_fetchmail_parallel
-}
-
-function teardown_file() {
-  docker rm -f mail_fetchmail_parallel
-}
-
-#
-# processes
-#
-
-@test "checking process: fetchmail 1 (fetchmail server enabled)" {
-  run docker exec mail_fetchmail_parallel /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/bin/fetchmail -f /etc/fetchmailrc.d/fetchmail-1.rc'"
-  assert_success
-}
-
-@test "checking process: fetchmail 2 (fetchmail server enabled)" {
-  run docker exec mail_fetchmail_parallel /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/bin/fetchmail -f /etc/fetchmailrc.d/fetchmail-2.rc'"
-  assert_success
-}
-
-#
-# fetchmail
-#
-
-@test "checking fetchmail: gerneral options in fetchmail-1.rc are loaded" {
-  run docker exec mail_fetchmail_parallel grep 'set syslog' /etc/fetchmailrc.d/fetchmail-1.rc
-  assert_success
-}
-
-@test "checking fetchmail: gerneral options in fetchmail-2.rc are loaded" {
-  run docker exec mail_fetchmail_parallel grep 'set syslog' /etc/fetchmailrc.d/fetchmail-2.rc
-  assert_success
-}
-
-@test "checking fetchmail: fetchmail-1.rc is loaded with pop3.example.com" {
-  run docker exec mail_fetchmail_parallel grep 'pop3.example.com' /etc/fetchmailrc.d/fetchmail-1.rc
-  assert_success
-}
-
-@test "checking fetchmail: fetchmail-1.rc is loaded without pop3-2.example.com" {
-  run docker exec mail_fetchmail_parallel grep 'pop3-2.example.com' /etc/fetchmailrc.d/fetchmail-1.rc
-  assert_failure
-}
-
-@test "checking fetchmail: fetchmail-2.rc is loaded without pop3.example.com" {
-  run docker exec mail_fetchmail_parallel grep 'pop3.example.com' /etc/fetchmailrc.d/fetchmail-2.rc
-  assert_failure
-}
-
-@test "checking fetchmail: fetchmail-2.rc is loaded with pop3-2.example.com" {
-  run docker exec mail_fetchmail_parallel grep 'pop3-2.example.com' /etc/fetchmailrc.d/fetchmail-2.rc
-  assert_success
-}
-
-#
-# supervisor
-#
-
-@test "checking restart of process: fetchmail-1" {
-  run docker exec mail_fetchmail_parallel /bin/bash -c "pkill fetchmail && sleep 10 && ps aux --forest | grep -v grep | grep '/usr/bin/fetchmail -f /etc/fetchmailrc.d/fetchmail-1.rc'"
-  assert_success
-}
-
-@test "checking restart of process: fetchmail-2" {
-  run docker exec mail_fetchmail_parallel /bin/bash -c "pkill fetchmail && sleep 10 && ps aux --forest | grep -v grep | grep '/usr/bin/fetchmail -f /etc/fetchmailrc.d/fetchmail-2.rc'"
-  assert_success
-}
diff --git a/test/tests/serial/mail_smtponly.bats b/test/tests/serial/mail_smtponly.bats
index ebd46368..9549ace5 100644
--- a/test/tests/serial/mail_smtponly.bats
+++ b/test/tests/serial/mail_smtponly.bats
@@ -29,11 +29,6 @@ function teardown_file() {
 # imap
 #
 
-@test "checking process: dovecot imaplogin (disabled using SMTP_ONLY)" {
-  run docker exec mail_smtponly /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/sbin/dovecot'"
-  assert_failure
-}
-
 @test "checking configuration: dovecot quota absent in postconf (disabled using SMTP_ONLY)" {
   run docker exec mail_smtponly /bin/bash -c "postconf | grep 'check_policy_service inet:localhost:65265'"
   assert_failure
diff --git a/test/tests/serial/mail_with_ldap.bats b/test/tests/serial/mail_with_ldap.bats
index 1f206c24..438c9a9e 100644
--- a/test/tests/serial/mail_with_ldap.bats
+++ b/test/tests/serial/mail_with_ldap.bats
@@ -64,13 +64,6 @@ function teardown_file() {
   docker network rm "${DMS_TEST_NETWORK}"
 }
 
-# processes
-
-@test "checking process: saslauthd (saslauthd server enabled)" {
-  run docker exec mail_with_ldap /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/sbin/saslauthd'"
-  assert_success
-}
-
 # postfix
 @test "checking postfix: ldap lookup works correctly" {
   run docker exec mail_with_ldap /bin/sh -c "postmap -q some.user@${FQDN_LOCALHOST_A} ldap:/etc/postfix/ldap-users.cf"
@@ -247,12 +240,3 @@ function teardown_file() {
   run docker exec mail_with_ldap grep "weekly" /etc/logrotate.d/maillog
   assert_success
 }
-
-#
-# supervisor
-#
-
-@test "checking restart of process: saslauthd (saslauthd server enabled)" {
-  run docker exec mail_with_ldap /bin/bash -c "pkill saslauthd && sleep 10 && ps aux --forest | grep -v grep | grep '/usr/sbin/saslauthd'"
-  assert_success
-}
diff --git a/test/tests/serial/mail_with_postgrey_disabled_by_default.bats b/test/tests/serial/mail_with_postgrey_disabled_by_default.bats
deleted file mode 100644
index 82041ccc..00000000
--- a/test/tests/serial/mail_with_postgrey_disabled_by_default.bats
+++ /dev/null
@@ -1,23 +0,0 @@
-load "${REPOSITORY_ROOT}/test/test_helper/common"
-
-function setup() {
-  local PRIVATE_CONFIG
-  PRIVATE_CONFIG=$(duplicate_config_for_container .)
-
-  CONTAINER=$(docker run -d \
-    -v "${PRIVATE_CONFIG}":/tmp/docker-mailserver \
-    -v "$(pwd)/test/test-files":/tmp/docker-mailserver-test:ro \
-    -h mail.my-domain.com -t "${NAME}")
-
-  # using postfix availability as start indicator, this might be insufficient for postgrey
-  wait_for_smtp_port_in_container "${CONTAINER}"
-}
-
-function teardown() {
-  docker rm -f "${CONTAINER}"
-}
-
-@test "checking process: postgrey (disabled in default configuration)" {
-  run docker exec "${CONTAINER}" /bin/bash -c "ps aux --forest | grep -v grep | grep 'postgrey'"
-  assert_failure
-}
diff --git a/test/tests/serial/setup-cli.bats b/test/tests/serial/setup-cli.bats
index 851f05ad..f07afa01 100644
--- a/test/tests/serial/setup-cli.bats
+++ b/test/tests/serial/setup-cli.bats
@@ -11,6 +11,7 @@ CONTAINER_NAME='dms-test_setup-cli'
 function setup_file() {
   # Initializes common default vars to prepare a DMS container with:
   init_with_defaults
+  mv "${TEST_TMP_CONFIG}/fetchmail/fetchmail.cf" "${TEST_TMP_CONFIG}/fetchmail.cf"
 
   # Creates and starts the container with additional ENV needed:
   # `LOG_LEVEL=debug` required for using `wait_until_change_detection_event_completes()`
diff --git a/test/tests/serial/tests.bats b/test/tests/serial/tests.bats
index a18d2206..ad16225c 100644
--- a/test/tests/serial/tests.bats
+++ b/test/tests/serial/tests.bats
@@ -72,54 +72,10 @@ teardown_file() {
   assert_success
 }
 
-#
-# processes
-#
-
-@test "checking process: postfix" {
-  run docker exec mail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/lib/postfix/sbin/master'"
-  assert_success
-}
-
-@test "checking process: clamd (is not runnning)" {
-  run docker exec mail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/sbin/clamd'"
-  assert_failure
-}
-
-@test "checking process: new" {
-  run docker exec mail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/sbin/amavisd-new'"
-  assert_success
-}
-
-@test "checking process: opendkim" {
-  run docker exec mail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/sbin/opendkim'"
-  assert_success
-}
-
-@test "checking process: opendmarc" {
-  run docker exec mail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/sbin/opendmarc'"
-  assert_success
-}
-
-@test "checking process: fail2ban (disabled in default configuration)" {
-  run docker exec mail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/bin/python3 /usr/bin/fail2ban-server'"
-  assert_failure
-}
-
-@test "checking process: fetchmail (disabled in default configuration)" {
-  run docker exec mail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/bin/fetchmail'"
-  assert_failure
-}
-
 #
 # imap
 #
 
-@test "checking process: dovecot imaplogin (enabled in default configuration)" {
-  run docker exec mail /bin/bash -c "ps aux --forest | grep -v grep | grep '/usr/sbin/dovecot'"
-  assert_success
-}
-
 @test "checking imap: server is ready with STARTTLS" {
   run docker exec mail /bin/bash -c "nc -w 2 0.0.0.0 143 | grep '* OK' | grep 'STARTTLS' | grep 'ready'"
   assert_success
@@ -223,12 +179,6 @@ teardown_file() {
   assert_success
 }
 
-@test "checking SRS: postsrsd running" {
-  run docker exec mail /bin/sh -c "ps aux | grep ^postsrsd"
-  assert_success
-}
-
-
 @test "checking SRS: fallback to hostname is handled correctly" {
   run docker exec mail grep "SRS_DOMAIN=example.test" /etc/default/postsrsd
   assert_success
@@ -712,22 +662,4 @@ EOF
 # supervisor
 #
 
-@test "checking restart of process: postfix" {
-  run docker exec mail /bin/bash -c "pkill master && sleep 10 && ps aux --forest | grep -v grep | grep '/usr/lib/postfix/sbin/master'"
-  assert_success
-}
 
-@test "checking restart of process: amavisd-new" {
-  run docker exec mail /bin/bash -c "pkill amavi && sleep 12 && ps aux --forest | grep -v grep | grep '/usr/sbin/amavisd-new (master)'"
-  assert_success
-}
-
-@test "checking restart of process: opendkim" {
-  run docker exec mail /bin/bash -c "pkill opendkim && sleep 10 && ps aux --forest | grep -v grep | grep '/usr/sbin/opendkim'"
-  assert_success
-}
-
-@test "checking restart of process: opendmarc" {
-  run docker exec mail /bin/bash -c "pkill opendmarc && sleep 10 && ps aux --forest | grep -v grep | grep '/usr/sbin/opendmarc'"
-  assert_success
-}