docker-mailserver/test
Brennan Kinney 672e9cf19a
tests: Ensure excessive FD limits are avoided (#2730)
* tests: Ensure excessive FD limits are avoided

Processes that run as daemons (`postsrsd` and `fail2ban-server`) initialize by closing all FDs (File Descriptors).

This behaviour queries that maximum limit and iterates through the entire range even if only a few FDs are open. In some environments (Docker, limit configured by distro) this can be a range exceeding 1 billion (from kernel default of 1024 soft, 4096 hard), causing an 8 minute delay with heavy CPU activity.

`postsrsd` has since been updated to use `close_range()` syscall, and `fail2ban` will now iterate through `/proc/self/fd` (open FDs) which should resolve the performance hit. Until those updates reach our Docker image, we need to workaround it with `--ulimit` option.

NOTE: If `docker.service` on a distro sets `LimitNOFILE=` to approx 1 million or lower, it should not be an issue. On distros such as Fedora 36, it is `LimitNOFILE=infinity` (approx 1 billion) that causes excessive delays.

* chore: Use Docker host limits instead

Typically on modern distros with systemd, this should equate to 1024 (soft) and 512K (hard) limits. A distro may override the built-in global defaults systemd sets via setting `DefaultLimitNOFILE=` in `/etc/systemd/user.conf` and `/etc/systemd/system.conf`.

* tests(fix): Better prevent non-deterministic failures

- `no_containers.bats` tests the external script `setup.sh` (without `-c`). It's expected that no existing DMS container is running  - otherwise it may attempt to use that container and fail. Detect this and fail early via `setup_file()` step.

- `mail_hostname.bats` had a odd timing failure with teardown due to the last tests bringing the containers down earlier (`docker stop` paired with the `docker run --rm`). Adding a moment of delay via `sleep` helps avoid that false positive scenario.
2022-08-23 11:24:23 +12:00
..
bats@410dd229a5 tests: Update submodules for bats (#2715) 2022-08-12 11:09:17 +12:00
config firewall: replace iptables with nftables (#2505) 2022-04-05 15:13:59 +02:00
docker-openldap formatting files according to standard (#1619) 2020-09-24 14:54:21 +02:00
linting scripts: revised linting script (#2737) 2022-08-22 16:22:46 +02:00
test-files tests(chore): Adjust supported FQDNs in test certs + add wildcard and acme.json configs (#2284) 2021-11-03 20:25:25 +00:00
test_helper chore: addmailuser - Remove delaying completion until /var/mail is ready (#2729) 2022-08-23 10:15:06 +12:00
default_relay_host.bats chore: Remove redundant capability SYS_PTRACE (#2624) 2022-06-07 01:20:13 +12:00
dovecot_inet_protocol.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
helper-functions.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_changedetector.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_disabled_clamav_spamassassin.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_dnsbl.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_fail2ban.bats tests: Ensure excessive FD limits are avoided (#2730) 2022-08-23 11:24:23 +12:00
mail_fetchmail.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_fetchmail_parallel.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_hostname.bats tests: Ensure excessive FD limits are avoided (#2730) 2022-08-23 11:24:23 +12:00
mail_lmtp_ip.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_pop3.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_postfix_inet.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_postscreen.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_privacy.bats chore: Remove redundant capability SYS_PTRACE (#2624) 2022-06-07 01:20:13 +12:00
mail_quotas_disabled.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_smtponly.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_spam_bounced.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_spam_junk_folder.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_special_use_folders.bats chore: Remove redundant capability SYS_PTRACE (#2624) 2022-06-07 01:20:13 +12:00
mail_ssl_letsencrypt.bats fix: typo in changedetector ready message (#2663) 2022-06-28 11:02:43 +02:00
mail_ssl_manual.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_time.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_tls_dhparams.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_undef_spam_subject.bats tests: Ensure excessive FD limits are avoided (#2730) 2022-08-23 11:24:23 +12:00
mail_with_imap.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_with_ldap.bats scripts: rework environment variables setup (#2716) 2022-08-22 08:31:32 +02:00
mail_with_mdbox.bats chore: Remove redundant capability SYS_PTRACE (#2624) 2022-06-07 01:20:13 +12:00
mail_with_postgrey.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_with_postgrey_disabled_by_default.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
mail_with_relays.bats chore: Remove redundant capability SYS_PTRACE (#2624) 2022-06-07 01:20:13 +12:00
mail_with_sdbox.bats chore: Remove redundant capability SYS_PTRACE (#2624) 2022-06-07 01:20:13 +12:00
no_container.bats tests: Ensure excessive FD limits are avoided (#2730) 2022-08-23 11:24:23 +12:00
open_dkim.bats chore: Remove redundant capability SYS_PTRACE (#2624) 2022-06-07 01:20:13 +12:00
permit_docker.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
security_tls_cipherlists.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
sedfile.bats tests(fix): Adjust for local testing conditions (#2606) 2022-05-30 12:53:30 +12:00
setup-cli.bats chore: addmailuser - Remove delaying completion until /var/mail is ready (#2729) 2022-08-23 10:15:06 +12:00
test_helper.bats chore(housekeeping): Create helpers/change-detection.sh (#2610) 2022-06-05 11:59:54 +12:00
tests.bats tests: Ensure excessive FD limits are avoided (#2730) 2022-08-23 11:24:23 +12:00