* ci(fix): Temporarily avoid specifying `provenance`
As the test workflow does not use the `docker-container` buildx driver, it uses the Docker Engine bundled BuildKit version which until v23 release does not support attestations.
Likewise the current buildx version in CI is `0.10.0` which does not respect `--provenance false`, the presence of the option appears to trigger a BuildKit version compatibility check and fail early before it considers the value of the option.
* chore: Use buildx `docker-container` driver instead
An alternative solution to omitting `provenance: false` (_not supported by buildx 0.10.0 with default `docker` driver when Docker Engine bundles BuildKit less than 0.10.0, which is the case prior to the Docker Engine v23 release_).
This approach provides more consistency with the build and publish workflows by using the same buildx `docker-container` driver (_and thus newer BuildKit, enabling support for `provenance: false`_).
* chore: Revise test workflow inline docs
Buildx `docker-container` driver is not needed here, but it does seem like it improves cache-hit ratio when building from the retrieved build cache (from the earlier build workflow). Possibly due to building with the same BuildKit version.
* chore: Remove `backup` target from Makefile
- The `backup` target is no longer serving any value to us. It was made redundant with changes added in Oct 2020.
- `clean` target inline docs revised.
- `.gitignore` remove test lines that are no longer valid.
* chore: Parallel test target split to multi-line
* tests(fix): Test `setup.sh` with temporary config dir
The `no_containers.bats` test has many redundant test cases already covered by `setup-cli`. They're basically identical. Removed all but one.
This removes some config dirs that were being explicitly created instead of using the test helper to generate a directory that can be used to test the `-p` option instead.
* ci: Ensure tests are run when `Makefile` is modified
These updates support running tests that have been relocated into `serial` and `parallel/set*` directories.
- `make tests` now calls the two make targets beneath it. The only difference is that `serial` continues the "1 test at a time" approach used prior to this PR, while the `parallel` target increases the `--jobs` arg to run multiple tests concurrently (_configured by `PARALLEL_JOBS`_).
- The `test/%` target leverages Bash syntax magic to ease running single tests without providing the exact path.
- This syntax also supports providing multiple test names (eg: `make test/clamav,template`) to run.
- `**` (globstar) allows for future improvements that can group multiple test files into sub-directories by their scope (eg: anti-spam, ssl, etc).
---
chore: Add `shopt -s globstar` to other targets
I realized that other targets should have this as well in case it is not set.
It is better to be more explicit here than to have weird errors due to `**` not expanding properly.
---
fix(Makefile): Add back `.PHONY` targets
I encountered `make` telling me the target was already up-to-date, which of course is nonsense.
I therefore added back the `.PHONY` targets to ensure tests are always run.
---
docs: Added instructions for running a single test
See https://github.com/docker-mailserver/docker-mailserver/pull/2857/files#r1008582760
This new script is a clean way of handling the installation of packages.
I think the huge `RUN` command in `Dockerfile` was hard to read and
maintain.
Using a script is a non-issue, as the image is rebuilt whenever the
script is touched.
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
The build arguments `VCS_REF` and `VCS_VER` were renamed and given
proper values according to their names.
1. `VCS_REVISION` holds the current SHA sum of the (git) HEAD pointer
2. `VCS_VERSION` now holds the contents of the `VERSION` file, i.e. a
semver version tag (one can now inspect the image and find a proper
version tag in the `org.opencontainers.image.version` label)
The build arguments were given defaults in order to allow the
`generic_build` and `generic_test` workflows to omit them (as they are
not need there anyways). When publishing images, this is fina as the
cache will rebuild almost all of the image except the last few layers
which are `LABEL`s anyways.
Mew re-usable workflows are introduced to handle building, testing and publishing the container
image in a uniform and easy way. Now, the `scheduled_builds`, `default_on_push`
and a part of the `test_merge_requests` workflow can use the same code
for building, testing and publishing the container images. This is DRY.
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* ci: Cache builds by splitting into two jobs
For the cache to work properly, we need to derive a cache key from the build context (files that affect the Dockerfile build) instead of the cache key changing by commit SHA.
We also need to avoid a test suite failure from preventing the caching of a build, thus splitting into separate jobs.
This first attempt used `upload-artifact` and `download-artifact` to transfer the built image, but it has quite a bit of overhead and prevented multi-platform build (without complicating the workflow further).
* ci: Transfer to dependent job via cache only
While `download-artifact` + `docker load` is a little faster than rebuilding the image from cached layers, `upload-artifact` takes about 2 minutes to upload the AMD64 (330MB) tar image export (likely due to compression during upload?).
The `actions/cache` approach however does not incur that hit and is very quick (<10 secs) to complete it's post upload work. The dependent job still gets a cache-hit, and the build job is able to properly support multi-platform builds.
Added additional notes about timing and size of including ARM builds.
* ci: Move Dockerfile ARG to end of build
When the ARG changes due to commit SHA, it invalidates all cache due to the LABEL layers at the start. Then any RUN layers implicitly invalidate, even when the ARG is not used.
Introduced basic multi-stage build, and relocated the container config / metadata to the end of the build. This avoids invalidating expensive caching layers (size and build time) needlessly.
The new version uses our `log.sh` helper to simplify logging
significantly. Moreover, the script was adjusted to the current style
and the GitHub workflow was streamlined. The workflow is ot providing
the version anymore (which was useless anyway), and has been compacted.
* docs(deps): bump mkdocs-material to v8.2.1
* feat(docs): enable mermaid integration
Configuration based on https://squidfunk.github.io/mkdocs-material/reference/diagrams/?h=mermaid#configuration
* fix: allow yaml value mapping
* chore: Adopt mkdocs-material mermaid integration support
Supported by the docs generator now, we no longer need to rely on external image generator or live editor link (both relied on large base64 encoding of mermaid markup). SVG will be rendered by docs now, although a little different style (can be fixed with custom CSS).
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
The prepare workflow runs in an untrusted context already and thus should not have anything worthwhile to exploit.
However care should still be taken to avoid interpolating expressions into shell scripts directly that is data a user can control the value of. Especially to avoid any maintainer referencing an existing workflow from copying a risky snippet unaware of different security contexts for workflows.
In this case, as per Github Documentation and referenced issue comment, the PR title is user controllable data, which if directly interpolated into the shell script being run (as it previously was), allows for injecting commands to execute.
* docs(deps): bump mkdocs-material to 8.0.2
* docs(deps): bump mkdocs-material to 8.0.3
* chore: add default version of docs
* feat: add version warning
* fix: remove version warning
* docs(deps): bump mkdocs-material to 8.0.5
* added code annotation feature
We can introduce new annotation with new PRs in the future. I'd advise against overhauling all code blocks with this feature in this PR - this PR should just introduce the feature.
* docs(deps): bump mkdocs-material to 8.1.0
* fix: remove unnecessary default value
re-add if version warning gets a thing in the future. See https://github.com/docker-mailserver/docker-mailserver/pull/2311#issuecomment-991805830
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
Using `set -ex` will exit the script as soon as a non-zero exit code is returned, such as when the docker image fails building the docs due to `build --strict` catching broken links. This also removes the need for `|| exit` when changing directory.
This seems fine for a small script, but AFAIK an alternative fix is just adding `|| exit` to the end of the `docker run` command too? There appears to be advice [against adopting `-e` carelessly](http://mywiki.wooledge.org/BashFAQ/105), while others [encourage `-e`](http://redsymbol.net/articles/unofficial-bash-strict-mode/). I know that several maintainers here have preference towards `set -e` so I've kept the original PR solution.
Additionally:
- `-x` is used to improve command visibility when reviewing the workflow log output.
- `--name` isn't necessary, but was part of the original PR.
- I've chosen not to include `-o pipefail`, only because no pipes are used in this script.
* docs(fix): Fix broken links
* ci(docs): Added inline docs
Extra documentation context for maintainers to quickly grok what's going on.
* chore(docs): Minor typo fix by wernerfred
Added from their related PR by request.
* ci: Fix lint check status update
The lint workflow is not important for this PR, but a fixed requirement to pass for merging.
As this workflow is triggered by `schedule` or `workflow_dispatch`, it will not trigger other events such as `pull_request` for other workflows to respond to.
Since the linting workflow is not important for this type of PR, we can pretend it was "skipped" and set the check status to "success". This is simpler than running the actual Lint workflow redundantly.
* ci: Remove workflow_run approach
This didn't work out, reverting.
This should resolve the issue of the lint workflow not being triggered by PRs opened via another workflow (`contributors.yml`).
This workflow will be triggered after the dependent workflow completes (regardless of status).
