mirror of
https://github.com/lobaro/restic-backup-docker.git
synced 2026-11-22 01:02:39 +00:00
docs: describe the Python test suite
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
d1a6744b8b
commit
5c03a2e31a
2 files changed
+29
-27
No files matched your search
+2
-2
@@ -3,11 +3,11 @@
|
|||||||
## Unreleased
|
## Unreleased
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
* Test suites in `test/` (compat, features, structure) that run the image as a real container, plus a `Test` GitHub Actions workflow running them on every pull request
|
* Test suites in `tests/` (compat, features, structure) that run the image as a real container, plus a `Test` GitHub Actions workflow running them on every pull request. The suites are written in Python (pytest) and run via `mise install` and `task test`; `task test:sabotage` proves they detect a broken backup
|
||||||
* `.gitattributes` pinning shell scripts, the Dockerfile and YAML files to LF line endings. With a Windows checkout (`core.autocrlf=true`) the scripts were checked out with CRLF, which made the built image fail to start with `exec /entry.sh: no such file or directory`
|
* `.gitattributes` pinning shell scripts, the Dockerfile and YAML files to LF line endings. With a Windows checkout (`core.autocrlf=true`) the scripts were checked out with CRLF, which made the built image fail to start with `exec /entry.sh: no such file or directory`
|
||||||
|
|
||||||
### Removed
|
### Removed
|
||||||
* `config-broken.yml` and `docker-compose.test.broken.yml`, replaced by `test/structure.yml` and the test workflow
|
* `config-broken.yml` and `docker-compose.test.broken.yml`, replaced by `tests/structure.yml` and the test workflow
|
||||||
|
|
||||||
## v1.4.2
|
## v1.4.2
|
||||||
### Removed
|
### Removed
|
||||||
|
|||||||
@@ -28,43 +28,45 @@ When behaviour of the image changes (Features, Bugfixes, Changes in the API) ple
|
|||||||
|
|
||||||
## Tests
|
## Tests
|
||||||
|
|
||||||
The test suites build the image and exercise it as a real container. They need
|
The tests build the image and exercise it as a real container. Tools are pinned
|
||||||
`bash` and `docker`, nothing else:
|
in `mise.toml` (Python, [uv](https://docs.astral.sh/uv/),
|
||||||
|
[Task](https://taskfile.dev/), container-structure-test), Python dependencies in
|
||||||
|
`uv.lock`. You need [mise](https://mise.jdx.dev/) and Docker, nothing else:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
bash test/run.sh # build the image and run every suite
|
mise install # install the pinned tools
|
||||||
bash test/run.sh --no-build # reuse an already built image
|
task test # build the image and run every suite
|
||||||
bash test/run.sh --suite compat # run a single suite
|
task test:sabotage # self test, see below
|
||||||
|
task lint # ruff check and format check of tests/
|
||||||
```
|
```
|
||||||
|
|
||||||
They run unchanged on Linux, macOS and Windows/Git-Bash. On Git-Bash the
|
Single suites: `task test:structure`, `task test:compat`, `task test:features`.
|
||||||
scripts set `MSYS_NO_PATHCONV=1` and `MSYS2_ARG_CONV_EXCL='*'` themselves,
|
Arguments after `--` go to pytest, e.g. `task test:compat -- -k hooks`. Set
|
||||||
because MSYS would otherwise rewrite container paths such as
|
`IMAGE` to test an already built image instead of `restic-backup-docker:test`.
|
||||||
`/usr/bin/backup` into `C:/Program Files/Git/usr/bin/backup` before `docker`
|
|
||||||
sees them. Nothing has to be exported by hand.
|
|
||||||
|
|
||||||
There are three suites:
|
The suites live in `tests/`:
|
||||||
|
|
||||||
* **compat** (`test/compat.sh`) - backup, restore, forget, check, hooks and
|
* **compat** (`tests/test_compat.py`) - backup, restore, forget, check, hooks and
|
||||||
failure handling against the default configuration. These tests describe the
|
failure handling against the default configuration. These tests describe the
|
||||||
behaviour users rely on. A change that turns one of them red is a breaking
|
behaviour users rely on. A change that turns one of them red is a breaking
|
||||||
change.
|
change.
|
||||||
* **features** (`test/features.sh`) - tests for optional features. Each block
|
* **features** (`tests/test_features.py`) - tests for optional features. Each
|
||||||
is guarded by a detection check against the built image, so a feature that is
|
test is marked `@pytest.mark.feature("<gate>")` and guarded by a detection
|
||||||
not part of the image is reported as `SKIP` instead of failing. Pull requests
|
check against the built image, so a feature that is not part of the image is
|
||||||
that add such a feature are therefore covered as soon as they are rebased,
|
reported as skipped instead of failing. Pull requests that add such a feature
|
||||||
without the author having to write the basic tests. After merging a feature,
|
are therefore covered as soon as they are rebased, without the author having
|
||||||
add its gate name to `test/required-features.txt` so that a missing feature
|
to write the basic tests. After merging a feature, add its gate name to
|
||||||
turns into a failure.
|
`tests/required-features.txt` so that a missing feature turns into a failure.
|
||||||
* **structure** (`test/structure.yml`) - image layout checks via
|
* **structure** (`tests/structure.yml`) - image layout checks via
|
||||||
[container-structure-test](https://github.com/GoogleContainerTools/container-structure-test).
|
[container-structure-test](https://github.com/GoogleContainerTools/container-structure-test).
|
||||||
When the tool is not installed, its official container image is used instead.
|
`tests/structure.py` runs it and removes the containers and anonymous volumes
|
||||||
|
it leaves behind.
|
||||||
|
|
||||||
`bash test/run.sh --sabotage` is a self test: it breaks the backup call on a
|
`task test:sabotage` is a self test: it breaks the backup call on a throwaway
|
||||||
throwaway copy of the tree and expects the compat suite to fail. CI runs it so
|
copy of the tree and expects the compat tests to fail. CI runs it so that the
|
||||||
that the suites cannot be watered down into always passing.
|
tests cannot be watered down into always passing.
|
||||||
|
|
||||||
When you add a feature, please add a block to `test/features.sh` for it.
|
When you add a feature, please add tests with a `feature` marker for it.
|
||||||
|
|
||||||
## Hooks
|
## Hooks
|
||||||
|
|
||||||
|
|||||||
Reference in new issue
Block a user