Files
restic-backup-docker/tests/conftest.py
Tobias KaupatandClaude Sonnet 5.5 be7d426f78 test: move container test suites to pytest (#112)
* test: scaffold pytest suite with mise, uv and Task

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* ci: run the pytest suite via mise and task

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix: make test cleanup survive repeated SIGINT

A group Ctrl-C reaches pytest a second time via task and uv while the
session fixture tears down, aborting the volume cleanup. Ignore SIGINT
and SIGTERM during cleanup, remove volumes in finally, and tolerate
already-removed objects.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix: pin Python 3.13 for uv without mise

Add .python-version (read by mise and uv) and cap requires-python at
<3.14 so bare uv uses 3.13 or fails instead of picking 3.14.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix: clean up test containers on SIGTERM

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* test: port compat tests to pytest

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* test: assert the success message first in the corruption test

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* test: assert the empty-tag snapshot carries no tags

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* test: remove anonymous volumes with short-lived test containers

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* test: port feature-gated tests to pytest

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* test: port sabotage self-test to task test:sabotage

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix: create parent directories in put_file

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* test: move structure.yml to tests and remove cst volume leak

container-structure-test left one anonymous volume per test container.
tests/structure.py runs it with --save and removes the new containers
with their volumes afterwards.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* test: remove the Bash suite

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* ci: run only the Python suite in the test job

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* docs: describe the Python test suite

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* test: isolate scenario steps, share helpers, use restore_latest, HEAD dir probe

Scenario fixtures record each step; a failing step fails only the tests
that assert on it. running() and raise_interrupt() live in helpers.py.
put_file probes parent directories with a HEAD request instead of a
get_archive stream. stream_run uses restore_latest and asserts its result.
Docstrings no longer refer to the removed Bash suite.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* test: sabotage check rejects errors and malformed reports

Only <failure> of the expected tests counts as red; errors in them and a
malformed JUnit report exit 2 (invalid) instead of passing or exiting 1.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* test: structure wrapper removes only its own containers

Restrict cleanup to containers of the image under test without the
rbd-test-run label and ignore NotFound.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* build: run all test suites and aggregate the exit code

task test no longer stops at the first failing suite.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix: sabotage check treats failed scenario steps as INVALID

StepFailed is no AssertionError any more; sabotage.py counts a JUnit failure
named StepFailed as error, so an expected-red test only counts when its own
assertion fails.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix: hard timeout for container-structure-test

CST_TIMEOUT (default 300 s) kills a hanging run; existing cleanup removes the
containers; exit code 124 with a clear message.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix: shorten sabotage message for lint

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* fix: handle JUnit failures without message in sabotage check

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 20:47:03 +02:00

172 lines
5.9 KiB
Python

"""Shared fixtures and feature gating for the container test suite."""
from __future__ import annotations
import contextlib
import os
import signal
import subprocess
import uuid
from collections.abc import Callable, Iterator
from pathlib import Path
import docker
import pytest
from helpers import Container, raise_interrupt, remove_quietly, run_once, start_container
ROOT = Path(__file__).resolve().parent.parent
IMAGE = os.environ.get("IMAGE", "restic-backup-docker:test")
REQUIRED_FEATURES_FILE = Path(
os.environ.get("REQUIRED_FEATURES_FILE", Path(__file__).with_name("required-features.txt"))
)
RUN_ID = uuid.uuid4().hex[:8]
# Shell probe run once in a throwaway container; one line per detected feature.
FEATURE_PROBE = """
grep -q RESTIC_STREAM /bin/backup && echo stream
command -v docker >/dev/null && echo docker-cli
command -v bash >/dev/null && echo bash
command -v msmtp >/dev/null && echo msmtp
true
"""
KNOWN_FEATURES = ("stream", "docker-cli", "bash", "msmtp")
def required_features() -> set[str]:
if not REQUIRED_FEATURES_FILE.is_file():
return set()
lines = (line.strip() for line in REQUIRED_FEATURES_FILE.read_text().splitlines())
return {line for line in lines if line and not line.startswith("#")}
def pytest_configure(config: pytest.Config) -> None:
"""Turn SIGTERM into KeyboardInterrupt so session teardown (cleanup) runs."""
signal.signal(signal.SIGTERM, raise_interrupt)
def pytest_addoption(parser: pytest.Parser) -> None:
parser.addoption("--build", action="store_true", help="build the image before testing")
@pytest.fixture(scope="session")
def docker_client() -> Iterator[docker.DockerClient]:
try:
client = docker.from_env()
client.ping()
except docker.errors.DockerException as exc:
pytest.exit(f"docker is not available: {exc}", returncode=2)
yield client
client.close()
@pytest.fixture(scope="session")
def image(request: pytest.FixtureRequest, docker_client: docker.DockerClient) -> str:
"""Tag of the image under test; built on demand with --build."""
if request.config.getoption("--build"):
subprocess.run(["docker", "build", "--quiet", "-t", IMAGE, str(ROOT)], check=True)
try:
docker_client.images.get(IMAGE)
except docker.errors.ImageNotFound:
pytest.exit(f"image {IMAGE} not found, run `task build` or pass --build", returncode=2)
return IMAGE
def _remove_leftovers(client: docker.DockerClient) -> None:
"""Remove containers, then volumes, labelled with this run; safe to call repeatedly."""
flt = {"label": f"rbd-test-run={RUN_ID}"}
try:
for container in client.containers.list(all=True, filters=flt):
with contextlib.suppress(docker.errors.NotFound):
container.remove(force=True, v=True)
finally:
for volume in client.volumes.list(filters=flt):
with contextlib.suppress(docker.errors.NotFound):
volume.remove(force=True)
@pytest.fixture(scope="session", autouse=True)
def _cleanup_leftovers(docker_client: docker.DockerClient) -> Iterator[None]:
"""Remove every container/volume this run created, even after failures or Ctrl-C.
A Ctrl-C in `task`/`uv` is forwarded as a second SIGINT while teardown runs, so
SIGINT/SIGTERM are ignored until cleanup is finished.
"""
try:
yield
finally:
old = {sig: signal.signal(sig, signal.SIG_IGN) for sig in (signal.SIGINT, signal.SIGTERM)}
try:
_remove_leftovers(docker_client)
finally:
for sig, handler in old.items():
signal.signal(sig, handler)
@pytest.fixture(scope="session")
def detected_features(docker_client: docker.DockerClient, image: str) -> set[str]:
out = run_once(
docker_client,
image,
entrypoint=["sh", "-c", FEATURE_PROBE],
labels={"rbd-test-run": RUN_ID},
)
return set(out.split())
def pytest_collection_modifyitems(items: list[pytest.Item]) -> None:
for item in items:
for marker in item.iter_markers("feature"):
item.fixturenames.append("_feature_gate")
if marker.args[0] not in KNOWN_FEATURES:
raise pytest.UsageError(f"unknown feature gate: {marker.args[0]}")
def _apply_feature_gate(node: pytest.Item | pytest.Class, detected: set[str]) -> None:
for marker in node.iter_markers("feature"):
name = marker.args[0]
if name in detected:
continue
if name in required_features():
pytest.fail(
f"feature '{name}' listed in {REQUIRED_FEATURES_FILE.name} "
"but not present in the image",
pytrace=False,
)
pytest.skip(f"feature '{name}' not in image yet")
@pytest.fixture
def _feature_gate(request: pytest.FixtureRequest, detected_features: set[str]) -> None:
"""Skip a feature test when the image lacks it; fail when it is required."""
_apply_feature_gate(request.node, detected_features)
@pytest.fixture(scope="class")
def feature_gate_class(request: pytest.FixtureRequest, detected_features: set[str]) -> None:
"""Same gate for class-scoped fixtures: request it first so no container starts when gated."""
_apply_feature_gate(request.node, detected_features)
LABELS = {"rbd-test-run": RUN_ID}
@pytest.fixture(scope="session")
def run_container(
docker_client: docker.DockerClient, image: str
) -> Callable[..., contextlib.AbstractContextManager[Container]]:
"""Factory: `with run_container("name", env={...}) as c:` starts, waits, always removes."""
@contextlib.contextmanager
def _run(
suffix: str, env: dict[str, str] | None = None, files: dict[str, str] | None = None
) -> Iterator[Container]:
container = start_container(
docker_client, image, f"rbd-test-{suffix}-{RUN_ID}", LABELS, env, files
)
try:
yield container
finally:
remove_quietly(container)
return _run