Testing¶

Layers¶

Layer	Location	Rules
Unit	`tests/unit/`	No network, no real providers, no real `agy`. Use `FakeProvider`, recorded yt-dlp fixtures (`tests/fixtures/ytdlp/.json`) and the hand-built Data API fixtures (`tests/fixtures/youtube_api/.json`, `build()` stubbed). Fast (< 1 s each).
Contract	`tests/contract/`	`test_provider_contract.py` is parametrised over every provider in the registry; a provider that cannot pass it cannot ship. Real providers are skipped unless `-m integration`. `test_source_contract.py` holds every `MetadataSource` (`ytdlp`, `api`) to one suite over the same fixture videos.
Integration	`tests/integration/`	Marked `integration`; hit live YouTube, the real Antigravity CLI, or testcontainers database environments. Excluded by default (`addopts = "-m 'not integration'"`). Run with `uv run pytest -m integration`.
Golden	`tests/golden/`, `tests/templates/golden/`	Marked `golden`; text overlay output compared against committed PNGs in `tests/golden/overlay/`, built-in prompts against committed text in `tests/templates/golden/`. Images compare exactly on decoded pixels (size, mode, `tobytes()`), not on the PNG file, whose compressed bytes may differ per OS; text compares exactly. A missing golden fails. Regenerate with `uv run pytest -m golden --update-golden` and review the files before committing. Only bundled fonts are used so output is stable across machines.

Commands¶

uv run pytest -q                       # unit + contract (default)
uv run pytest -m integration           # live systems, opt-in
uv run pytest -m golden                # image and prompt snapshots
uv run pytest -m golden --update-golden  # rewrite goldens, then review them
uv run pytest --cov --cov-fail-under=80

Fixtures¶

yt-dlp responses are recorded once by an integration-marked recorder test and committed as JSON. Re-record when yt-dlp changes shape; review the diff.
FakeProvider is deterministic: same prompt + seed → same bytes. Prompts containing [[FAIL_TRANSIENT]] / [[FAIL_PERMANENT]] raise the corresponding errors, which is how retry and resume paths are tested.
Antigravity adapter unit tests use a fake agy script on PATH that emits chosen JSON envelopes and exit codes, covering every row of the error-mapping table in PLAN.md §4.3.

What a test must do¶

Defend an observable contract: behaviour, boundaries, invariants, state transitions, error mapping. Do not test wiring, defaults, or source text. Delete tests that only pin implementation.

CI¶

ci.yml runs ruff, pyright, lint-imports, pytest with coverage (fail under 80 %), and zensical build on Ubuntu and Windows for Python 3.14.