Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 8s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 26s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m32s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m12s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m14s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m32s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
114 lines
7.0 KiB
Python
114 lines
7.0 KiB
Python
"""Keep the test suite out of the PRODUCTION hook-fire log (ersatztv#776, #809, #822).
|
|
|
|
Several suites here drive real hooks as subprocesses. Four of them predate the instrumentation —
|
|
`test_merge_consent_exemption.py`, `test_merge_consent_base_change.py`,
|
|
`test_prepush_rebase_check_tag_exemption.py` and `test_pr_changed_files.py`; `test_hook_fire_log.py`
|
|
and `test_worktree_ownership_guard.py` have joined them since. Once every hook records its own
|
|
execution, running `pytest scripts/tests` wrote its synthetic invocations into
|
|
`$HOME/.cache/ersatztv/hook-fire/`: 96 `pretooluse-merge-consent` fires including two `deny`s, and
|
|
12 `blocked` decisions from `prepush-rebase-check`, none of them a real session.
|
|
|
|
That is not untidiness, it is the defect the whole change exists to remove. `scripts/hook-fire-log.sh
|
|
report` is meant to answer "what did the harness actually do", and a log carrying test artifacts
|
|
answers a different question while looking identical — inference reintroduced one layer up. The
|
|
record has no field distinguishing a test invocation from a real one, and adding one would only move
|
|
the problem, so the fix is that tests never write to the real log at all.
|
|
|
|
TWO LAYERS, because one is not enough and the two fail differently.
|
|
|
|
`pytest_configure` puts an isolated directory into `os.environ` BEFORE any test module is imported.
|
|
That is what carries the isolation to suites other than the hook-driving four: a module that
|
|
snapshots `{**os.environ}` at import time captures the isolated value, because `pytest_configure`
|
|
runs before collection. #785 shipped exactly that snapshot, inside the file added to prove #776's
|
|
hooks, and it leaked 58 records per run while every assertion stayed green — the fire-log library is
|
|
fail-open by design, so nothing can surface from the hook side. The ordering this relies on was
|
|
verified by execution rather than read: a module-level `{**os.environ}` does carry a value set in
|
|
`pytest_configure`. The autouse fixture below still re-points the variable per test, so a test that
|
|
inspects its own log dir sees only its own records.
|
|
|
|
The route the first layer cannot see is an environment built from SCRATCH rather than derived from
|
|
`os.environ`, carrying no `ETV_HOOK_FIRE_LOG_DIR` at all — so `scripts/hook-fire-log.sh` falls back
|
|
to its own default. No snapshot is involved, so nothing about import ordering helps.
|
|
`ProductionLogGuard` closes it at the launch itself: a child that does not carry the isolated
|
|
directory fails the test that launched it. Its population is every launch that actually happens,
|
|
derived at runtime rather than from a list or from source text, so a new suite is covered by
|
|
existing — every launch, that is, that goes through `subprocess.Popen` and is the DIRECT child.
|
|
`ProductionLogGuard`'s own docstring enumerates what that leaves out (a re-exec, `os.execve`, a
|
|
script that unsets the variable itself, a `Popen` bound by name before this hook ran); none occur
|
|
here today, and the list is written down so the population is not read as total.
|
|
|
|
The rule is "CARRIES AN ISOLATED DIRECTORY", not "is not the production one", and the difference is
|
|
load-bearing: the sink's default has TWO branches — `$HOME/.cache/...` and, when `HOME` is unset or
|
|
empty, `/tmp/.cache/...` — so comparing against a single resolved production path models one and
|
|
waves the other through. On the same tree, the launches whose environment is built FROM SCRATCH —
|
|
the ones the first layer cannot reach — also number 83 across the helpers this change updates;
|
|
81 of those carry no `HOME` either and so RESOLVE TO `/tmp/.cache/ersatztv/hook-fire`, which
|
|
is just as shared, just as persistent, and which the predicate above cannot see. (Two different sets
|
|
of the same size, said out loud because a later reader would otherwise reconcile them as one.)
|
|
Resolve to, not write to: those particular children are `bash`, `git` and a nested pytest that
|
|
never source the sink, and that directory does not exist on this machine. The guard is deliberately
|
|
conservative about which launches it rejects — see `ProductionLogGuard` — because whether a given
|
|
command will reach the sink is not answerable at the launch. Requiring the variable covers both
|
|
branches and does not depend on what `HOME` is on the machine running the suite — which is also what
|
|
keeps it from firing on a legitimate launch wherever `HOME` is unset or is itself
|
|
`/tmp`. Its corollary is that a from-scratch environment must carry the variable; the guard
|
|
enumerates those sites, so a missed one is a red rather than a stale sentence here.
|
|
|
|
WHY NEITHER LAYER OBSERVES THE PRODUCTION DIRECTORY. The obvious check — snapshot it around each
|
|
test and compare — is what shipped, and its oracle is state every concurrent session on the machine
|
|
writes: that directory is shared by every live Claude Code session, so an unrelated session firing a
|
|
hook inside the window reddened the suite with `a test run modified the production hook-fire log` —
|
|
an accusation about the suite when the writer was another process (#822). Observed three times on
|
|
three different branches, each time green on an immediate re-run. Both layers here are structural
|
|
and read nothing under `$HOME`.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import os
|
|
import shutil
|
|
import tempfile
|
|
|
|
import pytest
|
|
|
|
from scripts.tests.hook_fire_isolation import ENV_VAR, ProductionLogGuard
|
|
|
|
_GUARD = ProductionLogGuard()
|
|
_SESSION_LOG_DIR: str | None = None
|
|
|
|
|
|
def pytest_configure(config):
|
|
"""Isolate BEFORE collection, so an import-time `{**os.environ}` snapshot inherits the isolation.
|
|
|
|
This is also what covers MODULE- and SESSION-scoped fixtures, which the autouse fixture below
|
|
cannot reach: they are set up outside any single test, so `os.environ` had no isolated value
|
|
while they ran. Measured once on the pre-change tree, asking which launches resolve to
|
|
`$HOME/.cache/ersatztv/hook-fire`: 83 — 37 during collection and 44 inside module- or
|
|
session-scoped fixtures. The remaining 2 came from an environment built from scratch, which is
|
|
the group this layer cannot help with and the reason there is a second one.
|
|
|
|
INSTRUMENT `Popen` ONLY. `subprocess.run`, `call` and `check_output` all reach `Popen`, so
|
|
wrapping `run` as well counts every launch twice and doubles every figure — which is how the
|
|
first version of this measurement was wrong.
|
|
"""
|
|
global _SESSION_LOG_DIR
|
|
_SESSION_LOG_DIR = tempfile.mkdtemp(prefix="etv-hook-fire-session-")
|
|
os.environ[ENV_VAR] = _SESSION_LOG_DIR
|
|
_GUARD.install()
|
|
|
|
|
|
def pytest_unconfigure(config):
|
|
_GUARD.uninstall()
|
|
if _SESSION_LOG_DIR:
|
|
shutil.rmtree(_SESSION_LOG_DIR, ignore_errors=True)
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def isolate_hook_fire_log(tmp_path_factory, monkeypatch):
|
|
"""Re-point per test, so a test reading its own log dir sees only its own records.
|
|
|
|
`monkeypatch` reverts to the value in place at setup — the session directory from
|
|
`pytest_configure`, never to unset — so the layer above survives every teardown.
|
|
"""
|
|
monkeypatch.setenv(ENV_VAR, str(tmp_path_factory.mktemp("hook-fire-log")))
|