From 56ed794ff70f1c1addd7bf1103746dff3f863054 Mon Sep 17 00:00:00 2001 From: Timothy Date: Fri, 14 Aug 2026 09:04:11 +0200 Subject: [PATCH 01/14] feat(776): every hook reports that it fired, and the report is measured MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In this harness version only `Stop` hooks emit a structured transcript record. `PreToolUse` and `PostToolUse` hooks leave no durable execution trace — which is every guard this repo relies on. #773 §5.4 could therefore only count filename mentions in settings dumps, i.e. inference, so a silently broken hook looked identical to a working one indefinitely. Each hook now sources one shared sink, `scripts/hook-fire-log.sh`, and records a `fire` record on entry and an `exit` record carrying its status and decision. The decision is parsed from the bytes the hook actually emitted rather than declared by the author: Claude hooks always exit 0 and decide by printing JSON, so stdout is diverted and replayed; git hooks decide by exit code and stream their output, so theirs is left alone. That avoided ~40 edits inside pretooluse-merge-consent and records what the harness sees rather than what the author meant. The instrumentation sits in the stdin/stdout path of the most authoritative guards in the repo, so transparency is the load-bearing claim, not coverage: test_instrumentation_changes_NOTHING_the_harness_can_see drives every hook with and without it over a payload matrix reaching each hook's deciding branch, and demands byte-equal stdout and equal exit status. It shipped with two defects that only that shape catches — `out=$(cat f)` stripping a trailing newline, and an unconditional stdin slurp that hangs on a TTY and would have frozen every interactive `git commit`. Both are now pinned, the second through a real pty. fixes #776 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 (1M context) --- .claude/hooks/decisions-guard.sh | 6 + .claude/hooks/design-sync-reminder.sh | 6 + .claude/hooks/posttooluse-worktree-marker.sh | 6 + .claude/hooks/prepush-clean-worktree-check.sh | 6 + .claude/hooks/prepush-donewhen.sh | 6 + .claude/hooks/prepush-rebase-check.sh | 6 + .claude/hooks/pretooluse-agent-model.sh | 6 + .claude/hooks/pretooluse-agent-ram.sh | 6 + .claude/hooks/pretooluse-bash-guard.sh | 6 + .claude/hooks/pretooluse-bom-guard.sh | 6 + .claude/hooks/pretooluse-merge-consent.sh | 6 + .claude/hooks/pretooluse-nav-guard.sh | 6 + .claude/hooks/pretooluse-worktree-guard.sh | 6 + docs/decisions/README.md | 1 + .../testing/hook-reports-its-own-execution.md | 70 +++ docs/guard-inventory.md | 16 +- scripts/hook-fire-log.sh | 364 +++++++++++++ scripts/tests/test_hook_fire_log.py | 490 ++++++++++++++++++ 18 files changed, 1015 insertions(+), 4 deletions(-) create mode 100644 docs/decisions/records/testing/hook-reports-its-own-execution.md create mode 100755 scripts/hook-fire-log.sh create mode 100644 scripts/tests/test_hook_fire_log.py diff --git a/.claude/hooks/decisions-guard.sh b/.claude/hooks/decisions-guard.sh index 91665164b..ce22df507 100755 --- a/.claude/hooks/decisions-guard.sh +++ b/.claude/hooks/decisions-guard.sh @@ -4,6 +4,12 @@ # body-diff, CI). This shim runs the structural validator over the working tree; the body-diff/no- # vanish checks run in CI where a base/head is available. Fail-open on any tooling trouble. set -uo pipefail + +# ersatztv#776 — report that this hook fired. MUST precede any stdin read. +# git hook: decides by exit code, and its stdout is live progress text. +ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)}/scripts/hook-fire-log.sh" +[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true +etv_hook_fire_begin decisions-guard "" stream || true cd "$(git rev-parse --show-toplevel)" || exit 0 command -v python3 >/dev/null 2>&1 || exit 0 # no python -> fail-open PYTHONPATH=. python3 scripts/decisions_validate.py diff --git a/.claude/hooks/design-sync-reminder.sh b/.claude/hooks/design-sync-reminder.sh index 43615897f..8a90d9e8e 100755 --- a/.claude/hooks/design-sync-reminder.sh +++ b/.claude/hooks/design-sync-reminder.sh @@ -17,6 +17,12 @@ # This is a reminder, never a hard gate — `start` only injects context; `finish` is a one-shot Stop nudge. set -euo pipefail +# ersatztv#776 — report that this hook fired. MUST precede any stdin read. +# Claude hook: decides by printed JSON, so stdout is captured. +ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)}/scripts/hook-fire-log.sh" +[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true +etv_hook_fire_begin design-sync-reminder "${1:-}" capture || true + UI_RE='(^|/)web/src/.*\.(tsx|css)$' TEST_RE='\.test\.(tsx|ts)$' diff --git a/.claude/hooks/posttooluse-worktree-marker.sh b/.claude/hooks/posttooluse-worktree-marker.sh index 588ed517d..f481bc407 100755 --- a/.claude/hooks/posttooluse-worktree-marker.sh +++ b/.claude/hooks/posttooluse-worktree-marker.sh @@ -4,6 +4,12 @@ # a sibling worktree another session created apart from this session's own. # Fail-safe: any parse trouble → do nothing (the guard stays fail-open without a marker). set -euo pipefail + +# ersatztv#776 — report that this hook fired. MUST precede any stdin read. +# Claude hook: decides by printed JSON, so stdout is captured. +ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)}/scripts/hook-fire-log.sh" +[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true +etv_hook_fire_begin posttooluse-worktree-marker "" capture || true input=$(cat) cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // ""' 2>/dev/null || true) cwd=$(printf '%s' "$input" | jq -r '.cwd // ""' 2>/dev/null || true) diff --git a/.claude/hooks/prepush-clean-worktree-check.sh b/.claude/hooks/prepush-clean-worktree-check.sh index 4b49ae874..a384ba607 100755 --- a/.claude/hooks/prepush-clean-worktree-check.sh +++ b/.claude/hooks/prepush-clean-worktree-check.sh @@ -15,6 +15,12 @@ # no origin/main, HEAD unresolved -> allow. Deliberate escape: ETV_ALLOW_DIRTY_PUSH=1. set -uo pipefail +# ersatztv#776 — report that this hook fired. MUST precede any stdin read. +# git hook: decides by exit code, and its stdout is live progress text. +ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)}/scripts/hook-fire-log.sh" +[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true +etv_hook_fire_begin prepush-clean-worktree-check "" stream || true + [ "${ETV_ALLOW_DIRTY_PUSH:-}" = "1" ] && exit 0 git rev-parse --git-dir >/dev/null 2>&1 || exit 0 diff --git a/.claude/hooks/prepush-donewhen.sh b/.claude/hooks/prepush-donewhen.sh index 4ca45400b..b7c64c16a 100755 --- a/.claude/hooks/prepush-donewhen.sh +++ b/.claude/hooks/prepush-donewhen.sh @@ -12,6 +12,12 @@ # Auth (never committed): ETV_GITEA_TOKEN or ETV_GITEA_BASICAUTH; ETV_GITEA_URL overrides the base. set -euo pipefail +# ersatztv#776 — report that this hook fired. MUST precede any stdin read. +# git hook: decides by exit code, and its stdout is live progress text. +ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)}/scripts/hook-fire-log.sh" +[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true +etv_hook_fire_begin prepush-donewhen "" stream || true + # git passes " " lines on stdin. refs=$(cat || true) printf '%s\n' "$refs" | grep -q 'refs/heads/main' || exit 0 # only gate pushes to main diff --git a/.claude/hooks/prepush-rebase-check.sh b/.claude/hooks/prepush-rebase-check.sh index 4dfa00f45..a76c24858 100755 --- a/.claude/hooks/prepush-rebase-check.sh +++ b/.claude/hooks/prepush-rebase-check.sh @@ -9,6 +9,12 @@ # a positively-proven "behind origin/main". Deliberate exception: ETV_SKIP_REBASE_CHECK=1. set -uo pipefail +# ersatztv#776 — report that this hook fired. MUST precede any stdin read. +# git hook: decides by exit code, and its stdout is live progress text. +ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)}/scripts/hook-fire-log.sh" +[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true +etv_hook_fire_begin prepush-rebase-check "" stream || true + [ "${ETV_SKIP_REBASE_CHECK:-}" = "1" ] && exit 0 git rev-parse --git-dir >/dev/null 2>&1 || exit 0 diff --git a/.claude/hooks/pretooluse-agent-model.sh b/.claude/hooks/pretooluse-agent-model.sh index 8b0583244..520f99dd1 100755 --- a/.claude/hooks/pretooluse-agent-model.sh +++ b/.claude/hooks/pretooluse-agent-model.sh @@ -40,6 +40,12 @@ # Fail-open by design: any parse trouble -> allow (exit 0, no output). set -uo pipefail +# ersatztv#776 — report that this hook fired. MUST precede any stdin read. +# Claude hook: decides by printed JSON, so stdout is captured. +ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)}/scripts/hook-fire-log.sh" +[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true +etv_hook_fire_begin pretooluse-agent-model "" capture || true + input=$(cat) tool=$(printf '%s' "$input" | jq -r '.tool_name // ""' 2>/dev/null || true) diff --git a/.claude/hooks/pretooluse-agent-ram.sh b/.claude/hooks/pretooluse-agent-ram.sh index bf26d8c4c..acf258f7f 100755 --- a/.claude/hooks/pretooluse-agent-ram.sh +++ b/.claude/hooks/pretooluse-agent-ram.sh @@ -3,6 +3,12 @@ # The historic 8-9-way crash was RAM starvation, not CPU load; gate on FREE RAM. # Fail-open: if memory_pressure is unavailable/unparsable → allow. set -euo pipefail + +# ersatztv#776 — report that this hook fired. MUST precede any stdin read. +# Claude hook: decides by printed JSON, so stdout is captured. +ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)}/scripts/hook-fire-log.sh" +[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true +etv_hook_fire_begin pretooluse-agent-ram "" capture || true free=$(memory_pressure -Q 2>/dev/null | grep -oE 'free percentage: [0-9]+' | grep -oE '[0-9]+' || true) [ -z "${free:-}" ] && exit 0 diff --git a/.claude/hooks/pretooluse-bash-guard.sh b/.claude/hooks/pretooluse-bash-guard.sh index d6b5121d2..c9da6e459 100755 --- a/.claude/hooks/pretooluse-bash-guard.sh +++ b/.claude/hooks/pretooluse-bash-guard.sh @@ -2,6 +2,12 @@ # PreToolUse / Bash — deny commands that violate a HARD RULE. # Fail-open: any parse trouble → allow (exit 0 with no output). set -euo pipefail + +# ersatztv#776 — report that this hook fired. MUST precede any stdin read. +# Claude hook: decides by printed JSON, so stdout is captured. +ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)}/scripts/hook-fire-log.sh" +[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true +etv_hook_fire_begin pretooluse-bash-guard "" capture || true input=$(cat) cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // ""' 2>/dev/null || true) diff --git a/.claude/hooks/pretooluse-bom-guard.sh b/.claude/hooks/pretooluse-bom-guard.sh index b687c433c..387c7fd92 100755 --- a/.claude/hooks/pretooluse-bom-guard.sh +++ b/.claude/hooks/pretooluse-bom-guard.sh @@ -18,6 +18,12 @@ # the reason a commit can't happen; CI is still the backstop. set -uo pipefail +# ersatztv#776 — report that this hook fired. MUST precede any stdin read. +# Claude hook: decides by printed JSON, so stdout is captured. +ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)}/scripts/hook-fire-log.sh" +[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true +etv_hook_fire_begin pretooluse-bom-guard "" capture || true + input=$(cat) cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // ""' 2>/dev/null || true) [ -n "$cmd" ] || exit 0 diff --git a/.claude/hooks/pretooluse-merge-consent.sh b/.claude/hooks/pretooluse-merge-consent.sh index ef81a9ea5..69ecc5157 100755 --- a/.claude/hooks/pretooluse-merge-consent.sh +++ b/.claude/hooks/pretooluse-merge-consent.sh @@ -40,6 +40,12 @@ # Gitea auth from env (never committed): ETV_GITEA_TOKEN (a token) OR ETV_GITEA_BASICAUTH (user:pass). # ETV_GITEA_URL overrides the base (default: the LAN instance; a LAN address, not a secret). set -euo pipefail + +# ersatztv#776 — report that this hook fired. MUST precede any stdin read. +# Claude hook: decides by printed JSON, so stdout is captured. +ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)}/scripts/hook-fire-log.sh" +[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true +etv_hook_fire_begin pretooluse-merge-consent "" capture || true input=$(cat) decide() { # $1=grant|allow|deny|ask $2=reason diff --git a/.claude/hooks/pretooluse-nav-guard.sh b/.claude/hooks/pretooluse-nav-guard.sh index cacc96360..1bae8aadc 100755 --- a/.claude/hooks/pretooluse-nav-guard.sh +++ b/.claude/hooks/pretooluse-nav-guard.sh @@ -2,6 +2,12 @@ # PreToolUse / browser-navigate — deny opening download/stream endpoints in a tab # (they hang the MCP session; curl them instead). Fail-open on parse trouble. set -euo pipefail + +# ersatztv#776 — report that this hook fired. MUST precede any stdin read. +# Claude hook: decides by printed JSON, so stdout is captured. +ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)}/scripts/hook-fire-log.sh" +[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true +etv_hook_fire_begin pretooluse-nav-guard "" capture || true input=$(cat) url=$(printf '%s' "$input" | jq -r '.tool_input.url // ""' 2>/dev/null || true) diff --git a/.claude/hooks/pretooluse-worktree-guard.sh b/.claude/hooks/pretooluse-worktree-guard.sh index 18ccb10c7..77eb450ba 100755 --- a/.claude/hooks/pretooluse-worktree-guard.sh +++ b/.claude/hooks/pretooluse-worktree-guard.sh @@ -8,6 +8,12 @@ # So the main tree (never marked) and pre-convention worktrees (no marker) are unaffected; # only a commit/merge into another session's marked worktree is blocked. set -euo pipefail + +# ersatztv#776 — report that this hook fired. MUST precede any stdin read. +# Claude hook: decides by printed JSON, so stdout is captured. +ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)}/scripts/hook-fire-log.sh" +[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true +etv_hook_fire_begin pretooluse-worktree-guard "" capture || true input=$(cat) cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // ""' 2>/dev/null || true) cwd=$(printf '%s' "$input" | jq -r '.cwd // ""' 2>/dev/null || true) diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 972275cb2..4510d26ad 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -196,6 +196,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `testing.enumerating-guard-identity-not-position` | A guard that cross-checks a hand-reviewed registry against call sites discovered across the whole repo must key each entry on properties INTRINSIC to the site — file, kind, and the value source text — and never on its absolute line or column. A registry keyed on position is a function of every other file in the repo, so a branch that never touches the guard can invalidate it; and because each PR is green against its own base, that failure is structurally invisible pre-merge and lands on `main` after review and after the merge gate. Dropping the position keeps every mutation the guard exists for — a NEW site, a REMOVED site and a CHANGED value each still fail, since each changes the identity multiset — and costs exactly ONE case, which must be stated rather than implied: a SAME-IDENTITY SUBSTITUTION within one file (delete a registered site, add a different unreviewed one with the same kind and value token, net-zero count) now passes. A REPORTED failure still prints the discovered line:column, because identity and diagnostics need not share a format. Comparison stays a MULTISET count rather than set membership, so two sites in one file sharing an identity must be discovered exactly that many times and a third occurrence still fails. A SCANNER test that asserts real AST positions against FIXED inline fixtures is the opposite case and keeps its line/column identity — it has no churn, because its input does not move. | 2026-07-27 | [link](records/testing/enumerating-guard-identity-not-position.md) | | `testing.guard-derives-population-from-source` | A guard that asserts a COMPLETENESS property enumerates its population from a machine-readable authoritative source — the enum, the generated OpenAPI document, the parsed workflow YAML, the provider list — and asserts SET EQUALITY in BOTH directions against it. It may not narrow that population with a filter, a `Where`, a `grep` or an early `continue` before the assertion, because a filter cannot see the member that is MISSING: the member whose absence is the defect is precisely the one the predicate excludes. A hand-written literal list of members is the same defect in slower motion — a filter frozen at authoring time, correct on the day it was written and unable to report the day it stopped being. Two boundaries bound the rule rather than weaken it. FIRST, filtering to select the SUBJECT of a PER-MEMBER property is legitimate and is not this defect: the excluded members satisfy the property vacuously, so the filtered walk and the whole walk assert the same thing (`ToolCatalogTests.Every_Query_Parameter_Should_Be_A_Declared_Property` filters to tools that declare query parameters, and a tool declaring none has nothing to check). The defect is filtering the population before a COMPLETENESS claim, which is what makes an absent member unrepresentable (#757 filtered on `QueryParameters is {Count: > 0}` and so could not see a tool that should have declared one and did not). SECOND, a population of VALUES always has an external authoritative source and this rule applies directly; a population of SITES IN CODE has no such list, needs find-all-references tooling, and is tracked separately in #777 — do not stretch a set-equality assertion over it. Distinguish the guard SCOPE (which subsystems it covers — a reviewed policy choice, legitimately hand-written) from the guard POPULATION (the members inside that scope — always derived). When the scope itself MIRRORS an authoritative source, the mirror needs its own equality check or a dated staleness marker, or the guard is complete within a scope that has silently gone stale. The canonical worked example in this repo is `ToolCatalogTests.Every_Tool_Should_Declare_Exactly_Its_OpenApi_Query_Parameters`; the canonical residual gap is `MARKED_JOBS` in `scripts/tests/test_ci_dropped_step_guard.py`. | 2026-08-13 | [link](records/testing/guard-derives-population-from-source.md) | | `testing.guard-ships-with-mutation-proof` | A guard is not considered tested because a test involving it passes. It ships with a MUTATION PROOF: remove or disarm THAT GUARD'S CLAUSE ALONE, and a NAMED test must go red. ONE NAMED EXCEPTION, with its limits, because the rule degenerates without it: where the guard IS a test (a checker enforcing a repo invariant, with no separate script behind it), disarming it makes it ABSENT rather than red, so the proof is the contrapositive — INTRODUCE THE DEFECT THE GUARD EXISTS TO CATCH into an isolated copy of the guarded artifact, and the named test must go red. That is a mutation of the guarded SYSTEM rather than of the assertion, and it is admissible ONLY for checker-guards and ONLY when the mutation was executed and witnessed. It is NOT a licence to grade an ordinary script-guard MUTATION for having a bad-input test: feeding a script an input its clause rejects is BEHAVIOUR-ONLY, which is what three rows were regraded for. A file-level grade under this exception covers the clause its cited case actually mutates, not every assertion that later lands in the same file; clause-level grading is tracked in #790. Three things this excludes, each of which has already shipped here as a green suite over a dead check. FIRST, a behavioural test — one that feeds the guard a good input and a bad input and checks it passes and fails — proves the guard REACTS, never that it is LOAD-BEARING; #685 had two guards on one condition where deleting either left the whole suite green while every behavioural test passed. SECOND, mutating the WHOLE FILE does not count (#510): a whole-file revert cannot show that a test reaches a particular clause, so the mutation must target the clause. THIRD, the guard being WIRED is not the guard RUNNING — #631's suite was invoked by no CI job, #751's step was dropped by the runner and the job reported success in 6s against a normal 14-17s, and #719's new logic was never connected to stdin. Every guard also carries an ANTI-VACUITY assertion, because the characteristic failure of a completeness check is reporting that it proved everything while its population was empty. Mechanical enforcement is possible for the BOOKKEEPING and not for the JUDGEMENT, and the split is the decision: `docs/guard-inventory.md` lists every guard file with its Kind, its Proof class (`MUTATION`/`BEHAVIOUR-ONLY`/`NONE`) and a `file::function` ref, and `scripts/tests/test_guard_inventory.py` derives the guard population from the filesystem and the call sites, asserts SET EQUALITY against the rows, and resolves every claimed ref to a real `def`. So a new guard cannot ship unclassified and a renamed test cannot leave a row silently claiming coverage. What stays with review, and is stated rather than papered over: nothing checks that a row claiming `MUTATION` is telling the truth. | 2026-08-13 | [link](records/testing/guard-ships-with-mutation-proof.md) | +| `testing.hook-reports-its-own-execution` | Every script in `.claude/hooks/` sources `scripts/hook-fire-log.sh` and calls `etv_hook_fire_begin