feat(776): every hook reports that it fired, and the report is measured (#795)
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 25s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m14s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m37s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m16s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m19s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 25s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m14s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m37s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m16s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m19s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
This commit was merged in pull request #795.
This commit is contained in:
@@ -4,6 +4,13 @@
|
||||
# body-diff, CI). This shim runs the structural validator over the working tree; the body-diff/no-
|
||||
# vanish checks run in CI where a base/head is available. Fail-open on any tooling trouble.
|
||||
set -uo pipefail
|
||||
|
||||
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
|
||||
# git hook: decides by exit code, and its stdout is live progress text.
|
||||
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
|
||||
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
|
||||
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
|
||||
etv_hook_fire_begin decisions-guard "" stream || true
|
||||
cd "$(git rev-parse --show-toplevel)" || exit 0
|
||||
command -v python3 >/dev/null 2>&1 || exit 0 # no python -> fail-open
|
||||
PYTHONPATH=. python3 scripts/decisions_validate.py
|
||||
|
||||
@@ -17,6 +17,13 @@
|
||||
# This is a reminder, never a hard gate — `start` only injects context; `finish` is a one-shot Stop nudge.
|
||||
set -euo pipefail
|
||||
|
||||
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
|
||||
# Claude hook: decides by printed JSON, so stdout is captured.
|
||||
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
|
||||
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
|
||||
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
|
||||
etv_hook_fire_begin design-sync-reminder "${1:-}" capture || true
|
||||
|
||||
UI_RE='(^|/)web/src/.*\.(tsx|css)$'
|
||||
TEST_RE='\.test\.(tsx|ts)$'
|
||||
|
||||
|
||||
@@ -4,6 +4,13 @@
|
||||
# a sibling worktree another session created apart from this session's own.
|
||||
# Fail-safe: any parse trouble → do nothing (the guard stays fail-open without a marker).
|
||||
set -euo pipefail
|
||||
|
||||
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
|
||||
# Claude hook: decides by printed JSON, so stdout is captured.
|
||||
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
|
||||
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
|
||||
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
|
||||
etv_hook_fire_begin posttooluse-worktree-marker "" capture || true
|
||||
input=$(cat)
|
||||
cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // ""' 2>/dev/null || true)
|
||||
cwd=$(printf '%s' "$input" | jq -r '.cwd // ""' 2>/dev/null || true)
|
||||
|
||||
@@ -15,6 +15,13 @@
|
||||
# no origin/main, HEAD unresolved -> allow. Deliberate escape: ETV_ALLOW_DIRTY_PUSH=1.
|
||||
set -uo pipefail
|
||||
|
||||
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
|
||||
# git hook: decides by exit code, and its stdout is live progress text.
|
||||
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
|
||||
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
|
||||
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
|
||||
etv_hook_fire_begin prepush-clean-worktree-check "" stream || true
|
||||
|
||||
[ "${ETV_ALLOW_DIRTY_PUSH:-}" = "1" ] && exit 0
|
||||
git rev-parse --git-dir >/dev/null 2>&1 || exit 0
|
||||
|
||||
|
||||
@@ -12,6 +12,13 @@
|
||||
# Auth (never committed): ETV_GITEA_TOKEN or ETV_GITEA_BASICAUTH; ETV_GITEA_URL overrides the base.
|
||||
set -euo pipefail
|
||||
|
||||
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
|
||||
# git hook: decides by exit code, and its stdout is live progress text.
|
||||
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
|
||||
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
|
||||
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
|
||||
etv_hook_fire_begin prepush-donewhen "" stream || true
|
||||
|
||||
# git passes "<localref> <localsha> <remoteref> <remotesha>" lines on stdin.
|
||||
refs=$(cat || true)
|
||||
printf '%s\n' "$refs" | grep -q 'refs/heads/main' || exit 0 # only gate pushes to main
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
# a positively-proven "behind origin/main". Deliberate exception: ETV_SKIP_REBASE_CHECK=1.
|
||||
set -uo pipefail
|
||||
|
||||
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
|
||||
# git hook: decides by exit code, and its stdout is live progress text.
|
||||
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
|
||||
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
|
||||
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
|
||||
etv_hook_fire_begin prepush-rebase-check "" stream || true
|
||||
|
||||
[ "${ETV_SKIP_REBASE_CHECK:-}" = "1" ] && exit 0
|
||||
git rev-parse --git-dir >/dev/null 2>&1 || exit 0
|
||||
|
||||
|
||||
@@ -40,6 +40,13 @@
|
||||
# Fail-open by design: any parse trouble -> allow (exit 0, no output).
|
||||
set -uo pipefail
|
||||
|
||||
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
|
||||
# Claude hook: decides by printed JSON, so stdout is captured.
|
||||
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
|
||||
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
|
||||
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
|
||||
etv_hook_fire_begin pretooluse-agent-model "" capture || true
|
||||
|
||||
input=$(cat)
|
||||
|
||||
tool=$(printf '%s' "$input" | jq -r '.tool_name // ""' 2>/dev/null || true)
|
||||
|
||||
@@ -3,6 +3,13 @@
|
||||
# The historic 8-9-way crash was RAM starvation, not CPU load; gate on FREE RAM.
|
||||
# Fail-open: if memory_pressure is unavailable/unparsable → allow.
|
||||
set -euo pipefail
|
||||
|
||||
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
|
||||
# Claude hook: decides by printed JSON, so stdout is captured.
|
||||
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
|
||||
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
|
||||
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
|
||||
etv_hook_fire_begin pretooluse-agent-ram "" capture || true
|
||||
free=$(memory_pressure -Q 2>/dev/null | grep -oE 'free percentage: [0-9]+' | grep -oE '[0-9]+' || true)
|
||||
[ -z "${free:-}" ] && exit 0
|
||||
|
||||
|
||||
@@ -2,6 +2,13 @@
|
||||
# PreToolUse / Bash — deny commands that violate a HARD RULE.
|
||||
# Fail-open: any parse trouble → allow (exit 0 with no output).
|
||||
set -euo pipefail
|
||||
|
||||
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
|
||||
# Claude hook: decides by printed JSON, so stdout is captured.
|
||||
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
|
||||
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
|
||||
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
|
||||
etv_hook_fire_begin pretooluse-bash-guard "" capture || true
|
||||
input=$(cat)
|
||||
cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // ""' 2>/dev/null || true)
|
||||
|
||||
|
||||
@@ -18,6 +18,13 @@
|
||||
# the reason a commit can't happen; CI is still the backstop.
|
||||
set -uo pipefail
|
||||
|
||||
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
|
||||
# Claude hook: decides by printed JSON, so stdout is captured.
|
||||
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
|
||||
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
|
||||
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
|
||||
etv_hook_fire_begin pretooluse-bom-guard "" capture || true
|
||||
|
||||
input=$(cat)
|
||||
cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // ""' 2>/dev/null || true)
|
||||
[ -n "$cmd" ] || exit 0
|
||||
|
||||
@@ -40,6 +40,13 @@
|
||||
# Gitea auth from env (never committed): ETV_GITEA_TOKEN (a token) OR ETV_GITEA_BASICAUTH (user:pass).
|
||||
# ETV_GITEA_URL overrides the base (default: the LAN instance; a LAN address, not a secret).
|
||||
set -euo pipefail
|
||||
|
||||
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
|
||||
# Claude hook: decides by printed JSON, so stdout is captured.
|
||||
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
|
||||
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
|
||||
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
|
||||
etv_hook_fire_begin pretooluse-merge-consent "" capture || true
|
||||
input=$(cat)
|
||||
|
||||
decide() { # $1=grant|allow|deny|ask $2=reason
|
||||
|
||||
@@ -2,6 +2,13 @@
|
||||
# PreToolUse / browser-navigate — deny opening download/stream endpoints in a tab
|
||||
# (they hang the MCP session; curl them instead). Fail-open on parse trouble.
|
||||
set -euo pipefail
|
||||
|
||||
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
|
||||
# Claude hook: decides by printed JSON, so stdout is captured.
|
||||
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
|
||||
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
|
||||
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
|
||||
etv_hook_fire_begin pretooluse-nav-guard "" capture || true
|
||||
input=$(cat)
|
||||
url=$(printf '%s' "$input" | jq -r '.tool_input.url // ""' 2>/dev/null || true)
|
||||
|
||||
|
||||
@@ -8,6 +8,13 @@
|
||||
# So the main tree (never marked) and pre-convention worktrees (no marker) are unaffected;
|
||||
# only a commit/merge into another session's marked worktree is blocked.
|
||||
set -euo pipefail
|
||||
|
||||
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
|
||||
# Claude hook: decides by printed JSON, so stdout is captured.
|
||||
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
|
||||
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
|
||||
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
|
||||
etv_hook_fire_begin pretooluse-worktree-guard "" capture || true
|
||||
input=$(cat)
|
||||
cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // ""' 2>/dev/null || true)
|
||||
cwd=$(printf '%s' "$input" | jq -r '.cwd // ""' 2>/dev/null || true)
|
||||
|
||||
@@ -196,6 +196,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
|
||||
| `testing.enumerating-guard-identity-not-position` | A guard that cross-checks a hand-reviewed registry against call sites discovered across the whole repo must key each entry on properties INTRINSIC to the site — file, kind, and the value source text — and never on its absolute line or column. A registry keyed on position is a function of every other file in the repo, so a branch that never touches the guard can invalidate it; and because each PR is green against its own base, that failure is structurally invisible pre-merge and lands on `main` after review and after the merge gate. Dropping the position keeps every mutation the guard exists for — a NEW site, a REMOVED site and a CHANGED value each still fail, since each changes the identity multiset — and costs exactly ONE case, which must be stated rather than implied: a SAME-IDENTITY SUBSTITUTION within one file (delete a registered site, add a different unreviewed one with the same kind and value token, net-zero count) now passes. A REPORTED failure still prints the discovered line:column, because identity and diagnostics need not share a format. Comparison stays a MULTISET count rather than set membership, so two sites in one file sharing an identity must be discovered exactly that many times and a third occurrence still fails. A SCANNER test that asserts real AST positions against FIXED inline fixtures is the opposite case and keeps its line/column identity — it has no churn, because its input does not move. | 2026-07-27 | [link](records/testing/enumerating-guard-identity-not-position.md) |
|
||||
| `testing.guard-derives-population-from-source` | A guard that asserts a COMPLETENESS property enumerates its population from a machine-readable authoritative source — the enum, the generated OpenAPI document, the parsed workflow YAML, the provider list — and asserts SET EQUALITY in BOTH directions against it. It may not narrow that population with a filter, a `Where`, a `grep` or an early `continue` before the assertion, because a filter cannot see the member that is MISSING: the member whose absence is the defect is precisely the one the predicate excludes. A hand-written literal list of members is the same defect in slower motion — a filter frozen at authoring time, correct on the day it was written and unable to report the day it stopped being. Two boundaries bound the rule rather than weaken it. FIRST, filtering to select the SUBJECT of a PER-MEMBER property is legitimate and is not this defect: the excluded members satisfy the property vacuously, so the filtered walk and the whole walk assert the same thing (`ToolCatalogTests.Every_Query_Parameter_Should_Be_A_Declared_Property` filters to tools that declare query parameters, and a tool declaring none has nothing to check). The defect is filtering the population before a COMPLETENESS claim, which is what makes an absent member unrepresentable (#757 filtered on `QueryParameters is {Count: > 0}` and so could not see a tool that should have declared one and did not). SECOND, a population of VALUES always has an external authoritative source and this rule applies directly; a population of SITES IN CODE has no such list, needs find-all-references tooling, and is tracked separately in #777 — do not stretch a set-equality assertion over it. Distinguish the guard SCOPE (which subsystems it covers — a reviewed policy choice, legitimately hand-written) from the guard POPULATION (the members inside that scope — always derived). When the scope itself MIRRORS an authoritative source, the mirror needs its own equality check or a dated staleness marker, or the guard is complete within a scope that has silently gone stale. The canonical worked example in this repo is `ToolCatalogTests.Every_Tool_Should_Declare_Exactly_Its_OpenApi_Query_Parameters`; the canonical residual gap is `MARKED_JOBS` in `scripts/tests/test_ci_dropped_step_guard.py`. | 2026-08-13 | [link](records/testing/guard-derives-population-from-source.md) |
|
||||
| `testing.guard-ships-with-mutation-proof` | A guard is not considered tested because a test involving it passes. It ships with a MUTATION PROOF: remove or disarm THAT GUARD'S CLAUSE ALONE, and a NAMED test must go red. ONE NAMED EXCEPTION, with its limits, because the rule degenerates without it: where the guard IS a test (a checker enforcing a repo invariant, with no separate script behind it), disarming it makes it ABSENT rather than red, so the proof is the contrapositive — INTRODUCE THE DEFECT THE GUARD EXISTS TO CATCH into an isolated copy of the guarded artifact, and the named test must go red. That is a mutation of the guarded SYSTEM rather than of the assertion, and it is admissible ONLY for checker-guards and ONLY when the mutation was executed and witnessed. It is NOT a licence to grade an ordinary script-guard MUTATION for having a bad-input test: feeding a script an input its clause rejects is BEHAVIOUR-ONLY, which is what three rows were regraded for. A file-level grade under this exception covers the clause its cited case actually mutates, not every assertion that later lands in the same file; clause-level grading is tracked in #790. Three things this excludes, each of which has already shipped here as a green suite over a dead check. FIRST, a behavioural test — one that feeds the guard a good input and a bad input and checks it passes and fails — proves the guard REACTS, never that it is LOAD-BEARING; #685 had two guards on one condition where deleting either left the whole suite green while every behavioural test passed. SECOND, mutating the WHOLE FILE does not count (#510): a whole-file revert cannot show that a test reaches a particular clause, so the mutation must target the clause. THIRD, the guard being WIRED is not the guard RUNNING — #631's suite was invoked by no CI job, #751's step was dropped by the runner and the job reported success in 6s against a normal 14-17s, and #719's new logic was never connected to stdin. Every guard also carries an ANTI-VACUITY assertion, because the characteristic failure of a completeness check is reporting that it proved everything while its population was empty. Mechanical enforcement is possible for the BOOKKEEPING and not for the JUDGEMENT, and the split is the decision: `docs/guard-inventory.md` lists every guard file with its Kind, its Proof class (`MUTATION`/`BEHAVIOUR-ONLY`/`NONE`) and a `file::function` ref, and `scripts/tests/test_guard_inventory.py` derives the guard population from the filesystem and the call sites, asserts SET EQUALITY against the rows, and resolves every claimed ref to a real `def`. So a new guard cannot ship unclassified and a renamed test cannot leave a row silently claiming coverage. What stays with review, and is stated rather than papered over: nothing checks that a row claiming `MUTATION` is telling the truth. | 2026-08-13 | [link](records/testing/guard-ships-with-mutation-proof.md) |
|
||||
| `testing.hook-reports-its-own-execution` | Every script in `.claude/hooks/` sources `scripts/hook-fire-log.sh` and calls `etv_hook_fire_begin <its-own-name> <label> <capture\|stream>` as its FIRST act, before anything reads stdin. Two records are appended per invocation — a `fire` record on entry and an `exit` record carrying the exit status and the decision — to a session-scoped JSONL log. THE DECISION IS READ FROM WHAT THE HOOK ACTUALLY EMITTED, never declared by the hook author: Claude Code hooks (`capture` mode) always exit 0 and communicate by PRINTING JSON, so their stdout is diverted and replayed, and the recorded decision is parsed from those bytes; git hooks (`stream` mode) decide by EXIT CODE and their stdout is live progress text a human is watching, so it is not diverted and the decision is the status. That split is not a tuning knob — capturing a slow pre-push hook's output would hold it back until the end and read as a hang, and inferring a git hook's decision from absent JSON would put the report back into the guessing business this record exists to end. The population is DERIVED from `.claude/hooks/*.sh` by `scripts/tests/test_hook_fire_log.py`, so a new hook is uninstrumented-and-red rather than silently unobserved, and the report lists every hook that EXISTS rather than every hook that appears in the log — a report built from the log alone can only show hooks that fired, which makes the never-fired hook, the one finding worth having, invisible. THE INSTRUMENTATION MUST BE INVISIBLE TO THE HARNESS, and this is the load-bearing half: it sits in the stdin and stdout path of the most authoritative guards in the repo, so a differential test drives EVERY hook with and without it over a payload matrix and demands byte-equal stdout and equal exit status. It fails OPEN in exactly one direction — if the log cannot be written the hook behaves exactly as before — because observability that breaks a guard is worse than the blindness it replaces. Two mechanical traps are pinned by tests rather than left to care: stdout must be replayed from the FILE, since `out=$(cat f)` strips trailing newlines and delivers a guard's JSON one byte short with no parser anywhere to complain; and stdin must never be slurped when it is a TTY, because an interactive `git commit` hands its hooks a terminal and `cat` would block forever, hanging the commit the instrumentation was added to observe. | 2026-08-14 | [link](records/testing/hook-reports-its-own-execution.md) |
|
||||
| `testing.live-e2e-prepush-timing` | Run live-E2E via `scripts/e2e-local.sh` before pushing a write-path or UI change, and exercise download endpoints with curl, never a browser tab. | 2026-07-21 | [link](records/testing/live-e2e-prepush-timing.md) |
|
||||
| `testing.playwright-mcp-download-and-recovery` | In Playwright-MCP E2E, fetch file-download endpoints with curl — never a browser tab or `window.open` — and if browser tools stall repeatedly, `pkill -f ms-playwright-mcp` and drive a fresh session. | 2026-07-21 | [link](records/testing/playwright-mcp-download-and-recovery.md) |
|
||||
| `testing.scripted-playout-golden-deferred` | The `PlayoutBuildGoldenTests` in-memory golden net covers Sequential (YAML) as of #381. Scripted's *end-to-end pipeline* is excluded — `ScriptedPlayoutBuilder` runs a user-authored external program that drives the engine over HTTP loopback, which the in-memory harness can't pin — so that full-pipeline (integration) harness is deferred to #563. But the scheduling *behavior* those scripts drive lives entirely in the in-process `SchedulingEngine` (the `ScriptedScheduleController` is a 1:1 pass-through to it), which IS directly unit/golden-testable; the earlier "Scripted is un-golden-able by construction" framing overstated the constraint by conflating transport with engine. #395 extracts that shared switch to `ContentEnumeratorBuilder` and adds a direct regression net (`ContentEnumeratorBuilderTests`) over it. | 2026-07-22 | [link](records/testing/scripted-playout-golden-deferred.md) |
|
||||
|
||||
@@ -0,0 +1,223 @@
|
||||
---
|
||||
key: testing.hook-reports-its-own-execution
|
||||
title: '2026-08-14 — every hook records that it fired and what it decided, through one shared sink (#776)'
|
||||
status: active
|
||||
since: '2026-08-14'
|
||||
supersedes: none
|
||||
superseded-by: none
|
||||
rule: 'Every script in `.claude/hooks/` sources `scripts/hook-fire-log.sh` and calls `etv_hook_fire_begin <its-own-name> <label> <capture|stream>` as its FIRST act, before anything reads stdin. Two records are appended per invocation — a `fire` record on entry and an `exit` record carrying the exit status and the decision — to a session-scoped JSONL log. THE DECISION IS READ FROM WHAT THE HOOK ACTUALLY EMITTED, never declared by the hook author: Claude Code hooks (`capture` mode) always exit 0 and communicate by PRINTING JSON, so their stdout is diverted and replayed, and the recorded decision is parsed from those bytes; git hooks (`stream` mode) decide by EXIT CODE and their stdout is live progress text a human is watching, so it is not diverted and the decision is the status. That split is not a tuning knob — capturing a slow pre-push hook''s output would hold it back until the end and read as a hang, and inferring a git hook''s decision from absent JSON would put the report back into the guessing business this record exists to end. The population is DERIVED from `.claude/hooks/*.sh` by `scripts/tests/test_hook_fire_log.py`, so a new hook is uninstrumented-and-red rather than silently unobserved, and the report lists every hook that EXISTS rather than every hook that appears in the log — a report built from the log alone can only show hooks that fired, which makes the never-fired hook, the one finding worth having, invisible. THE INSTRUMENTATION MUST BE INVISIBLE TO THE HARNESS, and this is the load-bearing half: it sits in the stdin and stdout path of the most authoritative guards in the repo, so a differential test drives EVERY hook with and without it over a payload matrix and demands byte-equal stdout and equal exit status. It fails OPEN in exactly one direction — if the log cannot be written the hook behaves exactly as before — because observability that breaks a guard is worse than the blindness it replaces. Two mechanical traps are pinned by tests rather than left to care: stdout must be replayed from the FILE, since `out=$(cat f)` strips trailing newlines and delivers a guard''s JSON one byte short with no parser anywhere to complain; and stdin must never be slurped when it is a TTY, because an interactive `git commit` hands its hooks a terminal and `cat` would block forever, hanging the commit the instrumentation was added to observe.'
|
||||
signals: 'hook fire log · did this hook actually fire · PreToolUse leaves no transcript record · make the system report it rather than infer it · capture vs stream mode · byte-equal stdout differential · fail-open observability · never slurp a tty · derived hook population · never-fired row · paths: `scripts/hook-fire-log.sh`, `scripts/tests/test_hook_fire_log.py`, `docs/guard-inventory.md`, `docs/defect-shapes-773.md` · issues: #776, #773, #775, #774, #756, #751, #719, #685, #631'
|
||||
mechanics: 'Read side: `scripts/hook-fire-log.sh report [--all|--json|--dir D|--session S]`. Logs default to `$HOME/.cache/ersatztv/hook-fire/<session-id>.jsonl`, outside the repo, overridable with `ETV_HOOK_FIRE_LOG_DIR`; `ETV_HOOK_FIRE_DISABLE=1` turns recording off without editing a hook. The session id comes from the payload''s `.session_id`, because this harness version does not export one to hooks. Records carry no tool arguments — only hook, label, event, tool name, phase, status and decision — so the log is not a second copy of the transcript.'
|
||||
---
|
||||
|
||||
**The blind spot this closes.** In this harness version only `Stop` hooks emit a structured
|
||||
transcript record. `PreToolUse` and `PostToolUse` hooks — which is *every* guard this repo actually
|
||||
relies on: merge consent, worktree ownership, BOM, agent model and RAM, the golden-file ban — leave
|
||||
no durable execution trace at all. When #773 tried to measure whether they fire, the only available
|
||||
signal was **filename mentions in settings dumps and prior audits**, which is not evidence of
|
||||
execution. So the guards carrying the most authority were the ones whose execution could only be
|
||||
inferred, and a silently broken hook would have looked identical to a working one indefinitely.
|
||||
|
||||
That is Family B (the check never ran) applied to the hook layer itself, and this project has
|
||||
already paid for it twice one layer down: #751's guard that reported success in 6 seconds having run
|
||||
nothing, and #756's fail-open twin that 30 green tests could not see. #756's standing lesson is the
|
||||
whole of this record — **make the system report it rather than infer it**.
|
||||
|
||||
**Why the decision is parsed from the emitted bytes rather than declared.** The obvious design has
|
||||
each decision site announce itself. `pretooluse-merge-consent.sh` alone reaches its output through a
|
||||
`decide` helper from roughly forty call sites, so that design is forty edits in the most
|
||||
load-bearing guard in the repo, and it records **what the author meant** — which is the inference
|
||||
being abolished, reintroduced one layer up. Capturing what the hook printed records what the
|
||||
*harness* sees. It cannot drift from the decision because it **is** the decision, and it needed no
|
||||
edit to any decision site.
|
||||
|
||||
**The risk this takes on, stated plainly.** Reading stdin before the hook and diverting its stdout
|
||||
puts this code directly in the path of every guard. Get it wrong and a `deny` is swallowed while the
|
||||
log cheerfully reports the guard firing — strictly worse than the blindness it replaces, and
|
||||
undetectable from the outside for the same reason the original gap was. So the proof obligation is
|
||||
not a unit test of the sink: a wrapper can be perfect alone and lethal in context.
|
||||
`test_instrumentation_changes_NOTHING_the_harness_can_see` runs every hook in both forms and
|
||||
compares stdout bytes, stderr bytes and exit status.
|
||||
|
||||
**A differential test is only as good as the cases it drives, and this one had to be measured rather
|
||||
than asserted.** A hook that emits nothing and exits 0 makes the comparison `("", 0) == ("", 0)`,
|
||||
which passes on a wrapper that discards everything. Measured on the first version of the matrix: 160
|
||||
of 165 comparisons were empty-against-empty and 165 of 165 compared exit 0 against exit 0, so
|
||||
deleting the entire replay left the test green for four hooks, two of which issue `deny`. The three
|
||||
places that claimed the matrix "reaches each hook's deciding branch" — including this record — were
|
||||
simply wrong. The structure that replaces the claim:
|
||||
|
||||
- a broad, cheap payload matrix, which reaches the deciding branch of a few hooks;
|
||||
- `positives()`, which CONSTRUCTS the rest — a sibling worktree carrying a foreign owner marker, an
|
||||
`*ersatztv*`-named repo with a staged BOM `.cs`, `memory_pressure` stubbed at 5% and 15%, a real
|
||||
bare remote so the three pre-push guards can be driven to a genuine block, and a stub Gitea so
|
||||
`prepush-donewhen` is exercised rather than exempted;
|
||||
- `test_the_AB_is_not_VACUOUS_for_any_hook`, which fails unless each hook has at least one case
|
||||
where the control side emits output or exits non-zero.
|
||||
|
||||
One hook is exempt and the exemption is stated rather than assumed: `posttooluse-worktree-marker.sh`
|
||||
emits nothing and exits 0 by design — its entire output is a marker file — so its transparency is
|
||||
asserted on that file instead.
|
||||
|
||||
**Five defects this shipped with, all found by cold review or by the tests it prompted.** Recorded
|
||||
because each was invisible to every other check, and because the ranking is instructive: the worst
|
||||
one was surfaced by a finding ranked *Low*.
|
||||
|
||||
| Defect | Why nothing else would catch it | Pinned by |
|
||||
|---|---|---|
|
||||
| `exec 0<"$sin" 2>/dev/null` sent the hook's **entire stderr** to /dev/null for its whole life — `exec` with no command redirects the shell permanently. Every husky guard's user-facing output is stderr, so guards blocked while explaining nothing | the A/B compared only stdout and exit status | `test_stderr_is_NOT_silenced` |
|
||||
| The fd restore lived inside the same conditional as the replay, and the replay read the PATH — so an unlinked-but-open scratch file meant a swallowed `deny` and an exit 0 | needs `mktemp` stubbed or the file removed mid-run | `test_output_SURVIVES_a_vanished_stdout_tempfile` |
|
||||
| `sed`/`tr` in the classifier are locale-aware, so one 0xE9 byte in a `permissionDecisionReason` printed `illegal byte sequence` AT THE HARNESS and filed a real `deny` as `output` | no test put a non-UTF-8 byte in a decision | `LC_ALL=C`, plus the stderr comparison in the A/B |
|
||||
| Payload fields were read from a `head -c 65536` prefix, so a large `tool_response` pushed `session_id` past the cap and those fires filed under `unknown-session` — reading as NEVER FIRED in the report | a truncating read is a sampling error, and no fixture was over 64 KB | fields now read from the file, unbounded |
|
||||
| `out=$(cat f)` strips trailing newlines, so a hook's JSON reached the harness one byte short | every JSON parser accepts it; only a byte comparison objects | `test_stdout_is_replayed_BYTE_EXACT` |
|
||||
| An unconditional stdin slurp blocks forever on a TTY, hanging every interactive `git commit` | the pipe-fed tests cannot reach a tty; needs a real pty | `test_a_TTY_stdin_is_not_slurped` |
|
||||
|
||||
**One mechanism was WITHDRAWN rather than fixed, and it is recorded here because the next reader
|
||||
will otherwise re-add it.** A `trap … INT TERM HUP` was added so a hook killed by the harness
|
||||
timeout would not lose output it had already printed. It produced a defect in each of three
|
||||
consecutive rounds — a handler ending in `exit "$?"` that turned a signal into consent and let a
|
||||
push reach `main`; a `local sig=…` that clobbered `$?` so a killed guard was *recorded* as having
|
||||
passed; and a second signal mid-flush that discarded the output entirely.
|
||||
|
||||
**A correction, because the first version of this record justified the withdrawal with a number that
|
||||
is wrong.** It claimed, as measured fact, that trapping made a hook take 30 s to die where it had
|
||||
taken 1 s. That came from a review finding accepted without re-measuring, and the A/B refutes it:
|
||||
signalling the pid gives control **4.01 s** and trapped **4.12 s**; signalling the process group
|
||||
gives control **0.002 s** and trapped **0.054 s**. The stall is bash deferring a fatal signal until
|
||||
its foreground command finishes, and it happens **identically without any trap** — the control
|
||||
stalls too. The trap cost ~0.1 s, not 29 s. Recorded rather than quietly fixed, because a decision
|
||||
record carrying a fabricated measurement is worse than one carrying none: it reads as checked and
|
||||
talks the next reader out of checking.
|
||||
|
||||
**A second correction, in the same paragraph as the first.** The replacement rationale asserted that
|
||||
the round-3 sink "returned rc 0 on SIGINT where the uninstrumented control returns −2". That is
|
||||
**inverted**, and it was again taken from a review summary without re-measuring — the identical
|
||||
mistake, one round later, in the artifact written to correct it. The measured 12-cell matrix
|
||||
({group, pid} × {TERM, INT, HUP} × {control, round-3 trapped, HEAD}):
|
||||
|
||||
| | TERM | INT | HUP |
|
||||
|---|---|---|---|
|
||||
| group kill — control / trapped (`6f006a1b7`) / HEAD | −15 / −15 / −15 | −2 / −2 / −2 | −1 / −1 / −1 |
|
||||
| pid kill — control / trapped (`6f006a1b7`) / HEAD | −15 / −15 / −15 | **0 / −2 / 0** | −1 / −1 / −1 |
|
||||
|
||||
Reproduce: spawn a `stream`-mode hook that prints a decision then `sleep`s, in its own session;
|
||||
signal 1 s in with `os.killpg` or `os.kill`; read the wait status. Independently re-measured, five
|
||||
repeats per cell, every cell unanimous, stdout byte-identical in all runs — and the "would not have
|
||||
caught round 3" claim carries its own positive control: patching the trapped handler's re-raise back
|
||||
to `exit 0` turns the same test red on all three signals.
|
||||
|
||||
So the trapped sink returned **−2 where the control returns 0** — a real transparency violation, but
|
||||
in the *restrictive* direction, which also makes "the consent bug survived" the wrong
|
||||
characterisation. **HEAD matches the control in all twelve cells.** And the claim that the
|
||||
exit-status assertion "catches the SIGINT case" is false as written: the test signals the process
|
||||
group, and in every group cell the trapped sink is indistinguishable, so it would not have caught
|
||||
round 3 at all. The test asserts the transparency of the current code, which is what it is for; it
|
||||
is not evidence that the trap was bad.
|
||||
|
||||
**What actually justifies the withdrawal**, with the two false reasons removed. Three defects in
|
||||
three rounds, each real and measured at the time — a handler that turned a signal into consent and
|
||||
let a push reach `main`, a `local` that clobbered the status so a killed guard was recorded as
|
||||
passing, and a nested signal that discarded the output. One measured transparency violation that
|
||||
survived the fixes (pid + SIGINT, above). And a rescue window that is sub-millisecond, because every
|
||||
capture-mode hook prints its decision and exits immediately. Machinery needing a signal-to-status
|
||||
map, an idempotent flush, correct re-raise semantics and `BASHPID` handling, to rescue that window,
|
||||
is not worth its failure modes.
|
||||
|
||||
What is lost by withdrawing it, enumerated rather than waved past, because
|
||||
`process.enumerate-workaround-behaviors-before-deleting` exists for precisely this move:
|
||||
|
||||
| Behaviour the trap provided | Status now |
|
||||
|---|---|
|
||||
| A killed hook's `exit` record | **LOST.** This is why `report` counts `fire` records, not `exit` records — a hook that fired and was killed still shows as having fired |
|
||||
| Stdout a killed hook had already printed | **LOST**, and near-unreachable: every capture-mode hook prints its decision and exits immediately, a sub-millisecond window |
|
||||
| Cleanup of the two temp files on a kill | **LOST**, bounded by `$TMPDIR` cleanup |
|
||||
| Idempotent flush (`ETV_HOOK_FIRE_FLUSHED`) | **RETAINED** — cheap, and the failure it prevents is silent |
|
||||
|
||||
The invariant that replaces the rescue is asserted directly: **under a signal, an instrumented hook
|
||||
is indistinguishable from an uninstrumented one** — compared on exit status, stdout, stderr and time
|
||||
to die. It **signals the process group, not the pid**, and that is not a detail: bash defers a fatal
|
||||
signal until its foreground command finishes, so signalling the pid alone makes both sides take the
|
||||
full sleep and the timing assertion distinguishes nothing.
|
||||
|
||||
Its limit, stated because the alternative is a coverage claim that is not true: **the group-kill
|
||||
arrangement makes the test blind to the round-3 trapped sink**, which passes every assertion there.
|
||||
The divergence that sink actually had is a pid-kill one (the table above). This test pins the
|
||||
transparency of the code that ships; it is not a proof that any earlier version was worse.
|
||||
|
||||
**A SECOND withdrawal, on the same reasoning.** A `test_no_redirection_in_the_sink_PRECEDES_its_stderr_redirect`
|
||||
generalised the `exec … 2>/dev/null` rule to any command by pattern-matching shell source. It went
|
||||
through four iterations, each fixing a false positive or a false negative the last had introduced —
|
||||
`>/dev/null 2>&1` flagged wrongly, then operators preceded by a digit (`exec 0<`) missed entirely,
|
||||
then multi-command lines flagged wrongly — and cold review then constructed more of both
|
||||
(`[[ "$x" < "$y" ]] 2>/dev/null`, a line continuation, a `>` inside a quoted string). It is deleted
|
||||
rather than patched a fifth time, following the precedent in `docs/guard-inventory.md`: a weak
|
||||
detector is itself the symptom-keyed mistake, and another regex patch adds confidence without
|
||||
soundness. The narrower `test_no_exec_in_the_sink_carries_a_STDERR_REDIRECT` is retained — it is a
|
||||
sound match on a fixed keyword and covers the dangerous subset, which is every case that has
|
||||
actually occurred.
|
||||
|
||||
**One transparency limit, measured and stated rather than papered over.** On a group kill with a
|
||||
live foreground child, bash prints its own job-control notice (`Terminated: 15 sleep 5`) that an
|
||||
uninstrumented hook does not. Isolated: sourcing the sink alone does not produce it, and a no-op
|
||||
`EXIT` trap does not either — doing ANY work at exit keeps the shell alive the ~50 ms bash needs to
|
||||
print, and doing work at exit is the sink's entire purpose. The A/B exempts exactly that notice and
|
||||
nothing else.
|
||||
|
||||
**The exemption is doing real work and the honest reading is uncomfortable**, so it is stated
|
||||
plainly: one observed form of the notice is
|
||||
`…/scripts/hook-fire-log.sh: line 109: <pid> Hangup: 1 sleep 25` — bash attributing its own
|
||||
message to whichever line of the sink was executing, which means the instrumentation's path appears
|
||||
on a guard's stderr where an uninstrumented hook emits nothing. That is a genuine, if small, leak,
|
||||
and the exemption hides it. It is accepted rather than fixed because it cannot be suppressed while
|
||||
the sink does any work at exit, and it costs one line in a run that is already being killed.
|
||||
|
||||
**CI found what six review rounds on one platform could not, and it was a live guard defect.** The
|
||||
non-vacuity assertion demands that `pretooluse-bom-guard.sh` reach a `deny`. On the Linux runner it
|
||||
could not, because the guard detects a BOM with `xxd -p` — and **`xxd` ships with vim and is absent
|
||||
there**, so the command substitution yielded empty, never matched, and the guard passed every BOM
|
||||
silently. It has been fail-open on any host without vim for its whole life. `od -A n -t x1 -N 3` is
|
||||
POSIX, byte-identical on both platforms, and is what it uses now.
|
||||
|
||||
Two things follow. The defect was invisible to every round of review because all of them ran on
|
||||
macOS, where `xxd` exists — **the environment was a sampled population too**, and the only reason it
|
||||
surfaced is that a test asserted the guard must actually decide something. And two of the three CI
|
||||
failures were my *tests* being platform-specific (a bare `Terminated` job notice where macOS prints
|
||||
`Terminated: 15 sleep 5`; locales that do not exist on the runner, so bash warned and the
|
||||
assertion measured the missing locale rather than the fix). A test suite is code, and it inherits
|
||||
the same blind spot as the code it checks.
|
||||
|
||||
**The generalisable lesson is about the fixing, not the bugs.** This took three review rounds, and
|
||||
every round found that the previous round's *fix* had introduced something beside its target. The
|
||||
signal fix is the sharpest case: it corrected a lost log line and, in doing so, turned a killed
|
||||
`main`-protection guard into a rubber stamp. The rule this repo already has —
|
||||
`process.consistency-fix-new-code-scrutiny`, a fix is new code and gets reviewed as new code —
|
||||
applies with full force to instrumentation, because the fix and the thing it instruments occupy the
|
||||
same execution path.
|
||||
|
||||
Two of those fixes read as done and were not, which is the more useful pattern to recognise. The
|
||||
path-traversal hardening reused the record scrubber, which deliberately permits `/` and `.`, so
|
||||
`../../escaped` passed through untouched: **passing a value through a scrub is not the same as
|
||||
passing it through the right scrub.** And `exec … 2>/dev/null` was fixed twice and reintroduced a
|
||||
third time, eight lines below the comment forbidding it — which is why
|
||||
`test_no_exec_in_the_sink_carries_a_STDERR_REDIRECT` now enforces the rule instead of restating it.
|
||||
Enumerating instances loses to removing the class, and the third instance is what proves it.
|
||||
|
||||
One measurement note, since it cost a round of confusion: a shared `TMPDIR` between the two halves
|
||||
of the differential let the control's one-shot throttle marker suppress the instrumented run, so the
|
||||
test reported a swallowed decision that never happened. A differential whose halves share mutable
|
||||
state stops measuring the difference and measures the state.
|
||||
|
||||
**What the first measurement immediately produced**, which is the argument for the whole thing. A
|
||||
live run recorded `pretooluse-agent-model` firing on an `Agent` dispatch and deciding `no-op` where
|
||||
`ask` was expected. Under the old regime that would have been an unanswerable suspicion. Instead the
|
||||
question was settled by dumping the real payload: the dispatch carried `model: haiku`, so `no-op`
|
||||
was correct and there was no bug. Note what did *not* happen — no argument from the hook's source
|
||||
about what it "must" do, which this repo's record says has been wrong repeatedly. One printed line
|
||||
settled it in one run.
|
||||
|
||||
**The limit, stated rather than implied.** A zero in the report means *not observed in this window*,
|
||||
which is two different findings wearing one number: a hook that is broken, and a hook whose trigger
|
||||
genuinely did not occur. The report says so in its own footer rather than presenting a zero as a
|
||||
verdict. `pretooluse-nav-guard` and `pretooluse-merge-consent` are the standing examples — neither
|
||||
can be exercised without either a browser session or a real merge, so their zeroes are expected and
|
||||
distinguishing them stays a human judgement.
|
||||
+85
-16
@@ -474,23 +474,92 @@ and 5,102 `Read` calls over the same corpus, so the search was demonstrably capa
|
||||
evidence in transcripts (`husky - dotnet format found whitespace/BOM issues`, `husky - commit message
|
||||
missing Co-Authored-By trailer`, `husky - refusing to commit root-level screenshot(s)`).
|
||||
|
||||
### 5.4 We cannot currently tell whether our own hooks fire
|
||||
Wiring was the strongest claim available for the hook rows when this table was written. It is no
|
||||
longer the ceiling — §5.4 is now measured — so read the hook rows against
|
||||
`scripts/hook-fire-log.sh report`, not against this paragraph.
|
||||
|
||||
The most consequential Part 2 finding, and it was volunteered by the inventory as a limit on its own
|
||||
evidence. In this harness version only **`Stop`** hooks emit a structured record
|
||||
(`stop_hook_summary`/`hookInfos`). `PreToolUse` and `PostToolUse` hooks — which is *every* guard that
|
||||
matters here: merge consent, worktree, BOM, agent model/RAM — leave **no durable execution trace**.
|
||||
What the audit could count for those hooks was filename mentions in settings dumps and prior audits,
|
||||
which is not proof of execution. Only `design-sync-reminder.sh`'s Stop path and the three husky
|
||||
message strings quoted in §5.3 are confirmed fires; the `.claude/hooks/` scripts are confirmed
|
||||
*wired*, which is a different claim.
|
||||
### 5.4 Whether our own hooks fire — now measured (#776 closed this)
|
||||
|
||||
So the guards this repo relies on most are exactly the ones whose execution we cannot observe. That
|
||||
is Family B (the check never ran) applied to the hook layer itself, and this project has already
|
||||
paid for it twice at the CI layer — #751's guard that reported success in 6s having run nothing, and
|
||||
#756's fail-open twin. The standing lesson from #756 applies unchanged: **make the system report it
|
||||
rather than infer it.** A one-line append to a session-scoped log from each hook would convert every
|
||||
row in that table from inference to measurement, and it is cheap.
|
||||
**The finding as originally recorded.** In this harness version only `Stop` hooks emit a structured
|
||||
record (`stop_hook_summary`/`hookInfos`). `PreToolUse` and `PostToolUse` hooks — which is *every*
|
||||
guard that matters here: merge consent, worktree, BOM, agent model/RAM — leave no durable execution
|
||||
trace. What this audit could count for those hooks was filename mentions in settings dumps, which is
|
||||
not evidence of execution. So the guards this repo relies on most were exactly the ones whose
|
||||
execution could only be inferred: Family B (the check never ran) applied to the hook layer itself,
|
||||
after this project had already paid for it twice at the CI layer (#751, #756).
|
||||
|
||||
**What replaced the inference.** Every hook now records its own execution through one shared sink,
|
||||
`scripts/hook-fire-log.sh` (`testing.hook-reports-its-own-execution`): a `fire` record on entry and
|
||||
an `exit` record carrying the status and the decision, where **the decision is parsed from the bytes
|
||||
the hook actually emitted** rather than declared by its author. Read it with
|
||||
`scripts/hook-fire-log.sh report [--all]`.
|
||||
|
||||
**Measured 2026-08-14** — a scripted headless session (a plain Bash call, a Bash call carrying
|
||||
`ETV_UPDATE_GOLDENS=1`, a `Write` to a SPA file, and one `Agent` dispatch) plus a real `git commit`
|
||||
and a `git push --dry-run` in a worktree. Snapshot boundary: this is one deliberately-constructed
|
||||
window, not a corpus statistic.
|
||||
|
||||
| Hook | Event | Fires | Decisions observed |
|
||||
|---|---|---|---|
|
||||
| `pretooluse-bash-guard` | PreToolUse/Bash | 2 | `deny` 1, `no-op` 1 |
|
||||
| `pretooluse-bom-guard` | PreToolUse/Bash | 2 | `no-op` 2 |
|
||||
| `pretooluse-worktree-guard` | PreToolUse/Bash | 2 | `no-op` 2 |
|
||||
| `posttooluse-worktree-marker` | PostToolUse/Bash | 1 | `no-op` 1 |
|
||||
| `design-sync-reminder` | PreToolUse/Write + Stop | 4 | `context` 1, `block` 1, `no-op` 2 |
|
||||
| `pretooluse-agent-model` | PreToolUse/Agent | 2 | `no-op` 2 |
|
||||
| `pretooluse-agent-ram` | PreToolUse/Agent | 2 | `no-op` 2 |
|
||||
| `decisions-guard` | git pre-commit | 2 | `pass` 2 |
|
||||
| `prepush-clean-worktree-check` | git pre-push | 1 | `pass` 1 |
|
||||
| `prepush-donewhen` | git pre-push | 1 | `pass` 1 |
|
||||
| `prepush-rebase-check` | git pre-push | 1 | `pass` 1 |
|
||||
| `pretooluse-nav-guard` | PreToolUse/navigate | **0** | not exercised — needs a live browser session |
|
||||
| `pretooluse-merge-consent` | PreToolUse/PR write | **0** | not exercised — needs a real merge attempt |
|
||||
|
||||
**11 of 13 hooks are confirmed firing, with the decision each reached.** The `deny` row is the load-
|
||||
bearing one: `pretooluse-bash-guard` did not merely run, it *blocked* the `ETV_UPDATE_GOLDENS=1`
|
||||
probe, so at least one guard in this set is demonstrably live rather than merely present.
|
||||
|
||||
**Re-verified against the shipped implementation.** The table was first measured against an early
|
||||
version of the sink, and the classifier changed materially afterwards, so the run was repeated
|
||||
against the final code: `pretooluse-bash-guard` `deny`+`no-op`, `pretooluse-bom-guard` and
|
||||
`pretooluse-worktree-guard` `no-op`, `posttooluse-worktree-marker` `no-op`, `design-sync-reminder`
|
||||
`context`+`block`+`no-op` — identical decisions. The repeat run exercised the five Claude hooks a
|
||||
Bash/Write session reaches; the `Agent` pair and the four git hooks are carried over from the
|
||||
original run and were not re-measured.
|
||||
|
||||
**Reproducing this table.** It is a constructed window, not a corpus statistic, and it is not
|
||||
reproducible from a reader's own `report` output — running the test suite alone would not produce
|
||||
it, and before `scripts/tests/conftest.py` landed, running the suite actively polluted the default
|
||||
log with synthetic fires. To re-derive: point `ETV_HOOK_FIRE_LOG_DIR` at an empty directory, run a
|
||||
headless session exercising the four tool paths above, then a `git commit` and a
|
||||
`git push --dry-run`, and read `scripts/hook-fire-log.sh report --all --dir <that directory>`.
|
||||
|
||||
**A zero is two findings wearing one number** — a hook that is broken, and a hook whose trigger did
|
||||
not occur — and the report says so rather than presenting a zero as a verdict.
|
||||
|
||||
For the two zeroes here, what is established and what is not, kept apart deliberately.
|
||||
**Established:** both script bodies work. `pretooluse-nav-guard` and `pretooluse-merge-consent` are
|
||||
each driven to their deciding branch in `scripts/tests/test_hook_fire_log.py` — a `deny` on an
|
||||
`/iptv/` URL and an `ask` on a merge call — emitting the correct decision with the instrumentation
|
||||
in place. **Not established:** that the harness would dispatch to them. Those tests invoke the
|
||||
scripts directly, so they bypass registration and matcher dispatch entirely; a typo in
|
||||
`.claude/settings.json`, a settings file that never loaded, or a matcher that does not match would
|
||||
leave both tests green while the production zero still meant broken wiring. A working script is a
|
||||
necessary condition, not the finding.
|
||||
|
||||
So these two zeroes remain genuinely ambiguous, and neither can be resolved without the thing that
|
||||
resolves it: a live browser session, or a real merge attempt. Manufacturing a merge to observe the
|
||||
merge guard is a worse idea than the gap it would close. Distinguishing the two meanings of zero
|
||||
stays a human judgement — `testing.hook-reports-its-own-execution` says so, and an earlier draft of
|
||||
this paragraph quietly contradicted it by treating "the script works" as "the wiring works".
|
||||
|
||||
**The measurement earned its keep on its first run.** `pretooluse-agent-model` was recorded deciding
|
||||
`no-op` on an `Agent` dispatch where `ask` was expected. Previously that would have been an
|
||||
unanswerable suspicion about a guard nobody could observe; instead the real payload was dumped and
|
||||
it carried `model: haiku`, so `no-op` was correct and there was no defect. The general form is the
|
||||
standing #756 lesson — **make the system report it rather than infer it** — and note what it
|
||||
displaced: an argument from the hook's source about what it "must" do, which is the reasoning shape
|
||||
§3 measures going wrong repeatedly.
|
||||
|
||||
### 5.5 Tools we do *not* have that would address a named class
|
||||
|
||||
@@ -533,7 +602,7 @@ This issue is analysis; it spawns implementation rather than doing it.
|
||||
|---|---|---|---|
|
||||
| #774 | **A** — a guard derives its population from the authoritative source and asserts set equality | Family A, 27 records (39%) | high |
|
||||
| #775 | **B** — every guard ships a mutation proof: delete that guard alone, see red | Family B, 9 records (13%) | high |
|
||||
| #776 | Make hooks report that they fired — PreToolUse/PostToolUse execution is unobservable | the whole hook layer | high |
|
||||
| #776 | Make hooks report that they fired — PreToolUse/PostToolUse execution is unobservable | the whole hook layer | high — **DONE**, §5.4 is measured |
|
||||
| #777 | `csharp-lsp` and `typescript-lsp` are broken; the "workflow agents must use csharp-lsp" note is stale | partial mitigation for Family A | medium |
|
||||
| #778 | **D** — pin a version or use compare-and-set for read-then-write against live remote state | Family D, 5 records | medium |
|
||||
| #779 | **F + G** — test the deny path with the production config value; assert full-replace field lists | fail-open + destructive-replace, ~8 records | medium |
|
||||
|
||||
+23
-4
@@ -101,6 +101,7 @@ all.
|
||||
| `scripts/e2e-functional.sh` | the Functional E2E job, on a failed HTTP contract assertion | GUARD | NONE | — |
|
||||
| `scripts/e2e-local.sh` | nothing (boots a local instance) | TOOLING | NONE | — |
|
||||
| `scripts/e2e-ui.sh` | nothing (drives the Playwright flows) | TOOLING | NONE | — |
|
||||
| `scripts/hook-fire-log.sh` | nothing (records that each hook fired, and reports it) | TOOLING | NONE | — |
|
||||
| `scripts/jq-preflight.sh` | the `script-tests` job, on a jq version change | GUARD | BEHAVIOUR-ONLY | `test_jq_preflight.py::test_below_the_floor_is_LOUD` |
|
||||
| `scripts/post-review-verdict.sh` | nothing (writes the verdict status) | GUARD | BEHAVIOUR-ONLY | `test_post_review_verdict.py::test_never_retargets_the_verdict_at_the_new_head` |
|
||||
| `scripts/pr-changed-files.sh` | the verdict exemption, on an incomplete enumeration | GUARD | BEHAVIOUR-ONLY | `test_pr_changed_files.py::test_a_SHORT_page_does_not_end_the_enumeration` |
|
||||
@@ -114,6 +115,7 @@ all.
|
||||
| `scripts/tests/test_decisions_lib.py` | the `script-tests` job | PROOF | NONE | — |
|
||||
| `scripts/tests/test_decisions_validate.py` | the `script-tests` job | PROOF | NONE | — |
|
||||
| `scripts/tests/test_guard_inventory.py` | the `script-tests` job, on an unclassified guard or a stale proof ref | GUARD | MUTATION | `test_guard_inventory.py::test_the_inventory_covers_exactly_the_guards_that_exist` |
|
||||
| `scripts/tests/test_hook_fire_log.py` | the `script-tests` job, on a hook that stops reporting that it fired, or whose reporting changes what the harness sees | GUARD | MUTATION | `test_hook_fire_log.py::test_a_hook_that_LOSES_its_instrumentation_is_DETECTED` |
|
||||
| `scripts/tests/test_jq_preflight.py` | the `script-tests` job | PROOF | NONE | — |
|
||||
| `scripts/tests/test_merge_consent_base_change.py` | the `script-tests` job | PROOF | NONE | — |
|
||||
| `scripts/tests/test_merge_consent_exemption.py` | the `script-tests` job | PROOF | NONE | — |
|
||||
@@ -124,7 +126,7 @@ all.
|
||||
|
||||
## What the numbers say
|
||||
|
||||
31 guards, 4 tooling scripts, 13 proof files. **5 guards carry a mutation proof; 6 are
|
||||
32 guards, 5 tooling scripts, 13 proof files. **6 guards carry a mutation proof; 6 are
|
||||
behaviour-only; 20 have none.** These figures are asserted against the table by
|
||||
`test_the_summary_counts_match_the_table` — they were wrong in the first draft (28/4/6/3/19 against
|
||||
a table holding 27/5/6/3/18), because a hand-maintained summary of a table is a second copy of it,
|
||||
@@ -171,14 +173,31 @@ dangerous direction — a token that a reviewer means as `BLOCKED` silently read
|
||||
approval, which writes a green `review-verdict/h10`. A token added to one script and not the other
|
||||
is untested by either, which is the whole of #788.
|
||||
|
||||
**A guard was WITHDRAWN from `test_hook_fire_log.py` after four iterations**, recorded here because
|
||||
this is where a future session will look for it. A `test_no_redirection_in_the_sink_PRECEDES_its_stderr_redirect`
|
||||
generalised the `exec … 2>/dev/null` rule to any command by pattern-matching shell source. Each
|
||||
iteration fixed a false positive or false negative the previous one introduced — `>/dev/null 2>&1`
|
||||
flagged wrongly, then operators preceded by a digit (`exec 0<`) missed entirely, then multi-command
|
||||
lines flagged wrongly — and cold review then constructed more of both (`[[ "$x" < "$y" ]] 2>/dev/null`,
|
||||
a backslash continuation, a `>` inside a quoted string). Deleted rather than patched a fifth time,
|
||||
on the same reasoning as the vocabulary-parity withdrawal above.
|
||||
|
||||
**`test_hook_fire_log.py` asserts TWO clauses
|
||||
|
||||
**That unproven set carries no number here on purpose.** It restated the count, drifted the moment
|
||||
the BOM guard was regraded, and `test_the_summary_counts_match_the_table` cannot see it — the parser
|
||||
checks the formatted summary sentence and nothing else. A second hand-maintained copy of a number is
|
||||
the duplication family this file argues against, so the copy is removed rather than corrected: the
|
||||
set is every `PreToolUse` hook except merge-consent and the BOM guard, and the count is in the
|
||||
summary above. It is also the population #776 found we
|
||||
cannot even observe firing. A guard that is neither proven nor observable is an assertion about the
|
||||
past.
|
||||
summary above.
|
||||
|
||||
**They are now observable but still unproven, and the two words carry different weight.** Observable:
|
||||
every hook records its own execution through `scripts/hook-fire-log.sh`, so "did this hook fire, and
|
||||
what did it decide" is a measurement — run `scripts/hook-fire-log.sh report` (#776). Unproven:
|
||||
nobody has demonstrated any of them is load-bearing, which is what #785 tracks. Observability tells
|
||||
you a guard ran; only a mutation tells you it would have caught anything. The BOM guard is the case
|
||||
that shows why the distinction matters — it was firing on every commit the whole time it was
|
||||
fail-open.
|
||||
|
||||
The gaps are not uniform in cost, and the ranking that matters is *what a silent failure would let
|
||||
through*, not test count:
|
||||
|
||||
Executable
+557
@@ -0,0 +1,557 @@
|
||||
#!/usr/bin/env bash
|
||||
# Hook fire log — the WRITE side and the READ side, deliberately in ONE file (ersatztv#776).
|
||||
#
|
||||
# WHY THIS EXISTS. In this harness version only `Stop` hooks emit a structured transcript record
|
||||
# (`stop_hook_summary`/`hookInfos`). `PreToolUse` and `PostToolUse` hooks leave no durable execution
|
||||
# trace — which is every guard this repo actually relies on. #773 §5.4 could therefore only count
|
||||
# *filename mentions in settings dumps*, i.e. inference. A guard that is neither proven nor
|
||||
# observable is an assertion about the past, and a silently broken hook looks identical to a working
|
||||
# one indefinitely. #756's standing lesson: make the system REPORT it rather than infer it.
|
||||
#
|
||||
# WHY ONE FILE. The reader and the writer share the record format. Two files means the format can
|
||||
# drift and the report can quietly describe a shape nothing writes any more — the same argument that
|
||||
# put `scripts/ci-step-ran.sh` in one script instead of an inline workflow body (#756).
|
||||
#
|
||||
# WHY IT CAPTURES STDOUT RATHER THAN BEING TOLD THE DECISION. Every hook here exits 0 always; the
|
||||
# decision is communicated by *printing* `hookSpecificOutput.permissionDecision` (PreToolUse) or
|
||||
# `decision` (Stop). `pretooluse-merge-consent.sh` alone reaches that print from ~40 call sites via
|
||||
# its `decide` helper. Asking each site to also set a variable would (a) be 40 edits in the most
|
||||
# load-bearing guard in the repo and (b) record what the author *meant*, which is the inference this
|
||||
# issue exists to abolish. Capturing the bytes the hook actually emits records what the HARNESS
|
||||
# sees. It cannot drift from the decision because it IS the decision.
|
||||
#
|
||||
# WHY IT CAPTURES STDIN. 8 of the 9 hooks already open with `input=$(cat)` — a full blocking slurp —
|
||||
# so reading stdin once here and replaying it via `exec 0<` is not a new risk, it is the read they
|
||||
# already perform, moved earlier. It buys the tool name and session id for the log, and it makes
|
||||
# `pretooluse-agent-ram.sh` (which reads no stdin at all today) observable on the same terms as the
|
||||
# rest instead of being a hole in the table.
|
||||
#
|
||||
# FAIL-OPEN, DELIBERATELY AND IN THIS DIRECTION ONLY. This file is observability, not a guard. If
|
||||
# anything here fails — no temp dir, unwritable log, missing `date` — the hook must behave EXACTLY as
|
||||
# it did before instrumentation. A logging bug that denies a merge, or that swallows a guard's deny
|
||||
# JSON, would be far worse than the blindness it is fixing. Every function returns 0, the stdout
|
||||
# replay is the first act of the exit path, and the original exit code is re-raised explicitly.
|
||||
#
|
||||
# USAGE (write side) — the first two lines of every hook, before anything reads stdin:
|
||||
# . "${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)}/scripts/hook-fire-log.sh"
|
||||
# etv_hook_fire_begin <hook-name> [label]
|
||||
#
|
||||
# USAGE (read side):
|
||||
# scripts/hook-fire-log.sh report # this session (or every session, see below)
|
||||
# scripts/hook-fire-log.sh report --all # every session in the log dir
|
||||
# scripts/hook-fire-log.sh report --json
|
||||
# scripts/hook-fire-log.sh path # where the current session logs
|
||||
#
|
||||
# `scripts/tests/test_hook_fire_log.py` is the guard: it derives the hook population from the
|
||||
# filesystem (never a list — `testing.guard-derives-population-from-source`) and fails if any hook
|
||||
# is uninstrumented, and it proves stdin, stdout and the exit code survive the wrapper.
|
||||
|
||||
# ---------------------------------------------------------------------------------------------
|
||||
# Shared: where the log lives, and how a record is written
|
||||
# ---------------------------------------------------------------------------------------------
|
||||
|
||||
etv_hook_fire_log_dir() {
|
||||
printf '%s' "${ETV_HOOK_FIRE_LOG_DIR:-${HOME:-/tmp}/.cache/ersatztv/hook-fire}"
|
||||
}
|
||||
|
||||
etv_hook_fire_session() {
|
||||
# The harness does not export a session id to hooks in this version, so the payload's
|
||||
# `.session_id` is the real source and the env var is only a fallback. Recorded rather than
|
||||
# assumed: several hooks already read `.session_id` from stdin for exactly this reason.
|
||||
printf '%s' "${ETV_HOOK_FIRE_SESSION:-${CLAUDE_SESSION_ID:-unknown-session}}"
|
||||
}
|
||||
|
||||
etv_hook_fire_log_file() {
|
||||
# SCRUBBED, like every other use of this value. The session id is extracted from the payload by a
|
||||
# `sed` that only excludes `"`, so a `/` or `..` in it would otherwise steer the write outside the
|
||||
# log dir. Harness-generated UUIDs make that unreachable today, but a file whose stated thesis is
|
||||
# "restrict the value space so there is no escaping bug to have" should not exempt the one use
|
||||
# that becomes a path.
|
||||
printf '%s/%s.jsonl' "$(etv_hook_fire_log_dir)" "$(etv_hook_fire_scrub_component "$(etv_hook_fire_session)")"
|
||||
}
|
||||
|
||||
# Sanitise a value to a safe JSON scalar charset. This is why no `jq` is needed on the write side:
|
||||
# with the value space restricted there is nothing to escape, so there is no escaping bug to have.
|
||||
# A quote, backslash or newline in a hook name or tool name is not a case worth supporting — it is a
|
||||
# case worth flattening, loudly, to `_`.
|
||||
etv_hook_fire_scrub() {
|
||||
printf '%s' "${1:-}" | tr -c 'A-Za-z0-9._/:+@=-' '_' | cut -c1-200
|
||||
}
|
||||
|
||||
# A STRICTER scrub for the one value that becomes a PATH. The record scrub above deliberately keeps
|
||||
# `/` and `.` — tool names like `mcp__gitea__x` and event paths read better with them — but those
|
||||
# are exactly the two characters that turn a session id into `../../escaped`. Passing a value
|
||||
# through a scrub is not the same as passing it through the RIGHT scrub, and the first version of
|
||||
# this fix reused the record scrubber and left the traversal wide open while reading as fixed.
|
||||
etv_hook_fire_scrub_component() {
|
||||
printf '%s' "${1:-}" | tr -c 'A-Za-z0-9_-' '_' | cut -c1-120
|
||||
}
|
||||
|
||||
# Append one record. Never fails; never writes a partial line (built whole, appended once).
|
||||
etv_hook_fire_record() {
|
||||
[ "${ETV_HOOK_FIRE_DISABLE:-0}" = "1" ] && return 0
|
||||
|
||||
local file line
|
||||
file="${ETV_HOOK_FIRE_LOG_FILE:-$(etv_hook_fire_log_file)}"
|
||||
mkdir -p "$(dirname "$file")" 2>/dev/null || return 0
|
||||
|
||||
line=$(printf '{"ts":"%s","session":"%s","pid":"%s","hook":"%s","label":"%s","event":"%s","tool":"%s","phase":"%s","code":"%s","decision":"%s"}' \
|
||||
"$(date -u '+%Y-%m-%dT%H:%M:%SZ' 2>/dev/null || printf 'unknown')" \
|
||||
"$(etv_hook_fire_scrub "$(etv_hook_fire_session)")" \
|
||||
"$(etv_hook_fire_scrub "${ETV_HOOK_FIRE_PID:-$$}")" \
|
||||
"$(etv_hook_fire_scrub "${1:-}")" \
|
||||
"$(etv_hook_fire_scrub "${2:-}")" \
|
||||
"$(etv_hook_fire_scrub "${3:-}")" \
|
||||
"$(etv_hook_fire_scrub "${4:-}")" \
|
||||
"$(etv_hook_fire_scrub "${5:-}")" \
|
||||
"$(etv_hook_fire_scrub "${6:-}")" \
|
||||
"$(etv_hook_fire_scrub "${7:-}")" 2>/dev/null) || return 0
|
||||
|
||||
# ORDER MATTERS: `2>` BEFORE `>>`. Redirections are applied left to right, so
|
||||
# `printf ... >> "$file" 2>/dev/null` opens the file FIRST and bash reports a failure to open it
|
||||
# on the stderr still in force — the hook prints `Operation not permitted` at the harness. An
|
||||
# existing-but-unwritable log file is the reachable case; the fail-open test missed it by using a
|
||||
# path that dies at `mkdir` instead. Redirecting stderr first covers the open failure too.
|
||||
printf '%s\n' "$line" 2>/dev/null >> "$file" || true
|
||||
return 0
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------------------------
|
||||
# Write side: what a hook calls
|
||||
# ---------------------------------------------------------------------------------------------
|
||||
|
||||
# Classify what the hook printed. The harness reads these two shapes and nothing else:
|
||||
# PreToolUse : {"hookSpecificOutput":{"permissionDecision":"allow|deny|ask", ...}}
|
||||
# Stop : {"decision":"block", "reason": ...}
|
||||
# Anything else printed on stdout is surfaced to the user but decides nothing.
|
||||
etv_hook_fire_classify() {
|
||||
local out="${1:-}" code="${2:-0}" mode="${3:-capture}" d=""
|
||||
# BYTE-ORIENTED, not locale-aware. One 0xE9 byte in a `permissionDecisionReason` made `sed` print
|
||||
# `illegal byte sequence` to the harness AND fail to extract, filing a real `deny` as `output`.
|
||||
#
|
||||
# `local -x`, NOT `local`. A plain `local` sets a shell variable without the export attribute, so
|
||||
# the child `sed`/`tr` never sees it — the fix was INERT and read as applied. It looked correct
|
||||
# only because this author's shell sets `LANG` alone; with an inherited `LC_CTYPE` (macOS
|
||||
# Terminal, ssh `SendEnv LC_*`, sudo `env_keep`) `LC_CTYPE` outranks the exported `LANG=C` and the
|
||||
# symptom returns in full.
|
||||
local -x LC_ALL=C LANG=C
|
||||
|
||||
# A `stream`-mode hook is a git hook: git reads its exit code and nothing else, so that is the
|
||||
# whole of its decision. Reporting `no-op` here because no JSON was captured would be an
|
||||
# inference, and inference is what this file exists to replace.
|
||||
if [ "$mode" = "stream" ]; then
|
||||
if [ "$code" = "0" ]; then printf 'pass'; else printf 'blocked'; fi
|
||||
return 0
|
||||
fi
|
||||
|
||||
if [ -n "$out" ]; then
|
||||
# `[^"]*`, not `[A-Za-z-]*`: a restricted class means an odd value fails to EXTRACT and is
|
||||
# filed as generic `output`, so "every non-canonical value is recorded as unrecognized" was
|
||||
# true only for values the class happened to admit. Extract anything, then judge it below.
|
||||
d=$(printf '%s' "$out" | sed -n 's/.*"permissionDecision"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' | head -n1)
|
||||
# A PRESENT-but-empty value is not the same as an absent key, and treating them alike let
|
||||
# `{"permissionDecision":""}` fall through to `output` — a malformed decision laundered into
|
||||
# "the hook just printed something".
|
||||
if [ -z "$d" ] && printf '%s' "$out" | grep -q '"permissionDecision"[[:space:]]*:'; then
|
||||
d="unrecognized"
|
||||
fi
|
||||
[ -z "$d" ] && d=$(printf '%s' "$out" | sed -n 's/.*"decision"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' | head -n1)
|
||||
|
||||
# A NON-CANONICAL VALUE IS RECORDED AS SUCH, not normalised into a valid one. An earlier version
|
||||
# lowercased whatever it found, so `"permissionDecision":"Deny"` was filed as a clean `deny` —
|
||||
# manufacturing a decision the harness may never have honoured. The documented values are
|
||||
# lowercase; anything else is a hook bug, and the log should say so rather than launder it.
|
||||
case "$d" in
|
||||
allow|deny|ask|block|"") ;;
|
||||
*) d="unrecognized" ;;
|
||||
esac
|
||||
# `additionalContext` with no decision (design-sync-reminder's `start` path) informs, it does
|
||||
# not gate. Naming it distinctly keeps it out of the allow/deny counts. Matched WITH its quotes
|
||||
# and colon — a bare substring test also fired on human prose that merely used the word.
|
||||
if [ -z "$d" ] && printf '%s' "$out" | grep -q '"additionalContext"[[:space:]]*:'; then d="context"; fi
|
||||
[ -z "$d" ] && d="output"
|
||||
fi
|
||||
|
||||
# Exit 2 is the harness's block channel and it DOMINATES: the call is blocked whatever the JSON
|
||||
# said. Recording a printed `allow` here would report a permit for an invocation that was refused,
|
||||
# which is the one direction a log of security decisions must never be wrong in.
|
||||
if [ "$code" = "2" ]; then printf 'deny-exit2'; return 0; fi
|
||||
|
||||
[ -z "$d" ] && d="no-op"
|
||||
|
||||
# A non-zero status does NOT annotate a parsed decision. It used to overwrite one (`deny` + exit 1
|
||||
# recorded as `error`), and the first correction over-swung the other way and appended `+error` —
|
||||
# inventing a composite state the harness does not report. The status already has its own field in
|
||||
# the record, so the decision field states the decision and nothing else. `error` means only "it
|
||||
# failed and said nothing classifiable".
|
||||
# `output` too, not just `no-op`: a hook that printed a diagnostic and then FAILED was being
|
||||
# filed as `output`, so the report showed `output=1` and the failure vanished from the histogram.
|
||||
# A parsed decision is still left alone — the status has its own field.
|
||||
case "$code:$d" in 0:*) ;; *:no-op|*:output) d="error" ;; esac
|
||||
|
||||
printf '%s' "$d"
|
||||
}
|
||||
|
||||
# etv_hook_fire_begin <hook-name> [label] [stdout-mode]
|
||||
#
|
||||
# `stdout-mode` is `capture` (default) or `stream`, and the split is not a tuning knob — the two
|
||||
# kinds of hook in this repo decide by different channels:
|
||||
#
|
||||
# Claude Code hooks (pretooluse-*, posttooluse-*, design-sync-reminder) always exit 0 and
|
||||
# communicate by PRINTING JSON. Their decision is only observable by capturing stdout, and their
|
||||
# output is a single line consumed by the harness after exit, so buffering costs nothing.
|
||||
#
|
||||
# Git hooks (prepush-*, decisions-guard) decide by EXIT CODE, and their stdout is progress text a
|
||||
# human is watching in real time. Capturing it would hold a slow pre-push hook's output back until
|
||||
# the end, turning a working progress display into an apparent hang. They pass `stream`, and their
|
||||
# decision is read from the exit code, which is what git reads too.
|
||||
etv_hook_fire_begin() {
|
||||
ETV_HOOK_FIRE_NAME="${1:-unknown-hook}"
|
||||
ETV_HOOK_FIRE_LABEL="${2:-}"
|
||||
ETV_HOOK_FIRE_MODE="${3:-capture}"
|
||||
ETV_HOOK_FIRE_PID="$$"
|
||||
ETV_HOOK_FIRE_STDIN_TMP=""
|
||||
ETV_HOOK_FIRE_STDOUT_TMP=""
|
||||
# RESET, never merely default. Inherited from the environment (exported by a parent, or a second
|
||||
# `begin` in one shell) a stale `1` made the first flush return immediately: stdout stayed
|
||||
# redirected and no exit record was ever written.
|
||||
ETV_HOOK_FIRE_FLUSHED=0
|
||||
|
||||
[ "${ETV_HOOK_FIRE_DISABLE:-0}" = "1" ] && return 0
|
||||
|
||||
ETV_HOOK_FIRE_LOG_FILE="$(etv_hook_fire_log_file)"
|
||||
mkdir -p "$(dirname "$ETV_HOOK_FIRE_LOG_FILE")" 2>/dev/null || { ETV_HOOK_FIRE_DISABLE=1; return 0; }
|
||||
|
||||
# --- stdin: slurp, replay, and read the payload's identifying fields ------------------------
|
||||
#
|
||||
# NEVER on a terminal. A `git commit` run interactively hands its hooks a TTY on fd 0, and `cat`
|
||||
# would block there forever — instrumentation hanging the commit it was added to observe. Claude
|
||||
# Code always writes the JSON payload and closes the pipe, which is why the 8 hooks that already
|
||||
# open with `input=$(cat)` are safe today; that guarantee does not extend to the git hooks, so the
|
||||
# capture is conditioned on stdin not being a tty rather than on which hook is calling.
|
||||
local sin payload="" event="" tool="" sess=""
|
||||
if [ ! -t 0 ]; then
|
||||
sin=$(mktemp "${TMPDIR:-/tmp}/etv-hook-stdin.XXXXXX" 2>/dev/null) || sin=""
|
||||
else
|
||||
sin=""
|
||||
fi
|
||||
if [ -n "$sin" ]; then
|
||||
ETV_HOOK_FIRE_STDIN_TMP="$sin"
|
||||
cat 2>/dev/null > "$sin" || true
|
||||
# Replay: even a partial capture is closer to the truth than the drained pipe the hook would
|
||||
# otherwise inherit.
|
||||
#
|
||||
# NO `2>/dev/null` ON THIS LINE, EVER. `exec` with redirections and no command applies them to
|
||||
# the shell PERMANENTLY, so `exec 0<"$sin" 2>/dev/null` does not suppress errors from this one
|
||||
# redirection — it sends the HOOK'S ENTIRE STDERR to /dev/null for the rest of its life. That
|
||||
# silenced every husky hook's user-facing output, which is stderr: the H6 "push to main BLOCKED"
|
||||
# message, the BOM guard's remediation text, `husky - commit message missing Co-Authored-By`.
|
||||
# The guards still blocked, and the human was told nothing about why.
|
||||
# Readability is tested instead of relying on redirection-failure suppression.
|
||||
if [ -r "$sin" ]; then exec 0<"$sin" || true; fi
|
||||
payload=1
|
||||
fi
|
||||
|
||||
if [ -n "$payload" ]; then
|
||||
# Read the fields from the FILE, byte-oriented, with no size cap. These were extracted from a
|
||||
# `head -c 65536` prefix, so a payload whose `tool_response` pushed `session_id` past 64 KB
|
||||
# filed its records under `unknown-session` with empty event and tool — and a report keyed on
|
||||
# the real session then showed those fires as NEVER HAVING HAPPENED. A truncating read is a
|
||||
# sampling error, and this one manufactured exactly the vacuity #776 exists to abolish.
|
||||
local -x LC_ALL=C LANG=C
|
||||
# FIRST occurrence, via `grep -o`. A `sed` substitution with a leading `.*` is GREEDY, and
|
||||
# payloads are one long line, so it selected the LAST match: a nested
|
||||
# `{"session_id":"...","tool_name":"..."}` inside a `tool_response` outranked the top-level one
|
||||
# and the whole invocation filed under the wrong session. Removing the 64 KB cap is what armed
|
||||
# it — the cap had been accidentally protecting the right answer, which is the kind of load a
|
||||
# bound can be silently carrying.
|
||||
#
|
||||
# The scan is bounded again at 256 KB, but now the bound is safe rather than load-bearing:
|
||||
# identity fields are at the head of the payload, and first-match means a later duplicate cannot
|
||||
# win. Unbounded cost 0.5s per scan on a 20 MB payload, three scans per fire.
|
||||
# A BOUNDED FAST PATH WITH AN UNBOUNDED FALLBACK. A plain cap is a truncating read, and a
|
||||
# truncating read is a sampling error: a payload whose `tool_response` pushes `session_id` past
|
||||
# the cap returns nothing, the record files under `unknown-session`, and that fire reads as
|
||||
# NEVER HAVING HAPPENED — the false vacuity this whole change exists to remove, reintroduced by
|
||||
# the bound added to make it fast. So the cap is an optimisation only: if the prefix yields
|
||||
# nothing, the whole payload is scanned. Identity fields sit at the head in practice, so the
|
||||
# fallback is rare; correctness no longer depends on that being true.
|
||||
#
|
||||
# `etv_hook_fire__field`, not `_etv_field`: a function defined inside another is still GLOBAL in
|
||||
# bash, so a short generic name leaks into the hook's namespace and can collide with something
|
||||
# the hook defines. It is unset after use.
|
||||
etv_hook_fire__field() {
|
||||
local v
|
||||
v=$(head -c 262144 "$sin" 2>/dev/null \
|
||||
| grep -o "\"$1\"[[:space:]]*:[[:space:]]*\"[^\"]*\"" 2>/dev/null \
|
||||
| head -n1 | sed 's/.*:[[:space:]]*"//; s/"$//' 2>/dev/null)
|
||||
if [ -z "$v" ]; then
|
||||
v=$(grep -o "\"$1\"[[:space:]]*:[[:space:]]*\"[^\"]*\"" "$sin" 2>/dev/null \
|
||||
| head -n1 | sed 's/.*:[[:space:]]*"//; s/"$//' 2>/dev/null)
|
||||
fi
|
||||
printf '%s' "$v"
|
||||
}
|
||||
event=$(etv_hook_fire__field hook_event_name)
|
||||
tool=$(etv_hook_fire__field tool_name)
|
||||
sess=$(etv_hook_fire__field session_id)
|
||||
unset -f etv_hook_fire__field 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# The payload's session id is authoritative when the env var is absent, which it is in this
|
||||
# harness version. Re-point the log file at it before the first record is written.
|
||||
if [ -n "$sess" ] && [ -z "${CLAUDE_SESSION_ID:-}" ] && [ -z "${ETV_HOOK_FIRE_SESSION:-}" ]; then
|
||||
ETV_HOOK_FIRE_SESSION="$sess"
|
||||
ETV_HOOK_FIRE_LOG_FILE="$(etv_hook_fire_log_file)"
|
||||
mkdir -p "$(dirname "$ETV_HOOK_FIRE_LOG_FILE")" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
ETV_HOOK_FIRE_EVENT="$event"
|
||||
ETV_HOOK_FIRE_TOOL="$tool"
|
||||
|
||||
etv_hook_fire_record "$ETV_HOOK_FIRE_NAME" "$ETV_HOOK_FIRE_LABEL" "$event" "$tool" "fire" "" ""
|
||||
|
||||
# --- stdout: divert, so the exit path can read the decision the hook emitted ----------------
|
||||
local sout=""
|
||||
if [ "$ETV_HOOK_FIRE_MODE" = "capture" ]; then
|
||||
sout=$(mktemp "${TMPDIR:-/tmp}/etv-hook-stdout.XXXXXX" 2>/dev/null) || sout=""
|
||||
fi
|
||||
if [ -n "$sout" ]; then
|
||||
ETV_HOOK_FIRE_STDOUT_TMP="$sout"
|
||||
exec 3>&1 || { ETV_HOOK_FIRE_STDOUT_TMP=""; rm -f "$sout" 2>/dev/null; return 0; }
|
||||
exec 1>"$sout" || { exec 1>&3 3>&-; ETV_HOOK_FIRE_STDOUT_TMP=""; rm -f "$sout" 2>/dev/null; return 0; }
|
||||
# A READ FD held open on the same file, so the replay does not depend on the PATH still
|
||||
# resolving at exit. Replaying with `cat "$path"` loses everything if anything unlinks the file
|
||||
# mid-run — a $TMPDIR reaper, a hook that clears its own scratch — because the write fd stays
|
||||
# valid on the unlinked inode while the name is gone: the hook prints its `deny` into a file
|
||||
# nothing can reopen. An fd survives unlink, which turns that from silent guard-disarming into
|
||||
# a non-event. Pinned by `test_output_SURVIVES_a_vanished_stdout_tempfile`.
|
||||
exec 4<"$sout" || true
|
||||
fi
|
||||
|
||||
# SIGNALS ARE DELIBERATELY NOT TRAPPED — a withdrawal, recorded because the next reader will
|
||||
# otherwise re-add this. A `trap ... TERM` was added so a hook killed by the harness timeout would
|
||||
# not lose output it had already printed (measured then at 52 bytes before, 0 after). It produced
|
||||
# three defects in three rounds and cost more than it bought:
|
||||
#
|
||||
# 1. the handler ended in `exit "$?"`, so a SIGTERMed `prepush-donewhen.sh` reported 0 and git
|
||||
# PUSHED TO MAIN — a signal became consent;
|
||||
# 2. `local sig=...` clobbered `$?`, so a killed guard was RECORDED as having passed;
|
||||
# 3. and the one that settles it: bash does not run a trap until the current foreground command
|
||||
# finishes, so a hook mid-`curl` took 30s to die where it had taken 1s. A TERM-then-KILL
|
||||
# supervisor therefore gets no flush AT ALL, plus a 29s stall — strictly worse than the
|
||||
# untrapped behaviour on the very path the trap existed for.
|
||||
#
|
||||
# WHAT IS LOST, enumerated rather than waved past: a hook killed by a signal loses its `exit`
|
||||
# record, loses stdout it had already written, and leaks its two temp files. The first is a real
|
||||
# gap in the log and is why `report` counts `fire` records, not `exit` records. The second is
|
||||
# near-unreachable — every capture-mode hook prints its decision and exits immediately, a
|
||||
# sub-millisecond window. The third is bounded by $TMPDIR cleanup.
|
||||
#
|
||||
# The invariant that replaces the rescue is stronger and is what the tests now assert: under a
|
||||
# signal an instrumented hook behaves EXACTLY as an uninstrumented one.
|
||||
trap 'etv_hook_fire_end "$?"' EXIT
|
||||
return 0
|
||||
}
|
||||
|
||||
# Flush and record, WITHOUT deciding how the process ends. `ETV_HOOK_FIRE_FLUSHED` keeps it
|
||||
# idempotent. That guard is retained after the signal traps were withdrawn, because it is cheap and
|
||||
# because the failure it prevents — two contradictory `exit` records for one invocation, a real
|
||||
# `deny` followed by a phantom `no-op` — is silent, and re-entrancy would return the moment anyone
|
||||
# adds a second caller.
|
||||
etv_hook_fire_flush() {
|
||||
local code="${1:-0}" out=""
|
||||
# The `tr -d '\000'` below is a child process too, and it sat outside every locale declaration:
|
||||
# `tr: Illegal byte sequence` reached the harness and truncated the classification copy.
|
||||
local -x LC_ALL=C LANG=C
|
||||
[ "${ETV_HOOK_FIRE_FLUSHED:-0}" = "1" ] && return 0
|
||||
ETV_HOOK_FIRE_FLUSHED=1
|
||||
|
||||
# RESTORE FIRST, UNCONDITIONALLY — then replay if there is anything to replay.
|
||||
#
|
||||
# These were one conditional, `[ -n "$TMP" ] && [ -f "$TMP" ]`, and that coupling was the bug:
|
||||
# `-f` asks "is there output to replay", but the fd restore must happen whenever the redirect was
|
||||
# ESTABLISHED, which is a different fact. Any route that makes `-f` false while the redirect is
|
||||
# live — the temp file unlinked by a $TMPDIR reaper mid-run (the fd stays valid, so the hook
|
||||
# writes happily into an unlinked inode), a $TMPDIR where mktemp yields a non-regular file — left
|
||||
# fd 1 still pointing at the temp target with the restore skipped, so nothing later could rescue
|
||||
# the bytes. A guard's `deny` was silently discarded and the hook still exited 0.
|
||||
#
|
||||
# Split, the worst case degrades from "the guard is disarmed" to "the log is short", which is the
|
||||
# correct failure direction for observability. Pinned by
|
||||
# `test_output_SURVIVES_a_vanished_stdout_tempfile`.
|
||||
if [ -n "${ETV_HOOK_FIRE_STDOUT_TMP:-}" ]; then
|
||||
# Same rule as the stdin replay: a trailing `2>/dev/null` here would permanently silence stderr
|
||||
# rather than suppress an error from this redirection. fd 3 is known open on this path.
|
||||
exec 1>&3 3>&- || true
|
||||
fi
|
||||
if [ -n "${ETV_HOOK_FIRE_STDOUT_TMP:-}" ]; then
|
||||
# ONE read, used for BOTH the replay and the classification, so the two cannot disagree. They
|
||||
# did: the classifier read fd 4 while the replay preferred the file, so a hook that used fd 4
|
||||
# itself replayed its `deny` correctly to the harness and filed it in the log as `no-op` — the
|
||||
# log quietly contradicting the decision it exists to record.
|
||||
#
|
||||
# The `printf X` / `%X` dance preserves trailing newlines, which `$(...)` strips. Without it the
|
||||
# rescue path delivered 52 bytes where the hook wrote 53, and every JSON parser downstream
|
||||
# accepts the short form without complaint. Pinned by `test_stdout_is_replayed_BYTE_EXACT`.
|
||||
#
|
||||
# NO `2>/dev/null` ON THE `exec` — see the stdin comment. An earlier version of THIS line had
|
||||
# it, eight lines below the comment forbidding it, which is why the rule is now restated at
|
||||
# every `exec` rather than once.
|
||||
if [ -r "${ETV_HOOK_FIRE_STDOUT_TMP:-}" ]; then
|
||||
# Common path: stream the FILE straight through. A shell variable cannot hold a NUL byte, so
|
||||
# replaying via `$(...)` silently drops them and warns on stderr; `cat` is byte-exact for any
|
||||
# content. Classification reads the SAME file, so the two cannot disagree — reading them from
|
||||
# different sources is what made the harness see `deny` while the log recorded `no-op`.
|
||||
cat "$ETV_HOOK_FIRE_STDOUT_TMP" 2>/dev/null || true
|
||||
# `tr -d '\000'` before the substitution: bash cannot hold a NUL in a variable and warns
|
||||
# about it ON STDERR, which the harness sees — an instrumentation message leaking into a
|
||||
# guard's output channel. Classification does not care about NULs; the replay above is
|
||||
# byte-exact regardless, because it streams the file rather than a variable.
|
||||
out=$( { head -c 65536 "$ETV_HOOK_FIRE_STDOUT_TMP" 2>/dev/null | tr -d '\000'; printf 'X'; } )
|
||||
out="${out%X}"
|
||||
else
|
||||
# Rescue path: the name is gone but the fd still reads the unlinked inode. This one goes
|
||||
# through a variable and is therefore NUL-lossy — stated rather than hidden, because the
|
||||
# alternative is losing the output entirely.
|
||||
out=$( { cat 2>/dev/null <&4 | tr -d '\000'; printf 'X'; } )
|
||||
out="${out%X}"
|
||||
[ -n "$out" ] && printf '%s' "$out"
|
||||
fi
|
||||
exec 4<&- || true
|
||||
fi
|
||||
|
||||
etv_hook_fire_record \
|
||||
"${ETV_HOOK_FIRE_NAME:-unknown-hook}" "${ETV_HOOK_FIRE_LABEL:-}" \
|
||||
"${ETV_HOOK_FIRE_EVENT:-}" "${ETV_HOOK_FIRE_TOOL:-}" \
|
||||
"exit" "$code" "$(etv_hook_fire_classify "$out" "$code" "${ETV_HOOK_FIRE_MODE:-capture}")" || true
|
||||
|
||||
rm -f "${ETV_HOOK_FIRE_STDIN_TMP:-}" "${ETV_HOOK_FIRE_STDOUT_TMP:-}" 2>/dev/null || true
|
||||
ETV_HOOK_FIRE_STDOUT_TMP=""
|
||||
return 0
|
||||
}
|
||||
|
||||
etv_hook_fire_end() {
|
||||
local code="${1:-0}"
|
||||
etv_hook_fire_flush "$code"
|
||||
# Re-raise the hook's own status explicitly rather than relying on the trap preserving it. Bash
|
||||
# does not re-enter an EXIT trap, so this is not recursive.
|
||||
exit "$code"
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------------------------
|
||||
# Read side: the report
|
||||
# ---------------------------------------------------------------------------------------------
|
||||
|
||||
# The hook population, DERIVED from the filesystem, never listed
|
||||
# (`testing.guard-derives-population-from-source`). A hook added tomorrow appears in the report as
|
||||
# NEVER-FIRED the moment it exists, rather than being invisible because nobody updated an array.
|
||||
etv_hook_fire_population() {
|
||||
local root="${1:-}" f
|
||||
[ -d "$root/.claude/hooks" ] || return 0
|
||||
for f in "$root"/.claude/hooks/*.sh; do
|
||||
[ -f "$f" ] || continue
|
||||
basename "$f" .sh
|
||||
done
|
||||
}
|
||||
|
||||
etv_hook_fire_repo_root() {
|
||||
if [ -n "${CLAUDE_PROJECT_DIR:-}" ] && [ -d "$CLAUDE_PROJECT_DIR/.claude/hooks" ]; then
|
||||
printf '%s' "$CLAUDE_PROJECT_DIR"
|
||||
return 0
|
||||
fi
|
||||
( cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." >/dev/null 2>&1 && pwd )
|
||||
}
|
||||
|
||||
etv_hook_fire_report() {
|
||||
local all=0 as_json=0 dir root files hook
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--all) all=1 ;;
|
||||
--json) as_json=1 ;;
|
||||
--dir) shift; ETV_HOOK_FIRE_LOG_DIR="${1:-}" ;;
|
||||
--session) shift; ETV_HOOK_FIRE_SESSION="${1:-}" ;;
|
||||
*) printf 'hook-fire-log: unknown report option %s\n' "$1" >&2; return 2 ;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
dir="$(etv_hook_fire_log_dir)"
|
||||
root="$(etv_hook_fire_repo_root)"
|
||||
|
||||
if [ "$all" = "1" ]; then
|
||||
files=$(find "$dir" -name '*.jsonl' -type f 2>/dev/null | sort)
|
||||
else
|
||||
files="$(etv_hook_fire_log_file)"
|
||||
[ -f "$files" ] || files=""
|
||||
fi
|
||||
|
||||
if [ -z "$files" ]; then
|
||||
printf 'hook-fire-log: no records under %s%s\n' "$dir" \
|
||||
"$([ "$all" = "1" ] || printf ' for session %s (try --all)' "$(etv_hook_fire_session)")" >&2
|
||||
fi
|
||||
|
||||
# ANTI-VACUITY. A report over an empty population would print a clean table of nothing and read
|
||||
# as "all hooks accounted for" — the exact failure this file exists to end.
|
||||
local pop
|
||||
pop=$(etv_hook_fire_population "$root")
|
||||
if [ -z "$pop" ]; then
|
||||
printf 'hook-fire-log: found NO hook scripts under %s/.claude/hooks — refusing to report, because a report over an empty population reads as full coverage.\n' "$root" >&2
|
||||
return 2
|
||||
fi
|
||||
|
||||
local total_fires=0 rows=""
|
||||
while IFS= read -r hook; do
|
||||
[ -n "$hook" ] || continue
|
||||
local fires decisions
|
||||
fires=0
|
||||
decisions=""
|
||||
if [ -n "$files" ]; then
|
||||
fires=$(cat $files 2>/dev/null | grep -c "\"hook\":\"$hook\",.*\"phase\":\"fire\"" || true)
|
||||
decisions=$(cat $files 2>/dev/null \
|
||||
| grep "\"hook\":\"$hook\",.*\"phase\":\"exit\"" \
|
||||
| sed -n 's/.*"decision":"\([^"]*\)".*/\1/p' \
|
||||
| sort | uniq -c | sort -rn \
|
||||
| awk '{printf "%s=%s ", $2, $1}')
|
||||
fi
|
||||
[ -z "$fires" ] && fires=0
|
||||
total_fires=$((total_fires + fires))
|
||||
if [ "$as_json" = "1" ]; then
|
||||
rows="${rows}{\"hook\":\"$hook\",\"fires\":$fires,\"decisions\":\"$(printf '%s' "$decisions" | tr -d '"')\"}\n"
|
||||
else
|
||||
rows="$(printf '%s%-34s %6s %s\n' "$rows" "$hook" "$fires" "${decisions:-—}")"$'\n'
|
||||
fi
|
||||
done <<EOF
|
||||
$pop
|
||||
EOF
|
||||
|
||||
if [ "$as_json" = "1" ]; then
|
||||
printf '{"log_dir":"%s","total_fires":%s,"hooks":[%s]}\n' "$dir" "$total_fires" \
|
||||
"$(printf '%b' "$rows" | sed '/^$/d' | paste -sd, -)"
|
||||
return 0
|
||||
fi
|
||||
|
||||
printf 'Hook fire log — %s\n' "$dir"
|
||||
printf '%-34s %6s %s\n' 'HOOK' 'FIRES' 'DECISIONS'
|
||||
printf '%s' "$rows"
|
||||
printf '\n%s hook scripts on disk, %s recorded fires.\n' "$(printf '%s\n' "$pop" | wc -l | tr -d ' ')" "$total_fires"
|
||||
printf 'A hook showing 0 has NOT been observed firing. That is a finding to investigate (broken\nwiring vs genuinely never matched in this window), not a blank to ignore.\n'
|
||||
return 0
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------------------------
|
||||
# CLI (only when executed, never when sourced)
|
||||
# ---------------------------------------------------------------------------------------------
|
||||
|
||||
if [ "${BASH_SOURCE[0]}" = "$0" ]; then
|
||||
set -uo pipefail
|
||||
case "${1:-report}" in
|
||||
report) shift 2>/dev/null || true; etv_hook_fire_report "$@" ;;
|
||||
path) etv_hook_fire_log_file; printf '\n' ;;
|
||||
record) shift; etv_hook_fire_record "$@" ;;
|
||||
*)
|
||||
printf 'usage: %s [report [--all|--json|--dir D|--session S] | path]\n' "$0" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
@@ -0,0 +1,29 @@
|
||||
"""Keep the test suite out of the PRODUCTION hook-fire log (ersatztv#776).
|
||||
|
||||
Four suites here drive real hooks as subprocesses — `test_merge_consent_exemption.py`,
|
||||
`test_merge_consent_base_change.py`, `test_prepush_rebase_check_tag_exemption.py`,
|
||||
`test_pr_changed_files.py` — and they predate the instrumentation. Once every hook records its own
|
||||
execution, running `pytest scripts/tests` wrote its synthetic invocations into
|
||||
`$HOME/.cache/ersatztv/hook-fire/`: 96 `pretooluse-merge-consent` fires including two `deny`s, and
|
||||
12 `blocked` decisions from `prepush-rebase-check`, none of them a real session.
|
||||
|
||||
That is not untidiness, it is the defect the whole change exists to remove. `scripts/hook-fire-log.sh
|
||||
report` is meant to answer "what did the harness actually do", and a log carrying test artifacts
|
||||
answers a different question while looking identical — inference reintroduced one layer up. The
|
||||
record has no field distinguishing a test invocation from a real one, and adding one would only move
|
||||
the problem, so the fix is that tests never write to the real log at all.
|
||||
|
||||
Autouse and session-independent: a new suite that drives a hook is isolated by existing, without
|
||||
having to know this file is here.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def isolate_hook_fire_log(tmp_path_factory, monkeypatch):
|
||||
monkeypatch.setenv(
|
||||
"ETV_HOOK_FIRE_LOG_DIR", str(tmp_path_factory.mktemp("hook-fire-log"))
|
||||
)
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user