feat(776): every hook reports that it fired, and the report is measured #795
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: timothy/ersatztv#795
Reference in New Issue
Block a user
Every hook now records that it fired and what it decided, through one shared sink.
fixes #776
The gap
In this harness version only
Stophooks emit a structured transcript record.PreToolUseandPostToolUsehooks — which is every guard this repo relies on: merge consent, worktree ownership,BOM, agent model/RAM, the golden-file ban — leave no durable execution trace. #773 §5.4 could only
count filename mentions in settings dumps, i.e. inference, so a silently broken hook looked
identical to a working one indefinitely.
What landed
scripts/hook-fire-log.sh— one script, write side and read side, so the record format cannotdrift between them. A
firerecord on entry, anexitrecord with status and decision.Every Claude hook here exits 0 always and decides by printing JSON;
pretooluse-merge-consent.shalone reaches that print from ~40 call sites. Capturing stdout avoided 40 edits in the repo's most
load-bearing guard and records what the harness sees rather than what the author meant.
capturestdout; git hooksstreamit anddecide by exit code — capturing a slow pre-push hook's output would hold it back and read as a hang.
scripts/tests/test_hook_fire_log.py(78 tests),scripts/tests/conftest.py, guard-inventoryrows, a decision record, and §5.4 of
defect-shapes-773.mdre-derived from measurement.Measured, not inferred
11 of 13 hooks confirmed firing with their decisions. The load-bearing row is a real
deny—pretooluse-bash-guardactually blocked anETV_UPDATE_GOLDENS=1probe, which is what separates"ran" from "present". The two zeroes need a live browser and a real merge; §5.4 states what is
established (the scripts work) separately from what is not (that the harness dispatches to them),
because unit tests bypass registration.
Review
Six rounds, two independent reviewers per round (cross-family Codex + cold Claude). Findings and
fixes are in the commit messages; the durable ones are in the decision record. Highlights:
"" == "". It nowconstructs a positive case per hook (bare remote, stub Gitea, foreign worktree marker, BOM
.cs,stubbed
memory_pressure) andtest_the_AB_is_not_VACUOUS_for_any_hookrefuses to let any hookride on empty comparisons.
exec 0<"$sin" 2>/dev/nullwas silencing every hook's stderr —execwith no commandredirects the shell permanently. Every husky guard's user-facing output is stderr.
rounds for a sub-millisecond rescue) and a generalised redirect-order guard (four iterations, then
constructed false positives and negatives). Both withdrawals enumerate what they lose.
and now prints the 12-cell matrix plus its reproduction instead of a conclusion. Independently
re-measured, five repeats per cell, with a positive control.
Verification
pytest scripts/tests— 663 passed, 1 skippedscripts/decisions_validate.py— OK; catalog regenerated…_LOSES_its_instrumentation_is_DETECTED) andtransparency (
…DELETING_the_replay_makes_the_differential_go_RED)Not applicable: no
.cs(BOM gate) and noweb/(SPA gate) files changed.Round 2, from two independent cold reviews. The headline defect was found by the test the reviews prompted rather than by the reviews themselves: exec 0<"$sin" 2>/dev/null reads as "suppress errors from this redirection". It is not. `exec` with redirections and no command applies them to the shell PERMANENTLY, so this sent each hook's ENTIRE stderr to /dev/null. Every husky guard's user-facing output is stderr — the H6 "push to main BLOCKED" message, the BOM guard's remediation text, the missing-trailer message — so the guards went on blocking correctly while telling the human nothing about why. Invisible to the original A/B, which compared only stdout and exit status. The finding that exposed it was ranked Low ("the A/B ignores stderr"). Also fixed, each with an executed proof: - The fd restore sat inside the same conditional as the replay, and the replay read the PATH, so an unlinked-but-open scratch file meant a swallowed `deny` at exit 0. Restore is now unconditional and the replay reads through a held fd, which survives unlink. - A SIGTERMed hook lost output it had already printed (52 bytes -> 0). Bash runs no EXIT trap for an untrapped fatal signal; INT/TERM/HUP are trapped now. - The classifier misreported an uppercase decision as prose, read the bare word additionalContext in free text as a decision, and let a non-zero exit ERASE a decision the hook really printed. - The session id reached the log FILENAME unscrubbed. The first fix reused the record scrubber, which deliberately keeps `/` and `.`, so `../../escaped` went through untouched: passing a value through a scrub is not the same as passing it through the right scrub. - Preamble line 1 could abort a `set -e` hook, turning a fail-open guard fail-closed; and a missing sink printed `command not found` to stderr. The big one is test quality. `test_instrumentation_changes_NOTHING_the_harness_ can_see` was 160/165 empty-vs-empty and 165/165 exit-0-vs-exit-0, so deleting the whole replay left it green for four hooks, two of which issue `deny`. Three places claimed the matrix reached every deciding branch, including the decision record. It now constructs the missing cases — a foreign worktree marker, a BOM `.cs` in an *ersatztv*-named repo, stubbed memory_pressure, a real bare remote for the three pre-push guards, a stub Gitea for prepush-donewhen — and test_the_AB_is_not_VACUOUS_for_any_hook fails unless each hook has a case where the control side actually says something. Running pytest also wrote 115 synthetic fires into the production log, so the report described the test suite while looking like a record of real sessions. scripts/tests/conftest.py isolates every test; verified live, a full suite run now leaves the real log untouched. Refs: #776 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>Round 6 found a Blocker, and it is in the decision record rather than the code: the paragraph written LAST round to correct a fabricated measurement contains an inverted one. Same mistake, one round later, in the artifact written to fix it — and by the same route, taking a review summary without re-measuring. Claimed: the round-3 trapped sink "returned rc 0 on SIGINT where the control returns -2". Measured myself, 12 cells ({group,pid} x {TERM,INT,HUP} x {control, round-3, HEAD}): pid + INT: control 0 round-3 -2 HEAD 0 Exactly inverted. The trapped sink returned -2 where the control returns 0 — a real transparency violation, but in the RESTRICTIVE direction, which also makes "the consent bug survived" the wrong characterisation. HEAD matches the control in all twelve cells. The record now prints the matrix instead of asserting a conclusion about it. A second false claim in the same paragraph, also removed: that the test's exit-status assertion "catches the SIGINT case". It does not. The test signals the process GROUP, and in every group cell the trapped sink is indistinguishable — so it would not have caught round 3 at all. The test pins the transparency of the code that ships; it is not evidence any earlier version was worse, and the record now says so. Also this round: - The job-notice exemption in the signal A/B hides a line that NAMES the sink's path on a guard's stderr (bash attributing its own message to the executing line). That is a real if small leak and the exemption conceals it. Accepted rather than fixed — it cannot be suppressed while the sink does any work at exit — but stated plainly instead of left as "bash's notice, harmless". - docs/guard-inventory.md records the withdrawn redirect-order guard and, more importantly, what the surviving exec-only guard does NOT cover. Refs: #776 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>4c9a829bcdtoba2b162238Review-verdict: MERGEABLE @
ba2b162Six independent cold review rounds, two reviewers each, plus a third-party re-measurement of the disputed signal matrix. Every round found a real defect; all fixed or explicitly withdrawn with lost behaviours enumerated. CI green on this head after rebase onto main. 667 tests pass. Two mutation proofs witnessed red. Known limits stated in the decision record rather than implied.