fix(797): the twin recipe was still broken, and the mutation proof was green over a dead guard
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 9s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 18s
PR Gates / Docs update reminder (pull_request) Successful in 18s
PR Gates / decisions lifecycle (pull_request) Successful in 28s
Review verdict / Set review-verdict status (pull_request_target) Successful in 17s
PR Gates / Script tests (pytest) (pull_request) Successful in 2m6s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m19s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m42s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m19s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 5s
review-verdict/h10 Review-verdict: MERGEABLE @ 6a6ce36 (base: main)

Two independent reviews of #798. No Blocker; the functional fix is confirmed
sound — one reviewer ran a 17-input matrix on BSD od, GNU od, and inside the
actual CI runner image, identical in every cell, with `xxd` confirmed absent
there. Everything below is what they found around it.

THE FIX STOPPED AT THE HOOK. `docs/decisions/records/process/bom-format-detection-recipe.md`
(active) and `docs/contributing.md:120` both still prescribed
`head -c3 | xxd -p | grep -q '^efbbbf'` as the standing manual pre-push check —
same absent xxd, same silent all-clean, on the path a human runs PRECISELY when
the hook is being bypassed with --no-verify. Worse, that record's "Verify your
detector" paragraph rejected an `od` form by name, so the active record argued
against the hook this PR ships.

Both corrected. The record keeps its original warning, which is still right and
was never about `od` as a tool: the form that reported all-clean over 19 dirty
files was `od -An -c | grep '357 273 277'` — octal character output, whose
spacing and escaping vary. The replacement is `od -A n -t x1 -N 3`, hexadecimal
bytes, measured across the matrix above. The record now says its own prescribed
detector had been an all-clean detector for three weeks, which is the thing it
was written to prevent.

THE MUTATION PROOF WAS GREEN OVER A DEAD GUARD. Run against the pre-fix hook on
Linux, `test_DISARMING_the_BOM_comparison_stops_detection` PASSED while the guard
detected nothing at all — "the mutant is silent" is trivially true when the
original is silent too. The inventory row cites that function by name. It now
asserts the unmutated guard DOES detect, first.

THE STAND-IN PATH STARVED THE HOOK. `_path_without_xxd` hand-listed six tools the
hook never calls and omitted `tail`, which it does call for cd-detection — so a
payload whose cwd differs from the repo failed open for a reason unrelated to
xxd. The list is derived from the hook's own source now, with a required-set
assertion so a starved PATH fails loudly instead of passing quietly.

Also: the inventory narrative restated the NONE count and drifted the moment this
guard was regraded, invisibly to the summary parser. The duplicate count is
removed rather than corrected.

Refs: #797

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-14 17:30:01 +02:00
co-authored by Claude Opus 5
parent 4873870727
commit 6a6ce36b2c
5 changed files with 58 additions and 13 deletions
+1 -1
View File
@@ -117,7 +117,7 @@ talks exclusively to the REST API. Every former Blazor route now 302-redirects t
`io.open(..., encoding='utf-8-sig')` *writes one back*, and sed/perl round-trips keep it. This cost
two sessions a red CI job on 2026-07-17 alone (PR #405, 6 files; PR #402, 19), which is what the
guard is for. To check by hand:
`for f in $(git diff --name-only origin/main...HEAD --cached -- '*.cs'; git diff --name-only -- '*.cs') ; do head -c3 "$f" | xxd -p | grep -q '^efbbbf' && echo "BOM: $f"; done` (covers branch + staged + dirty — a brand-new staged file is absent from `origin/main...HEAD`)
`for f in $(git diff --name-only origin/main...HEAD --cached -- '*.cs'; git diff --name-only -- '*.cs') ; do [ "$(od -A n -t x1 -N 3 < "$f" | tr -d ' \n')" = efbbbf ] && echo "BOM: $f"; done` (covers branch + staged + dirty — a brand-new staged file is absent from `origin/main...HEAD`)
(note `dotnet format --include` needs **bash**`mapfile` is bash-only, and under zsh the file list
comes out empty, which looks exactly like the tool silently doing nothing). (A one-time repo-wide normalization
is a separate, unmade decision; the touched-file rule is the standing one.)
+1 -1
View File
@@ -103,7 +103,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
| `media.remote-stream-probe` | `ValidatePlayoutItemPath` probes the Plex/Jellyfin/Emby remote-stream URL via `IRemoteStreamProber` before returning it; only a redirected 404 fails closed (`PlayoutItemNotAvailableFromMediaServer`), everything else fails open, and there is no toggle. | 2026-07-19 | [link](records/media/remote-stream-probe.md) |
| `media.remote-stream-probe-externaljson` | External-JSON playout channels' `StreamRemotely` now probes the remote-stream URL through the same `IRemoteStreamProber` seam as the generated-playout path, closing the #473 scope gap for a channel kind with no DB `PlayoutItem` rows. | 2026-07-20 | [link](records/media/remote-stream-probe-externaljson.md) |
| `media.source-mgmt-write-api` | Media-source management (local/Plex/Jellyfin/Emby) is a REST write API + SPA under `/app/libraries/*`, wrapping existing MediatR commands 1:1 with no new commands or DB migration; connection GETs never leak a stored `apiKey`, and each PUT-replace family's identity contract is documented per-family (not assumed uniform). | 2026-07-11 | [link](records/media/source-mgmt-write-api.md) |
| `process.bom-format-detection-recipe` | Before any push touching `.cs`, detect BOMs with the `xxd` byte check and verify the format gate with `dotnet format --include` run under `bash -c`, never bare zsh. | 2026-07-21 | [link](records/process/bom-format-detection-recipe.md) |
| `process.bom-format-detection-recipe` | Before any push touching `.cs`, detect BOMs with the `od -A n -t x1 -N 3` byte check and verify the format gate with `dotnet format --include` run under `bash -c`, never bare zsh. NOT `xxd`: it ships with vim and is absent on plain Linux hosts including this repo's CI runner, where the substitution yields empty, never matches, and the check reports all-clean — the same all-clean-detector failure this record was written about, in the detector it prescribed. | 2026-07-21 | [link](records/process/bom-format-detection-recipe.md) |
| `process.branch-off-feature-branch` | To fix work on an unmerged feature branch, branch off that branch and land by fast-forward push — and after creating a worktree, drive the first Edit/Read from ITS absolute paths and `git status` it before building. | 2026-07-21 | [link](records/process/branch-off-feature-branch.md) |
| `process.build-concurrency-limits` | Run at most 34 concurrent dotnet/npm builds on this Mac, gate launches on FREE RAM rather than CPU load, and never set `ETV_UPDATE_GOLDENS` / `ETV_UPDATE_PLAYOUT_GOLDENS`. | 2026-07-21 | [link](records/process/build-concurrency-limits.md) |
| `process.codex-cheap-worker-launch` | For bounded tool-bearing selector/recon work, launch a Codex worker with `codex exec -m gpt-5.4-mini -c model_reasoning_effort=low -s read-only`; `spawn_agent` buys parallelism but no cost savings. | 2026-07-21 | [link](records/process/codex-cheap-worker-launch.md) |
@@ -1,11 +1,11 @@
---
key: process.bom-format-detection-recipe
title: '2026-07-21 — BOM/format pre-push detection: use the `xxd` loop, and run `dotnet format --include` under `bash -c` (#542)'
title: '2026-07-21 — BOM/format pre-push detection: use the `od` byte check, and run `dotnet format --include` under `bash -c` (#542, detector corrected #797)'
status: active
since: '2026-07-21'
supersedes: none
superseded-by: none
rule: Before any push touching `.cs`, detect BOMs with the `xxd` byte check and verify the format gate with `dotnet format --include` run under `bash -c`, never bare zsh.
rule: 'Before any push touching `.cs`, detect BOMs with the `od -A n -t x1 -N 3` byte check and verify the format gate with `dotnet format --include` run under `bash -c`, never bare zsh. NOT `xxd`: it ships with vim and is absent on plain Linux hosts including this repo''s CI runner, where the substitution yields empty, never matches, and the check reports all-clean — the same all-clean-detector failure this record was written about, in the detector it prescribed.'
signals: 'UTF-8 BOM · `efbbbf` · `dotnet format --verify-no-changes` · `--include` · `mapfile` · zsh vs bash · #311 format gate · detector verification · paths: `ErsatzTV.sln`, `.editorconfig` · issues: #542, #311, #70, PR #402, PR #405'
mechanics: Local pre-push shell; CI's format job recipe (`ci.format-gate-folder-mode`). The BOM *policy* is `release.format-as-you-touch-rebase`; this record is the mechanics only.
---
@@ -13,11 +13,25 @@ mechanics: Local pre-push shell; CI's format job recipe (`ci.format-gate-folder-
Detection loop, verbatim:
```bash
for f in $(git diff --name-only origin/main...HEAD -- '*.cs'); do head -c3 "$f" | xxd -p | grep -q "^efbbbf" && echo "BOM: $f"; done
for f in $(git diff --name-only origin/main...HEAD -- '*.cs'); do [ "$(od -A n -t x1 -N 3 < "$f" | tr -d ' \n')" = efbbbf ] && echo "BOM: $f"; done
```
**Verify your detector.** An `od -An -c | grep '357 273 277'` check reported all-clean while 19 files
were actually dirty. A detector that can only say "ok" is worse than none — use the `xxd` form or read
**The prescribed detector was itself an all-clean detector, corrected 2026-08-14 (#797).** This
record was written because a detector that can only say "ok" is worse than none. It then prescribed
`xxd -p`, and **`xxd` ships with vim and is absent on plain Linux hosts, including this repo's CI
runner** — where the substitution yields the empty string, never matches `efbbbf`, and the loop
prints nothing for a tree full of BOMs. The rule failed its own test, in the recipe it recommends,
for three weeks.
Read the original warning precisely, because it is still right and it is not about `od`: the form
that reported all-clean over 19 dirty files was `od -An -c | grep '357 273 277'` — octal character
output, whose spacing and escaping vary. The replacement is `od -A n -t x1 -N 3`, which emits
hexadecimal bytes. That is a different invocation of the same tool, and it was measured across 17
inputs (empty, sub-3-byte, exactly the BOM, BOM+NUL, binary, UTF-16 BOM, unreadable, 20 MB, awkward
filenames) on BSD `od`, GNU `od`, and inside the actual CI runner image: identical in every cell.
`tr -d ' \n'` is load-bearing — BSD pads to a fixed column width and GNU does not.
**Verify your detector** — use the form above, or read
the bytes directly, and distrust a clean result you didn't prove can go dirty.
**`dotnet format --include` DOES work here** — an earlier note claiming it silently no-ops was WRONG.
+6 -1
View File
@@ -171,7 +171,12 @@ dangerous direction — a token that a reviewer means as `BLOCKED` silently read
approval, which writes a green `review-verdict/h10`. A token added to one script and not the other
is untested by either, which is the whole of #788.
The 21 include every `PreToolUse` hook except merge-consent — the same population #776 found we
**That unproven set carries no number here on purpose.** It restated the count, drifted the moment
the BOM guard was regraded, and `test_the_summary_counts_match_the_table` cannot see it — the parser
checks the formatted summary sentence and nothing else. A second hand-maintained copy of a number is
the duplication family this file argues against, so the copy is removed rather than corrected: the
set is every `PreToolUse` hook except merge-consent and the BOM guard, and the count is in the
summary above. It is also the population #776 found we
cannot even observe firing. A guard that is neither proven nor observable is an assertion about the
past.
+31 -5
View File
@@ -70,14 +70,29 @@ def _path_without_xxd(tmp_path: Path) -> dict:
in the same directory as `git` and `od`; removing that directory would starve the hook of tools
it legitimately needs and the test would pass for the wrong reason.
"""
import re
# DERIVED from the hook, not hand-listed. The hand-listed version symlinked six tools the hook
# never calls and omitted `tail`, which it does call for `cd`-detection — so a payload whose cwd
# differs from the repo took a `tail: command not found` path and the guard ALLOWED a BOM for a
# reason that had nothing to do with xxd. A stand-in PATH that starves the subject is a test
# passing for the wrong reason, which is the whole subject of this file.
words = set(re.findall(r"\b([a-z][a-z0-9_-]*)\b", HOOK.read_text()))
bindir = tmp_path / "nobin"
bindir.mkdir()
for tool in ("bash", "git", "od", "tr", "sort", "head", "grep", "sed", "jq", "rm", "printf",
"cat", "dirname", "basename", "mktemp", "date"):
linked = []
for tool in sorted(words - {"xxd"}):
found = shutil.which(tool)
if found:
if found and not (bindir / tool).exists():
(bindir / tool).symlink_to(found)
assert shutil.which("od", path=str(bindir)), "the stand-in PATH lost `od` itself"
linked.append(tool)
# `head` is deliberately NOT here: the `od` change removed the only `head` call, and asserting a
# tool the hook no longer uses is how a stand-in PATH drifts from its subject.
for required in ("bash", "git", "od", "tr", "sort", "tail"):
assert shutil.which(required, path=str(bindir)), (
f"the stand-in PATH lost `{required}`, which the hook needs — the test would then pass "
"because the guard was starved, not because it detected anything"
)
assert not shutil.which("xxd", path=str(bindir)), "the stand-in PATH still resolves xxd"
return {**os.environ, "PATH": str(bindir)}
@@ -114,7 +129,7 @@ def test_a_staged_BOM_is_DENIED_when_xxd_DOES_NOT_EXIST(tmp_path):
repo = _repo_with(tmp_path, "Bad.cs", BOM + b"class A {}\n")
rc, out = _run(HOOK, repo, _path_without_xxd(tmp_path))
assert rc == 0
assert b"deny" in out, (
assert b'"permissionDecision": "deny"' in out or b'"permissionDecision":"deny"' in out, (
"with `xxd` absent the guard allowed a BOM. That is the fail-open this file exists to "
f"prevent, and it is the state every non-vim host was in: {out!r}"
)
@@ -151,6 +166,17 @@ def test_DISARMING_the_BOM_comparison_stops_detection(tmp_path):
mutated.write_text(text.replace(marker, '= "deadbeef" ]; then', 1))
repo = _repo_with(tmp_path, "Bad.cs", BOM + b"class A {}\n")
# POSITIVE CONTROL FIRST. Without it this test passes when the guard detects NOTHING AT ALL —
# verified: run against the pre-fix hook on Linux, where xxd is absent, and "the mutant is
# silent" was trivially true. A green mutation proof over a dead check is the exact failure
# `guard-ships-with-mutation-proof` exists to stop, and the inventory row cites THIS function.
rc_live, out_live = _run(HOOK, repo, dict(os.environ))
assert rc_live == 0 and b"deny" in out_live, (
"the UNMUTATED guard did not detect the BOM, so 'the mutant is silent' proves nothing about "
f"the clause: {out_live!r}"
)
rc, out = _run(mutated, repo, dict(os.environ))
assert rc == 0
assert out == b"", (