diff --git a/docs/README.md b/docs/README.md index 81eed2b94..ab8ee63f5 100644 --- a/docs/README.md +++ b/docs/README.md @@ -28,7 +28,7 @@ doc below, or that changes which sections a task signal points to.** | Concurrency / optimistic-locking work | `docs/api-conventions.md` §7a/b/c + `docs/decisions/optimistic-concurrency.md` | | Auth / security-surface work | `docs/decisions/api-auth-security.md` | | CI / release pipeline work | `docs/ci-cd.md` + `docs/decisions/release-ci-governance.md` | -| Proposing a new guard / CI check / regression test convention | `docs/defect-shapes-773.md` §4 (detector menu + the classes where no detector is plausible), then the rules every guard must satisfy: `docs/decisions/records/testing/guard-derives-population-from-source.md`, `…/guard-ships-with-mutation-proof.md` and `…/mutation-claims-are-executed.md` (a `MUTATION` grade carries a DECLARED clause mutation that is re-run every suite — and so does a PROSE claim that some mutation reddens, or does not redden, a named test, wherever it is written: it is a `CLAIMS` entry in the same manifest, bound to its site and verbatim quote, or it is not written) — plus `…/verification-code-needs-its-own-proof.md`, which extends the same obligation BEYOND guards to the harness, wrapper or checker doing the checking, and says where its proof lives when the checker holds no row. Then `…/guard-pins-the-artifact-not-a-shape.md` for the SHAPE of the predicate itself: over an artifact whose grammar the predicate does not implement (shell, a workflow `run:` body, TOML, YAML, a jq expression), pin the artifact WHOLE by default — matching a shape inside it is the exception and must carry that record's four-part argument | +| Proposing a new guard / CI check / regression test convention | `docs/defect-shapes-773.md` §4 (detector menu + the classes where no detector is plausible), then the rules every guard must satisfy: `docs/decisions/records/testing/guard-derives-population-from-source.md`, `…/guard-ships-with-mutation-proof.md` and `…/mutation-claims-are-executed.md` (a `MUTATION` grade carries a DECLARED clause mutation that is re-run every suite — and so does a PROSE claim that some mutation reddens, or does not redden, a named test, wherever it is written: it is a `CLAIMS` entry in the same manifest, bound to its site and verbatim quote, or it is not written) — plus `…/verification-code-needs-its-own-proof.md`, which extends the same obligation BEYOND guards to the harness, wrapper or checker doing the checking, and says where its proof lives when the checker holds no row. Then `…/guard-pins-the-artifact-not-a-shape.md` for the SHAPE of the predicate itself: over an artifact whose GRAMMAR the predicate does not implement, pin the artifact WHOLE by default — matching a shape inside it is the exception and must carry that record's four-part argument; that record's `rule:` is the one place the qualifying grammars are enumerated | | Adding or bounding a consequential numeric config field (an FFmpeg profile tunable, a pipeline knob) | `docs/api-conventions.md` §3d — reject out of range with a 422 naming the bound and its consequence, never accept-then-rewrite; validate against the constants the renderer reads, keep the render-time clamp for pre-existing rows, and let an UNCHANGED legacy value through on update. Then `api.ffmpeg-profile-numeric-bounds` | | Testing a surface gated by config / an env var / a credential | `docs/decisions/records/testing/deny-path-at-production-config-value.md` — cover the setting absent, at its production value, and each opt-out, and assert the DENY branch | | Touching a full-replace write path or a hand-built request object | `docs/decisions/records/testing/full-replace-asserts-field-list.md` — derive the field list from the DTO and assert set equality; reconcile by id where child state exists. In the SPA the same rule is enforced by the type system: `docs/spa-conventions.md` §4b — build the body as `Complete`, annotating BOTH the wrapper parameter and every construction site | diff --git a/docs/decisions/records/testing/guard-pins-the-artifact-not-a-shape.md b/docs/decisions/records/testing/guard-pins-the-artifact-not-a-shape.md index bf95097e3..2c0d0d904 100644 --- a/docs/decisions/records/testing/guard-pins-the-artifact-not-a-shape.md +++ b/docs/decisions/records/testing/guard-pins-the-artifact-not-a-shape.md @@ -7,7 +7,7 @@ supersedes: none superseded-by: none rule: 'A predicate that decides something by matching a SHAPE inside an artifact whose GRAMMAR it does not implement pins the artifact WHOLE instead — the whole file, or a whole named span, compared for equality under a STATED normalisation. The boundary is the grammar and never the file type: shell source, a workflow `run:` body, TOML, YAML, a jq expression and JSON5 all qualify, and a shape matched over the OUTPUT of a real parser for that grammar does not. Extracting a `run:` value with a real YAML parser and then matching shell inside it IS in the class, because the shell is the unparsed half. THE PIN IS THE DEFAULT FOR A NEW PREDICATE; REJECTED is the alternative of writing the shape-matcher and pinning only once it has been defeated. The defeat a remedy policy waits for is a false GREEN, so it is found by a reviewer or by an incident and never by the guard: "not defeated yet" is a statement about who has looked, not about the input space. A SHAPE-MATCHER IS THE EXCEPTION, and its argument carries FOUR things, written where the predicate is and re-read whenever a clause of it is widened. (1) The GRAMMAR named, and the parser that implements it — if the shape is matched over a real parse there is no exception to write. (2) The INPUT SPACE as a CLOSED enumeration with the reason it is closed: a generator whose output form is fixed, or a canonicaliser the guard itself runs. A list of the spellings the author thought of is not that reason, and is the thing that was defeated nine, seven and five times. (3) The FAILURE DIRECTION measured rather than asserted: each defeat the matcher claims to catch is a DECLARED, executed mutation under `testing.mutation-claims-are-executed`, because a shape-matcher''s miss is a false green and an unrun claim about a fail direction is not evidence. (4) WHAT IT BUYS over the pin, priced in a cost the pin actually charges — the artifact is edited routinely or written by tooling, so a pin would redden as background noise and train its readers to ignore it; or the artifact''s population is DISCOVERED rather than fixed, so there is nothing to pin. Fewer than four is a refusal, and neither elegance nor an unwillingness to update a constant is one of the four. TWO OTHER BARS ARE REJECTED for this exception: a NUMERIC one ("survives N adversarial spellings"), which measures the reviewer''s imagination rather than the input space and would have licensed the parser above through most of nine rounds; and a REVIEWER SIGN-OFF, which makes the exception depend on the very signal the threshold below says arrives late. THE THRESHOLD for withdrawing a shape-matcher already shipped is not a count of spellings: it is the moment the NEXT spelling is found by the REVIEWER rather than by the author, which is evidence that the author''s model of the input space is not the input space. TWO RIDERS, both measured. A pin assumes it is pinning the artifact that still DECIDES — ask what else could decide this and whether the pin would still match: another FILE that outranks it, another OCCURRENCE in the same file, another WORKFLOW, or a HOOK the pinned command invokes. And WIDENING A CLAUSE RETIRES THE PROOFS CALIBRATED AGAINST THE NARROW ONE: a canary asserting that a finer mutation still SURVIVES has as its precondition that the guard is deliberately coarse there, so widening the main clause turns the canary into a tautology, which reads exactly like a passing proof — re-derive every proof calibrated against the narrow clause in the same commit that widens it. THE COST IS STATED RATHER THAN DISCOVERED LATER: any edit to a pinned artifact reddens, and the pin is updated deliberately in the same commit, with a reason. That red IS the review trigger the pin exists to be. A pin also declares its NORMALISATION and what the normalisation cannot see, because "pinned whole" invites a reader to assume byte equality.' signals: 'pin the artifact whole · shape-matcher is the exception that argues for itself · the boundary is the grammar not the file type · predicate does not implement the grammar it parses · defeated by the next spelling · the reviewer found the next spelling · false RED versus false GREEN · nine defects one mechanism · seven respellings of a partial config match · five spellings of a lexical preamble rule · the four-part exception argument: grammar and parser, closed input space, measured fail direction, what it buys · authority moves where the pin is not looking · an outranking config file · a decoy occurrence · an npm lifecycle hook · widening a clause retires its own canary · a survived-clause canary becomes a tautology · a pin declares its normalisation · instances: `ci.image-build-delegates-the-spa-suite`, `process.hook-resolves-inputs-from-repo-root` · paths: `scripts/tests/test_image_build_delegates_the_spa_suite.py`, `scripts/tests/test_hook_fire_log.py`, `docs/guard-inventory.md`, `docs/defect-shapes-773.md` · issues: #901, #887, #891, #774, #790, #881, #773' -mechanics: 'Two worked pins. `scripts/tests/test_image_build_delegates_the_spa_suite.py` pins command TEXT (`PINNED_STAGE_COMMANDS`), `web/package.json`''s script map (`PINNED_PACKAGE_SCRIPTS`) and `web/vite.config.ts` entire (`PINNED_VITE_CONFIG`), the last two after a SELECTION and then a partial match were defeated. The three normalise DIFFERENTLY, so a residual read off one of them NEED NOT transfer to the others — and one of them does transfer, because two share a primitive: the stage commands are compared after line continuations are joined and whitespace within each command is collapsed (`_normalise`), so a reflow is free and a line boundary is not; the script map is dict equality over parsed JSON, so the file''s own formatting and key ORDER are free while every script body is exact; and of the three only `PINNED_VITE_CONFIG` runs through `_normalise_lines`, which additionally drops blank lines. `_normalise_lines` IS `_normalise` applied per line, so the whitespace collapse — including inside a QUOTED STRING, where it can change meaning — belongs to both TEXT pins and not to the vite one alone; of the two only the vite test states it, in place of claiming byte equality, which is what the rule above asks of a pin. `scripts/tests/test_hook_fire_log.py` pins the hook preamble''s two lines byte for byte (`CANONICAL_SINK_ASSIGNMENT`, `CANONICAL_SINK_SOURCE`), named once so the checker, the mutation proofs and the hooks cannot come to mean different strings — but its normalisation is a SELECTION, and the rule above obliges this record to say what that selection cannot see, because the pin itself does not. The compared set is the lines that contain the literal `ETV_HOOK_FIRE_LIB` and are not comment-led, so the pin reaches exactly the two preamble lines and nothing else in the file: a later reassignment of that variable which never spells the literal — one composed at runtime and `eval`ed, say — is outside the selection, and therefore outside the pin, however the checker then behaves. Byte-identity WITHIN a selection is not byte-identity over the file — the pin''s reach ends where its selector does. That is stated here as a structural fact about the selector and NOT as a measured checker outcome, because an outcome claim about a mutation is a declared, harness-executed `CLAIMS` entry under `testing.mutation-claims-are-executed` or it is not written, wherever it is written — this record included. The residual is #891 code and predates this record; stating it is this record''s own rule applied to one of its own worked pins. Each of the three image-build pins faults with a request for the reason in the same commit, and two of them name the CONSTANT to update (`PINNED_STAGE_COMMANDS`, `PINNED_VITE_CONFIG`) while the script-map fault names the FILE and prints both maps. The hook-preamble fault asks for neither: it reports the divergent lines, since the two canonical strings ARE what a deliberate change edits.' +mechanics: 'Two worked pins. `scripts/tests/test_image_build_delegates_the_spa_suite.py` pins command TEXT (`PINNED_STAGE_COMMANDS`), `web/package.json`''s script map (`PINNED_PACKAGE_SCRIPTS`) and `web/vite.config.ts` entire (`PINNED_VITE_CONFIG`), the last two after a SELECTION and then a partial match were defeated. The three normalise DIFFERENTLY, so a residual read off one of them NEED NOT transfer to the others — and one of them does transfer, because two share a primitive: the stage commands are compared after line continuations are joined and whitespace within each command is collapsed (`_normalise`), so a reflow is free and a line boundary is not; the script map is dict equality over parsed JSON, so the file''s own formatting and key ORDER are free while every script body is exact; and of the three only `PINNED_VITE_CONFIG` runs through `_normalise_lines`, which additionally drops blank lines. `_normalise_lines` IS `_normalise` applied per line, so the whitespace collapse — including inside a QUOTED STRING, where it can change meaning — belongs to both TEXT pins and not to the vite one alone; of the two only the vite test states it, in place of claiming byte equality, which is what the rule above asks of a pin. `scripts/tests/test_hook_fire_log.py` pins the hook preamble''s two lines byte for byte (`CANONICAL_SINK_ASSIGNMENT`, `CANONICAL_SINK_SOURCE`) — but its normalisation is a SELECTION, and the rule above obliges this record to say what that selection cannot see, because the pin itself does not. The compared set is the lines that contain the literal `ETV_HOOK_FIRE_LIB` and are not comment-led, so the pin reaches exactly the two preamble lines and nothing else in the file: a later reassignment of that variable which never spells the literal — one composed at runtime and `eval`ed, say — is outside the selection, and therefore outside the pin, however the checker then behaves. Byte-identity WITHIN a selection is not byte-identity over the file — the pin''s reach ends where its selector does. That is stated here as a structural fact about the selector and NOT as a measured checker outcome, because an outcome claim about a mutation is a declared, harness-executed `CLAIMS` entry under `testing.mutation-claims-are-executed` or it is not written, wherever it is written — this record included. The residual is #891 code and predates this record; stating it is this record''s own rule applied to one of its own worked pins. Each of the three image-build pins faults with a request for the reason in the same commit, and two of them name the CONSTANT to update (`PINNED_STAGE_COMMANDS`, `PINNED_VITE_CONFIG`) while the script-map fault names the FILE and prints both maps. The hook-preamble fault asks for neither: it reports the divergent lines, since the two canonical strings ARE what a deliberate change edits.' --- **The class, and what approximating it costs.** A predicate over an artifact with a grammar is a partial reimplementation of that grammar, defeated not by a clever attacker but by an ordinary respelling — and each defeat @@ -15,9 +15,10 @@ looks local, one more case to handle, so the sequence stops only when someone st two unrelated subjects: a guard parsing shell text to decide whether a command runs the SPA suite was wrong **nine** ways from one mechanism (#887), its replacement — a partial match of `web/vite.config.ts` — **seven** further ways, and a lexical rule over a hook preamble **five** (#891). Every fix was locally correct; none of the three sequences -converged. All three ended in the same move: a pin cannot be defeated by a respelling of what it COMPARES, since it -need not RECOGNISE one in order to reject it. What stays exposed is whatever it does not compare: nothing for a -whole-file pin, and the SELECTOR's reach for a pin scoped by one — which is why `mechanics:` states the hook pin's. +converged. All three ended in the same move, and what the move buys is a fail DIRECTION: a pin's failure direction is +a false RED, which a human reads in the diff; a shape-matcher's is a false GREEN, which nobody reads. Which edits a +pin accepts is its NORMALISATION's business, so every pin — whole-file or selected — declares that normalisation and +what it cannot see, which is what `mechanics:` does for both worked pins. **Why the pin is the DEFAULT and not the remedy — the first thing #901 left open.** The rejected alternative is the tempting one: write the shape-matcher, and pin only once it has been defeated. It is rejected because of what the @@ -38,13 +39,12 @@ spent on it, and requirement 3, which forces the fail-direction claim to be exec **Rider 1 — a pin assumes it pins the artifact that still DECIDES.** This is where the pinned form fails, silently, so it is the question to ask of every new pin. In #887 every hole found after the first withdrawal was authority -moving where the pin was not looking: another FILE (vitest prefers `vitest.config.*`; `vite build` prefers -`vite.config.js`/`.mjs` to the pinned `.ts` — a VERSION-bound ordering, so the vite release it was read at is dated -in `docs/guard-inventory.md`'s `test_image_build_delegates_the_spa_suite.py` row rather than copied here), another -OCCURRENCE in the same file, another WORKFLOW (a `needs:` edge naming a different job called `test`), a HOOK the -pinned command invokes (a vite plugin's `buildStart`, an npm `prebuild`/`preinstall` script). A pin removes the -SPELLING problem outright and reduces the MEANING problem to a list that can be worked through; a pin sold as immune -to both stops being re-examined. +moving where the pin was not looking: another FILE that OUTRANKS the pinned one (the vite config ordering, a +VERSION-bound reading, is dated in `docs/guard-inventory.md`'s `test_image_build_delegates_the_spa_suite.py` row +rather than copied here), another OCCURRENCE in the same file, another WORKFLOW (a `needs:` edge naming a different +job called `test`), a HOOK the pinned command invokes (a vite plugin's `buildStart`, an npm `prebuild`/`preinstall` +script). A pin turns the SPELLING problem into its normalisation's and reduces the MEANING problem to a list that can +be worked through; a pin sold as immune to both stops being re-examined. **Rider 2 — widening a clause can retire its own evidence.** A canary asserting that a finer mutation still SURVIVES is calibrated against a deliberately coarse clause. Widen that clause and the canary does not go red, it goes diff --git a/docs/guard-inventory.md b/docs/guard-inventory.md index 0ea990399..32e040d5b 100644 --- a/docs/guard-inventory.md +++ b/docs/guard-inventory.md @@ -14,9 +14,9 @@ and `dotnet format`, and `grep -rn pytest .husky/` returns nothing. The runner i **Read `docs/decisions/records/testing/guard-derives-population-from-source.md`, `…/guard-ships-with-mutation-proof.md` and — for the SHAPE of the predicate itself, whenever it -decides by matching inside an artifact with a grammar the predicate does not implement (shell, a -workflow `run:` body, TOML, YAML, a jq expression, JSON5) — `…/guard-pins-the-artifact-not-a-shape.md` -before editing a guard or adding a row.** +decides by matching inside an artifact with a grammar the predicate does not implement (that +record's `rule:` enumerates which) — `…/guard-pins-the-artifact-not-a-shape.md` before editing a +guard or adding a row.** ## Columns diff --git a/scripts/tests/test_hook_fire_log.py b/scripts/tests/test_hook_fire_log.py index 8aa8a1c4f..9dd2b65db 100644 --- a/scripts/tests/test_hook_fire_log.py +++ b/scripts/tests/test_hook_fire_log.py @@ -77,9 +77,12 @@ _IMPORT_TIME_ENV = {**os.environ} # anything alone. It is kept for the diagnostic it produces, not for a verdict. _SOURCES_SINK = re.compile(r'^\[ -r "\$ETV_HOOK_FIRE_LIB" \] && \. "\$ETV_HOOK_FIRE_LIB" \|\| true$', re.M) -# THE TWO LINES, byte for byte. Named once so the checker, the mutation proofs and the hooks all mean -# the same string; a deliberate change to the preamble edits these and the suite tells you which -# hooks disagree. +# THE TWO LINES, byte for byte — what `instrumentation_faults` compares against, and what a +# deliberate change to the preamble edits. NOT single-sourced: the assignment's exact text is +# written out at THREE sites in this file — here, and as the `current` local of +# `test_an_ENV_VAR_resolved_sink_path_is_DETECTED` and of +# `test_the_NEXT_env_var_to_be_invented_is_DETECTED`, each of which asserts `current in text` +# against every hook before mutating. So editing the preamble is editing three sites, not one. CANONICAL_SINK_ASSIGNMENT = ( 'ETV_HOOK_FIRE_LIB="' '$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)'