timothy/ersatztv
1
0
Fork 0
Code Issues 54 Pull Requests 6 Actions Packages Projects Releases 9 Wiki Activity

Extend the machine-checked guard inventory to inline workflow-job guards #786

New Issue
Closed
opened 2026-08-13 21:59:11 +02:00 by timothy · 3 comments
timothy commented 2026-08-13 21:59:11 +02:00
Owner
Copy Link
Copy Source

Spawned by #774. scripts/tests/test_guard_inventory.py derives its population from guard files — a glob of .claude/hooks/ and .husky/ plus scripts/* references scraped from workflows and hooks. Guards implemented inline in workflow YAML are outside that population. docs/guard-inventory.md states the limit explicitly rather than implying coverage, but the limit is real.

The one that motivated #774 lives exactly there: pr-checks.yml:ci-image-pin stated an invariant it did not check. It was fixed by adding a structural test, not by bringing it into the inventory.

The blocker is that "which jobs are guards" needs a judgement call per job that the filesystem cannot supply — so it needs the same reviewed-registry-cross-checked-both-ways shape TOOLCHAIN_JOBS uses in test_ci_image_pin_population.py. 15 jobs across 6 workflows.

Also worth folding in: several jobs have no dropped-step guard at all (functional-e2e, build, api-docs, format). build's absence is self-documented as a known residual; the other three are not. Decide per job and record the decision, rather than extending the marker protocol reflexively — test_ci_dropped_step_guard.py already argues why the ban is scoped to where a drop is consequential.

Done-when

  • Workflow-job guards enumerated into the inventory with a reviewed registry, cross-checked both ways
  • The four jobs with no dropped-step guard each get a recorded decision (guard added, or reason stated)
  • docs/guard-inventory.md scope limit updated to match the new coverage
  • Adversarial review passed
Spawned by #774. `scripts/tests/test_guard_inventory.py` derives its population from guard **files** — a glob of `.claude/hooks/` and `.husky/` plus `scripts/*` references scraped from workflows and hooks. Guards implemented **inline in workflow YAML** are outside that population. `docs/guard-inventory.md` states the limit explicitly rather than implying coverage, but the limit is real. The one that motivated #774 lives exactly there: `pr-checks.yml:ci-image-pin` stated an invariant it did not check. It was fixed by adding a structural test, not by bringing it into the inventory. The blocker is that "which jobs are guards" needs a judgement call per job that the filesystem cannot supply — so it needs the same reviewed-registry-cross-checked-both-ways shape `TOOLCHAIN_JOBS` uses in `test_ci_image_pin_population.py`. 15 jobs across 6 workflows. Also worth folding in: several jobs have **no dropped-step guard at all** (`functional-e2e`, `build`, `api-docs`, `format`). `build`'s absence is self-documented as a known residual; the other three are not. Decide per job and record the decision, rather than extending the marker protocol reflexively — `test_ci_dropped_step_guard.py` already argues why the ban is scoped to where a drop is consequential. ## Done-when - [x] Workflow-job guards enumerated into the inventory with a reviewed registry, cross-checked both ways - [x] The four jobs with no dropped-step guard each get a recorded decision (guard added, or reason stated) - [x] `docs/guard-inventory.md` scope limit updated to match the new coverage - [x] Adversarial review passed
timothy added the ci-cdpriority: medium labels 2026-08-13 21:59:11 +02:00
timothy added the in-progress label 2026-08-28 19:10:53 +02:00
timothy commented 2026-08-28 19:11:16 +02:00
Author
Owner
Copy Link
Copy Source

Claiming for this session (Claude Code / Opus 5, orchestrator).

Bundling #786 + #789 — they are one mechanism, and working #786 alone would build the artifact #789 removes:

  • #786 asks for inline workflow-job guards to be enumerated with "the same reviewed-registry-cross-checked-both-ways shape TOOLCHAIN_JOBS uses".
  • #789 says TOOLCHAIN_JOBS is itself the defect and must be deleted in favour of a machine-readable per-job marker declared in the workflow, with both sets derived from the parsed YAML.

So the bundle resolves the tension in one direction: one per-job execution-class declaration owned by the workflow, both populations derived from it, unknown/missing marker a hard failure. Registry-shaped hand lists go away rather than multiply.

Progress and any scope cuts will be recorded here.

Claiming for this session (Claude Code / Opus 5, orchestrator). Bundling **#786 + #789** — they are one mechanism, and working #786 alone would build the artifact #789 removes: - #786 asks for inline workflow-job guards to be enumerated with "the same reviewed-registry-cross-checked-both-ways shape `TOOLCHAIN_JOBS` uses". - #789 says `TOOLCHAIN_JOBS` is itself the defect and must be **deleted** in favour of a machine-readable per-job marker declared in the workflow, with both sets derived from the parsed YAML. So the bundle resolves the tension in one direction: one per-job execution-class declaration owned by the workflow, both populations derived from it, unknown/missing marker a hard failure. Registry-shaped hand lists go away rather than multiply. Progress and any scope cuts will be recorded here.
timothy commented 2026-08-28 22:29:41 +02:00
Author
Owner
Copy Link
Copy Source

PR #872 is open, covering this and its sibling — the two were bundled because working either alone
would have built the artifact the other removes.

Review: four cold adversarial rounds, alternating model families (Claude / Codex gpt-5.6 /
both in parallel / Claude), each from a review-only brief in an isolated worktree. Severity
converged BLOCKER/HIGH → MEDIUM → LOW/NIT; the final confirmation pass verified the substance clean
(detector-extraction equivalence over 4000 randomized cases per detector, zero mismatches) and its
one blocking finding had already been fixed a commit before it reported.

The recurring defect class was prose drifting from code — over half of all findings, including a
mechanism claim in the decision record that execution refuted. Every factual claim in the changed
docs and comments has since been re-verified by execution.

Two corrections to the issues themselves, both measured rather than assumed, are recorded in the
closing records.

Awaiting CI on d3f470f, then the H10 verdict and the merge gate.

PR #872 is open, covering this and its sibling — the two were bundled because working either alone would have built the artifact the other removes. **Review:** four cold adversarial rounds, alternating model families (Claude / Codex gpt-5.6 / both in parallel / Claude), each from a review-only brief in an isolated worktree. Severity converged BLOCKER/HIGH → MEDIUM → LOW/NIT; the final confirmation pass verified the substance clean (detector-extraction equivalence over 4000 randomized cases per detector, zero mismatches) and its one blocking finding had already been fixed a commit before it reported. The recurring defect class was **prose drifting from code** — over half of all findings, including a mechanism claim in the decision record that execution refuted. Every factual claim in the changed docs and comments has since been re-verified by execution. Two corrections to the issues themselves, both measured rather than assumed, are recorded in the closing records. Awaiting CI on `d3f470f`, then the H10 verdict and the merge gate.
timothy commented 2026-08-28 22:55:14 +02:00
Author
Owner
Copy Link
Copy Source

Closing record

Outcome: Closed by PR #872. Every job in all six tracked workflows now declares
env.CI_JOB_ROLE (guard/advisory/none), and scripts/tests/test_workflow_job_guards.py
asserts set equality both ways between the guard-declaring jobs and a new Workflow-job guards
table in docs/guard-inventory.md. A missing or unrecognised marker is a hard failure. The
scope-limit entry that said this class was uncovered is closed in place, with its two residuals
stated. The four jobs named here as lacking a dropped-step guard each carry a recorded decision.

Root cause: the inventory derived its population from guard FILES, so a guard implemented inline
in workflow YAML belonged to no population at all — which is where pr-checks.yml:ci-image-pin was
sitting when #774 found it stating an invariant it did not check.

Decisions/conventions changed: added testing.workflow-declares-its-own-job-metadata. Bundled
with #789, whose TOOLCHAIN_JOBS literal this issue proposed to COPY — the record resolves that
tension in #789's direction (derive from a workflow-declared marker; never a literal in the checker).

Reusable knowledge:

  • The classification rule that works is "verdict vs artifact". A guard job's output is a
    verdict; a none job's output is an artifact, and its red means the build did not work rather
    than that an invariant was violated. The more natural-sounding rule — "a guard enforces an
    invariant about the REPOSITORY, so a job exercising the PRODUCT is none" — was tried, reads as
    principled, and is WRONG: it puts test and migrations outside the table, and those are the two
    REQUIRED status contexts on main, i.e. exactly where a failure to fire is fail-open against
    branch protection. Recorded in the doc as a rejected alternative so it is not re-adopted.
  • This issue's own framing was the trap. It asked for "the same reviewed-registry-cross-checked-
    both-ways shape TOOLCHAIN_JOBS uses" while #789 was open to DELETE that shape. Working it as
    written would have built the artifact the sibling issue removes. Check an issue's proposed
    mechanism against its open siblings before implementing it.
  • This issue's numbers were stale: it says 15 jobs across 6 workflows; there are 17.

Verification: scripts/tests 1185 passed / 2 skipped on the final rebased tree (baseline on main was 1083). The
new checker's declared mutation runs as its own harness case. Two cold adversarial reviews; the first
returned BLOCKED with four findings, all fixed and each demonstrated.

Deferred: test_hook_fire_log.py::test_the_suite_does_not_write_to_the_PRODUCTION_log reddens
when a second Claude session runs concurrently — it snapshots mtimes under the shared
~/.cache/ersatztv/hook-fire/. Observed once here with three session logs written during the window;
green in isolation before and after. That is #809 / #822, and evidence is posted there.

Docs updated: docs/guard-inventory.md (new section, scope limit, file-populations row, counts),
docs/ci-cd.md ("Per-job declarations"), docs/README.md (task-signal row),
docs/decisions/records/testing/workflow-declares-its-own-job-metadata.md + regenerated catalog.

## Closing record **Outcome:** Closed by PR #872. Every job in all six tracked workflows now declares `env.CI_JOB_ROLE` (`guard`/`advisory`/`none`), and `scripts/tests/test_workflow_job_guards.py` asserts set equality both ways between the guard-declaring jobs and a new **Workflow-job guards** table in `docs/guard-inventory.md`. A missing or unrecognised marker is a hard failure. The scope-limit entry that said this class was uncovered is closed in place, with its two residuals stated. The four jobs named here as lacking a dropped-step guard each carry a recorded decision. **Root cause:** the inventory derived its population from guard FILES, so a guard implemented inline in workflow YAML belonged to no population at all — which is where `pr-checks.yml:ci-image-pin` was sitting when #774 found it stating an invariant it did not check. **Decisions/conventions changed:** added `testing.workflow-declares-its-own-job-metadata`. Bundled with #789, whose `TOOLCHAIN_JOBS` literal this issue proposed to COPY — the record resolves that tension in #789's direction (derive from a workflow-declared marker; never a literal in the checker). **Reusable knowledge:** - **The classification rule that works is "verdict vs artifact".** A `guard` job's output is a verdict; a `none` job's output is an artifact, and its red means the build did not work rather than that an invariant was violated. The more natural-sounding rule — "a guard enforces an invariant about the REPOSITORY, so a job exercising the PRODUCT is `none`" — was tried, reads as principled, and is WRONG: it puts `test` and `migrations` outside the table, and those are the two REQUIRED status contexts on `main`, i.e. exactly where a failure to fire is fail-open against branch protection. Recorded in the doc as a rejected alternative so it is not re-adopted. - **This issue's own framing was the trap.** It asked for "the same reviewed-registry-cross-checked- both-ways shape `TOOLCHAIN_JOBS` uses" while #789 was open to DELETE that shape. Working it as written would have built the artifact the sibling issue removes. Check an issue's proposed mechanism against its open siblings before implementing it. - **This issue's numbers were stale**: it says 15 jobs across 6 workflows; there are 17. **Verification:** `scripts/tests` 1185 passed / 2 skipped on the final rebased tree (baseline on `main` was 1083). The new checker's declared mutation runs as its own harness case. Two cold adversarial reviews; the first returned BLOCKED with four findings, all fixed and each demonstrated. **Deferred:** `test_hook_fire_log.py::test_the_suite_does_not_write_to_the_PRODUCTION_log` reddens when a second Claude session runs concurrently — it snapshots mtimes under the shared `~/.cache/ersatztv/hook-fire/`. Observed once here with three session logs written during the window; green in isolation before and after. That is #809 / #822, and evidence is posted there. **Docs updated:** `docs/guard-inventory.md` (new section, scope limit, file-populations row, counts), `docs/ci-cd.md` ("Per-job declarations"), `docs/README.md` (task-signal row), `docs/decisions/records/testing/workflow-declares-its-own-job-metadata.md` + regenerated catalog.
timothy removed the in-progress label 2026-08-28 22:55:31 +02:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
ad-hoc
One-off / ad-hoc work not tracked by a dedicated issue
 api
REST API / HTTP endpoints
 bug
Something isn't working
 ci-cd
Build, test, deploy pipeline
 content
Channel content / schedules / playlists
 dependencies
Dependency updates (Renovate)
 enhancement
New feature or improvement
 frontend
ChicoryTV React SPA frontend
 in-progress
Claimed by an active session — do not pick up
 jellyfin
Jellyfin tuner / IPTV integration
 parked
Excluded from automatic queue pickup; work only when explicitly selected
 priority: high
priority: low
priority: medium
review
Adversarial review finding
 security
Security / vulnerability fix
No labels ci-cd priority: medium
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
renovate timothy (Timothy)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: timothy/ersatztv#786