ersatztv/scripts/tests/test_workflow_job_guards.py

"""Every workflow JOB that is a guard has a row in docs/guard-inventory.md (ersatztv#786).

WHAT THIS CLOSES. The inventory's population was guard FILES — `.claude/hooks/`, `.husky/`, and the
`scripts/…` paths workflows and hooks reference. A guard written INLINE in workflow YAML belonged to
no population at all, and the inventory said so in its own scope limit rather than implying coverage.
The motivating case lived exactly there: `pr-checks.yml:ci-image-pin` stated an invariant in its
error text — "Every container: job must pin ersatztv-ci:<7-char-sha>" — and did not check it (#774).
It was fixed by adding a structural test, not by bringing the job into any inventory, so the next one
would have been just as invisible.

WHY A MARKER RATHER THAN A LIST. "Which jobs are guards" is a judgement per job, and #786 says so:
the filesystem cannot supply it. The available shapes were a hand-written registry in this file, or a
declaration in the workflow. This takes the second, for the reason #789 rejected the first in
`test_ci_image_pin_population.py`: a literal list of members is a filter frozen at authoring time,
correct the day it is written and unable to report the day it stopped being. The judgement still has
to be made by a human — it is made in the workflow diff, next to the job, where a reviewer changing
that job sees it.

WHAT THE MARKER MEANS, because the obvious wrong readings make the population useless. THE LINE IS
WHAT THE JOB PRODUCES: a `guard` job's output is a VERDICT, a `none` job's output is an ARTIFACT.
A `none` job can still go red — that is an ERROR in producing the thing, not a finding about the
repo. `docker-build.yml::build` publishes an image and smoke-tests it, and is `none`; a failed smoke
test means the build did not work. Which jobs are `none` is read off the markers, not listed here:
a list in prose is a second copy of the workflow that rots on the next job added.

`report-only` is the third value and needs saying, because "guard vs none" does not imply it: a
`report-only` job produces a VERDICT IT CANNOT ENFORCE — every check step carries
`continue-on-error: true`. The limit, since this file is where someone will look for it: nothing
here detects a `guard` job whose checks are all `continue-on-error`. The comparison is marker
against ROW, never marker against the job's ability to fail.

TWO READINGS THAT DO NOT WORK, recorded so neither is re-adopted. "Any job containing a step that
can fail" makes every job a guard and the table distinguishes nothing. "A guard enforces an invariant
about the REPOSITORY, so a job exercising the PRODUCT is `none`" sounds more principled and is worse:
it puts `test` and `migrations` outside the population, and those are the two REQUIRED status
contexts on `main` — precisely where a failure to fire is fail-OPEN against branch protection, and
precisely the gap #786 exists to close. `test` also runs the C# and TypeScript structural guards
`docs/guard-inventory.md`'s scope-limit item 2 names, so "exercises the product" never described it.

THE RESIDUE, stated rather than left for review. This proves every guard JOB has been classified by
someone and has a row; it cannot prove the classification is CORRECT, and it cannot prove an inline
assertion works — that needs the job to run. `Proof: NONE` on an inline row is therefore the honest
entry and not a gap to be filled with a neighbouring file's proof. Two rows do cite a pytest, and
each names the PART of the job it covers, because a reference covering a fraction must not read as
covering the whole. Same residue `test_guard_inventory.py` records for its own table, and for the
same reason: the table is what review reads.
"""

from __future__ import annotations

import re
from pathlib import Path

import pytest
import yaml

from scripts.tests.tracked_files import tracked_paths

REPO_ROOT = Path(__file__).resolve().parents[2]
INVENTORY = REPO_ROOT / "docs" / "guard-inventory.md"
SECTION = "Workflow-job guards (ersatztv#786)"

# From the GIT INDEX, never `Path.glob` (ersatztv#806): an untracked scratch workflow left in the
# directory would otherwise join the population on one checkout and not in CI — red locally, green on
# the server, which is #778's third shape. `*.yaml` alongside `*.yml` because Gitea accepts both.
WORKFLOWS = (".gitea/workflows", ("*.yml", "*.yaml"))

ROLE_KEY = "CI_JOB_ROLE"
GUARD, REPORT_ONLY, NONE = "guard", "report-only", "none"
# A VOCABULARY, not a population — the legal values of the marker, not a list of jobs.
ROLES = frozenset({GUARD, REPORT_ONLY, NONE})
# role -> the Kind the row must carry. `none` is absent by construction: it takes no row.
ROLE_KIND = {GUARD: "GUARD", REPORT_ONLY: "REPORT-ONLY"}

# Keyed on the `<workflow>.yml::<job>` shape of the first cell, so this cannot match a row of the
# file-guard table (or any other) even before `inventory_section()` narrows the text.
#
# A MALFORMED ROW FOR AN OTHERWISE-UNLISTED JOB FAILS CLOSED, and the qualifier is the whole of it.
# `[^|]*` for the Blocks cell means a raw `|` inside it (a code span such as `grep … | cut …`) stops
# the row matching. When that job has no OTHER row, the unmatched row is an ABSENT row and the set
# equality reports a guard job with no entry — measured: injecting a pipe into one Blocks cell
# reddens `test_the_inventory_covers_exactly_the_guard_JOBS_that_exist` naming that job.
#
# THE CASE IT DOES NOT COVER, because an earlier version of this note claimed "every malformed row":
# a malformed DUPLICATE is invisible. Its job is already satisfied by the well-formed row, so set
# equality holds, `duplicate_row_faults` never sees the second occurrence, and the Kind check has
# nothing to compare. A contradictory row that also happens to be malformed therefore passes. No
# checker here can see it; a reader can.
_ROW = re.compile(r"^\|\s*`([a-z0-9.-]+\.ya?ml::[A-Za-z0-9_-]+)`\s*\|([^|]*)\|\s*([A-Z-]+)\s*\|", re.M)


def workflow_files() -> list[Path]:
    """THE WORKFLOW POPULATION, from the git index. Named so the shared proof in
    `test_guard_populations_derive_from_git.py` can assert it never admits an untracked file."""
    return tracked_paths(*WORKFLOWS)


def declared_roles(paths: list[Path] | None = None) -> dict[str, str | None]:
    """`<workflow>::<job>` -> the role it declares, or None where the marker is absent.

    None is KEPT rather than dropped: an unmarked job is the defect `role_faults` reports, and a dict
    that silently omitted it could not report anything.

    `paths` defaults to the tracked population and exists so a proof can drive THIS function over a
    fixture. Without the seam a test wanting to check the reader had to re-implement the read, which
    is how a proof ends up asserting a fact about PyYAML instead of about the checker.
    """
    out: dict[str, str | None] = {}
    for path in paths if paths is not None else workflow_files():
        doc = yaml.safe_load(path.read_text()) or {}
        for jid, job in (doc.get("jobs") or {}).items():
            if not isinstance(job, dict):
                continue
            value = (job.get("env") or {}).get(ROLE_KEY)
            out[f"{path.name}::{jid}"] = str(value) if value is not None else None
    return out


def env_shadow_faults(docs: list[tuple[str, dict]] | None = None) -> list[str]:
    """THE DETECTOR for a job-level `env:` key that shadows a workflow-level one.

    `docs` is `(name, parsed workflow)` pairs and defaults to the tracked population, so the proof
    below drives THIS function over a fixture instead of rebuilding the set intersection itself —
    a proof that reimplements the comparison stays green when the shipped one is disarmed.
    """
    if docs is None:
        docs = [(p.name, yaml.safe_load(p.read_text()) or {}) for p in workflow_files()]
    faults = []
    for name, doc in docs:
        workflow_env = set(doc.get("env") or {})
        if not workflow_env:
            continue
        for jid, job in (doc.get("jobs") or {}).items():
            if not isinstance(job, dict):
                continue
            clash = workflow_env & set(job.get("env") or {})
            if clash:
                faults.append(
                    f"{name}::{jid} declares job-level {sorted(clash)}, which also exists at "
                    "workflow level. The job value WINS and the workflow value is silently lost."
                )
    return faults


def role_faults(roles: dict[str, str | None]) -> list[str]:
    """MISSING OR UNKNOWN IS A HARD FAILURE.

    A marker whose absence defaults to `none` would stop applying the moment someone adds a job and
    forgets — and the job nobody remembered to mark is the job nobody reviewed. An unrecognised value
    is rejected for the same reason rather than read as "not a guard".
    """
    faults = []
    for job, value in sorted(roles.items()):
        if value is None:
            faults.append(
                f"job '{job}' declares no {ROLE_KEY}. Every job in every tracked workflow must "
                f"declare one under its `env:` — {sorted(ROLES)} — so that 'not a guard' is a "
                "recorded decision rather than an omission."
            )
        elif value not in ROLES:
            faults.append(f"job '{job}' declares {ROLE_KEY}: {value!r}, which is not one of {sorted(ROLES)}.")
    return faults


def inventory_section() -> str:
    """The body of the workflow-job section alone.

    Scoped, not a whole-file scan: `docs/guard-inventory.md` holds several tables and the file-guard
    one is read by `test_guard_inventory.py`. Two shape-keyed regexes over one document would each
    pull in the other's rows and report them as phantoms. A missing heading raises rather than
    returning empty — a silent empty section is the vacuous population this file exists to prevent.
    """
    text = INVENTORY.read_text()
    start = text.index(f"\n## {SECTION}\n")
    nxt = text.find("\n## ", start + 1)
    return text[start : nxt if nxt != -1 else len(text)]


def inventory_row_list() -> list[tuple[str, str]]:
    """(`<workflow>::<job>`, Kind) for each row, IN ORDER and WITH duplicates preserved."""
    return [(job.strip(), kind.strip()) for job, _blocks, kind in _ROW.findall(inventory_section())]


def duplicate_row_faults(rows: list[tuple[str, str]]) -> list[str]:
    """THE DETECTOR, named so its live test and its mutation proof share one implementation.

    An earlier draft inlined this loop in the live test and had the proof rebuild the comparison
    with its own `len(set(...))`. That proof stayed green with the live check disarmed, because it
    was exercising arithmetic it had written itself rather than the code that ships — the shape
    `testing.guard-ships-with-mutation-proof` calls a behavioural test dressed as a proof.
    """
    seen: dict[str, str] = {}
    dupes = []
    for job, kind in rows:
        if job in seen:
            dupes.append(f"'{job}' has rows claiming {seen[job]} and {kind}")
        seen[job] = kind
    return dupes


def kind_faults(roles: dict[str, str | None], listed: dict[str, str]) -> list[str]:
    """THE DETECTOR for a row whose Kind contradicts the job's marker. Shared with its proof."""
    return [
        f"'{job}' declares {ROLE_KEY}: {roles.get(job)} (Kind {ROLE_KIND.get(roles.get(job))}) but its row says {kind}"
        for job, kind in sorted(listed.items())
        if ROLE_KIND.get(roles.get(job)) != kind
    ]


def guard_jobs() -> set[str]:
    """The jobs whose declared role obliges them to hold a row."""
    return {j for j, r in declared_roles().items() if r in ROLE_KIND}


def coverage_faults(declared: set[str], listed: set[str]) -> tuple[set[str], set[str]]:
    """(missing, phantom) — THE DETECTOR the live check and its three proofs all call.

    Named for the same reason `duplicate_row_faults` is: three `test_MUTATION_*` below used to
    rebuild `declared - listed` inline, and a proof that reimplements the comparison stays green
    when the shipped one is disarmed — a behavioural test wearing a MUTATION name. Cold review
    demonstrated exactly that by flattening the live assertion and watching the file stay green.
    """
    return declared - listed, listed - declared


def inventory_jobs() -> dict[str, str]:
    """`<workflow>::<job>` -> Kind. One entry per job; see `test_no_job_has_TWO_rows`.

    A dict is the right shape for the comparisons below and the WRONG shape for detecting a
    duplicate: two rows for one job collapse, last one wins, and a table that contradicts itself
    reports green. That is checked separately against `inventory_row_list`, which keeps duplicates,
    rather than by making every caller handle a list.
    """
    return dict(inventory_row_list())


# ------------------------------------------------------------------------------------------------
# ANTI-VACUITY FIRST — a population or a row regex that stopped matching would make every assertion
# below compare two empty sets and report a fully-covered inventory. That is the failure this file
# exists to prevent, so it is checked before anything depends on it.
# ------------------------------------------------------------------------------------------------


def test_the_workflow_population_is_not_empty():
    files = workflow_files()
    assert files, (
        "no tracked workflow files were found — `tracked_paths` returned nothing. Every assertion "
        "below would compare empty sets and pass having checked nothing."
    )
    roles = declared_roles()
    assert roles, f"{len(files)} workflow file(s) parsed but no jobs were found in any of them"


def test_the_workflow_job_table_actually_parsed():
    rows = inventory_jobs()
    assert rows, (
        f"the `## {SECTION}` table parsed to ZERO rows. Either the table is gone or `_ROW` no "
        "longer matches its shape; both make the set equality below vacuous."
    )


# ------------------------------------------------------------------------------------------------
# THE LIVE ASSERTIONS
# ------------------------------------------------------------------------------------------------


def test_every_job_declares_a_known_role():
    faults = role_faults(declared_roles())
    assert not faults, "workflow jobs with no usable role:\n  " + "\n  ".join(faults)


def test_the_inventory_covers_exactly_the_guard_JOBS_that_exist():
    """SET EQUALITY, both directions — the two failures are opposite mistakes.

    Left-to-right: a job declared a guard with no row, which is the gap #786 was filed for.
    Right-to-left: a row for a job that no longer declares itself a guard (renamed, deleted, or
    downgraded), which is how a table becomes a comforting fiction.
    """
    missing, phantom = coverage_faults(guard_jobs(), set(inventory_jobs()))
    assert missing == set(), (
        f"these workflow jobs declare {ROLE_KEY} guard/report-only but have NO row in "
        f"`## {SECTION}` — {sorted(missing)}. Add a row saying what each blocks, where "
        "its assertion lives and what proof it carries."
    )
    assert phantom == set(), (
        f"`## {SECTION}` has rows for jobs that no longer declare a guard/report-only role — "
        f"{sorted(phantom)}. Either the job was renamed or removed, or its marker "
        "changed; the row must follow."
    )


def test_no_job_has_TWO_rows():
    """A duplicate row is a self-contradicting registry that every other check reports green.

    `inventory_jobs()` is a dict, so a second row for the same job silently overwrites the first and
    set equality still holds in both directions. The realistic arrival is a merge conflict or a
    copy/paste that leaves a `REPORT-ONLY` and a `GUARD` row for one job: the table then states both,
    the checker picks whichever is later, and nothing reports the contradiction. Counting rows
    against distinct keys is the only place that can see it.
    """
    rows = inventory_row_list()
    assert rows, "the table parsed to zero rows, so this check would be vacuous"
    dupes = duplicate_row_faults(rows)
    assert not dupes, (
        f"`## {SECTION}` lists the same job more than once:\n  "
        + "\n  ".join(dupes)
        + "\n\nThe dict built from these rows keeps only the LAST, so every other assertion in this "
        "file passes while the table contradicts itself."
    )


def test_MUTATION_a_DUPLICATE_row_is_reported():
    """Proof for the check above, with its own negative control."""
    rows = inventory_row_list()
    assert not duplicate_row_faults(rows), "NEGATIVE CONTROL FAILED: the unmutated table already contains a duplicate."
    victim = rows[0][0]
    mutated = rows + [(victim, "REPORT-ONLY" if rows[0][1] != "REPORT-ONLY" else "GUARD")]
    faults = duplicate_row_faults(mutated)
    assert faults and victim in faults[0], f"the DETECTOR did not report a second row for '{victim}': {faults}"


def test_each_row_carries_the_KIND_its_marker_declares():
    """The row and the workflow must not disagree about WHICH kind.

    Set equality alone is satisfied by a `report-only` job holding a `GUARD` row — the table would then
    claim a merge-blocking check where the workflow has one that cannot fail. `docs-reminder` is
    exactly that job (`continue-on-error: true` on every check step), so the case is live, not
    hypothetical.
    """
    faults = kind_faults(declared_roles(), inventory_jobs())
    assert not faults, "row Kind disagrees with the job's declared role:\n  " + "\n  ".join(faults)


def test_no_job_env_key_SHADOWS_a_workflow_level_one():
    """Adding a job-level `env:` to a workflow that has a workflow-level one must not shadow it.

    WHY THIS EXISTS. Before ersatztv#786 no job in `docker-build.yml` declared `env:` at all, and its
    workflow-level `env:` is memory-critical: `UseSharedCompilation`,
    `DOTNET_CLI_USE_MSBUILD_SERVER` and `MSBUILDDISABLENODEREUSE` are what keep a 7.8 GB
    VBCSCompiler off a RAM-oversubscribed runner (ersatztv#406). Actions merges the two scopes with
    the more specific one winning, so a job-level key of the SAME NAME silently replaces the
    workflow-level value — and the loss is invisible: the job still runs, just with the shared
    compiler back on.

    THAT THE SCOPES LAYER RATHER THAN REPLACE is not assumed here, and the evidence is in-repo and
    live: `build`'s "Smoke + IPTV E2E" step declares its own step-level `env:`
    (`SMOKE_SHORT_SHA`, `SMOKE_RUN_ID`) and on the same line consumes the WORKFLOW-level `${IMAGE}`
    — `IMG="${IMAGE}:${SMOKE_SHORT_SHA}"` — on pushes to `main` and `v*` tags (its `if:` also skips
    a docs-only push). A replacing runner would have broken that long ago.

    THE GAP IN THAT EVIDENCE, stated because it is a scope narrower than the claim: it is STEP-level
    layering, and what this branch newly relies on is JOB-level. On `origin/main` no job anywhere
    declared both a job-level `env:` and a consumed workflow-level var, so the job scope had no
    in-repo witness. The inference is that a runner layering one scope layers the other; the
    exposure if it did not is the two publish jobs — `ci-image.yml::build` reads `CI_IMAGE` and
    `docker-build.yml::build` reads `IMAGE`. NEITHER RUNS ON A PR, by two different mechanisms worth
    keeping straight: `docker-build.yml::build` carries `if: github.event_name != 'pull_request'`,
    while `ci-image.yml` declares no `pull_request` TRIGGER at all, so its job is never created on a
    PR rather than being created and skipped. Either way PR CI cannot exercise the job scope, and
    the first push to `main` would try to publish `":<sha>"` and fail loudly rather than silently.

    So this check is not what proves merging; it removes the remaining exposure, which is a NAME
    COLLISION. Merging is only safe while no job-level key shadows a workflow-level one, and that is
    a property of the files rather than of the runner, so it is the half worth asserting here.

    It also catches the reverse mistake — someone "documenting" `REGISTRY` on a job and pinning it to
    a stale value.
    """
    faults = env_shadow_faults()
    assert not faults, "job-level env shadows a workflow-level env key:\n  " + "\n  ".join(faults)


def test_MUTATION_a_SHADOWING_env_key_is_reported():
    """Proof for the check above, with its own negative control."""
    doc = {
        "env": {"UseSharedCompilation": "false"},
        "jobs": {"a": {"env": {"CI_JOB_ROLE": "none"}}},
    }
    assert not env_shadow_faults([("fixture.yml", doc)]), (
        "NEGATIVE CONTROL FAILED: the unmutated fixture already reports a collision"
    )
    doc["jobs"]["a"]["env"]["UseSharedCompilation"] = "true"
    faults = env_shadow_faults([("fixture.yml", doc)])
    assert faults and "UseSharedCompilation" in faults[0], (
        f"the DETECTOR did not report a shadowing job-level key: {faults}"
    )


# ------------------------------------------------------------------------------------------------
# MUTATION PROOFS — THE GUARD IS A TEST, so each mutation goes into the guarded ARTIFACT (the
# workflow, or the table) rather than into an assertion here. Disarming a checker makes it ABSENT
# rather than red; `testing.guard-ships-with-mutation-proof`'s checker-guard exception is what makes
# introducing the real defect the admissible proof.
# ------------------------------------------------------------------------------------------------


def _role_mutants():
    for job in sorted(declared_roles()):
        removed = dict(declared_roles())
        removed[job] = None
        yield f"{job}-marker-removed", removed

        unknown = dict(declared_roles())
        unknown[job] = "checker"
        yield f"{job}-marker-unknown-value", unknown


_ROLE_MUTANTS = list(_role_mutants())


@pytest.mark.parametrize("roles", [m for _, m in _ROLE_MUTANTS], ids=[i for i, _ in _ROLE_MUTANTS])
def test_a_missing_or_UNKNOWN_role_is_DETECTED(roles):
    assert role_faults(roles), "a job with no role, or an unrecognised one, was accepted"


def test_the_role_mutation_set_is_not_empty():
    """The NON-EMPTINESS assertion comes first, and it is the whole point of this test.

    `len(_ROLE_MUTANTS) == 2 * len(declared_roles())` alone is `2N == 2N` — a tautology that holds
    at N=0, so a `declared_roles()` degraded to empty would satisfy it while every parametrised
    proof above collected zero cases and the file reported green. That is the vacuous-positive-
    control shape this suite exists to prevent, reproduced inside its own control.
    """
    roles = declared_roles()
    assert roles, (
        "declared_roles() is EMPTY, so every role mutation above is vacuous and the parametrised "
        "proofs collected nothing. Either the workflow population broke or the marker key moved."
    )
    expected = 2 * len(roles)
    assert len(_ROLE_MUTANTS) == expected, f"expected 2 role mutations per job ({expected}), got {len(_ROLE_MUTANTS)}"


def test_MUTATION_a_guard_job_with_its_row_DELETED_is_reported():
    """The defect this guard exists to catch: a guard job that acquired no row.

    Mutating the guarded ARTIFACT (the table), not this checker's population — a shrunken population
    would make every real row report as a PHANTOM, demonstrating a false POSITIVE while proving
    nothing about the missing-row detection the inventory row claims.
    """
    declared = guard_jobs()
    listed = set(inventory_jobs())
    assert declared and listed, "nothing to mutate — the population or the table is empty"
    # NEGATIVE CONTROL, the same one its phantom twin carries. Without it a PRE-EXISTING
    # `declared - listed` gap satisfies every iteration below, so the loop would pass while proving
    # nothing about the deletion — the live assertion would be red in that state, but this proof
    # would be claiming coverage it does not have.
    assert coverage_faults(declared, listed)[0] == set(), (
        f"NEGATIVE CONTROL FAILED: guard jobs {sorted(coverage_faults(declared, listed)[0])} "
        "already have no row, so every deletion below would be satisfied by the pre-existing gap."
    )
    for victim in sorted(listed):
        missing, _ = coverage_faults(declared, listed - {victim})
        assert missing == {victim}, (
            f"the DETECTOR did not report deleting the row for '{victim}' as exactly that job missing"
        )


def test_MUTATION_a_PHANTOM_row_is_reported():
    """The opposite direction: a row for a job that does not declare a guard role.

    Carries its own NEGATIVE control. Asserting only that the mutated set contains a phantom is
    true even when `inventory_jobs()` returns nothing — the added literal supplies the phantom by
    itself, so the assertion could not fail and proved nothing about the parser. Pinning the
    UNMUTATED set to zero phantoms first is what makes the second assertion mean anything.
    """
    declared = guard_jobs()
    listed = set(inventory_jobs())
    assert listed, "the table parsed to nothing, so this proof would be vacuous"
    assert coverage_faults(declared, listed)[1] == set(), (
        f"NEGATIVE CONTROL FAILED: the unmutated table already reports phantoms "
        f"{sorted(coverage_faults(declared, listed)[1])}, so the mutation below would prove nothing."
    )
    phantom = "docker-build.yml::a-job-that-was-deleted"
    _, reported = coverage_faults(declared, listed | {phantom})
    assert reported == {phantom}, (
        "the DETECTOR did not report a row naming a non-guard job as exactly the phantom introduced"
    )


def test_MUTATION_a_row_whose_KIND_disagrees_is_reported():
    """`report-only` holding a `GUARD` row must not pass — the case `docs-reminder` makes live."""
    roles = declared_roles()
    report_only = [j for j, r in roles.items() if r == REPORT_ONLY]
    assert report_only, "no report-only job exists, so this proof is vacuous"
    listed = dict(inventory_jobs())
    assert not kind_faults(roles, listed), (
        "NEGATIVE CONTROL FAILED: the unmutated table already disagrees with a marker"
    )
    listed[report_only[0]] = "GUARD"
    faults = kind_faults(roles, listed)
    assert faults and report_only[0] in faults[0], (
        f"the DETECTOR did not report a report-only job carrying a GUARD row: {faults}"
    )


def test_a_workflow_job_added_without_a_marker_is_DETECTED():
    """A NEW job is the realistic arrival path, and it must not default to `none`."""
    roles = dict(declared_roles())
    roles["docker-build.yml::a-brand-new-job"] = None
    assert role_faults(roles), "a newly added job with no role marker was accepted"


def test_every_tracked_job_APPEARS_in_the_parse(tmp_path):
    """Key completeness, plus the ATTRIBUTION property a line-based reader would break.

    The first half is completeness only — every job of every tracked workflow appears as a key. It
    deliberately does NOT claim to catch a grep-based reader: a grep returning the right keys with
    wrong VALUES satisfies it. That regression is covered by
    `test_a_commented_out_marker_does_NOT_count`, which drives the reader over a fixture.

    The second half is the part a completeness check cannot express and that is worth its own
    fixture: a marker belongs to the job whose `env:` block contains it. A line-oriented reader that
    scanned for `CI_JOB_ROLE:` and attached the value to the nearest preceding job header would
    SMEAR the marker of one job onto a neighbour that declares none — and the smear is invisible to
    any assertion phrased over keys, because both jobs have keys either way.
    """
    roles = declared_roles()
    for path in workflow_files():
        doc = yaml.safe_load(path.read_text()) or {}
        for jid in doc.get("jobs") or {}:
            assert f"{path.name}::{jid}" in roles, f"{path.name}::{jid} was not parsed"
    assert all("::" in k for k in roles)

    wf = tmp_path / "two.yml"
    wf.write_text(f"jobs:\n  first:\n    steps: []\n  second:\n    env:\n      {ROLE_KEY}: {GUARD}\n    steps: []\n")
    assert declared_roles([wf]) == {"two.yml::first": None, "two.yml::second": GUARD}, (
        "the reader attributed a marker to the wrong job — `first` declares none and must read as "
        "None, `second` declares one and must read as its value"
    )


def test_a_commented_out_marker_does_NOT_count(tmp_path):
    """A commented-out marker must read as ABSENT — asserted through `declared_roles` ITSELF.

    An earlier draft called `yaml.safe_load` on a literal and asserted comments are not values. That
    is a fact about PyYAML, which this suite does not own and no change here could break: replacing
    `declared_roles` with a grep-based reader left that version GREEN. It now drives the real reader
    over a real file through the `paths` seam, so a reader that ever became a text scan fails here.
    """
    wf = tmp_path / "commented.yml"
    wf.write_text("jobs:\n  x:\n    env:\n      # CI_JOB_ROLE: guard\n      OTHER: 1\n    steps: []\n")

    roles = declared_roles([wf])
    assert roles == {"commented.yml::x": None}, (
        f"the real reader returned {roles!r}; a commented-out marker must read as absent"
    )
    # And the fault function must REPORT it, not merely fail to see it.
    assert role_faults(roles), "a job whose only marker is commented out was accepted as classified"

    # The text a grep-based reader would match IS in the file — which is what makes this a real
    # guard against the parser regressing to a text scan rather than a restatement of YAML rules.
    assert ROLE_KEY in wf.read_text()

    # A REAL marker in the same shape must still be read, so the assertion above cannot be satisfied
    # by a reader that returns None for everything.
    ok = tmp_path / "live.yml"
    ok.write_text("jobs:\n  x:\n    env:\n      CI_JOB_ROLE: guard\n    steps: []\n")
    assert declared_roles([ok]) == {"live.yml::x": GUARD}, "the reader stopped seeing a real marker"