ersatztv/scripts/tests/tracked_files.py

"""The authoritative population for a guard whose members are FILES: the git index (ersatztv#806).

`testing.guard-derives-population-from-source` (#774) says a completeness guard derives its
population from a machine-readable authoritative source, and its worked examples are an enum and the
generated OpenAPI document. It does not say what to do when the population is *files*, and every
guard in this repo answered that with a filesystem walk. **A filesystem walk is not an authoritative
source.** It reports build output, editor droppings and whatever else happens to be on disk, and it
differs per machine, so a guard derived from it asserts a different population in CI than on the
laptop of the person it is supposed to stop.

#778 got that wrong three times in one PR, each time with an argument for why the traversal
sufficed, and the third shape is the one that motivates this module:

  * a **content filter** on outbound-network tokens that omitted `git fetch`, making a hook that
    fetches `origin/main` and derives a push decision structurally invisible;
  * a **non-recursive `Path.glob`**, missing four nested files, one of which calls a live ErsatzTV
    API and acts on the reply;
  * **`Path.rglob`**, which then enumerated `.husky/_/` — 17 husky shims generated by `npm ci` via
    `web/package.json`'s `prepare` script, gitignored (`.husky/_/.gitignore` is `*`) and untracked.
    That made the guard **RED on every developer checkout and GREEN in CI**, whose `script-tests`
    job checks out and pip-installs but never runs `npm ci`. A guard that fails everywhere except
    where it runs teaches its readers to ignore it.

The index holds the same set of files every checkout receives from a clone, and excludes untracked
generated files **by construction** rather than by an exclusion list somebody has to maintain and
keep correct. It is not immutable and it is per-worktree; the claim is not that it never changes,
but that it changes only through a deliberate git operation — staging, a checkout, a reset, a
merge — whereas the disk changes whenever a build runs.
Note what that buys over `.gitignore`-awareness: `.husky/_/` happens to carry its own `.gitignore`,
but a stray untracked `foo.sh` in `.claude/hooks/` carries nothing, and only the index knows it is
not part of the repo.

**This is not "replace every glob".** The question per guard is whether it makes a COMPLETENESS
claim over tracked files. If it does, the index is the authoritative source. If it does not — a
fixture copying files into a tmp tree, a walk selecting the SUBJECT of a per-member property — say
so in the guard and leave it, per the boundary #774 already draws between scope and population.

`git ls-files` lists INDEX entries, so a file deleted in the working tree but not yet staged is
still reported. That is deliberate: callers that read member contents assert existence with their
own message rather than filtering, because filtering is what makes a missing member unrepresentable.
"""

from __future__ import annotations

import fnmatch
import subprocess
from pathlib import Path

REPO_ROOT = Path(__file__).resolve().parents[2]


def _git_ls_files() -> list[str]:
    """Every path git tracks, as repo-relative posix strings.

    The single place this package shells out to git, so a regression proof can narrow the tracked
    set once and have every derivation built on it react.

    Fails LOUDLY on an empty result rather than returning it: an empty population makes every
    completeness assertion downstream pass vacuously, which is the exact failure these guards exist
    to prevent.
    """
    proc = subprocess.run(
        ["git", "-C", str(REPO_ROOT), "ls-files", "-z"],
        capture_output=True,
        check=False,
    )
    # `check=True` would raise "returned non-zero exit status 128" and leave git's own diagnostic
    # trapped in `e.stderr`. The two real triggers — a tree that is not a repository, and CI's
    # `detected dubious ownership` — are both diagnosable ONLY from that text, and every module
    # that imports this one would otherwise die with the same inscrutable line.
    assert proc.returncode == 0, (
        f"`git ls-files` failed in {REPO_ROOT} (exit {proc.returncode}). Every file population here "
        f"comes from the index, so this is fatal rather than empty. git said:\n"
        f"{proc.stderr.decode(errors='replace').strip() or '(no stderr)'}"
    )
    paths = [p for p in proc.stdout.decode().split("\0") if p]
    assert paths, (
        f"`git ls-files` reported nothing under {REPO_ROOT} — the derivation is broken, not the "
        "repo. Every file population built on it would be empty and every completeness assertion "
        "would pass vacuously."
    )
    return paths


def tracked_children(directory: str, patterns: tuple[str, ...]) -> set[str]:
    """Tracked files that are DIRECT children of `directory` and match one of `patterns`.

    Direct children only, and that is the point rather than a limitation: every population here is
    a flat directory (`.claude/hooks/*.sh`, `.husky/*`, `.gitea/workflows/*.yml`,
    `scripts/tests/test_*.py`), and recursing is what dragged `.husky/_/` in. A guard that genuinely
    needs a nested population should say so and ask for it explicitly.

    Returns repo-relative posix paths, matching what the guards' inventories and error messages use.
    """
    found: set[str] = set()
    for path in _git_ls_files():
        parent, _, name = path.rpartition("/")
        if parent != directory:
            continue
        if any(fnmatch.fnmatch(name, pattern) for pattern in patterns):
            found.add(path)
    return found


def tracked_under(directory: str) -> set[str]:
    """Tracked files anywhere BENEATH `directory`, recursively.

    This is the explicit nested ask `tracked_children` directs callers to when direct children are
    not enough. It decides whether a path names a directory at all: a path with nothing beneath it
    is a file or absent, and a caller that assumed a directory would otherwise compare two spellings
    that select different sets.
    """
    prefix = directory.rstrip("/") + "/"
    return {path for path in _git_ls_files() if path.startswith(prefix)}


def tracked_paths(directory: str, patterns: tuple[str, ...]) -> list[Path]:
    """`tracked_children` as absolute `Path`s, sorted — for guards that read member contents.

    Existence is ASSERTED, never filtered: a path in the index with no file on disk means the tree
    is mid-edit, and reporting that is strictly better than silently shrinking the population, which
    is the defect this module exists to remove.
    """
    paths = []
    for rel in sorted(tracked_children(directory, patterns)):
        absolute = REPO_ROOT / rel
        assert absolute.is_file(), (
            f"git tracks {rel} but there is no file there. The population comes from the index, so "
            "a working tree mid-delete is reported rather than silently shrinking the population."
        )
        paths.append(absolute)
    return paths