Files
ersatztv/scripts/tests/test_complete_annotation_dispositions.py
T
timothyandtimothy e8f80c42ce
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 9s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 28s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 14m22s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 11m1s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 9m58s
Build ErsatzTV Image / Build & push image (amd64) (push) Failing after 1m58s
fix(820): derive where Complete<T> is APPLIED, not just what it means (#883)
`Complete<T>` (#807) makes SPA full-replace bodies fail typecheck when a builder omits a schema member. Nothing checked it was APPLIED: `completeRequest.guard.test.ts` proves the type's semantics and would stay green with every annotation deleted, and `test_optional_request_members.py`'s COVERED disposition — "the builder is annotated `Complete<T>`" — was a claim about another language's source that nothing verified.

Adds `completeAnnotationScan.ts` (compiler-API scanners) + `completeAnnotations.guard.test.ts`, with a synthetic-source fixture suite. Two derived populations: the `Complete<…>` annotations (SPA AST ∩ git index) and the droppable schemas (parsed from the generated `v1.d.ts`, a pass-through of the OpenAPI `required` array). It asserts a production annotation per schema dispositioned as needing one, NO annotation on the server-computed and load-bearing-omission schemas, that every `Complete<X>` resolves to a generated schema rather than a hand-written mirror, and set equality between droppable schemas and the reviewed dispositions. `test_complete_annotation_dispositions.py` cross-checks that table against the authoritative Python one and ships a declared, harness-executed mutation.

Found one live defect: `playouts.ts` declared two request types as hand-written mirrors SHADOWING generated schemas of the same name, so their `Complete<>` was checking a local copy rather than the contract — the #754 mechanism wearing the annotation meant to prevent it.

Eight review rounds, seven BLOCKED, two independent cold reviewers. A wrapper-signature scanner was built and REMOVED: every blocker traced to that one mechanism (obligation on the wrong population; reachability mistaken for protection, since `Complete<T>` is shallow; body discovery keyed on a parameter name, then parameter-vs-local; and finally `export function` → `export const` blinding the scanner and its cross-check together). Five defects from one mechanism, so the mechanism went rather than a sixth patch.

Residuals stated in §4b, the guard-inventory row and the record: per-SCHEMA not per-site or per-wrapper; token presence not liveness; the phantom direction unchecked (#777); a second `setupFiles` entry could discharge; and plugin-level population integrity borrowed from the sibling guard.

fixes #820

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-29 22:06:38 +00:00

159 lines
8.4 KiB
Python

"""ersatztv#820: the SPA guard's disposition table may not silently disagree with this one.
WHAT THIS BLOCKS. `web/src/api/completeAnnotations.guard.test.ts` carries a `DISPOSITIONS` table
saying, per schema, whether the SPA must annotate it `Complete<T>`, must not, or need not. It derives
its POPULATION (which schemas can drop a member) from the generated types and asserts set equality
against those keys — so a new optional member cannot ship undispositioned.
It did not check the VALUE, and cold review demonstrated that the value is a kill switch: flip
`MultiCollectionItemRequest` from `ANNOTATED` to `CREATE` and delete the `Complete<…>` from
`MultiCollectionsScreen.toItemRequest`, and the whole suite stays green while the defect #807 exists
to prevent — every weight silently reset to 1 on save — is live again. The row's own note still said
"a prose comment warned about it, and a comment is not a check", which is what the flip made of it.
WHY THE CHECK LIVES HERE. `testing.guard-derives-population-from-source`: "When the scope itself
MIRRORS an authoritative source, the mirror needs its own equality check against that source."
`test_optional_request_members.py` IS that source — it derives from `ErsatzTV/wwwroot/openapi/v1.json`
every run and its dispositions are reviewed. The TS table mirrors nine of its rows. Putting the
comparison on the Python side is what makes it cheap: this process already holds the authoritative
table and can read a file, whereas the SPA guard runs under Vite with no filesystem access to
`scripts/` and would need a virtual-module plugin to see it.
THE MAPPING IS ASYMMETRIC ON PURPOSE, and stating why is half the check:
COVERED -> ANNOTATED exact. This is the row whose claim the SPA guard verifies.
COMPUTED -> MUST-NOT-ANNOTATE exact. Annotating would force a caller to invent server-computed
values, so the prohibition must survive on both sides.
CREATE -> CREATE | MUST-NOT-ANNOTATE
TRIGGER -> TRIGGER | MUST-NOT-ANNOTATE
a STRICTER call on the SPA side is allowed, because forbidding an
annotation is never the unsafe direction. This is not hypothetical
slack: `CreateChannelFromLineupAdvancedOptionsRequest` is CREATE
here and MUST-NOT-ANNOTATE there, because an omitted override means
INHERIT the template value and `Complete<T>` would collapse that
third state into an explicit null.
Schemas the TS table holds that this one does not are NOT an error: that table scopes to every
generated schema with an optional member, this one to schemas reachable from a request body, so it
additionally sees response models. The reverse IS an error — a schema ruled on here and absent there
means the SPA guard is not covering something the API can drop.
"""
from __future__ import annotations
import importlib.util
import re
from pathlib import Path
import pytest
REPO_ROOT = Path(__file__).resolve().parents[2]
TS_GUARD = REPO_ROOT / "web" / "src" / "api" / "completeAnnotations.guard.test.ts"
# `Name: {` then an optional run of `//` comment lines, then `disposition: '...'`. Anchored on the
# two-space indent the file's formatter produces for a top-level entry, so a `disposition:` appearing
# in prose or in a nested object cannot be mistaken for a row.
_TS_ROW = re.compile(
r"^ (?P<schema>\w+): \{\n(?:\s*//[^\n]*\n)*\s*disposition: '(?P<disposition>[A-Z-]+)'",
re.MULTILINE,
)
# The compact single-line form the formatter emits for short rows.
_TS_ROW_INLINE = re.compile(r"^ (?P<schema>\w+): \{ disposition: '(?P<disposition>[A-Z-]+)'", re.MULTILINE)
ALLOWED = {
"COVERED": {"ANNOTATED"},
"COMPUTED": {"MUST-NOT-ANNOTATE"},
"CREATE": {"CREATE", "MUST-NOT-ANNOTATE"},
"TRIGGER": {"TRIGGER", "MUST-NOT-ANNOTATE"},
}
# A key that is not a bare identifier does not name a generated schema: `test_optional_request_members`
# also rules on INLINE request bodies, which it keys by endpoint (`POST /api/v1/artwork/uploads
# (multipart/form-data inline body)`). Those have no `components['schemas'][…]` entry, so the SPA
# guard — whose population is parsed out of the generated types — cannot carry a row for them and
# their absence is not a hole. `_comparable` is what keeps that exemption from widening into one: it
# is a SHAPE test, not a name list, so a new inline body is exempt automatically while a new named
# schema can never be.
_SCHEMA_NAME = re.compile(r"^\w+$")
def _comparable(schema: str) -> bool:
return _SCHEMA_NAME.match(schema) is not None
def _python_dispositions() -> dict[str, str]:
spec = importlib.util.spec_from_file_location(
"_optional_request_members", Path(__file__).with_name("test_optional_request_members.py")
)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return {schema: value[0] for schema, value in module.DISPOSITIONS.items()}
def _ts_dispositions() -> dict[str, str]:
text = TS_GUARD.read_text(encoding="utf-8")
found = {m.group("schema"): m.group("disposition") for m in _TS_ROW.finditer(text)}
found.update({m.group("schema"): m.group("disposition") for m in _TS_ROW_INLINE.finditer(text)})
return found
def test_the_TS_table_is_parseable_at_all() -> None:
"""Anti-vacuity, and the only thing standing between a formatting change and a silent no-op.
Every comparison below is over an intersection. If the regex stops matching — a prettier setting,
a rename, a move — the intersection empties and every other test here passes having compared
nothing. This is the check that turns that into a failure.
"""
ts = _ts_dispositions()
assert len(ts) >= 10, (
f"parsed only {len(ts)} rows out of {TS_GUARD.relative_to(REPO_ROOT)}; the table has more than "
f"that, so the parse is broken rather than the table being small. Every comparison in this "
f"file is over an intersection and would pass vacuously."
)
assert set(ts.values()) <= {"ANNOTATED", "MUST-NOT-ANNOTATE", "CREATE", "TRIGGER", "RESPONSE-ONLY"}
def test_the_PYTHON_table_is_non_empty_too() -> None:
"""The TS side has a floor; without one here the comparisons are still vacuous from the other end.
Every check below iterates the Python table. An empty one collects zero parametrised cases and
makes the missing-schema check pass over an empty set — green, having compared nothing. The
neighbour file has its own floor, but a guard that depends on a neighbour's floor for its own
anti-vacuity is borrowing the property rather than holding it.
"""
assert len(_python_dispositions()) >= 5
def test_the_inline_body_exemption_stays_small_and_shaped() -> None:
"""The exemption above is a shape test; this bounds how much it can be carrying."""
exempt = sorted(s for s in _python_dispositions() if not _comparable(s))
assert len(exempt) <= 2, f"more inline-body keys than expected: {exempt}"
assert all(" " in s or "/" in s for s in exempt), f"an exempt key looks like a schema name: {exempt}"
def test_every_schema_ruled_on_HERE_is_also_ruled_on_in_the_SPA_guard() -> None:
missing = sorted(s for s in _python_dispositions() if _comparable(s) and s not in _ts_dispositions())
assert not missing, (
f"schema(s) dispositioned in test_optional_request_members.py with no row in the SPA guard: "
f"{missing}. The API can drop a member of each, and the guard that checks the SPA's "
f"annotations is not covering it."
)
@pytest.mark.parametrize("schema", sorted(s for s in _python_dispositions() if _comparable(s)))
def test_the_two_dispositions_AGREE(schema: str) -> None:
"""The bypass this closes: the TS value alone decided whether an annotation was required."""
python_disposition = _python_dispositions()[schema]
ts = _ts_dispositions()
if schema not in ts:
pytest.skip("absence is asserted separately, so it is not re-reported per schema here")
allowed = ALLOWED[python_disposition]
assert ts[schema] in allowed, (
f"{schema}: this file says {python_disposition}, the SPA guard says {ts[schema]}, and "
f"{python_disposition} admits only {sorted(allowed)}. Flipping the SPA-side value is how a "
f"required `Complete<…>` annotation stops being required — decide which table is wrong."
)