fix(610): make the frontmatter READ path dependency-free — CI has no PyYAML
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 12s
PR Gates / Docs update reminder (pull_request) Successful in 17s
PR Gates / decisions lifecycle (pull_request) Successful in 17s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m15s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 6s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 16m41s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 20m6s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 12s
PR Gates / Docs update reminder (pull_request) Successful in 17s
PR Gates / decisions lifecycle (pull_request) Successful in 17s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m15s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 6s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 16m41s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 20m6s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
This is why `decisions lifecycle` went red, and it was NOT the known flake. I came close to dismissing it as one for the second time this session, because an earlier red on another branch genuinely was. The dual-format parser imported PyYAML to read frontmatter. `decisions-guard` does `setup-python` and installs NOTHING, so once the corpus was migrated every record became unparseable there: ModuleNotFoundError, job fails. The same would hit the Husky pre-commit hook and every contributor's machine. Installing PyYAML in CI is the wrong fix: READING happens everywhere -- CI, the hook, every dev -- while WRITING happens once, in a migration a human runs deliberately. So the read path is now dependency-free and only `migrate_decisions_split` (the writer) still imports yaml. A hand-rolled parser is only safe if it provably matches the library that WROTE the files, so `test_frontmatter_reader_matches_pyyaml_on_every_real_record` compares the two field-by-field across all 169 real records (importorskip, so it is skipped rather than failing where PyYAML is absent) with a >100-file guard against near-vacuity. It is narrow by construction: the frontmatter is machine-generated with default_flow_style=False and width=10**9, so every value is a single-line scalar, and the reader bails to None on anything nested. Verified by running all four affected entry points against a shim that makes `import yaml` raise: validate --base/--head, catalog --check, the kickoff guard, and the plain validate the pre-commit hook calls. All exit 0. refs #610
This commit is contained in:
+54
-11
@@ -148,9 +148,56 @@ def has_frontmatter(text: str) -> bool:
|
||||
return bool(lines) and lines[0].rstrip() == _FM_DELIM
|
||||
|
||||
|
||||
def _parse_frontmatter(text: str, source: Path) -> list[Record]:
|
||||
import yaml # local import: only the new form needs it
|
||||
def _unquote(raw: str) -> str:
|
||||
"""Undo the quoting `yaml.safe_dump` applies. Single-line scalars only.
|
||||
|
||||
The frontmatter this reads is MACHINE-GENERATED by `migrate_decisions_split.render_record`
|
||||
with `default_flow_style=False` and `width=10**9`, so every value is a single-line scalar,
|
||||
either plain or quoted. That narrowness is what makes a dependency-free reader safe.
|
||||
"""
|
||||
raw = raw.strip()
|
||||
if len(raw) >= 2 and raw[0] == raw[-1] == "'":
|
||||
return raw[1:-1].replace("''", "'")
|
||||
if len(raw) >= 2 and raw[0] == raw[-1] == '"':
|
||||
body = raw[1:-1]
|
||||
out, i = [], 0
|
||||
while i < len(body):
|
||||
if body[i] == "\\" and i + 1 < len(body):
|
||||
nxt = body[i + 1]
|
||||
out.append({"n": "\n", "t": "\t", '"': '"', "\\": "\\"}.get(nxt, nxt))
|
||||
i += 2
|
||||
else:
|
||||
out.append(body[i])
|
||||
i += 1
|
||||
return "".join(out)
|
||||
return raw
|
||||
|
||||
|
||||
def _read_frontmatter(block: str) -> dict[str, str] | None:
|
||||
"""Parse the frontmatter block WITHOUT PyYAML. None if it doesn't look like our format.
|
||||
|
||||
Deliberately dependency-free: this read path runs in CI's `decisions lifecycle` job, in the
|
||||
Husky pre-commit hook, and on every contributor's machine. Requiring PyYAML there made the
|
||||
validator crash with ModuleNotFoundError on a runner that installs nothing — the split's own
|
||||
records became unparseable. Writing still uses PyYAML (`migrate_decisions_split`), because that
|
||||
is a one-shot run by a human who can install it.
|
||||
|
||||
`test_frontmatter_reader_matches_pyyaml` asserts this agrees with PyYAML on every real record.
|
||||
"""
|
||||
meta: dict[str, str] = {}
|
||||
for line in block.splitlines():
|
||||
if not line.strip() or line.lstrip().startswith("#"):
|
||||
continue
|
||||
if line[:1].isspace():
|
||||
return None # nested/continued structure — not the flat form we emit
|
||||
k, sep, v = line.partition(":")
|
||||
if not sep:
|
||||
return None
|
||||
meta[k.strip()] = _unquote(v)
|
||||
return meta
|
||||
|
||||
|
||||
def _parse_frontmatter(text: str, source: Path) -> list[Record]:
|
||||
lines = text.splitlines()
|
||||
end = None
|
||||
for i, ln in enumerate(lines[1:], start=1):
|
||||
@@ -159,21 +206,17 @@ def _parse_frontmatter(text: str, source: Path) -> list[Record]:
|
||||
break
|
||||
if end is None:
|
||||
return [] # unterminated frontmatter — malformed; validator reports the missing record
|
||||
try:
|
||||
meta = yaml.safe_load("\n".join(lines[1:end])) or {}
|
||||
except yaml.YAMLError:
|
||||
return []
|
||||
if not isinstance(meta, dict):
|
||||
meta = _read_frontmatter("\n".join(lines[1:end]))
|
||||
if not meta:
|
||||
return []
|
||||
|
||||
rec = Record(heading=str(meta.get("title") or "").strip(), source=source, lineno=1)
|
||||
for fm_key, field in _FM_TO_FIELD.items():
|
||||
if fm_key not in meta:
|
||||
continue
|
||||
val = meta[fm_key]
|
||||
# `None` is what YAML gives for `key:` with no value — the empty-vs-absent distinction
|
||||
# that `stale-after` depends on (#603). Preserve it as "" rather than collapsing to None.
|
||||
setattr(rec, field, "" if val is None else str(val).strip())
|
||||
# "" is preserved rather than collapsed to None — the empty-vs-absent distinction
|
||||
# `stale-after` depends on (#603).
|
||||
setattr(rec, field, meta[fm_key].strip())
|
||||
if not rec.status:
|
||||
rec.status = "legacy-unmigrated"
|
||||
rec.body = "\n".join(lines[end + 1 :]).strip()
|
||||
|
||||
Reference in New Issue
Block a user