decisions_validate.py stays green on frontmatter PyYAML rejects — an apostrophe in a single-quoted scalar (hit twice in one session)
#674
Closed
opened 2026-07-27 01:12:45 +02:00 by timothy
·
2 comments
No Branch/Tag Specified
main
renovate/dotnet-monorepo
renovate/meziantou.analyzer-3.x
renovate/lucene.net
renovate/sqlitepclraw.bundle_e_sqlite3-3.x
renovate/cliwrap-3.x
913-scripted-playout-401
release/v26.15.0-notes
issue-806-guard-populations
scratch/767b-poisoned
scratch/767b-control
release/v26.14.0-notes
release/v26.14.0
docs/510-skill-logo-bug-policy
fix/510-watermark-resolution-policy
fix/629-verdict-classifier-falseopens
fix/609-decisions-edit-token-scope
issue-135-clear-to-none
release/v26.12.0-notes
fix/409b-lastscan-api-parity
fix/401-updatechannel-mirror-422
fix/327-playlist-rename-validation
fix/410-scancancel-log-level
fix/409-447-librariesscreen-neverscanned
fix/338-zap-exit-code
fix/367-plex-budget-message
fix/310-debom-legacy-cs
ci/604-lane-rebalance
feat/388-design-mirror
feat/247-test-ownership
feat/247-primary-action
feat/357-player-owned-playback
feat/357-jellyfin-plugin-poc
fix/289-mcp-hardening
issue58-mcp
feat/244-channels-extract
ci/auto-bump-prod-compose
feat/multi-rerun-collections-api
feat/collections-api
feat/quick-wins
feat/185-docs-part2
feat/140-collections-screen
feat/146-channel-edit
feat/147-classic-ui-link
issue22-renovate-dashboard
feat/91-cutover
feat/63-composite-create
feat/65-library-browse
feat/85-epg
feat/86-schedule-editor
feat/109-dashboard-data
feat/99-session-tracking
fix/dockerfile-node-tag
feat/59-spa-foundation
docs/59-ui-redesign-brief
feat/102-json-guide
feat/111-schedule-durations
feat/104-artwork-upload
feat/103-media-sources-api
feat/playouts-read-api
feat/108-health-api
feat/105-picker-list-endpoints
issue-97-channel-state-api
issue42-jellyfin-musicvideos
issue46-rest-api-error-contract
dependabot/nuget/ErsatzTV.FFmpeg.Tests/multi-d307a2e06f
qsv-improvements
hdr-vulkan-cuda-test
v26.16.0
v26.15.0
v26.14.0
v26.13.0
v26.12.0
v26.11.0
v26.10.0
v26.9.0
v26.8.0
v26.7.0
blazor-final
v26.6.0
v26.5.0
v26.4.0
v26.3.1
v26.3.0
v26.2.0
v26.1.1
v26.1.0
v25.9.0
v25.8.0
v25.7.1
v25.7.0
v25.6.0
v25.5.0
v25.4.0
v25.3.1
v25.3.0
v25.2.0
v25.1.0
v0.8.8-beta
v0.8.7-beta
v0.8.6-beta
v0.8.5-beta
v0.8.4-beta
v0.8.3-beta
v0.8.2-beta
v0.8.1-beta
v0.8.0-beta
v0.7.9-beta
v0.7.8-beta
v0.7.7-beta
v0.7.6-beta
v0.7.5-beta
v0.7.4-beta
v0.7.3-beta
v0.7.2-beta
v0.7.1-beta
v0.7.0-beta
v0.6.9-beta
v0.6.8-beta
v0.6.7-beta
v0.6.6-beta
v0.6.5-beta
v0.6.4-beta
v0.6.3-beta
v0.6.2-beta
v0.6.1-beta
v0.6.0-beta
v0.5.8-beta
v0.5.7-beta
v0.5.6-beta
v0.5.5-beta
v0.5.4-beta
v0.5.3-beta
v0.5.2-beta
v0.5.1-beta
v0.5.0-beta
v0.4.5-alpha
v0.4.4-alpha
v0.4.3-alpha
v0.4.2-alpha
v0.4.1-alpha
v0.4.0-alpha
v0.3.8-alpha
v0.3.7-alpha
develop
v0.3.6-alpha
v0.3.5-alpha
v0.3.4-alpha
v0.3.3-alpha
v0.3.2-alpha
v0.3.1-alpha
v0.3.0-alpha
v0.2.5-alpha
v0.2.4-alpha
v0.2.3-alpha
v0.2.2-alpha
v0.2.1-alpha
v0.2.0-alpha
v0.1.5-alpha
v0.1.4-alpha
v0.1.3-alpha
v0.1.2-alpha
v0.1.1-alpha
v0.1.0-alpha
v0.0.62-alpha
v0.0.61-alpha
v0.0.60-alpha
v0.0.59-alpha
v0.0.58-alpha
v0.0.57-alpha
v0.0.56-alpha
v0.0.55-alpha
v0.0.54-alpha
v0.0.53-alpha
v0.0.52-alpha
v0.0.51-alpha
v0.0.50-alpha
v0.0.49-prealpha
v0.0.48-prealpha
v0.0.47-prealpha
v0.0.46-prealpha
v0.0.45-prealpha
v0.0.44-prealpha
v0.0.43-prealpha
v0.0.42-prealpha
v0.0.41-prealpha
v0.0.40-prealpha
v0.0.39-prealpha
v0.0.38-prealpha
v0.0.37-prealpha
v0.0.36-prealpha
v0.0.35-prealpha
v0.0.34-prealpha
v0.0.33-prealpha
v0.0.32-prealpha
v0.0.31-prealpha
v0.0.30-prealpha
v0.0.29-prealpha
v0.0.28-prealpha
v0.0.27-prealpha
v0.0.26-prealpha
v0.0.25-prealpha
v0.0.24-prealpha
v0.0.23-prealpha
v0.0.22-prealpha
v0.0.21-prealpha
v0.0.20-prealpha
v0.0.19-prealpha
v0.0.18-prealpha
v0.0.17-prealpha
v0.0.16-prealpha
v0.0.15-prealpha
v0.0.14-prealpha
v0.0.13-prealpha
v0.0.12-prealpha
v0.0.11-prealpha
v0.0.10-prealpha
v0.0.9-prealpha
v0.0.8-prealpha
v0.0.7-prealpha
v0.0.6-prealpha
v0.0.5-prealpha
v0.0.4-prealpha
v0.0.3-prealpha
v0.0.2-prealpha
v0.0.1-prealpha
Labels
Clear labels
ad-hoc
api
bug
ci-cd
content
dependencies
enhancement
frontend
in-progress
jellyfin
parked
priority: high
priority: low
priority: medium
review
security
One-off / ad-hoc work not tracked by a dedicated issue
REST API / HTTP endpoints
Something isn't working
Build, test, deploy pipeline
Channel content / schedules / playlists
Dependency updates (Renovate)
New feature or improvement
ChicoryTV React SPA frontend
Claimed by an active session — do not pick up
Jellyfin tuner / IPTV integration
Excluded from automatic queue pickup; work only when explicitly selected
Adversarial review finding
Security / vulnerability fix
Milestone
No items
No Milestone
Projects
Clear projects
No projects
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: timothy/ersatztv#674
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Hit twice in one session, independently, by two different agents working unrelated branches (#578 and #651). That frequency is the reason to file it: it is not an exotic edge case, it is what happens when anyone writes ordinary English prose into a
rule:field.The gap
docs/decisions/records/**frontmatter values are single-quoted YAML scalars. A bare apostrophe inside one —the old type's space,SQLite's LOWER()— terminates the scalar early. PyYAML rejects the file.scripts/decisions_validate.pyreports OK.The validator uses a hand-rolled frontmatter parser, so it cannot see the defect. Only
scripts/tests(which parses with PyYAML) catches it.This is the same class as the already-documented unquoted-
#trap, with a different character. The existingunquoted-hash-in-decisions-frontmatterknowledge says the validator "CAN'T see it — runscripts/tests". That is accurate but it is a workaround for a tool that reports success on input it should reject.Why the hand parser exists — do not just import PyYAML
The decisions read path is deliberately dependency-free:
decisions-guardand the git hooks install nothing, and reading has to work everywhere while writing runs in one place. Swapping the parser for PyYAML would break that contract. Any fix has to preserve it.Plausible shapes:
#) and fail loudly, without becoming a general YAML parser.The second is attractive because it generalizes: it catches the next character class too, rather than enumerating hazards one at a time — which is precisely the mistake this session kept making elsewhere.
Why CI already mostly covers it
script-testsrunsscripts/testson every PR, so a broken record does not reachmain. The exposure is local: the validator is the tool a developer or agent reaches for directly, and it says OK.Done-when
decisions_validate.pyfails on a record whose frontmatter PyYAML rejects — including the apostrophe case and the existing unquoted-#casedecisions-guardor the git hooks executescripts/testspins both known hazards and asserts the validator itself now rejects them (today it would pass a file the suite rejects — the two disagree)unquoted-hash-in-decisions-frontmatterknowledge is updated to reflect the resolved stateContext: surfaced during #578 and #651.
Claiming this for a session working #674 + #688 together as a
ci-cdbundle — both are defects inscripts/decisions_validate.py/ its pytest suite, so they share a worktree and land as one reviewed change.Pre-claim checks run and clear: no open PR references either number, no remote branch names them, no prior claiming comment (#688's 2026-07-28 comment explicitly states "No claim on this issue"), and
origin/mainre-fetched.Closing record
Outcome: Shipped in PR #725 (merged,
efc34a3).decisions_validate.pynow runspyyaml_frontmatter_faults()over every record-wing file whenever PyYAML is importable, comparing its parse against the dependency-free reader and failing on either a PyYAML rejection or a value mismatch. Absent PyYAML the check is SKIPPED with a::notice::, never silently — the read path stays dependency-free.Root cause:
dl._read_frontmatteris a hand parser written for machine-generated single-line scalars. It has no concept of quoting rules, so it accepted input the writer's own library rejects. Nothing compared the two outside the test suite, so the tool and its own tests could disagree about the same file and the tool reported OK.Decisions/conventions changed: added
docs.frontmatter-pyyaml-crosscheck.docs.corpus-size-signalalso updated (see #688).Reusable knowledge: the two known hazards fail DIFFERENTLY and a try/except catches only one — a bare apostrophe in a single-quoted scalar makes PyYAML reject the document, while an unquoted
#parses fine and silently TRUNCATES the value (rule: use --flag #2->use --flag). Comparing parsed results rather than catching exceptions is what generalises past the two known characters. Also: PyYAML raises a bareValueError(notYAMLError) on a well-shaped but impossible date such asstale-after: 2026-06-31, and returns TYPED mapping keys, sosorted(set | set)needskey=str.Honest scope: in CI this changes little.
decisions-guardinstalls no PyYAML so the cross-check always skips there, andscript-testsalready went red on both hazards. Branch protection requires exactlyBuild & test (.NET),EF migration integrityandreview-verdict/h10— neitherscript-testsnordecisions-guard. The real fix is the LOCAL loop, where the validator is the tool an agent reaches for and it printed OK on a corrupt file, plus the tool/suite disagreement now being impossible (one implementation, two callers).Verification: 432
scripts/testspass; CI 10/10 green onefc34a3; ruff at baseline parity (47); cross-check clean on all 195 wing files (190 with frontmatter) — no false positives. A positive control pins thatrecord_wing_faultsALONE still reports both hazard files as clean, so the new red cannot pass for the wrong reason. Mutation-verified: stubbing the cross-check, feeding it[], droppingkey=str, narrowing the except, or unwiring it frommain()each fail a test.Deferred: none.
Docs updated:
docs/decisions/records/docs/frontmatter-pyyaml-crosscheck.md(new),docs/ci-cd.md,docs/decisions/README.md(regenerated).