100 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
timothy	83cd36e0de	test(491): run the dedupe fixture against MySql in CI; correct the collation claim ... The dedupe DML had zero automated coverage on MySql: the migrations job only applies migrations to a fresh EMPTY database, so no dedupe row ever executed there. Two MySql-only collation defects escaped that gate in this session and were caught only by hand-run containers. Parameterize LibraryFolderDedupeMigrationTests over both providers from ONE fixture body - same seeded rows, same expected survivors - rather than adding a MySql-only copy that would drift and recreate the gap. Assertions no longer use WHERE Path = '...', which is itself collation-dependent and would quietly mean something different per provider; rows are read once and compared ordinally in memory. A new step in the existing migrations job runs it against that job's mysql:8.4 service, on a per-test database of its own. Proven red when the collation is wrong: restoring COLLATE utf8mb4_bin fails the MySql half with survivors [1,4,5,6,7,9] - the trailing-space sibling deleted - while SQLite stays green. Proven non-skippable: without ETV_TEST_MYSQL_CONNECTION the fixture ignores visibly, and with ETV_REQUIRE_MYSQL_TESTS=1 (which CI sets) that skip becomes a hard failure, so it cannot pass having connected to nothing. Local runs need no MySql. Also correct an overstated comment. The schema pins only the utf8mb4 charset, never a collation, so the effective comparison is the server default: always case-insensitive, but PAD SPACE only on utf8mb4_general_ci - 8.4's default utf8mb4_0900_ai_ci is NO PAD, verified on the real column. The migration bug was independent of that because the old code applied an EXPLICIT utf8mb4_bin, which is PAD SPACE everywhere; the runtime simply tolerates both. Refs #488 #308 fix #491	2026-07-25 21:13:31 +02:00
timothy	48d41b9235	fix(491): make the MySql dedupe byte-exact, not just case-exact (PAD SPACE) ... Cross-family review of 1b4dd6d6 found that utf8mb4_bin - chosen to keep the dedupe case-exact - is a PAD SPACE collation, so trailing spaces are insignificant under it. Verified on MySQL 8.4: '/media/Foo' = '/media/Foo ' is TRUE, while case correctly compares unequal. Two distinct legal directories therefore grouped together and the second was DELETED irreversibly, even though PathUtils.GetPathHash hashes them differently and the unique index about to be created would have accepted both. The dedupe destroyed data the constraint never required it to destroy. Group and join on CONVERT(Path USING binary) instead - NO PAD and byte-exact, matching the hash. utf8mb4_0900_bin is also NO PAD but carries a server-version floor. This is the only path comparison in either migration (every other predicate keys off an integer id), so there is no mix of padded and unpadded comparisons across the keeper-selection, repoint and delete steps. SQLite's = on TEXT is byte-exact with no padding, so that migration was already correct - which is exactly why a SQLite-only test could not see the divergence. The two providers are now semantically equivalent, and the dedupe fixture is shared: same rows, same expected survivors (1,4,5,6,7,9,10), asserted by the SQLite test and reproduced by hand on MySQL 8.4. Runtime was never affected, and this is now stated and tested rather than assumed: GetFolder's SQL equality is a superset narrowing (both collation quirks make it more permissive, never less, so it cannot miss a byte-exact match) and ResolveExact settles identity with StringComparison.Ordinal, which compares length first. Added ResolveExact coverage for the trailing-space axis. Refs #488 #308 fix #491	2026-07-25 21:13:31 +02:00
timothy	50eff83628	docs(491): reattach the case-exact lookup rationale to GetFolder ... Review follow-up. ResolveExact was inserted between GetFolder's summary and GetFolder itself, leaving ResolveExact with two summaries and GetFolder with none -- so the H1 rationale (SQL equality is only a narrowing filter; OrderBy(Id) for determinism) described neither of the things ResolveExact does. On a fix that turns on exactly that reasoning, a misattached explanation is what misleads the next reader.	2026-07-25 21:13:31 +02:00
timothy	ecb763ea58	fix(491): review polish — heal cannot abort a scan, ordinal settle unit-tested ... Final low-severity items from the re-review of ee10f932. L2: DbUpdateConcurrencyException derives from DbUpdateException but carries no provider exception, so IsUniqueConstraintViolation does not classify it. A row deleted by a concurrent library edit between the heal's read and its save would propagate and fail the scan, contradicting the invariant stated directly above it. Admit it in the filter. L1: lift the in-memory ordinal settle into LibraryRepository.ResolveExact and unit-test it with both spellings in the candidate list. No SQLite-backed test can exercise it (SQLite's = on TEXT is already binary), so this converts the half that rested on hand-run MySQL evidence into automated coverage. The end-to-end companion test's comment no longer claims to be provider-independent. L4: assert PRAGMA foreign_keys is 1 before migrating, so the enforcement guard cannot silently degrade into the weak pre-fix form it was added to replace. L3: detach the failed heal, matching the insert path. N3: the heal's inner predicate now matches its IsNullOrEmpty outer guard, so a PathHash = '' row cannot enter the branch and silently never heal. N4: record that GetFolder returning null for a case-differing spelling makes MySQL insert a second row where it used to reuse one — correct, and now matching SQLite, but a real behaviour change on a case-insensitive filesystem. Refs #488 #308 fix #491	2026-07-25 21:13:30 +02:00
timothy	14e9b03433	fix(491): wire the unique-violation classifier in the scanner; make folder lookup case-exact ... Review of 491f5099 found the fix inert in the only process that runs it, plus a MySQL collation defect in the lookup. B1 — TvContext.IsUniqueConstraintViolation was assigned only in ErsatzTV/ Startup.cs, but ErsatzTV.Scanner is a separate executable and every production caller of GetOrAddFolder/SetEtag lives there. The classifier kept its '_ => false' default, so the catch never ran and the DbUpdateException failed the whole scan - worse than the duplicate row it replaced. Wire both provider branches in ErsatzTV.Scanner/Program.cs, and add ProviderStaticsWiringTests (architecture) asserting the scanner assigns every TvContext static the host assigns, with IsSqlite documented as the one exemption. H1 — GetFolder's 'Path == folder' is case-insensitive on MySQL while PathHash is case-sensitive, and FirstOrDefault was unordered: a scan of '/x/foo' could resolve the '/x/Foo' row and stamp the wrong hash onto it (verified on MySQL 8.4: the WHERE matches both, LIMIT 1 returns the wrong one). Treat the SQL equality as a narrowing filter, order by Id, and settle identity ordinally. Route the heal through EF and drop a classified violation, so an opportunistic maintenance write can never abort a scan. Also: run the dedupe migration test with foreign keys ON (matching prod), clear the keeper's etag, null out a self-parent, and document the cleanup's limits (NULL paths excluded, Down does not restore deleted rows, CI's fresh-DB apply covers none of the data mutation). Refs #488 #308 fix #491	2026-07-25 21:13:30 +02:00
timothy	9a4f3e832d	fix(491): unique index on LibraryFolder(LibraryPathId, PathHash) + tolerate concurrent insert ... GetOrAddFolder was a check-then-insert with no unique constraint behind it, so two callers racing the same folder could both miss the lookup and both insert. Enforce identity in the schema and make the loser adopt the winner. - LibraryFolder gains a SHA-256 PathHash (the MediaFile.Path/PathHash precedent): Path is MySQL longtext, which cannot be indexed without a prefix length and collates case-insensitively, so the unique index is on (LibraryPathId, PathHash) instead. - GetOrAddFolder and SetEtag catch a classified unique violation via the existing TvContext.IsUniqueConstraintViolation seam (#308) and re-read. - Dual-provider migration audits and collapses pre-existing duplicates (repointing MediaFile, ParentId and ImageFolderDuration) before creating the index; legacy rows keep a null hash and heal on the next scan. - Tests: deterministic cross-connection race, 8x10 barrier stress with an insert-attempt vacuity guard, classifier-inversion negative control, and a real-migration dedupe test. Refs #488 #308 fix #491	2026-07-25 21:13:22 +02:00
timothy	02c82b35ea	fix(610): make the frontmatter READ path dependency-free — CI has no PyYAML ... This is why `decisions lifecycle` went red, and it was NOT the known flake. I came close to dismissing it as one for the second time this session, because an earlier red on another branch genuinely was. The dual-format parser imported PyYAML to read frontmatter. `decisions-guard` does `setup-python` and installs NOTHING, so once the corpus was migrated every record became unparseable there: ModuleNotFoundError, job fails. The same would hit the Husky pre-commit hook and every contributor's machine. Installing PyYAML in CI is the wrong fix: READING happens everywhere -- CI, the hook, every dev -- while WRITING happens once, in a migration a human runs deliberately. So the read path is now dependency-free and only `migrate_decisions_split` (the writer) still imports yaml. A hand-rolled parser is only safe if it provably matches the library that WROTE the files, so `test_frontmatter_reader_matches_pyyaml_on_every_real_record` compares the two field-by-field across all 169 real records (importorskip, so it is skipped rather than failing where PyYAML is absent) with a >100-file guard against near-vacuity. It is narrow by construction: the frontmatter is machine-generated with default_flow_style=False and width=10**9, so every value is a single-line scalar, and the reader bails to None on anything nested. Verified by running all four affected entry points against a shim that makes `import yaml` raise: validate --base/--head, catalog --check, the kickoff guard, and the plain validate the pre-commit hook calls. All exit 0. refs #610	2026-07-25 19:57:22 +02:00
timothy	52786a545c	feat(610): re-key the body-diff guard to key; enforce path<->key; pin the filesystem invariant ... Three Done-when items that were genuinely unfinished. I audited the checklist before ticking it rather than after, and these were not done. 1. BODY-DIFF GUARD RE-KEYED FROM HEADING TO `key`. Heading-keying made a pure RENAME look like a removal plus an unrelated addition, so retitling a record failed CI as an "unlogged removal" -- a trap that has bitten this repo before. Records are now MATCHED by key and REPORTED by heading (a key alone is not enough for a human to find the record). Demotion, which is inherently about a record losing its key, is detected on the heading axis instead, and a demoted record is no longer double-reported as removed. Key-matching is also what lets the comparison work straight through the legacy->frontmatter migration, since `key` survives both the format change and the move between files. Pinned by a test driving the REAL git-backed diff engine: retitle a record, same key, byte-identical prose -> removed/rewritten/demoted all empty. Mutation-verified: restoring heading-keying makes it fail with exactly "a retitle was reported as a removal". 2. PATH <-> KEY ENFORCED BY THE VALIDATOR. The filename is derived from the key, which is what makes one-active-per-key a filesystem property rather than a check -- but only if the two cannot drift. Now an error when they disagree, with legacy multi-record files explicitly exempt (they have no key-derived path to match). 3. ONE-ACTIVE-PER-KEY PINNED AS A FILESYSTEM PROPERTY. Two records sharing a key derive the same path, so the filesystem refuses the second; the migration aborts on a destination collision rather than silently overwriting. Both are now tested, the latter end-to-end through `main()`. Also fixed a vacuous assertion ruff caught in my own new test -- `assert X or True` is always true. Note on verification: my first positive control for the guard reported it NOT firing. That was the probe, not the code -- "load 340" appears in both the `sources:` frontmatter field and the body, and replace(...,1) hit the metadata copy, which is correctly token-free. Re-run against genuine body prose, the guard fires with exit 1. refs #610	2026-07-25 19:53:08 +02:00
timothy	fba5233caf	feat(610): split the decision corpus into one YAML-frontmatter file per record ... 168 records -> docs/decisions/records/<area>/<topic>.md (163 active, 23 dirs) and docs/decisions/archive/<area>/<topic>.md (5 archived). The filename IS the key, so one-active-record-per-key becomes a filesystem property rather than a validator check, and supersession becomes a `git mv`. WHY: the monolith was a concurrency problem before an aesthetic one. A 3,900-line append target made parallel sessions collide -- PR #605 and PR #614 both hit append-vs-append conflicts during routine rebases, and hand-resolving those inside the corpus is exactly the operation the rationale-rewrite guard exists to police. HOW IT IS VERIFIED: a ~170-file diff cannot be meaningfully read, so correctness does not rest on reading it. The parser was taught BOTH formats first, so the body-diff guard parses the old form at the merge-base and the new form at head -- the migration validates itself, no bypass. The proof is a field-level equivalence harness: 168 records before and after, zero lost, zero gained, zero field mismatches, zero rationale bodies differing. Reviewers should scrutinise the harness; it is the actual evidence. What measuring caught that reading would not have: - ~500 lines sit OUTSIDE any record -- decisions.md's lifecycle schema and each topic file's preamble, mostly the only copy. Source files are kept and stripped, never deleted. They also cannot be filed per-area: topic files hold several areas and 4 of 23 areas span several files. - Archive discovery was a non-recursive glob; after the split it found ZERO archived records, surfacing as four bogus "supersedes points to unknown key" errors rather than an obvious failure. - ~32 live docs point into the corpus BY DATE, which the split dangles. Each stripped file now ends with a generated "Records formerly in this file" index, which also rescues the identical breadcrumbs in old issue comments. - decisions.md's "In this file:" list was 97 same-file anchor bullets that the split makes WRONG, not merely stale. Dropped; the generated index replaces them with links that resolve. The equivalence harness now runs against a checked-in FIXTURE, not the live corpus. The earlier version migrated the real tree, which made it a one-shot: the moment the migration landed there was nothing left to move and the tests failed for reasons unrelated to the code. A fixture keeps them testing the SCRIPT rather than the repo's current state. Keys preserved verbatim, warts included: `sched` (12) and `scheduling` (1) remain two directories for one concept. Renaming a key is not a move -- it changes identity, breaks the equivalence proof, and invalidates MemPalace's per-key drawers. Taxonomy normalisation is separate work. refs #610	2026-07-25 19:45:09 +02:00
timothy	8578dc1ca7	fix(610): de-brittle the count assertion; stop conflating index lines with preserved prose ... Both surfaced when main gained two records mid-flight. - test_record_count_is_the_expected_166 hardcoded the total, so it failed the moment a record landed on main -- a merge turning an unrelated test red. The real invariant is before == after; the count only needs to prove the harness isn't parsing a stub corpus, so it is now equality plus a floor. - The migration's "lines preserved" figure silently absorbed the generated where-did-it-go index once that was threaded into the preamble string, jumping 507 -> 759 with no new prose preserved. It now reports the two separately: 514 lines of original prose, plus 245 generated index lines. A number that quietly changes meaning is worse than no number.	2026-07-25 19:08:58 +02:00
timothy	64b65fd2db	feat(610): generated where-did-it-go index on each stripped file ... Live docs point into the corpus BY DATE -- "see `decisions.md` 2026-07-10" -- about 32 such references across 12 files, plus the same form in historical issue comments. The split would dangle every one of them. Each stripped file now ends with a generated "Records formerly in this file" index: date, title, and a link to the record's new path. A reader following a date pointer lands on the file it names and resolves from there. That is far cheaper and less error-prone than rewriting 32 references by hand, and it also covers the issue-comment breadcrumbs, which cannot be rewritten at all. Caught while verifying it: the generated `## Records formerly in this file` heading is itself an H2, so the record parser counted one legacy-unmigrated record per stripped file -- the notice went 0 -> 6. Same treatment as the existing `## Index` section: skip it by name. SKIP_HEADINGS moved to decisions_lib as the single source of truth, since three modules now need it. Found by reading the validator's notice output on a trial migration, not by inspection -- the corpus still validated OK, so nothing else would have flagged it.	2026-07-25 19:08:58 +02:00
timothy	8ec1f527e4	fix(610): catalog path links only for the split layout, not every file under docs/decisions ... The first cut tested `relative_to(TOPIC_DIR)`, which also matches the LEGACY multi-record topic files -- so every record in workflow-process.md et al. lost its anchor and linked to the top of the file instead of to its own record. Caught by `build_decisions_catalog --check` going stale on the unmigrated corpus, not by reading. Narrowed to `RECORDS_DIR in src.parents`, which by construction only matches one-record-per-file.	2026-07-25 19:08:58 +02:00
timothy	1af65b7bee	feat(610): budget counts PROSE, excluding YAML frontmatter ... The line budget exists to bound how much narrative a reader or agent must get through. Under the split each record carries ~11 frontmatter lines plus two fences -- 1789 lines across 166 records -- which are the structured restatement of what used to be one dense backtick line. Counting them inflates the metric without any new knowledge being added. Stated plainly because it flatters the number: this is a change of METRIC, not a consolidation. It re-measures the same corpus, it does not shrink it. Whole-file counting put the migrated corpus at 6837 against a 5600 budget; prose-only puts the same content at ~5048. The consolidation work is still worth doing -- it is simply no longer being signalled by a warning that was partly measuring punctuation. Inert pre-migration: no legacy file has frontmatter, so the branch is never taken and today's number is unchanged.	2026-07-25 19:08:58 +02:00
timothy	ef67be924c	feat(610): teach the validator and catalog the split layout ... Both surfaced in a trial migration against a disposable copy of the real corpus, not from reading: - Archive discovery was a NON-recursive glob. After the split, archived records live at archive/<area>/<topic>.md, so every one of them became invisible -- which surfaced as four bogus "supersedes points to unknown key" errors rather than as an obvious "no archive found". rglob at both sites. - Catalog links: a record is now a FILE, so the link is a plain relative path with no anchor -- nothing to slug, nothing to keep in sync with a heading. The legacy anchor form is kept for records still living inside a multi-record file, so the catalog is correct on either side of the migration.	2026-07-25 19:08:58 +02:00
timothy	d52268f2ed	feat(610): migration script + field-level equivalence harness ... The migration is a MOVE, so correctness is provable rather than reviewable: parse the corpus before, migrate, parse after, assert the Record sets are identical field-for-field with byte-identical rationale. scripts/migrate_decisions_split.py 166 records -> docs/decisions/records/<area>/<topic>.md (active) or docs/decisions/archive/<area>/<topic>.md (superseded/retired), 26 directories. Path is DERIVED from the key, so identity stays the key. Refuses to run if any record lacks a key, and aborts on a destination collision. Source files are KEPT, stripped to their narrative -- not deleted. 506 lines of the corpus sit outside any record: decisions.md's lifecycle-schema header (the status vocabulary, supersession rules, the edit-token contract) and each topic file's preamble explaining why those records exist. For most of it that is the only copy. It also cannot be filed per-area -- topic files hold several areas and 4 of 23 areas span several files -- so the files themselves stay. scripts/tests/test_migration_equivalence.py Runs the real migration against a COPY of the real corpus in tmp_path, never the working tree. Asserts: all 166 records survive with the same keys, every metadata field round-trips, titles round-trip from headings, every rationale body is byte-identical, path matches key, archived records land in the archive wing, the legacy files keep their narrative, and no parseable record is left behind in them. Proven non-vacuous: corrupting one migrated record's prose is caught by the byte-identical check, and deleting one is caught by the survival check. One test-authoring note: an early assertion string-matched "## " to prove no records were left in decisions.md. That is wrong -- the schema header quotes an illustrative "## 2026-07-17 ..." example in prose. Whether records remain is a PARSING question, so the parser-based leftover test is the real invariant.	2026-07-25 19:08:58 +02:00
timothy	e063fd065f	feat(610): dual-format decision-record parser (backtick line + YAML frontmatter) ... First step of the one-file-per-record split. `decisions_lib.parse_text` now dispatches on a leading `---`: the legacy "many ## records per file, metadata on a backtick line" form and the new "one record per file, YAML frontmatter" form both produce the same Record. This is the load-bearing move for the migration. Because the parser understands both, the body-diff guard can parse the OLD form at the merge-base and the NEW form at head -- both keyed on `key` -- so the commit that relocates 166 records validates itself with no one-time bypass and no escape hatch. Details: - Frontmatter uses the SAME field vocabulary as the backtick line (`superseded-by`, `stale-after`), so on-disk names don't shift mid-migration. - YAML `None` (a key with no value) is preserved as "" rather than collapsing to None, keeping the empty-vs-absent distinction `stale-after` depends on (#603). - `active_files()` also walks `docs/decisions/records/**`, so both layouts can coexist while the migration lands. - Malformed frontmatter (unterminated fence, non-dict, YAML error) yields no record rather than raising; the validator then reports it as a missing record.	2026-07-25 19:08:58 +02:00
timothyandClaude Opus 5	f49556b6ef	docs(609): qualify the two ci-cd.md mentions as non-merge ... Last review Low, marked safe-to-defer but it is two words in a file this PR already edits. Both prose mentions of the marker now say "non-merge commit", matching the decision Rule, the catalog, the module comment and the function docstring. Docs-only. fixes #609 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-25 18:40:55 +02:00
timothyandClaude Opus 5	e9f4444fec	docs(609): qualify two comments as non-merge, matching the implementation ... Review Low: the module comment and the _edit_trailer_armed summary still said "a commit message ... does arm" / "some commit", while the matcher excludes merges. The decision Rule and catalog already carried the qualification; these two did not. Comment-only. fixes #609 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-25 18:38:58 +02:00
timothyandClaude Opus 5	f00dfde0c5	docs(609): correct the --no-merges rationale — merges are discouraged, not blocked ... Review Low x2, both correct and both the stale-comment class: The claim that prepush-rebase-check.sh forbids merging main into a PR branch is false. That hook refuses a branch that is BEHIND origin/main; a merge makes origin/main an ancestor, so the push is allowed. Merging main in is discouraged by convention only. So --no-merges does cost a real false negative: an author who marks ONLY a conflict-resolving merge commit gets a legitimate rewrite rejected. Keeping --no-merges and stating the trade explicitly -- that failure is loud and costs one extra commit, whereas honoring forge-composed merge bodies disables the guard silently, which is #609 itself. The module comment also still claimed a quoted example cannot arm the guard, which contradicts the residual the decision record now states accurately. Aligned both, and narrowed the record's Rule line from "some commit" to "some NON-MERGE commit" so the stated contract matches the implementation. No logic change -- comments, docstring, record prose and regenerated catalog only. fixes #609 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-25 18:36:07 +02:00
timothyandClaude Opus 5	cc9481f541	fix(609): review fixes — exclude merge commits, unfold folded values, correct the fail-posture doc ... Cross-family review (High + Medium + Low), all three reproduced before fixing: High -- on a pull_request event actions/checkout lands on a synthetic merge commit whose body the forge composes from the PR description, so a description ending in an example marker armed a guard no author armed. Excluded merge commits from the range; merging main into a PR branch is separately forbidden, so no author-written commit is skipped. Medium -- a folded value (`no` + continuation ` yes`) was split into independent lines and the continuation armed on its own, inverting the value the author wrote. Read with `unfold` so the value is judged whole. Low -- the module docstring promised fail-open while marker resolution deliberately fails closed. The posture is right; the docstring was wrong. Documented as the one exception. Both new negative controls verified red against the unfixed matcher. Decision record amended to state the residual honestly rather than overclaim: a quoted example that is the FINAL paragraph of an ordinary commit is a trailer by git's own grammar and does arm. What the change buys is that discussing the marker can no longer disable anything. fixes #609 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-25 18:29:46 +02:00
timothyandClaude Opus 5	c597c49f02	fix(609): arm the decisions body-diff with a git trailer, not a bare substring ... The rationale-edit exemption was a substring test over the whole commit range, so any message containing the literal marker armed it -- including prose explaining why no marker was needed, which is how it fired live in PR #605: a green --base/--head run that was vacuous on the body-diff dimension, in the one PR that hand-resolved a merge conflict inside the corpus the guard exists to police. Now read as an affirmative `Decisions-Edit:` git trailer. Git parses trailers only in the final paragraph, so a quoted example message cannot arm it -- which matters because this commit and its decision record both quote one. A non-affirmative value (`no`) does not arm it either; the retired substring arms nothing and gets a ::warning:: nudge. Tests: negative controls (prose mention, quoted mid-body trailer, `no` value, retired substring) plus positive controls (trailer, uppercase, alongside Co-Authored-By). All four negative controls verified red against the old matcher. fixes #609 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-25 18:21:12 +02:00
timothy	b5b6e7f636	test(496,484): record projection failures during enumeration, not eagerly [decisions-edit] ... Re-review Low. The substitute incremented the failure counter inside .Returns(...), i.e. when the enumerable was handed out, while the real paginator records from ProjectToMusicVideo's catch DURING enumeration. A refactor that snapshotted Count before the enumeration completed would then break production while both replacement tests kept passing — exactly the regression the guard exists to prevent. Moves the recording into an async iterator, and corrects the decision record to describe #484's removed music-video test accurately and name its two replacements.	2026-07-25 17:36:34 +02:00
timothy	4bdad4bf52	fix(496,484): thread #484's projection-failure guard through the new music-video scanner [decisions-edit] ... Rebasing onto #612 exposed a silent gap rather than a conflict. #612 added a `projectionFailureCount` parameter to MediaServerReconciliationGuard.ShouldFlagMissing that refuses the sweep when the enumeration reported swallowed projection exceptions — but the parameter is OPTIONAL with a default of 0, so this scanner compiled unchanged while opting out of the protection entirely. ProjectToMusicVideo has exactly the swallowing catch #484 exists to defend against, so the music-video sweep would have been the only one unguarded. - MediaServerMusicVideoLibraryScanner creates one MediaServerProjectionFailureCounter per enumeration (never a field on the singleton api client, so concurrent scans of different libraries can't leak failures into each other's sweep decision), passes it to GetMusicVideoLibraryItems, and feeds its Count to ShouldFlagMissing. - The single guard call also gates the #496 legacy path diff, which is more exposed: a legacy row has no etag to fall back on. - ProjectToMusicVideo now returns MediaServerProjectionResult<JellyfinMusicVideo>, combining #612's Skipped/Failed distinction with #496's identity type. - main's own #484 music-video test targeted the pre-#496 hard-delete scanner (FindMusicVideoPaths /DeleteByPath) and no longer applies; it is replaced by two tests in the new architecture covering the identity sweep and the legacy sweep. Both proven non-vacuous — removing projectionFailures.Count from the guard call fails exactly those two. 4,277 tests green; format/BOM clean; decisions validator OK.	2026-07-25 17:27:50 +02:00
timothy	147166b053	fix(496): harden the legacy path sweep against MySQL collation (re-review Low) ... Re-review returned MERGEABLE with one Low, MySQL-collation-dependent edge in FlagFileNotFoundByPaths: the C#-side Except diff is ordinal, but `MF.Path IN @LocalPaths` runs under MySQL's case-insensitive default collation, so a still-reported identified row differing only in case from an absent legacy row could be matched and flagged missing. Match on the indexed PathHash instead of collated Path text, and re-state the NOT EXISTS (JellyfinMusicVideo) guard so the identity pass and the legacy pass are disjoint by construction rather than by the caller's diff being correct. SQLite was unaffected.	2026-07-25 17:20:53 +02:00
timothy	6bd1d954bd	fix(496): review fixes — thread the replaced local path, scope identity per library, sweep legacy rows [decisions-edit] ... Independent cold review (Codex) returned BLOCKED. Findings 1, 3 and 4 are fixed here; each has a regression test proven non-vacuous by a negative control. 1. Blocker — the replaced local path was discarded. The scanner computed localPath but GetOrAdd only received `incoming`, so the repository re-derived the path from the UNREPLACED projection. On any install with path replacements, adoption hashed the server-side path, missed the existing row, ALSO slipped past MediaFileAlreadyExists (which hashes that same wrong string) and inserted a duplicate row under a server path, leaving the original collection-linked row identity-less forever. The test harness hid this because its path-replacement stub was an identity function. → GetOrAdd now takes localPath explicitly and never reads the projection's path; BuildPathReplacement takes a real mapping and the new test genuinely replaces. 3. Medium — GetByItemId matched on ItemId alone, so two media sources presenting the same item id (cloned Jellyfin DB) resolved to each other's row, letting one library repoint another's. → filtered by LibraryPath.LibraryId. 4. Medium — a row predating the identity that the server had ALREADY stopped reporting was never adopted (adoption only runs for an incoming item) and carried no identity, so the itemId diff could not see it either: it sat Normal and schedulable forever, strictly worse than the hard delete it replaced. → GetExistingLegacyMusicVideoPaths + FlagFileNotFoundByPaths reconcile legacy rows by local path, and they are counted into the #477 empty-fetch guard (on the first scan after this ships they ARE the whole library, so a guard counting only identity rows would sweep all of them on a transient empty fetch). Finding 2 (the issue's Done-when #2) is a scope question, not a defect, and is unchanged: one file path is still one MediaItem row globally, so this lands music videos at parity with movies rather than eliminating shared-row trashing. Recorded honestly in the decision record; raised for an explicit call before the issue is closed. fixes #496	2026-07-25 17:20:53 +02:00
timothy	64decd492e	fix(496): give music videos a per-library server identity; itemId diff + soft trash ... Music videos carried no server identity, so JellyfinMusicVideoLibraryScanner had to reconcile by a (LibraryPathId, path) diff and HARD-delete the remainder. A file served by two libraries with overlapping local paths is a single row owned by whichever library scanned it first, so that owner's sweep destroyed a row another library still served — taking collection membership and playout references with it, irreversibly. This is #494's deferred "option 2": - New JellyfinMusicVideo : MusicVideo (ItemId/Etag), mirroring JellyfinMovie — TPT table, varchar(36), ItemId index. Dual-provider migration Add_JellyfinMusicVideo. - New IMediaServerMusicVideoRepository + JellyfinMusicVideoRepository: itemId-keyed existing-set/lookup and Flag{Normal,Unavailable,FileNotFound} seams, all scoped per library via LibraryPath.LibraryId. - New MediaServerMusicVideoLibraryScanner base; JellyfinMusicVideoLibraryScanner folds onto it and keeps the #177/#488/#497/#500 metadata-reconcile logic verbatim. - The sweep now soft-trashes (FileNotFound) instead of deleting, so removal is reversible and EmptyTrash-governed. DeleteEmptyArtists consequently no longer fires from a sweep. - Pre-identity rows are ADOPTED in place: the identity row is inserted against the same MediaItem id, scoped to the scanned library's own LibraryPath, so collection membership survives and a local/second-library row is never hijacked. - AddMusicVideo normalizes Path/PathHash to the path-REPLACED local path; the projection fills them from the server-reported path, which would break every later PathHash lookup. Docs: scan.musicvideo-reconciliation relocated to docs/decisions/archive/scan.md as superseded; new active record scan.musicvideo-server-identity. fixes #496	2026-07-25 17:20:53 +02:00
timothy	7cc12881de	docs(484): correct the counted-enumeration counts on the api client interfaces ... Review follow-up. The comments still said three (Jellyfin) / two (Emby) library-level enumerations after the nested season and episode ones were counted, which understates the reach of the very safety property this branch establishes -- a future auditor reading them would conclude the nested sweeps are unprotected.	2026-07-25 16:36:35 +02:00
timothy	6f4497e1ce	fix(484): guard the nested TV season and episode sweeps against projection failures ... Review finding 1 (blocking). ScanSeasons' FlagFileNotFoundSeasons and ScanEpisodes' FlagFileNotFoundEpisodes had no guard at all — neither #477's nor #484's — so ProjectToSeason / ProjectToEpisode returning Failed() was computed and discarded. #477 scoped those out because "the blast radius is one show's seasons / one season's episodes", which holds for a per-parent EMPTY fetch but not for a projection failure: that is systematic by construction. One bad code path fires on every parent, so every season enumerates zero episodes, existing.Except([]) is the whole episode library, and EmptyTrashHandler deletes it permanently. Threads the counter into GetSeasonLibraryItems / GetEpisodeLibraryItems(WithoutPeople) for Jellyfin and Emby using the same optional-trailing-param shape, and guards both sweeps with MediaServerReconciliationGuard.ShouldFlagMissingDescendants — the same class and the same private failure predicate as ShouldFlagMissing, deliberately WITHOUT #477's empty-fetch branch so per-parent empty behaviour (and #476's cascade, which depends on it) is unchanged. Also from the review: - finding 3: tests now pin the same-instance JOIN at every level (movie, show, season, episode, music video) by driving the real ScanLibrary entry point and recording the failure from inside the enumeration, so a refactor handing the api client a fresh counter goes red. - finding 4: the missing-library Failed() branch is documented as defensive and unreachable. - finding 2: the mass-Skip residual (Emby's response-shape-dependent MediaSources guard, Plex's pre-projection filter) is stated as a known limitation in the decision record. - finding 5: the log-contract change (only the #484 message when both refusals apply) is noted. fixes #484	2026-07-25 16:36:35 +02:00
timothy	e3645a2840	feat(484): suppress the media-server sweep on projection failures; reject the ratio threshold ... Extends MediaServerReconciliationGuard (#477) with a second deterministic refusal: when the enumeration that produced the incoming set silently dropped items whose projection THREW, the file-not-found sweep is refused. A dropped item the server did return is indistinguishable from a deletion at the reconcile step, so a projection regression could otherwise mass-flag a healthy library FileNotFound (which EmptyTrash then deletes permanently). Deliberate guard-clause skips (STRM files, virtual items, unsupported types) are explicitly NOT failures and never suppress a sweep — counting them would permanently disable reconciliation for any library holding a single STRM file. The ratio / missing-fraction threshold is REJECTED, not deferred: it is a two-sided heuristic with no tunable default and no telemetry, and the failure it approximates is exactly observable via the projection-failure count (a genuine bulk deletion produces zero failures). Seam is deliberately narrow — the private ProjectTo* contract inside each api client changed from Option<T> to MediaServerProjectionResult<T> (projected/skipped/failed), the paged helper counts IsFailure in one place, and the scanner reads it through an optional trailing MediaServerProjectionFailureCounter on only the five library-level methods that feed a sweep. The counter is per-enumeration state created by the scanner, never a field on an api client. fixes #484	2026-07-25 16:36:35 +02:00
timothy	f04412a95b	docs(603): correct the record's own no-backfill claim, which the backfill commit falsifies ... Caught in re-review. The record shipped in 0ad02db6 said 'no backfill was done here'; 8e1e15e4 then backfilled three ci.* records on the same branch. Corrected now while the record is still NEW at HEAD, so the body-diff guard does not apply and no edit token is needed -- after merge this would be surviving rationale prose and the same fix would cost one. Deliberately avoiding the literal bracketed token string here: decisions_validate matches it as a bare substring over the whole commit range, so a message merely DESCRIBING the mechanism arms it and suppresses the rationale-rewrite guard for the entire PR. Found exactly that way on the first draft of this commit.	2026-07-25 15:30:43 +02:00
timothy	8e1e15e4d7	docs(603): backfill stale-after + Sources onto the three ci.* outside-world records ... The mechanism shipped unused. First adopters are the records that encode measured host behavior, which is what drifts silently: - ci.runner-placement 2027-01-15 — bumblebee sizing (25 GiB/12 cores) + container caps. Premise has already partly moved: the media transcoders left for jazz on 2026-07-20 and the runners were retuned since. - ci.infra-shaped-red-under-load 2027-02-15 — a triage heuristic calibrated against load 243 on a host that has since been retuned. - ci.peak-anon-measurement 2027-03-15 — weakest of the three (mostly our own script) but it does assert that memory.peak is cache-inflated, a cgroup fact. Dates staggered so they don't all come due in the same week. Each Sources line cites the incident evidence already named in the record's own prose.	2026-07-25 15:22:50 +02:00
timothy	75139bbf31	fix(603): close four defects found in adversarial review of the stale-after fields ... 1. `date.fromisoformat` is not a YYYY-MM-DD validator. On Python >= 3.11 it also accepts ISO basic format ("20270101") and week dates ("2027-W01-1"), so a malformed-looking value passed the blocking check — and which forms parse depends on the interpreter, meaning the same corpus could validate differently on a dev machine and on the runner (pr-checks.yml pins only python-version '3.x'). Knock-on: the catalog's Review-due section sorts on the raw STRING, so an accepted "20270101" sorted AFTER "2027-01-15" ('-' < '0'), contradicting the section's own "sorted soonest-first" text. Gate on ^\d{4}-\d{2}-\d{2}$ first, which fixes both — a fixed-width zero-padded form makes string sort == date sort. 2. A present-but-empty `stale-after:` was collapsed to None by `or None` in the parser and then skipped by a truthiness guard in the validator, so it passed as "absent" — a field that silently never fires, which is the exact failure mode the blocking check exists to prevent. Keep "" distinct from None and test with `is not None`. 3. `test_catalog_is_date_independent` was partly vacuous: with no date in either render, both sides were trivially equal after the .replace(). It did still catch an injected clock-derived marker, but it passed with the feature deleted. Assert the dates are present. 4. The malformed-date check ran only over the active set, exempting archive records. Staleness is moot there, but a typo is still a typo — check both wings. Adds regression tests for each, plus a Review-due row for a topic-file record (pinning the `../decisions.md` vs bare-filename link forms).	2026-07-25 15:22:50 +02:00
timothy	0ad02db651	feat(603): adopt OKF's optional stale-after and Sources decision-record metadata ... Evaluated the Open Knowledge Format (GoogleCloudPlatform/knowledge-catalog okf v0.2, scaccogatto/okf-skills) as a replacement for our decision-record system and rejected it: its conformance rules are deliberately permissive exactly where ours are strict (broken links, unknown types and missing fields must all be tolerated; `deprecated` points at no successor), and its stable identity is the file path, which the breadcrumb rule tells agents not to trust. Adopted two of its optional families instead, additively: - `stale-after: YYYY-MM-DD` on the metadata line — marks a record asserting an outside-world fact as due for re-confirmation. Absolute date, no TTL. - `**Sources:**` in the metadata block — the evidence a record rests on, as distinct from `Signals:` (recall keywords). Neither is required; absence is never an error. A malformed `stale-after` is blocking (it would silently never fire), but a past-due record is only a non-blocking `::notice::` — going stale is the passage of time, not a defect in whatever commit is under test. The catalog's new "Review due" section renders the date only and never a clock-derived verdict, so it cannot drift `--check` red on a calendar boundary with no commit touching the corpus. No backfill: no existing record adopts either field here. fixes #603	2026-07-25 15:22:50 +02:00
timothy	39873811da	test(503): guard WatermarkLocation exhaustiveness and cover source-content margins ... Review follow-up. The 9-element ExpectedPositions table meant a future enum member would pass every case and silently render bottom-right again -- the exact bug #503 fixes. Assert the table covers Enum.GetValues instead, and exercise the previously untested SourceContentMargins() branch.	2026-07-25 15:19:23 +02:00
timothyandClaude Sonnet 5	97bc346539	fix(503): map WatermarkLocation.MiddleCenter to a centered overlay position ... OverlayWatermarkFilter.Position had no switch arm for MiddleCenter, so it silently fell into the BottomRight default and rendered bottom-right. OverlayWatermarkCudaFilter and OverlayWatermarkQsvFilter both inherit this Position property without overriding it, so they were affected too (the whole ErsatzTV.FFmpeg project has only this one WatermarkLocation switch). - Add an explicit MiddleCenter arm: x=(W-w)/2:y=(H-h)/2 - Make BottomRight an explicit arm instead of the fallthrough default - The default case now logs a warning (not throw - this runs on the playback hot path constructing ffmpeg args, and sibling filters in this project already use safe string fallbacks rather than throwing for an unmapped enum) and falls back to the BottomRight position - Strip the pre-existing UTF-8 BOM from OverlayWatermarkFilter.cs (#311 formatting gate: touching a legacy BOM'd file makes removing it ours) - Add ErsatzTV.FFmpeg.Tests/Filter/OverlayWatermarkFilterTests.cs asserting the exact position expression for every WatermarkLocation value across the software, CUDA, and QSV overlay filters, plus the unmapped-value fallback Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-25 15:19:23 +02:00
timothy	9798877631	ci(445): re-point the toolchain pin after the second rebase (1652fc5 -> 32747a0) ... Completes the second pin-recovery cycle. main moved twice during this branch's review and each rebase rewrites the sha of the commit that touched docker/ci, so the pin has to be re-pointed each time. Simulated the guard's full logic before pushing, including the length check main added in #598 (which came from this session's #594): length = 7 (guard requires exactly 7) expected = git log -1 --format=%H -- docker/ci .gitea/workflows/ci-image.yml = 32747a067e pin = 32747a0 -> resolves to the same commit => WOULD PASS Confirmed the registry actually holds ersatztv-ci:32747a0 before pinning it, so this cannot be the "pin resolves but no such tag exists" failure #594 describes. Refs #445 #594	2026-07-25 14:35:02 +02:00
timothy	32747a067e	docs(445): note the ci-image-pin length guard in the rebase-trap warning ... Second pin-recovery cycle on this branch — main moved twice during review and each rebase rewrites the sha of the commit that touched docker/ci. Records that #598 now length-checks the pin at exactly 7 chars, which turns a locally-computed 8-char `git rev-parse --short` into a loud gate failure instead of a confusing manifest-unknown at image-pull time. That guard came from #594, filed by this session after hitting exactly that ambiguity. This commit also IS the recovery: it touches docker/ci, so ci-image.yml tags it and the pin can be re-pointed in the follow-up commit. Refs #445 #594	2026-07-25 14:29:27 +02:00
timothy	352aa70634	refactor(445): drop e2e-ui.sh's port pre-flight, now owned by e2e-local.sh (#586/#598) ... main's #598 added a port pre-flight to scripts/e2e-local.sh while this branch was in review, and theirs is strictly better than the one I had here: - it probes BOTH bound ports. The app binds a SECOND (streaming) listener that defaults to 8409 regardless of ETV_UI_PORT, so my single-port check could certify a port free while the run still died binding 8409. That also means this PR's CI step (ETV_UI_PORT=8410) was only working because the curl step's server had already been killed — #598's fix, defaulting ETV_STREAMING_PORT to the given port, is what makes it correct rather than lucky. A real latent bug in my work, surfaced by their change. - it REPORTS rather than reaps, which is the #586 rule. Keeping mine would leave two divergent port checks on the same concern. Also records the app's SINGLE-INSTANCE guard, which is independent of ports: a stray instance blocks a run whatever port you choose, so 'pick another port' is not a workaround. Kill the leftover BY PID (#586). Refs #445 #586	2026-07-25 14:29:27 +02:00
timothy	e7bac06d1b	ci(445): re-point the toolchain pin after the rebase (e9fd26f -> 1652fc5) ... Second half of the post-rebase pin recovery. The preceding commit touched docker/ci, so ci-image.yml tagged it and published ersatztv-ci:1652fc5; this commit points all five container jobs (plus the header comment) at it. Simulated ci-image-pin's own logic before pushing rather than guessing: expected = git log -1 --format=%H -- docker/ci .gitea/workflows/ci-image.yml = 1652fc568e53184c92c7e0bc5a41546aed19744d pin = 1652fc5 -> resolves to the same commit => WOULD PASS Verified the published image is the one CI will actually consume: pulled :1652fc5 on the runner host, /ms-playwright holds chromium_headless_shell-1234, chromium launches (151.0.7922.34), dotnet 10.0.302 intact. Refs #445	2026-07-25 14:04:52 +02:00
timothy	5dd0b147e8	docs(445): record that rebasing invalidates the CI toolchain image pin ... Hit this for real on this branch. `ci-image-pin` went red after a rebase that was otherwise clean, and the failure is confusing on three counts: - the pin must equal the short sha of the commit that touched docker/ci, and a rebase REWRITES that sha (e9fd26f6 -> 9130274c here); - the pin still resolves to a real commit and the tagged image still exists in the registry, so nothing looks broken; - the force-push does NOT republish: ci-image.yml filters on `paths: docker/ci/**`, and a rebase that leaves the Dockerfile's content unchanged produces no diff for that path. And it cannot be fixed by re-dispatching ci-image.yml, because that tags `git rev-parse --short HEAD` — the branch HEAD, not the commit that touched docker/ci. The two coincide only when the docker/ci commit IS HEAD, which is why the original two-step worked and the post-rebase state does not. Recorded in docs/ci-cd.md with the recovery, plus the cheaper lesson: land a toolchain-image change on its OWN branch first, so the consuming branch never carries the docker/ci commit through a rebase. This commit is also the recovery itself — it touches docker/ci, so it becomes the commit ci-image.yml tags, restoring the pin dance. Refs #445	2026-07-25 14:04:52 +02:00
timothy	dbe72dbf03	docs(445): refresh web test count after rebase (983 -> 995) ... main added 12 web tests while this branch was in review, so the count I wrote into docs/testing.md went stale during the rebase. Verified by running the suite against the rebased tree: 995 tests / 105 files.	2026-07-25 14:04:52 +02:00
timothy	54919e4730	fix(445): pidfile-first cleanup ordering + clear BOOT_PID after wait ... Codex round-3 verdict was BLOCKED @ 29bc5119 with three High findings. All three are real; all three are fixed. They are narrower than the previous rounds — two are sub-millisecond races between adjacent statements — but the fixes are cheap and make the ordering correct rather than lucky. **Recycled-PID kill (the one that genuinely mattered).** `BOOT_PID` stayed set after `wait` had already reaped the launcher. A stale PID is not merely useless: the OS can RECYCLE that number during a long spec run, and cleanup would then SIGTERM an unrelated process. This is the same "never kill something you did not prove is yours" failure that got the earlier port-based reap deleted — it came back in a different disguise. Cleared immediately after `wait` returns. **Cleanup ordering.** cleanup() killed e2e-local.sh BEFORE reading the pidfile, so a signal landing between the server fork and the pidfile write could kill the launcher and lose the only handle on an already-running server. The pidfile is now read FIRST; stopping the launcher is best-effort and never the primary handle. **Fork/publish sliver.** A signal can also land between `... &` and `BOOT_PID=$!`, or between e2e-local.sh's fork and its pidfile write — the pidfile reads empty, then populates microseconds later, and the server is orphaned. cleanup now does a bounded grace re-read (~1s) when it has no PID. Verified this does NOT cost anything on a genuinely server-less abort: a bad ETV_BUILD_CONFIG still exits in 1s rather than sitting out the loop. ## Verification - passing run 3/3; failing run exits 1 - server-less abort exits in 1s (grace loop stays bounded) - targeted SIGTERM: exit 143, no listener left - zero orphaned processes after the full gate Refs #445	2026-07-25 14:04:52 +02:00
timothy	e8ad79623f	fix(445): make the boot wait interruptible so a mid-boot signal can't orphan the server ... Codex round-2 verdict was BLOCKED @ 2a5f26c0 on one High finding, which was correct and is the subtlest defect in this whole change. ## The defect Bash DEFERS a trapped signal while it waits on a FOREGROUND command. The boot was OUT="$(ETV_UI_PORT=... scripts/e2e-local.sh "$CONFIG_DIR")" so a TERM arriving during the (up to 120s) readiness wait did not run `on_signal` until boot completed — and a supervisor escalating TERM->KILL means the trap never runs at all, orphaning the server on its port. The pidfile added in the previous commit only helps IF the trap runs; this is the case where it doesn't. Proved the mechanism in isolation rather than asserting it, since the real boot is only ~2s locally and kept winning the race: foreground command substitution : TERM at t=1s -> trap ran at t=6s (deferred) background job + `wait` : TERM at t=1s -> trap ran at t=1s (prompt) ## The fix Boot in the background and `wait` on it. `wait` is interruptible, so the handler runs immediately; by then e2e-local.sh has already written $ETV_PIDFILE, so cleanup can reap a server whose PID this script has not yet parsed. Cleanup also now stops the backgrounded e2e-local.sh itself, so it cannot sit waiting on a server we just killed, and removes its temp output file. Boot diagnostics are preserved on the failure path (verified with a deliberately bad ETV_BUILD_CONFIG: the underlying "bin/<config> does not exist" error is surfaced, along with the exit status, which is now propagated rather than flattened to 1). ## Verification - mechanism: deferred-vs-prompt trap proven as above - passing run 3/3 green; failing run (--grep ZZZ_NOPE) exits 1 - boot-failure path prints e2e-local.sh's real error and its exit status - targeted SIGTERM: exit 143, no listener, no stray process - curl harness unaffected: 45/45 PASS - no orphaned processes after the full gate (an earlier count of 1 was a graceful shutdown still in flight, not a leak — re-checked clean) Refs #445	2026-07-25 14:04:52 +02:00
timothy	71706f3849	fix(445): silent-pass on spec failure + prove server ownership via pidfile [decisions-edit] ... Second review round. Codex returned BLOCKED @ e50a2624 with three High findings; all three were real and all three are fixed. One of them was severe and was introduced by my OWN previous "fix" commit. ## HIGH 1 — a FAILING spec run exited 0, silently passing CI `if ! npx playwright test; then status=$?; ... exit "$status"; fi` Under `!` negation bash sets `$?` to the LOGICAL NEGATION of the command's status, so inside the failure branch `$?` reads 0 — the script exited 0 on a failing run. Verified: `if ! (exit 42); then echo $?; fi` prints 0. A UI-E2E harness that reports success when its specs fail is worse than no harness. My five green local runs could never have caught this: the bug lives only on the failure path. Introduced by the log-tail improvement in e50a2624. Fixed with `set +e` / read `$?` / `set -e`, then exit that status explicitly. Verified: a deliberately-failing run (`--grep ZZZ_NOPE`) now exits 1. ## HIGH 2 + 3 — the pre-PID fallback needed lsof and only GUESSED ownership The mid-boot fallback reaped "whatever LISTENS on $PORT", which was wrong twice: - it needed `lsof`, which is ABSENT from the CI toolchain image (verified directly in the published image) — so it silently no-opped precisely where it was needed; - it INFERRED ownership from the earlier pre-flight rather than proving it, so a process that grabbed the port after the pre-flight — or a real instance on a shared host — could be killed. Reaping someone else's server is worse than the leak it was meant to fix. Replaced with an opt-in `ETV_PIDFILE`: e2e-local.sh writes the PID the instant it forks, BEFORE its readiness wait, which is exactly the window a mid-boot signal lands in. A pidfile we asked for PROVES ownership, needs no external tool, and works in CI. The port-based kill is gone; the lsof pre-flight remains as a friendly local check only. Verified: the pidfile is populated while still mid-boot (readiness not yet reached), names the real `dotnet ErsatzTV` process (not a subshell — which also re-confirms the `exec` fix), and killing that PID alone frees the port. Also dropped the `seq` dependency inside the trap (shell arithmetic instead), addressing the other reviewer's busybox concern. ## Docs-reviewer finding — my stated reasoning was wrong I justified amending `testing.e2e-local-fresh-config-dir` rather than superseding it partly on "renaming the heading trips CI". That's a true statement that does NOT bear on the choice: a supersession relocates to `archive/` with the heading INTACT (verified: archive/api.md keeps the #72 heading verbatim). Corrected to the actual reasons — the Rule never reversed, and the key is cited from docs/handoffs/chicorytv-issue-queue.md plus two docs/superpowers/ files, which a supersession would aim at an archived, stale-labelled record. ## Gotcha found by accident, now documented A flawed test of mine booted two e2e-local.sh instances concurrently and the first mysteriously failed to become ready. Cause: every run `rm -rf`s and re-copies the SAME build-output wwwroot, so a second run yanks the static files out from under a still-starting first instance. Documented in both the script header and docs/e2e-local.md, because the symptom (readiness timeout, or /app 404ing) looks nothing like a shared-directory race. CI is unaffected — its curl and UI-E2E steps are sequential. ## Budget: filed, not shaved This PR pushes the active decisions corpus 7 lines past its 5600-line soft budget (main was under). I trimmed my records repeatedly and each rewrite recovered ~1 line, because the content is load-bearing; continuing would have meant deleting useful rationale from a new convention record to hit an arbitrary cap. The validator's own remedy is "schedule a consolidation", so that is filed as #595 rather than paid for by starving the record. Non-blocking warning. Also filed #594 for the pre-existing `ci-image-pin` any-hex-length weakness. ## Verification - failing run exits 1 (was 0); passing run still 3/3 green - pidfile written mid-boot, names the real dotnet proc, reap frees the port - SIGTERM mid-run: exit 143, no orphan listener or process - curl harness unaffected: 45/45 PASS; ETV_PIDFILE unset => unchanged behaviour - decisions validator OK; zero orphaned processes after the full gate Refs #445 #533 #594 #595	2026-07-25 14:04:52 +02:00
timothy	0f1951340e	fix(445,533): harden e2e-ui lifecycle + retire stale #533 decision record [decisions-edit] ... Addresses the adversarial review round. Codex returned BLOCKED on the harness lifecycle contract; the second (cold) reviewer independently flagged the same trap-ordering defect, which is what made it credible. **Trap installed AFTER boot -> signal mid-boot orphans the server.** The window between `e2e-local.sh` returning and `trap ... EXIT INT TERM` had no handler, so a Ctrl-C/TERM there (or the PID-parse bail-out) left dotnet holding the port — violating the script's own "always kills the server" contract. The trap is now installed BEFORE boot. When the PID is not yet known, cleanup falls back to reaping whatever LISTENS on our port; attribution is sound because the pre-flight proved that port free moments earlier. **Signals were not re-raised.** A TERM landing beside a passing Playwright run could exit 0, reporting success for a cancelled run. INT/TERM now clean up and re-raise, so the wrapper dies BY the signal. Verified, not assumed: SIGTERM mid-run -> wrapper exits 143 (128+15) and leaves zero listeners and zero stray ErsatzTV processes. `e2e-local.sh` backgrounded a MULTI-command subshell, so `$!` is the subshell — not dotnet — wherever bash does not collapse it. Measured both ways: bash 3.2.57 (stock macOS /bin/bash), 2-command subshell : $! = SUBSHELL bash 5.3.15 (homebrew) : $! = leaf with `exec` (both versions) : $! = leaf Consequence on stock-macOS bash: every PID-based kill/liveness check targeted the wrong process, the escalation silently no-opped, and the server leaked. Fixed at the source with `exec`, which benefits all consumers (the CI step's trap and the e2e-functional.sh pairing), not just e2e-ui.sh. My first attempt to test this was WRONG and would have cleared the finding: a single-command subshell is collapsed on both versions. Only the 2-command form reproduces it. `testing.e2e-local-fresh-config-dir` was still `status: active` asserting the bug 'wait for either line' option today", "widening the probe ... is tracked as #533". `READY_LINE` existed ONLY in that record; nowhere in code. Amended rather than superseded: the operative rule (use a fresh config dir) is unchanged and still correct — only its RATIONALE moved from "the probe hangs" to "state bleed". Heading deliberately left alone: the validator matches records by `## HEADING`, so renaming fails CI as an "unlogged removal"; an explicit note now tells the reader the heading is historical. - Boot-failure diagnostics were lost: `OUT="$(...)"` aborts under `set -e` before the print. Now `if ! OUT=$(...)` so the output is shown. - Playwright failures surfaced no server-side evidence (nothing uploads the traces in web/e2e/.output). The server log tail is now printed on failure. - `lsof -ti :PORT` also matched outbound/TIME_WAIT sockets -> false positives. Now `-sTCP:LISTEN`. Documented honestly that the CI image ships no lsof, so the pre-flight is a local-developer guard only. - Predictable `/tmp/etv-pw-probe.$$` -> `mktemp` (symlink-redirect on a shared host). - Suggested escape-hatch port was 8411, which is scripts/security-scan.sh's default; 8409/8410 are prod/ersatztv-test on jazz. Now suggests 8419 and names the conflicts. - boot-gate.spec.ts comment overclaimed: after logout the browser holds NO cookie, so that assertion cannot prove stamp rotation (the curl harness does, by replaying the same cookie). Comment corrected to what it actually proves. Deliberately NOT changed: `ci-image-pin`'s regex accepts any hex length rather than the exact 7 chars ci-image.yml publishes (pre-existing guard weakness, not introduced here — filed as a follow-up rather than widened in this PR). Also trimmed the two records I had bloated: the active decisions corpus went over its 5600-line budget as a result of this PR (baseline on main was under), so the overflow was mine to pay down, not to pass on. - SIGTERM mid-run: exit 143, no orphan listener/process - curl harness unaffected by the `exec` change: 45/45 PASS - UI-E2E 2x clean; typecheck + lint clean; decisions validator OK, no size warning Refs #445 #533	2026-07-25 14:04:52 +02:00
timothy	f7fa57c816	docs(445): fix dangling decision key + clarify UI-E2E step reference [decisions-edit] ... Two review findings from the cold docs/convention pass. HIGH — the new `ci.ui-e2e-harness` record cited `` `ci.toolchain-image` `` as if it were a resolvable decision key. No such key exists anywhere in the corpus (verified: the only occurrence in all of docs/ was that citation itself). #390's toolchain-image work was deliberately never migrated to a standalone record — `ci.runner-placement`'s own Signals line says so. This is exactly the breadcrumb hazard docs/README.md -> "Knowledge retrieval" warns about: a future session (or a MemPalace lookup) resolving that key gets nothing back, and cannot distinguish "retired" from "never existed". Replaced with an explicit pointer to #390, `ci.runner-placement`, and docs/ci-cd.md, and stated outright that #390 has no standalone record. Swept the whole class rather than the one instance: every dotted key cited in added lines across the diff now resolves (ci.functional-e2e-harness, ci.runner-placement, ci.ui-e2e-harness). NIT — docs/ci-cd.md said the UI-E2E step is "Step 4 of this same job", which refers to the doc's own 4-item prose summary, not the YAML step list (where it is the 11th `steps:` entry). Reworded to name the actual step so a reader skimming the workflow isn't sent looking for a 4th YAML step. Refs #445	2026-07-25 14:03:59 +02:00
timothy	d8c0b3e752	feat(445,533): headless Playwright UI-E2E flows + fix e2e-local readiness probe [decisions-edit] ... Adds the last deferred #299/#363 follow-up: the flows that CANNOT be expressed as curl calls. Scope rule (the durable part) — assert only what the curl harness structurally cannot reach: 1. client-side form validation (the Setup confirm-password gate is pure React state and makes no request, so there is no HTTP contract to assert) 2. AuthGate's RENDERED states (Setup vs Login vs app) 3. the session cookie authenticating the SPA's OWN /api XHRs — curl proves the cookie works for curl, not that the app sends it 4. sign-out through the UserMenu back to the login gate New: web/e2e/boot-gate.spec.ts, web/playwright.config.ts, scripts/e2e-ui.sh (owns the whole lifecycle: fresh config dir -> boot -> specs -> always kill). Runs as a second step of the EXISTING advisory `functional-e2e` job rather than a new job: the dominant cost there is `npm ci` + the Release build, both already done, so this adds ~5s instead of duplicating a heavy job. It boots its own fresh instance on port 8410 because the first spec asserts the one-shot Setup gate that the curl step has already claimed on its config dir. Determinism (the issue asked for it explicitly): `serial`, `workers: 1`, `retries: 0` even in CI — a retry would let a flaky flow merge looking green. Measured 5 consecutive clean runs, ~2s each. Pins all five `container:` jobs to the toolchain image built by the preceding commit, which bakes `chromium-headless-shell`. Non-obvious coupling fixed: vitest's default include glob would have collected web/e2e/*.spec.ts and run it under jsdom. Excluded `e2e/**` by spreading `configDefaults.exclude` rather than narrowing `include` to `src/**`, because web/scripts/ holds a real vitest test an src-only include would silently stop running. `RebuildSearchIndexHandler` logs one of two mutually-exclusive lines just before `SystemStartup.SearchIndexIsReady()`: fresh config -> "Done migrating search index in {Duration}" reused config -> "Search index is already version {Version}" The probe watched only the first, so a reused dir waited out the full 120s timeout and then killed a perfectly healthy server. Widened to a `grep -Eq` alternation; the handler's if/else is exhaustive, so the pair covers every path to readiness. Verified with a negative control: on a reused dir the server is ready in 2s via the "already version" line, and the OLD probe string is genuinely ABSENT from that run's log — so the old code would have hung, i.e. the fix is load-bearing rather than incidentally passing. The "prefer a fresh config dir" guidance stays: that guards state bleed, which is a separate concern from the probe hanging. - `wait "$PID"` in the cleanup trap was a NO-OP: the server is a grandchild (launched in e2e-local.sh's subshell, which then exits), so `wait` fails instantly and was swallowed by `|| true` — cleanup did not actually ensure the port was released, exactly what its comment claimed. Replaced with a bounded `kill -0` poll, then SIGKILL. - Added a port pre-flight check: previously an occupied port surfaced as a 120s readiness timeout that reads like a broken build. Now fails in 0s naming the PIDs, and warns against blanket-killing `dotnet ErsatzTV.dll` (that reaps other sessions' servers). - UI-E2E: 5x clean (3 specs, ~2s); back-to-back runs pass with no manual cleanup - curl harness unaffected by the boot-script change: 45/45 PASS - web: 983 tests / 105 files green; typecheck + lint clean - vitest collection verified: excludes web/e2e, still collects web/scripts - Dockerfile sequence + browser launch validated verbatim in a container on the real amd64 base before committing; chromium launches as root with NO sandbox opt-out needed - decisions validator green; catalog regenerated - docs/decisions.md TOC repaired: it had drifted to 69 of 97 records and held a dangling anchor to the #72 record that #415 superseded into archive/. Regenerated with a generator validated against the 68 existing anchors (0 mismatches) -> 97/97, no dangling, no duplicates. Docs: docs/e2e-local.md (new "UI-E2E harness" section), docs/ci-cd.md (toolchain image + UI-E2E step), docs/testing.md, docs/README.md, docs/decisions.md (new `ci.ui-e2e-harness` record; `ci.functional-e2e-harness` amended — its Rule said "curl-only", now accurate). Refs #445 #533	2026-07-25 14:03:59 +02:00
timothy	cb5d1af65f	ci(445): bake headless Chromium into the CI toolchain image ... Adds `chromium-headless-shell` + its system deps to the shared CI toolchain image so the `functional-e2e` job can run the UI-E2E Playwright flows without installing a browser per run — the same "jobs install nothing at run time" rule as the rest of this image. Measured on this exact base (Ubuntu 24.04 ffmpeg base, amd64): - headless shell: 267M vs full chromium: 656M - `chromium.launch()` resolves to the shell anyway (the config never asks for headed), so the only cost is that a HEADED run inside this image would fail — CI-only, and deliberate. - Chromium launches as root in a container with NO --no-sandbox / chromiumSandbox:false opt-out, so the Playwright config needs no sandbox workaround. Verified rather than assumed. `ARG PLAYWRIGHT_VERSION` must stay equal to web/package.json's EXACT `@playwright/test` pin: Playwright ties a browser revision to the package version, so a mismatch leaves no usable browser. The build-time smoke test LAUNCHES the browser (not just a file test), so a missing system library fails the image build instead of a CI run. First half of the deliberate two-step toolchain bump: this commit is what ci-image.yml publishes as :<sha>; the follow-up commit pins it. Refs #445	2026-07-25 14:02:31 +02:00
timothy	0081bed4b1	docs(460): drop the "clean at rest" overstatement from the test header ... Follow-up to the cold review of 615e00a. The record was corrected to say only that no NEW sentinel rows are created, but the test file's header still carried the disclaimed "the data is clean at rest for every provider" claim — sitting on the very test that supposedly proved it. It now states what the tests actually pin, and names what they cannot speak to (rows written before this change, which keep the sentinel and are covered only by the read coercion). Also corrects docs/testing.md's ErsatzTV.Core.Tests count (543 -> ~650 measured; the neighbouring Scanner.Tests cell is corrected in #602 instead, to keep the two open PRs off the same line).	2026-07-25 13:46:41 +02:00
timothy	890745d1d4	fix(460): write null LastScan on disable-sync instead of the MinValue sentinel [decisions-edit] ... MediaSourceRepository's Plex/Jellyfin/Emby remove-and-recreate (disable-sync) flows stamped SystemTime.MinValueUtc into library.LastScan, so normal use kept minting 0001-01-01 sentinel rows. #409 fixed the READ side (the API coerces the sentinel to null and a migration cleaned the historical residue), so the wire contract was already correct — this stops new sentinel rows being written. Safe because every remaining LastScan reader either coalesces (LastScan ?? SystemTime.MinValueUtc) for its own non-nullable scan-comparison needs, or is the API read-boundary coercion itself — audited every reference under ErsatzTV{,.Application,.Core,.Infrastructure,.Scanner,.Mcp} plus web/. There is no Where/OrderBy/GroupBy on LastScan anywhere, so the SQL null-ordering divergence between SQLite and MySQL has no surface here. Scan-comparison behavior is unchanged. Deliberately ships NO second cleanup migration: rows written between #409's NullOutNeverScannedLastScan and this change keep the sentinel at rest, and the permanent read coercion — not a migration — is what keeps the contract honest for them (as it must be anyway for a restored or hand-edited DB). LibraryPath.LastScan is likewise left untouched: the flows re-add Paths with their original values, and its only readers are the local-library scan handlers, so it has no API surface and no remote-scan effect. Regression test per provider (MediaSourceRepositoryDisableSyncTests), proven non-vacuous: restoring the MinValue writes fails all three. [decisions-edit] — the media.lastscan-null-boundary record documented this write as an ONGOING sentinel source and rested its "the coercion is permanent" argument on it, so the rationale prose is corrected in the same PR per docs.decision-lifecycle. The Rule line is unchanged, so the generated decisions/README.md catalog is byte-identical. MediaSourceRepository.cs also loses its UTF-8 BOM (fix-as-you-touch, #311). fixes #460	2026-07-25 13:46:09 +02:00
timothy	2f2bcca681	fix(500): dedup incoming metadata collections so a duplicate name inserts once ... The remove-stale + add-new reconcile idiom materializes its add set with .ToList() BEFORE the loop mutates the existing collection, so the add filter (`incoming.All(x2 => x2.Name != x.Name)`) is evaluated against a snapshot. Two identically-named incoming entries whose name is not yet on the existing item therefore BOTH passed the filter and BOTH inserted — a duplicate row. Deduplicate the incoming set on the same key the filter compares (Name; Guid for Guids), in both copies of the idiom: - PlexMovieLibraryScanner.UpdateMetadata (the original) — genres, studios, actors, directors, writers, guids, tags. - JellyfinMusicVideoLibraryScanner.Reconcile{Genres,Tags,Studios,Artists} (added in #497, mirrors the Plex pattern verbatim). Plex ACTORS are the exception and get an artwork-preferring dedup hoisted out and shared with the remove filter, because that filter is keyed on (Name, artwork-presence) — it is the mechanism that drops an artwork-less actor so the add loop can re-add it WITH artwork. A bare DistinctBy(a => a.Name) there keeps the FIRST duplicate, so Plex listing the artwork-less copy first discarded the artwork; worse, the remove filter would still see the artwork-less duplicate, making its upgrade clause false, so the stale row was never removed and the artwork never arrived on ANY later scan either. Actor also carries Role/Order, which first-wins would silently drop too. Caught by the cold review of the first version of this commit. For the Jellyfin scanner the dedup sits at the incoming-list declaration, which also covers the remove filter — safe because all four of those filters only ask "is this name present at all", an answer duplicates cannot change. Tests: duplicate-collapse for both paths, the two Actors cases above, and a POSITIVE CONTROL proving distinct entries are still all added and stale ones still removed (without it, a mis-keyed dedup that collapsed genuinely different entries would pass every other assertion). Each proven non-vacuous. The Plex tests drive the protected UpdateMetadata through a minimal test-only subclass, as MediaServerMovieLibraryScannerTests already does. Low likelihood in practice (a media server emitting two identically-named genres for one item is unusual); this is defensive, with no observed occurrence. The same idiom is copied into 8 further scanners/repositories that this change deliberately does not touch (the issue scoped it to two paths) — filed as #600 so the class of bug is tracked rather than silently left in the majority of its instances. The dedup rule is recorded under scan.musicvideo-reconciliation. fixes #500	2026-07-25 13:04:38 +02:00
timothyandClaude Opus 5	c8e79f49f4	chore(586,594,485): PID-scoped E2E cleanup, ci-image-pin length guard, .gitignore core fix ... Three independent CI/repo-hygiene fixes swept together; disjoint file sets. fixes #586 — E2E cleanup is scoped by PID, never a pattern-wide pkill - New decision record `testing.e2e-cleanup-scope-by-pid`. - docs/e2e-local.md states the constraint where a BRIEF-WRITER sees it (the #586 root cause was a delegation gap, not agent error). - scripts/e2e-local.sh: reviewed against e2e-ui.sh's trap lifecycle and deliberately does NOT adopt it — its contract is to hand a running instance back to its caller, so an EXIT trap would kill the server the instant the launcher returned (both callers use `OUT="$(e2e-local.sh ...)"`). Recorded. - Instead it gains what actually prevents the incident: an lsof pre-flight that NAMES a foreign listener's PID rather than letting Kestrel fail its bind and surface as "process N exited before becoming ready". - Pre-flight probes BOTH bound ports, and ETV_STREAMING_PORT now defaults to ETV_UI_PORT. Program.cs binds a second listener whose port defaults to 8409 independently of ETV_UI_PORT, so `ETV_UI_PORT=8420` alone still bound 8409 and died against a foreign holder — i.e. the documented escape hatch was a dead end that led straight back to the confusion behind the pattern kill. fixes #594 — ci-image-pin accepts any hex length - Length is a separate invariant from correctness: the resolve/staleness checks compare resolved shas, so an 8-char pin of the right commit passes green while matching NO registry tag, and all five container: jobs then die at image-pull with `manifest unknown` (reads like a registry outage). - Guard fails at the gate and prints the exact tag to use. Verified against doctored pins: 7 green; 6/8/10 red. - Uses a literal 7 rather than a derived `--short=7`: in a full clone git may widen an ambiguous abbreviation, demanding a pin ci-image.yml can never publish. Escape hatch documented inline. - Also fixes a pre-existing misdiagnosis: zero pins reported "MORE THAN ONE". - docs/ci-cd.md documents the 7-char rule and `git rev-parse --short=7 HEAD`. fixes #485 — .gitignore `core` silently ignored `*/Core/` files - A bare `core` matched any path component named `core`; case-insensitively on macOS that swallowed every `*/Core/` SOURCE dir, so new untracked files were dropped by `git add -A` while tracked ones stayed fine — a clean local build and a CI checkout that fails to compile. - Now `/core` + `/core.[0-9]*`, both anchored (an unanchored `core.[0-9]*` would re-introduce the same silent-exclusion class this fixes). - Verified by diffing the full ignored-file set before/after: identical, and the three real Core/ dirs are trackable without -f. Docs updated in-PR: docs/e2e-local.md, docs/ci-cd.md, docs/decisions/ workflow-process.md (+ regenerated catalog), docs/handoffs/chicorytv-issue-queue.md. Follow-ups filed: #596 (the same shared-host reap in the Playwright-MCP recovery record) and the ci-image.yml `--short=7` publisher-side fix, which cannot ride this PR — editing ci-image.yml re-points ci-image-pin's `expected` at this commit and reds the gate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-25 12:46:13 +02:00
timothy	2fee3f0b94	docs(592): record that a skipped CI context is not red [decisions-edit] ... Amends ci.monitor-armed-at-pr-open (prose + Signals only; heading and key unchanged, no supersession) with the monitor-classification rules that were missing. Landing #436/#583 today, my CI monitor filtered per-context statuses on `!= "success"` and announced "NOT all green" on two fully green PRs, because `Build & push image (amd64)` reports `skipped`. Nothing was blocked — Gitea's combined /status already treats skipped as non-blocking and reported overall=success — but a false red costs a diagnosis cycle every time. Two corrections recorded: - The image job is skipped on EVERY PR (job-level `if: github.event_name != 'pull_request'`; images build only on push-to-main and tags), NOT because of the docs-only mechanism. Misattributing it to docs-only is a plausible-sounding wrong diagnosis, since docs-only gates STEPS precisely so required jobs still report success. decisions.md already stated the fact from the branch-protection angle; the monitor-authoring consequence was missing. - skipped / failure / cancelled are three distinct meanings and must not be collapsed. Prefer gating on the combined `.state`. The documented filter is verified in BOTH directions: silent on a green PR carrying a skipped build, and still dirty on a genuinely pending run. My first draft of it was itself broken — `select(.status != …)` after the pipeline had renamed `.status` to `.st`, so it compared against null and reported a green PR as nine failures. That failure is recorded in the note, per the "verify your detector" rule. fixes #592	2026-07-25 11:53:35 +02:00
timothy	22e89fb9a4	docs(440): correct two misleading text defects found in review ... Text-only follow-up; no behavior change (14/14 tests, lint clean, tsc clean). - AutoTuneScreen.tsx: the addSource comment claimed picking a hit that is already a base member makes "the existing row read as customised". It does not — patchSource(id, {}) materializes a DEFAULT draft, sourceCustomized is false for it, and sourcesRequest omits it, so the pick is a payload no-op whose only visible effect is the query clearing. Comment now states that. - spa-conventions.md §11: said WEIGHT_MIN/WEIGHT_MAX are "the same const pair the multi-collection editor uses". Same VALUES, separate screen-local consts — there is no shared module. The old wording invited a future reader to assume a shared seam that does not exist. Both were nits in the independent review of d3c89d87 (verdict MERGEABLE-WITH-NITS). Fixed rather than deferred because a comment that states the opposite of the code, and a doc that implies a nonexistent shared const, are exactly the kind of thing the next session reads and trusts. Remaining review nits deferred to a follow-up issue: exporting compile.ts's Lucene escaper instead of duplicating it, an exclude-all warning, and >50-source axis handling.	2026-07-25 11:30:36 +02:00
timothyandClaude Opus 5	e678e6653e	chore(docs): trim derivable content from CLAUDE.md, lazy-load the task-completion protocol ... Session context/config checkup (/doctor) found ~2.5k chars of always-loaded CLAUDE.md text that a session can reconstruct from the codebase, plus a task-specific workflow that only matters when closing an issue. Cut (derivable from the repo): - the `### Project Layout` table (what `ls` shows) - the `dotnet build` / `dotnet run` invocations (standard for the toolchain; the non-obvious `docker build -f docker/Dockerfile` line is kept) - the Language / Media / Functional C# bullets (stated by the csproj and Directory.Packages.props) Migrated to lazy loading: - the 7 mandatory completion steps and the `## Closing record` template move to .claude/skills/closing-an-issue/SKILL.md; only its one-line description stays resident. The `## Done-when` / merge-consent block stays in CLAUDE.md on purpose — it is safety-critical and describes hook behaviour that fires whether or not a skill was loaded. CLAUDE.md 13,306 -> 10,801 chars (~625 est. tokens saved per session). No convention, route, endpoint or decision changes, so no other doc updates are triggered by the docs-update table. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-25 11:22:48 +02:00
timothy	7e5d20be98	fix(583): gate every model-less dispatch, not just implementer-looking ones ... Independent review of d221a065 found the prompt-text heuristic both over- and under-fired. Confirmed repros: a read-only recon brief merely mentioning "worktree" nagged, while "author the change and open a PR", "land this on the branch" and "make the changes and commit them" all passed silently — the gate missed exactly the case it existed to catch. Root cause behind all three findings: the hook contradicted its own rule. The HARD CONSTRAINT says "every dispatched agent"; the hook gated only dispatches whose prose looked like an implementer. Prompt text is not a reliable signal for authority. Inverted: ask whenever `model` is absent, exempting only `fork` (the tool ignores a fork's model override, so a prompt could not be acted on). This dissolves all three findings rather than patching the regex — no phrasing dependence, no false-positive concept, and no "cannot commit" claim to be wrong about. The reviewer was right that Explore/Plan/claude-code-guide all carry Bash, so the old exemption rationale was false. The noise objection is answered by the escape hatch, not by scoping: naming a tier costs one parameter and the hook never fires. Self-eliminating for anyone following the rule. Decision record updated to record the rejected narrow design and why. Re-verified: 11 payloads incl. all three findings, the invariants (model named -> never fires; fork exempt; non-Agent passes), and robustness (malformed JSON, empty payload, missing tool_input, embedded backticks/quotes/newlines) — never crashes, never emits malformed JSON. decisions-validate: OK.	2026-07-25 11:19:31 +02:00
timothy	d3c89d87aa	feat(440): per-source weight steppers + exclude/add-untagged in Auto-Tune DetailPanel ... Wires the Auto-Tune DetailPanel's Content-sources pane to #425's backend: each member row gains a 1..1000 weight stepper and an include/exclude toggle, and a library typeahead adds a source that isn't in the axis's base set. Edits accumulate in the screen's per-channel draft (the existing §8/§11 guard covers them) and are flushed as the create request's `sources` array. - Only genuinely customised rows are sent, mirroring the server's own `customized` predicate — an all-default array is a backend no-op, so the field is omitted entirely and the channel keeps the cheaper fair-share shape. - Weights are clamped to 1..1000 on blur and again at save, so an out-of-range value never reaches the server as a raw 400 (spa-conventions §4a). - The add-untagged picker compiles typed text to `title:*…*` rather than forwarding raw Lucene: the index's default field does not match bare title words, so a raw forward would silently find nothing. - Removes the read-only #425 hint. Docs: spa-conventions.md §11 records the per-source correction-row convention. fixes #440	2026-07-25 11:15:20 +02:00
timothy	d221a06522	chore(583): make per-agent model routing a hard constraint + PreToolUse gate ... The kickoff tells the orchestrator to route agents by capability, but that rule lived only in a prose paragraph. On 2026-07-25 a session dispatched two implementers (#436, #440) with `model` omitted, silently inheriting the orchestrator tier — while following every bullet in HARD CONSTRAINTS in the same session. The bulleted list is what functions as the checklist; prose above it reads as background. - HARD CONSTRAINTS: keyed routing bullet requiring the tier to be stated in the dispatch itself, so an invisible omission becomes visible output. Prose paragraph tightened to point at the key rather than restate the table (the kickoff is pasted into every session — duplication is a per-session tax, #542). - .claude/hooks/pretooluse-agent-model.sh: PreToolUse on Agent, `ask` when a committing agent is dispatched with no explicit `model`. Narrow by design — passes through read-only/recon types, `fork` (model override ignored by the tool), and any dispatch already naming a tier, because a gate that fires on every fan-out trains one-shot dismissal. `ask` not `deny`: routing is a judgment call with no derivable right answer, unlike the H6/H10 merge gate. - New decision record `process.per-agent-model-routing`; catalog regenerated. Decision matrix verified against 9 payloads incl. a replay of the dispatch that missed. decisions-validate: OK. fixes #583	2026-07-25 11:02:42 +02:00
timothy	6cf99718ee	feat(436): arbitrary-depth rule-builder group nesting ... The rule builder's Group nesting was capped at one level (#176's Kodi model). Generalize it to recursive nesting bounded by a single shared constant, MAX_GROUP_DEPTH (types.ts, = 5, root group is depth 0): - parse.ts: replace the allowNested boolean with a depth counter that recurses to the cap; deeper input stays out of subset (null -> raw-text fallback), so parse remains the exact inverse of compile. Sub-group detection now requires the leading '(' to be the one closed by the trailing ')' (quote/escape aware), so '(a)x(b)' can't be mistaken for one wrapped group. - RuleBuilder.tsx: 'Add group' is offered while depth < MAX_GROUP_DEPTH instead of only at the root; nested group boxes get box-sizing: border-box so per-level padding can't overflow (no global reset). - roundtrip.test.ts: the 500-tree generator nests to the cap and asserts the corpus actually reached it; explicit depth-3 cases added to compile/parse/validation tests and a depth-gate test to RuleBuilder. compile.ts and validation.ts already recursed correctly and are unchanged. No backend/OpenAPI change. Docs: spa-conventions.md §12; decisions lifecycle — new active record spa.rulebuilder-nesting, predecessor spa.smartcollection-rule-builder relocated to docs/decisions/archive/spa.md as superseded. fixes #436	2026-07-25 11:01:35 +02:00
timothyandtimothy	65c0e09179	feat(415): per-channel fault detection — server-derived health object + Problems filter (#581) ... Closes #415. Server-derived health object on the channel list + detail DTOs (built-timeline detection, kind-agnostic across all 5 PlayoutScheduleKind; assessable gate keyed to the owning channel's mode), single "Problems" SPA filter with per-fault badges. Supersedes #72's api.channel-health-signal decision. Co-authored-by: Timothy <timothy.look@gmail.com> Co-committed-by: Timothy <timothy.look@gmail.com>	2026-07-23 20:45:19 +00:00
timothyandClaude Opus 4.8	5fba6187ce	feat(437): inline RuleBuilder smart-query authoring in Channel Builder ... Adopt the reusable RuleBuilder (#176) for inline query authoring in the Channel Builder (/app/new-channel). Extract CollectionsScreen's smart-collection dialog into a shared, self-contained component (SmartCollectionDialog) and consume it in both screens; the Channel Builder's Collections source gains a "New smart query" action that persists the authored query as a real SmartCollection and adds it to the lineup by smartCollectionId. Pure frontend — no REST/MCP surface change (the MCP already exposes ersatztv_create_smart_collection). The Auto-Tune half of #437 is a different primitive (group-by, not single-query filtering) and a backend epic; it is designed separately in docs/superpowers/specs/2026-07-23-auto-tune-arbitrary-field-design.md and filed as its own issue rather than wired here. Verification: web typecheck + lint clean; full vitest suite green (981, incl. a new inline-smart-query test); cold-context review clean; live-E2E on a real instance (query authored in the SPA persisted as SmartCollection "Action Picks" and added to the lineup, 0 console errors). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 22:08:41 +02:00
timothyandClaude Opus 4.8	7e6861a928	fix(434): accurate endpoint description (DB-sourced) + tests for case-insensitivity, tag exclusion, distinct ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:55:32 +02:00
timothyandClaude Opus 4.8	cc5ec712e0	chore(434): restore ISearchIndex/LuceneSearchIndex to main (revert incidental BOM strip) ... The GetFieldValues additions were reverted in the DB-sourcing rework; these two files had only an incidental BOM strip left, which pulled unrelated pre-existing whitespace debt into the scoped format gate. Restore byte-identical to origin/main. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:44:15 +02:00
timothyandClaude Opus 4.8	a654ae554a	fix(434): actually strip the BOM this time (previous commit staged before stripping) ... ISearchIndex.cs / LuceneSearchIndex.cs still carried a BOM after the prior commit — the strip ran after `git add`, so the staged (BOM'd) content was what got committed. No content change beyond the BOM. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:44:15 +02:00
timothyandClaude Opus 4.8	9cb51f6a73	docs(434): [decisions-edit] update field-values decision + api-conventions for DB sourcing ... Edits the api.search-field-values decision record's rationale prose in place (same key, same date, not a reversal) to describe the DB-sourced per-field distinct-values design and the narrowed allow-list, replacing the superseded Lucene-term-dictionary description. Updates the endpoint's api-conventions.md entry the same way. Regenerated docs/decisions/README.md via build_decisions_catalog.py; decisions_validate.py passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:44:15 +02:00
timothyandClaude Opus 4.8	fb5f609cef	rework(434): source facet typeahead from DB distinct values, not Lucene analyzed tokens ... The Lucene term dictionary stores lowercased word tokens for analyzed text fields ("Science Fiction" -> science/fiction), so the typeahead was suggesting fragments instead of whole values. GetSearchFieldValuesHandler now injects IDbContextFactory<TvContext> and resolves an explicit per-field-name distinct-values query (genre/studio/director/writer/actor/ artist/tag/network/collection/video_codec/album), with state and video_dynamic_range computed in memory and content_rating split on '/' to match what search actually matches on. title/show_title/album_artist have no distinct source and now correctly 404 (free-text fallback), same as before. Reverts the GetFieldValues additions to ISearchIndex/ LuceneSearchIndex/ElasticSearchIndex back to their pre-#434 state (BOM stripped per #311, otherwise byte-identical). Endpoint shape, DTO, controller, and OpenAPI are unchanged (no diff from ./scripts/update-openapi.sh). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:44:15 +02:00
timothyandClaude Opus 4.8	a1d3adda9c	fix(435): reject non-integer relative-date N (1e2/7.0) — strict digit-string validation ... - validation.ts: ruleError now uses /^\d+$/ regex instead of Number.isInteger - dateMacro.ts: isValidN now uses /^\d+$/ regex instead of Number.isInteger - Prevents accepting scientific/decimal notation (e.g. '1e2', '7.0') that break compile↔parse contract - Added tests for both invalid cases returning null/error Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:44:15 +02:00
timothyandClaude Opus 4.8	c2fb62dc88	docs(434,435,438): decisions records, spa-conventions §12, api-conventions ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:44:15 +02:00
timothyandClaude Opus 4.8	13bfcebd04	feat(434): facet-value typeahead combobox for text fields ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:43:27 +02:00
timothyandClaude Opus 4.8	8d805ecbd8	fix(434): ElasticSearch field-value typeahead returns empty list, not a 500 ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:43:27 +02:00
timothyandClaude Opus 4.8	b4ae0bde18	feat(434): distinct-values search endpoint (text fields, Lucene term enumeration) ... Adds GET /api/v1/search/fields/{name}/values?q=&limit= — the backend slice of the visual rule builder's facet-value typeahead (#434). Enumerates distinct Lucene term values for a text field via MultiFields.GetTerms + TermsEnum, filtered by a case-insensitive prefix, limit clamped to [1,50]. 404s when the field is absent from SearchFieldCatalog or is not type "text". ElasticSearchIndex (the optional external backend) throws NotSupportedException for this method — its text fields are analyzed, not keyword-mapped, so a terms aggregation isn't safe to guess at without verifying against a live cluster. Regenerated OpenAPI trio (v1.json, v1.d.ts, endpoint-index.md). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:43:27 +02:00
timothyandClaude Opus 4.8	b66a3400ae	feat(438,435): RuleBuilder UI — validation surface, relative-date inputs, single-child toggle ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:43:26 +02:00
timothyandClaude Opus 4.8	30a8a5752c	feat(435): parse relative-date macros; round-trip against normalizeGroup ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:43:26 +02:00
timothyandClaude Opus 4.8	d487ce3601	fix(438): drop empty nested subgroups instead of emitting "()" in compiled Lucene ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:43:26 +02:00
timothyandClaude Opus 4.8	575f652760	feat(438,435): compile relative-date macros; drop invalid rules instead of malformed Lucene ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:43:26 +02:00
timothyandClaude Opus 4.8	000a987a48	test(435): cover all relative-date combos, invalid-input rejection, composed round-trip ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:43:26 +02:00
timothyandClaude Opus 4.8	e91891a453	feat(435): dateMacro compile/parse mapping for relative-date operators ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:43:26 +02:00
timothyandClaude Opus 4.8	c301db10c4	feat(438,435): extend rule types; validation + single-child normalization helpers ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:43:26 +02:00
timothyandClaude Opus 4.8	5618ad12ea	plan(434,435,438): RuleBuilder bundle implementation plan ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:43:26 +02:00
timothyandClaude Opus 4.8	4e2183d9a3	design(434,435,438): RuleBuilder bundle spec — validation, relative-date operators, facet typeahead ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:43:26 +02:00
timothyandtimothy	8b9a7ed541	feat(414): stamp immutable Channel.Origin (auto-tuned vs user-created) and surface it (#575) ... Co-authored-by: Timothy <timothy.look@gmail.com> Co-committed-by: Timothy <timothy.look@gmail.com>	2026-07-23 18:12:25 +00:00
timothyandtimothy	7a7c611267	fix(458): reject duplicate names on playlist &amp; playlist-group rename (#576) ... Co-authored-by: Timothy <timothy.look@gmail.com> Co-committed-by: Timothy <timothy.look@gmail.com>	2026-07-23 18:07:43 +00:00
timothyandtimothy	0c063c23fb	harden(421,559): percent-encode access_token in IPTV URLs, redact from logs, no-store on tokened manifests (#574) ... Co-authored-by: Timothy <timothy.look@gmail.com> Co-committed-by: Timothy <timothy.look@gmail.com>	2026-07-23 16:30:59 +00:00
timothy	bb3e245b3c	docs(392): drop predecessor cross-ref line (append-only decisions diff)	2026-07-23 08:27:02 +02:00
timothyandClaude Opus 4.8	60fecd18b7	chore(392): regenerate migration + OpenAPI + decisions catalog after rebase onto main ... The rebase onto origin/main (which merged #74's ChannelGraphicsElement migration) left the PadToNearestMinute migration's embedded Designer.cs model snapshot stale — it still reflected the pre-#74 model, so EF's diff against it produced an empty Up()/Down() when naively regenerated. Reset TvContextModelSnapshot.cs to origin/main's true post-#74 state, then re-ran scripts/add-migration.sh so the migration's Designer.cs correctly folds in ChannelGraphicsElement and the migration's Up() adds only the PadToNearestMinute column. Verified has-pending-model-changes is clean for both providers. OpenAPI (v1.json/v1.d.ts/endpoint-index.md) and docs/decisions/README.md regenerated identically to the auto-merged state, so nothing to commit there — confirmed both padToNearestMinute and #74's graphics-elements endpoints/decision key are present. Stripped a UTF-8 BOM this dotnet-ef/dotnet-format toolchain wrote into the regenerated migration + snapshot files (known BOM trap, ersatztv#311). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 08:18:15 +02:00
timothy	767f96802e	fix(392): apply schedule-level pad to Fill-With-Group items (reverse nav lost in DeepCopy)	2026-07-23 08:03:31 +02:00
timothyandClaude Opus 4.8	6a05363e2a	design(392): mirror 60-min pad option into FillerPresets prototype ... Schedule-level pad control has no prototype counterpart (the schedule-level scalar form is not modeled in Schedules.jsx); nothing to mirror there. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 08:03:31 +02:00
timothy	e99e72ee71	docs(392): record per-schedule clock-padding decision + domain-model field	2026-07-23 08:03:31 +02:00
timothy	f4ba219000	feat(392): add pad-to-clock-boundary control to the schedule editor	2026-07-23 08:03:31 +02:00
timothyandClaude Opus 4.8	beab219a90	test(392): cover schedule pad through Flood/Duration/Multiple; make day-seam clamp precise ... Adds parameterized invariant tests (Schedule_clock_padded_offline_multimode) exercising schedule-level clock pad + offline advance across a 2-day window (two midnight crossings) through Flood, Duration, and Multiple — previously only PlayoutModeSchedulerOne had any coverage. Fixture uses sub-15-min content so each padded item occupies one :15 slot and Duration's fill-the-block contract tiles exactly (no off-boundary packing). Part 2 (precision): replaces the day-boundary anchor-clamp magnitude heuristic (overrun <= one pad interval on a padded schedule) with a precise signal — the last scheduler now reports the exact offline-pad target it advanced CurrentTime to (transient PlayoutSchedulerResult.ClockPadOfflineTarget, never persisted), and the clamp exempts only when CurrentTime equals that target exactly. A non-offline overrun (Duration/Flood/Multiple ending short of its natural end, a hard-stop, a tail advance) changes CurrentTime away from the target and still clamps, so persisted NextStart no longer shifts by up to an interval. No persisted-schema/migration change. Output byte-identical for all existing goldens. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 08:03:31 +02:00
timothy	7efe38dd04	feat(392): honor ProgramSchedule.PadToNearestMinute in the Classic builder	2026-07-23 08:03:31 +02:00
timothy	9ea2750cbf	feat(392): expose ProgramSchedule.padToNearestMinute on the REST API	2026-07-23 08:03:31 +02:00
timothy	75d5c385b2	feat(392): add ProgramSchedule.PadToNearestMinute column (dual-provider migration)	2026-07-23 08:03:31 +02:00
timothy	2f1ede6030	feat(392): add 60-minute option to filler-preset pad increment	2026-07-23 08:03:31 +02:00
timothy	0b814abf41	docs(392): extract shared ComputePadBoundary helper in plan (DRY)	2026-07-23 08:03:31 +02:00
timothyandClaude Opus 4.8	9385bc8acd	docs(392): implementation plan for per-schedule clock-boundary padding ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 08:03:31 +02:00
timothyandClaude Opus 4.8	425b4b65f9	docs(392): design spec for per-schedule clock-boundary padding toggle ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 08:03:31 +02:00
timothyandClaude Opus 4.8	892f4e354b	fix(570): On Now/Next overlay YAML renders — font_family + format_datetime ... The #74 seeded on-now-next.yml never rendered at transcode time: - format_datetime was called with 2 args; it needs 3 (DateTimeOffset, timeZoneId, format) and does the tz conversion itself, so the Scriban render threw. Drop the NEXT start-time (avoids hardcoding a timezone in a shipped default). - styles had no font_family; CustomFontMapper.TypefaceFromStyle crashes on a null family before its default-font fallback. Add font_family: Noto Sans. Verified live on ersatztv-test via frame capture. Adds a seeder test that deserializes the YAML and asserts base_style resolves + every style sets a font. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-22 23:10:35 +02:00
timothyandClaude Opus 4.8	b7d58bbf32	docs(74): tighten overlay suppression rule (Merge-during-filler also clears) ... Folds whole-branch review finding #2. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-22 22:11:46 +02:00
timothy	c80839b338	docs(74): channel-level graphics attachment + On Now/Next overlay	2026-07-22 22:11:46 +02:00