_anchor() mapped each space/hyphen to its own '-' without collapsing runs,
so the standard heading separator " — " (space, em-dash, space) produced a
double hyphen in every generated anchor. Since nearly every real decision
record heading uses that separator, this made the catalog emit a dead link
for essentially every row. Fix: after building the char list, collapse
consecutive '-' into one and strip leading/trailing '-' via re.sub, matching
how Goldmark/GitHub/Gitea sluggers behave.
Also fixed main() writing an extra trailing newline (want already ends in
"\n", then "+ \n" appended a second one) so docs/decisions/README.md now
ends with exactly one trailing newline; --check still compares via .strip().
Added test_anchor_collapses_em_dash_and_keeps_underscore to pin the anchor
behavior against the reported iptv.base_url case.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds scripts/build_decisions_catalog.py, which renders docs/decisions/README.md
as a compact table of only 'active' decision records (sorted by key), and its
test scripts/tests/test_build_catalog.py. Supports --check for CI drift
detection. No decision records are migrated yet, so the generated catalog is
currently empty (banner + header only) — expected at this stage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds scripts/decisions_validate.py: lifecycle invariant checks (unique
active key, key format, reciprocal supersession, removed-without-archive,
rationale-rewrite-without-token, catalog staleness, corpus budget) plus
git-diff helpers for merge-base-based CI checks. Deviates from the task
brief in one spot: REQUIRED_META narrowed to (key, status) — the brief's
(key, status, since, supersedes, superseded_by) makes its own
test_clean_corpus_passes fail, since since/supersedes/superseded_by
default to None on bare Record() instances built without going through
the markdown parser.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds scripts/decisions_lib.py, the shared parser for ErsatzTV decision
records (docs/decisions.md + docs/decisions/*.md). Parses H2 sections
into Record dataclasses, distinguishing migrated records (visible
metadata block: key/status/since/supersedes/superseded-by + Rule/
Signals/Mechanics) from legacy-unmigrated ones with no metadata line.
scripts/ is now an importable package (scripts/__init__.py,
scripts/tests/__init__.py) so later tools can `import scripts.decisions_lib`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The test-job memory instrument (#411) reported memory.peak — the high-water mark of
memory.current, which charges reclaimable page cache to the cgroup. A build does heavy
NuGet/npm/obj/bin/coverage I/O, so cache can dominate the peak, and page cache is reclaimed
under a tighter cap rather than OOM-killed. Sizing a per-job cap (server-management#604) off
memory.peak therefore inverts the decision. The OOM-forcing quantity is peak anon, which the
kernel exposes no counter for and which the end-of-job split misses (a job that peaks
mid-dotnet-test then frees reports a low anon).
New scripts/ci-peak-anon.sh: a `start` step (before Build/Test/Coverage) launches a detached
background sampler tracking the high-water mark of cgroup anon; a `report` step (last) stops it
and prints the sampled peak anon as the headline, keeping memory.peak + end-of-job split as a
cache-inflated ceiling and reference. Both continue-on-error + fail-open so they never redden a
build. Validated on bumblebee: survives step-boundary re-execs, catches a transient 2.5 GiB
anon spike the snapshot reports as 0, stops cleanly on kill, degrades gracefully.
Compiler-server A/B (swap-off, sampled peak-anon, n=2 interleaved): OFF (CI config) ~5.84 GiB
consistent; ON (defaults) 6.3-7.6 GiB, always higher, + a ~3 GiB resident VBCSCompiler.
Disabling the servers is worth it, but OFF sits right at 6 GiB for the build phase alone and the
test job adds test+coverage, so #406's "budget loosens well under 6 GiB" premise is not
supported. Size the cap off the live test-job sampler.
Docs: ci-cd.md instrument section rewritten (peak-anon headline + A/B table + premise verdict);
decisions.md entry added. No .cs touched.
fixes#412
Adds "Flow C" to scripts/e2e-functional.sh, the last deferred lock-contention flow from #363.
A playout build is enqueued onto the single-consumer WorkerService channel and the trigger
returns before BuildPlayoutHandler acquires the lock, so an accepted trigger does not prove the
lock is held. Flow C makes it deterministic: seed a Classic Flood schedule over a few short
ffmpeg episodes, crank PlayoutDaysToBuild=5 (~43k items ~= ~1s build), then POLL GET
/playouts/{id} until isLocked:true before firing. Asserts PUT /playouts/{id} -> 409, reset ->
409, and the list-projection isLocked:true while locked; then isLocked:false + PUT -> 200 after
the build (proving the 409 is lock-specific). Each racing assertion is guarded so a build that
finishes mid-flight degrades to an advisory skip, never a false red; the whole flow self-skips
without ffmpeg or if the build is never observed locked.
Sized by measurement on a fresh instance -- going wider is counter-productive (a 777k-item build
saturates the single worker with post-build gap/overlap jobs). Verified green across 6
fresh-instance runs; cold adversarial review MERGEABLE.
Docs: docs/e2e-local.md + docs/ci-cd.md updated to describe Flow C and drop it from the
"deferred" lists.
fixes#444
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The kickoff dispatches a cheap model to rank the backlog, and the lore then made the
orchestrator re-derive its mechanical claims (deps, milestone tiering, priority order,
in-progress state) because a small model kept getting them wrong. This pays that tax down:
the mechanical checks are now CODE — GET /dependencies exclusion, LOCAL
.milestone.state/review/priority tiering (never the no-op ?milestones= filter),
(tier,priority,issue#) ordering, in-progress/parked/PR exclusion — leaving only two
JUDGMENT flags (CLAIM?, UMBRELLA?) for a human/model to resolve.
- scripts/select-queue.sh: fail-open on no-creds/unreachable; ~1.3s; shellcheck clean;
UMBRELLA? computed in the jq pass (no per-candidate body fetch).
- handoff kickoff: run the script FIRST; trust its deps/tiering/ordering, recheck only flags.
- handoff lore: the three 'cheap selector unreliable → re-derive by hand' bullets kept as the
EVIDENCE for why the script exists; the prescription is redirected to 'run the script'.
Operator-requested this session: 'rather than have the lore make us redo the selector's work,
improve the selector.'
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- check seed_library_path succeeded (print lastrowid) so a silent seeding
failure surfaces as a FAIL instead of degrading Flow A to an advisory skip
with no diagnostic (Medium)
- anchor the scan-status match to '"libraryId":2,' so it can't substring-match
"libraryId":20/23 if the suite ever creates more libraries (Low)
- drop the no-op ?deep=true (local scans always ForceScan; deep only affects
Plex/Jellyfin/Emby) + note why (Low)
- soften "guaranteed 409" for the scan flow to note the tiny residual TOCTOU
gap the multi-second scan covers; Flow B stays race-free by construction (Low)
- correct the "WAL tolerates a second writer" wording to the real reason (the
busy-timeout retry serializes the writer) in the script + both docs (Nit)
- use TEST-NET-1 192.0.2.1 (RFC 5737) instead of RFC1918 10.255.255.1 for the
non-routable Jellyfin address (Nit)
Re-verified: fresh-instance harness runs green (38/38), lock section
deterministic. Functional E2E CI job already green on the prior head.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends scripts/e2e-functional.sh with the two IEntityLocker 409 contracts the
first cut (ersatztv#299) deferred as "racy", made DETERMINISTIC by firing the
racing request only once the lock is provably held (no sleep-and-hope):
- library-scan "already scanning" 409: seed ~60 tiny ffmpeg clips into the
built-in Shows library so the scanner subprocess runs a few seconds, poll
GET /libraries/scan-status until the library is active (that window is a
strict subset of the scan lock's held window — StartScan after LockLibrary,
EndScan before UnlockLibrary), then a second POST .../scan is a guaranteed
409. Self-skips (advisory) when ffmpeg is absent.
- external-collections "already scanning" 409: seed a Jellyfin media-source row
pointing at a non-routable address so the background sync hangs and the
per-family lock stays held; the lock is taken synchronously before the 202,
so the 202 proves it held. collections-scan-status corroborates; unknown
source 404.
Seeding uses python3's stdlib sqlite3 (already a harness dep) to insert rows the
API can't create (LibraryPath, media-source); WAL mode tolerates the second
writer. No new CI step/dependency — ffmpeg ships in the toolchain image.
Verified: 4/4 fresh-instance runs green (38/38), lock section deterministic.
Still deferred to #363 follow-ups: the playout-build lock 409 + isLocked
projection (#215) and the UI-interactive Playwright flows.
Docs updated same PR: docs/e2e-local.md, docs/ci-cd.md, the functional-e2e
job comment in .gitea/workflows/docker-build.yml.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The test/migrations jobs check out fetch-depth:1. A shallow clone has NO
origin/<base> tracking ref and no merge-base, so the three-dot
origin/main...HEAD errored -> empty diff -> docs_only=false -> EVERY docs-only PR
silently ran the full matrix (safe but the skip never fired). git fetch always
writes FETCH_HEAD, which resolves in a shallow clone; diff against it with a
two-dot tree diff (no merge-base). Confirmed in a real shallow file:// clone:
origin/main did NOT resolve and three-dot errored, while FETCH_HEAD two-dot
correctly returned the docs file. api-docs/format were unaffected only because
they use fetch-depth:0.
Refs #416
The changed-set git diff had rename detection on by default, so a code->docs
rename (Foo.cs -> docs/Foo.md) showed only the destination and was misclassified
as docs-only, skipping required tests on a code change. --no-renames surfaces the
source deletion -> full matrix. Empirically verified. This is the cold-review
MEDIUM; it was applied in the working tree but never committed before the first
push (index/worktree mismatch) — committing it now.
Refs #416
Docs-only changes (docs/** or *.md) ran the full docker-build matrix (~9 min).
Each heavy job (test, migrations, functional-e2e, build) now runs
scripts/ci-detect-docs-only.sh as its first post-checkout step and gates every
real step on docs_only!='true'. The jobs still RUN and report success in
seconds, so the two required contexts keep reporting — a docs-only PR stays
mergeable (never an if:-skipped required job; Gitea 1.25.4 reports if-skip as
'skipped', verified with a throwaway probe PR). build skips its image steps on a
docs-only push to main; tag builds force docs_only=false. Detection uses
--no-renames so a code->docs rename can never be misclassified as docs-only.
Refs #416
Re-review of the fix commits came back MERGEABLE with three findings worth acting on.
MEDIUM -- the PR documented a standing rule ("--memory without --memory-swap silently
grants 2x in swap") and then didn't apply it to the two sites this repo owns:
docker-build.yml's smoke container and scripts/migration-smoke.sh, both `--memory 2g` with
no --memory-swap. Pre-existing rather than a regression, but a rule you don't follow in
your own repo isn't a rule. The migration-smoke one matters most: it runs on the PROD host
in the release path, so a runaway migration should die against its cap rather than quietly
swap out the box serving media.
LOW -- and this is the important one: the docs leaned "peak 8305 MiB is probably mostly
reclaimable cache". An independent probe (full solution build, same CI image, shared
compilation off) measured peak 9457 MiB / anon 7134 MiB / file 421 MiB. ANON DOMINATED.
Having verified the *mechanism* (peak overstates because it counts page cache), I guessed
the *magnitude* in the direction I preferred -- the exact failure this entry criticises,
committed inside the entry criticising it. Corrected in ci-cd.md, decisions.md and on
server-management#604 (where the previous comment could have led to an unsafe 6g cap).
Consequences now recorded honestly: a 6g cap looks UNSAFE, #570's "6g proved too tight" is
the rule not an outlier, and #406's premise ("if this brings peak RSS well under 6 GiB the
whole budget loosens") is looking DEAD -- the 7134 MiB anon was measured with shared
compilation already off. The switches remain right; the looser budget they were meant to
buy does not follow.
NIT -- dropped the unverified claim that this also disables the Razor build server. The
UseRazorBuildServer -> UseSharedCompilation fallback is .NET 5-era; Razor has been an
in-process source generator since .NET 6, so there is likely no separate server to disable
on .NET 10. Unverified, zero impact, so it has no business in a doc arguing for
measurement over assumption.
[decisions-edit]: the touched docs/decisions.md lines were added by this PR's own earlier
commits, not settled entries on main -- net vs origin/main remains a pure insertion (0
deletions, verified). Also the sanctioned reason: the entry was factually wrong (see LOW).
Verified: both workflows parse; migration-smoke.sh passes bash -n; the parsed mysql option
string is `--memory=2g --memory-swap=2g --cpus=2`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codify the manual live-E2E curl flows sessions have been re-running by hand
into a CI regression net: a new `functional-e2e` job boots the app from source
(via scripts/e2e-local.sh, parameterized for Release) and drives scripts/
e2e-functional.sh against it.
First-cut contracts (all curl-only, deterministic, no seeded media/ffmpeg/browser):
- legacy->SPA redirect sweep + the /api,/artwork never-redirect exemption
- auth/CSRF/security-stamp flow (setup-claim, read-gate, CSRF, login, logout+revoke)
- library-scan status contract (404/202/scan-status)
- optimistic-concurrency If-Match/412 round-trip
Advisory by design (separate job, not a `build` dependency, not a required
check) so a functional-E2E flake can't block the unit-test gate; promote once
proven, mirroring the migrations-job rollout. SQLite default -> no DB service.
Deferred to #299 follow-ups (need scanner+seeded media or a browser to be
deterministic): the racy 409 re-trigger, playout-build lock 409, Playwright UI
flows.
Assertions verified 30/30 green against a real Release-built instance; caught
/artwork/* returning 400 (not the 404 a static read suggested).
Docs updated same PR: docs/ci-cd.md (new job), docs/e2e-local.md (harness),
docs/decisions.md (append-only).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cold review of PR #331:
- Move the ZAP-container reap from an inline post-scan line into cleanup() (the
EXIT trap) so a SIGINT/timeout kill mid-scan can't leave it running.
- semgrep absent now prints a stderr WARN instead of silently skipping the SAST
pass (no false confidence for a release gate).
refs #314
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scripts/security-scan.sh: boots a THROWAWAY container from the image under test
(fresh config volume; never the deployed prod/test container — the active scan
attacks write endpoints), reads its machine key, and runs an authenticated OWASP
ZAP api-scan that imports /openapi/v1.json (all 160 /api/v1 ops) and injects
X-Api-Key on every request via a replacer rule so it reaches the
[RequiresAuthentication] + RequireKeyForReads surface — then a semgrep SAST
cross-check. Wrapped in `timeout` because zap-api-scan can hang in post-scan
cleanup after the report is already written.
docs/ci-cd.md: new 'Security scanning' section (out-of-ecosystem black-box gate,
run on the docker host per-release like migration-smoke, not a per-PR CI job) +
the Microsoft.OpenApi 2.7.5 pin note in dependency management.
refs #314#197
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The new api-docs gate (#303 H4/H5) runs update-openapi.sh after only `dotnet
restore`. The script's `dotnet build -t:GenerateOpenApiDocuments` does not compile
the project (OpenApiGenerateDocumentsOnBuild=false), so in a clean tree getdocument
fails with 'ErsatzTV.deps.json does not exist' (exit 129). This PR is the first to
change the /api surface and thus the first to exercise the gate's regen path,
exposing the latent bug. Add a full `dotnet build` before the doc-gen target so the
assembly + deps.json exist. Verified: clean-tree regen now succeeds with zero diff.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adversarial review (MERGEABLE-WITH-NITS) findings:
- Remove the broad FAIL_RE log-scan (matched benign ErsatzTV startup noise — library
scans against absent media mounts, EF connection retries — risking a false-FAIL that
blocks a good release). It was also redundant: a failed migration faults the
BackgroundService -> default StopHost -> container exit, which the early-exit check
already catches reliably (per the reviewer's own analysis). Migration failure is now
detected by early container exit + timeout + the post-boot serve probe.
- Validate --timeout is a positive integer (was: '--timeout abc' -> 0 -> instant false-FAIL).
- Log the resolved image id after (attempted) pull, so a pull-failure that rehearses a
stale local :latest is visible to the operator.
Re-validated live on bumblebee: :latest vs the 283MB prod-copy -> migrations clean, PASS,
image digest logged, no leftover temp dir/container. shellcheck + bash -n clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CI's migrations job only proves a migration is well-formed against a fresh,
empty DB. It never exercises the migration — or ErsatzTV's startup data steps
(DatabaseMigratorService -> DbInitializer + PopulatePathHashes over the real
MediaFile table) — against the accumulated prod SQLite, so a migration green on
a fresh DB can still fail/corrupt on prod, found only mid-deploy.
scripts/migration-smoke.sh rehearses it on a THROWAWAY copy of the latest prod
backup: boots the new image against the copy, gates PASS on the "Done applying
database migrations" log line (the migrator is a BackgroundService running
concurrently with Kestrel, so HTTP readiness alone doesn't prove migrations
finished), FAILs on early container exit / migration exception / timeout / not
serving afterwards. Always operates on a copy, never the live DB; tears down its
container + temp dir (incl. the container's root-owned config files) on exit.
Validated live 2026-07-12: :latest vs a copy of the 283MB prod backup ->
migrations applied cleanly, app booted+served, temp dir removed.
Home split: this repo owns the script + docs; wiring it into the Komodo
pre-deploy step is server-management#589 (cross-repo). Docs: docs/ci-cd.md
(Migration integrity), docs/decisions.md (new entry, pure insertion).
fixes#315
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scripts/issue-qualification-audit.sh: lists OPEN issues missing a `priority:` label
(the #237 ranking keys off priority:/gate labels, so an unlabeled issue is invisible
to it). "Fully qualified" = has a priority: label; gate-vs-backlog derives from the
review label / milestone, and a milestone is NOT required (backlog is unmilestoned).
Advisory (exit 1 if any unqualified); fail-open without Gitea creds. Wired into the
kickoff session-end protocol + a lore bullet. Tested live (flagged 2) + no-creds no-op.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Only run generate-endpoint-index.py if the GenerateOpenApiDocuments build
succeeded, so a failed build can't render docs/endpoint-index.md from a
stale/partial v1.json.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds docs/testing.md as the authoritative testing map (consolidated
from docs/contributing.md §8, now shrunk to a pointer), and a
generated docs/endpoint-index.md via scripts/generate-endpoint-index.py
(hooked into scripts/update-openapi.sh). Updates docs/README.md's
reading order and removes the "still to come" placeholder.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* generate openapi definitions as separate build task
* first defns
* install etv-client module in docker
* include python entrypoint in docker
* update changelog
timothyandClaude Opus 4.8