ersatztv/docs/decisions/records/testing/scripted-engine-in-process-net.md

key, title, status, since, supersedes, superseded-by, rule, signals, mechanics

key	title	status	since	supersedes	superseded-by	rule	signals	mechanics
testing.scripted-engine-in-process-net	2026-09-05 — Scripted playout is covered in-process, engine plus controller adapter; the Cli.Wrap process and the HTTP transport are permanently out of the automated suite (#563)	active	2026-09-05	testing.scripted-playout-golden-deferred@2026-07-22	none	Scripted playout is characterized at two in-process levels and no further. (1) `SchedulingEngineTests` drives the engine build API directly with substituted repositories — the same setup order `ScriptedPlayoutBuilder` uses (`WithPlayoutId`/`WithMode`/`WithSeed`/`BuildBetween`/`WithReferenceData`/`RestoreOrReset`, and `WithReferenceData` MUST precede the last two or `PlayoutReferenceData.PlayoutHistory` NREs). (2) `ScriptedScheduleControllerTests` replays a committed script fixture (`ErsatzTV.Tests/Controllers/Fixtures/scripted-build.json`) through the REAL `ScriptedScheduleController` + `ScriptedPlayoutBuilderService.MockSession` + `SchedulingEngine` and pins the resulting `PlayoutItem`s as a snapshot in the golden line format — so exactly ONE action-to-engine mapping exists in the repo, the production one. Fixture bodies in that replay are deserialized with `ApiJsonSettings`, the configuration `Startup` applies to `AddNewtonsoftJson`, invoked from the same function rather than mirrored — and two tests WITNESS that choice instead of asserting it, each binding a body that separates the production binder from one plausible replacement: `Production_Body_Binder_Ignores_Required_Members` omits a C# `required` member, which Newtonsoft defaults and System.Text.Json rejects; `Production_Body_Binder_Keeps_Declared_Defaults_Over_An_Explicit_Null` sends an explicit `"order": null`, which `NullValueHandling.Ignore` drops and a bare `JsonSerializerSettings` writes through into a 400. What is shared is the CONFIGURATION, never MVC's settings OBJECT: `ApiJsonSettings.Create()` applies it to a bare settings object, so MVC's `MaxDepth` of 32 and its `ProblemDetails`/`ValidationProblemDetails` converters are absent (measured 2026-09-05, pinned by `ApiJsonSettingsTests`); neither is reachable from a scripted request body, and no test may generalize from `Create()` to production past that. What no test observes is the `Startup` registration itself, so the extraction REMOVES the duplicate rather than detecting its drift. The `Cli.Wrap` launch of the user-authored program (exit code, timeout, stdout capture), the Kestrel/middleware/`ApiAuthorizationFilter` transport it calls back over, and MVC model binding as a WRAPPER (the input formatter, model validation and the `[ApiController]` automatic 400 either produces before an action runs — the serializer inside that wrapper is covered, the wrapper around it is not) are NOT tested and are not scheduled to be; that residue is named below rather than argued away. Both fixtures must aim every trimming instruction at an instant BETWEEN two content boundaries: a pad or duration that happens to land on one never enters the engine's trim branch, so that action's `trim` argument reaches nothing the snapshot can show. The controller is a thin adapter, NOT a 1:1 pass-through: an unparseable playback order is a 400, an unparseable filler kind SILENTLY degrades to `FillerKind.None`, an unknown build id is a 404, and the engine no-progress `InvalidOperationException` is translated to a 400 — each pinned by a test, because a prose claim of equivalence decays.	why is there no scripted golden; scripted integration harness; scripted transport vs engine; SchedulingEngine is in-process testable; Cli.Wrap external process is transport only; shell-out scoped out; MockSession seam; in-process stand-in for a scripted script; engine build API characterization; committed script fixture replay; ScriptedScheduleController is an adapter not a pass-through; scripted filler kind silently falls back to None; scripted playout no-progress halt maps to 400; scripted build 401 blind spot; EnumeratorForContent · paths: `ErsatzTV.Core.Tests/Scheduling/Engine/SchedulingEngineTests.cs`, `ErsatzTV.Tests/Controllers/ScriptedScheduleControllerTests.cs`, `ErsatzTV.Tests/Controllers/Fixtures/scripted-build.json`, `ErsatzTV.Tests/Serialization/ApiJsonSettingsTests.cs`, `ErsatzTV/Serialization/ApiJsonSettings.cs`, `ErsatzTV.Core/Scheduling/Engine/SchedulingEngine.cs`, `ErsatzTV/Controllers/Api/ScriptedScheduleController.cs`, `ErsatzTV.Core/Scheduling/ScriptedScheduling/ScriptedPlayoutBuilder.cs`, `ErsatzTV.Core.Tests/Scheduling/Goldens/PlayoutBuildGoldenTests.cs` · issues: #563, #381, #395, #163	docs/testing.md → "Scripted playout coverage" and "Golden-file nets"

What carries over from the predecessor. Sequential (YAML) is golden-able and TZ-independent, and PlayoutBuildGoldenTests covers Classic / Block / Sequential. Scripted's end-to-end pipeline stays out of that net: ScriptedPlayoutBuilder builds nothing itself, it Cli.Wrap-executes a user-authored program, hands it http://localhost:{Settings.UiPort} plus a build id, and reads the result straight off the in-process engine afterwards. #395's ContentEnumeratorBuilderTests remains the direct net over the enumerator-construction switch Scripted and Sequential share.

What changed. The predecessor deferred the whole harness to #563 and described the controller as a "1:1 pass-through". Both are now settled: the coverage exists in-process, and the pass-through wording was wrong — the four adapter mappings above are real behaviour that only a test can hold.

Why not the full Kestrel + Cli.Wrap harness. Four measured obstacles, not a preference:

1. The production transport is not "a controller behind Kestrel". Startup 404s any request whose Host is not localhost and whose Connection.LocalPort != Settings.UiPort (a mutable process-wide static), and ApiAuthorizationFilter fail-closes every mutating verb. A hand-rolled minimal host would test a transport the product does not have; the real Startup drags in the whole DI graph (TvContext, ffmpeg workers, search index, background services), which is what #381 deferred in the first place.
2. The external program is user-authored by definition. Playout.ScheduleFile is an arbitrary command line, so any script we commit is a stand-in we wrote either way. The fork's own driver, scripts/scripted-schedules/entrypoint.py, imports etv_client, a generated package that exists only in the deploy environment, so it cannot run in the .NET test job without adding a client-generation step.
3. It would put an interpreter and process spawning inside the pure in-memory .NET suite, with the 30s Cli.Wrap timeout nested inside CI's --blame-hang-timeout 2m — a new flake class on a shared runner.
4. The stand-in loses nothing the harness would have added at the engine level. Replaying the fixture through the real controller and the real service keeps a single action-to-engine mapping under test; what is absent is the two transport hops plus MVC binding as a wrapper, and the wrapper is the shallow half — the serializer inside it is configured by ApiJsonSettings, the production configuration (not MVC's settings object, whose remaining defaults the rule enumerates).

The residue, named. Uncovered: process launch, exit code and timeout handling; Kestrel plus the forwarded-headers / compression / SPA-hosting middleware; MVC model binding as a wrapper — the input formatter (content-type selection, malformed-body handling), model validation (a non-nullable reference type picks up an implicit required check there) and the [ApiController] automatic 400 either produces before an action runs, since every test hands an action an already-bound object; and ApiAuthorizationFilter. The serializer inside that wrapper is deliberately not on this list — it is ApiJsonSettings itself, and two tests hold the two ways it can be replaced. ApiJsonSettings exists so that binder is defined once; it is not a drift detector, and the branch may not claim one, because nothing observes Startup.ConfigureServices and a byte-equal hand-copy of Apply is indistinguishable by construction. What the suites do separate is a mirror that has lost something. The mechanics doc tabulates which suite separates which half of the configuration, and what neither reaches.

Where the calibrating measurements are, and why not here. testing.mutation-claims-are-executed admits a sentence asserting that a specific mutation reddens — or does not redden — a named test only as a CLAIMS entry in scripts/tests/mutation_manifest.py, executed every run; otherwise it is not written. That harness resolves a proof to scripts/tests/<node id> and runs pytest, so an NUnit proof cannot be declared in it and this coverage is entirely NUnit. The binder and trim mutants run while building these tests therefore stay in ersatztv#563 — its pull request and the commit messages that ran them — dated. Durable prose carries the mechanism each test is built on instead — which serializer difference, which engine branch — because that is re-checkable by reading the code rather than by trusting a remembered outcome.

ApiAuthorizationFilter on that list is not hypothetical — ScriptedScheduleController carries no [SkipApiAuthorization] while ScriptedPlayoutBuilder passes the script only host, build id and mode, with no X-Api-Key, and entrypoint.py builds its client with no credential. Whether a scripted build therefore 401s on its first POST in this fork is unverified against a live instance and is tracked in ersatztv#913; it is recorded here as the concrete shape of what an engine-level net cannot see, so the scope-out is not read as a claim of equivalence.

Determinism constraints on the fixtures. Both use Chronological order and only instant-preserving instructions (AddCount, AddAll, AddDuration, PadUntilExact). WaitUntil(TimeOnly) and PadUntil(string) read the LOCAL day and time-of-day, and PadToNext becomes offset-sensitive once a call has localized CurrentTime, so introducing any of them — or a shuffle order — reopens the timezone and seed question the fixtures were built to close.