ersatztv/docs/decisions/records/testing/scripted-engine-in-process-net.md

key, title, status, since, supersedes, superseded-by, rule, signals, mechanics

key	title	status	since	supersedes	superseded-by	rule	signals	mechanics
testing.scripted-engine-in-process-net	2026-09-05 — Scripted playout is covered in-process, engine plus controller adapter; the Cli.Wrap process and the HTTP transport are permanently out of the automated suite (#563)	active	2026-09-05	testing.scripted-playout-golden-deferred@2026-07-22	none	Scripted playout is characterized at two in-process levels and no further. (1) `SchedulingEngineTests` drives the engine build API directly with substituted repositories — the same setup order `ScriptedPlayoutBuilder` uses (`WithPlayoutId`/`WithMode`/`WithSeed`/`BuildBetween`/`WithReferenceData`/`RestoreOrReset`, and `WithReferenceData` MUST precede the last two or `PlayoutReferenceData.PlayoutHistory` NREs). (2) `ScriptedScheduleControllerTests` replays a committed script fixture (`ErsatzTV.Tests/Controllers/Fixtures/scripted-build.json`) through the REAL `ScriptedScheduleController` + `ScriptedPlayoutBuilderService.MockSession` + `SchedulingEngine` and pins the resulting `PlayoutItem`s as a snapshot in the golden line format — so exactly ONE action-to-engine mapping exists in the repo, the production one. Fixture bodies in that replay are deserialized with `ApiJsonSettings`, the configuration `Startup` applies to `AddNewtonsoftJson`, invoked from the same function rather than mirrored — and two tests WITNESS that choice instead of asserting it: `Production_Body_Binder_Ignores_Required_Members` reddens under a System.Text.Json binder, `Production_Body_Binder_Keeps_Declared_Defaults_Over_An_Explicit_Null` reddens under a bare `JsonSerializerSettings`. What no test observes is the `Startup` registration itself, so the extraction REMOVES the duplicate rather than detecting its drift. The `Cli.Wrap` launch of the user-authored program (exit code, timeout, stdout capture), the Kestrel/middleware/`ApiAuthorizationFilter` transport it calls back over, and MVC model binding as a WRAPPER (the input formatter, model validation and the `[ApiController]` automatic 400 either produces before an action runs — the serializer inside that wrapper is covered, the wrapper around it is not) are NOT tested and are not scheduled to be; that residue is named below rather than argued away. The controller is a thin adapter, NOT a 1:1 pass-through: an unparseable playback order is a 400, an unparseable filler kind SILENTLY degrades to `FillerKind.None`, an unknown build id is a 404, and the engine no-progress `InvalidOperationException` is translated to a 400 — each pinned by a test, because a prose claim of equivalence decays.	why is there no scripted golden; scripted integration harness; scripted transport vs engine; SchedulingEngine is in-process testable; Cli.Wrap external process is transport only; shell-out scoped out; MockSession seam; in-process stand-in for a scripted script; engine build API characterization; committed script fixture replay; ScriptedScheduleController is an adapter not a pass-through; scripted filler kind silently falls back to None; scripted playout no-progress halt maps to 400; scripted build 401 blind spot; EnumeratorForContent · paths: `ErsatzTV.Core.Tests/Scheduling/Engine/SchedulingEngineTests.cs`, `ErsatzTV.Tests/Controllers/ScriptedScheduleControllerTests.cs`, `ErsatzTV.Tests/Controllers/Fixtures/scripted-build.json`, `ErsatzTV.Core/Scheduling/Engine/SchedulingEngine.cs`, `ErsatzTV/Controllers/Api/ScriptedScheduleController.cs`, `ErsatzTV.Core/Scheduling/ScriptedScheduling/ScriptedPlayoutBuilder.cs`, `ErsatzTV.Core.Tests/Scheduling/Goldens/PlayoutBuildGoldenTests.cs` · issues: #563, #381, #395, #163	docs/testing.md → "Scripted playout coverage" and "Golden-file nets"

What carries over from the predecessor. Sequential (YAML) is golden-able and TZ-independent, and PlayoutBuildGoldenTests covers Classic / Block / Sequential. Scripted's end-to-end pipeline stays out of that net: ScriptedPlayoutBuilder builds nothing itself, it Cli.Wrap-executes a user-authored program, hands it http://localhost:{Settings.UiPort} plus a build id, and reads the result straight off the in-process engine afterwards. #395's ContentEnumeratorBuilderTests remains the direct net over the enumerator-construction switch Scripted and Sequential share.

What changed. The predecessor deferred the whole harness to #563 and described the controller as a "1:1 pass-through". Both are now settled: the coverage exists in-process, and the pass-through wording was wrong — the four adapter mappings above are real behaviour that only a test can hold.

Why not the full Kestrel + Cli.Wrap harness. Four measured obstacles, not a preference:

1. The production transport is not "a controller behind Kestrel". Startup 404s any request whose Host is not localhost and whose Connection.LocalPort != Settings.UiPort (a mutable process-wide static), and ApiAuthorizationFilter fail-closes every mutating verb. A hand-rolled minimal host would test a transport the product does not have; the real Startup drags in the whole DI graph (TvContext, ffmpeg workers, search index, background services), which is what #381 deferred in the first place.
2. The external program is user-authored by definition. Playout.ScheduleFile is an arbitrary command line, so any script we commit is a stand-in we wrote either way. The fork's own driver, scripts/scripted-schedules/entrypoint.py, imports etv_client, a generated package that exists only in the deploy environment, so it cannot run in the .NET test job without adding a client-generation step.
3. It would put an interpreter and process spawning inside the pure in-memory .NET suite, with the 30s Cli.Wrap timeout nested inside CI's --blame-hang-timeout 2m — a new flake class on a shared runner.
4. The stand-in loses nothing the harness would have added at the engine level. Replaying the fixture through the real controller and the real service keeps a single action-to-engine mapping under test; what is absent is the two transport hops plus MVC binding as a wrapper, and the wrapper is the shallow half — the serializer inside it is configured from ApiJsonSettings, the production object itself.

The residue, named. Uncovered: process launch, exit code and timeout handling; Kestrel plus the forwarded-headers / compression / SPA-hosting middleware; MVC model binding as a wrapper — the input formatter (content-type selection, malformed-body handling), model validation (a non-nullable reference type picks up an implicit required check there) and the [ApiController] automatic 400 either produces before an action runs, since every test hands an action an already-bound object; and ApiAuthorizationFilter. The serializer inside that wrapper is deliberately not on this list: ScriptedScheduleControllerTests binds with ApiJsonSettings itself, and two tests hold the two ways that binder can be replaced (mutants run 2026-09-05, 9 tests in the fixture):

Replace the replay binder with	Result
System.Text.Json with web defaults	2 red — Newtonsoft has no notion of the C# required keyword, so a body omitting one deserializes to a default rather than being rejected
a bare new JsonSerializerSettings() — Newtonsoft, but without the production configuration	1 red — NullValueHandling.Ignore keeps ContentCollection.Order at its declared "shuffle" over an explicit "order": null, where Newtonsoft's own Include default writes the null through and AddCollection's Enum.TryParse then returns a 400

Both claims are about the SERIALIZER and are scoped to it: what the wrapper then does with such a body — non-nullable reference types pick up an implicit required check in MVC validation — is on the uncovered list above, not measured here.

What the extraction does and does not buy. ApiJsonSettings exists so the binder is defined once; it is not a drift detector, and the difference matters because the branch cannot claim one. Nothing in the suite observes Startup.ConfigureServices — re-inlining the AddNewtonsoftJson lambda as a hand-copy of Apply reddens no test, because a byte-equal mirror is behaviourally indistinguishable by construction. What the two suites do catch is a mirror that has lost something: the table above for the read path, and OpenApiSerializerContractTests for the write path, where dropping CustomContractResolver turns all 4 cases red on PascalCase keys (measured 2026-09-05). Drift confined to ReferenceLoopHandling or the StringEnumConverter is witnessed by neither, since neither suite serializes a cycle and OpenApiSerializerContractTests compares key names rather than values.

ApiAuthorizationFilter on that list is not hypothetical — ScriptedScheduleController carries no [SkipApiAuthorization] while ScriptedPlayoutBuilder passes the script only host, build id and mode, with no X-Api-Key, and entrypoint.py builds its client with no credential. Whether a scripted build therefore 401s on its first POST in this fork is unverified against a live instance and is tracked in ersatztv#913; it is recorded here as the concrete shape of what an engine-level net cannot see, so the scope-out is not read as a claim of equivalence.

Determinism constraints on the fixtures. Both use Chronological order and only instant-preserving instructions (AddCount, AddAll, AddDuration, PadUntilExact). WaitUntil(TimeOnly) and PadUntil(string) read the LOCAL day and time-of-day, and PadToNext becomes offset-sensitive once a call has localized CurrentTime, so introducing any of them — or a shuffle order — reopens the timezone and seed question the fixtures were built to close.