--- key: testing.scripted-engine-in-process-net title: '2026-09-05 — Scripted playout is covered in-process, engine plus controller adapter; the Cli.Wrap process and the HTTP transport are permanently out of the automated suite (#563)' status: active since: '2026-09-05' supersedes: testing.scripted-playout-golden-deferred@2026-07-22 superseded-by: none rule: 'Scripted playout is characterized at two in-process levels and no further. (1) `SchedulingEngineTests` drives the engine build API directly with substituted repositories — the same setup order `ScriptedPlayoutBuilder` uses (`WithPlayoutId`/`WithMode`/`WithSeed`/`BuildBetween`/`WithReferenceData`/`RestoreOrReset`, and `WithReferenceData` MUST precede the last two or `PlayoutReferenceData.PlayoutHistory` NREs). (2) `ScriptedScheduleControllerTests` replays a committed script fixture (`ErsatzTV.Tests/Controllers/Fixtures/scripted-build.json`) through the REAL `ScriptedScheduleController` + `ScriptedPlayoutBuilderService.MockSession` + `SchedulingEngine` and pins the resulting `PlayoutItem`s as a snapshot in the golden line format — so exactly ONE action-to-engine mapping exists in the repo, the production one. Fixture bodies in that replay are deserialized with `ApiJsonSettings`, the configuration `Startup` applies to `AddNewtonsoftJson`, invoked from the same function rather than mirrored — and two tests WITNESS that choice instead of asserting it, each binding a body that separates the production binder from one plausible replacement: `Production_Body_Binder_Ignores_Required_Members` omits a C# `required` member, which Newtonsoft defaults and System.Text.Json rejects; `Production_Body_Binder_Keeps_Declared_Defaults_Over_An_Explicit_Null` sends an explicit `"order": null`, which `NullValueHandling.Ignore` drops and a bare `JsonSerializerSettings` writes through into a 400. What is shared is the CONFIGURATION, never MVC''s settings OBJECT: `ApiJsonSettings.Create()` applies it to a bare settings object, so MVC''s `MaxDepth` of 32 and its `ProblemDetails`/`ValidationProblemDetails` converters are absent (measured 2026-09-05, pinned by `ApiJsonSettingsTests`); neither is reachable from a scripted request body, and no test may generalize from `Create()` to production past that. What no test observes is the `Startup` registration itself, so the extraction REMOVES the duplicate rather than detecting its drift. The `Cli.Wrap` launch of the user-authored program (exit code, timeout, stdout capture), the Kestrel/middleware/`ApiAuthorizationFilter` transport it calls back over, and MVC model binding as a WRAPPER (the input formatter, model validation and the `[ApiController]` automatic 400 either produces before an action runs — the serializer inside that wrapper is covered, the wrapper around it is not) are NOT tested and are not scheduled to be; that residue is named below rather than argued away. Both fixtures must aim every trimming instruction at an instant BETWEEN two content boundaries: a pad or duration that happens to land on one never enters the engine''s trim branch, so that action''s `trim` argument reaches nothing the snapshot can show. The controller is a thin adapter, NOT a 1:1 pass-through: an unparseable playback order is a 400, an unparseable filler kind SILENTLY degrades to `FillerKind.None`, an unknown build id is a 404, and the engine no-progress `InvalidOperationException` is translated to a 400 — each pinned by a test, because a prose claim of equivalence decays.' signals: 'why is there no scripted golden; scripted integration harness; scripted transport vs engine; SchedulingEngine is in-process testable; Cli.Wrap external process is transport only; shell-out scoped out; MockSession seam; in-process stand-in for a scripted script; engine build API characterization; committed script fixture replay; ScriptedScheduleController is an adapter not a pass-through; scripted filler kind silently falls back to None; scripted playout no-progress halt maps to 400; scripted build 401 blind spot; EnumeratorForContent · paths: `ErsatzTV.Core.Tests/Scheduling/Engine/SchedulingEngineTests.cs`, `ErsatzTV.Tests/Controllers/ScriptedScheduleControllerTests.cs`, `ErsatzTV.Tests/Controllers/Fixtures/scripted-build.json`, `ErsatzTV.Tests/Serialization/ApiJsonSettingsTests.cs`, `ErsatzTV/Serialization/ApiJsonSettings.cs`, `ErsatzTV.Core/Scheduling/Engine/SchedulingEngine.cs`, `ErsatzTV/Controllers/Api/ScriptedScheduleController.cs`, `ErsatzTV.Core/Scheduling/ScriptedScheduling/ScriptedPlayoutBuilder.cs`, `ErsatzTV.Core.Tests/Scheduling/Goldens/PlayoutBuildGoldenTests.cs` · issues: #563, #381, #395, #163' mechanics: docs/testing.md → "Scripted playout coverage" and "Golden-file nets" --- **What carries over from the predecessor.** Sequential (YAML) is golden-able and TZ-independent, and `PlayoutBuildGoldenTests` covers Classic / Block / Sequential. Scripted's *end-to-end pipeline* stays out of that net: `ScriptedPlayoutBuilder` builds nothing itself, it `Cli.Wrap`-executes a user-authored program, hands it `http://localhost:{Settings.UiPort}` plus a build id, and reads the result straight off the in-process engine afterwards. #395's `ContentEnumeratorBuilderTests` remains the direct net over the enumerator-construction switch Scripted and Sequential share. **What changed.** The predecessor deferred the whole harness to #563 and described the controller as a "1:1 pass-through". Both are now settled: the coverage exists in-process, and the pass-through wording was wrong — the four adapter mappings above are real behaviour that only a test can hold. **Why not the full Kestrel + `Cli.Wrap` harness.** Four measured obstacles, not a preference: 1. *The production transport is not "a controller behind Kestrel".* `Startup` 404s any request whose Host is not `localhost` and whose `Connection.LocalPort != Settings.UiPort` (a mutable process-wide static), and `ApiAuthorizationFilter` fail-closes every mutating verb. A hand-rolled minimal host would test a transport the product does not have; the real `Startup` drags in the whole DI graph (TvContext, ffmpeg workers, search index, background services), which is what #381 deferred in the first place. 2. *The external program is user-authored by definition.* `Playout.ScheduleFile` is an arbitrary command line, so any script we commit is a stand-in we wrote either way. The fork's own driver, `scripts/scripted-schedules/entrypoint.py`, imports `etv_client`, a generated package that exists only in the deploy environment, so it cannot run in the .NET test job without adding a client-generation step. 3. *It would put an interpreter and process spawning inside the pure in-memory .NET suite*, with the 30s `Cli.Wrap` timeout nested inside CI's `--blame-hang-timeout 2m` — a new flake class on a shared runner. 4. *The stand-in loses nothing the harness would have added at the engine level.* Replaying the fixture through the real controller and the real service keeps a single action-to-engine mapping under test; what is absent is the two transport hops plus MVC binding as a wrapper, and the wrapper is the shallow half — the serializer inside it is configured by `ApiJsonSettings`, the production configuration (not MVC's settings *object*, whose remaining defaults the rule enumerates). **The residue, named.** Uncovered: process launch, exit code and timeout handling; Kestrel plus the forwarded-headers / compression / SPA-hosting middleware; MVC model binding as a *wrapper* — the input formatter (content-type selection, malformed-body handling), model validation (a non-nullable reference type picks up an implicit required check there) and the `[ApiController]` automatic 400 either produces before an action runs, since every test hands an action an already-bound object; and `ApiAuthorizationFilter`. The *serializer* inside that wrapper is deliberately not on this list — it is `ApiJsonSettings` itself, and two tests hold the two ways it can be replaced. `ApiJsonSettings` exists so that binder is *defined* once; it is not a drift **detector**, and the branch may not claim one, because nothing observes `Startup.ConfigureServices` and a byte-equal hand-copy of `Apply` is indistinguishable by construction. What the suites do separate is a mirror that has *lost* something. The mechanics doc tabulates which suite separates which half of the configuration, and what neither reaches. **Where the calibrating measurements are, and why not here.** `testing.mutation-claims-are-executed` admits a sentence asserting that a specific mutation reddens — or does not redden — a named test only as a `CLAIMS` entry in `scripts/tests/mutation_manifest.py`, executed every run; otherwise it is not written. That harness resolves a proof to `scripts/tests/` and runs pytest, so an NUnit proof cannot be declared in it and this coverage is entirely NUnit. The binder and `trim` mutants run while building these tests therefore stay in ersatztv#563 — its pull request and the commit messages that ran them — dated. Durable prose carries the mechanism each test is built on instead — which serializer difference, which engine branch — because that is re-checkable by reading the code rather than by trusting a remembered outcome. `ApiAuthorizationFilter` on that list is not hypothetical — `ScriptedScheduleController` carries no `[SkipApiAuthorization]` while `ScriptedPlayoutBuilder` passes the script only host, build id and mode, with no `X-Api-Key`, and `entrypoint.py` builds its client with no credential. Whether a scripted build therefore 401s on its first POST in this fork is unverified against a live instance and is tracked in ersatztv#913; it is recorded here as the concrete shape of what an engine-level net cannot see, so the scope-out is not read as a claim of equivalence. **Determinism constraints on the fixtures.** Both use Chronological order and only instant-preserving instructions (`AddCount`, `AddAll`, `AddDuration`, `PadUntilExact`). `WaitUntil(TimeOnly)` and `PadUntil(string)` read the LOCAL day and time-of-day, and `PadToNext` becomes offset-sensitive once a call has localized `CurrentTime`, so introducing any of them — or a `shuffle` order — reopens the timezone and seed question the fixtures were built to close.