Compare commits
19
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0add212941 | ||
|
|
4ba56848f6 | ||
|
|
f71e2fd0d0 | ||
|
|
0e64328ca2 | ||
|
|
2e4de81309 | ||
|
|
e04f75e9ec | ||
|
|
3f7febf656 | ||
|
|
f21dc121d3 | ||
|
|
bcf6f7e04d | ||
|
|
403bf011c1 | ||
|
|
00424bf090 | ||
|
|
8acba80a1a | ||
|
|
41d604f0ae | ||
|
|
e919b925f8 | ||
|
|
e61bbb63b5 | ||
|
|
e458630f48 | ||
|
|
bdd4ed7bad | ||
|
|
02aa05d75b | ||
|
|
1f7b763a7b |
@@ -176,7 +176,7 @@ docker compose -f $D/compose.yaml up -d --no-deps ersatztv-test
|
||||
|
||||
### Mutation Endpoints (POST)
|
||||
```bash
|
||||
# Library scan
|
||||
# Library scan (?deep=true also rescans items a media server reports unchanged)
|
||||
POST /api/v1/libraries/{id}/scan
|
||||
|
||||
# Scan single show
|
||||
@@ -472,6 +472,11 @@ docker start ersatztv
|
||||
### Common Mistakes (check every time)
|
||||
- **Playout not building**: Three things must all be correct: (1) `ProgramScheduleOneItem` row exists for the schedule item, (2) `PlaybackOrder=3` (Shuffle), (3) `ScheduleKind=1` on Playout. Missing any one results in 0 playout items — this is the most common issue.
|
||||
- **Collection queries fail**: `CollectionItem` has no `Id` column — use `rowid` for counting. Content lookup goes through `MediaVersion.MovieId` → `MediaFile.Path` (not a generic MediaItemId join).
|
||||
- **Channel won't start — "has no video stream on record"** (images before v26.16.0, which carries the ersatztv#924 fix, log a bare
|
||||
`Sequence contains no matching element` at `FFmpegStreamSelector.SelectVideoStream`): the playing item's
|
||||
`MediaVersion` has no `MediaStream` rows. Deep-scan its library — `POST /api/v1/libraries/{id}/scan?deep=true`;
|
||||
a regular scan skips items whose server etag is unchanged and cannot heal them. If the file itself has no video
|
||||
stream, no scan heals it. Jellyfin music videos scanned before that fix were all stored without streams (`scan.mediaserver-statistics-from-server`).
|
||||
- **Channel logos forgotten**: After creating a channel, add an Artwork row (ArtworkKind=2) + logo file, then run `fix_logos.py` to push to Jellyfin. Without this, the channel shows no logo in the EPG.
|
||||
- **Playout reset required**: After any schedule/collection change, run `POST /api/v1/channels/{id}/playout/reset`. Wait 5-10s for the playout to build before verifying item count.
|
||||
|
||||
|
||||
@@ -546,16 +546,19 @@ jobs:
|
||||
- name: Install test dependencies
|
||||
run: python3 -m pip install --disable-pip-version-check --quiet pytest pyyaml
|
||||
# jq gets its OWN step because its VERSION, not merely its presence, is load-bearing
|
||||
# (ersatztv#648). `--expect` makes this a TRIPWIRE: scripts/tests exercises the jq 1.6 code path
|
||||
# only because this runner ships 1.6, so an upgrade would silently delete that coverage — and
|
||||
# the three divergences found in ersatztv#643/#647 all lived exactly there. Going red forces an
|
||||
# explicit human decision instead of letting the coverage evaporate.
|
||||
# (ersatztv#648). `--expect` makes this a TRIPWIRE over an accepted SET of versions: the
|
||||
# set is exactly {1.6, 1.7} (ersatztv#935). Both `small` runners that took PR #933/#934 reported
|
||||
# 1.7 (2026-10-28), but both map to the floating runner-images:ubuntu-latest image with no digest
|
||||
# pin, so a host may hold an older pull and 1.6 stays in the set. Any other version goes red. No
|
||||
# known runner reports 1.6 today, so real-jq 1.6 coverage of scripts/tests is currently absent.
|
||||
# server-management#1080 proposes a fleet digest pin; when it settles one version, narrow this
|
||||
# to that version.
|
||||
#
|
||||
# The pin lives HERE and deliberately NOT in review-verdict.yml: that workflow writes the
|
||||
# branch-protection-required `review-verdict/h10` status, so pinning a version there would turn
|
||||
# any jq bump on the runner into a repo-wide merge deadlock. It gets the floor-only mode.
|
||||
# See docs/ci-cd.md -> "The jq contract".
|
||||
- name: Preflight jq version
|
||||
run: ./scripts/jq-preflight.sh --expect 1.6
|
||||
run: ./scripts/jq-preflight.sh --expect 1.6,1.7
|
||||
- name: Run scripts/tests
|
||||
run: PYTHONPATH=. python3 -m pytest scripts/tests -q
|
||||
|
||||
@@ -572,7 +572,7 @@ jobs:
|
||||
# verdict and leaves a FORGED one — and this job's own red status is not a required check.
|
||||
#
|
||||
# The empty case is checked EXPLICITLY, not left to jq's exit status: `jq -e` over empty input
|
||||
# exits 4 on jq >= 1.7 but 0 on jq 1.6, and THE RUNNER SHIPS 1.6 (ersatztv#647) — so on a
|
||||
# exits 4 on jq >= 1.7 but 0 on jq 1.6, and a runner may report 1.6 (the floor; ersatztv#647) — so on a
|
||||
# transient error this guard passed, the row came back "", and the job posted over a
|
||||
# possibly-existing human verdict.
|
||||
#
|
||||
@@ -1232,7 +1232,7 @@ jobs:
|
||||
# COUNTED, NOT `jq -e`, and that is this function's own standing rule rather than a
|
||||
# style preference. The `kind` read above says it in full: `jq -e` reports the
|
||||
# truthiness of its LAST OUTPUT, which is the exit-status subtlety that already bit
|
||||
# this workflow at jq 1.6 (ersatztv#647), and the runner still ships 1.6. A count read
|
||||
# this workflow at jq 1.6 (ersatztv#647), and a runner may still report 1.6. A count read
|
||||
# as a VALUE and then range-checked in shell cannot be read two ways, and it matches
|
||||
# every other tally here. (Using `jq -e` here would be safe by accident — the
|
||||
# array is known non-empty by the check above, so the empty-input
|
||||
|
||||
@@ -87,7 +87,7 @@
|
||||
<PackageVersion Include="Serilog.Sinks.Debug" Version="3.0.0" />
|
||||
<PackageVersion Include="Serilog.Sinks.File" Version="7.0.0" />
|
||||
<PackageVersion Include="Shouldly" Version="4.3.0" />
|
||||
<PackageVersion Include="SixLabors.ImageSharp" Version="4.1.2" />
|
||||
<PackageVersion Include="SixLabors.ImageSharp" Version="3.2.0" />
|
||||
<PackageVersion Include="SkiaSharp" Version="3.119.1" />
|
||||
<PackageVersion Include="SkiaSharp.NativeAssets.Linux.NoDependencies" Version="3.119.1" />
|
||||
<PackageVersion Include="SonarAnalyzer.CSharp" Version="10.27.0.140913" />
|
||||
|
||||
@@ -191,4 +191,65 @@ public class FFmpegStreamSelectorTests
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ersatztv#924: a version stored without a video stream (Jellyfin music videos whose statistics were never
|
||||
// read) used to surface First()'s bare "Sequence contains no matching element" from HLS session start.
|
||||
[TestFixture]
|
||||
public class SelectVideoStream
|
||||
{
|
||||
private static FFmpegStreamSelector Selector() =>
|
||||
new(
|
||||
new ScriptEngine(Substitute.For<ILogger<ScriptEngine>>()),
|
||||
Substitute.For<IStreamSelectorRepository>(),
|
||||
Substitute.For<IConfigElementRepository>(),
|
||||
new MockFileSystem(),
|
||||
Substitute.For<ILanguageCodeService>(),
|
||||
Substitute.For<ILogger<FFmpegStreamSelector>>());
|
||||
|
||||
[Test]
|
||||
public async Task Should_Select_The_Video_Stream()
|
||||
{
|
||||
var version = new MediaVersion
|
||||
{
|
||||
Id = 12,
|
||||
Streams =
|
||||
[
|
||||
new MediaStream { Index = 0, MediaStreamKind = MediaStreamKind.Audio },
|
||||
new MediaStream { Index = 1, MediaStreamKind = MediaStreamKind.Video, Codec = "h264" }
|
||||
]
|
||||
};
|
||||
|
||||
MediaStream stream = await Selector().SelectVideoStream(version);
|
||||
|
||||
stream.Index.ShouldBe(1);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task Should_Name_The_Version_And_File_When_There_Is_No_Video_Stream()
|
||||
{
|
||||
var version = new MediaVersion
|
||||
{
|
||||
Id = 1520,
|
||||
MediaFiles = [new MediaFile { Path = "/data/music/artist1/song1.mkv" }],
|
||||
Streams = [new MediaStream { Index = 0, MediaStreamKind = MediaStreamKind.Audio }]
|
||||
};
|
||||
|
||||
InvalidOperationException ex =
|
||||
await Should.ThrowAsync<InvalidOperationException>(() => Selector().SelectVideoStream(version));
|
||||
|
||||
ex.Message.ShouldContain("Media version 1520 (/data/music/artist1/song1.mkv) has no video stream");
|
||||
ex.Message.ShouldContain("deep scan");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task Should_Fail_Clearly_When_Streams_And_Files_Were_Never_Loaded()
|
||||
{
|
||||
var version = new MediaVersion { Id = 7, MediaFiles = null, Streams = null };
|
||||
|
||||
InvalidOperationException ex =
|
||||
await Should.ThrowAsync<InvalidOperationException>(() => Selector().SelectVideoStream(version));
|
||||
|
||||
ex.Message.ShouldContain("Media version 7 ((no media file)) has no video stream");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
using System.Collections.Immutable;
|
||||
using System.Collections.Immutable;
|
||||
using System.Diagnostics;
|
||||
using System.Diagnostics.CodeAnalysis;
|
||||
using System.Globalization;
|
||||
@@ -38,8 +38,31 @@ public class FFmpegStreamSelector : IFFmpegStreamSelector
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
public Task<MediaStream> SelectVideoStream(MediaVersion version) =>
|
||||
version.Streams.First(s => s.MediaStreamKind == MediaStreamKind.Video).AsTask();
|
||||
// ersatztv#924: a version with no video stream on record (its statistics were never read, or were stored
|
||||
// empty) cannot be played. Name the version and its file and say how the statistics get populated, instead
|
||||
// of surfacing First()'s bare "Sequence contains no matching element".
|
||||
public Task<MediaStream> SelectVideoStream(MediaVersion version)
|
||||
{
|
||||
foreach (MediaStream videoStream in Optional(version.Streams).Flatten()
|
||||
.Filter(s => s.MediaStreamKind == MediaStreamKind.Video)
|
||||
.HeadOrNone())
|
||||
{
|
||||
return videoStream.AsTask();
|
||||
}
|
||||
|
||||
string path = Optional(version.MediaFiles).Flatten()
|
||||
.HeadOrNone()
|
||||
.Map(f => f.Path)
|
||||
.IfNone("(no media file)");
|
||||
|
||||
return Task.FromException<MediaStream>(
|
||||
new InvalidOperationException(
|
||||
$"Media version {version.Id} ({path}) has no video stream on record, so it cannot be played. " +
|
||||
"If its media statistics were never read, rescan the item's library to read them (for a " +
|
||||
"Jellyfin/Emby/Plex library, a deep scan, since a regular scan skips items the server reports " +
|
||||
"unchanged). If the file itself has no video stream, a rescan cannot help: remove it from " +
|
||||
"video channels."));
|
||||
}
|
||||
|
||||
public async Task<Option<MediaStream>> SelectAudioStream(
|
||||
MediaItemAudioVersion version,
|
||||
@@ -115,7 +138,7 @@ public class FFmpegStreamSelector : IFFmpegStreamSelector
|
||||
}
|
||||
|
||||
break;
|
||||
// let default fall through
|
||||
// let default fall through
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
|
||||
@@ -19,6 +19,7 @@ public class JellyfinMusicVideoLibraryScanner :
|
||||
{
|
||||
private readonly IJellyfinApiClient _jellyfinApiClient;
|
||||
private readonly IJellyfinMusicVideoRepository _jellyfinMusicVideoRepository;
|
||||
private readonly ILogger<JellyfinMusicVideoLibraryScanner> _logger;
|
||||
private readonly IMediaSourceRepository _mediaSourceRepository;
|
||||
private readonly IMetadataRepository _metadataRepository;
|
||||
private readonly IMusicVideoRepository _musicVideoRepository;
|
||||
@@ -50,6 +51,7 @@ public class JellyfinMusicVideoLibraryScanner :
|
||||
_mediaSourceRepository = mediaSourceRepository;
|
||||
_musicVideoRepository = musicVideoRepository;
|
||||
_metadataRepository = metadataRepository;
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
public async Task<Either<BaseError, Unit>> ScanLibrary(
|
||||
@@ -97,6 +99,41 @@ public class JellyfinMusicVideoLibraryScanner :
|
||||
library,
|
||||
projectionFailures);
|
||||
|
||||
protected override async Task<Option<MediaVersion>> GetMediaServerStatistics(
|
||||
JellyfinConnectionParameters connectionParameters,
|
||||
JellyfinLibrary library,
|
||||
MediaItemScanResult<JellyfinMusicVideo> result,
|
||||
JellyfinMusicVideo incoming)
|
||||
{
|
||||
_logger.LogDebug("Refreshing {Attribute} for {Path}", "Jellyfin Statistics", result.LocalPath);
|
||||
|
||||
Either<BaseError, MediaVersion> maybeVersion =
|
||||
await _jellyfinApiClient.GetPlaybackInfo(
|
||||
connectionParameters.Address,
|
||||
connectionParameters.AuthorizationHeader,
|
||||
library,
|
||||
incoming.ItemId);
|
||||
|
||||
foreach (BaseError error in maybeVersion.LeftToSeq())
|
||||
{
|
||||
_logger.LogWarning(
|
||||
"Failed to get music video statistics from Jellyfin for {Path}: {Error}",
|
||||
result.LocalPath,
|
||||
error.ToString());
|
||||
}
|
||||
|
||||
// Chapters arrive with the library-items projection, not with playback info. They are taken from the
|
||||
// INCOMING projection, not from result.Item as JellyfinMovieLibraryScanner does: the music-video
|
||||
// repository's GetByItemId does not eager-load chapters, so on an existing row result.Item's head version
|
||||
// carries none to write.
|
||||
foreach (MediaVersion version in maybeVersion.RightToSeq())
|
||||
{
|
||||
version.Chapters = incoming.GetHeadVersion().Chapters ?? [];
|
||||
}
|
||||
|
||||
return maybeVersion.ToOption();
|
||||
}
|
||||
|
||||
protected override async Task<Either<BaseError, MediaItemScanResult<JellyfinMusicVideo>>> UpdateMetadata(
|
||||
MediaItemScanResult<JellyfinMusicVideo> result,
|
||||
JellyfinMusicVideo incoming,
|
||||
|
||||
@@ -131,6 +131,7 @@ public abstract class MediaServerMusicVideoLibraryScanner<TConnectionParameters,
|
||||
Either<BaseError, MediaItemScanResult<TMusicVideo>> maybeMusicVideo =
|
||||
await ProcessMusicVideo(
|
||||
musicVideoRepository,
|
||||
connectionParameters,
|
||||
library,
|
||||
libraryPath,
|
||||
incoming,
|
||||
@@ -250,6 +251,7 @@ public abstract class MediaServerMusicVideoLibraryScanner<TConnectionParameters,
|
||||
|
||||
private async Task<Either<BaseError, MediaItemScanResult<TMusicVideo>>> ProcessMusicVideo(
|
||||
IMediaServerMusicVideoRepository<TLibrary, TMusicVideo, TEtag> musicVideoRepository,
|
||||
TConnectionParameters connectionParameters,
|
||||
TLibrary library,
|
||||
LibraryPath libraryPath,
|
||||
TMusicVideo incoming,
|
||||
@@ -279,7 +281,7 @@ public abstract class MediaServerMusicVideoLibraryScanner<TConnectionParameters,
|
||||
return result;
|
||||
})
|
||||
.BindT(result => UpdateMetadata(result, incoming, cancellationToken))
|
||||
.BindT(result => UpdateStatistics(result, incoming));
|
||||
.BindT(result => UpdateStatistics(connectionParameters, library, result, incoming, deepScan));
|
||||
}
|
||||
|
||||
private async Task<Either<BaseError, Artist>> GetOrAddArtist(LibraryPath libraryPath, TMusicVideo musicVideo)
|
||||
@@ -315,13 +317,40 @@ public abstract class MediaServerMusicVideoLibraryScanner<TConnectionParameters,
|
||||
return result.Map(scanResult => scanResult.Item);
|
||||
}
|
||||
|
||||
// ersatztv#924: the library-items projection carries NO streams and no width/height (the server only reports
|
||||
// those through its per-item playback info), so the projection's version must never be written as the
|
||||
// statistics — doing so stored a version FFmpegStreamSelector cannot play, and, because
|
||||
// IMetadataRepository.UpdateStatistics reconciles streams against the version it is given, removed every
|
||||
// stream an existing row already had. Statistics come from the media server, under the same condition
|
||||
// MediaServerMovieLibraryScanner.UpdateStatistics uses; when the server cannot supply them nothing is written,
|
||||
// so stored streams are kept rather than replaced by the projection's empty list. (`result.IsAdded` is implied
|
||||
// by the empty-streams arm while the projection carries no streams; it is kept so the condition stays the
|
||||
// movie scanner's.)
|
||||
private async Task<Either<BaseError, MediaItemScanResult<TMusicVideo>>> UpdateStatistics(
|
||||
TConnectionParameters connectionParameters,
|
||||
TLibrary library,
|
||||
MediaItemScanResult<TMusicVideo> result,
|
||||
TMusicVideo incoming)
|
||||
TMusicVideo incoming,
|
||||
bool deepScan)
|
||||
{
|
||||
if (await _metadataRepository.UpdateStatistics(result.Item, incoming.GetHeadVersion()))
|
||||
TMusicVideo existing = result.Item;
|
||||
|
||||
if (deepScan || result.IsAdded || MediaServerEtag(existing) != MediaServerEtag(incoming) ||
|
||||
existing.GetHeadVersion().Streams is not { Count: > 0 })
|
||||
{
|
||||
result.IsUpdated = true;
|
||||
Option<MediaVersion> maybeMediaVersion = await GetMediaServerStatistics(
|
||||
connectionParameters,
|
||||
library,
|
||||
result,
|
||||
incoming);
|
||||
|
||||
foreach (MediaVersion mediaVersion in maybeMediaVersion)
|
||||
{
|
||||
if (await _metadataRepository.UpdateStatistics(result.Item, mediaVersion))
|
||||
{
|
||||
result.IsUpdated = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
@@ -405,4 +434,13 @@ public abstract class MediaServerMusicVideoLibraryScanner<TConnectionParameters,
|
||||
MediaItemScanResult<TMusicVideo> result,
|
||||
TMusicVideo incoming,
|
||||
CancellationToken cancellationToken);
|
||||
|
||||
// ersatztv#924: abstract rather than the movie scanner's virtual-returning-None, because a None default is
|
||||
// exactly the defect — an implementation that forgot to override it would scan music videos with no streams.
|
||||
// The returned version carries the chapters to store alongside the statistics.
|
||||
protected abstract Task<Option<MediaVersion>> GetMediaServerStatistics(
|
||||
TConnectionParameters connectionParameters,
|
||||
TLibrary library,
|
||||
MediaItemScanResult<TMusicVideo> result,
|
||||
TMusicVideo incoming);
|
||||
}
|
||||
|
||||
@@ -726,6 +726,285 @@ public class JellyfinMusicVideoLibraryScannerTests
|
||||
metadata.Artists.Count.ShouldBe(1);
|
||||
}
|
||||
|
||||
// ersatztv#924: Jellyfin's library-items projection carries no streams and no width/height — those come only
|
||||
// from the per-item playback info. These run against the REAL MetadataRepository so the assertions read what
|
||||
// was persisted. Non-vacuous (measured 2026-10-21): restoring the scanner's former body,
|
||||
// `UpdateStatistics(result.Item, incoming.GetHeadVersion())`, reddens all five tests below.
|
||||
[Test]
|
||||
public async Task ScanLibrary_Should_Store_PlaybackInfo_Statistics_For_A_New_MusicVideo()
|
||||
{
|
||||
int libraryPathId = await SeedLibraryPath("/data/music");
|
||||
const string VideoPath = "/data/music/artist1/song1.mkv";
|
||||
|
||||
JellyfinMusicVideo Incoming()
|
||||
{
|
||||
JellyfinMusicVideo musicVideo = BuildIncoming(VideoPath, "Artist 1", "Song 1");
|
||||
musicVideo.MediaVersions[0].Chapters = [Chapter(0, "Intro")];
|
||||
return musicVideo;
|
||||
}
|
||||
|
||||
var playbackCalls = new List<string>();
|
||||
IJellyfinApiClient api = FakeApiWithPlaybackInfo(
|
||||
itemId =>
|
||||
{
|
||||
playbackCalls.Add(itemId);
|
||||
return PlaybackInfo(1920, 1080);
|
||||
},
|
||||
Incoming);
|
||||
|
||||
(JellyfinMusicVideoLibraryScanner scanner, _) =
|
||||
BuildScanner(api, metadataRepository: new MetadataRepository(_db.Factory));
|
||||
|
||||
(await scanner.ScanLibrary(
|
||||
ConnectionParameters,
|
||||
BuildLibrary(libraryPathId, "/data/music"),
|
||||
deepScan: false,
|
||||
CancellationToken.None)).IsRight.ShouldBeTrue();
|
||||
|
||||
playbackCalls.ShouldBe([ItemIdFor(VideoPath)]);
|
||||
|
||||
MediaVersion version = await HeadVersionOf(VideoPath);
|
||||
version.Streams.Count(s => s.MediaStreamKind == MediaStreamKind.Video).ShouldBe(1);
|
||||
version.Streams.Count(s => s.MediaStreamKind == MediaStreamKind.Audio).ShouldBe(1);
|
||||
version.Width.ShouldBe(1920);
|
||||
version.Height.ShouldBe(1080);
|
||||
|
||||
// the projection's chapters are stored alongside the statistics, not dropped by them
|
||||
version.Chapters.Select(c => c.Title).ShouldBe(["Intro"]);
|
||||
}
|
||||
|
||||
// The remediation path for rows already stored without streams: a regular scan skips an item whose etag is
|
||||
// unchanged (so it cannot heal them), and a deep scan re-reads the statistics.
|
||||
[Test]
|
||||
public async Task ScanLibrary_Deep_Scan_Should_Read_Statistics_For_An_Unchanged_MusicVideo_Without_Streams()
|
||||
{
|
||||
int libraryPathId = await SeedLibraryPath("/data/music");
|
||||
JellyfinLibrary library = BuildLibrary(libraryPathId, "/data/music");
|
||||
const string VideoPath = "/data/music/artist1/song1.mkv";
|
||||
|
||||
var playbackCalls = 0;
|
||||
var serverAnswers = false;
|
||||
(int Width, int Height) size = (1280, 720);
|
||||
IJellyfinApiClient api = FakeApiWithPlaybackInfo(
|
||||
_ =>
|
||||
{
|
||||
playbackCalls++;
|
||||
return serverAnswers
|
||||
? PlaybackInfo(size.Width, size.Height)
|
||||
: BaseError.New("playback info unavailable");
|
||||
},
|
||||
() => BuildIncoming(VideoPath, "Artist 1", "Song 1"));
|
||||
|
||||
(JellyfinMusicVideoLibraryScanner scanner, _) =
|
||||
BuildScanner(api, metadataRepository: new MetadataRepository(_db.Factory));
|
||||
|
||||
// 1. the item is added while the server cannot supply statistics: stored with no streams
|
||||
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
|
||||
.IsRight.ShouldBeTrue();
|
||||
(await HeadVersionOf(VideoPath)).Streams.ShouldBeEmpty();
|
||||
playbackCalls.ShouldBe(1);
|
||||
|
||||
// 2. a regular scan of the unchanged item does not look at it again
|
||||
serverAnswers = true;
|
||||
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
|
||||
.IsRight.ShouldBeTrue();
|
||||
playbackCalls.ShouldBe(1);
|
||||
(await HeadVersionOf(VideoPath)).Streams.ShouldBeEmpty();
|
||||
|
||||
// 3. a deep scan reads the statistics
|
||||
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: true, CancellationToken.None))
|
||||
.IsRight.ShouldBeTrue();
|
||||
playbackCalls.ShouldBe(2);
|
||||
MediaVersion version = await HeadVersionOf(VideoPath);
|
||||
version.Streams.Count(s => s.MediaStreamKind == MediaStreamKind.Video).ShouldBe(1);
|
||||
version.Width.ShouldBe(1280);
|
||||
version.Height.ShouldBe(720);
|
||||
|
||||
// 4. a deep scan re-reads them even when streams are already stored and the etag is unchanged
|
||||
size = (1920, 1080);
|
||||
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: true, CancellationToken.None))
|
||||
.IsRight.ShouldBeTrue();
|
||||
playbackCalls.ShouldBe(3);
|
||||
(await HeadVersionOf(VideoPath)).Width.ShouldBe(1920);
|
||||
}
|
||||
|
||||
// The "no stored streams" arm of the refresh condition: an Unavailable item whose file is back is scanned with
|
||||
// an UNCHANGED etag and no deep scan, so only the empty stream list says its statistics must be read.
|
||||
[Test]
|
||||
public async Task ScanLibrary_Should_Read_Statistics_For_A_Returning_Unavailable_MusicVideo_Without_Streams()
|
||||
{
|
||||
int libraryPathId = await SeedLibraryPath("/data/music");
|
||||
JellyfinLibrary library = BuildLibrary(libraryPathId, "/data/music");
|
||||
const string VideoPath = "/data/music/artist1/song1.mkv";
|
||||
|
||||
var serverAnswers = false;
|
||||
IJellyfinApiClient api = FakeApiWithPlaybackInfo(
|
||||
_ => serverAnswers ? PlaybackInfo(1280, 720) : BaseError.New("playback info unavailable"),
|
||||
() => BuildIncoming(VideoPath, "Artist 1", "Song 1"));
|
||||
|
||||
(JellyfinMusicVideoLibraryScanner scanner, _) =
|
||||
BuildScanner(api, metadataRepository: new MetadataRepository(_db.Factory));
|
||||
|
||||
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
|
||||
.IsRight.ShouldBeTrue();
|
||||
|
||||
int musicVideoId = await MusicVideoId(VideoPath);
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
MediaItem mediaItem = await context.MediaItems.SingleAsync(mi => mi.Id == musicVideoId);
|
||||
mediaItem.State = MediaItemState.Unavailable;
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
serverAnswers = true;
|
||||
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
|
||||
.IsRight.ShouldBeTrue();
|
||||
|
||||
MediaVersion version = await HeadVersionOf(VideoPath);
|
||||
version.Streams.Count(s => s.MediaStreamKind == MediaStreamKind.Video).ShouldBe(1);
|
||||
version.Width.ShouldBe(1280);
|
||||
(await MediaItemStateOf(musicVideoId)).ShouldBe(MediaItemState.Normal);
|
||||
}
|
||||
|
||||
// When the server cannot supply statistics on a rescan, nothing is written: the projection's empty stream
|
||||
// list must not replace the streams already stored (IMetadataRepository.UpdateStatistics removes every
|
||||
// stored stream the version it is given does not carry).
|
||||
[Test]
|
||||
public async Task ScanLibrary_Should_Keep_Stored_Statistics_When_PlaybackInfo_Fails_On_Rescan()
|
||||
{
|
||||
int libraryPathId = await SeedLibraryPath("/data/music");
|
||||
JellyfinLibrary library = BuildLibrary(libraryPathId, "/data/music");
|
||||
const string VideoPath = "/data/music/artist1/song1.mkv";
|
||||
|
||||
var title = "Song 1";
|
||||
IJellyfinApiClient api = FakeApiWithPlaybackInfo(
|
||||
_ => BaseError.New("playback info unavailable"),
|
||||
() => BuildIncoming(VideoPath, "Artist 1", title));
|
||||
|
||||
(JellyfinMusicVideoLibraryScanner scanner, _) =
|
||||
BuildScanner(api, metadataRepository: new MetadataRepository(_db.Factory));
|
||||
|
||||
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
|
||||
.IsRight.ShouldBeTrue();
|
||||
|
||||
// the row holds statistics from an earlier successful read
|
||||
int versionId = (await HeadVersionOf(VideoPath)).Id;
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
MediaVersion stored = await context.MediaVersions.SingleAsync(v => v.Id == versionId);
|
||||
stored.Width = 1920;
|
||||
stored.Height = 1080;
|
||||
await context.MediaStreams.AddAsync(
|
||||
new MediaStream
|
||||
{
|
||||
MediaVersionId = versionId,
|
||||
Index = 0,
|
||||
MediaStreamKind = MediaStreamKind.Video,
|
||||
Codec = "h264"
|
||||
});
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
// the server reports the item changed (BuildIncoming derives the etag from the title), but playback
|
||||
// info fails
|
||||
title = "Song 1 (Remastered)";
|
||||
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
|
||||
.IsRight.ShouldBeTrue();
|
||||
|
||||
MediaVersion version = await HeadVersionOf(VideoPath);
|
||||
version.Streams.Select(s => (s.Index, s.MediaStreamKind, s.Codec))
|
||||
.ShouldBe([(0, MediaStreamKind.Video, "h264")]);
|
||||
version.Width.ShouldBe(1920);
|
||||
version.Height.ShouldBe(1080);
|
||||
}
|
||||
|
||||
// Refreshed statistics take their chapters from the INCOMING projection: the repository does not eager-load
|
||||
// chapters on an existing row, so JellyfinMovieLibraryScanner's source (the stored item) would have none to
|
||||
// write.
|
||||
[Test]
|
||||
public async Task ScanLibrary_Should_Refresh_Statistics_And_Projection_Chapters_When_The_Etag_Changes()
|
||||
{
|
||||
int libraryPathId = await SeedLibraryPath("/data/music");
|
||||
JellyfinLibrary library = BuildLibrary(libraryPathId, "/data/music");
|
||||
const string VideoPath = "/data/music/artist1/song1.mkv";
|
||||
|
||||
var title = "Song 1";
|
||||
List<MediaChapter> chapters = [Chapter(0, "Intro")];
|
||||
(int Width, int Height) size = (1280, 720);
|
||||
|
||||
JellyfinMusicVideo Incoming()
|
||||
{
|
||||
JellyfinMusicVideo musicVideo = BuildIncoming(VideoPath, "Artist 1", title);
|
||||
musicVideo.MediaVersions[0].Chapters = chapters.Select(c => Chapter(c.ChapterId, c.Title)).ToList();
|
||||
return musicVideo;
|
||||
}
|
||||
|
||||
IJellyfinApiClient api = FakeApiWithPlaybackInfo(_ => PlaybackInfo(size.Width, size.Height), Incoming);
|
||||
(JellyfinMusicVideoLibraryScanner scanner, _) =
|
||||
BuildScanner(api, metadataRepository: new MetadataRepository(_db.Factory));
|
||||
|
||||
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
|
||||
.IsRight.ShouldBeTrue();
|
||||
|
||||
title = "Song 1 (Remastered)";
|
||||
chapters = [Chapter(0, "Intro"), Chapter(1, "Chorus")];
|
||||
size = (1920, 1080);
|
||||
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
|
||||
.IsRight.ShouldBeTrue();
|
||||
|
||||
MediaVersion version = await HeadVersionOf(VideoPath);
|
||||
version.Width.ShouldBe(1920);
|
||||
version.Height.ShouldBe(1080);
|
||||
version.Streams.Count(s => s.MediaStreamKind == MediaStreamKind.Video).ShouldBe(1);
|
||||
version.Chapters.OrderBy(c => c.ChapterId).Select(c => c.Title).ShouldBe(["Intro", "Chorus"]);
|
||||
}
|
||||
|
||||
private static MediaChapter Chapter(long chapterId, string title) =>
|
||||
new()
|
||||
{
|
||||
ChapterId = chapterId,
|
||||
Title = title,
|
||||
StartTime = TimeSpan.FromSeconds(chapterId * 30),
|
||||
EndTime = TimeSpan.FromSeconds((chapterId + 1) * 30)
|
||||
};
|
||||
|
||||
// What JellyfinApiClient.GetPlaybackInfo returns for a playable file: a version carrying the streams and the
|
||||
// dimensions. Built fresh per call, since UpdateStatistics attaches the stream objects to the stored version.
|
||||
private static Either<BaseError, MediaVersion> PlaybackInfo(int width, int height) =>
|
||||
new MediaVersion
|
||||
{
|
||||
Duration = TimeSpan.FromMinutes(4),
|
||||
Width = width,
|
||||
Height = height,
|
||||
SampleAspectRatio = "1:1",
|
||||
DisplayAspectRatio = "16:9",
|
||||
RFrameRate = "23.976",
|
||||
VideoScanKind = VideoScanKind.Progressive,
|
||||
Streams =
|
||||
[
|
||||
new MediaStream { Index = 0, MediaStreamKind = MediaStreamKind.Video, Codec = "h264" },
|
||||
new MediaStream { Index = 1, MediaStreamKind = MediaStreamKind.Audio, Codec = "aac", Channels = 2 }
|
||||
],
|
||||
Chapters = []
|
||||
};
|
||||
|
||||
private async Task<MediaVersion> HeadVersionOf(string path)
|
||||
{
|
||||
await using TvContext context = _db.CreateContext();
|
||||
List<MusicVideo> musicVideos = await context.MusicVideos
|
||||
.Include(mv => mv.MediaVersions)
|
||||
.ThenInclude(v => v.MediaFiles)
|
||||
.Include(mv => mv.MediaVersions)
|
||||
.ThenInclude(v => v.Streams)
|
||||
.Include(mv => mv.MediaVersions)
|
||||
.ThenInclude(v => v.Chapters)
|
||||
.ToListAsync();
|
||||
return musicVideos
|
||||
.Single(mv => mv.MediaVersions.Single().MediaFiles.Single().Path == path)
|
||||
.MediaVersions
|
||||
.Single();
|
||||
}
|
||||
|
||||
private static JellyfinConnectionParameters ConnectionParameters =>
|
||||
new("http://jellyfin", "api-key", 1);
|
||||
|
||||
@@ -810,7 +1089,8 @@ public class JellyfinMusicVideoLibraryScannerTests
|
||||
|
||||
private (JellyfinMusicVideoLibraryScanner Scanner, IScannerProxy ScannerProxy) BuildScanner(
|
||||
IJellyfinApiClient apiClient,
|
||||
Func<string, string> replacePath = null)
|
||||
Func<string, string> replacePath = null,
|
||||
IMetadataRepository metadataRepository = null)
|
||||
{
|
||||
IScannerProxy scannerProxy = BuildScannerProxy();
|
||||
|
||||
@@ -823,7 +1103,7 @@ public class JellyfinMusicVideoLibraryScannerTests
|
||||
new ArtistRepository(_db.Factory),
|
||||
new MusicVideoRepository(_db.Factory),
|
||||
new LibraryRepository(Substitute.For<IFileSystem>(), _db.Factory),
|
||||
Substitute.For<IMetadataRepository>(),
|
||||
metadataRepository ?? Substitute.For<IMetadataRepository>(),
|
||||
BuildFileSystem(),
|
||||
NullLogger<JellyfinMusicVideoLibraryScanner>.Instance);
|
||||
|
||||
@@ -912,7 +1192,14 @@ public class JellyfinMusicVideoLibraryScannerTests
|
||||
|
||||
// Each Func builds a fresh music video so the async stream can be re-enumerated across ScanLibrary calls
|
||||
// (an IAsyncEnumerable iterator is single-use, and the scanner mutates the incoming item).
|
||||
private static IJellyfinApiClient FakeApi(params Func<JellyfinMusicVideo>[] items)
|
||||
// Playback info succeeds by default (ersatztv#924: the scanner reads statistics from it for every scanned item).
|
||||
private static IJellyfinApiClient FakeApi(params Func<JellyfinMusicVideo>[] items) =>
|
||||
FakeApiWithPlaybackInfo(_ => PlaybackInfo(1920, 1080), items);
|
||||
|
||||
// `playbackInfo` answers GetPlaybackInfo per item id, standing in for the server's per-item playback info.
|
||||
private static IJellyfinApiClient FakeApiWithPlaybackInfo(
|
||||
Func<string, Either<BaseError, MediaVersion>> playbackInfo,
|
||||
params Func<JellyfinMusicVideo>[] items)
|
||||
{
|
||||
var apiClient = Substitute.For<IJellyfinApiClient>();
|
||||
apiClient.GetMusicVideoLibraryItems(
|
||||
@@ -921,9 +1208,16 @@ public class JellyfinMusicVideoLibraryScannerTests
|
||||
Arg.Any<JellyfinLibrary>(),
|
||||
Arg.Any<MediaServerProjectionFailureCounter>())
|
||||
.Returns(_ => Items(items));
|
||||
StubPlaybackInfo(apiClient, playbackInfo);
|
||||
return apiClient;
|
||||
}
|
||||
|
||||
private static void StubPlaybackInfo(
|
||||
IJellyfinApiClient apiClient,
|
||||
Func<string, Either<BaseError, MediaVersion>> playbackInfo) =>
|
||||
apiClient.GetPlaybackInfo(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<JellyfinLibrary>(), Arg.Any<string>())
|
||||
.Returns(ci => Task.FromResult(playbackInfo(ci.ArgAt<string>(3))));
|
||||
|
||||
// Same as FakeApi, but the enumeration also reports `failures` swallowed projection exceptions into the
|
||||
// counter the scanner handed in — what the real JellyfinApiClient does from ProjectToMusicVideo's catch.
|
||||
private static IJellyfinApiClient FakeApiWithProjectionFailures(
|
||||
@@ -940,6 +1234,7 @@ public class JellyfinMusicVideoLibraryScannerTests
|
||||
items,
|
||||
ci.ArgAt<MediaServerProjectionFailureCounter>(3),
|
||||
failures));
|
||||
StubPlaybackInfo(apiClient, _ => PlaybackInfo(1920, 1080));
|
||||
return apiClient;
|
||||
}
|
||||
|
||||
|
||||
+72
-40
@@ -9,7 +9,7 @@ server-management#481.
|
||||
|
||||
| Host | IP | Role |
|
||||
|---|---|---|
|
||||
| **jazz** | 192.168.1.29 | **Docker host for the media transcoders** — prod `ersatztv` (8409), `ersatztv-test` (8410), Jellyfin. Release scans (`security-scan.sh`) and the prod-copy `migration-smoke.sh` run here. |
|
||||
| **jazz** | 192.168.1.29 | **Docker host for the media transcoders** — prod `ersatztv` (8409), `ersatztv-test` (8410), Jellyfin. The prod-copy `migration-smoke.sh` runs here. The release scan (`security-scan.sh`) boots its own throwaway target from the registry, so it runs on any docker host that meets its requirements (Security scanning → Where/when). |
|
||||
| **bumblebee** | 192.168.1.99 | CI runners (`bumblebee-runner`, `small-runner`), plus every other Docker stack. All the memory/lane measurements below were taken here. |
|
||||
| **ci-runner** | VM 127 (pve4) | The other `ubuntu-latest` CI runner. |
|
||||
|
||||
@@ -42,6 +42,7 @@ Upstream's final release was **`v26.3.0`** (archived). Our line continues from t
|
||||
| `v26.13.0` | **RuleBuilder maturation** — arbitrary-depth group nesting (#436), inline smart-query authoring in Channel Builder (#437), DB-sourced facet typeahead + relative-date operators + validation (#434/#435/#438), and an artist typeahead covering music-video/song credits with `album_artist` no longer 404ing (#578). **Per-channel On Now/Next transient overlay** (#74/#570) and **per-schedule clock-boundary padding** (#392); in-browser channel preview (#60); Auto-Tune per-source weight steppers + exclude/add-untagged (#440). Library-browse pickers now resolve by search instead of a 100-row window, closing several silent at-cap truncations (#644/#650/#651/#634). Correctness: one watermark resolver for all four attachment points, incl. `MiddleCenter` (#503/#510); QSV HDR tonemaps through OpenCL because `vpp_qsv=tonemap` is a silent no-op (#505); `LibraryFolder` unique index + concurrent-insert tolerance (#491); per-library music-video identity with soft trash (#496); Jellyfin Album/Track music-video projection (#177); metadata-collection dedup (#500); accented facet values via a registered Unicode fold on SQLite (#668); `WorkAheadSlots` atomic slot claim, never a negative count (#536/#539); on-demand guide rebuild on thaw (#68). Process/CI: the H10 review-verdict gate became a sha-bound **required** commit status and was hardened through its false-open chain (#622/#629/#632/#648/#649/#672/#698), the decision corpus split to one YAML-frontmatter record per file (#610/#620), and headless Playwright UI-E2E flows landed (#445/#533). Five dual-provider migrations. |
|
||||
| `v26.14.0` | **Live TV no longer starves on embedded bitmap subtitles** — `-readrate` paces an input off its *furthest-behind* stream, and a PGS/DVD subtitle read through the video's own `-i` is sparse enough to drag the whole process to **0.53x realtime** against the 1.0x a client consumes, draining the buffer until the channel stalls. Fixed with a capability-gated `-readrate_catchup` (ffmpeg 8.0+) on realtime inputs, keeping `-readrate` on the frame-producing path so the `ffmpeg.qsv-extra-hw-frames-floor` bound is untouched; measured 0.533x → 1.067x on QSV and software, with a 240s QSV soak clean of allocation errors (#726). Affects items carrying an embedded bitmap subtitle matching the channel's subtitle mode — 3,182 of 24,646 media versions on prod, and a property of the *item*, not the channel, which is why the stall presented as random. Process/CI: the H10 review-verdict gate's repair sentinel became a fixed point and its write is now fenced on the timeline retarget count, closing a raced-sentinel false-open (#706/#707/#711). **The decisions validator now cross-checks its dependency-free frontmatter parse against PyYAML** and reports both the truncating unquoted `` #`` and the scalar-closing bare apostrophe as errors, so a record whose `rule:` silently halves under PyYAML fails the local gate instead of CI (#674/#688) — the ceiling-calibration claim was also split so the suite pins what the derivation MEANS rather than live-corpus order statistics. Dependencies: CliWrap 3.10.4, JetBrains.ReSharper.GlobalTools 2025.3.5. |
|
||||
| `v26.15.0` | **`main` became PR-only, which is what makes `review-verdict/h10` load-bearing** — branch protection now carries `enable_push: false` *and* `block_admin_merge_override: true`, so a direct `git push origin HEAD:main` is refused at pre-receive for every account including a site admin, and an admin cannot force-merge past a missing or red required context (#743/#749). Gitea only evaluates `status_check_contexts` on the PR merge path, so before this the whole gate was skippable with no forgery needed. The gate was then hardened through the rest of its false-open chain: CI's registry credential scoped so a head-resolved workflow cannot forge the status (#697), a stray expression delimiter **in a comment** that killed the gate outright and is now banned on the release path itself by a `scan` job `build` depends on (#751/#767), a positive verdict inherited only from an allow-listed reviewer (#742), the PR-timeline `HEAD` alias fenced on the `pull_push` count (#803/#664), and both `/statuses/{sha}` reads paged to a validated terminator (#763). App-facing: **FFmpeg profile numeric fields are bounded with a 422 that names the bound and its consequence** instead of being silently rewritten, and readrate pacing is exposed as a profile field (#735 — one dual-provider migration, `Add_FFmpegProfile_ReadRatePacing`). **On Now / Next gets a boxed background and is on by default** (#732). Field-level progressive disclosure lands as a shared `FieldHelp` trigger + portalled panel, adopted on FFmpeg Profiles and documented as `docs/spa-conventions.md` §15 (#734). Auto-Tune proposal rows align on a grid (#721); `AddItemsDialog` resolves by search instead of windowing a whole media-library type, with stale-response and is-mounted guards (#685/#740). Correctness: a paged handler now counts **the same query it pages**, so a filtered page stops reporting the unfiltered total across playouts, rerun collections and six sibling handlers (#690/#758); nullable primitive collections are guarded at the read site rather than mutated on a tracked entity — `SongMetadata.Artists` in both search indexers (#701/#824) and the scheduling `DaysOfMonth`/`MonthsOfYear`/`DaysOfWeek` columns, where a runtime NULL reads as **unrestricted** (#823); SPA full-replace bodies are built as `Complete<T>` so an omitted optional DTO field is a type error (#807); and every `ErsatzTV.Mcp` tool is pinned to its OpenAPI contract, with `graphicsElementIds` and `padToNearestMinute` declared so an MCP channel update stops silently stripping the overlay (#754/#757). Process/CI: one rule for how a guard derives its population and one for the proof it ships with, both enforced (#774/#775) — populations come from the git index, never a filesystem walk (#806/#819), every `MUTATION` grade is executed each suite rather than asserted in prose (#790), and verification code is itself code under test (#796). Workflows declare their own per-job metadata and guard populations derive from it (#786/#789); all six declare `permissions:` (#748); `ci-image.yml` publishes from `main` only and the persisted checkout credential is gone (#744/#746/#835). Hooks report their own execution and the report is measured (#776), and `docs.no-session-narrative` was generalized to every durable artifact and the sites its sweep IDENTIFIED were remediated as judgements rather than by regex — no closure is claimed over a phrasing space (#784/#812). A ruff config is committed and enforced in CI (#780). Dependencies: Scriban.Signed 7.2.6. **Cut at `736649b3b`, not at `main`'s head**: `completeAnnotations.guard.test.ts` (merged in the very next commit) needs a git index the Dockerfile's `web-build` stage does not have, so every image build after `736649b3b` fails — including a tag build (#887). `736649b3b` is the newest commit whose `Build & push image (amd64)` actually ran green. The scheduling-recurrence fix #880 therefore slips to the next release. |
|
||||
| `v26.16.0` | **Channels carrying Jellyfin music videos play again** — the music-video scanner wrote the library-items projection's empty stream list straight through, so every Jellyfin music video was stored with no `MediaStream` rows (1520/1520 on prod) and `SelectVideoStream` threw on tune-in while the guide looked normal; it now reads statistics from per-item playback info under the movie scanner's conditions, writes nothing when the server cannot answer, and `SelectVideoStream` fails with the version, file and remedy instead of a bare `Sequence contains no matching element`. **Existing rows heal only on a deep scan** (`POST /api/v1/libraries/{id}/scan?deep=true`), since a regular scan skips unchanged etags (#924/#928, `scan.mediaserver-statistics-from-server`). **Alternate-schedule and template recurrence arrays: an absent array means unrestricted, an explicit `[]` is rejected with a 422 that names the consequence** — the three recurrence sets are read conjunctively, so `?? []` on an omitted array returned 200 while storing an item that could never apply; absent and null now normalize to the same `All*()` sets the read side substitutes, and an unchanged pre-existing empty set is still accepted so no other item in a whole-list PUT becomes uneditable (#880/#892 — slipped from v26.15.0). **Unknown channel `graphicsElementIds` are rejected with a 422** (and the same bounded id-list validation applies to Deco updates), and the built-in graphics elements are identified by the seeded path, not the filename (#568/#922). The channel preview says why it is not playing and only the preview opts into muted autoplay (#554/#910); a failed write in the Collections screen reports to a surface that outlives the dialog that started it (#830/#878). Correctness: a classifier-unreadable branch-protection rule (a glob rule with an empty `branch_name`) is no longer resolved to a rule that matches nothing, which had let the merge hook auto-grant against a rule Gitea may be applying (#858/#859/#897); an empty PR-timeline page before the 20-page cap is no longer read as exhaustion, which a PR author could use to hide an ABA force-push behind 50 comments — narrowed, not closed (#870/#896); a head-authored run holds no stored secret, so the PR route reads the registry and status API anonymously (#885/#919); `Complete<T>` annotations are now derived where applied, not only where defined (#820/#883). **Image builds fixed**: the Dockerfile's `web-build` stage built the SPA and then ran a vitest suite with a hand-written `--exclude` list, so `completeAnnotations.guard.test.ts` (needs a git index the stage lacks) failed every `main` and tag image build after `736649b3b`; the stage now builds the SPA and does not test it, the suite runs once unfiltered in the `test` job `build` depends on, and a guard pins that delegation (#887/#899). Process/CI: the verdict gate replaces every unknown state and proves the clauses that claim to (#849/#890), the verdict writer checks the gate will honour what it just posted (#845/#889), a sourced path is code so every hook resolves it from its own tree (#891/#903), a mutation-outcome claim is executed wherever it is written (#881/#914), `--depth` is never passed to a checkout that may be complete (#836/#884), session narrative is swept from hooks, workflows, scripts, tests and code comments (#876/#906), a scripted-engine build API is characterized in-process against a committed script fixture (#563/#915), orchestrated sessions get a mechanics page, standing prompt and pick/build/resume workflows (#907/#908) with the review loop and concurrency cap tightened (#911/#912/#916/#918), and CI claims dated at Gitea 1.25.4 are re-established on 1.27.1 (#869/#893/#905). No dual-provider migrations and no dependency bumps in this release. **The image moves to the .NET 10.0.14 runtime** (prod had 10.0.11), so it carries the October 2026 .NET out-of-band security fixes, including the Brotli decompression overflow (CVE-2026-111497) and the SocketAsyncEventArgs memory corruption (CVE-2026-70197). No code change was needed, because `docker/Dockerfile` follows the floating `aspnet`/`sdk:10.0-noble-amd64` tags. Release scan on the immutable `:26.16.0` (`sha256:6d698162…`), 2026-10-27: ZAP 0 FAIL / 120 PASS / 1 WARN (100001, the SPA catch-all's `302 → /app` HTML, by design); semgrep 2 rule classes, both false positives (a constant-SQL test helper; JSX inside `design-system/` demo HTML, which the app doesn't bundle). **Known advisories carried unchanged from v26.15.0**: `System.Security.Cryptography.Pkcs` 9.0.4 (High, transitive via the unused Oracle/SQL Server BulkExtensions providers) and `SixLabors.ImageSharp` 3.1.12 (Moderate, fix only in 4.x), tracked in #929. |
|
||||
|
||||
**Before cutting a release — sweep `docs/decisions.md` + `docs/decisions/`** (ersatztv#521, supersedes
|
||||
the ersatztv#303 H9 append-only ritual). Supersession/retirement is now a same-PR act (add the new
|
||||
@@ -106,17 +107,21 @@ commit in the range (see the `decisions.md` header) — routine lifecycle metada
|
||||
> tag-protection rule (`name_pattern: v*`, `whitelist_usernames: ["timothy"]`,
|
||||
> `release.tag-protection-v-star`), because a `v*` push builds and publishes `:prod`. A push from
|
||||
> any other account is expected to be refused — configured, not exercised, since only the
|
||||
> `timothy` credential is available here. **The positive half is UNVERIFIED too**: the rule was
|
||||
> applied 2026-09-04 and only its negative half was measured (a non-`v*` tag still pushes), because
|
||||
> pushing a `v*` tag outside a real cut would publish `:prod`. This cut is its first real
|
||||
> exercise. If the tag push is refused, unblock with
|
||||
> `timothy` credential is available here. The positive half is verified: the rule was
|
||||
> applied 2026-09-04, its negative half measured that day (a non-`v*` tag still pushes), and a real
|
||||
> `v*` cut by `timothy` succeeded with the rule in force on 2026-10-27 (v26.16.0). If the tag push
|
||||
> is refused, unblock with
|
||||
> `DELETE /api/v1/repos/timothy/ersatztv/tag_protections/1`, push, then re-`POST` the rule
|
||||
> (prefer a `PATCH` adding the account if the cut has moved to a different operator, so the
|
||||
> protection is never left off) — and record the outcome on `release.tag-protection-v-star`,
|
||||
> which is waiting for it.
|
||||
2. Wait for tag CI to build `:prod` + the immutable `:<version>` + `:<sha>` images. Run
|
||||
`scripts/security-scan.sh` on jazz against **the immutable `:<version>` image**, not a
|
||||
moving tag, and triage every ZAP/semgrep finding.
|
||||
`scripts/security-scan.sh` against **the immutable `:<version>` image**, not a moving tag, from a
|
||||
checkout of the tagged commit on a docker host whose non-interactive `PATH` has semgrep: check with
|
||||
`ssh <host> command -v semgrep` first (bumblebee passed that check on 2026-10-27,
|
||||
server-management#1072). A host without semgrep fails the gate with exit 2; the only alternative is
|
||||
`ETV_SCAN_SKIP_SEMGREP=1` plus a separate semgrep run against the tagged commit, recorded as such.
|
||||
Triage every ZAP/semgrep finding. Requirements: Security scanning → Where/when.
|
||||
3. Only after the candidate passes, manually `DeployStack jazz-media` and observe its pre-deploy
|
||||
output. Prod's compose deliberately follows floating `:prod` (Timothy's 2026-07-11 decision), so
|
||||
no CI push or pin bump is needed.
|
||||
@@ -1303,7 +1308,7 @@ minutes, and it would still be wrong to filter on `scripts/**`.
|
||||
stay unpinned on purpose: a pytest release does not add assertions to your suite, a ruff release
|
||||
adds rules to your lint.
|
||||
- Lint runs **before the jq preflight**, and after the `git` one. `Preflight jq version` is a hard
|
||||
`--expect` tripwire; a lint sitting behind it goes dark for as long as the jq contract is broken,
|
||||
`--expect` tripwire; a lint sitting behind it goes dark for as long as the jq version is outside the accepted set,
|
||||
under a red that says "jq". `Preflight external tools` stays ahead, because the lint steps consume
|
||||
`git` — without it a missing git reaches them as an empty population and they blame the glob.
|
||||
- Neither step is `ruff check .`. Both pass an **explicit population** from
|
||||
@@ -1330,7 +1335,7 @@ which is itself a small demonstration of why the suite needed to run in CI at al
|
||||
against a temp repo it creates itself, never this repository's history.
|
||||
|
||||
Two **preflight steps** run before the suite. The first asserts `git` is on PATH; the second runs
|
||||
`scripts/jq-preflight.sh --expect 1.6`, which checks jq's **version**, not merely its presence (see
|
||||
`scripts/jq-preflight.sh --expect 1.6,1.7`, which checks jq's **version**, not merely its presence (see
|
||||
"The jq contract" below). Those two tests exec the real shell scripts, which shell out to `jq` ~26
|
||||
times; the tests shim `curl` on PATH but not `jq`, so a runner image without it would surface as ~20
|
||||
opaque assertion failures instead of one diagnosis. Both deliberately **check** rather than install —
|
||||
@@ -1339,13 +1344,16 @@ the runner image.
|
||||
|
||||
### The jq contract (ersatztv#648)
|
||||
|
||||
> Full rationale: `docs/decisions/records/ci/jq-version-contract.md`.
|
||||
> Full rationale: `docs/decisions/records/ci/jq-accepted-version-set.md` (`ci.jq-accepted-version-set`).
|
||||
|
||||
Every shell gate in this repo — `decisions-guard`, `script-tests`'s own harness,
|
||||
`pretooluse-merge-consent.sh`, `review-verdict.yml`, `scripts/pr-changed-files.sh` — is authored and
|
||||
tested on a developer Mac shipping **jq 1.8.x**. The CI runner ships **jq 1.6**. Author to the
|
||||
1.6-compatible subset; three concrete constructs diverge between the two and each one produced a real
|
||||
bug when it hit CI for the first time:
|
||||
tested on a developer Mac shipping **jq 1.8.x**. The CI runners' jq is **not pinned**: both `small`
|
||||
runners map to the floating `runner-images:ubuntu-latest` image with no digest pin, so each host keeps
|
||||
whatever it last pulled. Measured 2026-10-28: jq 1.7 on both hosts (PR #933 job 12347, PR #934 job
|
||||
12360); no known runner reports 1.6, but an unpinned host could (server-management#1080). Author to the
|
||||
1.6-compatible subset, which is the **floor**; three concrete constructs diverge between 1.6 and
|
||||
later versions and each one produced a real bug when it hit CI for the first time:
|
||||
|
||||
- **`jq -e` over EMPTY input.** Exits 4 on jq >= 1.7, but **0** on jq 1.6. A guard that infers
|
||||
"transport failure" from that exit status silently passes an empty/failed page on 1.6.
|
||||
@@ -1360,16 +1368,25 @@ bug when it hit CI for the first time:
|
||||
the parsed version and asserts a floor of 1.6) so a future divergence can be diagnosed from the log
|
||||
alone instead of guessing at the runner image.
|
||||
|
||||
**Pin vs floor is deliberately asymmetric.** `scripts/jq-preflight.sh --expect 1.6` additionally pins
|
||||
the version and fails loudly if it drifts, but that mode is used **only** by `script-tests`
|
||||
(`.gitea/workflows/pr-checks.yml`) — advisory, not a required check. `review-verdict.yml` runs the
|
||||
no-args floor-only mode and never pins, because that workflow writes `review-verdict/h10`, the
|
||||
branch-protection-**required** status check on `main`: a hard pin there would mean the day the
|
||||
runner's jq version changes (a base-image bump, a host reimage — nothing this repo controls), every
|
||||
PR on `main` stops merging until someone notices and re-pins. A required merge gate cannot fail
|
||||
because an upstream package manager did its job. The narrower pin on `script-tests` exists precisely
|
||||
because that job is the suite's only 1.6 coverage — if the runner's jq silently changed, that coverage
|
||||
would evaporate with no signal, so failing loudly there forces a human decision instead.
|
||||
**Accepted set, not a single pin (2026-10-29, ersatztv#935).** `scripts/jq-preflight.sh --expect
|
||||
1.6,1.7` additionally asserts the version is a member of the comma-separated set and fails loudly
|
||||
(exit 1) on any other version; a malformed set (`1.6,`, `1.x`) is a usage error (exit 2). It is used
|
||||
**only** by `script-tests` (`.gitea/workflows/pr-checks.yml`). The set is exactly {1.6, 1.7}: a
|
||||
1.8 runner stays red so a person decides. As of 2026-10-28 no known runner provides
|
||||
jq 1.6, so **real-jq 1.6 coverage of `scripts/tests/` is currently absent**; 1.6 stays in the set only
|
||||
because the tag is unpinned. In-repo jq shims (`test_pr_changed_files.py`,
|
||||
`test_merge_consent_exemption.py`) reproduce the empty-input `-e` quirk only; the NUL-truncation and
|
||||
parse-error-exit-code quirks have no coverage on a 1.7 runner. The floor stays at 1.6. **server-management#1080** (a fleet digest
|
||||
pin) is what narrows the set: when it settles one version, change `--expect` to that single version.
|
||||
The tripwire is advisory in name only: `pretooluse-merge-consent.sh` reads the combined status, so a
|
||||
red `script-tests` blocks the hook-mediated merge path until the `--expect` value is fixed.
|
||||
|
||||
**Pin vs floor is deliberately asymmetric.** `review-verdict.yml` runs the no-args floor-only mode and
|
||||
never `--expect`, because that workflow writes `review-verdict/h10`, the branch-protection-**required**
|
||||
status check on `main`: a version assertion there would mean the day a runner's jq version changes (a
|
||||
base-image pull, a host reimage — nothing this repo controls), every PR on `main` stops merging until
|
||||
someone notices and re-pins. A required merge gate cannot fail because an upstream package manager did
|
||||
its job.
|
||||
|
||||
Baking a pinned jq into `docker/ci/Dockerfile` was considered and rejected: `review-verdict.yml` is
|
||||
`runs-on: small` with no toolchain-image pin, and per `ci.small-lane-git-only` the small lane is
|
||||
@@ -1462,10 +1479,10 @@ they are fail-open and `--no-verify` bypasses them. Tag pushes go through a sepa
|
||||
are meant to stay unaffected for the release operator: since ersatztv#885 `tag_protections` carries
|
||||
one rule, `v*` whitelisted to `timothy` (`release.tag-protection-v-star`), which is intended to leave
|
||||
the release cut in "Cutting a release" working unchanged while refusing a `v*` tag from the `renovate`
|
||||
bot — a push that would otherwise have published a `:prod` image. Both halves of that are EXPECTED,
|
||||
UNVERIFIED: only the `timothy` credential exists here, so neither a real release cut nor a refused bot
|
||||
push has been exercised. `release.tag-protection-v-star` carries the same caveat and waits on the
|
||||
next cut.
|
||||
bot — a push that would otherwise have published a `:prod` image. The release-cut half is verified
|
||||
(the v26.16.0 cut, 2026-10-27, pushed its `v*` tag as `timothy` with the rule in force); the refused
|
||||
bot push is still UNVERIFIED, because only the `timothy` credential exists here
|
||||
(`release.tag-protection-v-star`).
|
||||
|
||||
**A fourth arm: guard-scope freshness (ersatztv#787).** `scripts/tests/test_ci_dropped_step_guard.py`
|
||||
decides which CI jobs must carry per-step execution markers by DERIVING that scope from
|
||||
@@ -2647,13 +2664,21 @@ running product from outside our C#/review stack.
|
||||
`/api/v1` operation, injecting `X-Api-Key` on every request via a ZAP replacer rule so it reaches the
|
||||
`[RequiresAuthentication]` + `RequireKeyForReads` surface (not just the `/app` shell an unauthenticated
|
||||
spider sees). Then a **semgrep** SAST cross-check (`p/security-audit` + `p/secrets` + `p/csharp`). The
|
||||
container is torn down on exit.
|
||||
- **Where/when.** Runs on the **docker host** (jazz — the Mac has no docker), like
|
||||
`migration-smoke.sh`: `scripts/security-scan.sh [IMAGE] [PORT]` (defaults `…:latest` / `8411`). It is a
|
||||
**manual release-gate**, deliberately **not** a per-PR CI job — it needs docker + a booted image, takes
|
||||
several minutes, and is noisy (expect to tune, not take raw). The continuous layer is the per-PR
|
||||
white-box gates + the weekly `dependency-scan`; this is the per-release black-box pass. Re-run it each
|
||||
release and before any change to the exposure posture.
|
||||
container is torn down on exit. **The SAST half is required** (ersatztv#931): with no semgrep on `PATH`
|
||||
the script exits 2 naming the tool, before the image is pulled. The only skip is the explicit opt-out
|
||||
`ETV_SCAN_SKIP_SEMGREP=1` (any other value, `0` included, does not skip). A run that sets it is DAST
|
||||
only and has not passed the SAST half: run semgrep separately and say so in the release record. A
|
||||
semgrep that is installed but exits non-zero (bad config, registry unreachable) also fails the script,
|
||||
since findings alone exit 0.
|
||||
- **Where/when.** Runs on a **docker host** with registry access, from a checkout of the commit under test
|
||||
(semgrep scans the checkout the script lives in), whose **non-interactive** `PATH` has semgrep: the
|
||||
`PATH` that `ssh <host> command -v semgrep` sees, which for zsh comes from `~/.zshenv`, not `.zshrc`. A
|
||||
host without it fails the gate by construction; the alternative is the opt-out plus a separate SAST
|
||||
run. Like `migration-smoke.sh`: `scripts/security-scan.sh [IMAGE] [PORT]` (defaults `…:latest` /
|
||||
`8411`). It is a **manual release-gate**, deliberately **not** a per-PR CI job — it needs docker + a
|
||||
booted image, takes several minutes, and is noisy (expect to tune, not take raw). The continuous layer
|
||||
is the per-PR white-box gates + the weekly `dependency-scan`; this is the per-release black-box pass.
|
||||
Re-run it each release and before any change to the exposure posture.
|
||||
- **Exit-code contract (ersatztv#338).** `zap-api-scan.py`'s raw exit code is NOT a simple pass/fail — it
|
||||
conflates a clean run with a warnings-only run unless you know its wrapper contract: **0** clean (no
|
||||
FAIL or WARN alerts), **2** WARN-only (triage required, but **not** release-blocking), **1** FAIL (at
|
||||
@@ -2661,12 +2686,19 @@ running product from outside our C#/review stack.
|
||||
hung post-scan cleanup (the report written before the hang is still usable — triage it), any other
|
||||
code means the scanner/tool itself errored (not a scan result at all). `scripts/security-scan.sh`
|
||||
encodes this in `classify_zap_exit()` and prints an unambiguous `==> ZAP result: <PASS|WARN|FAIL|
|
||||
TIMEOUT|TOOL ERROR> ...` line; the script's own exit status reflects that classification (0 for
|
||||
clean/WARN, 1 for FAIL/timeout/tool-error) rather than ZAP's raw code, so a warnings-only run no longer
|
||||
reads as a failed scan. Found when the v26.8.0 release scan (#335) returned raw exit 2 for a report with
|
||||
`FAIL-NEW: 0` and two known/expected warning classes — the shell result looked like a failure though the
|
||||
release gate had actually passed. Run `scripts/security-scan.sh --selftest` for a docker-free regression
|
||||
check of the classification logic.
|
||||
TIMEOUT|TOOL ERROR> ...` line. The script's **own** exit is not ZAP's raw code, so a warnings-only run
|
||||
no longer reads as a failed scan. Found when the v26.8.0 release scan (#335) returned raw exit 2 for a
|
||||
report with `FAIL-NEW: 0` and two known/expected warning classes — the shell result looked like a
|
||||
failure though the release gate had actually passed. Every non-zero script exit is release-blocking:
|
||||
**0** ZAP clean or WARN-only and the SAST half ran clean (or was opted out); **1** ZAP FAIL, timeout or
|
||||
tool error, or the throwaway target never came up or its key was unreadable; **2** — *not* ZAP's WARN —
|
||||
a scanner half did not run: semgrep missing (before the scan) or erroring (after it), or docker
|
||||
missing; if ZAP already classified non-zero, that status stands and the semgrep error prints beside
|
||||
it. A setup command that fails under `set -e` (e.g. `docker pull`) exits with its own status. Run
|
||||
`scripts/security-scan.sh --selftest` for a docker-free regression check: it pins every ZAP
|
||||
classification, and runs the script itself with shimmed tools (ZAP reporting clean) to pin the
|
||||
semgrep gate — missing → exit 2 before docker is checked, erroring → 2, succeeding → 0 with semgrep
|
||||
run, opt-out → 0 with semgrep not run (`=0` still runs it). It exercises no real ZAP or semgrep.
|
||||
- **Triage.** Triage each WARN/FAIL finding false-positive vs real. Real, in-scope, go-live-blocking
|
||||
findings get fixed (e.g. the security headers from the #319 baseline; the Microsoft.OpenApi pin above);
|
||||
LAN-expected noise (Private-IP disclosure) is revisited only for genuine remote exposure. nuclei
|
||||
|
||||
@@ -57,7 +57,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
|
||||
| `ci.grep-q-pipefail-inversion` | In any script running under `set -o pipefail`, a security or classification predicate of the form `producer \| grep -q…` is FORBIDDEN: `grep -q` exits at its first match, the producer then takes SIGPIPE and exits 141 once the data exceeds the pipe buffer (~64K), so `pipefail` reports the pipeline as FAILED even though grep MATCHED — inverting the predicate exactly when the input is large. A here-string (`grep -q… <<< "$data"`) is ALSO forbidden: bash materialises a large here-string via temporary storage, so it fails when temp space is full or unwritable, and inside an `if`/`!` that failure flips the predicate the same way. COUNT instead — `n=$(printf '%s\n' "$data" \| grep -cE "$re")` — because `grep -c` drains stdin (no early exit, no SIGPIPE) over an ordinary pipe (no temp file). Read grep's status honestly: exit 1 means a zero count and is a legitimate answer, anything >1 is a real error. Evaluate the counts ONCE at TOP LEVEL, never inline inside an `if`/`elif` condition: inside `$( )` an `exit` leaves only the subshell and `set -e` does not fire, so an error silently reads as "no match". Validate that each result is numeric and fail closed if not. This applies to both the enforced gate `.gitea/workflows/review-verdict.yml` and the advisory hook `.claude/hooks/pretooluse-merge-consent.sh`. | 2026-07-29 | [link](records/ci/grep-q-pipefail-inversion.md) |
|
||||
| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step's `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step's own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`'s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), `web/vite.config.ts` PINNED WHOLE — it decides what the suite collects and what `vite build` loads, and pinning a BLOCK of it was defeated seven measured ways across three rounds, so the partial match was withdrawn rather than respelled an eighth time; and the ABSENCE of any config file that outranks it — `vitest.config.*` for vitest, and `vite.config.js`/`.mjs` for `vite build`, whose `DEFAULT_CONFIG_FILES` lists them BEFORE `.ts`. Pinning one file is worthless while a second can outrank it, and that family went short twice. The gating job's `container:` image is left to `ci.image-pin-population` rather than guarded twice. What is unmodelled is a LIST, not an "only": an `ENV` changing what a pinned `RUN` resolves, the plugin BODIES, and a publish through another action. The count of such routes lives in the `docs/guard-inventory.md` row and nowhere else — it briefly existed in three places with two different values, which is how a number stays wrong. It is a running total, wrong at every count so far. They share one shape, which is the transferable part: A PIN ASSUMES IT IS PINNING THE ARTIFACT THAT STILL DECIDES, and every route found so far is authority moving to another file, another occurrence, another workflow, or a hook the pinned command invokes. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) |
|
||||
| `ci.infra-shaped-red-under-load` | When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure. | 2026-07-21 | [link](records/ci/infra-shaped-red-under-load.md) |
|
||||
| `ci.jq-version-contract` | Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset, because the CI runner ships jq 1.6 while every developer Mac ships 1.8.x. `scripts/jq-preflight.sh` (no args) prints the parsed version and asserts a floor of 1.6 in every gate job's log; `scripts/jq-preflight.sh --expect 1.6` additionally pins and fails loudly, but ONLY in the `script-tests` job. `review-verdict.yml` never pins — it writes the branch-protection-required `review-verdict/h10` status, so a hard pin there would turn any jq bump into a repo-wide merge deadlock. | 2026-07-26 | [link](records/ci/jq-version-contract.md) |
|
||||
| `ci.jq-accepted-version-set` | Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset (floor 1.6, unchanged). Both `small` runners map to the floating `runner-images:ubuntu-latest` image with no digest pin, so the fleet jq version is not guaranteed; measured 2026-10-28, both hosts report jq 1.7 and no known runner reports 1.6. `scripts/jq-preflight.sh --expect 1.6,1.7` in the `script-tests` job accepts exactly that comma-separated SET and fails loudly on any other version; 1.6 stays in the set only because the tag is unpinned, and real-jq 1.6 coverage of `scripts/tests/` is currently absent. server-management#1080 (fleet digest pin) is the event that narrows the set to one version. `review-verdict.yml` stays floor-only (no `--expect`): it writes the branch-protection-required `review-verdict/h10` status. | 2026-10-29 | [link](records/ci/jq-accepted-version-set.md) |
|
||||
| `ci.killed-job-triage` | Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure. | 2026-07-21 | [link](records/ci/killed-job-triage.md) |
|
||||
| `ci.monitor-armed-at-pr-open` | Arm a CI monitor on the PR head sha the moment the PR opens, polling the commit-status endpoint — not at the end of the work. | 2026-07-21 | [link](records/ci/monitor-armed-at-pr-open.md) |
|
||||
| `ci.no-host-health-gating` | Push when your work is validated — never SSH to bumblebee to sample load/RAM first, and never hand-schedule around other sessions' runs. | 2026-07-21 | [link](records/ci/no-host-health-gating.md) |
|
||||
@@ -151,7 +151,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
|
||||
| `release.prepush-clean-worktree-guard` | A fail-open pre-push hook blocks a push when any file in the branch's diff vs `origin/main` also has uncommitted working-tree or index changes, since a stale-index commit (e.g. `git reset --soft` + `git add` over an edited-but-unstaged fix) can silently push, CI-test, and get reviewed a different tree than the one on disk. Scope is precise to pushed-diff files; escape hatch `ETV_ALLOW_DIRTY_PUSH=1`. | 2026-07-17 | [link](records/release/prepush-clean-worktree-guard.md) |
|
||||
| `release.promotion-floating-prod` | Prod tracks the floating `:prod` image reference; a tag build's immutable `:<version>` image is scanned first, then promotion happens via a separate manual `DeployStack`, with daily auto-update only as a fallback — tag with enough runway before 03:00 to avoid an unscanned promotion. | 2026-07-13 | [link](records/release/promotion-floating-prod.md) |
|
||||
| `release.review-verdict-gate` | A PR may not merge until a `Review-verdict: <MERGEABLE\|APPROVED\|LGTM\|BLOCKED\|NOT-MERGEABLE> @ <head-sha>` comment references the PR's current head sha (short-sha prefix match against the verdict's OWN `@ <sha>` field, marker at COLUMN 0 (no indent, so indented code blocks cannot self-approve), whole-word verdict token, fenced code blocks stripped with markdown fence-length semantics, negative wins over positive on the same head); folds into the H6 merge-consent hook as condition (c). The grammar lives in ONE tested place, `scripts/check-review-verdict.sh` — #629 found three false-opens that survived because it was implemented inline and untested while this record described stricter behaviour than the code had. | 2026-07-12 | [link](records/release/review-verdict-gate.md) |
|
||||
| `release.tag-protection-v-star` | The repository carries one tag-protection rule: `name_pattern: v*`, `whitelist_usernames: ["timothy"]`. This matters because `docker-build.yml` triggers on `push.tags: [v*]` and a `v*` tag push BUILDS AND PUBLISHES the `:prod` image — the tag the `jazz-media` Komodo stack follows. `release.main-direct-push-disabled` closed every route to `main`, and left this one open: a tag push is a separate mechanism that branch protection does not govern, so it was the cheapest remaining single act that reaches production. It buys NOTHING against the admin actor (`timothy` is on the whitelist, and no self-administered control bounds an owner) and everything it buys is against `renovate`, the repo's other write-capable account (`permission: write`, `is_admin: false` from `GET /repos/timothy/ersatztv/collaborators/renovate/permission`, measured in #853 2026-08-30 and re-read 2026-09-04): the H10 verdict gate and the Renovate exemption rule bound what that bot can get MERGED, and nothing bounded what it could TAG. The rule's failure mode is a BROKEN RELEASE CUT, so it is verified in both directions before it is trusted: the negative half is done and recorded below; the POSITIVE half — a real `v*` release-cut tag push by `timothy` still succeeding — is DEFERRED to the operator's next real cut, because pushing a `v*` tag from any working session publishes `:prod` and a release is the operator's act, not a verification step. Treat that as OPEN until a cut lands. There is no committed mirror of this rule: it lives in Gitea, and this record's `mechanics:` carries the exact call and the read-back, the same shape `release.main-direct-push-disabled` uses. | 2026-09-04 | [link](records/release/tag-protection-v-star.md) |
|
||||
| `release.tag-protection-v-star` | The repository carries one tag-protection rule: `name_pattern: v*`, `whitelist_usernames: ["timothy"]`. This matters because `docker-build.yml` triggers on `push.tags: [v*]` and a `v*` tag push BUILDS AND PUBLISHES the `:prod` image — the tag the `jazz-media` Komodo stack follows. `release.main-direct-push-disabled` closed every route to `main`, and left this one open: a tag push is a separate mechanism that branch protection does not govern, so it was the cheapest remaining single act that reaches production. It buys NOTHING against the admin actor (`timothy` is on the whitelist, and no self-administered control bounds an owner) and everything it buys is against `renovate`, the repo's other write-capable account (`permission: write`, `is_admin: false` from `GET /repos/timothy/ersatztv/collaborators/renovate/permission`, measured in #853 2026-08-30 and re-read 2026-09-04): the H10 verdict gate and the Renovate exemption rule bound what that bot can get MERGED, and nothing bounded what it could TAG. The rule's failure mode is a BROKEN RELEASE CUT, so it is verified in both directions before it is trusted: both halves are done and recorded below: the negative half on 2026-09-04, and the POSITIVE half — a real `v*` release-cut tag push by `timothy` succeeding with the rule in force — on the v26.16.0 cut, 2026-10-27. There is no committed mirror of this rule: it lives in Gitea, and this record's `mechanics:` carries the exact call and the read-back, the same shape `release.main-direct-push-disabled` uses. | 2026-09-04 | [link](records/release/tag-protection-v-star.md) |
|
||||
| `release.verdict-status-check` | The H10 review verdict is written as a `review-verdict/h10` Gitea **commit status** on the exact reviewed sha by `scripts/post-review-verdict.sh`, and that context is a REQUIRED status check on `main`. Because a status belongs to one sha, a later commit cannot inherit it, so Gitea's own `merge_when_checks_succeed` refuses to merge a head no one reviewed. The PreToolUse hook additionally refuses to SCHEDULE an auto-merge unless that status is already green on head. A `pull_request_target` workflow auto-passes the two exempt classes (Renovate-authored, docs-only) unless the PR touches a protected path (`.claude/`, `.codex/`, `.gitea/`, `.husky/`, `scripts/`, `docker/ci/`). This extends — does not supersede — `release.review-verdict-gate` (#303 H10), whose comment convention remains the human-readable artifact and the hook's condition (c). | 2026-07-25 | [link](records/release/verdict-status-check.md) |
|
||||
| `release.verdict-vocabulary-shared` | The H10 verdict words live in exactly one place, `scripts/lib/review-verdict-vocabulary.sh`, as two arrays. `scripts/post-review-verdict.sh` (write: word to commit-status state) classifies through `etv_verdict_class`; `scripts/check-review-verdict.sh` (read: comment to classification) builds `POS_RE`/`NEG_RE` from `etv_verdict_alternation`. Neither script may restate a word, and neither may enumerate the vocabulary in a usage banner or an error string. Every word is validated to `[a-z][a-z-]*` before it reaches a regex. Validation is enforced by a DATA dependency, not a control-flow check: `etv_verdict_vocabulary_validate` sets `ETV_VERDICT_VOCABULARY_OK=1` on its final line, and the derived views refuse to hand out words without it. A vocabulary that is missing, unreadable, truncated, exits at top level, is declared as a scalar, or fails validation yields NO words on either side — exit 1 for the writer (nothing posted), exit 2 for the reader (callers fail closed). Do NOT reintroduce a text-comparison parity test alongside this. | 2026-08-26 | [link](records/release/verdict-vocabulary-shared.md) |
|
||||
| `release.verdict-writes-status-before-comment` | `scripts/post-review-verdict.sh` writes the sha-bound `review-verdict/h10` commit status FIRST and the human-readable `Review-verdict:` comment SECOND. Every refusal path still refuses (fail-closed, unchanged) and exits non-zero, and none of them may leave a verdict comment behind. An orphaned comment is therefore PREVENTED rather than tolerated. If the comment write fails after the status was written, that is an error too, but it degrades to an `ask` at the merge gate rather than to an apparent grant. | 2026-08-22 | [link](records/release/verdict-writes-status-before-comment.md) |
|
||||
@@ -160,6 +160,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
|
||||
| `scan.getoraddfolder-db-lookup` | `ILibraryRepository.GetOrAddFolder` resolves the existing folder via a DB query on `(LibraryPathId, Path)`, not the caller's `LibraryPath.LibraryFolders` in-memory navigation, since that navigation is only eager-loaded on the local scan path and is null on remote (Jellyfin) callers. | 2026-07-20 | [link](records/scan/getoraddfolder-db-lookup.md) |
|
||||
| `scan.jellyfin-mixed-content-library` | A Jellyfin library whose collection type is `mixed` (or absent) maps to one ErsatzTV library of `LibraryMediaKind.Mixed`, scanned by running the movie/television/music-video scanners in sequence against that single library — a library is a place, not a media kind. | 2026-07-20 | [link](records/scan/jellyfin-mixed-content-library.md) |
|
||||
| `scan.libraryfolder-unique-identity` | `LibraryFolder` uniqueness per `(LibraryPathId, Path)` is enforced by a database unique index over a SHA-256 `PathHash` (Path is unbounded and not portably indexable), and `LibraryRepository.GetOrAddFolder`/`SetEtag` tolerate the constraint violation by re-reading and adopting the winner's row. | 2026-07-25 | [link](records/scan/libraryfolder-unique-identity.md) |
|
||||
| `scan.mediaserver-statistics-from-server` | Every media-server scanner writes a head `MediaVersion`'s statistics (streams, width/height, aspect, frame rate) from what the server reports per item — `GetMediaServerStatistics` (Jellyfin/Emby playback info) or the metadata call that carries them (`ServerReturnsStatisticsWithMetadata`, Plex) — and NEVER writes the library-items projection's version, whose `Streams` list is empty. When the server cannot supply them, nothing is written. In `MediaServerMusicVideoLibraryScanner` the hook is `abstract`, not a virtual returning `None`. | 2026-10-21 | [link](records/scan/mediaserver-statistics-from-server.md) |
|
||||
| `scan.musicvideo-server-identity` | Jellyfin music videos carry a per-library server identity (`JellyfinMusicVideo : MusicVideo` with `ItemId`/`Etag`, TPT table + ItemId index), so `JellyfinMusicVideoLibraryScanner` folds onto a shared `MediaServerMusicVideoLibraryScanner` base that diffs the **server item id** and soft-trashes (`FlagFileNotFound`) instead of diffing local paths and hard-deleting. Rows predating the identity are **adopted in place** — the identity row is inserted against the same `MediaItem` id, scoped to the scanned library's own `LibraryPath` — never deleted and re-added. | 2026-07-25 | [link](records/scan/musicvideo-server-identity.md) |
|
||||
| `scan.projection-failure-sweep-guard` | `MediaServerReconciliationGuard` takes a per-enumeration projection-failure count and refuses the file-not-found sweep (logged loudly) whenever it is non-zero against a non-empty existing set — at all six sweeps, including the nested per-show season and per-season episode ones #477 left unguarded (via `ShouldFlagMissingDescendants`, which applies the failure refusal but not #477's empty-fetch branch). Deliberate guard-clause skips (STRM, virtual, unsupported type) are explicitly **not** failures and never suppress a sweep. The missing-fraction / ratio threshold floated by #477 is **rejected**, not deferred. | 2026-07-25 | [link](records/scan/projection-failure-sweep-guard.md) |
|
||||
| `scan.zero-item-fetch-guard` | A media-server library sweep refuses to flag missing items when a successful fetch returns zero incoming items against a non-empty existing set (`MediaServerReconciliationGuard.ShouldFlagMissing`), rather than treating an ambiguous empty result as a full-library deletion. | 2026-07-19 | [link](records/scan/zero-item-fetch-guard.md) |
|
||||
|
||||
+3
-3
@@ -1,11 +1,11 @@
|
||||
---
|
||||
key: ci.jq-version-contract
|
||||
title: '2026-07-26 — jq 1.6 is the FLOOR every shell gate must run on; `scripts/jq-preflight.sh` makes the version observable, and only `script-tests` pins it (#648)'
|
||||
status: active
|
||||
status: superseded
|
||||
since: '2026-07-26'
|
||||
supersedes: none
|
||||
superseded-by: none
|
||||
rule: 'Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset, because the CI runner ships jq 1.6 while every developer Mac ships 1.8.x. `scripts/jq-preflight.sh` (no args) prints the parsed version and asserts a floor of 1.6 in every gate job''s log; `scripts/jq-preflight.sh --expect 1.6` additionally pins and fails loudly, but ONLY in the `script-tests` job. `review-verdict.yml` never pins — it writes the branch-protection-required `review-verdict/h10` status, so a hard pin there would turn any jq bump into a repo-wide merge deadlock.'
|
||||
superseded-by: ci.jq-accepted-version-set@2026-10-29
|
||||
rule: '(superseded) Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset, because the CI runner ships jq 1.6 while every developer Mac ships 1.8.x. `scripts/jq-preflight.sh` (no args) prints the parsed version and asserts a floor of 1.6 in every gate job''s log; `scripts/jq-preflight.sh --expect 1.6` additionally pins and fails loudly, but ONLY in the `script-tests` job. `review-verdict.yml` never pins — it writes the branch-protection-required `review-verdict/h10` status, so a hard pin there would turn any jq bump into a repo-wide merge deadlock.'
|
||||
signals: 'jq version divergence, jq 1.6 vs 1.8, jq -e exit code on empty input, contains NUL false positive, jq parse-error exit code collision, jq-preflight, script-tests --expect, review-verdict jq floor, merge deadlock from a pinned dependency · paths: `scripts/jq-preflight.sh`, `.gitea/workflows/review-verdict.yml`, `.gitea/workflows/pr-checks.yml`, `docs/ci-cd.md` · issues: #643, #647, #648, #649'
|
||||
mechanics: '`scripts/jq-preflight.sh` (no args) -> floor+observability in every gate job; `scripts/jq-preflight.sh --expect 1.6` -> tripwire, `script-tests` job only; `docs/ci-cd.md` -> "The jq contract"'
|
||||
---
|
||||
@@ -0,0 +1,80 @@
|
||||
---
|
||||
key: ci.jq-accepted-version-set
|
||||
title: '2026-10-29 — jq 1.6 stays the FLOOR; `script-tests` accepts exactly the SET {1.6, 1.7} until server-management#1080 pins the runner fleet (#935)'
|
||||
status: active
|
||||
since: '2026-10-29'
|
||||
supersedes: ci.jq-version-contract@2026-07-26
|
||||
superseded-by: none
|
||||
rule: 'Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset (floor 1.6, unchanged). Both `small` runners map to the floating `runner-images:ubuntu-latest` image with no digest pin, so the fleet jq version is not guaranteed; measured 2026-10-28, both hosts report jq 1.7 and no known runner reports 1.6. `scripts/jq-preflight.sh --expect 1.6,1.7` in the `script-tests` job accepts exactly that comma-separated SET and fails loudly on any other version; 1.6 stays in the set only because the tag is unpinned, and real-jq 1.6 coverage of `scripts/tests/` is currently absent. server-management#1080 (fleet digest pin) is the event that narrows the set to one version. `review-verdict.yml` stays floor-only (no `--expect`): it writes the branch-protection-required `review-verdict/h10` status.'
|
||||
signals: 'jq version divergence, jq 1.6 vs 1.7, script-tests jq tripwire red, jq-1.7 (parsed 1.7; floor 1.6), --expect set, unpinned runner image, floating runner-images ubuntu-latest, server-management#1080, jq-preflight, merge blocked by combined status · paths: `scripts/jq-preflight.sh`, `scripts/tests/test_jq_preflight.py`, `.gitea/workflows/pr-checks.yml`, `.gitea/workflows/review-verdict.yml`, `docs/ci-cd.md` · issues: #935, #648, #643, #647'
|
||||
mechanics: '`scripts/jq-preflight.sh` (no args) -> floor+observability in every gate job; `scripts/jq-preflight.sh --expect 1.6,1.7` -> set tripwire, `script-tests` job only (`test_jq_preflight.py` pins the exact value); `docs/ci-cd.md` -> "The jq contract"'
|
||||
---
|
||||
|
||||
Supersedes `ci.jq-version-contract` (2026-07-26, archived), whose premise was that the CI runner
|
||||
ships jq 1.6. That is no longer a fact the repo can state.
|
||||
|
||||
**Measured 2026-10-28.** `script-tests`' tripwire (`--expect 1.6`) went red on jazz-small-runner
|
||||
(PR #933, job 12347) and on bumblebee's small-runner (PR #934, job 12360), each reporting
|
||||
`jq-1.7 (parsed 1.7; floor 1.6)`. Cause (server-management#1080): both `small` runners map to the
|
||||
floating `runner-images:ubuntu-latest` image with no digest pin or forced pull, so each host keeps
|
||||
whatever it last pulled and the fleet's jq version is a function of pull history. A digest pin across
|
||||
the fleet is proposed in server-management#1080 and not yet decided.
|
||||
|
||||
**The invariant.** `script-tests` accepts exactly the set `{1.6, 1.7}`:
|
||||
|
||||
- `scripts/jq-preflight.sh --expect 1.6,1.7` parses the argument as a comma-separated set of
|
||||
`<major>.<minor>` members; an empty or free-form member (`1.6,`, `1.6,,1.7`, `1.x`) is a usage
|
||||
error (exit 2), never a member that matches nothing or everything. Membership is exact on
|
||||
`major.minor`, not a prefix or range test (`1.10` and `1.71` do not match `1.7` / `1.6`).
|
||||
- A version outside the set (`1.8`, which every developer Mac ships, `2.x`) exits 1 with the tripwire
|
||||
message. That red is witnessed by `scripts/tests/test_jq_preflight.py`
|
||||
(`test_a_version_outside_the_accepted_set_is_LOUD`), and the exact `--expect 1.6,1.7` value in
|
||||
`pr-checks.yml` is pinned by `test_script_tests_pins_the_jq_version`.
|
||||
- The set is exactly {1.6, 1.7}, not "1.6 or newer": widening it to every version would delete the
|
||||
tripwire. Admitting 1.7 is evidence-driven (both measured hosts run it); admitting 1.8 is not, and
|
||||
a 1.8 runner should go red so a person decides.
|
||||
|
||||
**The floor stays at 1.6.** Raising it would assert that no host in the fleet can report 1.6, which
|
||||
the unpinned image does not guarantee, and which only the #1080 decision can establish. A gate
|
||||
authored to the 1.6 subset is harmless on 1.7, and the three divergences that motivated the contract
|
||||
(`jq -e` over empty input, `contains("\u0000")`, the parse-error exit code) remain real on whichever
|
||||
host still runs 1.6.
|
||||
|
||||
**What coverage is lost.** As of 2026-10-28 no known runner provides jq 1.6, so the real-jq 1.6 run
|
||||
of `scripts/tests/` is currently absent; it would happen only if a host held an older image. In-repo
|
||||
jq shims reproduce ONE of the three 1.6 divergences: `jq -e` over empty input exiting 0
|
||||
(`test_pr_changed_files.py`, `test_merge_consent_exemption.py`). The `contains("\u0000")` truncation
|
||||
and the parse-error exit code (4 vs 5) have no shim, so on a 1.7 run they are covered only by the
|
||||
constructs having been replaced with version-stable ones, and any NEW 1.6-only divergence is
|
||||
uncovered.
|
||||
|
||||
**What narrows the set.** server-management#1080. When it pins the fleet to one jq version, change
|
||||
the `--expect` value in `pr-checks.yml` to that single version (the script accepts a one-member set)
|
||||
and record the pin here by superseding this record. If the pin lands on 1.7, decide then whether the
|
||||
floor rises, with the fleet evidence in hand.
|
||||
|
||||
**Unchanged from the superseded record.**
|
||||
|
||||
- *Pin vs floor is asymmetric.* `review-verdict.yml` runs the no-args floor-only mode and never
|
||||
`--expect`: it writes `review-verdict/h10`, the branch-protection-required status on `main`, so a
|
||||
version assertion there would turn any runner jq change into a repo-wide merge deadlock.
|
||||
`test_review_verdict_never_pins_a_jq_version` holds that.
|
||||
- *Firing the tripwire blocks merges.* `.claude/hooks/pretooluse-merge-consent.sh` reads the combined
|
||||
commit status and denies on anything not `success`/`skipped` (ersatztv#598), and `script-tests` is a Gitea Actions job folded into it. Only the
|
||||
`merge_when_checks_succeed` path, which defers to Gitea's required checks, is spared. This is the
|
||||
cost of the tripwire, accepted because the remedy is a one-line edit to `--expect`, and the
|
||||
2026-10-28 firing is that cost being paid.
|
||||
- *Baking jq into `docker/ci/Dockerfile` cannot reach the gate.* `review-verdict.yml` is
|
||||
`runs-on: small` with no toolchain-image pin and `ci.small-lane-git-only` keeps that lane
|
||||
git-only, so it gets the host's jq whatever the toolchain image holds. The fleet pin in
|
||||
server-management#1080 is the only lever that reaches it.
|
||||
- *The parse is strictly fail-closed.* `jq-preflight.sh` accepts only a first line of the form
|
||||
`jq-<X>.<Y>` or `jq version <X>.<Y>`; anything else exits 1. If a wrapper banner ever breaks that
|
||||
in the required `review-verdict.yml` job, widen the accepted forms, never relax fail-closed: an
|
||||
unparsed version must not satisfy the floor.
|
||||
- *The three constructs to avoid:* never infer "empty input" from a jq exit status (check the string
|
||||
in shell first); never use `contains("\u0000")` (use `explode | index(0)`); never read `jq`'s
|
||||
exit code as a specific parse-error signal on ambiguous input (validate the response shape).
|
||||
|
||||
Full rationale for how the three divergences were found is in
|
||||
`docs/decisions/archive/ci/jq-version-contract.md` and `docs/decisions/records/ci/script-tests-job.md`.
|
||||
@@ -91,7 +91,7 @@ a judgement that type checking does not belong.
|
||||
|
||||
**The pin is the second half of the fix.** An unpinned `pip install ruff` re-introduces exactly the
|
||||
divergence the config closes, one layer up: the verdict becomes a function of *when* the job ran. Same
|
||||
argument as the `jq` pin in the same job (`ci.jq-version-contract`), and the same consequence — a bump
|
||||
argument as the `jq` pin in the same job (`ci.jq-accepted-version-set`), and the same consequence — a bump
|
||||
is a PR someone reads. `pytest` and `pyyaml` in the same job stay unpinned, and the
|
||||
asymmetry is the point rather than an oversight: a pytest release does not add assertions to your
|
||||
suite, a ruff release adds rules to your lint.
|
||||
|
||||
@@ -56,7 +56,7 @@ both reds were real:
|
||||
PyYAML). It passed locally only because the author's machine had PyYAML.
|
||||
2. A **fail-open in the merge-consent gate itself.** `jq -e` over EMPTY input exits 4 on jq ≥ 1.7
|
||||
but **0 on jq 1.6** (verified against both binaries), and the docs-only pagination guard leaned
|
||||
on that exit status to reject a transport failure. On jq 1.6 — which the CI runner ships — the
|
||||
on that exit status to reject a transport failure. On jq 1.6 — which the CI runner shipped on 2026-07-26 — the
|
||||
failed page passed the guard, the loop walked past it, the next page legitimately returned `[]`,
|
||||
and `files_complete=yes` was set over a PARTIAL list: the docs-only exemption firing over unread
|
||||
pages that may be pure code. Fixed by rejecting an empty body explicitly instead of inferring it
|
||||
@@ -68,7 +68,7 @@ on a developer Mac (jq 1.8) — only in CI, where the suite had never run. Two f
|
||||
the same shape (a `contains("\u0000")` false positive and a colliding parse-error exit code) turned
|
||||
up the same day; the durable contract that came out of all three — the exact constructs to avoid, and
|
||||
why `jq-preflight.sh` pins in `script-tests` but only floors the version in `review-verdict.yml` — is
|
||||
recorded once, in `ci.jq-version-contract` (`docs/decisions/records/ci/jq-version-contract.md`), and
|
||||
recorded once, in `ci.jq-accepted-version-set` (`docs/decisions/records/ci/jq-accepted-version-set.md`), and
|
||||
is not restated here.
|
||||
|
||||
An independent cross-family review of that fix then found **two further fail-opens in the same
|
||||
|
||||
@@ -131,7 +131,7 @@ Two reasons it read as deliberate rather than broken, both worth carrying forwar
|
||||
other.
|
||||
|
||||
The type is now read as a value (`case` over `jq -r 'type'`) rather than through `jq -e`, whose
|
||||
exit-status semantics already bit this workflow at jq 1.6 (`ci.jq-version-contract`), and both `null`
|
||||
exit-status semantics already bit this workflow at jq 1.6 (`ci.jq-accepted-version-set`), and both `null`
|
||||
and `[]` are read as the same EMPTY shape (which since #870 ends the walk only at page 1 or at the
|
||||
page cap, not in between). `test_the_fence_TRUSTS_the_count_and_POSTS_when_the_timeline_terminates`
|
||||
is parameterised over both shapes and asserts the POSTED STATUS rather than the log line — on the real
|
||||
|
||||
@@ -5,9 +5,9 @@ status: active
|
||||
since: '2026-09-04'
|
||||
supersedes: none
|
||||
superseded-by: none
|
||||
rule: 'The repository carries one tag-protection rule: `name_pattern: v*`, `whitelist_usernames: ["timothy"]`. This matters because `docker-build.yml` triggers on `push.tags: [v*]` and a `v*` tag push BUILDS AND PUBLISHES the `:prod` image — the tag the `jazz-media` Komodo stack follows. `release.main-direct-push-disabled` closed every route to `main`, and left this one open: a tag push is a separate mechanism that branch protection does not govern, so it was the cheapest remaining single act that reaches production. It buys NOTHING against the admin actor (`timothy` is on the whitelist, and no self-administered control bounds an owner) and everything it buys is against `renovate`, the repo''s other write-capable account (`permission: write`, `is_admin: false` from `GET /repos/timothy/ersatztv/collaborators/renovate/permission`, measured in #853 2026-08-30 and re-read 2026-09-04): the H10 verdict gate and the Renovate exemption rule bound what that bot can get MERGED, and nothing bounded what it could TAG. The rule''s failure mode is a BROKEN RELEASE CUT, so it is verified in both directions before it is trusted: the negative half is done and recorded below; the POSITIVE half — a real `v*` release-cut tag push by `timothy` still succeeding — is DEFERRED to the operator''s next real cut, because pushing a `v*` tag from any working session publishes `:prod` and a release is the operator''s act, not a verification step. Treat that as OPEN until a cut lands. There is no committed mirror of this rule: it lives in Gitea, and this record''s `mechanics:` carries the exact call and the read-back, the same shape `release.main-direct-push-disabled` uses.'
|
||||
rule: 'The repository carries one tag-protection rule: `name_pattern: v*`, `whitelist_usernames: ["timothy"]`. This matters because `docker-build.yml` triggers on `push.tags: [v*]` and a `v*` tag push BUILDS AND PUBLISHES the `:prod` image — the tag the `jazz-media` Komodo stack follows. `release.main-direct-push-disabled` closed every route to `main`, and left this one open: a tag push is a separate mechanism that branch protection does not govern, so it was the cheapest remaining single act that reaches production. It buys NOTHING against the admin actor (`timothy` is on the whitelist, and no self-administered control bounds an owner) and everything it buys is against `renovate`, the repo''s other write-capable account (`permission: write`, `is_admin: false` from `GET /repos/timothy/ersatztv/collaborators/renovate/permission`, measured in #853 2026-08-30 and re-read 2026-09-04): the H10 verdict gate and the Renovate exemption rule bound what that bot can get MERGED, and nothing bounded what it could TAG. The rule''s failure mode is a BROKEN RELEASE CUT, so it is verified in both directions before it is trusted: both halves are done and recorded below: the negative half on 2026-09-04, and the POSITIVE half — a real `v*` release-cut tag push by `timothy` succeeding with the rule in force — on the v26.16.0 cut, 2026-10-27. There is no committed mirror of this rule: it lives in Gitea, and this record''s `mechanics:` carries the exact call and the read-back, the same shape `release.main-direct-push-disabled` uses.'
|
||||
signals: 'tag_protections empty, v* tag push publishes prod, release cut blocked, renovate can push a tag, tag protection whitelist, release operator whitelist, prod image published by tag, deferred positive verification · paths: `.gitea/workflows/docker-build.yml` · issues: #885, #853, #743, #698, #742'
|
||||
mechanics: 'Applied 2026-09-04 with `POST /api/v1/repos/timothy/ersatztv/tag_protections` body `{"name_pattern": "v*", "whitelist_usernames": ["timothy"]}`, which returned `{"id": 1, "name_pattern": "v*", "whitelist_usernames": ["timothy"], "whitelist_teams": [], "created_at": "2026-09-04T23:18:50+02:00"}`. Read back the same day with `GET .../tag_protections` -> the one rule, fields as echoed. It is reversible with `DELETE .../tag_protections/1`. NOTE THE ROUTE NAME: `/repos/{owner}/{repo}/tags/protection` is NOT it — measured 2026-09-04, that path resolves `protection` as a TAG NAME and answers 404 `{"message": "tag doesn''t exist: protection"}`. Before this the endpoint returned `[]` (measured #743 2026-08-24 and again 2026-09-04). NEGATIVE VERIFICATION available from this session and performed: a NON-`v*` tag (`probe-885-tagpush`) pushed by `timothy` succeeded and was deleted, confirmed gone by `git ls-remote --tags` — so tag pushes still work at all and the rule is not mis-scoped to `*`. NOT VERIFIED, and it cannot be from a working session: that a real `v*` release-cut push still succeeds, and that a push by `renovate` is refused (only the `timothy` credential is available here). The first is the operator''s next cut; the second would require the bot''s credential.'
|
||||
mechanics: 'Applied 2026-09-04 with `POST /api/v1/repos/timothy/ersatztv/tag_protections` body `{"name_pattern": "v*", "whitelist_usernames": ["timothy"]}`, which returned `{"id": 1, "name_pattern": "v*", "whitelist_usernames": ["timothy"], "whitelist_teams": [], "created_at": "2026-09-04T23:18:50+02:00"}`. Read back the same day with `GET .../tag_protections` -> the one rule, fields as echoed. It is reversible with `DELETE .../tag_protections/1`. NOTE THE ROUTE NAME: `/repos/{owner}/{repo}/tags/protection` is NOT it — measured 2026-09-04, that path resolves `protection` as a TAG NAME and answers 404 `{"message": "tag doesn''t exist: protection"}`. Before this the endpoint returned `[]` (measured #743 2026-08-24 and again 2026-09-04). NEGATIVE VERIFICATION available from this session and performed: a NON-`v*` tag (`probe-885-tagpush`) pushed by `timothy` succeeded and was deleted, confirmed gone by `git ls-remote --tags` — so tag pushes still work at all and the rule is not mis-scoped to `*`. POSITIVE VERIFICATION, 2026-10-27: the v26.16.0 release-cut push (`git push origin v26.16.0` as `timothy`, tag on `1f7b763a7`) succeeded and `git ls-remote --tags origin v26.16.0` read it back; `GET .../tag_protections` the same day returned rule `id: 1` unchanged (`updated_at` 2026-09-04T23:18:50+02:00), so the push ran WITH the rule in force, not past a deleted one. STILL NOT VERIFIED: that a push by `renovate` is refused, which needs the bot''s credential (only `timothy`''s is available here).'
|
||||
---
|
||||
|
||||
**What this closes.** `release.main-direct-push-disabled` made every change to `main` go through a PR
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
---
|
||||
key: scan.mediaserver-statistics-from-server
|
||||
title: 2026-10-21 — A media-server scanner reads a version's statistics from the server, never from the library-items projection (#924)
|
||||
status: active
|
||||
since: '2026-10-21'
|
||||
supersedes: none
|
||||
superseded-by: none
|
||||
rule: 'Every media-server scanner writes a head `MediaVersion`''s statistics (streams, width/height, aspect, frame rate) from what the server reports per item — `GetMediaServerStatistics` (Jellyfin/Emby playback info) or the metadata call that carries them (`ServerReturnsStatisticsWithMetadata`, Plex) — and NEVER writes the library-items projection''s version, whose `Streams` list is empty. When the server cannot supply them, nothing is written. In `MediaServerMusicVideoLibraryScanner` the hook is `abstract`, not a virtual returning `None`.'
|
||||
signals: 'zero MediaStream rows, Width/Height 0, Sequence contains no matching element, SelectVideoStream, playback info, GetPlaybackInfo, projection Streams = [], deep scan to heal, music videos will not play · paths: `MediaServerMusicVideoLibraryScanner.UpdateStatistics`, `JellyfinMusicVideoLibraryScanner.GetMediaServerStatistics`, `MediaServerMovieLibraryScanner.UpdateStatistics`, `IMetadataRepository.UpdateStatistics`, `FFmpegStreamSelector.SelectVideoStream` · issues: #924, #496'
|
||||
mechanics: '`JellyfinMusicVideoLibraryScannerTests` (`ScanLibrary_Should_Store_PlaybackInfo_Statistics_For_A_New_MusicVideo`, `..._Deep_Scan_Should_Read_Statistics_For_An_Unchanged_MusicVideo_Without_Streams`, `..._Should_Read_Statistics_For_A_Returning_Unavailable_MusicVideo_Without_Streams`, `..._Should_Keep_Stored_Statistics_When_PlaybackInfo_Fails_On_Rescan`, `..._Should_Refresh_Statistics_And_Projection_Chapters_When_The_Etag_Changes`, against the real `MetadataRepository`); `FFmpegStreamSelectorTests.SelectVideoStream`'
|
||||
---
|
||||
|
||||
The Jellyfin and Emby library-items projections build every movie, episode and music-video version with
|
||||
`Streams = []` and no dimensions; only the per-item playback info carries them. The movie and TV scanners fetch
|
||||
statistics when the item is new, changed, deep-scanned or has no stored streams. The music-video scanner wrote
|
||||
the projection's version instead, so every Jellyfin music video was stored without streams and
|
||||
`FFmpegStreamSelector.SelectVideoStream` threw on play. Measured on prod 2026-10-21 (#924): the Jellyfin "Music Videos" library (id 16), 1520 of
|
||||
1520 music-video versions with zero `MediaStream` rows (movies 0, episodes 4).
|
||||
|
||||
- **Writing the projection is worse than writing nothing.** `IMetadataRepository.UpdateStatistics` reconciles
|
||||
the stored streams against the version it is handed, so an empty stream list REMOVES every stream an existing
|
||||
row has. A failed statistics read therefore leaves the row untouched, matching the movie scanner.
|
||||
- **The refresh condition is the movie scanner's**: `deepScan || IsAdded || etag changed || no stored streams`.
|
||||
For music videos `IsAdded` is implied by the empty-streams arm while the projection carries no streams; it is
|
||||
kept so the two conditions read the same.
|
||||
- **`abstract`, not virtual-`None`.** The movie base defaults `GetMediaServerStatistics` to `None`; Jellyfin and
|
||||
Emby override it, and Plex takes statistics from its metadata call (`ServerReturnsStatisticsWithMetadata`).
|
||||
Music videos have one implementation, and a `None` default is the defect itself, so a future Emby/Plex
|
||||
music-video scanner must answer the question to compile.
|
||||
- **Chapters come from the incoming projection**, not from the stored item as in
|
||||
`JellyfinMovieLibraryScanner`: `JellyfinMusicVideoRepository.GetByItemId` does not eager-load chapters, so the
|
||||
stored item has none to write.
|
||||
- **Healing stored rows takes a deep scan.** A regular scan skips an item whose etag is unchanged before the
|
||||
refresh condition is reached, so rows already stored without streams stay that way until their library is
|
||||
deep-scanned or the server changes the item. On a live instance that scan is a channel operation
|
||||
(media-management), not part of shipping the fix.
|
||||
- **The player names the gap.** `SelectVideoStream` on a version with no video stream fails with the version id,
|
||||
its file and the remediation (rescan; deep scan for a media-server library) instead of `First()`'s bare
|
||||
`Sequence contains no matching element`.
|
||||
- **Known residual, shared with movies:** when the server cannot answer, the item's etag is still recorded, so a
|
||||
transient playback-info failure on a new or changed item leaves it without streams until the next deep scan or
|
||||
server-side change. Not changed here: skipping the etag write would retry the item on every scan, and the same
|
||||
trade-off already holds for movies and episodes.
|
||||
@@ -320,7 +320,7 @@ different fixes, and conflating them is why several of these recurred.
|
||||
| **B** | **Every guard ships with a proof it can go red: delete or disarm *that guard alone* and the suite must fail.** Not "a test exists" — a mutation. | Family B: 9 records (13%) | **rule-present-unenforced** — stated in #685's record and in project memory, enforced nowhere |
|
||||
| **C** | **Dedup by construction.** When two copies of one rule exist, delete one and have both callers invoke it — *unless* deriving the list would feed the judged input back into the judge, which is #711's reasoned exception. | Family C: 9 records (13%) | **rule-missing** |
|
||||
| **D** | **Anything read-then-written against live Gitea/remote state pins a version or uses compare-and-set.** See the honesty note below — this is a fix pattern, and its *detector* is detector A applied to an enumerated inventory of such sites. | Family D: 5 records (7%) | **rule-missing** (partially addressed for the merge path by the per-sha required check from #622) |
|
||||
| **E** | **Run the check under the interpreter/provider it will actually run under**, and preflight-log the version. | Environment divergence: 7 records | **rule-present** and working — `scripts/jq-preflight.sh` + `ci.jq-version-contract` closed the jq axis after #643/#647/#648. **Unclosed axes**: SQLite-vs-MySQL query semantics (#668), GPU generation (#505), CI-VM speed (#512) |
|
||||
| **E** | **Run the check under the interpreter/provider it will actually run under**, and preflight-log the version. | Environment divergence: 7 records | **rule-present** and working — `scripts/jq-preflight.sh` + `ci.jq-accepted-version-set` closed the jq axis after #643/#647/#648. **Unclosed axes**: SQLite-vs-MySQL query semantics (#668), GPU generation (#505), CI-VM speed (#512) |
|
||||
| **F** | **Test the DENY path with the production config value.** #756's lesson generalised: a fixture that omits a field tests only the default, so a fail-open in the production value stays invisible. Parametrise the whole matrix, including "config absent". | Fail-open-by-default: ~5 records | **rule-missing** |
|
||||
| **G** | **Full-replace endpoints assert their complete field list in a test**, and prefer reconcile-by-id over delete-and-reinsert where child state exists. | Destructive replace: ~3 records | **rule-missing** |
|
||||
|
||||
|
||||
@@ -100,8 +100,8 @@ classifier="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/branch-rule-classi
|
||||
# comparison was made", which is the truth for a non-array body.
|
||||
#
|
||||
# "To produce a finding" rather than "before the classifier sees it", and the distinction is the
|
||||
# `ci.jq-version-contract` gap (ersatztv#643): on jq 1.6 — the version `scripts/jq-preflight.sh`
|
||||
# pins for `script-tests` — `jq -e` over EMPTY input exits 0, so an empty 200 body is not diverted
|
||||
# `ci.jq-accepted-version-set` gap (ersatztv#643): on jq 1.6 — the floor `scripts/jq-preflight.sh`
|
||||
# admits for `script-tests` — `jq -e` over EMPTY input exits 0, so an empty 200 body is not diverted
|
||||
# here and does reach the classifier. Measured on both 1.6 and 1.8.2: the class word is `unreadable`
|
||||
# either way, because the classifier then produces nothing and the catch-all answers. The outcome is
|
||||
# version-independent; only the mechanism differs, so the comment states the outcome.
|
||||
|
||||
@@ -139,8 +139,8 @@ comments=$(cat)
|
||||
# NOTE the NUL test is `explode | index(0)`, NOT `contains("\u0000")` (ersatztv#647). On jq 1.6 the
|
||||
# escape truncates the literal to the EMPTY string, and every string contains "" — so that form
|
||||
# returns true for ALL input, making this guard reject every comment body as malformed. Verified
|
||||
# against both binaries: 1.6 says true for "hello", 1.7+ says false. The CI runner ships jq 1.6, so
|
||||
# the whole verdict classifier was inert there. `explode | index(0)` agrees on both.
|
||||
# against both binaries: 1.6 says true for "hello", 1.7+ says false. A CI runner reporting jq 1.6 (the floor)
|
||||
# had the whole verdict classifier inert. `explode | index(0)` agrees on both.
|
||||
#
|
||||
# `Review<NUL>-verdict: MERGEABLE @ <head>` would arrive at the matcher as a valid verdict line —
|
||||
# text that is not a verdict silently becoming one.
|
||||
|
||||
+47
-24
@@ -2,9 +2,10 @@
|
||||
# Make the jq version a job's shell gates run under OBSERVABLE, and any drift LOUD.
|
||||
#
|
||||
# ersatztv#648. Every shell gate in this repo is authored and tested on a developer Mac shipping
|
||||
# jq 1.8.x. The CI runner ships jq 1.6. Nothing pinned or checked that, and until ersatztv#631 the one
|
||||
# thing that could have noticed (scripts/tests/) never ran on the runner. Three independent divergences
|
||||
# surfaced in a single day:
|
||||
# jq 1.8.x. The CI runners shipped jq 1.6 when this was written (2026-07-26); on 2026-10-28 both
|
||||
# `small` runners reported jq 1.7 (server-management#1080). Nothing pinned or checked that, and until
|
||||
# ersatztv#631 the one thing that could have noticed (scripts/tests/) never ran on the runner. Three
|
||||
# independent divergences surfaced in a single day:
|
||||
#
|
||||
# ersatztv#643 `jq -e` over EMPTY input -> exit 4 on 1.8, exit 0 on 1.6 (a transport failure
|
||||
# passed the docs-only pagination guard)
|
||||
@@ -20,11 +21,15 @@
|
||||
# WHY A FLOOR AND NOT A PIN EVERYWHERE. The obvious fix — bake a pinned jq into the CI toolchain image
|
||||
# (docker/ci/Dockerfile) — provably does NOT cover the gate that actually broke. `.gitea/workflows/
|
||||
# review-verdict.yml` is `runs-on: small`, carries no toolchain-image pin, and per `ci.small-lane-git-only`
|
||||
# the small lane is git-only. It therefore gets the HOST's jq 1.6 no matter what the image contains.
|
||||
# the small lane is git-only. It therefore gets the HOST's jq, whatever the image contains.
|
||||
# That was checked, not assumed (ersatztv#648's first Done-when box).
|
||||
#
|
||||
# So the contract is the other way round: 1.6 is the FLOOR every gate must work on, and it is the
|
||||
# runner's own jq that provides the 1.6 coverage `scripts/tests/` runs under.
|
||||
# So the contract is the other way round: 1.6 is the FLOOR every gate must work on. Both `small`
|
||||
# runners map to the floating runner-images:ubuntu-latest image with no digest pin, so each host keeps
|
||||
# whatever it last pulled (2026-10-28: both measured hosts reported 1.7; a host may hold an older
|
||||
# pull). No known runner reports 1.6, so real-jq 1.6 coverage of `scripts/tests/` is currently absent;
|
||||
# 1.6 stays accepted only because the tag is unpinned. server-management#1080 proposes a fleet digest
|
||||
# pin; when it settles one version, narrow the `--expect` set in pr-checks.yml to that version.
|
||||
#
|
||||
# TWO MODES, deliberately asymmetric:
|
||||
#
|
||||
@@ -34,20 +39,21 @@
|
||||
# `main`, so a hard pin there would turn any jq upgrade on the runner into a
|
||||
# repo-wide merge deadlock. Observability without a deadlock risk.
|
||||
#
|
||||
# --expect X.Y Additionally assert the version is exactly X.Y, and FAIL if not. Used by the
|
||||
# `script-tests` job. This is the tripwire: `scripts/tests/` currently exercises
|
||||
# the 1.6 path only because the runner happens to ship 1.6. If the runner were
|
||||
# upgraded, that coverage would vanish SILENTLY and the whole class of bug above
|
||||
# would go untested again. Going red forces a human to decide — re-pin, or add a
|
||||
# real 1.6 matrix leg — rather than letting the coverage evaporate unnoticed.
|
||||
# --expect X.Y[,X.Y] Additionally assert the version is exactly one member of the comma-separated
|
||||
# set, and FAIL if not. Used by the `script-tests` job with `1.6,1.7` (since
|
||||
# 2026-10-29, until server-management#1080 pins the fleet). This is the
|
||||
# tripwire: a version outside the set means the runner image moved on and the
|
||||
# coverage the suite assumed has changed, so going red forces a human to decide
|
||||
# rather than letting it change unnoticed.
|
||||
#
|
||||
# Usage: jq-preflight.sh [--expect <major.minor>]
|
||||
# Usage: jq-preflight.sh [--expect <major.minor>[,<major.minor>...]]
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# The lowest jq every shell gate in this repo must run correctly on. Do not raise this without
|
||||
# confirming the CI runner has actually been upgraded first — the runner, not the dev Mac, is the
|
||||
# binding constraint.
|
||||
# The lowest jq every shell gate in this repo must run correctly on. Do not raise this until every
|
||||
# runner that can take a job is confirmed above it — the runner fleet, not the dev Mac, is the binding
|
||||
# constraint, and while the runner image tag is unpinned (server-management#1080) a host may still hold
|
||||
# a 1.6 pull.
|
||||
MIN_VERSION="1.6"
|
||||
|
||||
expect=""
|
||||
@@ -156,18 +162,35 @@ if [ "$major" -lt "$min_major" ] || { [ "$major" -eq "$min_major" ] && [ "$minor
|
||||
fi
|
||||
|
||||
if [ -n "$expect" ]; then
|
||||
if [ "${major}.${minor}" != "$expect" ]; then
|
||||
echo "jq-preflight: expected jq ${expect}, found ${major}.${minor}." >&2
|
||||
# --expect is a comma-separated SET. Validate the WHOLE string first: every member must be a plain
|
||||
# <digits>.<digits>, so an empty member (`1.6,`, `,1.6`, `1.6,,1.7`) or a free-form one is a usage
|
||||
# error, never a member that silently matches nothing (or, via a later split, something it should not).
|
||||
member_re='[0-9]{1,9}\.[0-9]{1,9}'
|
||||
if [[ ! "$expect" =~ ^${member_re}(,${member_re})*$ ]]; then
|
||||
echo "jq-preflight: --expect must be <major.minor>[,<major.minor>...]; got '${expect}'" >&2
|
||||
exit 2
|
||||
fi
|
||||
matched=""
|
||||
IFS=',' read -r -a expect_set <<<"$expect"
|
||||
for member in "${expect_set[@]}"; do
|
||||
if [ "$member" = "${major}.${minor}" ]; then
|
||||
matched="yes"
|
||||
fi
|
||||
done
|
||||
if [ -z "$matched" ]; then
|
||||
echo "jq-preflight: expected jq in {${expect}}, found ${major}.${minor}." >&2
|
||||
echo "" >&2
|
||||
echo "This is a TRIPWIRE, not a defect in your change (ersatztv#648). scripts/tests/ was pinned to" >&2
|
||||
echo "jq ${expect} because that is what this runner shipped; it now reports ${major}.${minor}. The ${expect}" >&2
|
||||
echo "coverage the suite assumed has therefore just disappeared, silently — and jq 1.7 altered NUL" >&2
|
||||
echo "handling, exit codes, @base64d and number precision, every one of which a gate here depends on." >&2
|
||||
echo "This is a TRIPWIRE, not a defect in your change (ersatztv#648, #935). scripts/tests/ is run" >&2
|
||||
echo "under whichever jq the runner that took this job ships, and the accepted set is {${expect}}." >&2
|
||||
echo "This runner reports ${major}.${minor}, so the jq the suite assumed has changed — jq 1.7 altered" >&2
|
||||
echo "NUL handling, exit codes, @base64d and number precision, every one of which a gate here" >&2
|
||||
echo "depends on." >&2
|
||||
echo "" >&2
|
||||
echo "Decide explicitly, then update the --expect value in .gitea/workflows/pr-checks.yml:" >&2
|
||||
echo " * re-pin to the new version after re-reading docs/ci-cd.md -> 'The jq contract', or" >&2
|
||||
echo " * widen or re-pin the set after re-reading docs/ci-cd.md -> 'The jq contract'" >&2
|
||||
echo " (server-management#1080 is the fleet digest pin that narrows it to one version), or" >&2
|
||||
echo " * add a real matrix leg that runs the suite under ${MIN_VERSION} as well." >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "jq-preflight: version matches the expected pin (${expect})."
|
||||
echo "jq-preflight: version ${major}.${minor} is a member of the expected set (${expect})."
|
||||
fi
|
||||
|
||||
@@ -79,7 +79,8 @@
|
||||
# ETV_GITEA_URL | GITEA_BASE_URL -> API base; defaults to the homelab Gitea. A value ending in
|
||||
# /api/v1 is used as-is, otherwise /api/v1 is appended.
|
||||
#
|
||||
# jq COMPATIBILITY (ersatztv#648). This runs on the CI runner, which ships **jq 1.6**, while it is
|
||||
# jq COMPATIBILITY (ersatztv#648). This runs on CI runners whose jq is unpinned (1.7 measured 2026-10-28;
|
||||
# floor 1.6), while it is
|
||||
# authored on Macs shipping 1.8.x. It is therefore written to the 1.6-compatible subset:
|
||||
# * never rely on `jq -e`'s exit status over EMPTY input — 1.6 exits 0 where >=1.7 exits 4, which is
|
||||
# precisely the fail-open that ersatztv#647 found live in the enforced gate. Emptiness is always
|
||||
@@ -180,7 +181,7 @@ while [ "$page" -le "$MAX_PAGES" ]; do
|
||||
raw=$(gq "repos/$owner/$repo/pulls/$pr/files?limit=${PAGE_SIZE}&page=${page}")
|
||||
|
||||
# An EMPTY body is rejected in SHELL, before jq sees it. `jq -e` over empty input exits 4 on
|
||||
# jq >= 1.7 but 0 on jq 1.6, and the runner ships 1.6 — leaving this to jq's exit status is the
|
||||
# jq >= 1.7 but 0 on jq 1.6, and a runner may still report 1.6 (unpinned image) — leaving this to jq's exit status is the
|
||||
# exact fail-open ersatztv#647 found in the enforced copy. A transport failure must never
|
||||
# masquerade as a legitimate short final page.
|
||||
if [ -z "${raw//[[:space:]]/}" ]; then
|
||||
|
||||
+102
-10
@@ -10,7 +10,8 @@
|
||||
# injection reflections, known-CVE fingerprints. It is a #197 exit criterion — a HARD GATE before remote
|
||||
# exposure — and should be re-run each release (see docs/ci-cd.md → Security scanning).
|
||||
#
|
||||
# It runs on the docker host (bumblebee) against a THROWAWAY container booted from the image under test —
|
||||
# It runs on a docker host with registry access and semgrep on its non-interactive PATH (docs/ci-cd.md →
|
||||
# Security scanning → Where/when), against a THROWAWAY container booted from the image under test —
|
||||
# NEVER the deployed prod (or test) container: the authenticated active scan sends attack payloads to
|
||||
# write endpoints (creating/deleting rows, triggering playout builds), so it must target a disposable
|
||||
# instance with a fresh empty config volume. The container is torn down on exit.
|
||||
@@ -21,7 +22,7 @@
|
||||
# PORT host port for the throwaway app (default: 8411)
|
||||
# Env:
|
||||
# ETV_SCAN_OUT host dir for reports (default: /tmp/etv-scan-out)
|
||||
# ETV_SCAN_SKIP_SEMGREP=1 skip the SAST pass (DAST only)
|
||||
# ETV_SCAN_SKIP_SEMGREP=1 skip the SAST pass (DAST only) — the ONLY skip; a missing semgrep otherwise exits 2
|
||||
#
|
||||
# Reports land in $ETV_SCAN_OUT: zap-api-report.{html,md,json}. zap-api-scan.py's exit code is NOT a
|
||||
# simple pass/fail — its wrapper contract (ersatztv#338, found via the #335 exact-image release scan
|
||||
@@ -31,8 +32,14 @@
|
||||
# 1 FAIL — at least one FAIL-level alert — release-blocking
|
||||
# 124 our own `timeout` wrapper killed a hung wrapper (see the NOTE below) — report is still usable
|
||||
# other the scanner/tool itself errored (bad args, crash) — not a scan result at all
|
||||
# classify_zap_exit() (below) is the single place this contract is encoded; the script's own final exit
|
||||
# code reflects the CLASSIFICATION (0 for clean/WARN, 1 for FAIL/timeout/tool-error), not ZAP's raw code.
|
||||
# classify_zap_exit() (below) is the single place this contract is encoded. The SCRIPT's own exit is not
|
||||
# ZAP's raw code; every non-zero exit is release-blocking:
|
||||
# 0 ZAP clean or WARN-only, AND the SAST half ran clean (or was opted out: ETV_SCAN_SKIP_SEMGREP=1)
|
||||
# 1 ZAP FAIL / timeout / tool error, or the throwaway target never came up or its key was unreadable
|
||||
# 2 NOT ZAP's WARN: a scanner half did not run — semgrep missing (checked before the scan) or erroring
|
||||
# (after it), or docker missing. If ZAP already classified non-zero, that status stands and the
|
||||
# semgrep error is printed alongside it.
|
||||
# A setup command that fails under `set -e` (e.g. `docker pull`) exits with that command's own status.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
@@ -50,6 +57,24 @@ classify_zap_exit() {
|
||||
esac
|
||||
}
|
||||
|
||||
# require_semgrep — the SAST half is part of the gate: a missing semgrep is an ERROR naming the tool, never a
|
||||
# warning that still exits 0 (ersatztv#931 — a v26.16.0 release gate could pass with its SAST half never run).
|
||||
# The only skip is the explicit opt-out ETV_SCAN_SKIP_SEMGREP=1. Standalone so `--selftest` can exercise it.
|
||||
require_semgrep() {
|
||||
[ "${ETV_SCAN_SKIP_SEMGREP:-0}" = "1" ] && return 0
|
||||
command -v semgrep >/dev/null && return 0
|
||||
echo "error: semgrep not found on PATH — the SAST half of the release gate cannot run. Install semgrep, or set ETV_SCAN_SKIP_SEMGREP=1 to skip it explicitly (DAST only)." >&2
|
||||
return 2
|
||||
}
|
||||
|
||||
# run_semgrep ROOT — the SAST pass. Without --error semgrep exits 0 on findings (they print above), so ANY
|
||||
# non-zero exit means semgrep itself failed (bad config, registry unreachable, crash): the SAST half did not
|
||||
# run, which must fail the gate exactly like a missing semgrep (ersatztv#931). Returns semgrep's own status.
|
||||
run_semgrep() {
|
||||
semgrep scan --config p/security-audit --config p/secrets --config p/csharp \
|
||||
--exclude web/node_modules --exclude design-system --metrics off --quiet "$1"
|
||||
}
|
||||
|
||||
if [ "${1:-}" = "--selftest" ]; then
|
||||
# Lightweight regression check for the classification contract above (ersatztv#338 Done-when) — no
|
||||
# docker/ZAP required, so it can run anywhere (incl. off bumblebee, incl. in per-PR CI if ever added).
|
||||
@@ -70,8 +95,71 @@ if [ "${1:-}" = "--selftest" ]; then
|
||||
selftest_check 124 1 "TIMEOUT"
|
||||
selftest_check 3 1 "TOOL ERROR"
|
||||
selftest_check 77 1 "TOOL ERROR"
|
||||
# semgrep gate (ersatztv#931): absent -> non-zero naming the tool; opt-out -> 0; present -> 0.
|
||||
semgrep_check() {
|
||||
local label="$1" want_status="$2" want_substr="$3" got_status err
|
||||
err="$(require_semgrep 2>&1 >/dev/null)" && got_status=0 || got_status=$?
|
||||
if [ "$got_status" != "$want_status" ] || [[ "$err" != *"$want_substr"* ]]; then
|
||||
echo "SELFTEST FAIL: semgrep gate ($label) -> status=$got_status msg='$err' (want status=$want_status, msg containing '$want_substr')" >&2
|
||||
return 1
|
||||
else
|
||||
echo "selftest ok: semgrep gate ($label) -> status=$got_status"
|
||||
fi
|
||||
}
|
||||
sg_dir="$(mktemp -d)"; sg_empty="$(mktemp -d)"
|
||||
printf '#!/bin/sh\n' > "$sg_dir/semgrep"; chmod +x "$sg_dir/semgrep"
|
||||
(PATH="$sg_empty"; unset ETV_SCAN_SKIP_SEMGREP; semgrep_check "absent" 2 "semgrep") || selftest_failed=1
|
||||
(PATH="$sg_empty"; ETV_SCAN_SKIP_SEMGREP=0; semgrep_check "absent, skip=0" 2 "semgrep") || selftest_failed=1
|
||||
(PATH="$sg_empty"; ETV_SCAN_SKIP_SEMGREP=1; semgrep_check "absent, explicit opt-out" 0 "") || selftest_failed=1
|
||||
(PATH="$sg_dir"; unset ETV_SCAN_SKIP_SEMGREP; semgrep_check "present" 0 "") || selftest_failed=1
|
||||
# End-to-end through the real main path (pins the call sites, not just the helper). Absent semgrep + no
|
||||
# docker on PATH: must exit 2 with the semgrep message BEFORE the docker check (which also exits 2).
|
||||
out="$(env -u ETV_SCAN_SKIP_SEMGREP PATH="$sg_empty" "$BASH" "${BASH_SOURCE[0]}" 2>&1)" && rc=0 || rc=$?
|
||||
if [ "$rc" = 2 ] && [[ "$out" == *"error: semgrep not found"* ]]; then
|
||||
echo "selftest ok: script, semgrep absent -> exit 2 naming semgrep"
|
||||
else
|
||||
echo "SELFTEST FAIL: script with semgrep absent -> rc=$rc out='$out' (want rc=2 naming semgrep)" >&2; selftest_failed=1
|
||||
fi
|
||||
out="$(ETV_SCAN_SKIP_SEMGREP=1 PATH="$sg_empty" "$BASH" "${BASH_SOURCE[0]}" 2>&1)" && rc=0 || rc=$?
|
||||
if [ "$rc" = 2 ] && [[ "$out" == *"docker not found"* ]] && [[ "$out" != *"semgrep not found"* ]]; then
|
||||
echo "selftest ok: script, explicit opt-out -> gate skipped (stops at docker check)"
|
||||
else
|
||||
echo "SELFTEST FAIL: script with opt-out -> rc=$rc out='$out' (want to pass the semgrep gate)" >&2; selftest_failed=1
|
||||
fi
|
||||
# Full runs through the MAIN-path SAST block, with shimmed docker/curl/timeout (ZAP reports clean, so the
|
||||
# script's exit is decided by the semgrep half alone). The semgrep shim records each call in
|
||||
# $ETV_SCAN_OUT/semgrep.called and exits $SG_SHIM_RC, so a case can assert both the exit and whether
|
||||
# SAST actually ran. These pin the main-path opt-out test and the sg_rc check, not just the helper.
|
||||
sg_shim="$(mktemp -d)"; sg_out="$(mktemp -d)"
|
||||
printf '#!/bin/sh\n[ "$1" = exec ] && echo shimkey\nexit 0\n' > "$sg_shim/docker"
|
||||
printf '#!/bin/sh\necho 200\n' > "$sg_shim/curl"
|
||||
printf '#!/bin/sh\nshift\nexec "$@"\n' > "$sg_shim/timeout"
|
||||
printf '#!/bin/sh\necho called >> "$ETV_SCAN_OUT/semgrep.called"\n[ "${SG_SHIM_RC:-0}" = 0 ] || echo "fake semgrep: registry unreachable" >&2\nexit "${SG_SHIM_RC:-0}"\n' > "$sg_shim/semgrep"
|
||||
chmod +x "$sg_shim"/*
|
||||
# full_check LABEL SG_SHIM_RC SKIP WANT_RC WANT_CALLED WANT_SUBSTR — SKIP is "unset" or the value to export.
|
||||
full_check() {
|
||||
local label="$1" sg_shim_rc="$2" skip="$3" want_rc="$4" want_called="$5" want_substr="$6" out rc called
|
||||
rm -f "$sg_out/semgrep.called"
|
||||
if [ "$skip" = unset ]; then
|
||||
out="$(env -u ETV_SCAN_SKIP_SEMGREP SG_SHIM_RC="$sg_shim_rc" PATH="$sg_shim:$PATH" ETV_SCAN_OUT="$sg_out" "$BASH" "${BASH_SOURCE[0]}" 2>&1)" && rc=0 || rc=$?
|
||||
else
|
||||
out="$(ETV_SCAN_SKIP_SEMGREP="$skip" SG_SHIM_RC="$sg_shim_rc" PATH="$sg_shim:$PATH" ETV_SCAN_OUT="$sg_out" "$BASH" "${BASH_SOURCE[0]}" 2>&1)" && rc=0 || rc=$?
|
||||
fi
|
||||
if [ -e "$sg_out/semgrep.called" ]; then called=1; else called=0; fi
|
||||
if [ "$rc" = "$want_rc" ] && [ "$called" = "$want_called" ] && [[ "$out" == *"$want_substr"* ]]; then
|
||||
echo "selftest ok: script, $label -> exit $rc, semgrep called=$called"
|
||||
else
|
||||
echo "SELFTEST FAIL: script, $label -> rc=$rc called=$called (want rc=$want_rc called=$want_called, output containing '$want_substr')" >&2
|
||||
selftest_failed=1
|
||||
fi
|
||||
}
|
||||
full_check "semgrep succeeds" 0 unset 0 1 "==> semgrep SAST cross-check"
|
||||
full_check "semgrep succeeds, skip=0" 0 0 0 1 "==> semgrep SAST cross-check"
|
||||
full_check "semgrep fails" 2 unset 2 1 "error: semgrep failed (exit 2)"
|
||||
full_check "explicit opt-out, semgrep present" 0 1 0 0 "semgrep SAST skipped (ETV_SCAN_SKIP_SEMGREP=1)"
|
||||
rm -rf "$sg_dir" "$sg_empty" "$sg_shim" "$sg_out"
|
||||
if [ "$selftest_failed" = 0 ]; then
|
||||
echo "selftest: all ZAP exit classifications OK"; exit 0
|
||||
echo "selftest: all ZAP exit classifications and semgrep-gate cases OK"; exit 0
|
||||
else
|
||||
echo "selftest: FAILURES ABOVE" >&2; exit 1
|
||||
fi
|
||||
@@ -83,6 +171,9 @@ OUT="${ETV_SCAN_OUT:-/tmp/etv-scan-out}"
|
||||
TARGET_NAME="etv-secscan-target"
|
||||
ZAP_IMAGE="ghcr.io/zaproxy/zaproxy:stable"
|
||||
|
||||
# Fail fast, before the image pull and the multi-minute ZAP run, if the SAST half cannot run.
|
||||
require_semgrep || exit $?
|
||||
|
||||
command -v docker >/dev/null || { echo "error: docker not found — run this on the docker host (bumblebee)" >&2; exit 2; }
|
||||
|
||||
CFG="$(mktemp -d /tmp/etv-secscan-cfg-XXXXXX)"
|
||||
@@ -145,14 +236,15 @@ echo "==> ZAP result: $ZAP_CLASS_MSG (raw exit $zap_rc)"
|
||||
|
||||
if [ "${ETV_SCAN_SKIP_SEMGREP:-0}" = "1" ]; then
|
||||
echo "==> semgrep SAST skipped (ETV_SCAN_SKIP_SEMGREP=1)"
|
||||
elif ! command -v semgrep >/dev/null; then
|
||||
echo "WARN: semgrep not installed — SAST cross-check SKIPPED (install semgrep for full coverage)" >&2
|
||||
else
|
||||
echo "==> semgrep SAST cross-check (repo root)"
|
||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
semgrep scan --config p/security-audit --config p/secrets --config p/csharp \
|
||||
--exclude web/node_modules --exclude design-system --metrics off --quiet \
|
||||
"$REPO_ROOT" || echo "(semgrep reported findings — triage in the report above)"
|
||||
sg_rc=0
|
||||
run_semgrep "$REPO_ROOT" || sg_rc=$?
|
||||
if [ "$sg_rc" != 0 ]; then
|
||||
echo "error: semgrep failed (exit $sg_rc) — the SAST half did not run; this is a gate failure, not a finding" >&2
|
||||
[ "$zap_class_status" != 0 ] || zap_class_status=2
|
||||
fi
|
||||
fi
|
||||
|
||||
exit "$zap_class_status"
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
"""Tests for `scripts/jq-preflight.sh` — the jq version contract (ersatztv#648).
|
||||
|
||||
The axis this guards. Every shell gate in this repo is authored on a Mac shipping jq 1.8.x; the CI
|
||||
runner ships jq 1.6. Nothing pinned or checked that, and three independent divergences surfaced in a
|
||||
runners shipped jq 1.6 (2026-07-26) and both measured runners report 1.7 (2026-10-28). Nothing pinned
|
||||
or checked that, and three independent divergences surfaced in a
|
||||
single day — `jq -e` over empty input (exit 4 vs 0), `contains("<NUL>")` (false vs true for every
|
||||
string), and the parse-error exit code (5 vs 4, colliding with "no output"). Each was patched with a
|
||||
version-stable construct, but patching constructs one at a time leaves the AXIS untested.
|
||||
@@ -15,6 +16,7 @@ reproducible: it re-runs on every PR instead of living in one CI run's history.
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
@@ -132,13 +134,16 @@ def test_build_suffixes_still_parse_as_1_6(preflight, version_line):
|
||||
assert preflight.run("--expect", "1.6").returncode == 0, version_line
|
||||
|
||||
|
||||
ACCEPTED = "1.6,1.7" # the set pr-checks.yml passes until server-management#1080 pins the fleet
|
||||
|
||||
|
||||
def test_expect_mismatch_is_LOUD(preflight):
|
||||
"""THE TRIPWIRE. scripts/tests exercises the jq 1.6 path only because the runner ships 1.6. If
|
||||
the runner were upgraded that coverage would vanish silently, so the pin must go red instead."""
|
||||
"""THE TRIPWIRE. A version outside the accepted set must go red, so a runner image moving on
|
||||
changes the suite's jq visibly instead of silently."""
|
||||
preflight.with_jq("jq-1.7.1")
|
||||
r = preflight.run("--expect", "1.6")
|
||||
assert r.returncode == 1
|
||||
assert "expected jq 1.6, found 1.7" in r.stderr
|
||||
assert "expected jq in {1.6}, found 1.7" in r.stderr
|
||||
|
||||
|
||||
def test_expect_match_passes(preflight):
|
||||
@@ -146,6 +151,34 @@ def test_expect_match_passes(preflight):
|
||||
assert preflight.run("--expect", "1.6").returncode == 0
|
||||
|
||||
|
||||
@pytest.mark.parametrize("version_line", ["jq-1.6", "jq-1.7", "jq-1.7.1", "jq-1.6-dirty"])
|
||||
def test_the_accepted_set_passes_on_either_member(preflight, version_line):
|
||||
"""The unpinned fleet (server-management#1080): whichever version the runner holds must be green."""
|
||||
preflight.with_jq(version_line)
|
||||
r = preflight.run("--expect", ACCEPTED)
|
||||
assert r.returncode == 0, r.stderr
|
||||
assert "member of the expected set" in r.stdout
|
||||
|
||||
|
||||
@pytest.mark.parametrize("version_line", ["jq-1.8.1", "jq-1.10", "jq-2.6", "jq-1.71"])
|
||||
def test_a_version_outside_the_accepted_set_is_LOUD(preflight, version_line):
|
||||
"""The witnessed red. 1.8 is what the dev Macs ship and 1.10/1.71 are string-prefix lookalikes of
|
||||
1.7/1.6 members: membership is exact on major.minor, not a prefix or numeric-range test."""
|
||||
preflight.with_jq(version_line)
|
||||
r = preflight.run("--expect", ACCEPTED)
|
||||
assert r.returncode == 1, version_line
|
||||
assert "expected jq in {1.6,1.7}" in r.stderr
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bad", ["1.6,", ",1.6", "1.6,,1.7", "1.6 1.7", "1.x", "1", "1.6;1.7", "1.6,1.7,"])
|
||||
def test_a_malformed_expect_set_is_a_usage_error(preflight, bad):
|
||||
"""An empty or free-form member must not become a member that matches nothing (or everything)."""
|
||||
preflight.with_jq("jq-1.6")
|
||||
r = preflight.run("--expect", bad)
|
||||
assert r.returncode == 2, bad
|
||||
assert "--expect must be" in r.stderr
|
||||
|
||||
|
||||
def test_unknown_argument_is_a_usage_error(preflight):
|
||||
preflight.with_jq("jq-1.6")
|
||||
assert preflight.run("--pin", "1.6").returncode == 2
|
||||
@@ -345,6 +378,11 @@ def test_script_tests_pins_the_jq_version():
|
||||
"""
|
||||
pr_checks = (WORKFLOWS / "pr-checks.yml").read_text()
|
||||
assert "jq-preflight.sh --expect" in pr_checks, "script-tests must pin the jq version — that pin is the tripwire"
|
||||
# Exactly the accepted set until server-management#1080 settles one version (#935): not 1.6 alone
|
||||
# (red on a 1.7 runner), not a wider set (a 1.8 runner must stay red).
|
||||
assert re.search(rf"^\s*run: \./scripts/jq-preflight\.sh --expect {re.escape(ACCEPTED)}\s*$", pr_checks, re.M), (
|
||||
f"script-tests must run --expect {ACCEPTED}"
|
||||
)
|
||||
|
||||
|
||||
def test_review_verdict_never_pins_a_jq_version():
|
||||
|
||||
@@ -493,12 +493,12 @@ def test_a_SHORT_page_does_not_end_the_enumeration(enumerate_files):
|
||||
assert "ErsatzTV/Program.cs" in enumerate_files.paths()
|
||||
|
||||
|
||||
# --- The same guard, under the runner's jq 1.6 ------------------------------------------------
|
||||
# --- The same guard, under jq 1.6 ------------------------------------------------
|
||||
#
|
||||
# `test_transport_failure_mid_pagination_fails_closed` above does NOT isolate the explicit
|
||||
# `if [ -z "${raw//[[:space:]]/}" ]` clause — mutation confirms it: deleting that clause leaves
|
||||
# the whole suite green on a developer Mac, because jq 1.8 rejects empty
|
||||
# input on its own. jq 1.6 does not, and the runner ships 1.6 — so the one environment where the
|
||||
# input on its own. jq 1.6 does not, and a runner may report 1.6 — so the one environment where the
|
||||
# clause is load-bearing was the one environment with no coverage. That is the #643/#647 failure
|
||||
# class exactly, reproduced in the test suite meant to prevent it.
|
||||
#
|
||||
@@ -1840,7 +1840,7 @@ def _run_classify(
|
||||
curl.write_text(WORKFLOW_STUB_CURL)
|
||||
curl.chmod(0o755)
|
||||
if jq16:
|
||||
# Reproduce the runner's jq 1.6 (`-e` over EMPTY input exits 0, not 4). The shim is the one
|
||||
# Reproduce jq 1.6 (`-e` over EMPTY input exits 0, not 4). The shim is the one
|
||||
# already used for pr-changed-files.sh in this file, so its fidelity is covered by that
|
||||
# suite's own verify-the-verifier test.
|
||||
jq = bindir / "jq"
|
||||
@@ -2184,7 +2184,7 @@ def test_a_transport_failure_under_jq_1_6_STILL_refuses_to_classify(tmp_path):
|
||||
"""The ersatztv#647 fail-open, tested where it actually lives: jq 1.6.
|
||||
|
||||
The shell emptiness check exists because `jq -e` over EMPTY input exits 4 on jq >= 1.7 but **0 on
|
||||
jq 1.6, which is what the runner ships**. Remove that check and, on a dev Mac's 1.8, the guard
|
||||
jq 1.6, which a runner may report (the floor)**. Remove that check and, on a dev Mac's 1.8, the guard
|
||||
still fires and every test stays green — the bug is invisible locally.
|
||||
|
||||
Pinning the construct STRUCTURALLY instead, on the grounds that "no behavioural test can catch
|
||||
@@ -3345,8 +3345,8 @@ def test_count_pr_mutations_uses_NO_jq_e(tmp_path):
|
||||
|
||||
The function's own comment states the rule and the reason: `jq -e` reports the truthiness of its
|
||||
LAST OUTPUT, and over empty input jq 1.6 exits 0 where >= 1.7 exits 4 — the divergence that
|
||||
already took this workflow's enforced gate down once (ersatztv#647), on a runner that still ships
|
||||
1.6. Every tally here is therefore read with `jq -r` and range-checked in shell.
|
||||
already took this workflow's enforced gate down once (ersatztv#647), on a runner that may report
|
||||
1.6 (the floor). Every tally here is therefore read with `jq -r` and range-checked in shell.
|
||||
|
||||
This exists because the rule was BROKEN one commit after it was quoted (ersatztv#803 added a
|
||||
row-validation guard using `jq -e`, three lines below the comment forbidding it) and nothing
|
||||
@@ -3391,7 +3391,7 @@ def test_count_pr_mutations_uses_NO_jq_e(tmp_path):
|
||||
offenders = [ln.strip() for ln in joined if _uses_jq_exit_status(ln)]
|
||||
assert not offenders, (
|
||||
"`jq -e` appeared inside `count_pr_mutations`, whose own comment bans it because the runner "
|
||||
"ships jq 1.6 and `-e`'s exit status over empty input diverges there (ersatztv#647/#803). "
|
||||
"may report jq 1.6 and `-e`'s exit status over empty input diverges there (ersatztv#647/#803). "
|
||||
"Read the value with `jq -r` and range-check it in shell instead:\n " + "\n ".join(offenders)
|
||||
)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user