ci(390): shared CI toolchain image + drop 110s apt-ffmpeg + rebalance runner lanes #399
@@ -0,0 +1,136 @@
|
||||
name: Build CI Toolchain Image
|
||||
|
||||
# Builds the shared CI toolchain image (.NET 10 SDK + Node 22 + prod-identical ffmpeg) and
|
||||
# pushes it to the Gitea container registry (ersatztv#390). The toolchain jobs in
|
||||
# docker-build.yml consume it via `container:`, pinned to an immutable :<sha>.
|
||||
#
|
||||
# push touching docker/ci/** -> :<short-sha> (+ :latest only from main)
|
||||
# workflow_dispatch -> manual rebuild
|
||||
# schedule (weekly) -> picks up base-image security updates
|
||||
#
|
||||
# Deliberately separate from docker-build.yml: this image changes rarely (a Dockerfile edit or
|
||||
# the weekly cron), while docker-build.yml runs on every push/PR. Coupling them would rebuild a
|
||||
# ~2GB toolchain image on every commit.
|
||||
#
|
||||
# ROLLOUT NOTE: the jobs pin an immutable :<sha>, never :latest — a broken toolchain image would
|
||||
# otherwise block every converted job the moment it was pushed. Bumping the toolchain is therefore
|
||||
# a deliberate two-step: merge a docker/ci/Dockerfile change (this workflow publishes a new :<sha>),
|
||||
# then update the pin in docker-build.yml in a follow-up PR whose CI proves the new image works.
|
||||
# See docs/ci-cd.md -> "CI toolchain image".
|
||||
#
|
||||
# Like docker-build.yml: the Gitea registry is HTTP-only, so BuildKit needs the inline
|
||||
# `http = true` config (it does not inherit the host daemon's insecure-registries setting).
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
paths:
|
||||
- 'docker/ci/**'
|
||||
- '.gitea/workflows/ci-image.yml'
|
||||
schedule:
|
||||
# Mondays 05:00 UTC. Gitea registers `schedule` only from the default branch (main).
|
||||
#
|
||||
# What this cron does and does NOT do — it does **not** update any running job. The jobs in
|
||||
# docker-build.yml pin an immutable :<sha> (deliberately), so a rebuilt image is consumed only
|
||||
# when a human bumps that pin. Its actual value is twofold:
|
||||
# 1. a weekly CANARY — catches "the toolchain image no longer builds" (a NodeSource/apt/base
|
||||
# change) at a time of our choosing, rather than when you next need to bump the pin;
|
||||
# 2. it leaves a freshly-patched :latest so the next pin bump starts from a current base.
|
||||
# `no-cache` on this path is what makes both real: with the shared :buildcache, the
|
||||
# `apt-get update && apt-get install` layer would restore from cache and re-fetch nothing.
|
||||
- cron: '0 5 * * 1'
|
||||
|
||||
# Serialize per ref: concurrent builds would race on the shared :buildcache tag.
|
||||
# No cancel-in-progress — a half-pushed toolchain image is worse than a redundant build.
|
||||
concurrency:
|
||||
group: ersatztv-ci-image-${{ github.ref }}
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
REGISTRY: 192.168.1.95:3000
|
||||
CI_IMAGE: 192.168.1.95:3000/timothy/ersatztv-ci
|
||||
|
||||
jobs:
|
||||
build:
|
||||
name: Build & push CI image
|
||||
# `small` = the small-jobs runner lane. This is a docker-only job (no toolchain needed —
|
||||
# it *builds* the toolchain), same as docker-build.yml's `build` job.
|
||||
runs-on: small
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
# only docker/ci/Dockerfile is needed; no git describe/log here
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Compute tags
|
||||
id: meta
|
||||
run: |
|
||||
set -euo pipefail
|
||||
SHORT=$(git rev-parse --short HEAD)
|
||||
# Always publish the immutable :<sha> — that is what docker-build.yml pins.
|
||||
TAGS=("${CI_IMAGE}:${SHORT}")
|
||||
# :latest is a convenience/floating pointer for humans and the weekly rebuild; jobs must
|
||||
# never consume it. Only main may move it.
|
||||
if [ "${GITHUB_REF}" = "refs/heads/main" ]; then
|
||||
TAGS+=("${CI_IMAGE}:latest")
|
||||
fi
|
||||
echo "short=${SHORT}" >> "$GITHUB_OUTPUT"
|
||||
{
|
||||
echo "tags<<__EOT__"
|
||||
printf '%s\n' "${TAGS[@]}"
|
||||
echo "__EOT__"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
printf 'tag: %s\n' "${TAGS[@]}"
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
with:
|
||||
buildkitd-config-inline: |
|
||||
[registry."192.168.1.95:3000"]
|
||||
http = true
|
||||
|
||||
- name: Login to Gitea registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ secrets.REGISTRY_USER }}
|
||||
password: ${{ secrets.REGISTRY_PASSWORD }}
|
||||
|
||||
- name: Build and push
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
file: ./docker/ci/Dockerfile
|
||||
platforms: linux/amd64
|
||||
push: true
|
||||
provenance: false
|
||||
tags: ${{ steps.meta.outputs.tags }}
|
||||
# The scheduled rebuild must bypass the cache or it is pointless: `mode=max` buildcache
|
||||
# would restore the `apt-get update && apt-get install` layer verbatim and pull in none of
|
||||
# the base updates the cron exists to collect. Push-triggered builds keep the cache.
|
||||
no-cache: ${{ github.event_name == 'schedule' }}
|
||||
cache-from: type=registry,ref=192.168.1.95:3000/timothy/ersatztv-ci:buildcache
|
||||
cache-to: type=registry,ref=192.168.1.95:3000/timothy/ersatztv-ci:buildcache,mode=max,ignore-error=true
|
||||
|
||||
# The Dockerfile's own build-time smoke test (dotnet --info, node, ffmpeg, ...) already ran
|
||||
# inside the build. This re-checks the *pushed* artifact end-to-end: that the registry copy
|
||||
# pulls and its toolchain runs, which is exactly what `container:` will do on every job.
|
||||
- name: Verify the pushed image
|
||||
run: |
|
||||
set -euo pipefail
|
||||
IMG="${CI_IMAGE}:${{ steps.meta.outputs.short }}"
|
||||
echo "Pulling ${IMG}"
|
||||
docker pull "$IMG"
|
||||
docker run --rm --entrypoint /bin/bash "$IMG" -euxc '
|
||||
dotnet --version
|
||||
dotnet ef --version
|
||||
node --version
|
||||
ffmpeg -version | head -1
|
||||
git --version
|
||||
python3 --version
|
||||
# reportgenerator --version exits 1 ("No report files specified"); probe the shim.
|
||||
command -v reportgenerator
|
||||
'
|
||||
echo "CI image OK. Pin this in .gitea/workflows/docker-build.yml -> CI_IMAGE_REF:"
|
||||
echo " ${IMG}"
|
||||
@@ -12,6 +12,20 @@ name: Build ErsatzTV Image
|
||||
#
|
||||
# `:latest` is intentionally the test/dev channel (per ersatztv#3); prod pins
|
||||
# `:prod`, never `:latest` (enforced in the prod compose — server-management#481).
|
||||
#
|
||||
# TOOLCHAIN IMAGE (ersatztv#390): the jobs that need a toolchain (`test`, `migrations`,
|
||||
# `functional-e2e`, `api-docs`, `format`) run inside our shared CI image via `container:`
|
||||
# instead of installing .NET/Node/ffmpeg per run. It ships the .NET 10 SDK, Node 22,
|
||||
# prod-identical ffmpeg, and the dotnet-ef/reportgenerator global tools — so those jobs carry
|
||||
# no setup-dotnet, no setup-node, no apt, no `dotnet tool install`. Built by ci-image.yml from
|
||||
# docker/ci/Dockerfile. Project deps (NuGet/npm) are NOT baked in and stay on actions/cache.
|
||||
#
|
||||
# The pin below is an IMMUTABLE :<sha>, never :latest — a bad toolchain push would otherwise
|
||||
# break every converted job at once. It is repeated per job because `jobs.<id>.container.image`
|
||||
# cannot read the workflow `env` context. **Bump all five together**; see docs/ci-cd.md ->
|
||||
# "CI toolchain image" for the two-step procedure.
|
||||
#
|
||||
# CI image pin: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
@@ -32,6 +46,16 @@ concurrency:
|
||||
group: ersatztv-build-${{ github.event_name }}-${{ github.ref }}
|
||||
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
||||
|
||||
# Inside a `container:`, act_runner does NOT default `run` steps to bash — it falls back to
|
||||
# `sh -e {0}` (dash), because it can't assume bash exists in an arbitrary image. Every multi-line
|
||||
# script here is bash (`set -o pipefail`, arrays, `shopt`, `mapfile`), so dash fails them
|
||||
# immediately: `set: Illegal option -o pipefail`. Declare the shell once for the whole workflow
|
||||
# rather than per step. Non-container jobs already defaulted to bash, so this changes nothing for
|
||||
# them. (ersatztv#390 — see docs/ci-cd.md -> "CI toolchain image".)
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
|
||||
env:
|
||||
REGISTRY: 192.168.1.95:3000
|
||||
IMAGE: 192.168.1.95:3000/timothy/ersatztv
|
||||
@@ -40,6 +64,11 @@ jobs:
|
||||
test:
|
||||
name: Build & test (.NET)
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8
|
||||
credentials:
|
||||
username: ${{ secrets.REGISTRY_USER }}
|
||||
password: ${{ secrets.REGISTRY_PASSWORD }}
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
@@ -48,11 +77,6 @@ jobs:
|
||||
# are only needed by the `build` job's `git describe` (ersatztv#190)
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Setup .NET
|
||||
uses: actions/setup-dotnet@v4
|
||||
with:
|
||||
dotnet-version: '10.0.x'
|
||||
|
||||
- name: Cache NuGet packages
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
@@ -63,12 +87,14 @@ jobs:
|
||||
- name: Restore
|
||||
run: dotnet restore
|
||||
|
||||
- name: Setup Node
|
||||
uses: actions/setup-node@v4
|
||||
# Replaces setup-node's built-in `cache: npm`. The toolchain image supplies node/npm, but
|
||||
# the SPA's package downloads are project deps, so they stay cached per lockfile.
|
||||
- name: Cache npm packages
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
node-version: '22.x'
|
||||
cache: npm
|
||||
cache-dependency-path: web/package-lock.json
|
||||
path: ~/.npm
|
||||
key: npm-${{ runner.os }}-${{ hashFiles('web/package-lock.json') }}
|
||||
restore-keys: npm-${{ runner.os }}-
|
||||
|
||||
- name: Install SPA dependencies
|
||||
working-directory: web
|
||||
@@ -121,10 +147,8 @@ jobs:
|
||||
exit 0
|
||||
fi
|
||||
echo "Found ${#reports[@]} coverage report(s)."
|
||||
# `update` is install-or-update (idempotent, unlike `install` which errors if the tool
|
||||
# is already present under set -e); pinned for reproducible summary output.
|
||||
dotnet tool update --global dotnet-reportgenerator-globaltool --version 5.5.10 >/dev/null
|
||||
export PATH="$PATH:$HOME/.dotnet/tools"
|
||||
# reportgenerator is baked into the CI toolchain image (docker/ci/Dockerfile) and already
|
||||
# on PATH — no per-run `dotnet tool` install. Bump its version there (ersatztv#390).
|
||||
reportgenerator \
|
||||
"-reports:coverage/**/coverage.cobertura.xml" \
|
||||
"-targetdir:coverage/report" \
|
||||
@@ -139,6 +163,11 @@ jobs:
|
||||
migrations:
|
||||
name: EF migration integrity (SQLite + MySql)
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8
|
||||
credentials:
|
||||
username: ${{ secrets.REGISTRY_USER }}
|
||||
password: ${{ secrets.REGISTRY_PASSWORD }}
|
||||
# Independent gate (not a 'needs' of build yet) so the new MySql-service dependency
|
||||
# can't block image builds until it's proven reliable on the runner. Promote to a
|
||||
# required check / build dependency once green. (ersatztv#13)
|
||||
@@ -162,11 +191,6 @@ jobs:
|
||||
# default fetch-depth: 1 -- this job never runs git describe/log, only
|
||||
# actions/checkout@v4's default (shallow) history is needed (ersatztv#190)
|
||||
|
||||
- name: Setup .NET
|
||||
uses: actions/setup-dotnet@v4
|
||||
with:
|
||||
dotnet-version: '10.0.x'
|
||||
|
||||
- name: Cache NuGet packages
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
@@ -180,14 +204,13 @@ jobs:
|
||||
- name: Build
|
||||
run: dotnet build --configuration Release --no-restore
|
||||
|
||||
- name: Install dotnet-ef
|
||||
run: dotnet tool install --global dotnet-ef --version 9.0.12
|
||||
# dotnet-ef is baked into the CI toolchain image (docker/ci/Dockerfile) and already on PATH
|
||||
# — no per-run `dotnet tool install`. Bump its version there (ersatztv#390).
|
||||
|
||||
# SQLite is the prod provider; both checks validated locally.
|
||||
- name: SQLite — model drift + apply all migrations to a fresh DB
|
||||
run: |
|
||||
set -euo pipefail
|
||||
export PATH="$PATH:$HOME/.dotnet/tools"
|
||||
echo "::group::SQLite model drift (has-pending-model-changes)"
|
||||
dotnet ef migrations has-pending-model-changes --no-build --configuration Release \
|
||||
--context TvContext --startup-project ErsatzTV --project ErsatzTV.Infrastructure.Sqlite -- --provider Sqlite
|
||||
@@ -210,7 +233,6 @@ jobs:
|
||||
MySql__ConnectionString: "Server=mysql;Port=3306;Database=ersatztv_migrations;Uid=root;Pwd=ersatztv;DefaultCommandTimeout=300;"
|
||||
run: |
|
||||
set -euo pipefail
|
||||
export PATH="$PATH:$HOME/.dotnet/tools"
|
||||
echo "::group::MySql model drift (has-pending-model-changes)"
|
||||
dotnet ef migrations has-pending-model-changes --no-build --configuration Release \
|
||||
--context TvContext --startup-project ErsatzTV --project ErsatzTV.Infrastructure.MySql -- --provider MySql
|
||||
@@ -246,17 +268,17 @@ jobs:
|
||||
# reliable (same rollout the `migrations` job used). SQLite default provider -> no DB service.
|
||||
# Runs on PRs and on main (regression net); skipped for v* tag builds.
|
||||
if: github.event_name == 'pull_request' || github.ref == 'refs/heads/main'
|
||||
container:
|
||||
image: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8
|
||||
credentials:
|
||||
username: ${{ secrets.REGISTRY_USER }}
|
||||
password: ${{ secrets.REGISTRY_PASSWORD }}
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Setup .NET
|
||||
uses: actions/setup-dotnet@v4
|
||||
with:
|
||||
dotnet-version: '10.0.x'
|
||||
|
||||
- name: Cache NuGet packages
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
@@ -267,12 +289,12 @@ jobs:
|
||||
- name: Restore
|
||||
run: dotnet restore
|
||||
|
||||
- name: Setup Node
|
||||
uses: actions/setup-node@v4
|
||||
- name: Cache npm packages
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
node-version: '22.x'
|
||||
cache: npm
|
||||
cache-dependency-path: web/package-lock.json
|
||||
path: ~/.npm
|
||||
key: npm-${{ runner.os }}-${{ hashFiles('web/package-lock.json') }}
|
||||
restore-keys: npm-${{ runner.os }}-
|
||||
|
||||
- name: Install SPA dependencies
|
||||
working-directory: web
|
||||
@@ -285,8 +307,11 @@ jobs:
|
||||
- name: Build (Release)
|
||||
run: dotnet build ErsatzTV.sln --configuration Release --no-restore
|
||||
|
||||
- name: Ensure ffmpeg is available
|
||||
run: command -v ffmpeg >/dev/null 2>&1 || (sudo apt-get update && sudo apt-get install -y ffmpeg)
|
||||
# The old `command -v ffmpeg || sudo apt-get install ffmpeg` step is gone (ersatztv#390):
|
||||
# the toolchain image ships the same ffmpeg build prod runs, so the binary is already here.
|
||||
# That step also cost 110s of every run to install a dependency this harness never uses —
|
||||
# it boots the app (whose only ffmpeg touch at startup is a LogWarning from
|
||||
# FFmpegLocatorService) and drives curl-only contracts that never transcode.
|
||||
|
||||
- name: Boot instance and run functional-E2E harness
|
||||
run: |
|
||||
@@ -428,6 +453,57 @@ jobs:
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# BLOCKING (ersatztv#390): the CI toolchain image pin in this file must name the image that
|
||||
# ci-image.yml actually last published — i.e. the short sha of the last commit to touch the image's
|
||||
# sources. Without this detector, a PR that edits docker/ci/** publishes a NEW image but runs its own
|
||||
# jobs against the OLD pin: CI green-lights a toolchain it never executed, and once merged, main's
|
||||
# Dockerfile silently disagrees with what CI runs. **Renovate actively generates exactly that PR** —
|
||||
# it manages docker/ci/Dockerfile's base pins (dockerfile manager) but cannot bump an opaque
|
||||
# `:<sha>` in `container.image`, so it would leave the pin behind every time.
|
||||
#
|
||||
# Failing here forces the documented two-step (docs/ci-cd.md -> "CI toolchain image"): push the
|
||||
# Dockerfile change, let ci-image.yml publish `:<sha>`, then update the pin to that sha. Seconds-long
|
||||
# git+grep -> keep it off the build runners.
|
||||
ci-image-pin:
|
||||
name: CI image pin matches docker/ci
|
||||
runs-on: small
|
||||
if: github.event_name == 'pull_request'
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
# need real history: `git log -- <path>` on a shallow clone can't find the last
|
||||
# commit that touched the image sources
|
||||
fetch-depth: 0
|
||||
- name: Verify the pin matches the last-published image
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# ci-image.yml tags the image `git rev-parse --short HEAD` of the push that built it, and it
|
||||
# only builds on pushes touching these paths — so the published image is named by the last
|
||||
# commit to touch them.
|
||||
#
|
||||
# Compare RESOLVED FULL shas, never the abbreviations: git auto-scales abbreviation length
|
||||
# with the repo's object count, so the tag built in CI from a `fetch-depth: 1` shallow clone
|
||||
# is 7 chars while `%h` here (full clone) is 8. Comparing those strings would fail always.
|
||||
expected="$(git log -1 --format=%H -- docker/ci .gitea/workflows/ci-image.yml)"
|
||||
mapfile -t pins < <(grep -oE 'ersatztv-ci:[0-9a-f]+' .gitea/workflows/docker-build.yml | cut -d: -f2 | sort -u)
|
||||
echo "Image sources last changed in: ${expected}"
|
||||
echo "Pins found in docker-build.yml: ${pins[*]} (${#pins[@]} distinct)"
|
||||
if [ "${#pins[@]}" -ne 1 ]; then
|
||||
echo "::error::docker-build.yml pins MORE THAN ONE ersatztv-ci tag (${pins[*]}). All jobs must pin the same image — bump them together."
|
||||
exit 1
|
||||
fi
|
||||
pin_full="$(git rev-parse --verify --quiet "${pins[0]}^{commit}" || true)"
|
||||
if [ -z "$pin_full" ]; then
|
||||
echo "::error::The pinned CI image tag ersatztv-ci:${pins[0]} does not resolve to a commit in this repo, so it cannot correspond to an image ci-image.yml built from these sources. Rebuild the image and pin the sha it prints."
|
||||
exit 1
|
||||
fi
|
||||
if [ "$pin_full" != "$expected" ]; then
|
||||
echo "::error::CI toolchain image pin is stale: docker-build.yml pins ersatztv-ci:${pins[0]} ($pin_full), but docker/ci was last changed in $expected. Your jobs are testing an image that is NOT built from this PR's docker/ci. Let ci-image.yml publish the new :<sha>, then update the pin in ALL jobs to it (docs/ci-cd.md -> 'CI toolchain image')."
|
||||
exit 1
|
||||
fi
|
||||
echo "Pin is current: ersatztv-ci:${pins[0]} resolves to $pin_full = docker/ci's last change."
|
||||
|
||||
# Non-blocking nudge: if a PR migrates/adds a route but forgets the parity tracker, warn.
|
||||
# The rule lives in CLAUDE.md → Conventions; this only surfaces an easy-to-miss omission.
|
||||
# Deliberately no setup-dotnet/setup-node (and thus no actions/cache) so it can't hit the
|
||||
@@ -503,7 +579,20 @@ jobs:
|
||||
# API path changed, the expensive steps skip and the job passes trivially.
|
||||
api-docs:
|
||||
name: API docs in sync (OpenAPI + endpoint index)
|
||||
runs-on: ubuntu-latest
|
||||
# `small` lane (ersatztv#390): this job is ~5s on the ~90% of PRs that touch no API path, but
|
||||
# it was queueing ~29 min behind the heavy jobs in the contended `ubuntu-latest` lane, which
|
||||
# only has bumblebee-runner (capacity 2) + ci-runner. `small` has capacity 4, the same base
|
||||
# image, and answers in ~5s. Moving it here (and `format`) also drops `ubuntu-latest` from 5
|
||||
# jobs to 3, which shortens the queue for `test`/`migrations`/`functional-e2e` too.
|
||||
# This is only possible because `container:` makes the job self-contained — it no longer needs
|
||||
# the runner image to supply .NET/Node. Caveat: on an API-touching PR this job does a full
|
||||
# `dotnet build`, so it is not always a "small" job; capacity 4 absorbs that.
|
||||
runs-on: small
|
||||
container:
|
||||
image: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8
|
||||
credentials:
|
||||
username: ${{ secrets.REGISTRY_USER }}
|
||||
password: ${{ secrets.REGISTRY_PASSWORD }}
|
||||
if: github.event_name == 'pull_request'
|
||||
steps:
|
||||
- name: Checkout
|
||||
@@ -526,12 +615,6 @@ jobs:
|
||||
echo "No API-surface change -> skipping regeneration (job passes)."
|
||||
fi
|
||||
|
||||
- name: Setup .NET
|
||||
if: steps.detect.outputs.api_changed == 'true'
|
||||
uses: actions/setup-dotnet@v4
|
||||
with:
|
||||
dotnet-version: '10.0.x'
|
||||
|
||||
- name: Cache NuGet packages
|
||||
if: steps.detect.outputs.api_changed == 'true'
|
||||
uses: actions/cache@v4
|
||||
@@ -544,13 +627,13 @@ jobs:
|
||||
if: steps.detect.outputs.api_changed == 'true'
|
||||
run: dotnet restore
|
||||
|
||||
- name: Setup Node
|
||||
- name: Cache npm packages
|
||||
if: steps.detect.outputs.api_changed == 'true'
|
||||
uses: actions/setup-node@v4
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
node-version: '22.x'
|
||||
cache: npm
|
||||
cache-dependency-path: web/package-lock.json
|
||||
path: ~/.npm
|
||||
key: npm-${{ runner.os }}-${{ hashFiles('web/package-lock.json') }}
|
||||
restore-keys: npm-${{ runner.os }}-
|
||||
|
||||
- name: Install SPA dependencies
|
||||
if: steps.detect.outputs.api_changed == 'true'
|
||||
@@ -585,7 +668,14 @@ jobs:
|
||||
# a status, so it is safe as a required check).
|
||||
format:
|
||||
name: Formatting (changed .cs conform to .editorconfig)
|
||||
runs-on: ubuntu-latest
|
||||
# `small` lane (ersatztv#390) — 37s of work that was queueing ~29 min in the contended
|
||||
# `ubuntu-latest` lane. See the api-docs job above for the full rationale.
|
||||
runs-on: small
|
||||
container:
|
||||
image: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8
|
||||
credentials:
|
||||
username: ${{ secrets.REGISTRY_USER }}
|
||||
password: ${{ secrets.REGISTRY_PASSWORD }}
|
||||
if: github.event_name == 'pull_request'
|
||||
steps:
|
||||
- name: Checkout
|
||||
@@ -609,12 +699,6 @@ jobs:
|
||||
echo "No .cs change -> skipping format verify (job passes)."
|
||||
fi
|
||||
|
||||
- name: Setup .NET
|
||||
if: steps.detect.outputs.cs_changed == 'true'
|
||||
uses: actions/setup-dotnet@v4
|
||||
with:
|
||||
dotnet-version: '10.0.x'
|
||||
|
||||
- name: Cache NuGet packages
|
||||
if: steps.detect.outputs.cs_changed == 'true'
|
||||
uses: actions/cache@v4
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
# Shared CI toolchain image (ersatztv#390) — .NET 10 SDK + Node 22 + prod-identical ffmpeg.
|
||||
#
|
||||
# Built/pushed by .gitea/workflows/ci-image.yml to 192.168.1.95:3000/timothy/ersatztv-ci.
|
||||
# The toolchain jobs in docker-build.yml run *inside* this image via `container:`, so they
|
||||
# install nothing at run time: no setup-dotnet, no setup-node, no `apt-get install ffmpeg`,
|
||||
# no `dotnet tool install`. Project dependencies (NuGet/npm) are NOT baked in — those change
|
||||
# per commit and stay on actions/cache.
|
||||
#
|
||||
# Layering rationale: we start from our OWN ffmpeg base and copy the .NET SDK on top, which is
|
||||
# exactly the pattern docker/Dockerfile uses for the prod image (`FROM ersatztv-ffmpeg` +
|
||||
# `COPY --from=dotnet-runtime /usr/share/dotnet`). That base is ghcr.io/linuxserver/baseimage-ubuntu:noble,
|
||||
# the same Ubuntu release as mcr.microsoft.com/dotnet/sdk:10.0-noble, so the copied SDK matches
|
||||
# the base's glibc/ICU. It also gives CI the *prod* ffmpeg build rather than a generic apt one —
|
||||
# the fidelity that the ersatztv#299 seeded-media/scanner E2E follow-ups will need.
|
||||
#
|
||||
# Keep the pins below in sync with what CI actually needs; Renovate tracks the image pins
|
||||
# (see renovate.json). The two dotnet tool versions mirror what docker-build.yml used to
|
||||
# `dotnet tool install` per run — bump them here, not in the workflow.
|
||||
|
||||
# Keep the ffmpeg tag on the FROM below equal to the one docker/Dockerfile pins, so CI's ffmpeg
|
||||
# stays prod-identical. Renovate manages both pins (dockerfile manager); mirror any bump here.
|
||||
FROM --platform=linux/amd64 192.168.1.95:3000/timothy/ersatztv-ffmpeg:8.1.2 AS ci-base
|
||||
|
||||
# The ffmpeg base sets ENTRYPOINT ["ffmpeg"] / CMD ["--help"] because it ships as an ffmpeg CLI.
|
||||
# This image is a CI *job container*: the runner starts it long-lived and execs steps into it,
|
||||
# so an ffmpeg entrypoint is wrong here. Reset both.
|
||||
ENTRYPOINT []
|
||||
CMD ["/bin/bash"]
|
||||
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
|
||||
# Toolchain the workflow steps and the actions themselves need:
|
||||
# git — actions/checkout
|
||||
# ca-certificates, curl — NodeSource setup + general fetches
|
||||
# tar, zstd, unzip, xz — actions/cache archive round-trip
|
||||
# python3 — scripts/generate-endpoint-index.py (the api-docs job)
|
||||
# jq — ad-hoc JSON in CI steps
|
||||
# The ffmpeg base is a runtime image, so none of these are guaranteed present.
|
||||
RUN apt-get update && \
|
||||
apt-get install -y --no-install-recommends \
|
||||
ca-certificates \
|
||||
curl \
|
||||
git \
|
||||
jq \
|
||||
python3 \
|
||||
tar \
|
||||
unzip \
|
||||
xz-utils \
|
||||
zstd && \
|
||||
apt-get clean -y && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Node 22 (NodeSource) — matches the `node-version: '22.x'` the workflow used to request from
|
||||
# setup-node, and the node:22 base docker/Dockerfile builds the SPA with. act_runner also needs a
|
||||
# `node` on PATH inside the job container to execute JavaScript actions (actions/checkout, cache).
|
||||
RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - && \
|
||||
apt-get install -y --no-install-recommends nodejs && \
|
||||
apt-get clean -y && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# actions/checkout clones as root into a runner-mounted workspace, which trips git's
|
||||
# "detected dubious ownership in repository" guard and breaks every `git` call in a step
|
||||
# (scripts/e2e-local.sh's `git rev-parse --show-toplevel`, the base-ref diffs, ...). Trusting all
|
||||
# paths is the right call for a throwaway single-tenant CI container.
|
||||
RUN git config --global --add safe.directory '*'
|
||||
|
||||
# .NET 10 SDK. Copied from the official SDK image rather than dotnet-install.sh: same mechanism as
|
||||
# docker/Dockerfile's runtime copy, one pinned artifact, no install script to fail at build time.
|
||||
# global.json pins sdk 10.0.0 with rollForward latestMinor, which this satisfies.
|
||||
COPY --from=mcr.microsoft.com/dotnet/sdk:10.0-noble-amd64 /usr/share/dotnet /usr/share/dotnet
|
||||
|
||||
ENV DOTNET_ROOT=/usr/share/dotnet \
|
||||
PATH="/usr/share/dotnet:/root/.dotnet/tools:${PATH}" \
|
||||
DOTNET_CLI_TELEMETRY_OPTOUT=1 \
|
||||
DOTNET_NOLOGO=1 \
|
||||
DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1
|
||||
|
||||
# Global dotnet tools the jobs used to install per run (5s + ~4s each, every run, forever).
|
||||
# dotnet-ef — the migrations job (was: `dotnet tool install --global dotnet-ef --version 9.0.12`)
|
||||
# dotnet-reportgenerator-globaltool — the test job's coverage summary
|
||||
# Baked here so the versions live in one tracked place instead of inline workflow strings.
|
||||
RUN dotnet tool install --global dotnet-ef --version 9.0.12 && \
|
||||
dotnet tool install --global dotnet-reportgenerator-globaltool --version 5.5.10
|
||||
|
||||
# Build-time smoke test: fail the IMAGE build, not a CI run, if any part of the toolchain is
|
||||
# unusable. Notably `dotnet --info` catches a missing ICU/glibc dependency in the base — the one
|
||||
# real risk of copying the SDK onto a non-Microsoft base.
|
||||
RUN set -eux; \
|
||||
dotnet --info; \
|
||||
dotnet ef --version; \
|
||||
node --version; \
|
||||
npm --version; \
|
||||
git --version; \
|
||||
python3 --version; \
|
||||
ffmpeg -version; \
|
||||
ffprobe -version; \
|
||||
# reportgenerator has no clean version probe: `--version` prints the banner and then exits 1
|
||||
# ("No report files specified"), so asserting on it would fail this build. Probe that the
|
||||
# dotnet-tool shim resolves on PATH instead.
|
||||
command -v reportgenerator
|
||||
+90
-1
@@ -90,6 +90,29 @@ jobs capped `--cpus=4 --memory=10g` so CI can't starve prod media playback), and
|
||||
runner task even when its `if` skips it, and those skip-tasks used to wait behind
|
||||
long builds (observed 31 min) stalling every PR run.
|
||||
|
||||
**Lane assignment (ersatztv#390).** The `ubuntu-latest` lane has **4 slots** (2 + 2) and the
|
||||
`small` lane has 4. A 2026-07-17 audit of the Actions API found the `ubuntu-latest` lane
|
||||
saturated and the `small` lane idle — **queue wait exceeded every job's runtime**:
|
||||
|
||||
| Job | Runtime | Queue wait | Lane |
|
||||
|---|---|---|---|
|
||||
| `test` | 354s | 1363s | ubuntu-latest |
|
||||
| `migrations` | 639s | 1428s | ubuntu-latest |
|
||||
| `functional-e2e` | 520s | 1447s | ubuntu-latest |
|
||||
| `api-docs` | 5s | **1722s** | ubuntu-latest → now `small` |
|
||||
| `format` | 37s | **1731s** | ubuntu-latest → now `small` |
|
||||
| `docs-reminder` / `decisions-guard` | 10s | **5s** | small |
|
||||
|
||||
So `api-docs` and `format` — genuinely small jobs — moved to `small`, which also drops the
|
||||
contended lane from 5 jobs to 3 and shortens the queue for the three heavy jobs. Both lanes run
|
||||
the *identical* `runner-images:ubuntu-latest` base, so `small` is a label with spare capacity, not
|
||||
a different capability. This move is only possible because those jobs now run in the **CI toolchain
|
||||
image** (below) and no longer need the runner image to supply .NET/Node. Caveat: on an
|
||||
API-touching PR `api-docs` does a full `dotnet build`, so it is not *always* small.
|
||||
|
||||
Queue wait is still the dominant cost and capacity is server-management's boundary — tracked in
|
||||
**server-management#604**. The redundant triple-build behind those runtimes is **ersatztv#398**.
|
||||
|
||||
### `test` job
|
||||
|
||||
`dotnet restore` → strip the Scanner project ref (`sed -i '/Scanner/d'`, matching the
|
||||
@@ -142,7 +165,8 @@ Boots the app **from source** and drives the manual live-E2E curl flows sessions
|
||||
re-run by hand, turning them into a CI regression net (ersatztv#299). It is the automatable half of
|
||||
`docs/e2e-local.md`; the two scripts it chains run identically locally and in CI:
|
||||
|
||||
1. `npm ci` + `npm run build` (SPA), `dotnet build ErsatzTV.sln -c Release`, ensure `ffmpeg` is on PATH.
|
||||
1. `npm ci` + `npm run build` (SPA), `dotnet build ErsatzTV.sln -c Release`. (ffmpeg comes from the
|
||||
CI toolchain image; the old `apt-get install ffmpeg` step is gone — see below.)
|
||||
2. `ETV_BUILD_CONFIG=Release scripts/e2e-local.sh <fresh-config>` — copies `wwwroot`, launches
|
||||
`dotnet ErsatzTV.dll` in the background (logging to a file so the launch step returns once the app
|
||||
is ready), prints `PID`/`CONFIG_DIR`.
|
||||
@@ -191,6 +215,71 @@ accepts the token in *any* commit of the range). The local hook is therefore the
|
||||
gate; CI is the backstop for direct pushes or bypassed hooks. Like `docs-reminder`, it's a
|
||||
seconds-long `git diff` with no dotnet/node setup (`runs-on: small`).
|
||||
|
||||
## CI toolchain image (`docker/ci/Dockerfile`, `.gitea/workflows/ci-image.yml`)
|
||||
|
||||
The jobs that need a toolchain — `test`, `migrations`, `functional-e2e`, `api-docs`, `format` —
|
||||
run **inside** a shared image via `container:` instead of installing their toolchain per run
|
||||
(ersatztv#390). They therefore carry **no** `setup-dotnet`, **no** `setup-node`, **no** `apt-get`,
|
||||
and **no** `dotnet tool install`.
|
||||
|
||||
**What it ships:** .NET 10 SDK, Node 22, prod-identical ffmpeg/ffprobe, `git`/`python3`/`jq`/`zstd`,
|
||||
and the `dotnet-ef` + `dotnet-reportgenerator-globaltool` global tools (which the `migrations` and
|
||||
`test` jobs used to install on every run — **bump those versions in the Dockerfile, not the
|
||||
workflow**). Project dependencies (NuGet/npm) are deliberately **not** baked in — they change per
|
||||
commit and stay on `actions/cache` (`~/.nuget/packages`, `~/.npm`).
|
||||
|
||||
**How it's layered:** `FROM ersatztv-ffmpeg:8.1.2` + `COPY --from=mcr.microsoft.com/dotnet/sdk:10.0-noble-amd64
|
||||
/usr/share/dotnet` — the same pattern `docker/Dockerfile` uses for the prod image. Our ffmpeg base is
|
||||
`ghcr.io/linuxserver/baseimage-ubuntu:noble`, the same Ubuntu release as the SDK image, so the copied
|
||||
SDK matches the base's glibc/ICU. Keep the ffmpeg tag on that `FROM` equal to the one
|
||||
`docker/Dockerfile` pins, so CI's ffmpeg stays prod-identical — that fidelity is what the
|
||||
ersatztv#299 seeded-media/scanner E2E follow-ups will need.
|
||||
|
||||
**Bumping the toolchain is a deliberate two-step.** The jobs pin an immutable `:<sha>`, never
|
||||
`:latest`, so a bad toolchain push cannot break every job at once:
|
||||
|
||||
1. Merge a `docker/ci/Dockerfile` change. `ci-image.yml` publishes `ersatztv-ci:<sha>` (+ `:latest`
|
||||
from `main` only — a human pointer; **jobs must never consume it**).
|
||||
2. In a follow-up PR, update the pin in `docker-build.yml` — **all five jobs together**. That PR's own
|
||||
CI is what proves the new image works. The pin is repeated per job because
|
||||
`jobs.<id>.container.image` cannot read the workflow `env` context.
|
||||
|
||||
`ci-image.yml` triggers on pushes touching `docker/ci/**`, `workflow_dispatch`, and a weekly Monday
|
||||
05:00 UTC cron (base-image security updates; Gitea registers `schedule` only from `main`). It runs on
|
||||
the `small` lane and, like `docker-build.yml`, needs BuildKit's inline `http = true` for the HTTP
|
||||
registry. Renovate tracks the Dockerfile's image pins (`dockerfile` manager, see `renovate.json`).
|
||||
|
||||
**Three container-specific gotchas** — worth knowing if you add a job or a step:
|
||||
- **`sh`, not bash, is the default shell inside a `container:`.** act_runner runs steps as
|
||||
`sh -e {0}` (dash) because it can't assume bash exists in an arbitrary image — even though ours
|
||||
has it. Every bashism (`set -o pipefail`, arrays, `shopt`, `mapfile`) then dies instantly with
|
||||
`set: Illegal option -o pipefail`. `docker-build.yml` therefore declares a workflow-level
|
||||
`defaults: run: shell: bash`. **If you add a workflow with containerized jobs, do the same** —
|
||||
outside a container the shell defaults to bash, so this failure only appears once you containerize
|
||||
and it looks nothing like a shell problem (it surfaced as the `migrations` job dying in 0.13s).
|
||||
- `actions/checkout` clones as root into a mounted workspace, which trips git's *"detected dubious
|
||||
ownership"* guard and breaks every `git` call in a step. Fixed in the Dockerfile with
|
||||
`git config --global --add safe.directory '*'`.
|
||||
- *(defensive, not load-bearing)* The ffmpeg base sets `ENTRYPOINT ["ffmpeg"]` because it ships as an
|
||||
ffmpeg CLI, so the Dockerfile resets `ENTRYPOINT`/`CMD`. act overrides the entrypoint anyway
|
||||
(`entrypoint=["/bin/sleep" "10800"]`), so this is belt-and-braces for anyone running the image by
|
||||
hand — unlike the two above, which are real.
|
||||
|
||||
**Bumping the pin is enforced, not remembered.** The `ci-image-pin` job (blocking, PR-only) fails if
|
||||
`docker-build.yml`'s pin isn't the short sha of the last commit to touch `docker/ci/**` or
|
||||
`ci-image.yml`, or if the five jobs ever pin different tags. This exists because **Renovate manages
|
||||
`docker/ci/Dockerfile`'s base pins but cannot bump an opaque `:<sha>`** in `container.image` — so a
|
||||
Renovate base bump would otherwise publish a new image, test the *old* one, and merge with the
|
||||
Dockerfile disagreeing with the pin. A red `ci-image-pin` means: let `ci-image.yml` publish the new
|
||||
`:<sha>`, then update all five pins to it.
|
||||
|
||||
**What it is and isn't worth.** Measured honestly (ersatztv#390): the image saves ~15–40s per job
|
||||
(`setup-dotnet` is 8–19s, `setup-node` 2–5s cached, the two tool installs ~9s) plus the 110s
|
||||
apt-ffmpeg step — roughly 3–8% of runtime. It is **not** where CI time goes; see the lane table above
|
||||
(queue wait, server-management#604) and ersatztv#398 (742s of redundant compilation). Its durable
|
||||
value is prod-identical ffmpeg, a pinned/consistent toolchain, and making jobs runner-agnostic — the
|
||||
last is what allowed the lane rebalance.
|
||||
|
||||
## Dockerfile notes (`docker/Dockerfile`)
|
||||
|
||||
- Base image: **`192.168.1.95:3000/timothy/ersatztv-ffmpeg:8.1.2`** (our Gitea fork of
|
||||
|
||||
Reference in New Issue
Block a user