diff --git a/.gitea/workflows/ci-image.yml b/.gitea/workflows/ci-image.yml new file mode 100644 index 000000000..b0fb53fae --- /dev/null +++ b/.gitea/workflows/ci-image.yml @@ -0,0 +1,136 @@ +name: Build CI Toolchain Image + +# Builds the shared CI toolchain image (.NET 10 SDK + Node 22 + prod-identical ffmpeg) and +# pushes it to the Gitea container registry (ersatztv#390). The toolchain jobs in +# docker-build.yml consume it via `container:`, pinned to an immutable :. +# +# push touching docker/ci/** -> : (+ :latest only from main) +# workflow_dispatch -> manual rebuild +# schedule (weekly) -> picks up base-image security updates +# +# Deliberately separate from docker-build.yml: this image changes rarely (a Dockerfile edit or +# the weekly cron), while docker-build.yml runs on every push/PR. Coupling them would rebuild a +# ~2GB toolchain image on every commit. +# +# ROLLOUT NOTE: the jobs pin an immutable :, never :latest — a broken toolchain image would +# otherwise block every converted job the moment it was pushed. Bumping the toolchain is therefore +# a deliberate two-step: merge a docker/ci/Dockerfile change (this workflow publishes a new :), +# then update the pin in docker-build.yml in a follow-up PR whose CI proves the new image works. +# See docs/ci-cd.md -> "CI toolchain image". +# +# Like docker-build.yml: the Gitea registry is HTTP-only, so BuildKit needs the inline +# `http = true` config (it does not inherit the host daemon's insecure-registries setting). + +on: + workflow_dispatch: + push: + paths: + - 'docker/ci/**' + - '.gitea/workflows/ci-image.yml' + schedule: + # Mondays 05:00 UTC. Gitea registers `schedule` only from the default branch (main). + # + # What this cron does and does NOT do — it does **not** update any running job. The jobs in + # docker-build.yml pin an immutable : (deliberately), so a rebuilt image is consumed only + # when a human bumps that pin. Its actual value is twofold: + # 1. a weekly CANARY — catches "the toolchain image no longer builds" (a NodeSource/apt/base + # change) at a time of our choosing, rather than when you next need to bump the pin; + # 2. it leaves a freshly-patched :latest so the next pin bump starts from a current base. + # `no-cache` on this path is what makes both real: with the shared :buildcache, the + # `apt-get update && apt-get install` layer would restore from cache and re-fetch nothing. + - cron: '0 5 * * 1' + +# Serialize per ref: concurrent builds would race on the shared :buildcache tag. +# No cancel-in-progress — a half-pushed toolchain image is worse than a redundant build. +concurrency: + group: ersatztv-ci-image-${{ github.ref }} + cancel-in-progress: false + +env: + REGISTRY: 192.168.1.95:3000 + CI_IMAGE: 192.168.1.95:3000/timothy/ersatztv-ci + +jobs: + build: + name: Build & push CI image + # `small` = the small-jobs runner lane. This is a docker-only job (no toolchain needed — + # it *builds* the toolchain), same as docker-build.yml's `build` job. + runs-on: small + steps: + - name: Checkout + uses: actions/checkout@v4 + with: + # only docker/ci/Dockerfile is needed; no git describe/log here + fetch-depth: 1 + + - name: Compute tags + id: meta + run: | + set -euo pipefail + SHORT=$(git rev-parse --short HEAD) + # Always publish the immutable : — that is what docker-build.yml pins. + TAGS=("${CI_IMAGE}:${SHORT}") + # :latest is a convenience/floating pointer for humans and the weekly rebuild; jobs must + # never consume it. Only main may move it. + if [ "${GITHUB_REF}" = "refs/heads/main" ]; then + TAGS+=("${CI_IMAGE}:latest") + fi + echo "short=${SHORT}" >> "$GITHUB_OUTPUT" + { + echo "tags<<__EOT__" + printf '%s\n' "${TAGS[@]}" + echo "__EOT__" + } >> "$GITHUB_OUTPUT" + printf 'tag: %s\n' "${TAGS[@]}" + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v3 + with: + buildkitd-config-inline: | + [registry."192.168.1.95:3000"] + http = true + + - name: Login to Gitea registry + uses: docker/login-action@v3 + with: + registry: ${{ env.REGISTRY }} + username: ${{ secrets.REGISTRY_USER }} + password: ${{ secrets.REGISTRY_PASSWORD }} + + - name: Build and push + uses: docker/build-push-action@v6 + with: + context: . + file: ./docker/ci/Dockerfile + platforms: linux/amd64 + push: true + provenance: false + tags: ${{ steps.meta.outputs.tags }} + # The scheduled rebuild must bypass the cache or it is pointless: `mode=max` buildcache + # would restore the `apt-get update && apt-get install` layer verbatim and pull in none of + # the base updates the cron exists to collect. Push-triggered builds keep the cache. + no-cache: ${{ github.event_name == 'schedule' }} + cache-from: type=registry,ref=192.168.1.95:3000/timothy/ersatztv-ci:buildcache + cache-to: type=registry,ref=192.168.1.95:3000/timothy/ersatztv-ci:buildcache,mode=max,ignore-error=true + + # The Dockerfile's own build-time smoke test (dotnet --info, node, ffmpeg, ...) already ran + # inside the build. This re-checks the *pushed* artifact end-to-end: that the registry copy + # pulls and its toolchain runs, which is exactly what `container:` will do on every job. + - name: Verify the pushed image + run: | + set -euo pipefail + IMG="${CI_IMAGE}:${{ steps.meta.outputs.short }}" + echo "Pulling ${IMG}" + docker pull "$IMG" + docker run --rm --entrypoint /bin/bash "$IMG" -euxc ' + dotnet --version + dotnet ef --version + node --version + ffmpeg -version | head -1 + git --version + python3 --version + # reportgenerator --version exits 1 ("No report files specified"); probe the shim. + command -v reportgenerator + ' + echo "CI image OK. Pin this in .gitea/workflows/docker-build.yml -> CI_IMAGE_REF:" + echo " ${IMG}" diff --git a/.gitea/workflows/docker-build.yml b/.gitea/workflows/docker-build.yml index fee965db9..cda0be28e 100644 --- a/.gitea/workflows/docker-build.yml +++ b/.gitea/workflows/docker-build.yml @@ -12,6 +12,20 @@ name: Build ErsatzTV Image # # `:latest` is intentionally the test/dev channel (per ersatztv#3); prod pins # `:prod`, never `:latest` (enforced in the prod compose — server-management#481). +# +# TOOLCHAIN IMAGE (ersatztv#390): the jobs that need a toolchain (`test`, `migrations`, +# `functional-e2e`, `api-docs`, `format`) run inside our shared CI image via `container:` +# instead of installing .NET/Node/ffmpeg per run. It ships the .NET 10 SDK, Node 22, +# prod-identical ffmpeg, and the dotnet-ef/reportgenerator global tools — so those jobs carry +# no setup-dotnet, no setup-node, no apt, no `dotnet tool install`. Built by ci-image.yml from +# docker/ci/Dockerfile. Project deps (NuGet/npm) are NOT baked in and stay on actions/cache. +# +# The pin below is an IMMUTABLE :, never :latest — a bad toolchain push would otherwise +# break every converted job at once. It is repeated per job because `jobs..container.image` +# cannot read the workflow `env` context. **Bump all five together**; see docs/ci-cd.md -> +# "CI toolchain image" for the two-step procedure. +# +# CI image pin: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8 on: workflow_dispatch: @@ -32,6 +46,16 @@ concurrency: group: ersatztv-build-${{ github.event_name }}-${{ github.ref }} cancel-in-progress: ${{ github.event_name == 'pull_request' }} +# Inside a `container:`, act_runner does NOT default `run` steps to bash — it falls back to +# `sh -e {0}` (dash), because it can't assume bash exists in an arbitrary image. Every multi-line +# script here is bash (`set -o pipefail`, arrays, `shopt`, `mapfile`), so dash fails them +# immediately: `set: Illegal option -o pipefail`. Declare the shell once for the whole workflow +# rather than per step. Non-container jobs already defaulted to bash, so this changes nothing for +# them. (ersatztv#390 — see docs/ci-cd.md -> "CI toolchain image".) +defaults: + run: + shell: bash + env: REGISTRY: 192.168.1.95:3000 IMAGE: 192.168.1.95:3000/timothy/ersatztv @@ -40,6 +64,11 @@ jobs: test: name: Build & test (.NET) runs-on: ubuntu-latest + container: + image: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8 + credentials: + username: ${{ secrets.REGISTRY_USER }} + password: ${{ secrets.REGISTRY_PASSWORD }} steps: - name: Checkout uses: actions/checkout@v4 @@ -48,11 +77,6 @@ jobs: # are only needed by the `build` job's `git describe` (ersatztv#190) fetch-depth: 1 - - name: Setup .NET - uses: actions/setup-dotnet@v4 - with: - dotnet-version: '10.0.x' - - name: Cache NuGet packages uses: actions/cache@v4 with: @@ -63,12 +87,14 @@ jobs: - name: Restore run: dotnet restore - - name: Setup Node - uses: actions/setup-node@v4 + # Replaces setup-node's built-in `cache: npm`. The toolchain image supplies node/npm, but + # the SPA's package downloads are project deps, so they stay cached per lockfile. + - name: Cache npm packages + uses: actions/cache@v4 with: - node-version: '22.x' - cache: npm - cache-dependency-path: web/package-lock.json + path: ~/.npm + key: npm-${{ runner.os }}-${{ hashFiles('web/package-lock.json') }} + restore-keys: npm-${{ runner.os }}- - name: Install SPA dependencies working-directory: web @@ -121,10 +147,8 @@ jobs: exit 0 fi echo "Found ${#reports[@]} coverage report(s)." - # `update` is install-or-update (idempotent, unlike `install` which errors if the tool - # is already present under set -e); pinned for reproducible summary output. - dotnet tool update --global dotnet-reportgenerator-globaltool --version 5.5.10 >/dev/null - export PATH="$PATH:$HOME/.dotnet/tools" + # reportgenerator is baked into the CI toolchain image (docker/ci/Dockerfile) and already + # on PATH — no per-run `dotnet tool` install. Bump its version there (ersatztv#390). reportgenerator \ "-reports:coverage/**/coverage.cobertura.xml" \ "-targetdir:coverage/report" \ @@ -139,6 +163,11 @@ jobs: migrations: name: EF migration integrity (SQLite + MySql) runs-on: ubuntu-latest + container: + image: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8 + credentials: + username: ${{ secrets.REGISTRY_USER }} + password: ${{ secrets.REGISTRY_PASSWORD }} # Independent gate (not a 'needs' of build yet) so the new MySql-service dependency # can't block image builds until it's proven reliable on the runner. Promote to a # required check / build dependency once green. (ersatztv#13) @@ -162,11 +191,6 @@ jobs: # default fetch-depth: 1 -- this job never runs git describe/log, only # actions/checkout@v4's default (shallow) history is needed (ersatztv#190) - - name: Setup .NET - uses: actions/setup-dotnet@v4 - with: - dotnet-version: '10.0.x' - - name: Cache NuGet packages uses: actions/cache@v4 with: @@ -180,14 +204,13 @@ jobs: - name: Build run: dotnet build --configuration Release --no-restore - - name: Install dotnet-ef - run: dotnet tool install --global dotnet-ef --version 9.0.12 + # dotnet-ef is baked into the CI toolchain image (docker/ci/Dockerfile) and already on PATH + # — no per-run `dotnet tool install`. Bump its version there (ersatztv#390). # SQLite is the prod provider; both checks validated locally. - name: SQLite — model drift + apply all migrations to a fresh DB run: | set -euo pipefail - export PATH="$PATH:$HOME/.dotnet/tools" echo "::group::SQLite model drift (has-pending-model-changes)" dotnet ef migrations has-pending-model-changes --no-build --configuration Release \ --context TvContext --startup-project ErsatzTV --project ErsatzTV.Infrastructure.Sqlite -- --provider Sqlite @@ -210,7 +233,6 @@ jobs: MySql__ConnectionString: "Server=mysql;Port=3306;Database=ersatztv_migrations;Uid=root;Pwd=ersatztv;DefaultCommandTimeout=300;" run: | set -euo pipefail - export PATH="$PATH:$HOME/.dotnet/tools" echo "::group::MySql model drift (has-pending-model-changes)" dotnet ef migrations has-pending-model-changes --no-build --configuration Release \ --context TvContext --startup-project ErsatzTV --project ErsatzTV.Infrastructure.MySql -- --provider MySql @@ -246,17 +268,17 @@ jobs: # reliable (same rollout the `migrations` job used). SQLite default provider -> no DB service. # Runs on PRs and on main (regression net); skipped for v* tag builds. if: github.event_name == 'pull_request' || github.ref == 'refs/heads/main' + container: + image: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8 + credentials: + username: ${{ secrets.REGISTRY_USER }} + password: ${{ secrets.REGISTRY_PASSWORD }} steps: - name: Checkout uses: actions/checkout@v4 with: fetch-depth: 1 - - name: Setup .NET - uses: actions/setup-dotnet@v4 - with: - dotnet-version: '10.0.x' - - name: Cache NuGet packages uses: actions/cache@v4 with: @@ -267,12 +289,12 @@ jobs: - name: Restore run: dotnet restore - - name: Setup Node - uses: actions/setup-node@v4 + - name: Cache npm packages + uses: actions/cache@v4 with: - node-version: '22.x' - cache: npm - cache-dependency-path: web/package-lock.json + path: ~/.npm + key: npm-${{ runner.os }}-${{ hashFiles('web/package-lock.json') }} + restore-keys: npm-${{ runner.os }}- - name: Install SPA dependencies working-directory: web @@ -285,8 +307,11 @@ jobs: - name: Build (Release) run: dotnet build ErsatzTV.sln --configuration Release --no-restore - - name: Ensure ffmpeg is available - run: command -v ffmpeg >/dev/null 2>&1 || (sudo apt-get update && sudo apt-get install -y ffmpeg) + # The old `command -v ffmpeg || sudo apt-get install ffmpeg` step is gone (ersatztv#390): + # the toolchain image ships the same ffmpeg build prod runs, so the binary is already here. + # That step also cost 110s of every run to install a dependency this harness never uses — + # it boots the app (whose only ffmpeg touch at startup is a LogWarning from + # FFmpegLocatorService) and drives curl-only contracts that never transcode. - name: Boot instance and run functional-E2E harness run: | @@ -428,6 +453,57 @@ jobs: exit 1 fi + # BLOCKING (ersatztv#390): the CI toolchain image pin in this file must name the image that + # ci-image.yml actually last published — i.e. the short sha of the last commit to touch the image's + # sources. Without this detector, a PR that edits docker/ci/** publishes a NEW image but runs its own + # jobs against the OLD pin: CI green-lights a toolchain it never executed, and once merged, main's + # Dockerfile silently disagrees with what CI runs. **Renovate actively generates exactly that PR** — + # it manages docker/ci/Dockerfile's base pins (dockerfile manager) but cannot bump an opaque + # `:` in `container.image`, so it would leave the pin behind every time. + # + # Failing here forces the documented two-step (docs/ci-cd.md -> "CI toolchain image"): push the + # Dockerfile change, let ci-image.yml publish `:`, then update the pin to that sha. Seconds-long + # git+grep -> keep it off the build runners. + ci-image-pin: + name: CI image pin matches docker/ci + runs-on: small + if: github.event_name == 'pull_request' + steps: + - name: Checkout + uses: actions/checkout@v4 + with: + # need real history: `git log -- ` on a shallow clone can't find the last + # commit that touched the image sources + fetch-depth: 0 + - name: Verify the pin matches the last-published image + run: | + set -euo pipefail + # ci-image.yml tags the image `git rev-parse --short HEAD` of the push that built it, and it + # only builds on pushes touching these paths — so the published image is named by the last + # commit to touch them. + # + # Compare RESOLVED FULL shas, never the abbreviations: git auto-scales abbreviation length + # with the repo's object count, so the tag built in CI from a `fetch-depth: 1` shallow clone + # is 7 chars while `%h` here (full clone) is 8. Comparing those strings would fail always. + expected="$(git log -1 --format=%H -- docker/ci .gitea/workflows/ci-image.yml)" + mapfile -t pins < <(grep -oE 'ersatztv-ci:[0-9a-f]+' .gitea/workflows/docker-build.yml | cut -d: -f2 | sort -u) + echo "Image sources last changed in: ${expected}" + echo "Pins found in docker-build.yml: ${pins[*]} (${#pins[@]} distinct)" + if [ "${#pins[@]}" -ne 1 ]; then + echo "::error::docker-build.yml pins MORE THAN ONE ersatztv-ci tag (${pins[*]}). All jobs must pin the same image — bump them together." + exit 1 + fi + pin_full="$(git rev-parse --verify --quiet "${pins[0]}^{commit}" || true)" + if [ -z "$pin_full" ]; then + echo "::error::The pinned CI image tag ersatztv-ci:${pins[0]} does not resolve to a commit in this repo, so it cannot correspond to an image ci-image.yml built from these sources. Rebuild the image and pin the sha it prints." + exit 1 + fi + if [ "$pin_full" != "$expected" ]; then + echo "::error::CI toolchain image pin is stale: docker-build.yml pins ersatztv-ci:${pins[0]} ($pin_full), but docker/ci was last changed in $expected. Your jobs are testing an image that is NOT built from this PR's docker/ci. Let ci-image.yml publish the new :, then update the pin in ALL jobs to it (docs/ci-cd.md -> 'CI toolchain image')." + exit 1 + fi + echo "Pin is current: ersatztv-ci:${pins[0]} resolves to $pin_full = docker/ci's last change." + # Non-blocking nudge: if a PR migrates/adds a route but forgets the parity tracker, warn. # The rule lives in CLAUDE.md → Conventions; this only surfaces an easy-to-miss omission. # Deliberately no setup-dotnet/setup-node (and thus no actions/cache) so it can't hit the @@ -503,7 +579,20 @@ jobs: # API path changed, the expensive steps skip and the job passes trivially. api-docs: name: API docs in sync (OpenAPI + endpoint index) - runs-on: ubuntu-latest + # `small` lane (ersatztv#390): this job is ~5s on the ~90% of PRs that touch no API path, but + # it was queueing ~29 min behind the heavy jobs in the contended `ubuntu-latest` lane, which + # only has bumblebee-runner (capacity 2) + ci-runner. `small` has capacity 4, the same base + # image, and answers in ~5s. Moving it here (and `format`) also drops `ubuntu-latest` from 5 + # jobs to 3, which shortens the queue for `test`/`migrations`/`functional-e2e` too. + # This is only possible because `container:` makes the job self-contained — it no longer needs + # the runner image to supply .NET/Node. Caveat: on an API-touching PR this job does a full + # `dotnet build`, so it is not always a "small" job; capacity 4 absorbs that. + runs-on: small + container: + image: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8 + credentials: + username: ${{ secrets.REGISTRY_USER }} + password: ${{ secrets.REGISTRY_PASSWORD }} if: github.event_name == 'pull_request' steps: - name: Checkout @@ -526,12 +615,6 @@ jobs: echo "No API-surface change -> skipping regeneration (job passes)." fi - - name: Setup .NET - if: steps.detect.outputs.api_changed == 'true' - uses: actions/setup-dotnet@v4 - with: - dotnet-version: '10.0.x' - - name: Cache NuGet packages if: steps.detect.outputs.api_changed == 'true' uses: actions/cache@v4 @@ -544,13 +627,13 @@ jobs: if: steps.detect.outputs.api_changed == 'true' run: dotnet restore - - name: Setup Node + - name: Cache npm packages if: steps.detect.outputs.api_changed == 'true' - uses: actions/setup-node@v4 + uses: actions/cache@v4 with: - node-version: '22.x' - cache: npm - cache-dependency-path: web/package-lock.json + path: ~/.npm + key: npm-${{ runner.os }}-${{ hashFiles('web/package-lock.json') }} + restore-keys: npm-${{ runner.os }}- - name: Install SPA dependencies if: steps.detect.outputs.api_changed == 'true' @@ -585,7 +668,14 @@ jobs: # a status, so it is safe as a required check). format: name: Formatting (changed .cs conform to .editorconfig) - runs-on: ubuntu-latest + # `small` lane (ersatztv#390) — 37s of work that was queueing ~29 min in the contended + # `ubuntu-latest` lane. See the api-docs job above for the full rationale. + runs-on: small + container: + image: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8 + credentials: + username: ${{ secrets.REGISTRY_USER }} + password: ${{ secrets.REGISTRY_PASSWORD }} if: github.event_name == 'pull_request' steps: - name: Checkout @@ -609,12 +699,6 @@ jobs: echo "No .cs change -> skipping format verify (job passes)." fi - - name: Setup .NET - if: steps.detect.outputs.cs_changed == 'true' - uses: actions/setup-dotnet@v4 - with: - dotnet-version: '10.0.x' - - name: Cache NuGet packages if: steps.detect.outputs.cs_changed == 'true' uses: actions/cache@v4 diff --git a/docker/ci/Dockerfile b/docker/ci/Dockerfile new file mode 100644 index 000000000..b3e43b265 --- /dev/null +++ b/docker/ci/Dockerfile @@ -0,0 +1,100 @@ +# Shared CI toolchain image (ersatztv#390) — .NET 10 SDK + Node 22 + prod-identical ffmpeg. +# +# Built/pushed by .gitea/workflows/ci-image.yml to 192.168.1.95:3000/timothy/ersatztv-ci. +# The toolchain jobs in docker-build.yml run *inside* this image via `container:`, so they +# install nothing at run time: no setup-dotnet, no setup-node, no `apt-get install ffmpeg`, +# no `dotnet tool install`. Project dependencies (NuGet/npm) are NOT baked in — those change +# per commit and stay on actions/cache. +# +# Layering rationale: we start from our OWN ffmpeg base and copy the .NET SDK on top, which is +# exactly the pattern docker/Dockerfile uses for the prod image (`FROM ersatztv-ffmpeg` + +# `COPY --from=dotnet-runtime /usr/share/dotnet`). That base is ghcr.io/linuxserver/baseimage-ubuntu:noble, +# the same Ubuntu release as mcr.microsoft.com/dotnet/sdk:10.0-noble, so the copied SDK matches +# the base's glibc/ICU. It also gives CI the *prod* ffmpeg build rather than a generic apt one — +# the fidelity that the ersatztv#299 seeded-media/scanner E2E follow-ups will need. +# +# Keep the pins below in sync with what CI actually needs; Renovate tracks the image pins +# (see renovate.json). The two dotnet tool versions mirror what docker-build.yml used to +# `dotnet tool install` per run — bump them here, not in the workflow. + +# Keep the ffmpeg tag on the FROM below equal to the one docker/Dockerfile pins, so CI's ffmpeg +# stays prod-identical. Renovate manages both pins (dockerfile manager); mirror any bump here. +FROM --platform=linux/amd64 192.168.1.95:3000/timothy/ersatztv-ffmpeg:8.1.2 AS ci-base + +# The ffmpeg base sets ENTRYPOINT ["ffmpeg"] / CMD ["--help"] because it ships as an ffmpeg CLI. +# This image is a CI *job container*: the runner starts it long-lived and execs steps into it, +# so an ffmpeg entrypoint is wrong here. Reset both. +ENTRYPOINT [] +CMD ["/bin/bash"] + +ENV DEBIAN_FRONTEND=noninteractive + +# Toolchain the workflow steps and the actions themselves need: +# git — actions/checkout +# ca-certificates, curl — NodeSource setup + general fetches +# tar, zstd, unzip, xz — actions/cache archive round-trip +# python3 — scripts/generate-endpoint-index.py (the api-docs job) +# jq — ad-hoc JSON in CI steps +# The ffmpeg base is a runtime image, so none of these are guaranteed present. +RUN apt-get update && \ + apt-get install -y --no-install-recommends \ + ca-certificates \ + curl \ + git \ + jq \ + python3 \ + tar \ + unzip \ + xz-utils \ + zstd && \ + apt-get clean -y && \ + rm -rf /var/lib/apt/lists/* + +# Node 22 (NodeSource) — matches the `node-version: '22.x'` the workflow used to request from +# setup-node, and the node:22 base docker/Dockerfile builds the SPA with. act_runner also needs a +# `node` on PATH inside the job container to execute JavaScript actions (actions/checkout, cache). +RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - && \ + apt-get install -y --no-install-recommends nodejs && \ + apt-get clean -y && \ + rm -rf /var/lib/apt/lists/* + +# actions/checkout clones as root into a runner-mounted workspace, which trips git's +# "detected dubious ownership in repository" guard and breaks every `git` call in a step +# (scripts/e2e-local.sh's `git rev-parse --show-toplevel`, the base-ref diffs, ...). Trusting all +# paths is the right call for a throwaway single-tenant CI container. +RUN git config --global --add safe.directory '*' + +# .NET 10 SDK. Copied from the official SDK image rather than dotnet-install.sh: same mechanism as +# docker/Dockerfile's runtime copy, one pinned artifact, no install script to fail at build time. +# global.json pins sdk 10.0.0 with rollForward latestMinor, which this satisfies. +COPY --from=mcr.microsoft.com/dotnet/sdk:10.0-noble-amd64 /usr/share/dotnet /usr/share/dotnet + +ENV DOTNET_ROOT=/usr/share/dotnet \ + PATH="/usr/share/dotnet:/root/.dotnet/tools:${PATH}" \ + DOTNET_CLI_TELEMETRY_OPTOUT=1 \ + DOTNET_NOLOGO=1 \ + DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 + +# Global dotnet tools the jobs used to install per run (5s + ~4s each, every run, forever). +# dotnet-ef — the migrations job (was: `dotnet tool install --global dotnet-ef --version 9.0.12`) +# dotnet-reportgenerator-globaltool — the test job's coverage summary +# Baked here so the versions live in one tracked place instead of inline workflow strings. +RUN dotnet tool install --global dotnet-ef --version 9.0.12 && \ + dotnet tool install --global dotnet-reportgenerator-globaltool --version 5.5.10 + +# Build-time smoke test: fail the IMAGE build, not a CI run, if any part of the toolchain is +# unusable. Notably `dotnet --info` catches a missing ICU/glibc dependency in the base — the one +# real risk of copying the SDK onto a non-Microsoft base. +RUN set -eux; \ + dotnet --info; \ + dotnet ef --version; \ + node --version; \ + npm --version; \ + git --version; \ + python3 --version; \ + ffmpeg -version; \ + ffprobe -version; \ + # reportgenerator has no clean version probe: `--version` prints the banner and then exits 1 + # ("No report files specified"), so asserting on it would fail this build. Probe that the + # dotnet-tool shim resolves on PATH instead. + command -v reportgenerator diff --git a/docs/ci-cd.md b/docs/ci-cd.md index c400c9f1a..532bb8666 100644 --- a/docs/ci-cd.md +++ b/docs/ci-cd.md @@ -90,6 +90,29 @@ jobs capped `--cpus=4 --memory=10g` so CI can't starve prod media playback), and runner task even when its `if` skips it, and those skip-tasks used to wait behind long builds (observed 31 min) stalling every PR run. +**Lane assignment (ersatztv#390).** The `ubuntu-latest` lane has **4 slots** (2 + 2) and the +`small` lane has 4. A 2026-07-17 audit of the Actions API found the `ubuntu-latest` lane +saturated and the `small` lane idle — **queue wait exceeded every job's runtime**: + +| Job | Runtime | Queue wait | Lane | +|---|---|---|---| +| `test` | 354s | 1363s | ubuntu-latest | +| `migrations` | 639s | 1428s | ubuntu-latest | +| `functional-e2e` | 520s | 1447s | ubuntu-latest | +| `api-docs` | 5s | **1722s** | ubuntu-latest → now `small` | +| `format` | 37s | **1731s** | ubuntu-latest → now `small` | +| `docs-reminder` / `decisions-guard` | 10s | **5s** | small | + +So `api-docs` and `format` — genuinely small jobs — moved to `small`, which also drops the +contended lane from 5 jobs to 3 and shortens the queue for the three heavy jobs. Both lanes run +the *identical* `runner-images:ubuntu-latest` base, so `small` is a label with spare capacity, not +a different capability. This move is only possible because those jobs now run in the **CI toolchain +image** (below) and no longer need the runner image to supply .NET/Node. Caveat: on an +API-touching PR `api-docs` does a full `dotnet build`, so it is not *always* small. + +Queue wait is still the dominant cost and capacity is server-management's boundary — tracked in +**server-management#604**. The redundant triple-build behind those runtimes is **ersatztv#398**. + ### `test` job `dotnet restore` → strip the Scanner project ref (`sed -i '/Scanner/d'`, matching the @@ -142,7 +165,8 @@ Boots the app **from source** and drives the manual live-E2E curl flows sessions re-run by hand, turning them into a CI regression net (ersatztv#299). It is the automatable half of `docs/e2e-local.md`; the two scripts it chains run identically locally and in CI: -1. `npm ci` + `npm run build` (SPA), `dotnet build ErsatzTV.sln -c Release`, ensure `ffmpeg` is on PATH. +1. `npm ci` + `npm run build` (SPA), `dotnet build ErsatzTV.sln -c Release`. (ffmpeg comes from the + CI toolchain image; the old `apt-get install ffmpeg` step is gone — see below.) 2. `ETV_BUILD_CONFIG=Release scripts/e2e-local.sh ` — copies `wwwroot`, launches `dotnet ErsatzTV.dll` in the background (logging to a file so the launch step returns once the app is ready), prints `PID`/`CONFIG_DIR`. @@ -191,6 +215,71 @@ accepts the token in *any* commit of the range). The local hook is therefore the gate; CI is the backstop for direct pushes or bypassed hooks. Like `docs-reminder`, it's a seconds-long `git diff` with no dotnet/node setup (`runs-on: small`). +## CI toolchain image (`docker/ci/Dockerfile`, `.gitea/workflows/ci-image.yml`) + +The jobs that need a toolchain — `test`, `migrations`, `functional-e2e`, `api-docs`, `format` — +run **inside** a shared image via `container:` instead of installing their toolchain per run +(ersatztv#390). They therefore carry **no** `setup-dotnet`, **no** `setup-node`, **no** `apt-get`, +and **no** `dotnet tool install`. + +**What it ships:** .NET 10 SDK, Node 22, prod-identical ffmpeg/ffprobe, `git`/`python3`/`jq`/`zstd`, +and the `dotnet-ef` + `dotnet-reportgenerator-globaltool` global tools (which the `migrations` and +`test` jobs used to install on every run — **bump those versions in the Dockerfile, not the +workflow**). Project dependencies (NuGet/npm) are deliberately **not** baked in — they change per +commit and stay on `actions/cache` (`~/.nuget/packages`, `~/.npm`). + +**How it's layered:** `FROM ersatztv-ffmpeg:8.1.2` + `COPY --from=mcr.microsoft.com/dotnet/sdk:10.0-noble-amd64 +/usr/share/dotnet` — the same pattern `docker/Dockerfile` uses for the prod image. Our ffmpeg base is +`ghcr.io/linuxserver/baseimage-ubuntu:noble`, the same Ubuntu release as the SDK image, so the copied +SDK matches the base's glibc/ICU. Keep the ffmpeg tag on that `FROM` equal to the one +`docker/Dockerfile` pins, so CI's ffmpeg stays prod-identical — that fidelity is what the +ersatztv#299 seeded-media/scanner E2E follow-ups will need. + +**Bumping the toolchain is a deliberate two-step.** The jobs pin an immutable `:`, never +`:latest`, so a bad toolchain push cannot break every job at once: + +1. Merge a `docker/ci/Dockerfile` change. `ci-image.yml` publishes `ersatztv-ci:` (+ `:latest` + from `main` only — a human pointer; **jobs must never consume it**). +2. In a follow-up PR, update the pin in `docker-build.yml` — **all five jobs together**. That PR's own + CI is what proves the new image works. The pin is repeated per job because + `jobs..container.image` cannot read the workflow `env` context. + +`ci-image.yml` triggers on pushes touching `docker/ci/**`, `workflow_dispatch`, and a weekly Monday +05:00 UTC cron (base-image security updates; Gitea registers `schedule` only from `main`). It runs on +the `small` lane and, like `docker-build.yml`, needs BuildKit's inline `http = true` for the HTTP +registry. Renovate tracks the Dockerfile's image pins (`dockerfile` manager, see `renovate.json`). + +**Three container-specific gotchas** — worth knowing if you add a job or a step: +- **`sh`, not bash, is the default shell inside a `container:`.** act_runner runs steps as + `sh -e {0}` (dash) because it can't assume bash exists in an arbitrary image — even though ours + has it. Every bashism (`set -o pipefail`, arrays, `shopt`, `mapfile`) then dies instantly with + `set: Illegal option -o pipefail`. `docker-build.yml` therefore declares a workflow-level + `defaults: run: shell: bash`. **If you add a workflow with containerized jobs, do the same** — + outside a container the shell defaults to bash, so this failure only appears once you containerize + and it looks nothing like a shell problem (it surfaced as the `migrations` job dying in 0.13s). +- `actions/checkout` clones as root into a mounted workspace, which trips git's *"detected dubious + ownership"* guard and breaks every `git` call in a step. Fixed in the Dockerfile with + `git config --global --add safe.directory '*'`. +- *(defensive, not load-bearing)* The ffmpeg base sets `ENTRYPOINT ["ffmpeg"]` because it ships as an + ffmpeg CLI, so the Dockerfile resets `ENTRYPOINT`/`CMD`. act overrides the entrypoint anyway + (`entrypoint=["/bin/sleep" "10800"]`), so this is belt-and-braces for anyone running the image by + hand — unlike the two above, which are real. + +**Bumping the pin is enforced, not remembered.** The `ci-image-pin` job (blocking, PR-only) fails if +`docker-build.yml`'s pin isn't the short sha of the last commit to touch `docker/ci/**` or +`ci-image.yml`, or if the five jobs ever pin different tags. This exists because **Renovate manages +`docker/ci/Dockerfile`'s base pins but cannot bump an opaque `:`** in `container.image` — so a +Renovate base bump would otherwise publish a new image, test the *old* one, and merge with the +Dockerfile disagreeing with the pin. A red `ci-image-pin` means: let `ci-image.yml` publish the new +`:`, then update all five pins to it. + +**What it is and isn't worth.** Measured honestly (ersatztv#390): the image saves ~15–40s per job +(`setup-dotnet` is 8–19s, `setup-node` 2–5s cached, the two tool installs ~9s) plus the 110s +apt-ffmpeg step — roughly 3–8% of runtime. It is **not** where CI time goes; see the lane table above +(queue wait, server-management#604) and ersatztv#398 (742s of redundant compilation). Its durable +value is prod-identical ffmpeg, a pinned/consistent toolchain, and making jobs runner-agnostic — the +last is what allowed the lane rebalance. + ## Dockerfile notes (`docker/Dockerfile`) - Base image: **`192.168.1.95:3000/timothy/ersatztv-ffmpeg:8.1.2`** (our Gitea fork of