ci: harden build pipeline per adversarial review
- Add concurrency group so the single jazz runner can't run the push-main-then-push-tag release flow in parallel (shared :buildcache + smoke container would collide). - Add pull_request trigger running the test job only (PRs had no gate); skip the build job on PRs. - Only push images from main or a v* tag (workflow_dispatch from other refs now builds without publishing, instead of clobbering :latest/:prod). - Replace the log-grep smoke check with a real HTTP readiness probe (docker exec python3 -> http://localhost:8409/), unique container name, and trap-based cleanup to avoid leaks on cancel. - dotnet test now runs -c Release --no-build (was rebuilding in Debug). - Directory.Build.props: WarningsAsErrors=NU1904 so critical NuGet advisories block in every project, not just ones with TreatWarningsAsErrors. - Dockerfile copies global.json + .editorconfig too, so the image build matches CI's SDK pin and analyzer severities. - Remove dead .github/dependabot.yml + FUNDING.yml (upstream-pointed). - Rewrite docs/ci-cd.md to the implemented pipeline. Refs #4, #3, #8.
This commit is contained in:
@@ -1,22 +1,33 @@
|
||||
name: Build ErsatzTV Image
|
||||
|
||||
# Builds the fork's own amd64 image and pushes it to the Gitea container registry.
|
||||
# pull_request -> test job only (no image build/push)
|
||||
# push to main -> :latest + :<short-sha> (test image; does NOT touch prod)
|
||||
# push tag v* -> :prod + :<version> + :<short-sha> (prod release)
|
||||
# workflow_dispatch -> manual run (behaves like the branch it runs on)
|
||||
# workflow_dispatch -> manual run; only publishes when the ref is main or a v* tag
|
||||
#
|
||||
# Runner + registry provisioned in server-management#172. The Gitea registry is
|
||||
# HTTP-only, so BuildKit needs the inline `http = true` config below (it does not
|
||||
# inherit the host daemon's insecure-registries setting).
|
||||
#
|
||||
# `:latest` is intentionally the test/dev channel (per ersatztv#3); prod pins
|
||||
# `:prod`, never `:latest` (enforced in the prod compose — server-management#481).
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
tags:
|
||||
- 'v*'
|
||||
|
||||
# Single runner on jazz: serialize all runs so the push-main-then-tag release
|
||||
# flow can't collide on the shared :buildcache tag or the smoke container.
|
||||
concurrency:
|
||||
group: ersatztv-build
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
REGISTRY: 192.168.1.95:3000
|
||||
IMAGE: 192.168.1.95:3000/timothy/ersatztv
|
||||
@@ -46,12 +57,13 @@ jobs:
|
||||
run: dotnet build --configuration Release --no-restore
|
||||
|
||||
- name: Test
|
||||
run: dotnet test --blame-hang-timeout "2m" --no-restore --verbosity normal
|
||||
run: dotnet test --configuration Release --no-build --blame-hang-timeout "2m" --verbosity normal
|
||||
|
||||
build:
|
||||
name: Build & push image (amd64)
|
||||
runs-on: ubuntu-latest
|
||||
needs: test
|
||||
if: github.event_name != 'pull_request'
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
@@ -101,7 +113,8 @@ jobs:
|
||||
context: .
|
||||
file: ./docker/Dockerfile
|
||||
platforms: linux/amd64
|
||||
push: true
|
||||
# only publish from main or a v* tag; other refs (e.g. branch dispatch) build only
|
||||
push: ${{ github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v') }}
|
||||
provenance: false
|
||||
build-args: |
|
||||
INFO_VERSION=${{ steps.meta.outputs.info_version }}
|
||||
@@ -109,32 +122,43 @@ jobs:
|
||||
cache-from: type=registry,ref=192.168.1.95:3000/timothy/ersatztv:buildcache
|
||||
cache-to: type=registry,ref=192.168.1.95:3000/timothy/ersatztv:buildcache,mode=max,ignore-error=true
|
||||
|
||||
- name: Smoke test (container boots)
|
||||
- name: Smoke test (HTTP readiness)
|
||||
if: ${{ github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v') }}
|
||||
run: |
|
||||
IMG="${IMAGE}:${{ steps.meta.outputs.short }}"
|
||||
docker rm -f etv-smoke >/dev/null 2>&1 || true
|
||||
NAME="etv-smoke-${{ github.run_id }}"
|
||||
trap 'docker rm -f "$NAME" >/dev/null 2>&1 || true' EXIT
|
||||
echo "Pulling ${IMG}"
|
||||
docker pull "$IMG"
|
||||
docker run -d --name etv-smoke \
|
||||
docker run -d --name "$NAME" --memory 2g \
|
||||
-e ETV_CONFIG_FOLDER=/tmp/etv/config \
|
||||
-e ETV_TRANSCODE_FOLDER=/tmp/etv/transcode \
|
||||
"$IMG"
|
||||
# probe ErsatzTV's web server from inside the container (image ships python3)
|
||||
cat > probe.py <<'PY'
|
||||
import urllib.request, urllib.error, sys
|
||||
try:
|
||||
urllib.request.urlopen("http://localhost:8409/", timeout=3)
|
||||
except urllib.error.HTTPError:
|
||||
pass # any HTTP status means the server is serving
|
||||
except Exception:
|
||||
sys.exit(1) # not listening yet
|
||||
PY
|
||||
ok=0
|
||||
for _ in $(seq 1 45); do
|
||||
if [ -z "$(docker ps -q --filter name=etv-smoke --filter status=running)" ]; then
|
||||
for _ in $(seq 1 60); do
|
||||
if [ -z "$(docker ps -q --filter name="$NAME" --filter status=running)" ]; then
|
||||
echo "Container exited early"; break
|
||||
fi
|
||||
if docker logs etv-smoke 2>&1 | grep -Eiq "service started|Located ffmpeg|Now listening on"; then
|
||||
if docker exec -i "$NAME" python3 - < probe.py >/dev/null 2>&1; then
|
||||
ok=1; break
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
echo "===== container logs (tail) ====="
|
||||
docker logs etv-smoke 2>&1 | tail -n 60 || true
|
||||
docker logs "$NAME" 2>&1 | tail -n 40 || true
|
||||
echo "================================="
|
||||
docker rm -f etv-smoke >/dev/null 2>&1 || true
|
||||
if [ "$ok" != "1" ]; then
|
||||
echo "Smoke test FAILED: app did not report startup"
|
||||
echo "Smoke test FAILED: ErsatzTV did not serve HTTP on :8409"
|
||||
exit 1
|
||||
fi
|
||||
echo "Smoke test passed"
|
||||
echo "Smoke test passed: ErsatzTV served HTTP on :8409"
|
||||
|
||||
@@ -1,2 +0,0 @@
|
||||
github: jasongdove
|
||||
custom: "https://www.paypal.me/jasongdove"
|
||||
@@ -1,26 +0,0 @@
|
||||
version: 2
|
||||
updates:
|
||||
- package-ecosystem: nuget
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: daily
|
||||
assignees:
|
||||
- jasongdove
|
||||
- package-ecosystem: docker
|
||||
directory: "/docker"
|
||||
schedule:
|
||||
interval: daily
|
||||
assignees:
|
||||
- jasongdove
|
||||
- package-ecosystem: docker
|
||||
directory: "/docker/nvidia"
|
||||
schedule:
|
||||
interval: daily
|
||||
assignees:
|
||||
- jasongdove
|
||||
- package-ecosystem: docker
|
||||
directory: "/docker/vaapi"
|
||||
schedule:
|
||||
interval: daily
|
||||
assignees:
|
||||
- jasongdove
|
||||
@@ -8,7 +8,10 @@
|
||||
TreatWarningsAsErrors=true, which would otherwise fail `dotnet restore`
|
||||
on advisories we can't immediately fix. Demote low/moderate/high audit
|
||||
advisories to warnings (still printed in build logs); NU1904 (critical)
|
||||
stays an error so criticals still block. Track fixes separately. -->
|
||||
stays an error so criticals still block. Track fixes separately.
|
||||
WarningsAsErrors promotes NU1904 in EVERY project (even those without
|
||||
TreatWarningsAsErrors), so "criticals block" actually holds repo-wide. -->
|
||||
<WarningsNotAsErrors>$(WarningsNotAsErrors);NU1901;NU1902;NU1903</WarningsNotAsErrors>
|
||||
<WarningsAsErrors>$(WarningsAsErrors);NU1904</WarningsAsErrors>
|
||||
</PropertyGroup>
|
||||
</Project>
|
||||
|
||||
+4
-3
@@ -24,9 +24,10 @@ COPY scripts/scripted-schedules/. /app/scripted-schedules/
|
||||
|
||||
# copy csproj and restore as distinct layers
|
||||
COPY *.sln .
|
||||
# repo-wide MSBuild props/targets (incl. NuGet-audit warning exemptions) must be
|
||||
# present before restore so the image build matches local/CI builds
|
||||
COPY Directory.Build.props Directory.Build.targets ./
|
||||
# repo-wide build config (MSBuild props/targets incl. NuGet-audit warning
|
||||
# exemptions, SDK pin, analyzer severities) must be present before restore so the
|
||||
# image build matches local/CI builds
|
||||
COPY Directory.Build.props Directory.Build.targets global.json .editorconfig ./
|
||||
COPY artwork/* ./artwork/
|
||||
COPY ErsatzTV/*.csproj ./ErsatzTV/
|
||||
COPY ErsatzTV.Application/*.csproj ./ErsatzTV.Application/
|
||||
|
||||
+85
-49
@@ -1,58 +1,94 @@
|
||||
# CI/CD Details for ErsatzTV Fork
|
||||
# CI/CD for the ErsatzTV Fork
|
||||
|
||||
## Upstream GitHub Actions (to adapt for Gitea)
|
||||
The fork builds its own Docker image via **Gitea Actions** on the homelab and pushes
|
||||
to the **Gitea container registry**. Runner + registry were provisioned in
|
||||
server-management#172; the build pipeline is ersatztv#4; test/prod containers are
|
||||
server-management#481.
|
||||
|
||||
### Workflows
|
||||
| File | Trigger | Purpose |
|
||||
|------|---------|---------|
|
||||
| `ci.yml` | Push to main | Version calc → docker.yml + artifacts.yml |
|
||||
| `release.yml` | GitHub Release | Same but with release version tags |
|
||||
| `docker.yml` | Reusable workflow | Multi-arch Docker build+push (amd64, arm32v7, arm64) |
|
||||
| `artifacts.yml` | Reusable workflow | Platform binaries (macOS DMG, Windows exe, Linux) |
|
||||
| `pr.yml` | Pull request | Build+test on Linux, Windows, Mac |
|
||||
## The workflow: `.gitea/workflows/docker-build.yml`
|
||||
|
||||
### What to keep for Gitea fork
|
||||
- **Docker build**: Simplify to amd64-only (jazz is x86_64), push to Gitea registry or local registry
|
||||
- **PR checks**: Build + test on Linux only (single runner on jazz)
|
||||
- **Drop**: macOS/Windows artifacts, code signing, multi-arch, GHCR/DockerHub push
|
||||
Single workflow, two jobs (`test` → `build`).
|
||||
|
||||
### Build Steps (from pr.yml — the test workflow)
|
||||
```bash
|
||||
dotnet restore
|
||||
sed -i '/Scanner/d' ErsatzTV/ErsatzTV.csproj # strip Scanner project ref
|
||||
dotnet build --configuration Release --no-restore
|
||||
dotnet test --blame-hang-timeout "2m" --no-restore --verbosity normal
|
||||
```
|
||||
### Triggers & tags
|
||||
|
||||
### Docker Build (from docker.yml)
|
||||
- Uses `docker/build-push-action@v5`
|
||||
- Dockerfile: `docker/Dockerfile` (amd64), `docker/arm32v7/Dockerfile`, `docker/arm64/Dockerfile`
|
||||
- Build arg: `INFO_VERSION` for version stamp
|
||||
- Base image: `ghcr.io/ersatztv/ersatztv-ffmpeg:7.1.1` (custom ffmpeg)
|
||||
- Needs Java in build stage (OpenAPI generator)
|
||||
- Multi-stage: ffmpeg base → .NET runtime → build → final
|
||||
| Trigger | `test` job | `build` job | Image tags pushed |
|
||||
|---------|:----------:|:-----------:|-------------------|
|
||||
| `pull_request` | ✅ | — (skipped) | none |
|
||||
| push to `main` | ✅ | ✅ | `:latest` + `:<short-sha>` |
|
||||
| push tag `v*` | ✅ | ✅ | `:prod` + `:<version>` + `:<short-sha>` |
|
||||
| `workflow_dispatch` | ✅ | ✅ | only if ref is `main`/`v*`, else build-only (no push) |
|
||||
|
||||
### Image References to Update
|
||||
- `jasongdove/ersatztv` → fork's registry image name
|
||||
- `ghcr.io/ersatztv/ersatztv` → Gitea registry or local registry
|
||||
- `ghcr.io/ersatztv/ersatztv-ffmpeg:7.1.1` — still needed as base; could mirror locally
|
||||
`:latest` is the **test/dev** channel (every `main` commit). Prod pins **`:prod`**,
|
||||
never `:latest` — enforced in the prod compose (server-management#481). `:prod` is
|
||||
only produced by pushing a `v*` tag.
|
||||
|
||||
### Gitea Actions Compatibility Notes
|
||||
- Gitea Actions is GitHub Actions compatible but some actions need replacements
|
||||
- `actions/checkout@v4` → works in Gitea Actions
|
||||
- `actions/setup-dotnet@v4` → works in Gitea Actions
|
||||
- `docker/login-action@v3` → needs Gitea registry credentials instead
|
||||
- `docker/build-push-action@v5` → works but target registry changes
|
||||
- `actions/upload-artifact@v4` / `download-artifact@v4` → works in Gitea Actions
|
||||
- Reusable workflows (`workflow_call`) → supported in Gitea Actions
|
||||
`concurrency: { group: ersatztv-build, cancel-in-progress: false }` serializes all
|
||||
runs — the single runner on jazz can't safely run the push-`main`-then-push-`v*`
|
||||
release flow in parallel (shared `:buildcache` tag, shared smoke container).
|
||||
|
||||
### Container Registry Options (from server-management#172)
|
||||
1. **Gitea built-in** (Packages feature) — images at `192.168.1.95:3000/timothy/<package>`, needs HTTPS or `--insecure-registry`
|
||||
2. **Local Docker registry** — `registry:2` on jazz at port 5000
|
||||
### `test` job
|
||||
|
||||
### Runner Setup (from server-management#172)
|
||||
- `gitea/act_runner:latest` container on jazz
|
||||
- Needs Docker socket for DinD builds
|
||||
- Register via Gitea Admin → Runners → token
|
||||
- Labels: `ubuntu-latest`, `docker`
|
||||
- jazz has 128GB RAM, plenty for .NET builds (2-4GB)
|
||||
`dotnet restore` → strip the Scanner project ref (`sed -i '/Scanner/d'`, matching the
|
||||
Docker build) → `dotnet build -c Release` → `dotnet test -c Release --no-build`. Gates
|
||||
the image build.
|
||||
|
||||
### `build` job
|
||||
|
||||
1. Compute `INFO_VERSION` (`git describe` + short sha on `main`; tag version on `v*`).
|
||||
2. `docker/setup-buildx-action` with `buildkitd-config-inline` setting `http = true`
|
||||
for `192.168.1.95:3000` — **BuildKit does not inherit the host daemon's
|
||||
`insecure-registries`**, so without this, cache/base-image/push over the HTTP
|
||||
registry fails (`http: server gave HTTP response to HTTPS client`).
|
||||
3. `docker/login-action` with repo secrets `REGISTRY_USER` / `REGISTRY_PASSWORD`.
|
||||
4. `docker/build-push-action@v6`: amd64-only, `docker/Dockerfile`, `INFO_VERSION`
|
||||
build-arg, registry layer cache (`type=registry,ref=…:buildcache`,
|
||||
`cache-to … ignore-error=true`).
|
||||
5. **Smoke test**: pull the just-pushed `:<sha>`, run it, and poll ErsatzTV's web
|
||||
server from inside the container (`docker exec … python3` → `http://localhost:8409/`).
|
||||
Unique container name + `trap … EXIT` cleanup. Fails if it never serves HTTP.
|
||||
|
||||
## Dockerfile notes (`docker/Dockerfile`)
|
||||
|
||||
- Base image: **`192.168.1.95:3000/timothy/ersatztv-ffmpeg:7.1.1`** (our Gitea fork of
|
||||
the archived `ghcr.io/ersatztv/ersatztv-ffmpeg`). FFmpeg 8 upgrade is backlogged:
|
||||
base image → ersatztv-ffmpeg#4, app-side compat → ersatztv#9.
|
||||
- Copies `Directory.Build.props`, `Directory.Build.targets`, `global.json`,
|
||||
`.editorconfig` before `dotnet restore` so the image build uses the same MSBuild
|
||||
config, SDK pin, and analyzer severities as local/CI builds (it previously copied
|
||||
only `*.sln`).
|
||||
- amd64-only (jazz is x86_64). No arm32/arm64, no DMG/exe artifacts, no GHCR/DockerHub.
|
||||
|
||||
## NuGet audit
|
||||
|
||||
.NET 10 runs NuGet audit on restore. Several projects set `TreatWarningsAsErrors=true`,
|
||||
so vulnerable transitive packages failed the build. `Directory.Build.props` demotes
|
||||
low/moderate/high advisories (NU1901-1903) to warnings and promotes NU1904 (critical)
|
||||
to an error in **every** project via `WarningsAsErrors`. Underlying vulnerable deps are
|
||||
tracked in ersatztv#8.
|
||||
|
||||
## Registry
|
||||
|
||||
Gitea Packages, HTTP-only at `192.168.1.95:3000`. jazz's Docker daemon has it as an
|
||||
insecure-registry (server-management#172). Images: `192.168.1.95:3000/timothy/ersatztv:<tag>`.
|
||||
|
||||
## Test / prod environments
|
||||
|
||||
Container/compose wiring lives in **server-management** (project boundary): test
|
||||
`ersatztv-test` on 8410 (`:latest`), prod `ersatztv` on 8409 (`:prod`). See
|
||||
server-management#481 for the full spec (registry pull on jazz, volumes, Jellyfin
|
||||
isolation for test, Watchtower/manual promotion).
|
||||
|
||||
## Retired upstream workflows
|
||||
|
||||
The upstream `.github/workflows/` (`ci.yml`, `docker.yml`, `artifacts.yml`,
|
||||
`release.yml`, `pr.yml`, `issue-stale.yml`) were removed — they targeted
|
||||
GHCR/DockerHub + Azure/Apple signing and called reusable workflows at dead
|
||||
`ersatztv/ersatztv@main` paths, and ran as noise (incl. a daily stale-issue cron) on
|
||||
the Gitea runner. Upstream is archived, so there are no future merges to preserve them
|
||||
for. The dead `.github/dependabot.yml` and `FUNDING.yml` (upstream-pointed) were also
|
||||
removed.
|
||||
|
||||
## Known follow-ups
|
||||
|
||||
- Pin third-party actions to commit SHAs (currently floating major tags cloned from
|
||||
github.com at runtime) — low priority for a homelab; tracked informally.
|
||||
|
||||
Reference in New Issue
Block a user