fix(620): signal corpus size per RECORD; the aggregate becomes an unthresholded trend
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 13s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 20s
PR Gates / Docs update reminder (pull_request) Successful in 28s
PR Gates / decisions lifecycle (pull_request) Successful in 30s
Review verdict / Set review-verdict status (pull_request) Successful in 12s
PR Gates / Script tests (pytest) (pull_request) Successful in 42s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m31s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 18m33s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 21m27s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 22m24s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
review-verdict/h10 Review-verdict: MERGEABLE @ fefd11d
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 13s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 20s
PR Gates / Docs update reminder (pull_request) Successful in 28s
PR Gates / decisions lifecycle (pull_request) Successful in 30s
Review verdict / Set review-verdict status (pull_request) Successful in 12s
PR Gates / Script tests (pytest) (pull_request) Successful in 42s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m31s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 18m33s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 21m27s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 22m24s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
review-verdict/h10 Review-verdict: MERGEABLE @ fefd11d
Squashed from 7 commits (4 review rounds) to keep the rebase onto #621 tractable; the full round-by-round history is on PR #642. corpus was 5658/5600 — over budget and warning again — 3h35m after #619 put it at 5228, with nobody consolidating anything. So this does NOT re-baseline. An aggregate over a monotonically growing corpus can only ratchet; that is the "permanently red, therefore no signal" state #542 re-baselined away from, and growth is not even a smooth rate to plan against (the corpus FELL from 5089 to 5042 across four days, then gained 427 in one evening as two large records landed). Replaces it with a per-record prose ceiling (default 60), non-blocking, naming each record over it — not monotonic, so it can go red AND green, and it points at a file. The aggregate is still printed as an unthresholded trend notice, with record prose and non-record scaffolding reported separately because they are not the same unit. The GENERATED catalog is no longer counted at all: it gains one row per record and no consolidation can shrink it, which made the metric partly a record COUNT in a line-count costume. The calibration test took FOUR versions, and the failures are the durable lesson: v1 true by construction (`max(under) <= 60 < min(over)` over lists built by that test) v2 a gap WIDTH — a ceiling of 200 also sits in a wide gap, so it passed v3 fraction band + "clear air" vs the nearest record above — hostage to an unrelated record: one ordinary 62-line addition reddened it with the ceiling correctly placed, and the only remedy was to RAISE the ceiling. That is this very treadmill, as a hard failure in what #631 makes a blocking job. v4 `p90 <= ceiling <= p95` — the property stated directly and scale-free. Two rules recorded: a guard test must depend only on the thing it guards, and a threshold over a growing population must be expressed in that population's own terms. Candidates: all over-ceiling records assessed, each actioned or declined with a reason. The largest (scan.libraryfolder-unique-identity, 230 lines) is a legitimate DECLINE — a dozen-odd distinct traps whose only copy that is. Nothing pruned, so no archive or supersession was required. An automated redundancy metric is explicitly rejected. Also: `--budget` is accepted but announces its retirement rather than no-opping silently; the dead `budget_ok` parameter is gone; and five "untresholded" typos are fixed, one of which was propagating into the generated catalog row and MemPalace's per-key drawer. Refs #620 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
+19
-7
@@ -51,13 +51,25 @@ step is now continuous. The release boundary is instead where you:
|
||||
`docs/decisions/archive/` (the validator fails this, but eyeball it at the boundary too).
|
||||
3. Regenerate the active catalog: `PYTHONPATH=. python3 scripts/build_decisions_catalog.py` and
|
||||
commit any drift.
|
||||
4. Check the aggregate active-corpus budget (`decisions_validate.py --budget <n>`, default **4800**
|
||||
lines across `docs/decisions.md` + topic files + the catalog — replaces the old single-file
|
||||
1800-line floor). **Re-baselined 2026-07-21 (#520)**: the corpus is now fully migrated at
|
||||
~4366 lines; 4800 gives headroom so the warning fires on real future growth, not on the expected
|
||||
post-migration size. Going over budget is a **non-blocking warning** (`::warning::` to stderr,
|
||||
not a validator error) — a ratchet/reminder to extract a new topic file or archive more history,
|
||||
not a release gate.
|
||||
4. Read the corpus size signals. Since **ersatztv#620** these are two separate things:
|
||||
- a **per-record prose ceiling** (`decisions_validate.py --record-ceiling <n>`, default **60**)
|
||||
— a **non-blocking `::warning::`** naming every record over it. This is the actionable signal:
|
||||
it points at a file. The 60 is derived from the distribution, not picked as a round number.
|
||||
A test pins what that derivation MEANS rather than any particular numbers: the ceiling must sit
|
||||
between the **90th and 95th percentile** of record lengths, i.e. at the tail boundary. Stated
|
||||
as percentiles it is scale-free, so ordinary corpus growth cannot ratchet it — it fires only
|
||||
when the ceiling genuinely stops marking the tail and should be re-derived.
|
||||
- the **aggregate prose total**, printed every run as an unthresholded `::notice::` **trend**.
|
||||
It has no pass/fail. A total over a monotonically growing corpus can only ratchet: the old
|
||||
4800→5600 budget went quiet at 5228 after #610 changed the metric and was back over at 5658
|
||||
**three and a half hours later the same evening**, with nobody consolidating anything — the
|
||||
"permanently red = no signal" failure, not in slow motion at all. It reports record prose and
|
||||
non-record scaffolding separately, because they are not the same unit. The generated catalog is no longer counted at all — it
|
||||
gains one row per record and cannot be consolidated away.
|
||||
|
||||
**Being listed by the ceiling is an invitation to check for redundancy, not an instruction to
|
||||
cut.** A long record that is entirely distinct findings is a legitimate decline — say so in the
|
||||
record and move on. (`--budget` is still accepted and ignored, so old invocations keep working.)
|
||||
5. Report the remaining `legacy-unmigrated` count (the validator prints it as a `::notice::`) so the
|
||||
backlog is visible, even though it isn't required to hit zero before a release.
|
||||
A genuine rationale-prose rewrite still needs a `Decisions-Edit: yes` git trailer on a **non-merge**
|
||||
|
||||
+5
-3
@@ -50,9 +50,11 @@ it after any status change; CI's `decisions lifecycle` job fails on drift (`--ch
|
||||
|
||||
**Enforcement**: `scripts/decisions_validate.py` checks metadata well-formedness (including a required
|
||||
`Signals:` line), one-active-record-per-key, reciprocal links, that no record vanishes from the active
|
||||
set without an archive copy, that the active catalog is in sync, and an aggregate active-corpus line
|
||||
budget (replaces the old 1800-line
|
||||
floor on this single file). The Husky `pre-commit` hook runs the structural checks over the working
|
||||
set without an archive copy, and that the active catalog is in sync. Separately, and as a
|
||||
NON-BLOCKING warning rather than an error, it reports a per-record prose ceiling (ersatztv#620).
|
||||
The aggregate active-corpus line budget that used to sit in this list — itself the replacement for
|
||||
an older 1800-line floor on this single file — is RETIRED: the total is now printed as an
|
||||
unthresholded trend notice only. The Husky `pre-commit` hook runs the structural checks over the working
|
||||
tree; the CI `decisions lifecycle` job additionally runs the body-diff check with `--base`/`--head`.
|
||||
|
||||
**`Decisions-Edit: yes`** (a git **trailer** — the message's final paragraph, alongside
|
||||
|
||||
@@ -64,6 +64,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
|
||||
| `concurrency.replace-all-contract` | Replace-all aggregate PUTs carry a uniform plain `int Version` concurrency token (EF `.IsConcurrencyToken()`), checked pre-save and enforced by the EF UPDATE guard, returning 412 (not 409) on a stale `If-Match`. | 2026-07-11 | [link](records/concurrency/replace-all-contract.md) |
|
||||
| `concurrency.schedule-item-child-identity` | `PUT /api/schedules/{id}/items` reconciles by an optional round-tripped child `Id` (null/absent/0 ⇒ new item), never by array position, so fill-group/shuffle state follows the logical item across reorders; an unknown or duplicate id is rejected 422 (checked after the §7a `CheckVersion`, so 412 precedes 422). | 2026-07-11 | [link](records/concurrency/schedule-item-child-identity.md) |
|
||||
| `docs.convention-docs-session-start` | Docs-first, not source-first: conventions (api-conventions, spa-conventions, e2e-local, blazor-route-parity, domain-model, decisions, README) are read from docs, not reverse-engineered from code, via `docs/README.md`'s task-signal map — only the sections it points to for the task at hand, not the whole set. Each doc is updated in the same PR that changes what it documents, replacing deferred/follow-up doc updates. | 2026-07-07 | [link](records/docs/convention-docs-session-start.md) |
|
||||
| `docs.corpus-size-signal` | The corpus's size signal is a per-record prose ceiling (`decisions_validate.py --record-ceiling`, default 60, chosen at a natural gap in the distribution), reported as a NON-BLOCKING `::warning::` naming each record over it. The aggregate prose total is still printed every run but carries NO threshold — it is a `::notice::` trend only — because a total over a monotonically growing corpus can only ratchet, and the generated catalog (`docs/decisions/README.md`) is no longer counted at all since it gains one row per record and cannot be consolidated away. Being listed by the ceiling is an invitation to check for REDUNDANCY, never an instruction to cut: a long record that is all distinct findings is a legitimate decline, and should be recorded as one. | 2026-07-26 | [link](records/docs/corpus-size-signal.md) |
|
||||
| `docs.decision-lifecycle` | every decision `##` record (active or archived) carries a 5-field metadata block (`key`, `status`, `since`, `supersedes`, `superseded-by`) checked by `scripts/decisions_validate.py`; a record is never deleted or line-edited to reverse a call — it is moved to `docs/decisions/archive/` with `status: superseded`/`retired` and a reciprocal `superseded-by`/`supersedes` key pair to its replacement. | 2026-07-21 | [link](records/docs/decision-lifecycle.md) |
|
||||
| `docs.decision-one-file-per-record` | Each decision record is its own file at `docs/decisions/records/<area>/<topic>.md` (archived ones at `docs/decisions/archive/<area>/<topic>.md`) with YAML frontmatter; the filename IS the key, so one-active-record-per-key is a filesystem property rather than a validator check, and supersession is a `git mv`. | 2026-07-25 | [link](records/docs/decision-one-file-per-record.md) |
|
||||
| `docs.decision-optional-provenance` | Decision records gain two OPTIONAL fields — `stale-after: YYYY-MM-DD` on the metadata line and a `**Sources:**` line in the metadata block; the Open Knowledge Format (OKF) itself is NOT adopted as the record format. | 2026-07-25 | [link](records/docs/decision-optional-provenance.md) |
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
---
|
||||
key: docs.corpus-size-signal
|
||||
title: '2026-07-26 — The decision corpus signals size per RECORD, not as an aggregate budget; the aggregate becomes an unthresholded trend (#620)'
|
||||
status: active
|
||||
since: '2026-07-26'
|
||||
supersedes: none
|
||||
superseded-by: none
|
||||
rule: 'The corpus''s size signal is a per-record prose ceiling (`decisions_validate.py --record-ceiling`, default 60, chosen at a natural gap in the distribution), reported as a NON-BLOCKING `::warning::` naming each record over it. The aggregate prose total is still printed every run but carries NO threshold — it is a `::notice::` trend only — because a total over a monotonically growing corpus can only ratchet, and the generated catalog (`docs/decisions/README.md`) is no longer counted at all since it gains one row per record and cannot be consolidated away. Being listed by the ceiling is an invitation to check for REDUNDANCY, never an instruction to cut: a long record that is all distinct findings is a legitimate decline, and should be recorded as one.'
|
||||
signals: 'aggregate active-corpus budget, schedule a consolidation warning, permanently red ratchet, per-record ceiling, 60-line prose ceiling, generated catalog counted in budget, corpus consolidation has no owner, size is not redundancy · paths: `scripts/decisions_validate.py`, `scripts/tests/test_decisions_validate.py`, `docs/ci-cd.md` · issues: #620, #610, #603, #542, #520'
|
||||
mechanics: '`scripts/decisions_validate.py` -> `oversized_records` / `_budget_total`; `docs/ci-cd.md` -> "`decisions-guard` job"'
|
||||
---
|
||||
|
||||
#620 named a real risk: after #610 changed the budget to count prose rather than whole files, the
|
||||
warning went quiet (5228/5600) **without the underlying condition changing** — a check that stops
|
||||
complaining because it started measuring something else reads, to a future session, exactly like a
|
||||
problem that got fixed.
|
||||
|
||||
**The premise had already expired when the issue was picked up.** Measured on `main` at `f394d6ce`
|
||||
the corpus was **5658/5600 — over budget and warning again**. #619 put it at 5228 at `8e84191a`
|
||||
(2026-07-25T18:36Z); `f394d6ce` is 2026-07-25T22:11Z — **3 hours 35 minutes later, the same
|
||||
evening.** That is the finding, not a footnote: the aggregate went quiet and loud again inside one
|
||||
evening, with nobody consolidating anything. (An earlier draft of this record said "four days"; the
|
||||
measured interval is stronger than the claim it replaced.)
|
||||
|
||||
**So the aggregate is the wrong instrument, and re-baselining it would restart the treadmill.** It
|
||||
measures a monotonically growing quantity, so any fixed total is a ratchet that must periodically be
|
||||
raised — the "permanently red, therefore no signal" state #542 re-baselined away from. The growth is
|
||||
not a smooth rate to plan a threshold around: measured with one consistent metric the corpus sat at
|
||||
5089 on 07-21, **5042** on 07-25T18:36 (it went DOWN across four days), then 5469 three and a half
|
||||
hours later. Effectively all of it was two large records landing in one evening (#491's 230-line
|
||||
record, #622's 147-line one). A threshold cannot be set against that: two ordinary records can cross
|
||||
any headroom in an evening, while quiet weeks drift downward. #620 asked for "a threshold that can actually go red again"; the honest answer is that
|
||||
this one can never go durably *green*, which is the same defect wearing the opposite sign.
|
||||
|
||||
Two further defects in what it counted: the **generated catalog** (189 of the 5658 lines) was added
|
||||
on top, though it gains one row per record and no consolidation can shrink it — making the metric
|
||||
partly a record COUNT in a line-count costume, and pointing at work that cannot be done. And an
|
||||
aggregate **names no file**: "58 lines over" is not actionable.
|
||||
|
||||
**The ceiling is calibrated, not round.** On `f394d6ce` the records ran 2..59 prose lines (median
|
||||
26, p90 52) and then jumped to **83**, with nothing between; 60 sits in that gap.
|
||||
|
||||
**A gap is a live property, not a fact you write down once.** Adding any record can narrow it —
|
||||
an early draft of this very record landed inside the 59..83 gap it claimed was empty, and the test
|
||||
meant to prevent exactly that could not see it: `max(under) <= 60 < min(over)` is true **by
|
||||
construction** of the two lists it builds, and passes on a distribution with no gap at all. A
|
||||
rationale-guarding test that cannot fail is worse than none, because it launders the claim.
|
||||
|
||||
It is replaced by `test_real_corpus_ceiling_sits_at_the_TAIL_BOUNDARY_of_the_distribution`, which
|
||||
states the property directly and scale-free: **the ceiling sits between the 90th and 95th percentile
|
||||
of record lengths** — that is what "marks the start of the tail" means — and reads the value from
|
||||
`RECORD_CEILING_DEFAULT` so test and CLI cannot drift.
|
||||
|
||||
Getting there took four versions, and the failures are the useful part:
|
||||
|
||||
| | assertion | why it failed |
|
||||
|---|---|---|
|
||||
| v1 | `max(under) <= 60 < min(over)` | true **by construction** of those two lists |
|
||||
| v2 | a minimum gap WIDTH | a ceiling of 200 also sits in a wide gap — it passed |
|
||||
| v3 | 2-12% fraction band + "clear air" above | **hostage to an unrelated record**: one ordinary 62-line addition reddened it with the ceiling correctly placed, and the only remedy was to RAISE the ceiling — this very treadmill, as a hard failure in what #631 makes a blocking job. The fraction band had the same coupling more slowly (12 more long records breached it), and `0 <= headroom` was vacuous. |
|
||||
| v4 | `p90 <= ceiling <= p95` | percentiles move WITH the corpus, so routine growth cannot ratchet it; it fires only when the ceiling genuinely stops marking the tail |
|
||||
|
||||
Two rules came out of that sequence, and they outlive this metric:
|
||||
**a guard test must depend only on the thing it guards**, and
|
||||
**a threshold over a growing population must be expressed in the population's own terms** — a
|
||||
percentile, not a count and not a raw distance. **No line count in this record
|
||||
is restated as a self-referential fact** for the same reason: the warning reports the current
|
||||
numbers on every run, and a number frozen in prose is one edit away from being a lie.
|
||||
|
||||
**Size is a proxy for the thing we actually care about, and the proxy is demonstrably wrong.** Of
|
||||
the records over the ceiling, the largest by ~1.6x —
|
||||
`scan.libraryfolder-unique-identity`, 230 lines — is a dozen-odd **distinct** hard-won traps (MySQL
|
||||
`utf8mb4_bin` PAD SPACE, create-the-composite-index-before-dropping-its-predecessor, clearing the
|
||||
connection pool per MySQL fixture, lazy hash healing that must never abort a scan…). Shortening it
|
||||
would delete the only copy of most of them, which is the exact trap #620 itself flags via
|
||||
`docs.tracker-comment-retrofit`. That is why the warning's wording invites a redundancy check and
|
||||
explicitly blesses "this is all distinct, moving on" as an outcome.
|
||||
|
||||
**What this deliberately stops signalling.** Pure record-COUNT growth is now unmeasured: 500 records
|
||||
of 20 lines each trips nothing, and the term removed from the total — the generated catalog — was
|
||||
precisely the one that tracked count. That is accepted, not overlooked. Retrieval is by key and
|
||||
semantic search, never front-to-back reading, so total size is not what degrades; and shrinking the
|
||||
corpus is the archive lifecycle's job (a positive supersede + `git mv`), not a warning's. The trend
|
||||
notice still prints the record count every run, so the number stays visible without pretending to
|
||||
gate anything.
|
||||
|
||||
**What is NOT adopted.** An automated redundancy metric (similarity between sibling records) was
|
||||
considered and rejected: it would be noisy, unfalsifiable, and would fire hardest on exactly the
|
||||
adjacent-but-distinct records the corpus is supposed to keep apart. Genuine redundancy is already
|
||||
handled by the lifecycle — supersede and `git mv` to `archive/` — which is a positive act with a
|
||||
reciprocal link, not a size heuristic. Making either signal **blocking** is also still rejected
|
||||
(#520): a size condition turning an unrelated PR red is a self-inflicted CI outage.
|
||||
|
||||
**Consolidation candidates at #620** — identified, and each either actioned or declined here rather
|
||||
than deferred to yet another issue that does not exist. **This table is a point-in-time assessment,
|
||||
measured at `f394d6ce`; the over-ceiling SET moves as records are added** (`release.review-verdict-gate`
|
||||
joined it during this PR's own review cycle). The live list is whatever the warning prints — the
|
||||
table records the judgement, not the membership:
|
||||
|
||||
| Record | Prose | Verdict |
|
||||
|---|---|---|
|
||||
| `scan.libraryfolder-unique-identity` | 230 | **Decline** — 13 distinct traps, no restatement |
|
||||
| `release.verdict-status-check` | 147 | **Decline** — 4 claims + the #622 server-side enforcement chain |
|
||||
| `ffmpeg.remote-image-fetcher-bounded` | 131 | **Decline** — 9 distinct bounds (deadline, size cap, decoder frames) |
|
||||
| `ci.runner-placement` | 112 | **Decline** — 13 distinct claims, densest record in the corpus |
|
||||
| `scan.projection-failure-sweep-guard` | 112 | **Keep listed** — prose-heavy, overlaps `scan.zero-item-fetch-guard`; genuine candidate |
|
||||
| `sched.weighted-shuffle` | 98 | **Keep listed** — prose-heavy, overlaps `sched.weightedshuffle-editor`; genuine candidate |
|
||||
| `security.session-auth-dual-credential` | 96 | **Decline** — 9 distinct claims |
|
||||
| `docs.corpus-size-signal` (this record) | — | **Decline** — over its own ceiling; the candidate table and the review-correction history are the bulk, and both are the deliverable |
|
||||
| `media.source-mgmt-write-api` | 83 | **Decline** — 4 claims, per-family identity contracts are the point |
|
||||
|
||||
The two "keep listed" rows are left to the standing warning deliberately: they are the only pair with
|
||||
a plausible sibling to merge into, and merging records changes `key` identity (invalidating
|
||||
MemPalace's per-key drawers) — a call for the owning area, not a drive-by edit inside a metric
|
||||
change. The warning names them every run, which tracks them better than an issue nobody reads.
|
||||
Taxonomy normalisation stays out of scope for the same identity reason, as #620 states.
|
||||
|
||||
**This record is itself listed by its own ceiling, and that is the intended behaviour rather than an
|
||||
embarrassment.** What it contains is the per-candidate table the issue asked for plus the
|
||||
review-correction history; cutting either to hit the number would be exactly the target-chasing the
|
||||
rule above forbids — so, per that rule: checked for redundancy, none found, declining. A ceiling
|
||||
nobody is ever over is a ceiling that is not measuring anything. (Two successive drafts of this
|
||||
paragraph stated a specific self-count, and both were wrong within one commit — which is why the
|
||||
number is now simply absent. The warning prints it on every run.)
|
||||
@@ -48,6 +48,10 @@ _LEGACY_EDIT_TOKEN = "[decisions-edit]" # noqa: S105 (a commit-message marker,
|
||||
# sorting by date because this regex forces a fixed-width zero-padded form.
|
||||
_STALE_AFTER_RE = re.compile(r"^\d{4}-\d{2}-\d{2}$")
|
||||
|
||||
# Per-record prose ceiling (#620). A module constant, not a bare argparse default, so the test that
|
||||
# guards the calibration claim asserts against the SAME value the CLI uses and the two cannot drift.
|
||||
RECORD_CEILING_DEFAULT = 60
|
||||
|
||||
|
||||
def _parse_stale_after(value: str | None) -> date | None:
|
||||
"""`stale-after` as a date, or None if absent, empty, or malformed.
|
||||
@@ -91,7 +95,6 @@ def validate(
|
||||
*,
|
||||
archive_keys,
|
||||
catalog_ok,
|
||||
budget_ok,
|
||||
removed,
|
||||
rewritten,
|
||||
archive_records=None,
|
||||
@@ -199,9 +202,6 @@ def validate(
|
||||
|
||||
if not catalog_ok:
|
||||
errs.append("docs/decisions/README.md active catalog is stale — run build_decisions_catalog.py")
|
||||
if not budget_ok:
|
||||
# non-blocking: reported as a warning by the caller (main()), never added here.
|
||||
pass
|
||||
return errs
|
||||
|
||||
|
||||
@@ -573,8 +573,9 @@ def _archive_keys() -> set[str]:
|
||||
def _budget_total() -> int:
|
||||
"""Lines of PROSE in the active corpus — YAML frontmatter excluded.
|
||||
|
||||
The budget exists to bound how much narrative a reader/agent must get through, so it counts
|
||||
prose, not metadata. Under the #610 split each record carries ~11 frontmatter lines plus two
|
||||
Since #620 this is a TREND figure with no threshold attached; the name is historical. It counts
|
||||
prose rather than metadata because the question it answers is how much narrative a reader/agent
|
||||
must get through. Under the #610 split each record carries ~11 frontmatter lines plus two
|
||||
fences (1789 lines across 166 records), which are the structured restatement of what used to be
|
||||
one dense backtick line — counting them would inflate the metric without any new knowledge
|
||||
being added.
|
||||
@@ -584,6 +585,14 @@ def _budget_total() -> int:
|
||||
over budget; prose-only puts the same content comfortably under. The consolidation work is still
|
||||
worth doing — it is simply no longer being signalled by a warning that was partly measuring
|
||||
punctuation. Pre-migration this function is inert: no legacy file has frontmatter.
|
||||
|
||||
Since #620 this is reported as an informational TREND only — it no longer carries a threshold.
|
||||
The GENERATED catalog (`docs/decisions/README.md`) is also no longer counted: it used to be
|
||||
added on top, which quietly made this partly a record-COUNT metric wearing a line-count
|
||||
costume, since the catalog gains exactly one row per active record and no amount of
|
||||
consolidating prose can shrink it (189 of the 5658 lines it last reported were that file).
|
||||
Counting un-prunable generated output in a number whose stated remedy was "schedule a
|
||||
consolidation" pointed the reader at work that cannot be done.
|
||||
"""
|
||||
total = 0
|
||||
for f in dl.active_files():
|
||||
@@ -596,14 +605,41 @@ def _budget_total() -> int:
|
||||
if end is not None:
|
||||
lines = lines[end + 1 :]
|
||||
total += len(lines)
|
||||
cat = dl.TOPIC_DIR / "README.md"
|
||||
if cat.exists():
|
||||
total += len(cat.read_text(encoding="utf-8").splitlines())
|
||||
return total
|
||||
|
||||
|
||||
def _budget_ok(limit: int) -> bool:
|
||||
return _budget_total() <= limit
|
||||
def record_prose_lines(rec) -> int:
|
||||
"""Prose lines in one record's body (frontmatter already stripped by the parser)."""
|
||||
return len((rec.body or "").splitlines())
|
||||
|
||||
|
||||
def oversized_records(records, ceiling: int) -> list[tuple[str, int]]:
|
||||
"""Active records whose prose exceeds `ceiling`, longest first (#620).
|
||||
|
||||
This REPLACES the aggregate line budget as the corpus's actionable size signal. The aggregate
|
||||
measured a monotonically growing quantity: a healthy project's decision corpus only gets
|
||||
bigger, so any fixed total is a ratchet that must periodically be raised — which is exactly the
|
||||
"permanently red, therefore no signal at all" state #542 re-baselined away from and #620 was
|
||||
filed about. Re-baselining it again would only restart that treadmill: the growth is not a smooth
|
||||
rate to set a threshold against — the corpus FELL from 5089 to 5042 across four days, then gained
|
||||
427 in a single evening as two large records landed.
|
||||
|
||||
A per-record ceiling is NOT monotonic. It measures the shape of individual records rather than
|
||||
the size of the corpus, so it can go red and green again, and it names a file the reader can
|
||||
act on instead of asserting that "the corpus" is too big.
|
||||
|
||||
The ceiling sits at a natural gap in the real distribution rather than a round number: at #620
|
||||
the records run 0..59 prose lines (median 26, p90 52) and then jump straight to 83, with
|
||||
nothing in between. 60 separates the bulk from the tail without splitting a cluster.
|
||||
|
||||
IMPORTANT — a prompt for judgement, not a target. Length is a PROXY for "grown past what a
|
||||
reader can absorb", and the proxy is demonstrably wrong sometimes: the largest record in the
|
||||
corpus (`scan.libraryfolder-unique-identity`, 230 lines) is thirteen distinct hard-won traps,
|
||||
and shortening it would delete the only copy of most of them. Appearing in this list is an
|
||||
invitation to check for REDUNDANCY, never an instruction to cut.
|
||||
"""
|
||||
out = [(r.key, record_prose_lines(r)) for r in records if r.key]
|
||||
return sorted([kv for kv in out if kv[1] > ceiling], key=lambda kv: -kv[1])
|
||||
|
||||
|
||||
def _catalog_ok() -> bool:
|
||||
@@ -621,26 +657,37 @@ def main(argv=None) -> int:
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--base")
|
||||
ap.add_argument("--head")
|
||||
# Aggregate active-corpus budget. Re-baselined 2026-07-21 (#542): 4800 -> 5600. The corpus grew
|
||||
# by ~530 lines because 31 workflow/CI/review rules were MOVED into it from
|
||||
# docs/handoffs/chicorytv-issue-queue.md, which was their only copy. That is the corpus doing its
|
||||
# job, not drift — the knowledge existed already, it just wasn't retrievable. Do not raise this
|
||||
# again to accommodate genuinely new records; that is what the warning is for.
|
||||
ap.add_argument("--budget", type=int, default=5600)
|
||||
# Per-record prose ceiling — the corpus's size signal since #620, replacing the aggregate
|
||||
# budget. See `oversized_records` for why a total was the wrong instrument (it ratchets on a
|
||||
# monotonically growing quantity) and why 60 (a natural gap in the distribution, not a round
|
||||
# number). `--budget` is still ACCEPTED so an existing caller keeps working, but it no longer
|
||||
# does anything — and it SAYS SO when passed (below) rather than no-opping quietly. A flag that
|
||||
# takes a threshold and silently enforces nothing is the same "reports success while doing
|
||||
# nothing" defect as #603's `stale-after` and #609's marker; retiring one signal must not
|
||||
# introduce another.
|
||||
ap.add_argument("--record-ceiling", type=int, default=RECORD_CEILING_DEFAULT)
|
||||
ap.add_argument("--budget", type=int, default=None, help=argparse.SUPPRESS)
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
if args.budget is not None:
|
||||
print(
|
||||
f"::warning::decisions-validate: --budget {args.budget} is RETIRED and was IGNORED "
|
||||
f"(#620) — the aggregate is now an unthresholded trend. Use --record-ceiling for the "
|
||||
f"enforceable per-record signal.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
|
||||
records = dl.all_active_records()
|
||||
archive_records = []
|
||||
if dl.ARCHIVE_DIR.exists():
|
||||
for f in dl.ARCHIVE_DIR.rglob("*.md"): # rglob: archive is nested by area after #610
|
||||
archive_records += dl.parse_file(f)
|
||||
removed, rewritten, demoted = _diff_findings(args.base, args.head) if args.base and args.head else ([], [], [])
|
||||
budget_ok = _budget_ok(args.budget)
|
||||
oversized = oversized_records(records, args.record_ceiling)
|
||||
errs = validate(
|
||||
records,
|
||||
archive_keys=_archive_keys(),
|
||||
catalog_ok=_catalog_ok(),
|
||||
budget_ok=budget_ok,
|
||||
removed=removed,
|
||||
rewritten=rewritten,
|
||||
archive_records=archive_records,
|
||||
@@ -648,10 +695,31 @@ def main(argv=None) -> int:
|
||||
wing_faults=record_wing_faults(),
|
||||
)
|
||||
|
||||
if not budget_ok:
|
||||
# Aggregate: an unthresholded TREND, not a gate (#620). Printed every run so the number stays
|
||||
# visible, with no pass/fail attached — a total over a monotonically growing corpus can only
|
||||
# ratchet, and a permanently-tripped warning is indistinguishable from no warning at all.
|
||||
keyed = [r for r in records if r.key]
|
||||
record_prose = sum(record_prose_lines(r) for r in keyed)
|
||||
total_prose = _budget_total()
|
||||
print(
|
||||
f"::notice::decisions-validate: {record_prose} prose lines across {len(keyed)} records "
|
||||
f"(mean {record_prose // max(len(keyed), 1)}), plus {total_prose - record_prose} lines of "
|
||||
f"non-record scaffolding = {total_prose} total. Trend only — no threshold; the per-record "
|
||||
f"ceiling below is the actionable signal.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
|
||||
# Per-record ceiling: NON-BLOCKING by design (#520 — a size condition must never turn an
|
||||
# unrelated PR red). Names the files, so the reader can act instead of being told the corpus is
|
||||
# "too big". Being listed is an invitation to check for redundancy, NOT an instruction to cut:
|
||||
# the longest record in the corpus is 13 distinct traps and is a legitimate decline (#620).
|
||||
if oversized:
|
||||
listed = "; ".join(f"{k} ({n} lines)" for k, n in oversized)
|
||||
print(
|
||||
f"::warning::decisions-validate: aggregate active-corpus is {_budget_total()} lines "
|
||||
f"(budget {args.budget}) — schedule a consolidation",
|
||||
f"::warning::decisions-validate: {len(oversized)} record(s) exceed the "
|
||||
f"{args.record_ceiling}-line prose ceiling — check each for redundancy against its "
|
||||
f"siblings (a long record that is all distinct findings is fine, say so and move on): "
|
||||
f"{listed}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
|
||||
|
||||
@@ -27,7 +27,6 @@ def _v(recs: list[dl.Record], **kw: Any) -> list[str]:
|
||||
args: dict[str, Any] = dict(
|
||||
archive_keys=set(),
|
||||
catalog_ok=True,
|
||||
budget_ok=True,
|
||||
removed=[],
|
||||
rewritten=[],
|
||||
archive_records=[],
|
||||
@@ -938,3 +937,144 @@ def test_an_empty_ARCHIVE_wing_is_legitimate(tmp_path):
|
||||
records, archive = _wing(tmp_path)
|
||||
(records / "ci" / "good.md").write_text(_GOOD)
|
||||
assert dv.record_wing_faults(records, archive) == [], "an empty archive must not fault"
|
||||
|
||||
|
||||
# --- #620: per-record ceiling replaces the aggregate budget --------------------------------------
|
||||
|
||||
|
||||
_MINIMAL_RECORD = (
|
||||
"---\n"
|
||||
"key: ci.a\n"
|
||||
"title: '2026-01-01 — A (#1)'\n"
|
||||
"status: active\n"
|
||||
"since: '2026-01-01'\n"
|
||||
"supersedes: none\n"
|
||||
"superseded-by: none\n"
|
||||
"rule: 'r'\n"
|
||||
"signals: 's'\n"
|
||||
"---\n\nprose.\n"
|
||||
)
|
||||
|
||||
|
||||
def _rec_body(key: str, lines: int) -> dl.Record:
|
||||
return _rec(key=key, heading=key, body="\n".join(f"prose line {i}" for i in range(lines)))
|
||||
|
||||
|
||||
def test_oversized_records_flags_only_those_over_the_ceiling():
|
||||
recs = [_rec_body("a.short", 10), _rec_body("b.exact", 60), _rec_body("c.long", 61)]
|
||||
assert dv.oversized_records(recs, 60) == [("c.long", 61)], "the ceiling must be exclusive"
|
||||
|
||||
|
||||
def test_oversized_records_sorts_longest_first():
|
||||
recs = [_rec_body("a.mid", 80), _rec_body("b.big", 200), _rec_body("c.small", 70)]
|
||||
assert [k for k, _ in dv.oversized_records(recs, 60)] == ["b.big", "a.mid", "c.small"]
|
||||
|
||||
|
||||
def test_oversized_records_can_go_green():
|
||||
"""The property the aggregate budget structurally lacked: a corpus that satisfies it.
|
||||
|
||||
An aggregate over a monotonically growing corpus can only ratchet — it goes red and stays red
|
||||
until someone raises the number. A per-record ceiling is satisfiable and re-satisfiable, which
|
||||
is what makes a red mean something."""
|
||||
recs = [_rec_body("a.short", 10), _rec_body("b.short", 20)]
|
||||
assert dv.oversized_records(recs, 60) == []
|
||||
|
||||
|
||||
def test_budget_total_excludes_the_generated_catalog(tmp_path, monkeypatch):
|
||||
"""#620: the catalog gains one row per record and cannot be consolidated away.
|
||||
|
||||
Counting it made the 'schedule a consolidation' number partly a record COUNT, pointing the
|
||||
reader at work that cannot be done. Mutation-sensitive by construction: the catalog here is
|
||||
large and distinctively sized, so re-adding it would change the total by exactly 500."""
|
||||
topic = tmp_path / "docs" / "decisions"
|
||||
records = topic / "records" / "ci"
|
||||
records.mkdir(parents=True)
|
||||
(records / "a.md").write_text(_MINIMAL_RECORD)
|
||||
(topic / "README.md").write_text("\n".join(f"| row {i} |" for i in range(500)))
|
||||
|
||||
monkeypatch.setattr(dv.dl, "REPO_ROOT", tmp_path)
|
||||
monkeypatch.setattr(dv.dl, "DECISIONS_MD", tmp_path / "docs" / "decisions.md")
|
||||
monkeypatch.setattr(dv.dl, "TOPIC_DIR", topic)
|
||||
monkeypatch.setattr(dv.dl, "ARCHIVE_DIR", topic / "archive")
|
||||
monkeypatch.setattr(dv.dl, "RECORDS_DIR", records)
|
||||
|
||||
total = dv._budget_total()
|
||||
assert total < 100, f"the 500-line generated catalog leaked into the total ({total})"
|
||||
|
||||
|
||||
def test_real_corpus_ceiling_sits_at_the_TAIL_BOUNDARY_of_the_distribution():
|
||||
"""Guards the calibration claim. This is the FOURTH version; the failures are the lesson.
|
||||
|
||||
v1 `max(under) <= 60 < min(over)` — true by construction of those two lists.
|
||||
v2 a minimum gap WIDTH — but a ceiling of 200 also sits in a wide gap, so it passed.
|
||||
v3 a 2-12% fraction band plus "clear air" measured against `min(over)` — the nearest
|
||||
record ABOVE the ceiling. That made the test a hostage to an unrelated record: one
|
||||
ordinary 62-line addition reddened it with the ceiling correctly placed, and the only
|
||||
remedy the assertion admitted was to RAISE the ceiling. That is the ratchet this whole
|
||||
change abolishes, reinstated as a hard failure in what #631 makes a blocking CI job.
|
||||
The fraction band had the same coupling more slowly (12 more long records breached it),
|
||||
and `0 <= headroom` was vacuous — `max(under)` is by construction <= ceiling.
|
||||
|
||||
v4 states the property directly and scale-free: **the ceiling marks the start of the tail**,
|
||||
i.e. it sits between the 90th and 95th percentile of record lengths. Percentiles move WITH the
|
||||
corpus, so routine growth cannot ratchet this; it fires only when the ceiling genuinely stops
|
||||
marking the tail boundary, which is exactly when it should be re-derived.
|
||||
"""
|
||||
recs = [r for r in dl.all_active_records() if r.key]
|
||||
assert len(recs) > 100, f"corpus looks empty ({len(recs)}) — this check would be vacuous"
|
||||
|
||||
ceiling = dv.RECORD_CEILING_DEFAULT # the value the CLI actually uses; cannot drift from here
|
||||
lengths = sorted(dv.record_prose_lines(r) for r in recs)
|
||||
p90 = lengths[int(len(lengths) * 0.90)]
|
||||
p95 = lengths[int(len(lengths) * 0.95)]
|
||||
|
||||
assert p90 <= ceiling <= p95, (
|
||||
f"the ceiling ({ceiling}) no longer marks the tail boundary: p90={p90}, p95={p95}. "
|
||||
f"Below p90 it cuts into the bulk and every author will learn to ignore it; above p95 it is "
|
||||
f"parked among the outliers and signals nothing. Re-derive it from the distribution."
|
||||
)
|
||||
|
||||
|
||||
def test_retired_budget_flag_says_it_is_ignored(capsys):
|
||||
"""A retired flag must announce itself, not no-op silently.
|
||||
|
||||
Accepting `--budget 100` and enforcing nothing would be the same 'reports success while doing
|
||||
nothing' defect this change exists to retire (#603 stale-after, #609 marker)."""
|
||||
dv.main(["--budget", "100"])
|
||||
err = capsys.readouterr().err
|
||||
assert "--budget 100 is RETIRED and was IGNORED" in err, err
|
||||
|
||||
|
||||
def test_no_budget_flag_means_no_retirement_warning(capsys):
|
||||
dv.main([])
|
||||
assert "RETIRED" not in capsys.readouterr().err
|
||||
|
||||
|
||||
|
||||
def test_main_actually_REPORTS_the_ceiling_and_the_trend(capsys):
|
||||
"""The new signal's live wiring was untested: `if oversized:` -> `if False:`, or bumping the
|
||||
default ceiling to 999999, left every test green while main() reported nothing. Only the pure
|
||||
function `oversized_records()` was covered — so the replacement signal could silently do
|
||||
nothing, which is the exact defect this change exists to retire."""
|
||||
dv.main([])
|
||||
err = capsys.readouterr().err
|
||||
assert "prose lines across" in err, "the aggregate trend notice must always print"
|
||||
assert "exceed the" in err and "prose ceiling" in err, "the per-record ceiling warning must print"
|
||||
over = [r.key for r in dl.all_active_records()
|
||||
if r.key and dv.record_prose_lines(r) > dv.RECORD_CEILING_DEFAULT]
|
||||
assert over, "precondition: the live corpus has at least one over-ceiling record"
|
||||
assert any(k in err for k in over), "the warning must NAME the offending records"
|
||||
|
||||
|
||||
def test_trend_notice_reports_record_prose_and_scaffolding_separately(capsys):
|
||||
"""The notice used to print `_budget_total()` 'across N records', mixing two incompatible
|
||||
definitions of prose: the total includes ~600 lines of headings and keyless scaffolding
|
||||
belonging to no record, so dividing it by the record count and comparing that to the 60-line
|
||||
ceiling compares incommensurable units — a small instance of the 'the metric is not what it
|
||||
says it is' defect this change indicts."""
|
||||
dv.main([])
|
||||
err = capsys.readouterr().err
|
||||
recs = [r for r in dl.all_active_records() if r.key]
|
||||
record_prose = sum(dv.record_prose_lines(r) for r in recs)
|
||||
assert f"{record_prose} prose lines across {len(recs)} records" in err, err
|
||||
assert "non-record scaffolding" in err, err
|
||||
|
||||
Reference in New Issue
Block a user