docs(176): design spec — SmartCollection visual rule builder (sub-project #1)
Scopes #176 down after recon (Autotune #69 + composite create-channel #63 already shipped): the remaining leg is the visual WYSIWYG rule builder. Kodi one-level-nested model, compile-only to a closed Lucene subset (no schema change), backed by a new read-only GET /api/v1/search/fields catalog. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.8
@@ -0,0 +1,119 @@
|
||||
# SmartCollection visual rule builder — design (sub-project #1 of #176)
|
||||
|
||||
**Issue:** [ersatztv#176](http://192.168.1.95:3000/timothy/ersatztv/issues/176) — "Rethink channel/playout/schedule creation: PseudoTV-style channel-first flow with a Kodi-smart-playlist-like query builder."
|
||||
|
||||
**Date:** 2026-07-17 · **Status:** design approved, pre-implementation.
|
||||
|
||||
---
|
||||
|
||||
## 1. Context & why the scope shrank
|
||||
|
||||
#176 as originally filed described three legs of a PseudoTV-style vision. Recon (2026-07-17) shows two of them already shipped since the issue was written:
|
||||
|
||||
- **Autotune** (channel-first bulk create) — shipped as #69: `/app/auto-tune`, a 3-step Configure → Preview → Create wizard, each channel backed by an auto-generated `SmartCollection` query (`AutoTuneScreen.tsx`, `CreateAutoTunedChannelsHandler.cs`, `AutoTuneAxisMap.cs`).
|
||||
- **Composite Create-Channel wizard** — shipped as #63: `web/src/builder/ChannelBuilder.tsx` at `/app/new-channel`, composing collection-lineup + template + playout + advanced overrides in one call (`POST /api/v1/channels/from-lineup`).
|
||||
|
||||
The **one genuinely missing leg is the visual WYSIWYG rule builder.** Authoring a `SmartCollection` query today is a *single free-text box* (`CollectionsScreen.tsx:271`) over the Lucene grammar — no field/operator/AND-OR UI. This design covers **sub-project #1: a visual rule builder landing in the SmartCollection create/edit dialog.** Inline adoption by ChannelBuilder and Auto-Tune is deliberately deferred (the builder is built as a reusable component so that later adoption is a trivial plug-in).
|
||||
|
||||
## 2. Approved architectural decisions
|
||||
|
||||
Three pillars, settled during brainstorming:
|
||||
|
||||
1. **Scope = the rule builder in the SmartCollection editor** (highest-traffic surface, standalone value, no dependency on the wizard).
|
||||
2. **Compile-only, closed subset, no stored AST, no schema change.** The builder emits a *canonical closed subset* of the Lucene grammar and parses exactly that subset back. Because compile and parse are exact inverses over a closed subset, anything the builder produces round-trips losslessly with nothing new persisted — the `SmartCollection` keeps storing a plain Lucene query string, exactly as today. Queries outside the subset (hand-written fuzzy/boost/wildcard-heavy Lucene) open in a raw-text fallback. This was chosen over persisting a rule AST because *even an authoritative AST would still need a Lucene→rules parser* for the many pre-existing free-text queries (every Auto-Tune query included), so the migration cost of storing an AST buys almost nothing.
|
||||
3. **Field catalog on the backend, compile/parse on the client.** A new read-only `GET /api/v1/search/fields` is the single source of truth for the field vocabulary (no client/index drift; MCP can introspect it). The compile/parse/round-trip logic lives in the SPA.
|
||||
|
||||
## 3. The rule model (Kodi model, one level of nesting)
|
||||
|
||||
```
|
||||
Rule = { field: string, operator: Operator, value: string | [string, string] }
|
||||
Group = { match: "all" | "any", children: Array<Rule | Group> }
|
||||
```
|
||||
|
||||
- **Top level is a `Group`.** Its `children` may be `Rule`s and/or `Group`s.
|
||||
- **Sub-groups are one level deep only** — a sub-`Group`'s children are a flat list of `Rule`s (no `Group` inside a `Group` inside a `Group`).
|
||||
- **Why one level:** pure-flat (match all/any only) can't express the most common real query, `type:movie AND (genre:Horror OR genre:Thriller)`; one level of nesting expresses it exactly (`all: [type:movie, any:[Horror, Thriller]]`) and matches what Kodi ships. Arbitrary nesting is YAGNI for channel queries and bloats the compiler/parser/property-test surface.
|
||||
|
||||
### The closed Lucene subset
|
||||
|
||||
- **rule → clause:**
|
||||
- text `is` → `field:"value"` · `is not` → `NOT field:"value"` · `contains` → `field:*value*` · `starts with` → `field:value*`
|
||||
- fulltext (`plot`) `contains` → `field:value` · `not contains` → `NOT field:value`
|
||||
- number `=` → `field:N` · `>` → `field:{N TO *}` · `<` → `field:{* TO N}` · `between` → `field:[a TO b]`
|
||||
- date `before` → `field:{* TO d}` · `after` → `field:{d TO *}` · `between` → `field:[a TO b]`
|
||||
- enum `is`/`is not` → `field:"value"` / `NOT field:"value"` (value from a fixed allowed set)
|
||||
- **group → clauses joined by `AND` (match=all) or `OR` (match=any)**, wrapped in parentheses when the group is nested.
|
||||
- Quoting: values containing whitespace or reserved characters are quoted; the compiler is the sole author of quoting so the parser can rely on it.
|
||||
|
||||
Any string not matching this shape is **unparseable** → the editor stays in raw-text mode.
|
||||
|
||||
## 4. The field catalog endpoint
|
||||
|
||||
**`GET /api/v1/search/fields`** — read-only, returns a curated, typed, labeled catalog derived from `LuceneSearchIndex`:
|
||||
|
||||
```json
|
||||
[{ "name": "genre", "label": "Genre", "type": "text", "group": "General" },
|
||||
{ "name": "minutes", "label": "Duration (min)", "type": "number", "group": "Technical" },
|
||||
{ "name": "release_date", "label": "Release date", "type": "date", "group": "Dates" },
|
||||
{ "name": "type", "label": "Item type", "type": "enum", "group": "General",
|
||||
"values": ["movie", "episode", "music_video", "other_video", "song", "image"] }]
|
||||
```
|
||||
|
||||
- **Curation happens server-side.** Internal/index-plumbing fields are excluded: `id`, `library_id`, `library_folder_id`, `tag_full`, `jump_letter`, `title_and_year_search`, `language_tag`, `sub_language_tag`, `metadata_kind` (kept only if it proves user-meaningful). The rest get friendly labels + a `group` (`General` / `TV` / `Movie` / `Music` / `Technical` / `Dates`).
|
||||
- **Type taxonomy:** `text` (keyword-exact metadata), `fulltext` (tokenized — `plot`), `number` (`minutes`, `seconds`, `height`, `width`, `season_number`, `episode_number`, `video_bit_depth`, `chapters`), `date` (`added_date`, `release_date`), `enum` (small closed sets — `type`, `video_dynamic_range`, `state`; each carries a `values` array).
|
||||
- The endpoint is the single source of truth: the SPA renders exactly what it returns, and MCP can introspect the same list.
|
||||
|
||||
**Follow the api-conventions checklist**, then regenerate `v1.json` + `endpoint-index.md` via `./scripts/update-openapi.sh` and `npm run generate:api`.
|
||||
|
||||
## 5. UI integration in the SmartCollection dialog
|
||||
|
||||
The create/edit dialog (`CollectionsScreen.tsx:200-283`) gains a **`Builder | Advanced` mode toggle**:
|
||||
|
||||
- **Builder mode** — the rule-group UI: a `Match: All | Any` toggle, a list of rule rows (`field ▾` → `operator ▾` → value input), `+ Add rule` / `+ Add group` (one nested level), remove buttons. The value input is chosen by field type: text box (text/fulltext), number input(s) (number, single or a-to-b pair), date picker(s) (date), dropdown (enum). Every edit **debounced-compiles → live preview** (item count + sample) via the existing `getLibraryBrowseItems({ query })`.
|
||||
- **Advanced mode** — today's raw text box (also live-previewed).
|
||||
- **Mode switching:** Builder→Advanced fills the text with the compiled query. Advanced→Builder best-effort parses; if the text is outside the closed subset, it **stays in Advanced** with an inline note ("This query is too advanced to show in the builder").
|
||||
- **Opening an existing SmartCollection:** parse the stored query — fits the subset → open in Builder with rules populated; otherwise → open in Advanced. (Every Auto-Tune query like `type:episode AND genre:"Horror"` fits, so those open cleanly in the builder.)
|
||||
- **Save is unchanged** — posts the compiled (or raw) Lucene string to the existing SmartCollection write path. **No backend write-path change**; the only new backend surface is the read-only `GET /api/v1/search/fields`.
|
||||
|
||||
### Component structure (isolated & reusable)
|
||||
|
||||
```
|
||||
web/src/builder/rules/ (co-located with ChannelBuilder; final path validated vs spa-conventions.md)
|
||||
types.ts Rule | Group | Match | Operator | FieldType
|
||||
fieldCatalog.ts useSearchFields() hook over GET /api/v1/search/fields
|
||||
compile.ts rules → lucene subset (sole author of quoting)
|
||||
parse.ts lucene subset → rules (exact inverse; returns null on out-of-subset)
|
||||
RuleBuilder.tsx the group/rule UI (consumed by CollectionsScreen; later by ChannelBuilder/Auto-Tune)
|
||||
```
|
||||
|
||||
Each unit has one purpose and a narrow interface: `compile(Group) → string`, `parse(string) → Group | null`, `RuleBuilder({ value: Group, onChange })`. `RuleBuilder` can be understood and tested without reading `compile`/`parse` internals, and those can change without touching the UI.
|
||||
|
||||
## 6. Testing
|
||||
|
||||
- **Round-trip property test (keystone):** a deterministic generator produces rule trees within the closed subset → `compile → parse →` assert deep-equal. This is what makes "compile-only, no stored AST" safe. Hand-rolled generator over the finite field/operator space (deterministic; introduce `fast-check` only if it is already a `web/` dependency).
|
||||
- **Compile/parse unit tests:** each operator → expected Lucene; each Lucene shape → expected rules; representative **out-of-subset strings → `parse` returns null** (drives the raw-text fallback).
|
||||
- **RuleBuilder component tests** (vitest + testing-library): add/remove rule & group, match toggle, mode-switch fills text, editing an existing query populates the builder. Give heavy-render tests explicit vitest timeouts (CI-VM guidance).
|
||||
- **Backend** (NUnit + Shouldly): `GET /api/v1/search/fields` returns the curated list, excludes internal fields, and reports correct types.
|
||||
- **Live-E2E:** *not gated* by the write-path rule — the only new backend surface is a read-only GET and the SmartCollection write path is unchanged. Still run a light Playwright smoke (build a rule → preview count → save → reopen in builder) and a **cold-context adversarial review over the diff** (the parser is subtle enough to warrant it even though the skip rubric would permit skipping).
|
||||
|
||||
## 7. Docs to update in the same PR
|
||||
|
||||
- **New endpoint** → `docs/api-conventions.md` checklist, then regenerate `v1.json` + `endpoint-index.md` (`./scripts/update-openapi.sh` + `npm run generate:api`).
|
||||
- **`docs/decisions.md`** (append-only) — record the compile-only / closed-subset / no-stored-AST decision and the one-level-nesting choice.
|
||||
- **`docs/spa-conventions.md`** — document the reusable rule-builder component pattern if it establishes one.
|
||||
- **Not touched:** `blazor-route-parity.md` / `domain-model.md` — no route change, no new entity.
|
||||
|
||||
## 8. Scope boundaries — deferred as tracked follow-up issues
|
||||
|
||||
These are explicitly **out** of sub-project #1 and will be filed as separate issues:
|
||||
|
||||
1. **Facet-value typeahead (v1b)** — a `GET /api/v1/search/fields/{name}/values?q=` distinct-values endpoint turning text values into autocomplete comboboxes. Purely additive: swap the text input for a combobox, nothing else changes. (Keyword-exact matching means typos silently return zero; live-preview count is the v1 safety net.)
|
||||
2. **Relative dates** ("in the last N days") — can't be stored as *relative* in a compile-only Lucene string; needs a query-time macro.
|
||||
3. **Nested groups beyond one level.**
|
||||
4. **Inline adoption in ChannelBuilder & Auto-Tune** — reuse `RuleBuilder` for inline query authoring in the channel wizard and to generalize Auto-Tune's fixed 3-axis picker.
|
||||
|
||||
## 9. Risks
|
||||
|
||||
- **Round-trip correctness** is the central risk; the property test is the mitigation, and the compiler being the *sole* author of quoting keeps the parser's job bounded.
|
||||
- **Field/label curation drift** — mitigated by deriving the catalog from `LuceneSearchIndex` server-side (single source of truth). Adding an index field simply won't appear until curated — additive, low harm.
|
||||
- **Live-preview chatter** — debounce compile→preview; reuse the existing preview call so no new load pattern is introduced.
|
||||
Reference in New Issue
Block a user