# SmartCollection visual rule builder — design (sub-project #1 of #176)

**Issue:** [ersatztv#176](http://192.168.1.95:3000/timothy/ersatztv/issues/176) — "Rethink channel/playout/schedule creation: PseudoTV-style channel-first flow with a Kodi-smart-playlist-like query builder."

**Date:** 2026-07-17 · **Status:** design approved, pre-implementation.

---

## 1. Context & why the scope shrank

#176 as originally filed described three legs of a PseudoTV-style vision. Recon (2026-07-17) shows two of them already shipped since the issue was written:

- **Autotune** (channel-first bulk create) — shipped as #69: `/app/auto-tune`, a 3-step Configure → Preview → Create wizard, each channel backed by an auto-generated `SmartCollection` query (`AutoTuneScreen.tsx`, `CreateAutoTunedChannelsHandler.cs`, `AutoTuneAxisMap.cs`).
- **Composite Create-Channel wizard** — shipped as #63: `web/src/builder/ChannelBuilder.tsx` at `/app/new-channel`, composing collection-lineup + template + playout + advanced overrides in one call (`POST /api/v1/channels/from-lineup`).

The **one genuinely missing leg is the visual WYSIWYG rule builder.** Authoring a `SmartCollection` query today is a *single free-text box* (`CollectionsScreen.tsx:271`) over the Lucene grammar — no field/operator/AND-OR UI. This design covers **sub-project #1: a visual rule builder landing in the SmartCollection create/edit dialog.** Inline adoption by ChannelBuilder and Auto-Tune is deliberately deferred (the builder is built as a reusable component so that later adoption is a trivial plug-in).

## 2. Approved architectural decisions

Three pillars, settled during brainstorming:

1. **Scope = the rule builder in the SmartCollection editor** (highest-traffic surface, standalone value, no dependency on the wizard).
2. **Compile-only, closed subset, no stored AST, no schema change.** The builder emits a *canonical closed subset* of the Lucene grammar and parses exactly that subset back. Because compile and parse are exact inverses over a closed subset, anything the builder produces round-trips losslessly with nothing new persisted — the `SmartCollection` keeps storing a plain Lucene query string, exactly as today. Queries outside the subset (hand-written fuzzy/boost/wildcard-heavy Lucene) open in a raw-text fallback. This was chosen over persisting a rule AST because *even an authoritative AST would still need a Lucene→rules parser* for the many pre-existing free-text queries (every Auto-Tune query included), so the migration cost of storing an AST buys almost nothing.
3. **Field catalog on the backend, compile/parse on the client.** A new read-only `GET /api/v1/search/fields` is the single source of truth for the field vocabulary (no client/index drift; MCP can introspect it). The compile/parse/round-trip logic lives in the SPA.

## 3. The rule model (Kodi model, one level of nesting)

```
Rule  = { field: string, operator: Operator, value: string | [string, string] }
Group = { match: "all" | "any", children: Array<Rule | Group> }
```

- **Top level is a `Group`.** Its `children` may be `Rule`s and/or `Group`s.
- **Sub-groups are one level deep only** — a sub-`Group`'s children are a flat list of `Rule`s (no `Group` inside a `Group` inside a `Group`).
- **Why one level:** pure-flat (match all/any only) can't express the most common real query, `type:movie AND (genre:Horror OR genre:Thriller)`; one level of nesting expresses it exactly (`all: [type:movie, any:[Horror, Thriller]]`) and matches what Kodi ships. Arbitrary nesting is YAGNI for channel queries and bloats the compiler/parser/property-test surface.

### The closed Lucene subset

- **rule → clause:**
  - text `is` → `field:"value"` · `is not` → `NOT field:"value"` · `contains` → `field:*value*` · `starts with` → `field:value*`
  - fulltext (`plot`) `contains` → `field:value` · `not contains` → `NOT field:value`
  - number `=` → `field:N` · `>` → `field:{N TO *}` · `<` → `field:{* TO N}` · `between` → `field:[a TO b]`
  - date `before` → `field:{* TO d}` · `after` → `field:{d TO *}` · `between` → `field:[a TO b]`
  - enum `is`/`is not` → `field:"value"` / `NOT field:"value"` (value from a fixed allowed set)
- **group → clauses joined by `AND` (match=all) or `OR` (match=any)**, wrapped in parentheses when the group is nested.
- Quoting: values containing whitespace or reserved characters are quoted; the compiler is the sole author of quoting so the parser can rely on it.

Any string not matching this shape is **unparseable** → the editor stays in raw-text mode.

## 4. The field catalog endpoint

**`GET /api/v1/search/fields`** — read-only, returns a curated, typed, labeled catalog derived from `LuceneSearchIndex`:

```json
[{ "name": "genre", "label": "Genre", "type": "text", "group": "General" },
 { "name": "minutes", "label": "Duration (min)", "type": "number", "group": "Technical" },
 { "name": "release_date", "label": "Release date", "type": "date", "group": "Dates" },
 { "name": "type", "label": "Item type", "type": "enum", "group": "General",
   "values": ["movie", "episode", "music_video", "other_video", "song", "image"] }]
```

- **Curation happens server-side.** Internal/index-plumbing fields are excluded: `id`, `library_id`, `library_folder_id`, `tag_full`, `jump_letter`, `title_and_year_search`, `language_tag`, `sub_language_tag`, `metadata_kind` (kept only if it proves user-meaningful). The rest get friendly labels + a `group` (`General` / `TV` / `Movie` / `Music` / `Technical` / `Dates`).
- **Type taxonomy:** `text` (keyword-exact metadata), `fulltext` (tokenized — `plot`), `number` (`minutes`, `seconds`, `height`, `width`, `season_number`, `episode_number`, `video_bit_depth`, `chapters`), `date` (`added_date`, `release_date`), `enum` (small closed sets — `type`, `video_dynamic_range`, `state`; each carries a `values` array).
- The endpoint is the single source of truth: the SPA renders exactly what it returns, and MCP can introspect the same list.

**Follow the api-conventions checklist**, then regenerate `v1.json` + `endpoint-index.md` via `./scripts/update-openapi.sh` and `npm run generate:api`.

## 5. UI integration in the SmartCollection dialog

The create/edit dialog (`CollectionsScreen.tsx:200-283`) gains a **`Builder | Advanced` mode toggle**:

- **Builder mode** — the rule-group UI: a `Match: All | Any` toggle, a list of rule rows (`field ▾` → `operator ▾` → value input), `+ Add rule` / `+ Add group` (one nested level), remove buttons. The value input is chosen by field type: text box (text/fulltext), number input(s) (number, single or a-to-b pair), date picker(s) (date), dropdown (enum). Every edit **debounced-compiles → live preview** (item count + sample) via the existing `getLibraryBrowseItems({ query })`.
- **Advanced mode** — today's raw text box (also live-previewed).
- **Mode switching:** Builder→Advanced fills the text with the compiled query. Advanced→Builder best-effort parses; if the text is outside the closed subset, it **stays in Advanced** with an inline note ("This query is too advanced to show in the builder").
- **Opening an existing SmartCollection:** parse the stored query — fits the subset → open in Builder with rules populated; otherwise → open in Advanced. (Every Auto-Tune query like `type:episode AND genre:"Horror"` fits, so those open cleanly in the builder.)
- **Save is unchanged** — posts the compiled (or raw) Lucene string to the existing SmartCollection write path. **No backend write-path change**; the only new backend surface is the read-only `GET /api/v1/search/fields`.

### Component structure (isolated & reusable)

```
web/src/builder/rules/            (co-located with ChannelBuilder; final path validated vs spa-conventions.md)
  types.ts         Rule | Group | Match | Operator | FieldType
  fieldCatalog.ts  useSearchFields() hook over GET /api/v1/search/fields
  compile.ts       rules → lucene subset (sole author of quoting)
  parse.ts         lucene subset → rules (exact inverse; returns null on out-of-subset)
  RuleBuilder.tsx  the group/rule UI (consumed by CollectionsScreen; later by ChannelBuilder/Auto-Tune)
```

Each unit has one purpose and a narrow interface: `compile(Group) → string`, `parse(string) → Group | null`, `RuleBuilder({ value: Group, onChange })`. `RuleBuilder` can be understood and tested without reading `compile`/`parse` internals, and those can change without touching the UI.

## 6. Testing

- **Round-trip property test (keystone):** a deterministic generator produces rule trees within the closed subset → `compile → parse →` assert deep-equal. This is what makes "compile-only, no stored AST" safe. Hand-rolled generator over the finite field/operator space (deterministic; introduce `fast-check` only if it is already a `web/` dependency).
- **Compile/parse unit tests:** each operator → expected Lucene; each Lucene shape → expected rules; representative **out-of-subset strings → `parse` returns null** (drives the raw-text fallback).
- **RuleBuilder component tests** (vitest + testing-library): add/remove rule & group, match toggle, mode-switch fills text, editing an existing query populates the builder. Give heavy-render tests explicit vitest timeouts (CI-VM guidance).
- **Backend** (NUnit + Shouldly): `GET /api/v1/search/fields` returns the curated list, excludes internal fields, and reports correct types.
- **Live-E2E:** *not gated* by the write-path rule — the only new backend surface is a read-only GET and the SmartCollection write path is unchanged. Still run a light Playwright smoke (build a rule → preview count → save → reopen in builder) and a **cold-context adversarial review over the diff** (the parser is subtle enough to warrant it even though the skip rubric would permit skipping).

## 7. Docs to update in the same PR

- **New endpoint** → `docs/api-conventions.md` checklist, then regenerate `v1.json` + `endpoint-index.md` (`./scripts/update-openapi.sh` + `npm run generate:api`).
- **`docs/decisions.md`** (append-only) — record the compile-only / closed-subset / no-stored-AST decision and the one-level-nesting choice.
- **`docs/spa-conventions.md`** — document the reusable rule-builder component pattern if it establishes one.
- **Not touched:** `blazor-route-parity.md` / `domain-model.md` — no route change, no new entity.

## 8. Scope boundaries — deferred as tracked follow-up issues

These are explicitly **out** of sub-project #1 and will be filed as separate issues:

1. **Facet-value typeahead (v1b)** — a `GET /api/v1/search/fields/{name}/values?q=` distinct-values endpoint turning text values into autocomplete comboboxes. Purely additive: swap the text input for a combobox, nothing else changes. (Keyword-exact matching means typos silently return zero; live-preview count is the v1 safety net.)
2. **Relative dates** ("in the last N days") — can't be stored as *relative* in a compile-only Lucene string; needs a query-time macro.
3. **Nested groups beyond one level.**
4. **Inline adoption in ChannelBuilder & Auto-Tune** — reuse `RuleBuilder` for inline query authoring in the channel wizard and to generalize Auto-Tune's fixed 3-axis picker.

## 9. Risks

- **Round-trip correctness** is the central risk; the property test is the mitigation, and the compiler being the *sole* author of quoting keeps the parser's job bounded.
- **Field/label curation drift** — mitigated by deriving the catalog from `LuceneSearchIndex` server-side (single source of truth). Adding an index field simply won't appear until curated — additive, low harm.
- **Live-preview chatter** — debounce compile→preview; reuse the existing preview call so no new load pattern is introduced.
