1294 lines
58 KiB
Markdown
1294 lines
58 KiB
Markdown
# SmartCollection Visual Rule Builder — Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** Add a Kodi-style visual rule builder to the SmartCollection create/edit dialog that compiles to (and parses back from) a closed subset of the existing Lucene query grammar, with a read-only backend field catalog as the single source of truth.
|
||
|
||
**Architecture:** A new read-only `GET /api/v1/search/fields` returns a curated, typed, labeled field catalog (MediatR query + static list in `ErsatzTV.Application`). The SPA gains pure-TS `compile`/`parse` modules over a closed Lucene subset (exact inverses, proven by a round-trip property test), a `RuleBuilder` React component, and a `Builder | Advanced` mode toggle in the existing `SmartDialog`. Nothing new is persisted — the SmartCollection still stores a plain Lucene query string, so there is **no DB migration and no write-path change**.
|
||
|
||
**Tech Stack:** C# / .NET 10, MediatR (CQRS), ASP.NET Core controllers; React + TypeScript SPA (Vite), vitest + @testing-library/react + jsdom. NUnit + Shouldly for backend tests.
|
||
|
||
## Global Constraints
|
||
|
||
- **.NET 10**; functional C# with LanguageExt where the surrounding code uses it.
|
||
- **Backend tests: NUnit + Shouldly + NSubstitute only** — xUnit is not used here.
|
||
- **No new NuGet or npm packages.** Central Package Management (repo-root `Directory.Packages.props`); the round-trip generator is hand-rolled (do **not** add `fast-check`).
|
||
- **Every API action needs:** `[ApiController]`, absolute versioned route `[HttpGet("/api/v1/...", Name="...")]`, `[Tags("Search")]`, `[EndpointSummary(...)]`, **`[EndpointGroupName("general")]` (REQUIRED — omitting it drops the endpoint from the OpenAPI doc)**, and `[ProducesResponseType(typeof(X), StatusCodes.Status200OK)]`.
|
||
- **After any `/api/*` change:** build the app FIRST, then `./scripts/update-openapi.sh`, then `cd web && npm run generate:api`. Regenerated `ErsatzTV/wwwroot/openapi/v1.json`, `docs/endpoint-index.md`, and `web/src/api/generated/v1.d.ts` are committed.
|
||
- **Docs in the same PR:** tick `docs/api-conventions.md`; append to `docs/decisions.md`; update `docs/spa-conventions.md` for the new component pattern. (No `blazor-route-parity.md` / `domain-model.md` — no route/entity change.)
|
||
- **Before every push touching `.cs`:** BOM-check the touched set — `for f in $(git diff --name-only origin/main...HEAD -- '*.cs'); do head -c3 "$f" | xxd -p | grep -q "^efbbbf" && echo "BOM: $f"; done` — strip any hit (`charset=utf-8` ⇒ no BOM).
|
||
- **Worktree:** all work in `.claude/worktrees/176-rule-builder` (branch `feat/176-smartcollection-rule-builder`, off `origin/main`). Copy `web/node_modules` from the shared tree read-only if needed: `cp -R /Users/timothy/ersatztv/web/node_modules web/node_modules`.
|
||
- **The compiler is the SOLE author of quoting/escaping** so the parser can rely on canonical forms.
|
||
|
||
---
|
||
|
||
## File Structure
|
||
|
||
**Backend (new):**
|
||
- `ErsatzTV.Core/Api/Search/SearchFieldResponseModel.cs` — response DTO (record).
|
||
- `ErsatzTV.Application/Search/Queries/GetSearchFieldCatalog.cs` — MediatR query (marker record).
|
||
- `ErsatzTV.Application/Search/Queries/GetSearchFieldCatalogHandler.cs` — returns the curated static catalog.
|
||
- `ErsatzTV.Application/Search/SearchFieldCatalog.cs` — the curated static list (product-owned, cross-references `LuceneSearchIndex`).
|
||
- `ErsatzTV/Controllers/Api/SearchController.cs` — **modify**: add the `GET /api/v1/search/fields` action.
|
||
- `ErsatzTV.Tests/Application/Search/GetSearchFieldCatalogHandlerTests.cs` — NUnit test (new; Application handler tests live under `ErsatzTV.Tests/Application/<Domain>/`, namespace `ErsatzTV.Tests.Application.<Domain>`, run via `dotnet test ErsatzTV.Tests`).
|
||
|
||
**SPA (new — `web/src/builder/rules/`):**
|
||
- `types.ts` — `FieldType`, `Operator`, `Rule`, `Group`, `Match`, `isGroup`.
|
||
- `compile.ts` — `compile(group) → string`.
|
||
- `parse.ts` — `parse(str, fieldTypes) → Group | null`.
|
||
- `compile.test.ts`, `parse.test.ts`, `roundtrip.test.ts`.
|
||
- `fieldCatalog.ts` — `getSearchFields()` + `useSearchFields()` hook.
|
||
- `RuleBuilder.tsx` — the group/rule UI.
|
||
- `RuleBuilder.test.tsx`.
|
||
|
||
**SPA (modify):**
|
||
- `web/src/api/search.ts` — add `getSearchFields` + `SearchField` type.
|
||
- `web/src/screens/CollectionsScreen.tsx` — `SmartDialog`: `Builder | Advanced` toggle wiring.
|
||
|
||
**Docs (modify):** `docs/api-conventions.md`, `docs/decisions.md`, `docs/spa-conventions.md`.
|
||
|
||
---
|
||
|
||
## Task 1: Field catalog — response model, curated list, MediatR handler
|
||
|
||
**Files:**
|
||
- Create: `ErsatzTV.Core/Api/Search/SearchFieldResponseModel.cs`
|
||
- Create: `ErsatzTV.Application/Search/SearchFieldCatalog.cs`
|
||
- Create: `ErsatzTV.Application/Search/Queries/GetSearchFieldCatalog.cs`
|
||
- Create: `ErsatzTV.Application/Search/Queries/GetSearchFieldCatalogHandler.cs`
|
||
- Test: `ErsatzTV.Tests/Application/Search/GetSearchFieldCatalogHandlerTests.cs`
|
||
|
||
**Interfaces:**
|
||
- Produces: `record SearchFieldResponseModel(string Name, string Label, string Type, string Group, string[] Values)`; `record GetSearchFieldCatalog : IRequest<List<SearchFieldResponseModel>>`; `SearchFieldCatalog.Fields` (static `List<SearchFieldResponseModel>`).
|
||
|
||
- [ ] **Step 1: (resolved) Test location**
|
||
|
||
Application handler tests live in `ErsatzTV.Tests/Application/<Domain>/` with namespace `ErsatzTV.Tests.Application.<Domain>`, run via `dotnet test ErsatzTV.Tests`. The new test goes in `ErsatzTV.Tests/Application/Search/`. This repo uses **explicit `using`s** (no ImplicitUsings) — include the `System.*` usings shown in Step 5. Exemplar: `ErsatzTV.Tests/Application/Channels/PreviewAutoTuneChannelsHandlerTests.cs`.
|
||
|
||
- [ ] **Step 2: Write the response model**
|
||
|
||
`ErsatzTV.Core/Api/Search/SearchFieldResponseModel.cs`:
|
||
|
||
```csharp
|
||
namespace ErsatzTV.Core.Api.Search;
|
||
|
||
/// <summary>
|
||
/// One filterable field in the visual rule builder's catalog.
|
||
/// <c>Type</c> is one of: text, fulltext, number, date, enum.
|
||
/// <c>Values</c> is populated only for <c>enum</c> fields (allowed dropdown values); empty otherwise.
|
||
/// </summary>
|
||
public record SearchFieldResponseModel(string Name, string Label, string Type, string Group, string[] Values);
|
||
```
|
||
|
||
- [ ] **Step 3: Write the curated catalog**
|
||
|
||
`ErsatzTV.Application/Search/SearchFieldCatalog.cs`. This is a **product-curated** subset of the Lucene index fields (see `ErsatzTV.Infrastructure/Search/LuceneSearchIndex.cs`); internal/index-plumbing fields (`id`, `tag_full`, `library_id`, `library_folder_id`, `jump_letter`, `language_tag`, `sub_language_tag`, `title_and_year_search`, `metadata_kind`) are intentionally omitted. Curation lives here by design, not mirrored mechanically, so Application does not depend on Infrastructure.
|
||
|
||
```csharp
|
||
using ErsatzTV.Core.Api.Search;
|
||
|
||
namespace ErsatzTV.Application.Search;
|
||
|
||
public static class SearchFieldCatalog
|
||
{
|
||
private static readonly string[] None = [];
|
||
|
||
// Allowed values for the `type` enum — mirror the lowercase tokens the Lucene index stores for
|
||
// the `type` field. VERIFY against LuceneSearchIndex.cs during implementation and adjust if the
|
||
// tokens differ; the GetSearchFieldCatalogHandlerTests asserts this list is non-empty.
|
||
private static readonly string[] ItemTypes =
|
||
["movie", "show", "season", "episode", "artist", "music_video", "other_video", "song", "image"];
|
||
|
||
public static readonly List<SearchFieldResponseModel> Fields =
|
||
[
|
||
// General
|
||
new("title", "Title", "text", "General", None),
|
||
new("genre", "Genre", "text", "General", None),
|
||
new("tag", "Tag", "text", "General", None),
|
||
new("plot", "Plot", "fulltext", "General", None),
|
||
new("content_rating", "Content rating", "text", "General", None),
|
||
new("studio", "Studio", "text", "General", None),
|
||
new("collection", "Collection", "text", "General", None),
|
||
new("state", "State", "text", "General", None),
|
||
new("type", "Item type", "enum", "General", ItemTypes),
|
||
|
||
// TV
|
||
new("network", "Network", "text", "TV", None),
|
||
new("show_title", "Show title", "text", "TV", None),
|
||
new("show_genre", "Show genre", "text", "TV", None),
|
||
new("season_number", "Season number", "number", "TV", None),
|
||
new("episode_number", "Episode number", "number", "TV", None),
|
||
|
||
// Movie / People
|
||
new("director", "Director", "text", "Movie", None),
|
||
new("writer", "Writer", "text", "Movie", None),
|
||
new("actor", "Actor", "text", "Movie", None),
|
||
|
||
// Music
|
||
new("artist", "Artist", "text", "Music", None),
|
||
new("album", "Album", "text", "Music", None),
|
||
new("album_artist", "Album artist", "text", "Music", None),
|
||
|
||
// Technical
|
||
new("minutes", "Duration (min)", "number", "Technical", None),
|
||
new("height", "Height (px)", "number", "Technical", None),
|
||
new("width", "Width (px)", "number", "Technical", None),
|
||
new("video_codec", "Video codec", "text", "Technical", None),
|
||
new("video_dynamic_range", "Dynamic range", "text", "Technical", None),
|
||
|
||
// Dates
|
||
new("added_date", "Date added", "date", "Dates", None),
|
||
new("release_date", "Release date", "date", "Dates", None)
|
||
];
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Write the MediatR query + handler**
|
||
|
||
`ErsatzTV.Application/Search/Queries/GetSearchFieldCatalog.cs`:
|
||
|
||
```csharp
|
||
using ErsatzTV.Core.Api.Search;
|
||
using MediatR;
|
||
|
||
namespace ErsatzTV.Application.Search.Queries;
|
||
|
||
public record GetSearchFieldCatalog : IRequest<List<SearchFieldResponseModel>>;
|
||
```
|
||
|
||
`ErsatzTV.Application/Search/Queries/GetSearchFieldCatalogHandler.cs`:
|
||
|
||
```csharp
|
||
using ErsatzTV.Core.Api.Search;
|
||
using MediatR;
|
||
|
||
namespace ErsatzTV.Application.Search.Queries;
|
||
|
||
public class GetSearchFieldCatalogHandler : IRequestHandler<GetSearchFieldCatalog, List<SearchFieldResponseModel>>
|
||
{
|
||
public Task<List<SearchFieldResponseModel>> Handle(
|
||
GetSearchFieldCatalog request,
|
||
CancellationToken cancellationToken) =>
|
||
Task.FromResult(SearchFieldCatalog.Fields);
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 5: Write the failing NUnit test**
|
||
|
||
`ErsatzTV.Tests/Application/Search/GetSearchFieldCatalogHandlerTests.cs`:
|
||
|
||
```csharp
|
||
using System.Collections.Generic;
|
||
using System.Linq;
|
||
using System.Threading;
|
||
using System.Threading.Tasks;
|
||
using ErsatzTV.Application.Search.Queries;
|
||
using ErsatzTV.Core.Api.Search;
|
||
using NUnit.Framework;
|
||
using Shouldly;
|
||
|
||
namespace ErsatzTV.Tests.Application.Search;
|
||
|
||
[TestFixture]
|
||
public class GetSearchFieldCatalogHandlerTests
|
||
{
|
||
[Test]
|
||
public async Task Returns_Curated_Catalog_Without_Internal_Fields()
|
||
{
|
||
var handler = new GetSearchFieldCatalogHandler();
|
||
|
||
List<SearchFieldResponseModel> result = await handler.Handle(new GetSearchFieldCatalog(), CancellationToken.None);
|
||
|
||
result.ShouldNotBeEmpty();
|
||
result.Select(f => f.Name).ShouldNotContain("id");
|
||
result.Select(f => f.Name).ShouldNotContain("tag_full");
|
||
result.Select(f => f.Name).ShouldNotContain("library_folder_id");
|
||
}
|
||
|
||
[Test]
|
||
public async Task Every_Field_Has_A_Known_Type_And_A_Group()
|
||
{
|
||
var handler = new GetSearchFieldCatalogHandler();
|
||
string[] knownTypes = ["text", "fulltext", "number", "date", "enum"];
|
||
|
||
List<SearchFieldResponseModel> result = await handler.Handle(new GetSearchFieldCatalog(), CancellationToken.None);
|
||
|
||
foreach (SearchFieldResponseModel field in result)
|
||
{
|
||
knownTypes.ShouldContain(field.Type);
|
||
field.Group.ShouldNotBeNullOrWhiteSpace();
|
||
field.Label.ShouldNotBeNullOrWhiteSpace();
|
||
}
|
||
}
|
||
|
||
[Test]
|
||
public async Task Enum_Fields_Carry_NonEmpty_Values_And_NonEnum_Do_Not()
|
||
{
|
||
var handler = new GetSearchFieldCatalogHandler();
|
||
|
||
List<SearchFieldResponseModel> result = await handler.Handle(new GetSearchFieldCatalog(), CancellationToken.None);
|
||
|
||
foreach (SearchFieldResponseModel field in result)
|
||
{
|
||
if (field.Type == "enum")
|
||
{
|
||
field.Values.ShouldNotBeEmpty();
|
||
}
|
||
else
|
||
{
|
||
field.Values.ShouldBeEmpty();
|
||
}
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 6: Run the test to verify it fails, then passes after building**
|
||
|
||
Run: `cd /Users/timothy/ersatztv/.claude/worktrees/176-rule-builder && dotnet test ErsatzTV.Tests --filter GetSearchFieldCatalogHandlerTests`
|
||
Expected: fails to compile first if any type is missing, then PASS once Steps 2–4 are in. **Grep the build for `error CS` before trusting a `--no-build` result.**
|
||
|
||
- [ ] **Step 7: Commit**
|
||
|
||
```bash
|
||
git add ErsatzTV.Core/Api/Search ErsatzTV.Application/Search ErsatzTV.Tests/Application/Search
|
||
git -c core.hooksPath=/dev/null commit -m "feat(176): search field catalog query + curated list"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 2: Wire the endpoint, regenerate OpenAPI + TS types, tick api-conventions
|
||
|
||
**Files:**
|
||
- Modify: `ErsatzTV/Controllers/Api/SearchController.cs`
|
||
- Modify (generated): `ErsatzTV/wwwroot/openapi/v1.json`, `docs/endpoint-index.md`, `web/src/api/generated/v1.d.ts`
|
||
- Modify: `docs/api-conventions.md`
|
||
|
||
**Interfaces:**
|
||
- Produces: `GET /api/v1/search/fields` → `List<SearchFieldResponseModel>`; generated TS type `components['schemas']['SearchFieldResponseModel']`.
|
||
|
||
- [ ] **Step 1: Add the action to `SearchController`**
|
||
|
||
Add `using ErsatzTV.Application.Search.Queries;` and `using ErsatzTV.Core.Api.Search;` at the top (some may already be present), then this action inside the class:
|
||
|
||
```csharp
|
||
[HttpGet("/api/v1/search/fields", Name = "GetSearchFields")]
|
||
[Tags("Search")]
|
||
[EndpointSummary("List the filterable fields for the visual rule builder")]
|
||
[EndpointDescription(
|
||
"Returns the curated catalog of searchable fields (name, friendly label, type, UI group, and " +
|
||
"allowed values for enum fields). Drives the SmartCollection rule builder and is introspectable by MCP.")]
|
||
[EndpointGroupName("general")]
|
||
[ProducesResponseType(typeof(List<SearchFieldResponseModel>), StatusCodes.Status200OK)]
|
||
public Task<List<SearchFieldResponseModel>> GetSearchFields(CancellationToken cancellationToken) =>
|
||
mediator.Send(new GetSearchFieldCatalog(), cancellationToken);
|
||
```
|
||
|
||
- [ ] **Step 2: Build the app, regenerate the OpenAPI doc + endpoint index**
|
||
|
||
Run: `cd /Users/timothy/ersatztv/.claude/worktrees/176-rule-builder && ./scripts/update-openapi.sh`
|
||
Expected: builds `ErsatzTV`, regenerates `v1.json` and `docs/endpoint-index.md`. Confirm `GetSearchFields` appears: `grep -n "search/fields" docs/endpoint-index.md`.
|
||
|
||
- [ ] **Step 3: Regenerate the SPA client types**
|
||
|
||
Run: `cd web && npm run generate:api && grep -n "SearchFieldResponseModel" src/api/generated/v1.d.ts`
|
||
Expected: the schema is present in `v1.d.ts`.
|
||
|
||
- [ ] **Step 4: Tick the api-conventions checklist**
|
||
|
||
In `docs/api-conventions.md`, add `GET /api/v1/search/fields` to the endpoint inventory/exemplar list in the same style as neighboring read-only GET entries (a one-line mention that it returns the rule-builder field catalog). This is the "update the doc in the same PR" requirement for a new endpoint.
|
||
|
||
- [ ] **Step 5: Sanity-run the app and curl the endpoint (optional but recommended)**
|
||
|
||
If a local run is convenient: `dotnet run --project ErsatzTV` then `curl -s localhost:<port>/api/v1/search/fields | jq '.[0]'`.
|
||
Expected: a JSON object `{ "name": "title", "label": "Title", "type": "text", "group": "General", "values": [] }`.
|
||
|
||
- [ ] **Step 6: Commit (BOM-check first)**
|
||
|
||
```bash
|
||
for f in $(git diff --name-only origin/main...HEAD -- '*.cs'); do head -c3 "$f" | xxd -p | grep -q "^efbbbf" && echo "BOM: $f"; done
|
||
git add ErsatzTV/Controllers/Api/SearchController.cs ErsatzTV/wwwroot/openapi/v1.json docs/endpoint-index.md docs/api-conventions.md web/src/api/generated/v1.d.ts
|
||
git -c core.hooksPath=/dev/null commit -m "feat(176): GET /api/v1/search/fields endpoint + regenerated api artifacts"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 3: TS rule model + compiler (`types.ts`, `compile.ts`)
|
||
|
||
**Files:**
|
||
- Create: `web/src/builder/rules/types.ts`
|
||
- Create: `web/src/builder/rules/compile.ts`
|
||
- Test: `web/src/builder/rules/compile.test.ts`
|
||
|
||
**Interfaces:**
|
||
- Produces: `FieldType = 'text'|'fulltext'|'number'|'date'|'enum'`; `Operator`; `Rule`; `Group`; `Match`; `isGroup(node)`; `compile(group: Group): string`.
|
||
- The canonical compiled forms (parser in Task 4 is the exact inverse):
|
||
- text `is` → `field:"v"` · `isNot` → `NOT field:"v"` · `contains` → `field:*v*` · `startsWith` → `field:v*`
|
||
- fulltext `matches` → `field:"v"` · `notMatches` → `NOT field:"v"` (distinct operator names from text's `contains`, so `compileRule` maps each operator to exactly one canonical form; parse disambiguates the shared `field:"v"` shape by field type: text/enum→`is`, fulltext→`matches`)
|
||
- enum `is`/`isNot` → `field:"v"` / `NOT field:"v"`
|
||
- number `eq` → `field:v` · `gt` → `field:{v TO *}` · `lt` → `field:{* TO v}` · `between` → `field:[v TO v2]`
|
||
- date `before` → `field:{* TO v}` · `after` → `field:{v TO *}` · `between` → `field:[v TO v2]`
|
||
- group: children joined by ` AND ` (match `all`) / ` OR ` (match `any`); a **nested** group is wrapped in `( … )`. Top-level group is not wrapped.
|
||
|
||
- [ ] **Step 1: Write `types.ts`**
|
||
|
||
```ts
|
||
export type FieldType = 'text' | 'fulltext' | 'number' | 'date' | 'enum';
|
||
export type Match = 'all' | 'any';
|
||
|
||
export type Operator =
|
||
| 'is' | 'isNot' | 'contains' | 'startsWith' // text
|
||
| 'matches' | 'notMatches' // fulltext
|
||
| 'eq' | 'gt' | 'lt' | 'between' // number
|
||
| 'before' | 'after'; // date
|
||
|
||
export interface Rule {
|
||
field: string;
|
||
operator: Operator;
|
||
value: string;
|
||
value2?: string; // upper bound for `between`
|
||
}
|
||
|
||
export interface Group {
|
||
match: Match;
|
||
children: Array<Rule | Group>;
|
||
}
|
||
|
||
export function isGroup(node: Rule | Group): node is Group {
|
||
return (node as Group).children !== undefined;
|
||
}
|
||
|
||
// Which fields each catalog type exposes as operators (used by the UI and tests).
|
||
export const OPERATORS_BY_TYPE: Record<FieldType, Operator[]> = {
|
||
text: ['is', 'isNot', 'contains', 'startsWith'],
|
||
fulltext: ['matches', 'notMatches'],
|
||
enum: ['is', 'isNot'],
|
||
number: ['eq', 'gt', 'lt', 'between'],
|
||
date: ['before', 'after', 'between']
|
||
};
|
||
```
|
||
|
||
- [ ] **Step 2: Write the failing compiler test**
|
||
|
||
`web/src/builder/rules/compile.test.ts`:
|
||
|
||
```ts
|
||
import { describe, expect, it } from 'vitest';
|
||
import { compile } from './compile';
|
||
import type { Group } from './types';
|
||
|
||
describe('compile', () => {
|
||
it('quotes text is', () => {
|
||
const g: Group = { match: 'all', children: [{ field: 'genre', operator: 'is', value: 'Horror' }] };
|
||
expect(compile(g)).toBe('genre:"Horror"');
|
||
});
|
||
|
||
it('emits NOT for isNot', () => {
|
||
const g: Group = { match: 'all', children: [{ field: 'genre', operator: 'isNot', value: 'Horror' }] };
|
||
expect(compile(g)).toBe('NOT genre:"Horror"');
|
||
});
|
||
|
||
it('wildcards text contains and startsWith', () => {
|
||
expect(compile({ match: 'all', children: [{ field: 'title', operator: 'contains', value: 'night' }] })).toBe('title:*night*');
|
||
expect(compile({ match: 'all', children: [{ field: 'title', operator: 'startsWith', value: 'The' }] })).toBe('title:The*');
|
||
});
|
||
|
||
it('quotes fulltext matches / notMatches (distinct from text contains)', () => {
|
||
expect(compile({ match: 'all', children: [{ field: 'plot', operator: 'matches', value: 'car chase' }] })).toBe('plot:"car chase"');
|
||
expect(compile({ match: 'all', children: [{ field: 'plot', operator: 'notMatches', value: 'car' }] })).toBe('NOT plot:"car"');
|
||
});
|
||
|
||
it('emits numeric ranges', () => {
|
||
expect(compile({ match: 'all', children: [{ field: 'minutes', operator: 'gt', value: '30' }] })).toBe('minutes:{30 TO *}');
|
||
expect(compile({ match: 'all', children: [{ field: 'minutes', operator: 'lt', value: '90' }] })).toBe('minutes:{* TO 90}');
|
||
expect(compile({ match: 'all', children: [{ field: 'minutes', operator: 'between', value: '30', value2: '90' }] })).toBe('minutes:[30 TO 90]');
|
||
expect(compile({ match: 'all', children: [{ field: 'minutes', operator: 'eq', value: '42' }] })).toBe('minutes:42');
|
||
});
|
||
|
||
it('emits date ranges', () => {
|
||
expect(compile({ match: 'all', children: [{ field: 'release_date', operator: 'after', value: '2000-01-01' }] })).toBe('release_date:{2000-01-01 TO *}');
|
||
expect(compile({ match: 'all', children: [{ field: 'release_date', operator: 'before', value: '2010-01-01' }] })).toBe('release_date:{* TO 2010-01-01}');
|
||
});
|
||
|
||
it('joins by AND / OR and parenthesizes nested groups', () => {
|
||
const g: Group = {
|
||
match: 'all',
|
||
children: [
|
||
{ field: 'type', operator: 'is', value: 'movie' },
|
||
{ match: 'any', children: [
|
||
{ field: 'genre', operator: 'is', value: 'Horror' },
|
||
{ field: 'genre', operator: 'is', value: 'Thriller' }
|
||
] }
|
||
]
|
||
};
|
||
expect(compile(g)).toBe('type:"movie" AND (genre:"Horror" OR genre:"Thriller")');
|
||
});
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 3: Run it to verify it fails**
|
||
|
||
Run: `cd web && npx vitest run src/builder/rules/compile.test.ts`
|
||
Expected: FAIL — `compile` not found.
|
||
|
||
- [ ] **Step 4: Write `compile.ts`**
|
||
|
||
```ts
|
||
import { isGroup, type Group, type Rule } from './types';
|
||
|
||
// The compiler is the sole author of quoting. Quoted phrases escape only " and \.
|
||
function quote(value: string): string {
|
||
return `"${value.replace(/(["\\])/g, '\\$1')}"`;
|
||
}
|
||
|
||
// Wildcard/prefix values escape Lucene specials but keep the wildcard the operator adds.
|
||
function escapeWild(value: string): string {
|
||
return value.replace(/([+\-!(){}[\]^"~:\\/])/g, '\\$1');
|
||
}
|
||
|
||
function compileRule(rule: Rule): string {
|
||
const f = rule.field;
|
||
const v = rule.value;
|
||
switch (rule.operator) {
|
||
case 'is':
|
||
case 'matches': // fulltext: same quoted form as text `is`, disambiguated by field type on parse
|
||
return `${f}:${quote(v)}`;
|
||
case 'isNot':
|
||
case 'notMatches':
|
||
return `NOT ${f}:${quote(v)}`;
|
||
case 'contains':
|
||
return `${f}:*${escapeWild(v)}*`;
|
||
case 'startsWith':
|
||
return `${f}:${escapeWild(v)}*`;
|
||
case 'eq':
|
||
return `${f}:${v}`;
|
||
case 'gt':
|
||
case 'after':
|
||
return `${f}:{${v} TO *}`;
|
||
case 'lt':
|
||
case 'before':
|
||
return `${f}:{* TO ${v}}`;
|
||
case 'between':
|
||
return `${f}:[${v} TO ${rule.value2 ?? ''}]`;
|
||
default:
|
||
return '';
|
||
}
|
||
}
|
||
|
||
function compileGroup(group: Group): string {
|
||
const conn = group.match === 'all' ? ' AND ' : ' OR ';
|
||
return group.children
|
||
.map((child) => (isGroup(child) ? `(${compileGroup(child)})` : compileRule(child)))
|
||
.filter((s) => s.length > 0)
|
||
.join(conn);
|
||
}
|
||
|
||
export function compile(group: Group): string {
|
||
return compileGroup(group);
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 5: Run the test to verify it passes**
|
||
|
||
Run: `cd web && npx vitest run src/builder/rules/compile.test.ts`
|
||
Expected: PASS.
|
||
|
||
- [ ] **Step 6: Commit**
|
||
|
||
```bash
|
||
git add web/src/builder/rules/types.ts web/src/builder/rules/compile.ts web/src/builder/rules/compile.test.ts
|
||
git -c core.hooksPath=/dev/null commit -m "feat(176): rule model + Lucene-subset compiler"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 4: The parser (`parse.ts`) — exact inverse, null on out-of-subset
|
||
|
||
**Files:**
|
||
- Create: `web/src/builder/rules/parse.ts`
|
||
- Test: `web/src/builder/rules/parse.test.ts`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `types.ts`, the canonical forms from Task 3.
|
||
- Produces: `parse(input: string, fieldTypes: Record<string, FieldType>): Group | null`. Returns `null` for any string outside the closed subset (unknown field, mixed AND/OR at one level, nesting deeper than one level, unrecognized clause shape) — this drives the raw-text fallback. A single atom with no connective parses to `{ match: 'all', children: [rule] }`.
|
||
|
||
- [ ] **Step 1: Write the failing parser test**
|
||
|
||
`web/src/builder/rules/parse.test.ts`:
|
||
|
||
```ts
|
||
import { describe, expect, it } from 'vitest';
|
||
import { parse } from './parse';
|
||
import type { FieldType } from './types';
|
||
|
||
const FIELDS: Record<string, FieldType> = {
|
||
genre: 'text', title: 'text', plot: 'fulltext', type: 'enum', minutes: 'number', release_date: 'date'
|
||
};
|
||
|
||
describe('parse', () => {
|
||
it('parses a single quoted text atom as is', () => {
|
||
expect(parse('genre:"Horror"', FIELDS)).toEqual({ match: 'all', children: [{ field: 'genre', operator: 'is', value: 'Horror' }] });
|
||
});
|
||
|
||
it('parses NOT as isNot / notMatches by field type', () => {
|
||
expect(parse('NOT genre:"Horror"', FIELDS)).toEqual({ match: 'all', children: [{ field: 'genre', operator: 'isNot', value: 'Horror' }] });
|
||
expect(parse('NOT plot:"car"', FIELDS)).toEqual({ match: 'all', children: [{ field: 'plot', operator: 'notMatches', value: 'car' }] });
|
||
});
|
||
|
||
it('parses a quoted fulltext atom as matches', () => {
|
||
expect(parse('plot:"car"', FIELDS)).toEqual({ match: 'all', children: [{ field: 'plot', operator: 'matches', value: 'car' }] });
|
||
});
|
||
|
||
it('parses wildcard forms', () => {
|
||
expect(parse('title:*night*', FIELDS)).toEqual({ match: 'all', children: [{ field: 'title', operator: 'contains', value: 'night' }] });
|
||
expect(parse('title:The*', FIELDS)).toEqual({ match: 'all', children: [{ field: 'title', operator: 'startsWith', value: 'The' }] });
|
||
});
|
||
|
||
it('parses number/date ranges disambiguated by field type', () => {
|
||
expect(parse('minutes:{30 TO *}', FIELDS)).toEqual({ match: 'all', children: [{ field: 'minutes', operator: 'gt', value: '30' }] });
|
||
expect(parse('release_date:{* TO 2010-01-01}', FIELDS)).toEqual({ match: 'all', children: [{ field: 'release_date', operator: 'before', value: '2010-01-01' }] });
|
||
expect(parse('minutes:[30 TO 90]', FIELDS)).toEqual({ match: 'all', children: [{ field: 'minutes', operator: 'between', value: '30', value2: '90' }] });
|
||
expect(parse('minutes:42', FIELDS)).toEqual({ match: 'all', children: [{ field: 'minutes', operator: 'eq', value: '42' }] });
|
||
});
|
||
|
||
it('parses a nested group', () => {
|
||
expect(parse('type:"movie" AND (genre:"Horror" OR genre:"Thriller")', FIELDS)).toEqual({
|
||
match: 'all',
|
||
children: [
|
||
{ field: 'type', operator: 'is', value: 'movie' },
|
||
{ match: 'any', children: [
|
||
{ field: 'genre', operator: 'is', value: 'Horror' },
|
||
{ field: 'genre', operator: 'is', value: 'Thriller' }
|
||
] }
|
||
]
|
||
});
|
||
});
|
||
|
||
it('returns null on out-of-subset input', () => {
|
||
expect(parse('genre:"a" AND genre:"b" OR genre:"c"', FIELDS)).toBeNull(); // mixed AND/OR at one level
|
||
expect(parse('unknown_field:"x"', FIELDS)).toBeNull(); // unknown field
|
||
expect(parse('genre:"a" AND (type:"movie" AND (genre:"b"))', FIELDS)).toBeNull(); // two levels of nesting
|
||
expect(parse('title:jo~2', FIELDS)).toBeNull(); // fuzzy — not in subset
|
||
});
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 2: Run it to verify it fails**
|
||
|
||
Run: `cd web && npx vitest run src/builder/rules/parse.test.ts`
|
||
Expected: FAIL — `parse` not found.
|
||
|
||
- [ ] **Step 3: Write `parse.ts`**
|
||
|
||
```ts
|
||
import type { FieldType, Group, Operator, Rule } from './types';
|
||
|
||
// Split a group body into top-level parts separated by a single connective, respecting quotes and
|
||
// one level of parens. Returns the parts and the connective, or null if the split is malformed or the
|
||
// connectives are mixed (mixed AND/OR at one level is out of subset).
|
||
function splitTopLevel(input: string): { parts: string[]; match: 'all' | 'any' } | null {
|
||
const parts: string[] = [];
|
||
const ops: string[] = [];
|
||
let depth = 0;
|
||
let quoted = false;
|
||
let token = '';
|
||
|
||
for (let i = 0; i < input.length; i++) {
|
||
const ch = input[i];
|
||
if (quoted) {
|
||
token += ch;
|
||
if (ch === '"' && input[i - 1] !== '\\') quoted = false;
|
||
continue;
|
||
}
|
||
if (ch === '"') { quoted = true; token += ch; continue; }
|
||
if (ch === '(') { depth++; token += ch; continue; }
|
||
if (ch === ')') { depth--; token += ch; continue; }
|
||
if (depth === 0 && input.startsWith(' AND ', i)) { parts.push(token); ops.push('AND'); token = ''; i += 4; continue; }
|
||
if (depth === 0 && input.startsWith(' OR ', i)) { parts.push(token); ops.push('OR'); token = ''; i += 3; continue; }
|
||
token += ch;
|
||
}
|
||
if (depth !== 0 || quoted) return null;
|
||
parts.push(token);
|
||
|
||
if (ops.length === 0) return { parts, match: 'all' };
|
||
const allAnd = ops.every((o) => o === 'AND');
|
||
const allOr = ops.every((o) => o === 'OR');
|
||
if (!allAnd && !allOr) return null; // mixed connectives
|
||
return { parts, match: allAnd ? 'all' : 'any' };
|
||
}
|
||
|
||
function unquote(value: string): string {
|
||
return value.replace(/\\(["\\])/g, '$1');
|
||
}
|
||
|
||
function parseAtom(atom: string, fieldTypes: Record<string, FieldType>): Rule | null {
|
||
const trimmed = atom.trim();
|
||
const negate = trimmed.startsWith('NOT ');
|
||
const body = negate ? trimmed.slice(4).trim() : trimmed;
|
||
|
||
const colon = body.indexOf(':');
|
||
if (colon < 0) return null;
|
||
const field = body.slice(0, colon);
|
||
const raw = body.slice(colon + 1);
|
||
const type = fieldTypes[field];
|
||
if (!type) return null;
|
||
|
||
// Ranges: {a TO b} (exclusive) and [a TO b] (inclusive between)
|
||
const range = raw.match(/^([[{])(\S+) TO (\S+)([\]}])$/);
|
||
if (range) {
|
||
const [, , lo, hi, close] = range;
|
||
if (type !== 'number' && type !== 'date') return null;
|
||
if (close === ']') return mk(field, 'between', lo, hi);
|
||
if (lo === '*') return mk(field, type === 'date' ? 'before' : 'lt', hi);
|
||
if (hi === '*') return mk(field, type === 'date' ? 'after' : 'gt', lo);
|
||
return null;
|
||
}
|
||
|
||
// Quoted phrase → is/isNot (text, enum) or matches/notMatches (fulltext) by field type
|
||
if (raw.startsWith('"') && raw.endsWith('"') && raw.length >= 2) {
|
||
const v = unquote(raw.slice(1, -1));
|
||
if (type === 'fulltext') return mk(field, negate ? 'notMatches' : 'matches', v);
|
||
if (type === 'text' || type === 'enum') return mk(field, negate ? 'isNot' : 'is', v);
|
||
return null;
|
||
}
|
||
|
||
if (negate) return null; // NOT only valid with quoted forms above
|
||
|
||
// Wildcards (text only): *v* → contains, v* → startsWith
|
||
if (raw.startsWith('*') && raw.endsWith('*') && raw.length >= 2) {
|
||
if (type !== 'text') return null;
|
||
return mk(field, 'contains', unescapeWild(raw.slice(1, -1)));
|
||
}
|
||
if (/^[^*]+\*$/.test(raw)) {
|
||
if (type !== 'text') return null;
|
||
return mk(field, 'startsWith', unescapeWild(raw.slice(0, -1)));
|
||
}
|
||
|
||
// Bare token → numeric eq only (reject fuzzy ~, boosts ^, etc.)
|
||
if (type === 'number' && /^[0-9.]+$/.test(raw)) return mk(field, 'eq', raw);
|
||
|
||
return null;
|
||
}
|
||
|
||
function unescapeWild(value: string): string {
|
||
return value.replace(/\\([+\-!(){}[\]^"~:\\/])/g, '$1');
|
||
}
|
||
|
||
function mk(field: string, operator: Operator, value: string, value2?: string): Rule {
|
||
return value2 === undefined ? { field, operator, value } : { field, operator, value, value2 };
|
||
}
|
||
|
||
function parseGroup(input: string, fieldTypes: Record<string, FieldType>, allowNested: boolean): Group | null {
|
||
const split = splitTopLevel(input.trim());
|
||
if (!split) return null;
|
||
|
||
const children: Array<Rule | Group> = [];
|
||
for (const part of split.parts) {
|
||
const p = part.trim();
|
||
if (p.startsWith('(') && p.endsWith(')')) {
|
||
if (!allowNested) return null; // deeper than one level
|
||
const inner = parseGroup(p.slice(1, -1), fieldTypes, false);
|
||
if (!inner) return null;
|
||
children.push(inner);
|
||
} else {
|
||
const rule = parseAtom(p, fieldTypes);
|
||
if (!rule) return null;
|
||
children.push(rule);
|
||
}
|
||
}
|
||
if (children.length === 0) return null;
|
||
return { match: split.match, children };
|
||
}
|
||
|
||
export function parse(input: string, fieldTypes: Record<string, FieldType>): Group | null {
|
||
if (!input.trim()) return null;
|
||
return parseGroup(input, fieldTypes, true);
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Run the parser test to verify it passes**
|
||
|
||
Run: `cd web && npx vitest run src/builder/rules/parse.test.ts`
|
||
Expected: PASS. If the `startsWith` branch misbehaves, note the intent: a value that ends in a single trailing `*` and contains no other `*` is `startsWith`; the regex `/^[^*]+\*$/` captures exactly that.
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add web/src/builder/rules/parse.ts web/src/builder/rules/parse.test.ts
|
||
git -c core.hooksPath=/dev/null commit -m "feat(176): Lucene-subset parser (exact inverse of compiler)"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 5: Round-trip property test (the keystone)
|
||
|
||
**Files:**
|
||
- Test: `web/src/builder/rules/roundtrip.test.ts`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `compile`, `parse`, `types.ts`. No new production code — this test is what makes "compile-only, no stored AST" safe.
|
||
|
||
- [ ] **Step 1: Write the generator + property test**
|
||
|
||
`web/src/builder/rules/roundtrip.test.ts`. A deterministic pseudo-random generator (seeded LCG — no `Math.random`, so failures reproduce) builds rule trees within the closed subset; each must satisfy `parse(compile(tree)) deep-equals tree`. Groups always have ≥2 children (a 1-child group's `match` is meaningless and would not round-trip); string values are drawn from an alphabet that INCLUDES Lucene specials, spaces, quotes and backslashes — escaping is total (see the `escapeWild`/`quote` + tokenizer fix, commit `6591c6ef`), so every value must round-trip regardless of content. A generator restricted to `[A-Za-z0-9]` would pass blind to escaping bugs — do not narrow it.
|
||
|
||
```ts
|
||
import { describe, expect, it } from 'vitest';
|
||
import { compile } from './compile';
|
||
import { parse } from './parse';
|
||
import type { FieldType, Group, Operator, Rule } from './types';
|
||
|
||
const FIELDS: Record<string, FieldType> = {
|
||
genre: 'text', title: 'text', plot: 'fulltext', type: 'enum', minutes: 'number', release_date: 'date'
|
||
};
|
||
const BY_TYPE: Record<FieldType, string[]> = {
|
||
text: ['genre', 'title'], fulltext: ['plot'], enum: ['type'], number: ['minutes'], date: ['release_date']
|
||
};
|
||
const OPS: Record<FieldType, Operator[]> = {
|
||
text: ['is', 'isNot', 'contains', 'startsWith'],
|
||
fulltext: ['matches', 'notMatches'],
|
||
enum: ['is', 'isNot'],
|
||
number: ['eq', 'gt', 'lt', 'between'],
|
||
date: ['before', 'after', 'between']
|
||
};
|
||
|
||
// Deterministic LCG so a failing case is reproducible.
|
||
function lcg(seed: number) {
|
||
let s = seed >>> 0;
|
||
return () => {
|
||
s = (1664525 * s + 1013904223) >>> 0;
|
||
return s / 0xffffffff;
|
||
};
|
||
}
|
||
const pick = <T,>(rng: () => number, arr: T[]): T => arr[Math.floor(rng() * arr.length)];
|
||
// Include Lucene specials, spaces, quotes and backslashes: escaping must be TOTAL, so string
|
||
// values must round-trip regardless of content (this is what catches escaping bugs the pure
|
||
// alphanumeric generator would miss). Numbers/dates are generated separately in makeRule.
|
||
const VALUE_CHARS = 'abcdefghijABCDEFGHIJ0123456789 *?"\\():-';
|
||
const token = (rng: () => number): string => {
|
||
const n = 3 + Math.floor(rng() * 5);
|
||
let out = '';
|
||
for (let i = 0; i < n; i++) out += VALUE_CHARS[Math.floor(rng() * VALUE_CHARS.length)];
|
||
return out;
|
||
};
|
||
|
||
function makeRule(rng: () => number): Rule {
|
||
const type = pick(rng, Object.keys(BY_TYPE) as FieldType[]);
|
||
const field = pick(rng, BY_TYPE[type]);
|
||
const operator = pick(rng, OPS[type]);
|
||
if (operator === 'between') {
|
||
return type === 'date'
|
||
? { field, operator, value: '2000-01-01', value2: '2010-01-01' }
|
||
: { field, operator, value: '10', value2: '90' };
|
||
}
|
||
if (type === 'number') return { field, operator, value: String(1 + Math.floor(rng() * 500)) };
|
||
if (type === 'date') return { field, operator, value: '2005-06-15' };
|
||
return { field, operator, value: token(rng) };
|
||
}
|
||
|
||
function makeGroup(rng: () => number, allowNested: boolean): Group {
|
||
const match = rng() < 0.5 ? 'all' : 'any';
|
||
const count = 2 + Math.floor(rng() * 3); // >= 2 children
|
||
const children: Array<Rule | Group> = [];
|
||
for (let i = 0; i < count; i++) {
|
||
if (allowNested && rng() < 0.25) children.push(makeGroup(rng, false));
|
||
else children.push(makeRule(rng));
|
||
}
|
||
return { match, children };
|
||
}
|
||
|
||
describe('round-trip: parse(compile(tree)) === tree', () => {
|
||
it('holds over 500 generated trees', () => {
|
||
const rng = lcg(12345);
|
||
for (let i = 0; i < 500; i++) {
|
||
const tree = makeGroup(rng, true);
|
||
const text = compile(tree);
|
||
const back = parse(text, FIELDS);
|
||
expect(back, `seed-iter ${i} failed for: ${text}`).toEqual(tree);
|
||
}
|
||
});
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 2: Run it**
|
||
|
||
Run: `cd web && npx vitest run src/builder/rules/roundtrip.test.ts`
|
||
Expected: PASS. If a case fails, the assertion message prints the exact compiled string — fix `compile`/`parse` to be exact inverses for that shape (do NOT weaken the generator to hide it, unless the shape is genuinely out-of-subset, in which case document why).
|
||
|
||
- [ ] **Step 3: Commit**
|
||
|
||
```bash
|
||
git add web/src/builder/rules/roundtrip.test.ts
|
||
git -c core.hooksPath=/dev/null commit -m "test(176): compile/parse round-trip property test"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 6: Field catalog client (`fieldCatalog.ts` + api)
|
||
|
||
**Files:**
|
||
- Modify: `web/src/api/search.ts`
|
||
- Create: `web/src/builder/rules/fieldCatalog.ts`
|
||
- Test: `web/src/api/search.test.ts` (add a case), `web/src/builder/rules/fieldCatalog.test.ts`
|
||
|
||
**Interfaces:**
|
||
- Produces: `getSearchFields(): Promise<SearchField[]>` and `type SearchField`; `useSearchFields()` → `{ fields, fieldTypes, byGroup, status }`.
|
||
|
||
- [ ] **Step 1: Add `getSearchFields` to `web/src/api/search.ts`**
|
||
|
||
Add near the top (after the existing type exports):
|
||
|
||
```ts
|
||
export type SearchField = components['schemas']['SearchFieldResponseModel'];
|
||
|
||
export function getSearchFields(): Promise<SearchField[]> {
|
||
return request<SearchField[]>('/api/v1/search/fields');
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Add an api test (`web/src/api/search.test.ts`)**
|
||
|
||
Follow the existing tests in that file (they mock `request`/`fetch`). Add:
|
||
|
||
```ts
|
||
it('getSearchFields calls the catalog endpoint', async () => {
|
||
// Match the mocking style already used in this file (fetch or ./client mock).
|
||
const result = await getSearchFields();
|
||
expect(Array.isArray(result)).toBe(true);
|
||
});
|
||
```
|
||
|
||
Adapt to the file's actual mock harness — read the top of `search.test.ts` first and mirror it (do not invent a new mocking approach).
|
||
|
||
- [ ] **Step 3: Write `fieldCatalog.ts` (hook)**
|
||
|
||
The generated `SearchField` type is **all-nullable** (`ErsatzTV.Core` compiles with `<Nullable>disable</Nullable>`, so every response-model string is `string | null` in `v1.d.ts`). To keep that null-noise out of the UI (Tasks 7/8), the hook coerces to a clean **`RuleField`** (all non-null) — drop entries with no `name`, default `type→'text'`, `group→'Other'`, `label→name`, `values→[]`.
|
||
|
||
```ts
|
||
import { useEffect, useState } from 'react';
|
||
import { getSearchFields, type SearchField } from '../../api/search';
|
||
import type { FieldType } from './types';
|
||
|
||
// Clean, non-null field the builder UI consumes (SearchField from the generated client is all-nullable).
|
||
export interface RuleField {
|
||
name: string;
|
||
label: string;
|
||
type: FieldType;
|
||
group: string;
|
||
values: string[];
|
||
}
|
||
|
||
export interface FieldCatalog {
|
||
fields: RuleField[];
|
||
fieldTypes: Record<string, FieldType>;
|
||
byGroup: Array<{ group: string; fields: RuleField[] }>;
|
||
status: 'loading' | 'success' | 'error';
|
||
}
|
||
|
||
function toRuleField(f: SearchField): RuleField | null {
|
||
if (!f.name) return null;
|
||
return {
|
||
name: f.name,
|
||
label: f.label ?? f.name,
|
||
type: (f.type as FieldType | null) ?? 'text',
|
||
group: f.group ?? 'Other',
|
||
values: f.values ?? []
|
||
};
|
||
}
|
||
|
||
export function useSearchFields(): FieldCatalog {
|
||
const [raw, setRaw] = useState<SearchField[]>([]);
|
||
const [status, setStatus] = useState<'loading' | 'success' | 'error'>('loading');
|
||
|
||
useEffect(() => {
|
||
let active = true;
|
||
getSearchFields()
|
||
.then((f) => { if (active) { setRaw(f); setStatus('success'); } })
|
||
.catch(() => { if (active) setStatus('error'); });
|
||
return () => { active = false; };
|
||
}, []);
|
||
|
||
const fields = raw.map(toRuleField).filter((f): f is RuleField => f !== null);
|
||
|
||
const fieldTypes: Record<string, FieldType> = {};
|
||
for (const f of fields) fieldTypes[f.name] = f.type;
|
||
|
||
const groups = new Map<string, RuleField[]>();
|
||
for (const f of fields) {
|
||
const list = groups.get(f.group) ?? [];
|
||
list.push(f);
|
||
groups.set(f.group, list);
|
||
}
|
||
const byGroup = [...groups.entries()].map(([group, gf]) => ({ group, fields: gf }));
|
||
|
||
return { fields, fieldTypes, byGroup, status };
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Run tests + typecheck**
|
||
|
||
Run: `cd web && npx vitest run src/api/search.test.ts && npx tsc -b --pretty false`
|
||
Expected: PASS, no type errors.
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add web/src/api/search.ts web/src/api/search.test.ts web/src/builder/rules/fieldCatalog.ts
|
||
git -c core.hooksPath=/dev/null commit -m "feat(176): field-catalog api + useSearchFields hook"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 7: `RuleBuilder.tsx` component
|
||
|
||
**Files:**
|
||
- Create: `web/src/builder/rules/RuleBuilder.tsx`
|
||
- Test: `web/src/builder/rules/RuleBuilder.test.tsx`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `types.ts` (`Group`, `Rule`, `Operator`, `OPERATORS_BY_TYPE`), `RuleField` (from `./fieldCatalog` — clean, non-null).
|
||
- Produces: `RuleBuilder({ value, onChange, fields }: { value: Group; onChange: (g: Group) => void; fields: RuleField[] })`. Renders the top group's `Match: All | Any` toggle, a row per child (`field ▾` → `operator ▾` → value input by type; `between` shows two inputs; `enum` shows a value dropdown), `+ Add rule` / `+ Add group` (one nested level), and remove buttons. Purely controlled — every edit calls `onChange` with the next `Group`.
|
||
|
||
- [ ] **Step 1: Write the component**
|
||
|
||
Use the existing SPA primitives (`../../components` exports `Button`, `IconButton`, `Input`, `Switch`; native `<select>` is fine — check `web/src/components` for a `Select` and prefer it if present). Keep it controlled and small.
|
||
|
||
```tsx
|
||
import { Plus, Trash2 } from 'lucide-react';
|
||
import { Button, IconButton, Input } from '../../components';
|
||
import type { RuleField } from './fieldCatalog';
|
||
import { isGroup, OPERATORS_BY_TYPE, type FieldType, type Group, type Operator, type Rule } from './types';
|
||
|
||
const OP_LABEL: Record<Operator, string> = {
|
||
is: 'is', isNot: 'is not', contains: 'contains', startsWith: 'starts with',
|
||
matches: 'contains', notMatches: 'does not contain',
|
||
eq: '=', gt: '>', lt: '<', between: 'between', before: 'before', after: 'after'
|
||
};
|
||
|
||
function typeOf(fields: RuleField[], name: string): FieldType {
|
||
return (fields.find((f) => f.name === name)?.type as FieldType) ?? 'text';
|
||
}
|
||
|
||
function defaultRule(fields: RuleField[]): Rule {
|
||
const field = fields[0]?.name ?? 'title';
|
||
const type = typeOf(fields, field);
|
||
return { field, operator: OPERATORS_BY_TYPE[type][0], value: '' };
|
||
}
|
||
|
||
function RuleRow({ rule, fields, onChange, onRemove }: {
|
||
rule: Rule; fields: RuleField[]; onChange: (r: Rule) => void; onRemove: () => void;
|
||
}) {
|
||
const type = typeOf(fields, rule.field);
|
||
const ops = OPERATORS_BY_TYPE[type];
|
||
const enumField = fields.find((f) => f.name === rule.field && f.type === 'enum');
|
||
|
||
return (
|
||
<div style={{ display: 'flex', gap: 8, alignItems: 'center', marginBottom: 8 }}>
|
||
<select
|
||
aria-label="Field"
|
||
value={rule.field}
|
||
onChange={(e) => {
|
||
const nextType = typeOf(fields, e.target.value);
|
||
onChange({ field: e.target.value, operator: OPERATORS_BY_TYPE[nextType][0], value: '' });
|
||
}}
|
||
>
|
||
{fields.map((f) => <option key={f.name} value={f.name}>{f.label}</option>)}
|
||
</select>
|
||
|
||
<select aria-label="Operator" value={rule.operator} onChange={(e) => onChange({ ...rule, operator: e.target.value as Operator })}>
|
||
{ops.map((op) => <option key={op} value={op}>{OP_LABEL[op]}</option>)}
|
||
</select>
|
||
|
||
{enumField ? (
|
||
<select aria-label="Value" value={rule.value} onChange={(e) => onChange({ ...rule, value: e.target.value })}>
|
||
<option value="">—</option>
|
||
{enumField.values.map((v) => <option key={v} value={v}>{v}</option>)}
|
||
</select>
|
||
) : (
|
||
<Input aria-label="Value" value={rule.value} onChange={(e) => onChange({ ...rule, value: e.target.value })}
|
||
type={type === 'number' ? 'number' : type === 'date' ? 'date' : 'text'} />
|
||
)}
|
||
|
||
{rule.operator === 'between' && (
|
||
<Input aria-label="Upper bound" value={rule.value2 ?? ''} onChange={(e) => onChange({ ...rule, value2: e.target.value })}
|
||
type={type === 'number' ? 'number' : 'date'} />
|
||
)}
|
||
|
||
<IconButton aria-label="Remove rule" onClick={onRemove}><Trash2 size={14} /></IconButton>
|
||
</div>
|
||
);
|
||
}
|
||
|
||
function MatchToggle({ match, onChange }: { match: 'all' | 'any'; onChange: (m: 'all' | 'any') => void }) {
|
||
return (
|
||
<div style={{ display: 'flex', gap: 6, alignItems: 'center', marginBottom: 8 }}>
|
||
<span>Match</span>
|
||
<select aria-label="Match" value={match} onChange={(e) => onChange(e.target.value as 'all' | 'any')}>
|
||
<option value="all">All</option>
|
||
<option value="any">Any</option>
|
||
</select>
|
||
<span>of the following:</span>
|
||
</div>
|
||
);
|
||
}
|
||
|
||
function GroupEditor({ group, fields, depth, onChange, onRemove }: {
|
||
group: Group; fields: RuleField[]; depth: number; onChange: (g: Group) => void; onRemove?: () => void;
|
||
}) {
|
||
const setChild = (i: number, child: Rule | Group) => {
|
||
const children = group.children.slice();
|
||
children[i] = child;
|
||
onChange({ ...group, children });
|
||
};
|
||
const removeChild = (i: number) => onChange({ ...group, children: group.children.filter((_, j) => j !== i) });
|
||
|
||
return (
|
||
<div style={{ border: depth > 0 ? '1px solid var(--ctv-border, #333)' : 'none', borderRadius: 8, padding: depth > 0 ? 12 : 0, marginBottom: 8 }}>
|
||
<MatchToggle match={group.match} onChange={(m) => onChange({ ...group, match: m })} />
|
||
{group.children.map((child, i) =>
|
||
isGroup(child)
|
||
? <GroupEditor key={i} group={child} fields={fields} depth={depth + 1} onChange={(g) => setChild(i, g)} onRemove={() => removeChild(i)} />
|
||
: <RuleRow key={i} rule={child} fields={fields} onChange={(r) => setChild(i, r)} onRemove={() => removeChild(i)} />
|
||
)}
|
||
<div style={{ display: 'flex', gap: 8 }}>
|
||
<Button size="sm" variant="secondary" startIcon={<Plus size={14} />}
|
||
onClick={() => onChange({ ...group, children: [...group.children, defaultRule(fields)] })}>Add rule</Button>
|
||
{depth === 0 && (
|
||
<Button size="sm" variant="secondary" startIcon={<Plus size={14} />}
|
||
onClick={() => onChange({ ...group, children: [...group.children, { match: 'any', children: [defaultRule(fields)] }] })}>Add group</Button>
|
||
)}
|
||
{onRemove && <IconButton aria-label="Remove group" onClick={onRemove}><Trash2 size={14} /></IconButton>}
|
||
</div>
|
||
</div>
|
||
);
|
||
}
|
||
|
||
export function RuleBuilder({ value, onChange, fields }: { value: Group; onChange: (g: Group) => void; fields: RuleField[] }) {
|
||
return <GroupEditor group={value} fields={fields} depth={0} onChange={onChange} />;
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Write the component test**
|
||
|
||
`web/src/builder/rules/RuleBuilder.test.tsx`:
|
||
|
||
```tsx
|
||
import { render, screen, fireEvent } from '@testing-library/react';
|
||
import { describe, expect, it, vi } from 'vitest';
|
||
import { RuleBuilder } from './RuleBuilder';
|
||
import type { RuleField } from './fieldCatalog';
|
||
import type { Group } from './types';
|
||
|
||
const FIELDS: RuleField[] = [
|
||
{ name: 'genre', label: 'Genre', type: 'text', group: 'General', values: [] },
|
||
{ name: 'type', label: 'Item type', type: 'enum', group: 'General', values: ['movie', 'episode'] }
|
||
];
|
||
|
||
function setup(initial: Group) {
|
||
const onChange = vi.fn();
|
||
const utils = render(<RuleBuilder value={initial} onChange={onChange} fields={FIELDS} />);
|
||
return { onChange, ...utils };
|
||
}
|
||
|
||
describe('RuleBuilder', () => {
|
||
it('adds a rule', () => {
|
||
const { onChange } = setup({ match: 'all', children: [] });
|
||
fireEvent.click(screen.getByText('Add rule'));
|
||
expect(onChange).toHaveBeenCalledWith(expect.objectContaining({ match: 'all', children: expect.arrayContaining([expect.objectContaining({ field: 'genre' })]) }));
|
||
});
|
||
|
||
it('removes a rule', () => {
|
||
const { onChange } = setup({ match: 'all', children: [{ field: 'genre', operator: 'is', value: 'Horror' }] });
|
||
fireEvent.click(screen.getByLabelText('Remove rule'));
|
||
expect(onChange).toHaveBeenCalledWith({ match: 'all', children: [] });
|
||
});
|
||
|
||
it('adds a nested group only at top level', () => {
|
||
const { onChange } = setup({ match: 'all', children: [] });
|
||
fireEvent.click(screen.getByText('Add group'));
|
||
expect(onChange).toHaveBeenCalledWith(expect.objectContaining({ children: expect.arrayContaining([expect.objectContaining({ match: 'any' })]) }));
|
||
});
|
||
|
||
it('shows enum values as a dropdown', () => {
|
||
setup({ match: 'all', children: [{ field: 'type', operator: 'is', value: 'movie' }] });
|
||
expect(screen.getByRole('option', { name: 'episode' })).toBeInTheDocument();
|
||
});
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 3: Run the tests + typecheck**
|
||
|
||
Run: `cd web && npx vitest run src/builder/rules/RuleBuilder.test.tsx && npx tsc -b --pretty false`
|
||
Expected: PASS. (If `IconButton`/`Button`/`Input` prop names differ, read `web/src/components/index.ts` and adjust — mirror how `CollectionsScreen.tsx` uses them.)
|
||
|
||
- [ ] **Step 4: Commit**
|
||
|
||
```bash
|
||
git add web/src/builder/rules/RuleBuilder.tsx web/src/builder/rules/RuleBuilder.test.tsx
|
||
git -c core.hooksPath=/dev/null commit -m "feat(176): RuleBuilder component"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 8: Integrate into `SmartDialog` (Builder | Advanced toggle)
|
||
|
||
**Files:**
|
||
- Modify: `web/src/screens/CollectionsScreen.tsx` (the `SmartDialog` function, ~lines 204-300)
|
||
- Test: extend `web/src/screens/CollectionsScreen.test.tsx` if it exists; otherwise add a focused `SmartDialog` test file.
|
||
|
||
**Interfaces:**
|
||
- Consumes: `RuleBuilder`, `useSearchFields`, `compile`, `parse`. No change to `SmartDialog`'s props or `onSubmit` — `query` (the compiled/raw Lucene string) remains the single value submitted.
|
||
|
||
- [ ] **Step 1: Add imports at the top of `CollectionsScreen.tsx`**
|
||
|
||
```ts
|
||
import { RuleBuilder } from '../builder/rules/RuleBuilder';
|
||
import { useSearchFields } from '../builder/rules/fieldCatalog';
|
||
import { compile } from '../builder/rules/compile';
|
||
import { parse } from '../builder/rules/parse';
|
||
import type { Group } from '../builder/rules/types';
|
||
```
|
||
|
||
- [ ] **Step 2: Add builder state in `SmartDialog` and derive the initial mode from the seeded query**
|
||
|
||
Inside `SmartDialog`, after the existing `query` state (around line 221), add:
|
||
|
||
```ts
|
||
const { fields, fieldTypes, status: fieldsStatus } = useSearchFields();
|
||
const seededGroup = parse(initial?.query ?? '', fieldTypes);
|
||
const [mode, setMode] = useState<'builder' | 'advanced'>(() =>
|
||
initial?.query && !parse(initial.query, fieldTypes) ? 'advanced' : 'builder'
|
||
);
|
||
const [group, setGroup] = useState<Group>(seededGroup ?? { match: 'all', children: [] });
|
||
```
|
||
|
||
Note: `fieldTypes` is empty on first render (catalog still loading), so `parse` returns `null` and the dialog opens in `advanced` until the catalog arrives. To seed the builder once fields load, add:
|
||
|
||
```ts
|
||
useEffect(() => {
|
||
if (fieldsStatus !== 'success' || !initial?.query) return;
|
||
const parsed = parse(initial.query, fieldTypes);
|
||
if (parsed) { setGroup(parsed); setQuery(compile(parsed)); setMode('builder'); } else { setMode('advanced'); }
|
||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||
}, [fieldsStatus]);
|
||
```
|
||
|
||
- [ ] **Step 3: Keep `query` in sync when in builder mode**
|
||
|
||
Add an effect so the compiled string is what gets submitted/previewed:
|
||
|
||
```ts
|
||
useEffect(() => {
|
||
if (mode === 'builder') setQuery(compile(group));
|
||
}, [mode, group]);
|
||
```
|
||
|
||
- [ ] **Step 4: Replace the raw `Input` (lines ~272-280) with the mode toggle + conditional body**
|
||
|
||
```tsx
|
||
<div style={{ display: 'flex', gap: 8, marginTop: 12, marginBottom: 8 }}>
|
||
<Button size="sm" variant={mode === 'builder' ? 'primary' : 'secondary'}
|
||
disabled={query.trim().length > 0 && parse(query, fieldTypes) === null}
|
||
onClick={() => { const p = parse(query, fieldTypes); if (p) { setGroup(p); setQuery(compile(p)); setMode('builder'); } }}>Builder</Button>
|
||
<Button size="sm" variant={mode === 'advanced' ? 'primary' : 'secondary'}
|
||
onClick={() => { setQuery(compile(group)); setMode('advanced'); }}>Advanced</Button>
|
||
</div>
|
||
|
||
{mode === 'builder' ? (
|
||
fieldsStatus === 'success'
|
||
? <RuleBuilder value={group} onChange={setGroup} fields={fields} />
|
||
: <Spinner />
|
||
) : (
|
||
<div>
|
||
<Input label="Search query" onChange={(event) => setQuery(event.target.value)}
|
||
placeholder='e.g. genre:"action" AND type:"movie"'
|
||
style={{ fontFamily: 'var(--font-mono)' }} value={query} />
|
||
{parse(query, fieldTypes) === null && query.trim().length > 0 && (
|
||
<span className="ctv-field-hint">This query is too advanced to show in the builder.</span>
|
||
)}
|
||
</div>
|
||
)}
|
||
```
|
||
|
||
- [ ] **Step 5: Add a focused test**
|
||
|
||
Read `CollectionsScreen.test.tsx` (if present) for the existing harness; otherwise create `web/src/screens/CollectionsScreen.smartDialog.test.tsx` that mocks `../builder/rules/fieldCatalog` (`useSearchFields` → success with a small catalog) and `../api` (so `getLibraryBrowseItems` is stubbed). Assert:
|
||
|
||
```tsx
|
||
it('opens an existing subset query in builder mode', () => {
|
||
// render the CollectionsScreen with a smart collection whose query is 'genre:"Horror"'
|
||
// then: expect(screen.getByLabelText('Field')).toHaveValue('genre');
|
||
});
|
||
|
||
it('falls back to advanced mode for an out-of-subset query', () => {
|
||
// query 'genre:jo~2' → expect the raw Search query input to be visible
|
||
});
|
||
```
|
||
|
||
Fill these in against the real screen harness (the exact render/props mirror the existing CollectionsScreen tests). If wiring a full-screen render is heavy, extract `SmartDialog` is **not** required — test via the screen as the existing tests do.
|
||
|
||
- [ ] **Step 6: Run web tests, typecheck, lint**
|
||
|
||
Run: `cd web && npx vitest run && npx tsc -b --pretty false && npm run lint`
|
||
Expected: all PASS. Give any heavy-render test an explicit vitest timeout (e.g. `it('…', { timeout: 15000 }, …)`) — the CI VM is slower than local.
|
||
|
||
- [ ] **Step 7: Commit**
|
||
|
||
```bash
|
||
git add web/src/screens/CollectionsScreen.tsx web/src/screens/CollectionsScreen*.test.tsx
|
||
git -c core.hooksPath=/dev/null commit -m "feat(176): Builder|Advanced toggle in the SmartCollection dialog"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 9: Docs, full verification, live smoke, PR
|
||
|
||
**Files:**
|
||
- Modify: `docs/decisions.md`, `docs/spa-conventions.md`
|
||
|
||
- [ ] **Step 1: Append the decision to `docs/decisions.md`**
|
||
|
||
Add a dated entry (append-only) recording: the compile-only / closed-subset / **no-stored-AST** model for the SmartCollection rule builder (why: even an authoritative AST would still need a Lucene→rules parser for pre-existing free-text queries, so persisting an AST buys almost nothing while costing a dual-provider migration); the **one-level-nesting Kodi model**; and the read-only `GET /api/v1/search/fields` catalog as the field source of truth. Note the deferred follow-ups.
|
||
|
||
- [ ] **Step 2: Document the reusable component pattern in `docs/spa-conventions.md`**
|
||
|
||
Add a short subsection: the `web/src/builder/rules/` module (types + compile/parse + `useSearchFields` + `RuleBuilder`) is a reusable, controlled component that compiles to the Lucene query string; screens embed it and own the `query` string. Note it is intended for later reuse by ChannelBuilder / Auto-Tune.
|
||
|
||
- [ ] **Step 3: Full local verification**
|
||
|
||
Run:
|
||
```bash
|
||
cd /Users/timothy/ersatztv/.claude/worktrees/176-rule-builder
|
||
dotnet build ErsatzTV.sln 2>&1 | grep -E "error|Build succeeded"
|
||
dotnet test ErsatzTV.Tests --filter GetSearchFieldCatalogHandlerTests
|
||
cd web && npx vitest run && npx tsc -b --pretty false && npm run lint && npm run check:api
|
||
```
|
||
Expected: `Build succeeded`, all tests PASS, `check:api` clean (no uncommitted generated diff).
|
||
|
||
- [ ] **Step 4: Live smoke (Playwright, headless)**
|
||
|
||
Per `docs/e2e-local.md`, bring up a local instance with a seeded library, then drive: open Collections → Smart tab → New → Builder mode → add `genre is Horror` → confirm the live preview count updates → Save → reopen → confirm it opens in Builder with the rule populated. (Live-E2E is **not gated** here — read-only endpoint, unchanged write path — but this smoke is cheap insurance.) Never open download endpoints in a browser tab.
|
||
|
||
- [ ] **Step 5: BOM-check, push, open PR**
|
||
|
||
```bash
|
||
for f in $(git diff --name-only origin/main...HEAD -- '*.cs'); do head -c3 "$f" | xxd -p | grep -q "^efbbbf" && echo "BOM: $f"; done
|
||
# rebase on origin/main if it moved: git fetch origin main && git rebase origin/main (regenerate v1.json/v1.d.ts/endpoint-index if they conflict)
|
||
git push -u origin feat/176-smartcollection-rule-builder
|
||
```
|
||
Open a PR with `fixes #176` (or `part of #176` if you split the deferred follow-ups). Arm a CI monitor on the head sha at PR-open.
|
||
|
||
- [ ] **Step 6: Cold-context adversarial review + follow-up issues**
|
||
|
||
Dispatch a cold, review-only agent (cross-model if available) scoped to the diff — focus on the parser/compiler round-trip correctness and the escape/quote handling. Post a `Review-verdict: <verdict> @ <head-sha>` PR comment. File the deferred follow-ups as separate issues: facet typeahead, relative dates, deeper nesting, ChannelBuilder/Auto-Tune inline adoption. Run the H12 qualification audit and label anything new.
|
||
|
||
---
|
||
|
||
## Self-Review (checked against the spec)
|
||
|
||
**Spec coverage:** §3 rule model → Task 3 (`types.ts`) + Tasks 7-8; §3 closed subset compile → Task 3; parse/round-trip → Tasks 4-5; §4 catalog endpoint → Tasks 1-2; §5 UI integration + mode switching + open-existing behavior → Task 8; §6 testing (property, compile/parse, component, backend NUnit, live smoke, cold review) → Tasks 1,3,4,5,7,8,9; §7 docs → Tasks 2,9; §8 deferred follow-ups → Task 9 Step 6. No spec section is unmapped.
|
||
|
||
**Type consistency:** `Group`/`Rule`/`Operator`/`Match`/`FieldType`/`isGroup` defined in Task 3 and consumed unchanged in Tasks 4-8; `compile(group)`/`parse(input, fieldTypes)` signatures identical everywhere; `SearchFieldResponseModel(Name,Label,Type,Group,Values)` (C#) ↔ `SearchField` (TS generated) fields align; `useSearchFields()` returns `{ fields, fieldTypes, byGroup, status }` and every consumer uses those names.
|
||
|
||
**Placeholder scan:** the two intentionally implementer-adapted spots (the exact `*.Tests` project name in Task 1 Step 1; mirroring the existing api/screen test harness in Tasks 6 & 8) are gated by an explicit "read the sibling file first" instruction, not vague TODOs. The `type` enum token list carries an explicit VERIFY-against-`LuceneSearchIndex` step backed by a test asserting non-emptiness.
|