Cross-family review of 1b4dd6d6 found that utf8mb4_bin - chosen to keep the
dedupe case-exact - is a PAD SPACE collation, so trailing spaces are
insignificant under it. Verified on MySQL 8.4: '/media/Foo' = '/media/Foo ' is
TRUE, while case correctly compares unequal. Two distinct legal directories
therefore grouped together and the second was DELETED irreversibly, even though
PathUtils.GetPathHash hashes them differently and the unique index about to be
created would have accepted both. The dedupe destroyed data the constraint
never required it to destroy.
Group and join on CONVERT(Path USING binary) instead - NO PAD and byte-exact,
matching the hash. utf8mb4_0900_bin is also NO PAD but carries a server-version
floor. This is the only path comparison in either migration (every other
predicate keys off an integer id), so there is no mix of padded and unpadded
comparisons across the keeper-selection, repoint and delete steps.
SQLite's = on TEXT is byte-exact with no padding, so that migration was already
correct - which is exactly why a SQLite-only test could not see the divergence.
The two providers are now semantically equivalent, and the dedupe fixture is
shared: same rows, same expected survivors (1,4,5,6,7,9,10), asserted by the
SQLite test and reproduced by hand on MySQL 8.4.
Runtime was never affected, and this is now stated and tested rather than
assumed: GetFolder's SQL equality is a superset narrowing (both collation quirks
make it more permissive, never less, so it cannot miss a byte-exact match) and
ResolveExact settles identity with StringComparison.Ordinal, which compares
length first. Added ResolveExact coverage for the trailing-space axis.
Refs #488#308fix#491
Final low-severity items from the re-review of ee10f932.
L2: DbUpdateConcurrencyException derives from DbUpdateException but carries no
provider exception, so IsUniqueConstraintViolation does not classify it. A row
deleted by a concurrent library edit between the heal's read and its save would
propagate and fail the scan, contradicting the invariant stated directly above
it. Admit it in the filter.
L1: lift the in-memory ordinal settle into LibraryRepository.ResolveExact and
unit-test it with both spellings in the candidate list. No SQLite-backed test
can exercise it (SQLite's = on TEXT is already binary), so this converts the
half that rested on hand-run MySQL evidence into automated coverage. The
end-to-end companion test's comment no longer claims to be provider-independent.
L4: assert PRAGMA foreign_keys is 1 before migrating, so the enforcement guard
cannot silently degrade into the weak pre-fix form it was added to replace.
L3: detach the failed heal, matching the insert path.
N3: the heal's inner predicate now matches its IsNullOrEmpty outer guard, so a
PathHash = '' row cannot enter the branch and silently never heal.
N4: record that GetFolder returning null for a case-differing spelling makes
MySQL insert a second row where it used to reuse one — correct, and now matching
SQLite, but a real behaviour change on a case-insensitive filesystem.
Refs #488#308fix#491
Review of 491f5099 found the fix inert in the only process that runs it, plus
a MySQL collation defect in the lookup.
B1 — TvContext.IsUniqueConstraintViolation was assigned only in ErsatzTV/
Startup.cs, but ErsatzTV.Scanner is a separate executable and every production
caller of GetOrAddFolder/SetEtag lives there. The classifier kept its '_ =>
false' default, so the catch never ran and the DbUpdateException failed the
whole scan - worse than the duplicate row it replaced. Wire both provider
branches in ErsatzTV.Scanner/Program.cs, and add ProviderStaticsWiringTests
(architecture) asserting the scanner assigns every TvContext static the host
assigns, with IsSqlite documented as the one exemption.
H1 — GetFolder's 'Path == folder' is case-insensitive on MySQL while PathHash
is case-sensitive, and FirstOrDefault was unordered: a scan of '/x/foo' could
resolve the '/x/Foo' row and stamp the wrong hash onto it (verified on MySQL
8.4: the WHERE matches both, LIMIT 1 returns the wrong one). Treat the SQL
equality as a narrowing filter, order by Id, and settle identity ordinally.
Route the heal through EF and drop a classified violation, so an opportunistic
maintenance write can never abort a scan.
Also: run the dedupe migration test with foreign keys ON (matching prod), clear
the keeper's etag, null out a self-parent, and document the cleanup's limits
(NULL paths excluded, Down does not restore deleted rows, CI's fresh-DB apply
covers none of the data mutation).
Refs #488#308fix#491
GetOrAddFolder was a check-then-insert with no unique constraint behind it,
so two callers racing the same folder could both miss the lookup and both
insert. Enforce identity in the schema and make the loser adopt the winner.
- LibraryFolder gains a SHA-256 PathHash (the MediaFile.Path/PathHash
precedent): Path is MySQL longtext, which cannot be indexed without a
prefix length and collates case-insensitively, so the unique index is on
(LibraryPathId, PathHash) instead.
- GetOrAddFolder and SetEtag catch a classified unique violation via the
existing TvContext.IsUniqueConstraintViolation seam (#308) and re-read.
- Dual-provider migration audits and collapses pre-existing duplicates
(repointing MediaFile, ParentId and ImageFolderDuration) before creating
the index; legacy rows keep a null hash and heal on the next scan.
- Tests: deterministic cross-connection race, 8x10 barrier stress with an
insert-attempt vacuity guard, classifier-inversion negative control, and
a real-migration dedupe test.
Refs #488#308fix#491
timothy