Issue #2029 direct-write lock window oracle review

Reviewed source head: the commit that updates this record on lock-window-validate-at-apply. The production change is unchanged from 6cf681fb4. The oracle file is as of b6747be44.

Contract and evidence

In a persisted Query Collection, each source commit waits for the persistence wrapper's apply lock. While another task holds the lock, a committed refetch waits there, and core has not accepted it yet. A direct write (writeInsert, writeUpdate, writeDelete, writeUpsert) in that window must be validated against, and apply on top of, the rows of every earlier commit. A refetch that returns while the write waits follows the merge rule from #2030: a fetch that started before the write was called keeps the write's rows on the write's keys, and a fetch that started after the write wins.

The authority is #2029, the direct-write contract in docs/collections/query-collection.md, and the merge rule in packages/query-db-collection/src/query.ts. The reference is the same Query Collection without persistence, which has no lock.

The oracle is packages/query-db-collection/tests/persisted-direct-write-window.oracle.test.ts. It has five parts:

A 16-case matrix: 4 write types, 2 lock holders (a held durable write of an earlier refetch, and startup hydration), and 2 key cases (only the waiting refetch holds k, or both the applied rows and the refetch hold it). It compares the write's outcome, the visible rows and the Query cache with the reference, and the stored rows with the visible rows.
A 4-case ordering matrix: a third refetch starts before or after the write is called, and changes the same key or a different key. It returns while the write waits.
A witness for a persisted writeInsert of an existing key, with no lock window.
A real-SQLite subset: each write type with the key that only the waiting refetch holds and an earlier refetch's durable write holding the lock, plus the four ordering cases. It runs the same drivers over the node SQLite adapter for sync-present collections, with the same gate on its applyCommittedTx.
A calibration test for the harness's failure report.

Parts 1 to 3 use an in-memory fake adapter. The fake also seeds stored rows, which the startup holder needs.

RED and GREEN

Check	main 482196ec4	Branch
16-case matrix	10 fail. Example: writeUpdate of k rejects with UpdateOperationItemNotFoundError; the reference returns ok.	16 pass
Ordering matrix	Not run on main, which has no wait. On the branch before the ordering fix, the two after cases failed: k was 9, where the reference gave 6 and 5.	4 pass
Duplicate insert	Fails: expected 'ok' to be 'CollectionOperationError'	Passes
Real-SQLite subset (8 cases)	5 fail: insert, update and delete of k, and both ordering cases where the third refetch started before the write. Example: sqlite {"outcome":"UpdateOperationItemNotFoundError",…} != reference {"outcome":"ok",…}. Upsert and the two after cases pass, because main applies the write at once.	8 pass

The oracle file passes 30 tests, and its type check reports no errors. The Query Collection package passes 972 of 972 tests with type-checking on, measured before the real-SQLite subset was added.

Mutant results

Each mutant was applied to the reviewed source and reverted afterwards.

Mutant	Outcome
A direct write does not wait for earlier commits	Assertion failure, 9 of 16 matrix cases
No duplicate check for a persisted insert	Assertion failure, 4 of 16 matrix cases
The write waits only for the first pending commit	Assertion failure, 3 of 16 matrix cases
A result that arrives while a write waits is not deferred	Assertion failure, 2 ordering cases
The write's own cache update runs the stale-fetch merge	Assertion failure, 2 ordering cases
A deferred result is not restored into the cache	Assertion failure, 3 ordering cases
The write takes its generation when it applies, not when it is called	Assertion failure, 2 ordering cases
The harness rethrows the first cleanup error instead of the AggregateError	Assertion failure in the calibration test

External review of 2e1000a77

An external review of the published head 2e1000a77 reported four correctness findings and two test findings. All six were confirmed. The differential matrix missed the four correctness findings for two reasons. Its reference collection runs the same direct-write code. Its ordering cases observe rows only after every promise settles and 20 more event-loop turns, and each has only one write. A new section of the oracle, the precedence model, takes its expected rows from stated rules instead.

Finding	Law	RED on 2e1000a77	Fix
Rejected writes take precedence over a refetch	Only an accepted write gains precedence over an older fetch	8 of 8 rejected-write cases: expected [ { id: 'k', value: 1 } ] or [] to deeply equal [ { id: 'k', value: 2 } ]	The write reserves its position when it is called and claims its keys only after validation passes
Cleanup lets a waiting write fulfill	A write whose sync run cleanup retired while it waited rejects and stores nothing	3 of 3 cases: expected 'ok' not to be 'ok'	The waiting write checks that its sync context is still current, and rejects with SyncTransactionAbortedError
A deferred refetch settles before its rows apply	await refetch() settles after its result's rows apply	expected [ { id: 'k', value: 9 } ] to deeply equal [ { id: 'k', value: 6 } ]; with the merged write held, expected true to be false	A stale or deferred result merges once no write waits, and its waiter settles after the merged rows apply
A deferred refetch overwrites a later write	A fetch that started before a write never overwrites it	Same key: rows, cache and storage end at 6, not 10. Sibling key: j ends at 1, not 10	The deferred result keeps its own fetch start, and key positions are kept while a deferred result exists
The duplicate-insert witness ignores timing	Validation errors reject the returned promise	A synchronous-throw mutant passed the old witness	The witness uses rejects.toBeInstanceOf(DuplicateKeySyncError) for both collections
tx: any in the fake adapter	The adapter matches the persistence contract	No runtime failure; the type was unchecked	The adapter uses PersistedTx, and stored rows and keys are narrowed

The non-deferred stale-merge path had the same settlement fault as the deferred one: it wrote the cache and returned without a settlement. Both paths are now one path.

Mutants against the precedence model, all assertion failures:

Mutant	Result
Claim keys before validation	8 failures (the rejected-write cases)
No current-context check after the wait	3 failures (the cleanup cases)
Settle before the merged rows apply	1 failure: the case whose merged write is held. The case without a held merge cannot tell this mutant apart, because the rows apply synchronously there.
The deferred result loses its fetch start	6 failures, including the 4 ownership-lifecycle merge cases
Key positions cleared while a deferred result exists	6 failures, including the ordering and real-SQLite cases
Synchronous throw for a duplicate insert	1 failure (the duplicate-insert witness)
Store an unnarrowed transaction value in the fake adapter	Type error at the rows.set call

ORC-012 requirement audit

Requirement	Outcome
ORC-001	Applicable. The law and its authority are above. The full matrix uses an in-memory adapter and the two lock holders named. The real-SQLite subset covers the held-durable-write holder only.
ORC-002	Applicable. The matrix's expected result comes from a Query Collection without persistence. It does not use the persistence wrapper, which is the code under judgment. It shares the Query direct-write code, so the precedence model covers that code with expected rows written from its stated rules.
ORC-003	Applicable. The opening prose states the law, the reference, the lock holders, the key cases, the observations and the limits. The ordering section and the witness each have their own prose.
ORC-004	Inapplicable. The oracle is a finite matrix, not a generated history. Every case in the claimed matrix runs.
ORC-005	Applicable. The driver calls the public utils.write* and utils.refetch on a collection built with persistedCollectionOptions. It checks that it reached the window: when the write runs, storage does not hold the waiting refetch's row. It observes the write's outcome, the visible rows, the Query cache and the stored rows after every promise settles.
ORC-006	Applicable. Eight mutants fail at the intended checkpoint, as listed above.
ORC-007	Inapplicable. No generated property exists.
ORC-008	Inapplicable. No stateful reference model exists.
ORC-009	Applicable. "Window" means the time between a refetch commit and its durable write while another task holds the lock. "Lock holder" means the task that holds the apply lock. Neither is a production state.
ORC-010	Applicable. Each failure message carries both observations. Every driver runs its cleanup through checked. When the driver fails and a cleanup step also fails, checked throws an AggregateError whose cause is the driver's failure and whose errors list it first, then each cleanup failure. The calibration test forces a cleanup failure after an assertion failure and checks that the report names the assertion. Comparisons in each test body run after the driver returns, so a cleanup failure cannot hide them.
ORC-011	Applicable. The shared Query direct-write code was a real shared-fault risk: the external review found two faults the differential reference could not see. The precedence model is the second formulation, with expected rows from stated rules.
ORC-013	Applicable to the ordering rule. The before and after cases are opposite sides of the boundary "the fetch started before or after the write was called". A mutant that takes the generation when the write applies moves that boundary, and the after cases reject it.
ORC-014	Applicable. The fake adapter supplies the durable-write delay in parts 1 to 3. Part 4 is the receiving witness: the node SQLite adapter supplies the same premise, a held durable write of an earlier source commit, and stores the rows. It fails on main and passes on the branch. The startup holder has no real-SQLite witness.
Mutation confirmation as a lock holder	Inapplicable. persistAndConfirmCollectionMutations (packages/db-sqlite-persistence-core/src/persisted.ts:2141) is called only by the sync-absent wrappers wrappedOnInsert, wrappedOnUpdate and wrappedOnDelete (persisted.ts:5025, 5043, 5061) and through acceptMutations (persisted.ts:2200, exposed at 5069). persistedCollectionOptions (persisted.ts:4906) builds those only when the options have no sync key (persisted.ts:4935). A Query Collection always has sync, so it is sync-present and never takes the lock this way.