Failed transaction retention review#

Evidence by revision, on fix-failed-transaction-retention, based on origin/main 2ab7f3e55:

• First campaign: oracle commit fb9a682e9, fix 89de5ea0a. It removed a transaction in both isPersisted handlers.
• Review follow-up: f777321fd (tests) and 64f1c1045 (fix), plus 8274b81f3, which keeps an owned transaction's unobserved rejection handled. They move removal into the recompute pass, remove scheduleTransactionCleanup, and make touchCollection() recompute every Collection before it rethrows. The mutant results and CI counts below ran on 8274b81f3. The RED results for findings 1, 2 and 5 below were run on 89de5ea0a, and finding 5 also on 2ab7f3e55.
• Medium-review follow-up (PR head 609e97162, findings M1–M10 below): the owning-Collection set, removal in the overlay pass, settlement that survives a throwing conflicting rollback, and the transaction-ownership oracle. RED runs used 609e97162 and origin/main 07ffcfe47.

Contract and evidence#

A Collection tracks its optimistic transactions in state.transactions. Every per-mutation pass walks that map: recomputeOptimisticState, overlayActiveTransactions, hasPersistingTransaction, commitLocalOnlyDirect, and the Query Collection's isMutationPublicationBlocked. A settled transaction must leave the map, whether it succeeded or failed. Otherwise per-mutation cost grows with history, not with live work.

The law has three parts:

1. After its Collections recompute, a Collection tracks exactly its unsettled transactions. A completed row that a queued sync transaction still holds stays in the held-row layer (heldOptimisticRows), which records it during the recompute. The transaction itself does not stay tracked for it.
2. Removing an entry never removes a different transaction. createTransaction accepts a caller id, and offline restoration reuses ids, so two transactions can share an id over time.
3. One mutation's work does not depend on how many transactions settled before it.

Authority: no reader needs a settled transaction. Every loop over the map skips failed entries. A completed entry is read once, by the recompute that records its held rows. Readers outside packages/db are the offline-transactions restore path and an Electric test that reads a completed transaction.

Design#

Removal now happens in recomputeOptimisticState. The pass that records a completed transaction's held rows collects every settled entry and deletes those entries after the loop. Deleting during the loop would skip the next entry, because SortedMap.delete splices the key array that the loop iterates.

Every way into the map settles through a recompute: a commit or rollback calls touchCollection(), and offline restoration calls rollback() or recomputes itself. So the sweep covers offline restoration without a separate cleanup path. scheduleTransactionCleanup and its per-mutate() handler registrations are gone.

The sweep deletes by map entry, so a same-id successor that replaced its predecessor's entry stays tracked.

The removed isPersisted handlers also kept a rejected isPersisted promise from reaching unhandledrejection, as main does. A Collection now marks the promise handled once, when it first owns the transaction. Without that, the db and offline-transactions suites reported 48 and 154 unhandled rejections while every test passed.

touchCollection() now recomputes every Collection of a settled transaction before it rethrows the first error. Before, a throwing subscriber in one Collection left the next Collection showing the settled transaction's optimistic row. This was a pre-existing visibility bug.

Oracle changes#

• The optimistic-history driver (optimistic-history-oracle.ts) now checks, after every step, that collection._state.transactions.size equals the model's persisting transactions. The first campaign used "at most persisting plus held", which could hide a retained failed transaction while a held one existed, and could not detect an early removal. The opening prose states the law.
• collection.test.ts adds these focused witnesses:
  • 200 rejected updates and manual rollbacks leave nothing tracked.
  • One mutation counts the same Map and Set steps, including Map#get, after 0 and after 200 rollbacks. This is the public cost law; it does not read the internal map.
  • A same-id successor stays tracked and visible after its predecessor rolls back.
  • Three transactions that settle before one recompute all leave the map.
  • A rollback across two Collections recomputes the second Collection when the first one's subscriber throws.
• offline-e2e.test.ts: a restored transaction that fails permanently is no longer tracked after its rollback.

Results#

The first campaign measured 4,000 mutate and rollback cycles on 10k rows: the median per cycle for the last 200 cycles was 0.67–0.94 ms on origin/main and 0.008 ms on the branch. scripts/bench/incremental-update.ts gave a write geomean of 0.88× and 0.84× against origin/main, with same-code noise at 1.03×. The bench rolls back after each optimistic iteration, so its numbers on main include this leak.

Mutants#

All mutants ran on the follow-up revision.

ORC-012 requirement audit#

Medium-review follow-up#

The review of 609e97162 found that removal reached only the Collections that the transaction's current mutations touch, while the removed isPersisted cleanup had reached every Collection that ever tracked it.

The transaction-ownership oracle (packages/db/tests/transaction-ownership-oracle.property.test.ts, registered as transaction-ownership.settlement-release) owns the law across two Collections. It runs a fixed campaign (seed 2080) and a random or replayed campaign, 200 runs each, plus a pinned history. Its grammar covers writes, a pair that merges away, commits, successful and failed settlements, rollbacks, truncate-listener rollbacks, and a throwing subscriber during a settling step. After each synchronous settling step it compares the tracked ids in the same call stack, then compares tracked ids, rows and isPersisted settlement after promises flush. The model's first draft treated an update that keeps the visible value as a write; production creates no mutation for it, which the contract permits, so the model now skips it.

Low-review follow-up#

A low review of 26d34291d found that the ownership driver removed its throwing subscriber in a synchronous finally, before a commit's settlement resumed in a later microtask. A subscriber therefore never threw during an asynchronous success or failure settlement. Measuring the reach showed a wider gap: steps chose a transaction index from 0 to 3 at random, so most pointed at a missing transaction or one in the wrong state, and in one run the settle step ran 0 times in 400 histories.

The driver now keeps the subscriber installed until promises flush, and it asserts that the subscriber ran whenever the model predicts that the step changes that Collection's rows. A step's tx number now chooses among the transactions whose state allows the step, and settle and commit steps are weighted higher. A pinned history throws during an asynchronous success and an asynchronous failure settlement.

The fixed seed alone does not catch the first mutant; the pinned history is the reliable witness for asynchronous settlement with a throwing subscriber.

Medium review round 2 (reviewed head 7fd5de08b)#

Probes on 7fd5de08b and on main fe284ccbd, from a scratch probe file:

The ownership oracle gained:

• an error-shape law, checked for each settling call and each commit rejection against the subscriber errors the step raised, with throwing subscribers on one or both Collections;
• an open that reuses an earlier id, with the duplicate-id rule in the model;
• a mutate() callback that writes and then throws;
• a retention observation: a settled transaction's set of tracking Collections is empty (its mutations still name their Collection);
• identity comparison of tracked transactions;
• four pinned histories (scope identity, duplicate id, flat aggregate, commit error).

A focused witness covers a Collection cleaned up before its transaction settles.

All before the fix, on the oracle above: 8 of 8 cases failed on 7fd5de08b. After the fix: 8 of 8 pass.

The optimistic-history oracle's tracked-transaction check counted only persisting transactions as unsettled. main's pending manual edits (#2078) showed that the model was wrong, not production: a pending transaction is tracked. The check now counts pending and persisting transactions.

The collections CI group then failed to build PowerSync. A private settlement helper made a handler's TransactionWithMutations unassignable to Transaction<any>, because that type omits a key from Transaction, and Omit drops private members. The db typecheck missed it because nothing in db made that assignment. The helper was then @internal; the simplification below removed it. mutation-handler-compatibility.test-d.ts asserts the assignability. The test fails with the private method and passes without it.

Simplification: one error, every step#

Review decision: precise reports for these rare settlement failures do not earn their code. Correct handling does. The AggregateError, its code 234 and the internal error lists are removed:

• touchCollection() recomputes every tracking Collection, then rethrows the first subscriber error.
• rollbackConflictingTransactions() rolls back every conflicting transaction, then rethrows the first error.
• rollback() rejects isPersisted and recomputes its Collections in a finally, so a throwing conflicting rollback cannot skip them.
• commit() rejects with the mutation error when the rollback also throws.

The ownership oracle's error-shape law now requires one of the step's subscriber errors, as is, and the mutation error for a failed commit. It does not choose which subscriber error, because the contract does not.

Targeted review (head 9e7ebb0a8)#

Probes on 9e7ebb0a8 and on main 2c98b4992:

The focused witnesses in packages/offline-transactions/tests/restoration-ownership.test.ts and packages/db/tests/transactions.test.ts failed on 9e7ebb0a8 and pass now.

Unresolved#

• The generated ownership grammar reaches a conflicting rollback that throws, and an asynchronous settlement with a throwing subscriber, only sometimes; the pinned histories are their reliable witnesses.
• Two Collection instances with one id, and a Collection cleaned up before its transaction settles, are covered only by focused witnesses.
• Offline restoration's tracking path is covered by offline-transactions' witnesses, not by the ownership oracle or any generated history.
• A direct write whose subscriber throws leaves its transaction pending, on main and here. No oracle grammar authors a direct write.
• The ownership oracle does not combine queued sync transactions that hold a completed row with a second Collection. The optimistic-history oracle owns held rows for one Collection.
• One random seed of the partial-update history in collection-state-retention-oracle.property.test.ts (TANSTACK_DB_ORACLE_SEED=-526998278) fails identically on main (fe284ccbd). A partial sync update publishes a value that the model does not allow. It is unrelated to transaction retention and is not addressed here.
• The per-mutation pass is still O(unsettled transactions). A key-scoped recompute would remove that term, but it touches every settlement law.

Follow-up review (head 16009c138)#

The review had nine findings from reading the code. Each behavioral claim was probed on this head, and on main's core where the claim compared them.

Noted, outside this change. OfflineTransaction.mutate() with autoCommit rethrows a failed auto-commit inside a .catch, so a failed auto-commit is an unhandled rejection. This is on main as well.