feat(network): announce send chunks as an all-or-nothing batch#53
Merged
Merged
Conversation
One send_changes call = one batch: every chunk announces the same global window [send_checkpoint+1 .. watermark] plus batchId (UUIDv7 per call), chunkIndex and isFinal, replacing the per-chunk announce-window tiling. The server can then advance optimistic only when the final chunk is received and confirm the whole window only when every chunk of the batch applied - a failed batch is never merged, so a partially applied db_version can no longer be masked as covered; the client re-sends the window under a new batchId. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
/applycoverage on the server is db_version-granular, but a send chunk is not a db_version: one chunk can span several db_versions (no db_version-boundary flush in the chunk builder), and a value larger thanmax_chunk_sizefragments across several chunks within one db_version. So the first chunk touching db_version N marks N covered before N is complete — a later chunk's failure is masked (failures.applyhidden, columns silently missing) andlastOptimisticVersionadvances on a partially received db_version.Fix (client side)
One
send_changescall = one all-or-nothing batch:dbVersionMin = send_checkpoint+1,dbVersionMax = watermark, plus three new fields:batchId(UUIDv7 per call),chunkIndex(0-based),isFinal(true on the last chunk; already computed by thecloudsync_payload_chunksvtab).network_announce_minremoved) — the fixed window covers db_versions consumed by non-local-change transactions by construction.batchId. The client checkpoint already followslastOptimisticVersionbackward, so the regress-on-failure signal works unchanged.batchId→ the server keeps today's per-payload behavior, which is correct because a monolithic payload is a complete db_version set.Changes
src/network/network.c: batch UUID per send call; fixed global window announce;batchId/chunkIndex/isFinalin the/applyJSON (bothblobandurltransports); send-loop SELECT readsis_finalinstead ofdb_version_min/max; tiling logic removed.src/network/network_private.h:network_announce_minhelper removed;network_apply_json_payloadexposed for the network unit test.test/network_unit.c: announce-tiling test replaced bytest_apply_json_payload_batch(fixed window, batch fields, both transports, missing-batch-id rejection).Compatibility
The current server ignores the new fields and appends the announced window per chunk, so this client can ship ahead of the server change. Response wire format is unchanged (
lastOptimisticVersion/lastConfirmedVersionstay integer db_versions).Testing
make unittest— all pass (incl. memory-leaks check)make network-unittest— all pass (incl. the new batch-payload test)make e2eagainst the live (pre-batch) server — all pass, incl. Send Gap From Clock Hole and all Chunked tests🤖 Generated with Claude Code