Keep the received log per sender epoch#6604
Open
afck wants to merge 6 commits into
Open
Conversation
Partition ChainStateView::received_log by the epoch of the sender
block's certificate, and key received_certificate_trackers by validator
and epoch. Cross-chain updates already carry each bundle's epoch, so the
receiving worker files every entry under the epoch that certified it.
Clients now query the log per epoch (ReceivedLogQuery { epoch, skip }),
iterating exactly the live epochs: revocation is monotone, so these are
the epochs from the lowest unrevoked one up to the current one. Entries
of revoked epochs are useless to clients — the certificates they point
to can no longer be verified on their own, and blocks that still matter
are re-certified via previous_block_hash / previous_message_blocks links
— so validators drop a revoked epoch's log wholesale the next time a
client asks for a live epoch's log on that chain.
Also remove the unused count_received_log field from ChainInfo.
Use CustomCollectionView with a big-endian CustomSerialize impl for Epoch, so the received log's per-epoch entries are stored in numeric epoch order. Pruning then walks the stored epochs below the lowest requested one from newest to oldest and stops at the first revoked epoch: revocation is monotone, so everything below it is pruned without further checks. Replace the per-epoch received-log queries with a single batched query: the client names all live epochs and their tracker offsets at once, and the validator serves them in ascending epoch order under one total entry bound per response. If the bound cuts an epoch's log short, the client repeats the request with advanced offsets. Turn received_certificate_trackers from a register holding one growing map blob into a MapView keyed by (validator, epoch): updates only touch the changed keys, and entries for revoked epochs — whose logs are pruned and never queried again — are removed whenever the trackers are updated. The field is node-local synchronization bookkeeping, so it is no longer exposed via GraphQL.
A receiver syncing via the received log only sees live epochs' entries, but a sender block from a revoked epoch may still be undelivered. Process the lowest listed block together with its sending ancestors, so that the pruned-epoch message is re-certified and delivered; otherwise the newer messages could never be scheduled behind the missing one.
Key the pruning off the highest revoked epoch below the request, and drop the now-redundant single-epoch query helper.
This was referenced Jul 13, 2026
ma2bd
reviewed
Jul 14, 2026
ma2bd
reviewed
Jul 14, 2026
| /// per sender epoch. Node-local synchronization bookkeeping, not exposed via | ||
| /// GraphQL. | ||
| #[cfg_attr(with_graphql, graphql(skip))] | ||
| pub received_certificate_trackers: MapView<C, (ValidatorPublicKey, Epoch), u64>, |
Contributor
There was a problem hiding this comment.
Could this be MapView<C, ValidatorPublicKey, Vec<(Epoch, u64)>> to spare DB keys?
Contributor
Author
There was a problem hiding this comment.
Done in d1e9680.
But note that this field is only nonempty on the client side. Not sure how important it is to save DB keys there?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Validators keep every chain's received log forever, including entries that point at certificates from revoked epochs — which clients cannot verify on their own anymore. The log can't simply be truncated: clients track their position in it by index, so removing entries would shift every index behind them.
Proposal
Partition the received log by the epoch of the sender's certificate, so a revoked epoch's log can be dropped wholesale, and its client-side tracker with it. Validators prune revoked epochs when a client asks for a live epoch's log. Clients keep one tracker per validator and epoch, and request all live epochs in a single query, so a network with many live epochs doesn't pay for one round trip per epoch.
Since a sync then only sees live epochs' entries, an undelivered message from a pruned epoch could otherwise block the messages behind it forever. The client therefore processes the lowest listed block together with its sending ancestors, which re-certifies the pruned-epoch message through the links introduced in the previous PR.
Test Plan
CI, plus unit tests covering per-epoch log serving and pruning, the batched multi-epoch query, and a regression test for bulk-syncing across a revoked epoch.
Release Plan
devnetand release a newSDK soon.
Links