State sync is now [merged](https://github.com/tendermint/tendermint/pull/4705). Up-to-date ABCI documentation is [available](https://github.com/tendermint/spec/pull/90), refer to it rather than this ADR for details.
This ADR outlines the plan for an initial state sync prototype, and is subject to change as we gain feedback and experience. It builds on discussions and findings in [ADR-042](./adr-042-state-sync.md), see that for background information.
## Changelog
@ -22,6 +24,8 @@ This ADR outlines the plan for an initial state sync prototype, and is subject t
* ABCI: chunks are now 0-indexed, for parity with `chunk_hashes` array.
* Reduced maximum chunk size to 16 MB, and increased snapshot message size to 4 MB.
* 2020-04-29: Update with final released ABCI interface (Erik Grinaker)
## Context
State sync will allow a new node to receive a snapshot of the application state without downloading blocks or going through consensus. This bootstraps the node significantly faster than the current fast sync system, which replays all historical blocks.
@ -48,10 +52,11 @@ A node can have multiple snapshots taken at various heights. Snapshots can be ta
```proto
message Snapshot {
uint64 height = 1; // The height at which the snapshot was taken
uint32 format = 2; // The application-specific snapshot format
repeated bytes chunk_hashes = 3; // SHA-256 checksums of all chunks, in order
repeated uint32 refetch_chunks = 2; // Chunks to refetch and reapply (regardless of result)
repeated string reject_senders = 3; // Chunk senders to reject and ban (regardless of result)
enum Result {
accept = 0; // Chunk successfully accepted
abort = 1; // Abort all snapshot restoration
retry = 2; // Retry chunk, combine with refetch and reject as appropriate
retry_snapshot = 3; // Retry snapshot, combine with refetch and reject as appropriate
reject_snapshot = 4; // Reject this snapshot, try a different one but keep sender rejections
}
}
```
@ -141,15 +153,15 @@ When starting an empty node with state sync and fast sync enabled, snapshots are
4. The node requests available snapshots via P2P from peers, via `RequestListSnapshots`. Peers will return the 10 most recent snapshots, one message per snapshot.
5. The node aggregates snapshots from multiple peers, ordered by height and format (in reverse). If there are `chunk_hashes`mismatches between different snapshots, the one hosted by the largest amount of peers is chosen. The node iterates over all snapshots in reverse order by height and format until it finds one that satisfies all of the following conditions:
5. The node aggregates snapshots from multiple peers, ordered by height and format (in reverse). If there are mismatches between different snapshots, the one hosted by the largest amount of peers is chosen. The node iterates over all snapshots in reverse order by height and format until it finds one that satisfies all of the following conditions:
* The snapshot height's block is considered trustworthy by the light client (i.e. snapshot height is greater than trusted header and within unbonding period of the latest trustworthy block).
* The snapshot's height or format hasn't been explicitly rejected by an earlier `RequestOfferSnapshot` call (via `invalid_height` or `invalid_format`).
* The snapshot's height or format hasn't been explicitly rejected by an earlier `RequestOfferSnapshot`.
* The application accepts the `RequestOfferSnapshot` call.
6. The node downloads chunks in parallel from multiple peers, via `RequestLoadSnapshotChunk`, and both the sender and receiver verifies their checksums. Chunk messages cannot exceed 16 MB.
6. The node downloads chunks in parallel from multiple peers, via `RequestLoadSnapshotChunk`. Chunk messages cannot exceed 16 MB.
7. The node passes chunks sequentially to the app via `RequestApplySnapshotChunk`.
@ -191,38 +203,6 @@ Taking consistent snapshots of IAVL trees is greatly simplified by them being ve
Snapshots must also be garbage collected after some configurable time, e.g. by keeping the latest `n` snapshots.
## Experimental Prototype
An experimental but functional state sync prototype is available in the `erik/statesync-prototype` branches of the Tendermint, IAVL, Cosmos SDK, and Gaia repositories. To fetch the necessary branches:
Erik Grinaker
4 years ago
GitHub