You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

202 lines
7.1 KiB

  1. # RFC 005: ReverseSync - fetching historical data
  2. ## Changelog
  3. - 2021-04-19: Use P2P to gossip necessary data for reverse sync.
  4. - 2021-03-03: Simplify proposal to the state sync case.
  5. - 2021-02-17: Add notes on asynchronicity of processes.
  6. - 2020-12-10: Rename backfill blocks to reverse sync.
  7. - 2020-11-25: Initial draft.
  8. ## Author(s)
  9. - Callum Waters (@cmwaters)
  10. ## Context
  11. Two new features: [Block pruning](https://github.com/tendermint/tendermint/issues/3652)
  12. and [State sync](https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-042-state-sync.md)
  13. meant nodes no longer needed a complete history of the blockchain. This
  14. introduced some challenges of its own which were covered and subsequently
  15. tackled with [RFC-001](https://github.com/tendermint/spec/blob/master/rfc/001-block-retention.md).
  16. The RFC allowed applications to set a block retention height; an upper bound on
  17. what blocks would be pruned. However nodes who state sync past this upper bound
  18. (which is necessary as snapshots must be saved within the trusting period for
  19. the assisting light client to verify) have no means of backfilling the blocks
  20. to meet the retention limit. This could be a problem as nodes who state sync and
  21. then eventually switch to consensus (or fast sync) may not have the block and
  22. validator history to verify evidence causing them to panic if they see 2/3
  23. commit on what the node believes to be an invalid block.
  24. Thus, this RFC sets out to instil a minimum block history invariant amongst
  25. honest nodes.
  26. ## Proposal
  27. A backfill mechanism can simply be defined as an algorithm for fetching,
  28. verifying and storing, headers and validator sets of a height prior to the
  29. current base of the node's blockchain. In matching the terminology used for
  30. other data retrieving protocols (i.e. fast sync and state sync), we
  31. call this method **ReverseSync**.
  32. We will define the mechanism in four sections:
  33. - Usage
  34. - Design
  35. - Verification
  36. - Termination
  37. ### Usage
  38. For now, we focus purely on the case of a state syncing node, whom after
  39. syncing to a height will need to verify historical data in order to be capable
  40. of processing new blocks. We can denote the earliest height that the node will
  41. need to verify and store in order to be able to verify any evidence that might
  42. arise as the `max_historical_height`/`time`. Both height and time are necessary
  43. as this maps to the BFT time used for evidence expiration. After acquiring
  44. `State`, we calculate these parameters as:
  45. ```go
  46. max_historical_height = max(state.InitialHeight, state.LastBlockHeight - state.ConsensusParams.EvidenceAgeHeight)
  47. max_historical_time = max(GenesisTime, state.LastBlockTime.Sub(state.ConsensusParams.EvidenceAgeTime))
  48. ```
  49. Before starting either fast sync or consensus, we then run the following
  50. synchronous process:
  51. ```go
  52. func ReverseSync(max_historical_height int64, max_historical_time time.Time) error
  53. ```
  54. Where we fetch and verify blocks until a block `A` where
  55. `A.Height <= max_historical_height` and `A.Time <= max_historical_time`.
  56. Upon successfully reverse syncing, a node can now safely continue. As this
  57. feature is only used as part of state sync, one can think of this as merely an
  58. extension to it.
  59. In the future we may want to extend this functionality to allow nodes to fetch
  60. historical blocks for reasons of accountability or data accessibility.
  61. ### Design
  62. This section will provide a high level overview of some of the more important
  63. characteristics of the design, saving the more tedious details as an ADR.
  64. #### P2P
  65. Implementation of this RFC will require the addition of a new channel and two
  66. new messages.
  67. ```proto
  68. message LightBlockRequest {
  69. uint64 height = 1;
  70. }
  71. ```
  72. ```proto
  73. message LightBlockResponse {
  74. Header header = 1;
  75. Commit commit = 2;
  76. ValidatorSet validator_set = 3;
  77. }
  78. ```
  79. The P2P path may also enable P2P networked light clients and a state sync that
  80. also doesn't need to rely on RPC.
  81. ### Verification
  82. ReverseSync is used to fetch the following data structures:
  83. - `Header`
  84. - `Commit`
  85. - `ValidatorSet`
  86. Nodes will also need to be able to verify these. This can be achieved by first
  87. retrieving the header at the base height from the block store. From this trusted
  88. header, the node hashes each of the three data structures and checks that they are correct.
  89. 1. The trusted header's last block ID matches the hash of the new header
  90. ```go
  91. header[height].LastBlockID == hash(header[height-1])
  92. ```
  93. 2. The trusted header's last commit hash matches the hash of the new commit
  94. ```go
  95. header[height].LastCommitHash == hash(commit[height-1])
  96. ```
  97. 3. Given that the node now trusts the new header, check that the header's validator set
  98. hash matches the hash of the validator set
  99. ```go
  100. header[height-1].ValidatorsHash == hash(validatorSet[height-1])
  101. ```
  102. ### Termination
  103. ReverseSync draws a lot of parallels with fast sync. An important consideration
  104. for fast sync that also extends to ReverseSync is termination. ReverseSync will
  105. finish it's task when one of the following conditions have been met:
  106. 1. It reaches a block `A` where `A.Height <= max_historical_height` and
  107. `A.Time <= max_historical_time`.
  108. 2. None of it's peers reports to have the block at the height below the
  109. processes current block.
  110. 3. A global timeout.
  111. This implies that we can't guarantee adequate history and thus the term
  112. "invariant" can't be used in the strictest sense. In the case that the first
  113. condition isn't met, the node will log an error and optimistically attempt
  114. to continue with either fast sync or consensus.
  115. ## Alternative Solutions
  116. The need for a minimum block history invariant stems purely from the need to
  117. validate evidence (although there may be some application relevant needs as
  118. well). Because of this, an alternative, could be to simply trust whatever the
  119. 2/3+ majority has agreed upon and in the case where a node is at the head of the
  120. blockchain, you simply abstain from voting.
  121. As it stands, if 2/3+ vote on evidence you can't verify, in the same manner if
  122. 2/3+ vote on a header that a node sees as invalid (perhaps due to a different
  123. app hash), the node will halt.
  124. Another alternative is the method with which the relevant data is retrieved.
  125. Instead of introducing new messages to the P2P layer, RPC could have been used
  126. instead.
  127. The aforementioned data is already available via the following RPC endpoints:
  128. `/commit` for `Header`'s' and `/validators` for `ValidatorSet`'s'. It was
  129. decided predominantly due to the instability of the current RPC infrastructure
  130. that P2P be used instead.
  131. ## Status
  132. Proposed
  133. ## Consequences
  134. ### Positive
  135. - Ensures a minimum block history invariant for honest nodes. This will allow
  136. nodes to verify evidence.
  137. ### Negative
  138. - Statesync will be slower as more processing is required.
  139. ### Neutral
  140. - By having validator sets served through p2p, this would make it easier to
  141. extend p2p support to light clients and state sync.
  142. - In the future, it may also be possible to extend this feature to allow for
  143. nodes to freely fetch and verify prior blocks
  144. ## References
  145. - [RFC-001: Block retention](https://github.com/tendermint/spec/blob/master/rfc/001-block-retention.md)
  146. - [Original issue](https://github.com/tendermint/tendermint/issues/4629)