You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

660 lines
20 KiB

7 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
7 years ago
7 years ago
  1. package state
  2. import (
  3. "bytes"
  4. "errors"
  5. "fmt"
  6. "github.com/gogo/protobuf/proto"
  7. "github.com/google/orderedcode"
  8. dbm "github.com/tendermint/tm-db"
  9. abci "github.com/tendermint/tendermint/abci/types"
  10. tmmath "github.com/tendermint/tendermint/libs/math"
  11. tmstate "github.com/tendermint/tendermint/proto/tendermint/state"
  12. tmproto "github.com/tendermint/tendermint/proto/tendermint/types"
  13. "github.com/tendermint/tendermint/types"
  14. )
  15. const (
  16. // persist validators every valSetCheckpointInterval blocks to avoid
  17. // LoadValidators taking too much time.
  18. // https://github.com/tendermint/tendermint/pull/3438
  19. // 100000 results in ~ 100ms to get 100 validators (see BenchmarkLoadValidators)
  20. valSetCheckpointInterval = 100000
  21. )
  22. //------------------------------------------------------------------------
  23. const (
  24. // prefixes are unique across all tm db's
  25. prefixValidators = int64(5)
  26. prefixConsensusParams = int64(6)
  27. prefixABCIResponses = int64(7)
  28. prefixState = int64(8)
  29. )
  30. func encodeKey(prefix int64, height int64) []byte {
  31. res, err := orderedcode.Append(nil, prefix, height)
  32. if err != nil {
  33. panic(err)
  34. }
  35. return res
  36. }
  37. func validatorsKey(height int64) []byte {
  38. return encodeKey(prefixValidators, height)
  39. }
  40. func consensusParamsKey(height int64) []byte {
  41. return encodeKey(prefixConsensusParams, height)
  42. }
  43. func abciResponsesKey(height int64) []byte {
  44. return encodeKey(prefixABCIResponses, height)
  45. }
  46. // stateKey should never change after being set in init()
  47. var stateKey []byte
  48. func init() {
  49. var err error
  50. stateKey, err = orderedcode.Append(nil, prefixState)
  51. if err != nil {
  52. panic(err)
  53. }
  54. }
  55. //----------------------
  56. //go:generate ../../scripts/mockery_generate.sh Store
  57. // Store defines the state store interface
  58. //
  59. // It is used to retrieve current state and save and load ABCI responses,
  60. // validators and consensus parameters
  61. type Store interface {
  62. // Load loads the current state of the blockchain
  63. Load() (State, error)
  64. // LoadValidators loads the validator set at a given height
  65. LoadValidators(int64) (*types.ValidatorSet, error)
  66. // LoadABCIResponses loads the abciResponse for a given height
  67. LoadABCIResponses(int64) (*tmstate.ABCIResponses, error)
  68. // LoadConsensusParams loads the consensus params for a given height
  69. LoadConsensusParams(int64) (types.ConsensusParams, error)
  70. // Save overwrites the previous state with the updated one
  71. Save(State) error
  72. // SaveABCIResponses saves ABCIResponses for a given height
  73. SaveABCIResponses(int64, *tmstate.ABCIResponses) error
  74. // SaveValidatorSet saves the validator set at a given height
  75. SaveValidatorSets(int64, int64, *types.ValidatorSet) error
  76. // Bootstrap is used for bootstrapping state when not starting from a initial height.
  77. Bootstrap(State) error
  78. // PruneStates takes the height from which to prune up to (exclusive)
  79. PruneStates(int64) error
  80. }
  81. // dbStore wraps a db (github.com/tendermint/tm-db)
  82. type dbStore struct {
  83. db dbm.DB
  84. }
  85. var _ Store = (*dbStore)(nil)
  86. // NewStore creates the dbStore of the state pkg.
  87. func NewStore(db dbm.DB) Store {
  88. return dbStore{db}
  89. }
  90. // LoadState loads the State from the database.
  91. func (store dbStore) Load() (State, error) {
  92. return store.loadState(stateKey)
  93. }
  94. func (store dbStore) loadState(key []byte) (state State, err error) {
  95. buf, err := store.db.Get(key)
  96. if err != nil {
  97. return state, err
  98. }
  99. if len(buf) == 0 {
  100. return state, nil
  101. }
  102. sp := new(tmstate.State)
  103. err = proto.Unmarshal(buf, sp)
  104. if err != nil {
  105. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  106. panic(fmt.Sprintf("data has been corrupted or its spec has changed: %+v", err))
  107. }
  108. sm, err := FromProto(sp)
  109. if err != nil {
  110. return state, err
  111. }
  112. return *sm, nil
  113. }
  114. // Save persists the State, the ValidatorsInfo, and the ConsensusParamsInfo to the database.
  115. // This flushes the writes (e.g. calls SetSync).
  116. func (store dbStore) Save(state State) error {
  117. return store.save(state, stateKey)
  118. }
  119. func (store dbStore) save(state State, key []byte) error {
  120. batch := store.db.NewBatch()
  121. defer batch.Close()
  122. nextHeight := state.LastBlockHeight + 1
  123. // If first block, save validators for the block.
  124. if nextHeight == 1 {
  125. nextHeight = state.InitialHeight
  126. // This extra logic due to Tendermint validator set changes being delayed 1 block.
  127. // It may get overwritten due to InitChain validator updates.
  128. if err := store.saveValidatorsInfo(nextHeight, nextHeight, state.Validators, batch); err != nil {
  129. return err
  130. }
  131. }
  132. // Save next validators.
  133. err := store.saveValidatorsInfo(nextHeight+1, state.LastHeightValidatorsChanged, state.NextValidators, batch)
  134. if err != nil {
  135. return err
  136. }
  137. // Save next consensus params.
  138. if err := store.saveConsensusParamsInfo(nextHeight,
  139. state.LastHeightConsensusParamsChanged, state.ConsensusParams, batch); err != nil {
  140. return err
  141. }
  142. if err := batch.Set(key, state.Bytes()); err != nil {
  143. return err
  144. }
  145. return batch.WriteSync()
  146. }
  147. // BootstrapState saves a new state, used e.g. by state sync when starting from non-zero height.
  148. func (store dbStore) Bootstrap(state State) error {
  149. height := state.LastBlockHeight + 1
  150. if height == 1 {
  151. height = state.InitialHeight
  152. }
  153. batch := store.db.NewBatch()
  154. defer batch.Close()
  155. if height > 1 && !state.LastValidators.IsNilOrEmpty() {
  156. if err := store.saveValidatorsInfo(height-1, height-1, state.LastValidators, batch); err != nil {
  157. return err
  158. }
  159. }
  160. if err := store.saveValidatorsInfo(height, height, state.Validators, batch); err != nil {
  161. return err
  162. }
  163. if err := store.saveValidatorsInfo(height+1, height+1, state.NextValidators, batch); err != nil {
  164. return err
  165. }
  166. if err := store.saveConsensusParamsInfo(height,
  167. state.LastHeightConsensusParamsChanged, state.ConsensusParams, batch); err != nil {
  168. return err
  169. }
  170. if err := batch.Set(stateKey, state.Bytes()); err != nil {
  171. return err
  172. }
  173. return batch.WriteSync()
  174. }
  175. // PruneStates deletes states up to the height specified (exclusive). It is not
  176. // guaranteed to delete all states, since the last checkpointed state and states being pointed to by
  177. // e.g. `LastHeightChanged` must remain. The state at retain height must also exist.
  178. // Pruning is done in descending order.
  179. func (store dbStore) PruneStates(retainHeight int64) error {
  180. if retainHeight <= 0 {
  181. return fmt.Errorf("height %v must be greater than 0", retainHeight)
  182. }
  183. // NOTE: We need to prune consensus params first because the validator
  184. // sets have always one extra height. If validator sets were pruned first
  185. // we could get a situation where we prune up to the last validator set
  186. // yet don't have the respective consensus params at that height and thus
  187. // return an error
  188. if err := store.pruneConsensusParams(retainHeight); err != nil {
  189. return err
  190. }
  191. if err := store.pruneValidatorSets(retainHeight); err != nil {
  192. return err
  193. }
  194. if err := store.pruneABCIResponses(retainHeight); err != nil {
  195. return err
  196. }
  197. return nil
  198. }
  199. // pruneValidatorSets calls a reverse iterator from base height to retain height (exclusive), deleting
  200. // all validator sets in between. Due to the fact that most validator sets stored reference an earlier
  201. // validator set, it is likely that there will remain one validator set left after pruning.
  202. func (store dbStore) pruneValidatorSets(retainHeight int64) error {
  203. valInfo, err := loadValidatorsInfo(store.db, retainHeight)
  204. if err != nil {
  205. return fmt.Errorf("validators at height %v not found: %w", retainHeight, err)
  206. }
  207. // We will prune up to the validator set at the given "height". As we don't save validator sets every
  208. // height but only when they change or at a check point, it is likely that the validator set at the height
  209. // we prune to is empty and thus dependent on the validator set saved at a previous height. We must find
  210. // that validator set and make sure it is not pruned.
  211. lastRecordedValSetHeight := lastStoredHeightFor(retainHeight, valInfo.LastHeightChanged)
  212. lastRecordedValSet, err := loadValidatorsInfo(store.db, lastRecordedValSetHeight)
  213. if err != nil || lastRecordedValSet.ValidatorSet == nil {
  214. return fmt.Errorf("couldn't find validators at height %d (height %d was originally requested): %w",
  215. lastStoredHeightFor(retainHeight, valInfo.LastHeightChanged),
  216. retainHeight,
  217. err,
  218. )
  219. }
  220. // if this is not equal to the retain height, prune from the retain height to the height above
  221. // the last saved validator set. This way we can skip over the dependent validator set.
  222. if lastRecordedValSetHeight < retainHeight {
  223. err := store.pruneRange(
  224. validatorsKey(lastRecordedValSetHeight+1),
  225. validatorsKey(retainHeight),
  226. )
  227. if err != nil {
  228. return err
  229. }
  230. }
  231. // prune all the validators sets up to last saved validator set
  232. return store.pruneRange(
  233. validatorsKey(1),
  234. validatorsKey(lastRecordedValSetHeight),
  235. )
  236. }
  237. // pruneConsensusParams calls a reverse iterator from base height to retain height batch deleting
  238. // all consensus params in between. If the consensus params at the new base height is dependent
  239. // on a prior height then this will keep that lower height too.
  240. func (store dbStore) pruneConsensusParams(retainHeight int64) error {
  241. paramsInfo, err := store.loadConsensusParamsInfo(retainHeight)
  242. if err != nil {
  243. return fmt.Errorf("consensus params at height %v not found: %w", retainHeight, err)
  244. }
  245. // As we don't save the consensus params at every height, only when there is a consensus params change,
  246. // we must not prune (or save) the last consensus params that the consensus params info at height
  247. // is dependent on.
  248. if paramsInfo.ConsensusParams.Equal(&tmproto.ConsensusParams{}) {
  249. // sanity check that the consensus params at the last height it was changed is there
  250. lastRecordedConsensusParams, err := store.loadConsensusParamsInfo(paramsInfo.LastHeightChanged)
  251. if err != nil || lastRecordedConsensusParams.ConsensusParams.Equal(&tmproto.ConsensusParams{}) {
  252. return fmt.Errorf(
  253. "couldn't find consensus params at height %d (height %d was originally requested): %w",
  254. paramsInfo.LastHeightChanged,
  255. retainHeight,
  256. err,
  257. )
  258. }
  259. // prune the params above the height with which it last changed and below the retain height.
  260. err = store.pruneRange(
  261. consensusParamsKey(paramsInfo.LastHeightChanged+1),
  262. consensusParamsKey(retainHeight),
  263. )
  264. if err != nil {
  265. return err
  266. }
  267. }
  268. // prune all the consensus params up to either the last height the params changed or if the params
  269. // last changed at the retain height, then up to the retain height.
  270. return store.pruneRange(
  271. consensusParamsKey(1),
  272. consensusParamsKey(paramsInfo.LastHeightChanged),
  273. )
  274. }
  275. // pruneABCIResponses calls a reverse iterator from base height to retain height batch deleting
  276. // all abci responses in between
  277. func (store dbStore) pruneABCIResponses(height int64) error {
  278. return store.pruneRange(abciResponsesKey(1), abciResponsesKey(height))
  279. }
  280. // pruneRange is a generic function for deleting a range of keys in reverse order.
  281. // we keep filling up batches of at most 1000 keys, perform a deletion and continue until
  282. // we have gone through all of keys in the range. This avoids doing any writes whilst
  283. // iterating.
  284. func (store dbStore) pruneRange(start []byte, end []byte) error {
  285. var err error
  286. batch := store.db.NewBatch()
  287. defer batch.Close()
  288. end, err = store.reverseBatchDelete(batch, start, end)
  289. if err != nil {
  290. return err
  291. }
  292. // iterate until the last batch of the pruning range in which case we will perform a
  293. // write sync
  294. for !bytes.Equal(start, end) {
  295. if err := batch.Write(); err != nil {
  296. return err
  297. }
  298. if err := batch.Close(); err != nil {
  299. return err
  300. }
  301. batch = store.db.NewBatch()
  302. // fill a new batch of keys for deletion over the remainding range
  303. end, err = store.reverseBatchDelete(batch, start, end)
  304. if err != nil {
  305. return err
  306. }
  307. }
  308. return batch.WriteSync()
  309. }
  310. // reverseBatchDelete runs a reverse iterator (from end to start) filling up a batch until either
  311. // (a) the iterator reaches the start or (b) the iterator has added a 1000 keys (this avoids the
  312. // batch from growing too large)
  313. func (store dbStore) reverseBatchDelete(batch dbm.Batch, start, end []byte) ([]byte, error) {
  314. iter, err := store.db.ReverseIterator(start, end)
  315. if err != nil {
  316. return end, fmt.Errorf("iterator error: %w", err)
  317. }
  318. defer iter.Close()
  319. size := 0
  320. for ; iter.Valid(); iter.Next() {
  321. if err := batch.Delete(iter.Key()); err != nil {
  322. return end, fmt.Errorf("pruning error at key %X: %w", iter.Key(), err)
  323. }
  324. // avoid batches growing too large by capping them
  325. size++
  326. if size == 1000 {
  327. return iter.Key(), iter.Error()
  328. }
  329. }
  330. return start, iter.Error()
  331. }
  332. //------------------------------------------------------------------------
  333. // ABCIResponsesResultsHash returns the root hash of a Merkle tree of
  334. // ResponseDeliverTx responses (see ABCIResults.Hash)
  335. //
  336. // See merkle.SimpleHashFromByteSlices
  337. func ABCIResponsesResultsHash(ar *tmstate.ABCIResponses) []byte {
  338. return types.NewResults(ar.DeliverTxs).Hash()
  339. }
  340. // LoadABCIResponses loads the ABCIResponses for the given height from the
  341. // database. If not found, ErrNoABCIResponsesForHeight is returned.
  342. //
  343. // This is useful for recovering from crashes where we called app.Commit and
  344. // before we called s.Save(). It can also be used to produce Merkle proofs of
  345. // the result of txs.
  346. func (store dbStore) LoadABCIResponses(height int64) (*tmstate.ABCIResponses, error) {
  347. buf, err := store.db.Get(abciResponsesKey(height))
  348. if err != nil {
  349. return nil, err
  350. }
  351. if len(buf) == 0 {
  352. return nil, ErrNoABCIResponsesForHeight{height}
  353. }
  354. abciResponses := new(tmstate.ABCIResponses)
  355. err = abciResponses.Unmarshal(buf)
  356. if err != nil {
  357. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  358. panic(fmt.Sprintf("data has been corrupted or its spec has changed: %+v", err))
  359. }
  360. // TODO: ensure that buf is completely read.
  361. return abciResponses, nil
  362. }
  363. // SaveABCIResponses persists the ABCIResponses to the database.
  364. // This is useful in case we crash after app.Commit and before s.Save().
  365. // Responses are indexed by height so they can also be loaded later to produce
  366. // Merkle proofs.
  367. //
  368. // Exposed for testing.
  369. func (store dbStore) SaveABCIResponses(height int64, abciResponses *tmstate.ABCIResponses) error {
  370. return store.saveABCIResponses(height, abciResponses)
  371. }
  372. func (store dbStore) saveABCIResponses(height int64, abciResponses *tmstate.ABCIResponses) error {
  373. var dtxs []*abci.ResponseDeliverTx
  374. // strip nil values,
  375. for _, tx := range abciResponses.DeliverTxs {
  376. if tx != nil {
  377. dtxs = append(dtxs, tx)
  378. }
  379. }
  380. abciResponses.DeliverTxs = dtxs
  381. bz, err := abciResponses.Marshal()
  382. if err != nil {
  383. return err
  384. }
  385. return store.db.SetSync(abciResponsesKey(height), bz)
  386. }
  387. // SaveValidatorSets is used to save the validator set over multiple heights.
  388. // It is exposed so that a backfill operation during state sync can populate
  389. // the store with the necessary amount of validator sets to verify any evidence
  390. // it may encounter.
  391. func (store dbStore) SaveValidatorSets(lowerHeight, upperHeight int64, vals *types.ValidatorSet) error {
  392. batch := store.db.NewBatch()
  393. defer batch.Close()
  394. // batch together all the validator sets from lowerHeight to upperHeight
  395. for height := lowerHeight; height <= upperHeight; height++ {
  396. if err := store.saveValidatorsInfo(height, lowerHeight, vals, batch); err != nil {
  397. return err
  398. }
  399. }
  400. return batch.WriteSync()
  401. }
  402. //-----------------------------------------------------------------------------
  403. // LoadValidators loads the ValidatorSet for a given height.
  404. // Returns ErrNoValSetForHeight if the validator set can't be found for this height.
  405. func (store dbStore) LoadValidators(height int64) (*types.ValidatorSet, error) {
  406. valInfo, err := loadValidatorsInfo(store.db, height)
  407. if err != nil {
  408. return nil, ErrNoValSetForHeight{height}
  409. }
  410. if valInfo.ValidatorSet == nil {
  411. lastStoredHeight := lastStoredHeightFor(height, valInfo.LastHeightChanged)
  412. valInfo2, err := loadValidatorsInfo(store.db, lastStoredHeight)
  413. if err != nil || valInfo2.ValidatorSet == nil {
  414. return nil,
  415. fmt.Errorf("couldn't find validators at height %d (height %d was originally requested): %w",
  416. lastStoredHeight,
  417. height,
  418. err,
  419. )
  420. }
  421. vs, err := types.ValidatorSetFromProto(valInfo2.ValidatorSet)
  422. if err != nil {
  423. return nil, err
  424. }
  425. vs.IncrementProposerPriority(tmmath.SafeConvertInt32(height - lastStoredHeight)) // mutate
  426. vi2, err := vs.ToProto()
  427. if err != nil {
  428. return nil, err
  429. }
  430. valInfo2.ValidatorSet = vi2
  431. valInfo = valInfo2
  432. }
  433. vip, err := types.ValidatorSetFromProto(valInfo.ValidatorSet)
  434. if err != nil {
  435. return nil, err
  436. }
  437. return vip, nil
  438. }
  439. func lastStoredHeightFor(height, lastHeightChanged int64) int64 {
  440. checkpointHeight := height - height%valSetCheckpointInterval
  441. return tmmath.MaxInt64(checkpointHeight, lastHeightChanged)
  442. }
  443. // CONTRACT: Returned ValidatorsInfo can be mutated.
  444. func loadValidatorsInfo(db dbm.DB, height int64) (*tmstate.ValidatorsInfo, error) {
  445. buf, err := db.Get(validatorsKey(height))
  446. if err != nil {
  447. return nil, err
  448. }
  449. if len(buf) == 0 {
  450. return nil, errors.New("value retrieved from db is empty")
  451. }
  452. v := new(tmstate.ValidatorsInfo)
  453. err = v.Unmarshal(buf)
  454. if err != nil {
  455. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  456. panic(fmt.Sprintf("data has been corrupted or its spec has changed: %+v", err))
  457. }
  458. // TODO: ensure that buf is completely read.
  459. return v, nil
  460. }
  461. // saveValidatorsInfo persists the validator set.
  462. //
  463. // `height` is the effective height for which the validator is responsible for
  464. // signing. It should be called from s.Save(), right before the state itself is
  465. // persisted.
  466. func (store dbStore) saveValidatorsInfo(
  467. height, lastHeightChanged int64,
  468. valSet *types.ValidatorSet,
  469. batch dbm.Batch,
  470. ) error {
  471. if lastHeightChanged > height {
  472. return errors.New("lastHeightChanged cannot be greater than ValidatorsInfo height")
  473. }
  474. valInfo := &tmstate.ValidatorsInfo{
  475. LastHeightChanged: lastHeightChanged,
  476. }
  477. // Only persist validator set if it was updated or checkpoint height (see
  478. // valSetCheckpointInterval) is reached.
  479. if height == lastHeightChanged || height%valSetCheckpointInterval == 0 {
  480. pv, err := valSet.ToProto()
  481. if err != nil {
  482. return err
  483. }
  484. valInfo.ValidatorSet = pv
  485. }
  486. bz, err := valInfo.Marshal()
  487. if err != nil {
  488. return err
  489. }
  490. return batch.Set(validatorsKey(height), bz)
  491. }
  492. //-----------------------------------------------------------------------------
  493. // ConsensusParamsInfo represents the latest consensus params, or the last height it changed
  494. // Allocate empty Consensus params at compile time to avoid multiple allocations during runtime
  495. var (
  496. empty = types.ConsensusParams{}
  497. emptypb = tmproto.ConsensusParams{}
  498. )
  499. // LoadConsensusParams loads the ConsensusParams for a given height.
  500. func (store dbStore) LoadConsensusParams(height int64) (types.ConsensusParams, error) {
  501. paramsInfo, err := store.loadConsensusParamsInfo(height)
  502. if err != nil {
  503. return empty, fmt.Errorf("could not find consensus params for height #%d: %w", height, err)
  504. }
  505. if paramsInfo.ConsensusParams.Equal(&emptypb) {
  506. paramsInfo2, err := store.loadConsensusParamsInfo(paramsInfo.LastHeightChanged)
  507. if err != nil {
  508. return empty, fmt.Errorf(
  509. "couldn't find consensus params at height %d (height %d was originally requested): %w",
  510. paramsInfo.LastHeightChanged,
  511. height,
  512. err,
  513. )
  514. }
  515. paramsInfo = paramsInfo2
  516. }
  517. return types.ConsensusParamsFromProto(paramsInfo.ConsensusParams), nil
  518. }
  519. func (store dbStore) loadConsensusParamsInfo(height int64) (*tmstate.ConsensusParamsInfo, error) {
  520. buf, err := store.db.Get(consensusParamsKey(height))
  521. if err != nil {
  522. return nil, err
  523. }
  524. if len(buf) == 0 {
  525. return nil, errors.New("value retrieved from db is empty")
  526. }
  527. paramsInfo := new(tmstate.ConsensusParamsInfo)
  528. if err = paramsInfo.Unmarshal(buf); err != nil {
  529. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  530. panic(fmt.Sprintf(`data has been corrupted or its spec has changed: %+v`, err))
  531. }
  532. // TODO: ensure that buf is completely read.
  533. return paramsInfo, nil
  534. }
  535. // saveConsensusParamsInfo persists the consensus params for the next block to disk.
  536. // It should be called from s.Save(), right before the state itself is persisted.
  537. // If the consensus params did not change after processing the latest block,
  538. // only the last height for which they changed is persisted.
  539. func (store dbStore) saveConsensusParamsInfo(
  540. nextHeight, changeHeight int64,
  541. params types.ConsensusParams,
  542. batch dbm.Batch,
  543. ) error {
  544. paramsInfo := &tmstate.ConsensusParamsInfo{
  545. LastHeightChanged: changeHeight,
  546. }
  547. if changeHeight == nextHeight {
  548. paramsInfo.ConsensusParams = params.ToProto()
  549. }
  550. bz, err := paramsInfo.Marshal()
  551. if err != nil {
  552. return err
  553. }
  554. return batch.Set(consensusParamsKey(nextHeight), bz)
  555. }