You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

666 lines
20 KiB

7 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
7 years ago
7 years ago
  1. package state
  2. import (
  3. "bytes"
  4. "errors"
  5. "fmt"
  6. "github.com/gogo/protobuf/proto"
  7. "github.com/google/orderedcode"
  8. dbm "github.com/tendermint/tm-db"
  9. abci "github.com/tendermint/tendermint/abci/types"
  10. tmmath "github.com/tendermint/tendermint/libs/math"
  11. tmstate "github.com/tendermint/tendermint/proto/tendermint/state"
  12. tmproto "github.com/tendermint/tendermint/proto/tendermint/types"
  13. "github.com/tendermint/tendermint/types"
  14. )
  15. const (
  16. // persist validators every valSetCheckpointInterval blocks to avoid
  17. // LoadValidators taking too much time.
  18. // https://github.com/tendermint/tendermint/pull/3438
  19. // 100000 results in ~ 100ms to get 100 validators (see BenchmarkLoadValidators)
  20. valSetCheckpointInterval = 100000
  21. )
  22. //------------------------------------------------------------------------
  23. const (
  24. // prefixes are unique across all tm db's
  25. prefixValidators = int64(5)
  26. prefixConsensusParams = int64(6)
  27. prefixABCIResponses = int64(7)
  28. prefixState = int64(8)
  29. )
  30. func encodeKey(prefix int64, height int64) []byte {
  31. res, err := orderedcode.Append(nil, prefix, height)
  32. if err != nil {
  33. panic(err)
  34. }
  35. return res
  36. }
  37. func validatorsKey(height int64) []byte {
  38. return encodeKey(prefixValidators, height)
  39. }
  40. func consensusParamsKey(height int64) []byte {
  41. return encodeKey(prefixConsensusParams, height)
  42. }
  43. func abciResponsesKey(height int64) []byte {
  44. return encodeKey(prefixABCIResponses, height)
  45. }
  46. // stateKey should never change after being set in init()
  47. var stateKey []byte
  48. func init() {
  49. var err error
  50. stateKey, err = orderedcode.Append(nil, prefixState)
  51. if err != nil {
  52. panic(err)
  53. }
  54. }
  55. //----------------------
  56. //go:generate ../../scripts/mockery_generate.sh Store
  57. // Store defines the state store interface
  58. //
  59. // It is used to retrieve current state and save and load ABCI responses,
  60. // validators and consensus parameters
  61. type Store interface {
  62. // Load loads the current state of the blockchain
  63. Load() (State, error)
  64. // LoadValidators loads the validator set at a given height
  65. LoadValidators(int64) (*types.ValidatorSet, error)
  66. // LoadABCIResponses loads the abciResponse for a given height
  67. LoadABCIResponses(int64) (*tmstate.ABCIResponses, error)
  68. // LoadConsensusParams loads the consensus params for a given height
  69. LoadConsensusParams(int64) (types.ConsensusParams, error)
  70. // Save overwrites the previous state with the updated one
  71. Save(State) error
  72. // SaveABCIResponses saves ABCIResponses for a given height
  73. SaveABCIResponses(int64, *tmstate.ABCIResponses) error
  74. // SaveValidatorSet saves the validator set at a given height
  75. SaveValidatorSets(int64, int64, *types.ValidatorSet) error
  76. // Bootstrap is used for bootstrapping state when not starting from a initial height.
  77. Bootstrap(State) error
  78. // PruneStates takes the height from which to prune up to (exclusive)
  79. PruneStates(int64) error
  80. // Close closes the connection with the database
  81. Close() error
  82. }
  83. // dbStore wraps a db (github.com/tendermint/tm-db)
  84. type dbStore struct {
  85. db dbm.DB
  86. }
  87. var _ Store = (*dbStore)(nil)
  88. // NewStore creates the dbStore of the state pkg.
  89. func NewStore(db dbm.DB) Store {
  90. return dbStore{db}
  91. }
  92. // LoadState loads the State from the database.
  93. func (store dbStore) Load() (State, error) {
  94. return store.loadState(stateKey)
  95. }
  96. func (store dbStore) loadState(key []byte) (state State, err error) {
  97. buf, err := store.db.Get(key)
  98. if err != nil {
  99. return state, err
  100. }
  101. if len(buf) == 0 {
  102. return state, nil
  103. }
  104. sp := new(tmstate.State)
  105. err = proto.Unmarshal(buf, sp)
  106. if err != nil {
  107. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  108. panic(fmt.Sprintf("data has been corrupted or its spec has changed: %+v", err))
  109. }
  110. sm, err := FromProto(sp)
  111. if err != nil {
  112. return state, err
  113. }
  114. return *sm, nil
  115. }
  116. // Save persists the State, the ValidatorsInfo, and the ConsensusParamsInfo to the database.
  117. // This flushes the writes (e.g. calls SetSync).
  118. func (store dbStore) Save(state State) error {
  119. return store.save(state, stateKey)
  120. }
  121. func (store dbStore) save(state State, key []byte) error {
  122. batch := store.db.NewBatch()
  123. defer batch.Close()
  124. nextHeight := state.LastBlockHeight + 1
  125. // If first block, save validators for the block.
  126. if nextHeight == 1 {
  127. nextHeight = state.InitialHeight
  128. // This extra logic due to Tendermint validator set changes being delayed 1 block.
  129. // It may get overwritten due to InitChain validator updates.
  130. if err := store.saveValidatorsInfo(nextHeight, nextHeight, state.Validators, batch); err != nil {
  131. return err
  132. }
  133. }
  134. // Save next validators.
  135. err := store.saveValidatorsInfo(nextHeight+1, state.LastHeightValidatorsChanged, state.NextValidators, batch)
  136. if err != nil {
  137. return err
  138. }
  139. // Save next consensus params.
  140. if err := store.saveConsensusParamsInfo(nextHeight,
  141. state.LastHeightConsensusParamsChanged, state.ConsensusParams, batch); err != nil {
  142. return err
  143. }
  144. if err := batch.Set(key, state.Bytes()); err != nil {
  145. return err
  146. }
  147. return batch.WriteSync()
  148. }
  149. // BootstrapState saves a new state, used e.g. by state sync when starting from non-zero height.
  150. func (store dbStore) Bootstrap(state State) error {
  151. height := state.LastBlockHeight + 1
  152. if height == 1 {
  153. height = state.InitialHeight
  154. }
  155. batch := store.db.NewBatch()
  156. defer batch.Close()
  157. if height > 1 && !state.LastValidators.IsNilOrEmpty() {
  158. if err := store.saveValidatorsInfo(height-1, height-1, state.LastValidators, batch); err != nil {
  159. return err
  160. }
  161. }
  162. if err := store.saveValidatorsInfo(height, height, state.Validators, batch); err != nil {
  163. return err
  164. }
  165. if err := store.saveValidatorsInfo(height+1, height+1, state.NextValidators, batch); err != nil {
  166. return err
  167. }
  168. if err := store.saveConsensusParamsInfo(height,
  169. state.LastHeightConsensusParamsChanged, state.ConsensusParams, batch); err != nil {
  170. return err
  171. }
  172. if err := batch.Set(stateKey, state.Bytes()); err != nil {
  173. return err
  174. }
  175. return batch.WriteSync()
  176. }
  177. // PruneStates deletes states up to the height specified (exclusive). It is not
  178. // guaranteed to delete all states, since the last checkpointed state and states being pointed to by
  179. // e.g. `LastHeightChanged` must remain. The state at retain height must also exist.
  180. // Pruning is done in descending order.
  181. func (store dbStore) PruneStates(retainHeight int64) error {
  182. if retainHeight <= 0 {
  183. return fmt.Errorf("height %v must be greater than 0", retainHeight)
  184. }
  185. // NOTE: We need to prune consensus params first because the validator
  186. // sets have always one extra height. If validator sets were pruned first
  187. // we could get a situation where we prune up to the last validator set
  188. // yet don't have the respective consensus params at that height and thus
  189. // return an error
  190. if err := store.pruneConsensusParams(retainHeight); err != nil {
  191. return err
  192. }
  193. if err := store.pruneValidatorSets(retainHeight); err != nil {
  194. return err
  195. }
  196. if err := store.pruneABCIResponses(retainHeight); err != nil {
  197. return err
  198. }
  199. return nil
  200. }
  201. // pruneValidatorSets calls a reverse iterator from base height to retain height (exclusive), deleting
  202. // all validator sets in between. Due to the fact that most validator sets stored reference an earlier
  203. // validator set, it is likely that there will remain one validator set left after pruning.
  204. func (store dbStore) pruneValidatorSets(retainHeight int64) error {
  205. valInfo, err := loadValidatorsInfo(store.db, retainHeight)
  206. if err != nil {
  207. return fmt.Errorf("validators at height %v not found: %w", retainHeight, err)
  208. }
  209. // We will prune up to the validator set at the given "height". As we don't save validator sets every
  210. // height but only when they change or at a check point, it is likely that the validator set at the height
  211. // we prune to is empty and thus dependent on the validator set saved at a previous height. We must find
  212. // that validator set and make sure it is not pruned.
  213. lastRecordedValSetHeight := lastStoredHeightFor(retainHeight, valInfo.LastHeightChanged)
  214. lastRecordedValSet, err := loadValidatorsInfo(store.db, lastRecordedValSetHeight)
  215. if err != nil || lastRecordedValSet.ValidatorSet == nil {
  216. return fmt.Errorf("couldn't find validators at height %d (height %d was originally requested): %w",
  217. lastStoredHeightFor(retainHeight, valInfo.LastHeightChanged),
  218. retainHeight,
  219. err,
  220. )
  221. }
  222. // if this is not equal to the retain height, prune from the retain height to the height above
  223. // the last saved validator set. This way we can skip over the dependent validator set.
  224. if lastRecordedValSetHeight < retainHeight {
  225. err := store.pruneRange(
  226. validatorsKey(lastRecordedValSetHeight+1),
  227. validatorsKey(retainHeight),
  228. )
  229. if err != nil {
  230. return err
  231. }
  232. }
  233. // prune all the validators sets up to last saved validator set
  234. return store.pruneRange(
  235. validatorsKey(1),
  236. validatorsKey(lastRecordedValSetHeight),
  237. )
  238. }
  239. // pruneConsensusParams calls a reverse iterator from base height to retain height batch deleting
  240. // all consensus params in between. If the consensus params at the new base height is dependent
  241. // on a prior height then this will keep that lower height too.
  242. func (store dbStore) pruneConsensusParams(retainHeight int64) error {
  243. paramsInfo, err := store.loadConsensusParamsInfo(retainHeight)
  244. if err != nil {
  245. return fmt.Errorf("consensus params at height %v not found: %w", retainHeight, err)
  246. }
  247. // As we don't save the consensus params at every height, only when there is a consensus params change,
  248. // we must not prune (or save) the last consensus params that the consensus params info at height
  249. // is dependent on.
  250. if paramsInfo.ConsensusParams.Equal(&tmproto.ConsensusParams{}) {
  251. // sanity check that the consensus params at the last height it was changed is there
  252. lastRecordedConsensusParams, err := store.loadConsensusParamsInfo(paramsInfo.LastHeightChanged)
  253. if err != nil || lastRecordedConsensusParams.ConsensusParams.Equal(&tmproto.ConsensusParams{}) {
  254. return fmt.Errorf(
  255. "couldn't find consensus params at height %d (height %d was originally requested): %w",
  256. paramsInfo.LastHeightChanged,
  257. retainHeight,
  258. err,
  259. )
  260. }
  261. // prune the params above the height with which it last changed and below the retain height.
  262. err = store.pruneRange(
  263. consensusParamsKey(paramsInfo.LastHeightChanged+1),
  264. consensusParamsKey(retainHeight),
  265. )
  266. if err != nil {
  267. return err
  268. }
  269. }
  270. // prune all the consensus params up to either the last height the params changed or if the params
  271. // last changed at the retain height, then up to the retain height.
  272. return store.pruneRange(
  273. consensusParamsKey(1),
  274. consensusParamsKey(paramsInfo.LastHeightChanged),
  275. )
  276. }
  277. // pruneABCIResponses calls a reverse iterator from base height to retain height batch deleting
  278. // all abci responses in between
  279. func (store dbStore) pruneABCIResponses(height int64) error {
  280. return store.pruneRange(abciResponsesKey(1), abciResponsesKey(height))
  281. }
  282. // pruneRange is a generic function for deleting a range of keys in reverse order.
  283. // we keep filling up batches of at most 1000 keys, perform a deletion and continue until
  284. // we have gone through all of keys in the range. This avoids doing any writes whilst
  285. // iterating.
  286. func (store dbStore) pruneRange(start []byte, end []byte) error {
  287. var err error
  288. batch := store.db.NewBatch()
  289. defer batch.Close()
  290. end, err = store.reverseBatchDelete(batch, start, end)
  291. if err != nil {
  292. return err
  293. }
  294. // iterate until the last batch of the pruning range in which case we will perform a
  295. // write sync
  296. for !bytes.Equal(start, end) {
  297. if err := batch.Write(); err != nil {
  298. return err
  299. }
  300. if err := batch.Close(); err != nil {
  301. return err
  302. }
  303. batch = store.db.NewBatch()
  304. // fill a new batch of keys for deletion over the remainding range
  305. end, err = store.reverseBatchDelete(batch, start, end)
  306. if err != nil {
  307. return err
  308. }
  309. }
  310. return batch.WriteSync()
  311. }
  312. // reverseBatchDelete runs a reverse iterator (from end to start) filling up a batch until either
  313. // (a) the iterator reaches the start or (b) the iterator has added a 1000 keys (this avoids the
  314. // batch from growing too large)
  315. func (store dbStore) reverseBatchDelete(batch dbm.Batch, start, end []byte) ([]byte, error) {
  316. iter, err := store.db.ReverseIterator(start, end)
  317. if err != nil {
  318. return end, fmt.Errorf("iterator error: %w", err)
  319. }
  320. defer iter.Close()
  321. size := 0
  322. for ; iter.Valid(); iter.Next() {
  323. if err := batch.Delete(iter.Key()); err != nil {
  324. return end, fmt.Errorf("pruning error at key %X: %w", iter.Key(), err)
  325. }
  326. // avoid batches growing too large by capping them
  327. size++
  328. if size == 1000 {
  329. return iter.Key(), iter.Error()
  330. }
  331. }
  332. return start, iter.Error()
  333. }
  334. //------------------------------------------------------------------------
  335. // ABCIResponsesResultsHash returns the root hash of a Merkle tree of
  336. // ResponseDeliverTx responses (see ABCIResults.Hash)
  337. //
  338. // See merkle.SimpleHashFromByteSlices
  339. func ABCIResponsesResultsHash(ar *tmstate.ABCIResponses) []byte {
  340. return types.NewResults(ar.DeliverTxs).Hash()
  341. }
  342. // LoadABCIResponses loads the ABCIResponses for the given height from the
  343. // database. If not found, ErrNoABCIResponsesForHeight is returned.
  344. //
  345. // This is useful for recovering from crashes where we called app.Commit and
  346. // before we called s.Save(). It can also be used to produce Merkle proofs of
  347. // the result of txs.
  348. func (store dbStore) LoadABCIResponses(height int64) (*tmstate.ABCIResponses, error) {
  349. buf, err := store.db.Get(abciResponsesKey(height))
  350. if err != nil {
  351. return nil, err
  352. }
  353. if len(buf) == 0 {
  354. return nil, ErrNoABCIResponsesForHeight{height}
  355. }
  356. abciResponses := new(tmstate.ABCIResponses)
  357. err = abciResponses.Unmarshal(buf)
  358. if err != nil {
  359. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  360. panic(fmt.Sprintf("data has been corrupted or its spec has changed: %+v", err))
  361. }
  362. // TODO: ensure that buf is completely read.
  363. return abciResponses, nil
  364. }
  365. // SaveABCIResponses persists the ABCIResponses to the database.
  366. // This is useful in case we crash after app.Commit and before s.Save().
  367. // Responses are indexed by height so they can also be loaded later to produce
  368. // Merkle proofs.
  369. //
  370. // Exposed for testing.
  371. func (store dbStore) SaveABCIResponses(height int64, abciResponses *tmstate.ABCIResponses) error {
  372. return store.saveABCIResponses(height, abciResponses)
  373. }
  374. func (store dbStore) saveABCIResponses(height int64, abciResponses *tmstate.ABCIResponses) error {
  375. var dtxs []*abci.ResponseDeliverTx
  376. // strip nil values,
  377. for _, tx := range abciResponses.DeliverTxs {
  378. if tx != nil {
  379. dtxs = append(dtxs, tx)
  380. }
  381. }
  382. abciResponses.DeliverTxs = dtxs
  383. bz, err := abciResponses.Marshal()
  384. if err != nil {
  385. return err
  386. }
  387. return store.db.SetSync(abciResponsesKey(height), bz)
  388. }
  389. // SaveValidatorSets is used to save the validator set over multiple heights.
  390. // It is exposed so that a backfill operation during state sync can populate
  391. // the store with the necessary amount of validator sets to verify any evidence
  392. // it may encounter.
  393. func (store dbStore) SaveValidatorSets(lowerHeight, upperHeight int64, vals *types.ValidatorSet) error {
  394. batch := store.db.NewBatch()
  395. defer batch.Close()
  396. // batch together all the validator sets from lowerHeight to upperHeight
  397. for height := lowerHeight; height <= upperHeight; height++ {
  398. if err := store.saveValidatorsInfo(height, lowerHeight, vals, batch); err != nil {
  399. return err
  400. }
  401. }
  402. return batch.WriteSync()
  403. }
  404. //-----------------------------------------------------------------------------
  405. // LoadValidators loads the ValidatorSet for a given height.
  406. // Returns ErrNoValSetForHeight if the validator set can't be found for this height.
  407. func (store dbStore) LoadValidators(height int64) (*types.ValidatorSet, error) {
  408. valInfo, err := loadValidatorsInfo(store.db, height)
  409. if err != nil {
  410. return nil, ErrNoValSetForHeight{height}
  411. }
  412. if valInfo.ValidatorSet == nil {
  413. lastStoredHeight := lastStoredHeightFor(height, valInfo.LastHeightChanged)
  414. valInfo2, err := loadValidatorsInfo(store.db, lastStoredHeight)
  415. if err != nil || valInfo2.ValidatorSet == nil {
  416. return nil,
  417. fmt.Errorf("couldn't find validators at height %d (height %d was originally requested): %w",
  418. lastStoredHeight,
  419. height,
  420. err,
  421. )
  422. }
  423. vs, err := types.ValidatorSetFromProto(valInfo2.ValidatorSet)
  424. if err != nil {
  425. return nil, err
  426. }
  427. vs.IncrementProposerPriority(tmmath.SafeConvertInt32(height - lastStoredHeight)) // mutate
  428. vi2, err := vs.ToProto()
  429. if err != nil {
  430. return nil, err
  431. }
  432. valInfo2.ValidatorSet = vi2
  433. valInfo = valInfo2
  434. }
  435. vip, err := types.ValidatorSetFromProto(valInfo.ValidatorSet)
  436. if err != nil {
  437. return nil, err
  438. }
  439. return vip, nil
  440. }
  441. func lastStoredHeightFor(height, lastHeightChanged int64) int64 {
  442. checkpointHeight := height - height%valSetCheckpointInterval
  443. return tmmath.MaxInt64(checkpointHeight, lastHeightChanged)
  444. }
  445. // CONTRACT: Returned ValidatorsInfo can be mutated.
  446. func loadValidatorsInfo(db dbm.DB, height int64) (*tmstate.ValidatorsInfo, error) {
  447. buf, err := db.Get(validatorsKey(height))
  448. if err != nil {
  449. return nil, err
  450. }
  451. if len(buf) == 0 {
  452. return nil, errors.New("value retrieved from db is empty")
  453. }
  454. v := new(tmstate.ValidatorsInfo)
  455. err = v.Unmarshal(buf)
  456. if err != nil {
  457. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  458. panic(fmt.Sprintf("data has been corrupted or its spec has changed: %+v", err))
  459. }
  460. // TODO: ensure that buf is completely read.
  461. return v, nil
  462. }
  463. // saveValidatorsInfo persists the validator set.
  464. //
  465. // `height` is the effective height for which the validator is responsible for
  466. // signing. It should be called from s.Save(), right before the state itself is
  467. // persisted.
  468. func (store dbStore) saveValidatorsInfo(
  469. height, lastHeightChanged int64,
  470. valSet *types.ValidatorSet,
  471. batch dbm.Batch,
  472. ) error {
  473. if lastHeightChanged > height {
  474. return errors.New("lastHeightChanged cannot be greater than ValidatorsInfo height")
  475. }
  476. valInfo := &tmstate.ValidatorsInfo{
  477. LastHeightChanged: lastHeightChanged,
  478. }
  479. // Only persist validator set if it was updated or checkpoint height (see
  480. // valSetCheckpointInterval) is reached.
  481. if height == lastHeightChanged || height%valSetCheckpointInterval == 0 {
  482. pv, err := valSet.ToProto()
  483. if err != nil {
  484. return err
  485. }
  486. valInfo.ValidatorSet = pv
  487. }
  488. bz, err := valInfo.Marshal()
  489. if err != nil {
  490. return err
  491. }
  492. return batch.Set(validatorsKey(height), bz)
  493. }
  494. //-----------------------------------------------------------------------------
  495. // ConsensusParamsInfo represents the latest consensus params, or the last height it changed
  496. // Allocate empty Consensus params at compile time to avoid multiple allocations during runtime
  497. var (
  498. empty = types.ConsensusParams{}
  499. emptypb = tmproto.ConsensusParams{}
  500. )
  501. // LoadConsensusParams loads the ConsensusParams for a given height.
  502. func (store dbStore) LoadConsensusParams(height int64) (types.ConsensusParams, error) {
  503. paramsInfo, err := store.loadConsensusParamsInfo(height)
  504. if err != nil {
  505. return empty, fmt.Errorf("could not find consensus params for height #%d: %w", height, err)
  506. }
  507. if paramsInfo.ConsensusParams.Equal(&emptypb) {
  508. paramsInfo2, err := store.loadConsensusParamsInfo(paramsInfo.LastHeightChanged)
  509. if err != nil {
  510. return empty, fmt.Errorf(
  511. "couldn't find consensus params at height %d (height %d was originally requested): %w",
  512. paramsInfo.LastHeightChanged,
  513. height,
  514. err,
  515. )
  516. }
  517. paramsInfo = paramsInfo2
  518. }
  519. return types.ConsensusParamsFromProto(paramsInfo.ConsensusParams), nil
  520. }
  521. func (store dbStore) loadConsensusParamsInfo(height int64) (*tmstate.ConsensusParamsInfo, error) {
  522. buf, err := store.db.Get(consensusParamsKey(height))
  523. if err != nil {
  524. return nil, err
  525. }
  526. if len(buf) == 0 {
  527. return nil, errors.New("value retrieved from db is empty")
  528. }
  529. paramsInfo := new(tmstate.ConsensusParamsInfo)
  530. if err = paramsInfo.Unmarshal(buf); err != nil {
  531. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  532. panic(fmt.Sprintf(`data has been corrupted or its spec has changed: %+v`, err))
  533. }
  534. // TODO: ensure that buf is completely read.
  535. return paramsInfo, nil
  536. }
  537. // saveConsensusParamsInfo persists the consensus params for the next block to disk.
  538. // It should be called from s.Save(), right before the state itself is persisted.
  539. // If the consensus params did not change after processing the latest block,
  540. // only the last height for which they changed is persisted.
  541. func (store dbStore) saveConsensusParamsInfo(
  542. nextHeight, changeHeight int64,
  543. params types.ConsensusParams,
  544. batch dbm.Batch,
  545. ) error {
  546. paramsInfo := &tmstate.ConsensusParamsInfo{
  547. LastHeightChanged: changeHeight,
  548. }
  549. if changeHeight == nextHeight {
  550. paramsInfo.ConsensusParams = params.ToProto()
  551. }
  552. bz, err := paramsInfo.Marshal()
  553. if err != nil {
  554. return err
  555. }
  556. return batch.Set(consensusParamsKey(nextHeight), bz)
  557. }
  558. func (store dbStore) Close() error {
  559. return store.db.Close()
  560. }