You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

670 lines
21 KiB

7 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
7 years ago
7 years ago
7 years ago
7 years ago
7 years ago
  1. package state
  2. import (
  3. "bytes"
  4. "errors"
  5. "fmt"
  6. "github.com/gogo/protobuf/proto"
  7. "github.com/google/orderedcode"
  8. dbm "github.com/tendermint/tm-db"
  9. abci "github.com/tendermint/tendermint/abci/types"
  10. tmmath "github.com/tendermint/tendermint/libs/math"
  11. tmos "github.com/tendermint/tendermint/libs/os"
  12. tmstate "github.com/tendermint/tendermint/proto/tendermint/state"
  13. tmproto "github.com/tendermint/tendermint/proto/tendermint/types"
  14. "github.com/tendermint/tendermint/types"
  15. )
  16. const (
  17. // persist validators every valSetCheckpointInterval blocks to avoid
  18. // LoadValidators taking too much time.
  19. // https://github.com/tendermint/tendermint/pull/3438
  20. // 100000 results in ~ 100ms to get 100 validators (see BenchmarkLoadValidators)
  21. valSetCheckpointInterval = 100000
  22. )
  23. //------------------------------------------------------------------------
  24. const (
  25. // prefixes are unique across all tm db's
  26. prefixValidators = int64(5)
  27. prefixConsensusParams = int64(6)
  28. prefixABCIResponses = int64(7)
  29. prefixState = int64(8)
  30. )
  31. func encodeKey(prefix int64, height int64) []byte {
  32. res, err := orderedcode.Append(nil, prefix, height)
  33. if err != nil {
  34. panic(err)
  35. }
  36. return res
  37. }
  38. func validatorsKey(height int64) []byte {
  39. return encodeKey(prefixValidators, height)
  40. }
  41. func consensusParamsKey(height int64) []byte {
  42. return encodeKey(prefixConsensusParams, height)
  43. }
  44. func abciResponsesKey(height int64) []byte {
  45. return encodeKey(prefixABCIResponses, height)
  46. }
  47. // stateKey should never change after being set in init()
  48. var stateKey []byte
  49. func init() {
  50. var err error
  51. stateKey, err = orderedcode.Append(nil, prefixState)
  52. if err != nil {
  53. panic(err)
  54. }
  55. }
  56. //----------------------
  57. //go:generate mockery --case underscore --name Store
  58. // Store defines the state store interface
  59. //
  60. // It is used to retrieve current state and save and load ABCI responses,
  61. // validators and consensus parameters
  62. type Store interface {
  63. // Load loads the current state of the blockchain
  64. Load() (State, error)
  65. // LoadValidators loads the validator set at a given height
  66. LoadValidators(int64) (*types.ValidatorSet, error)
  67. // LoadABCIResponses loads the abciResponse for a given height
  68. LoadABCIResponses(int64) (*tmstate.ABCIResponses, error)
  69. // LoadConsensusParams loads the consensus params for a given height
  70. LoadConsensusParams(int64) (types.ConsensusParams, error)
  71. // Save overwrites the previous state with the updated one
  72. Save(State) error
  73. // SaveABCIResponses saves ABCIResponses for a given height
  74. SaveABCIResponses(int64, *tmstate.ABCIResponses) error
  75. // SaveValidatorSet saves the validator set at a given height
  76. SaveValidatorSets(int64, int64, *types.ValidatorSet) error
  77. // Bootstrap is used for bootstrapping state when not starting from a initial height.
  78. Bootstrap(State) error
  79. // PruneStates takes the height from which to prune up to (exclusive)
  80. PruneStates(int64) error
  81. }
  82. // dbStore wraps a db (github.com/tendermint/tm-db)
  83. type dbStore struct {
  84. db dbm.DB
  85. }
  86. var _ Store = (*dbStore)(nil)
  87. // NewStore creates the dbStore of the state pkg.
  88. func NewStore(db dbm.DB) Store {
  89. return dbStore{db}
  90. }
  91. // LoadState loads the State from the database.
  92. func (store dbStore) Load() (State, error) {
  93. return store.loadState(stateKey)
  94. }
  95. func (store dbStore) loadState(key []byte) (state State, err error) {
  96. buf, err := store.db.Get(key)
  97. if err != nil {
  98. return state, err
  99. }
  100. if len(buf) == 0 {
  101. return state, nil
  102. }
  103. sp := new(tmstate.State)
  104. err = proto.Unmarshal(buf, sp)
  105. if err != nil {
  106. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  107. tmos.Exit(fmt.Sprintf(`LoadState: Data has been corrupted or its spec has changed:
  108. %v\n`, err))
  109. }
  110. sm, err := StateFromProto(sp)
  111. if err != nil {
  112. return state, err
  113. }
  114. return *sm, nil
  115. }
  116. // Save persists the State, the ValidatorsInfo, and the ConsensusParamsInfo to the database.
  117. // This flushes the writes (e.g. calls SetSync).
  118. func (store dbStore) Save(state State) error {
  119. return store.save(state, stateKey)
  120. }
  121. func (store dbStore) save(state State, key []byte) error {
  122. batch := store.db.NewBatch()
  123. defer batch.Close()
  124. nextHeight := state.LastBlockHeight + 1
  125. // If first block, save validators for the block.
  126. if nextHeight == 1 {
  127. nextHeight = state.InitialHeight
  128. // This extra logic due to Tendermint validator set changes being delayed 1 block.
  129. // It may get overwritten due to InitChain validator updates.
  130. if err := store.saveValidatorsInfo(nextHeight, nextHeight, state.Validators, batch); err != nil {
  131. return err
  132. }
  133. }
  134. // Save next validators.
  135. err := store.saveValidatorsInfo(nextHeight+1, state.LastHeightValidatorsChanged, state.NextValidators, batch)
  136. if err != nil {
  137. return err
  138. }
  139. // Save next consensus params.
  140. if err := store.saveConsensusParamsInfo(nextHeight,
  141. state.LastHeightConsensusParamsChanged, state.ConsensusParams, batch); err != nil {
  142. return err
  143. }
  144. if err := batch.Set(key, state.Bytes()); err != nil {
  145. return err
  146. }
  147. return batch.WriteSync()
  148. }
  149. // BootstrapState saves a new state, used e.g. by state sync when starting from non-zero height.
  150. func (store dbStore) Bootstrap(state State) error {
  151. height := state.LastBlockHeight + 1
  152. if height == 1 {
  153. height = state.InitialHeight
  154. }
  155. batch := store.db.NewBatch()
  156. defer batch.Close()
  157. if height > 1 && !state.LastValidators.IsNilOrEmpty() {
  158. if err := store.saveValidatorsInfo(height-1, height-1, state.LastValidators, batch); err != nil {
  159. return err
  160. }
  161. }
  162. if err := store.saveValidatorsInfo(height, height, state.Validators, batch); err != nil {
  163. return err
  164. }
  165. if err := store.saveValidatorsInfo(height+1, height+1, state.NextValidators, batch); err != nil {
  166. return err
  167. }
  168. if err := store.saveConsensusParamsInfo(height,
  169. state.LastHeightConsensusParamsChanged, state.ConsensusParams, batch); err != nil {
  170. return err
  171. }
  172. if err := batch.Set(stateKey, state.Bytes()); err != nil {
  173. return err
  174. }
  175. return batch.WriteSync()
  176. }
  177. // PruneStates deletes states up to the height specified (exclusive). It is not
  178. // guaranteed to delete all states, since the last checkpointed state and states being pointed to by
  179. // e.g. `LastHeightChanged` must remain. The state at retain height must also exist.
  180. // Pruning is done in descending order.
  181. func (store dbStore) PruneStates(retainHeight int64) error {
  182. if retainHeight <= 0 {
  183. return fmt.Errorf("height %v must be greater than 0", retainHeight)
  184. }
  185. // NOTE: We need to prune consensus params first because the validator
  186. // sets have always one extra height. If validator sets were pruned first
  187. // we could get a situation where we prune up to the last validator set
  188. // yet don't have the respective consensus params at that height and thus
  189. // return an error
  190. if err := store.pruneConsensusParams(retainHeight); err != nil {
  191. return err
  192. }
  193. if err := store.pruneValidatorSets(retainHeight); err != nil {
  194. return err
  195. }
  196. if err := store.pruneABCIResponses(retainHeight); err != nil {
  197. return err
  198. }
  199. return nil
  200. }
  201. // pruneValidatorSets calls a reverse iterator from base height to retain height (exclusive), deleting
  202. // all validator sets in between. Due to the fact that most validator sets stored reference an earlier
  203. // validator set, it is likely that there will remain one validator set left after pruning.
  204. func (store dbStore) pruneValidatorSets(retainHeight int64) error {
  205. valInfo, err := loadValidatorsInfo(store.db, retainHeight)
  206. if err != nil {
  207. return fmt.Errorf("validators at height %v not found: %w", retainHeight, err)
  208. }
  209. // We will prune up to the validator set at the given "height". As we don't save validator sets every
  210. // height but only when they change or at a check point, it is likely that the validator set at the height
  211. // we prune to is empty and thus dependent on the validator set saved at a previous height. We must find
  212. // that validator set and make sure it is not pruned.
  213. lastRecordedValSetHeight := lastStoredHeightFor(retainHeight, valInfo.LastHeightChanged)
  214. lastRecordedValSet, err := loadValidatorsInfo(store.db, lastRecordedValSetHeight)
  215. if err != nil || lastRecordedValSet.ValidatorSet == nil {
  216. return fmt.Errorf("couldn't find validators at height %d (height %d was originally requested): %w",
  217. lastStoredHeightFor(retainHeight, valInfo.LastHeightChanged),
  218. retainHeight,
  219. err,
  220. )
  221. }
  222. // if this is not equal to the retain height, prune from the retain height to the height above
  223. // the last saved validator set. This way we can skip over the dependent validator set.
  224. if lastRecordedValSetHeight < retainHeight {
  225. err := store.pruneRange(
  226. validatorsKey(lastRecordedValSetHeight+1),
  227. validatorsKey(retainHeight),
  228. )
  229. if err != nil {
  230. return err
  231. }
  232. }
  233. // prune all the validators sets up to last saved validator set
  234. return store.pruneRange(
  235. validatorsKey(1),
  236. validatorsKey(lastRecordedValSetHeight),
  237. )
  238. }
  239. // pruneConsensusParams calls a reverse iterator from base height to retain height batch deleting
  240. // all consensus params in between. If the consensus params at the new base height is dependent
  241. // on a prior height then this will keep that lower height too.
  242. func (store dbStore) pruneConsensusParams(retainHeight int64) error {
  243. paramsInfo, err := store.loadConsensusParamsInfo(retainHeight)
  244. if err != nil {
  245. return fmt.Errorf("consensus params at height %v not found: %w", retainHeight, err)
  246. }
  247. // As we don't save the consensus params at every height, only when there is a consensus params change,
  248. // we must not prune (or save) the last consensus params that the consensus params info at height
  249. // is dependent on.
  250. if paramsInfo.ConsensusParams.Equal(&tmproto.ConsensusParams{}) {
  251. // sanity check that the consensus params at the last height it was changed is there
  252. lastRecordedConsensusParams, err := store.loadConsensusParamsInfo(paramsInfo.LastHeightChanged)
  253. if err != nil || lastRecordedConsensusParams.ConsensusParams.Equal(&tmproto.ConsensusParams{}) {
  254. return fmt.Errorf(
  255. "couldn't find consensus params at height %d (height %d was originally requested): %w",
  256. paramsInfo.LastHeightChanged,
  257. retainHeight,
  258. err,
  259. )
  260. }
  261. // prune the params above the height with which it last changed and below the retain height.
  262. err = store.pruneRange(
  263. consensusParamsKey(paramsInfo.LastHeightChanged+1),
  264. consensusParamsKey(retainHeight),
  265. )
  266. if err != nil {
  267. return err
  268. }
  269. }
  270. // prune all the consensus params up to either the last height the params changed or if the params
  271. // last changed at the retain height, then up to the retain height.
  272. return store.pruneRange(
  273. consensusParamsKey(1),
  274. consensusParamsKey(paramsInfo.LastHeightChanged),
  275. )
  276. }
  277. // pruneABCIResponses calls a reverse iterator from base height to retain height batch deleting
  278. // all abci responses in between
  279. func (store dbStore) pruneABCIResponses(height int64) error {
  280. return store.pruneRange(abciResponsesKey(1), abciResponsesKey(height))
  281. }
  282. // pruneRange is a generic function for deleting a range of keys in reverse order.
  283. // we keep filling up batches of at most 1000 keys, perform a deletion and continue until
  284. // we have gone through all of keys in the range. This avoids doing any writes whilst
  285. // iterating.
  286. func (store dbStore) pruneRange(start []byte, end []byte) error {
  287. var err error
  288. batch := store.db.NewBatch()
  289. defer batch.Close()
  290. end, err = store.reverseBatchDelete(batch, start, end)
  291. if err != nil {
  292. return err
  293. }
  294. // iterate until the last batch of the pruning range in which case we will perform a
  295. // write sync
  296. for !bytes.Equal(start, end) {
  297. if err := batch.Write(); err != nil {
  298. return err
  299. }
  300. if err := batch.Close(); err != nil {
  301. return err
  302. }
  303. batch = store.db.NewBatch()
  304. // fill a new batch of keys for deletion over the remainding range
  305. end, err = store.reverseBatchDelete(batch, start, end)
  306. if err != nil {
  307. return err
  308. }
  309. }
  310. return batch.WriteSync()
  311. }
  312. // reverseBatchDelete runs a reverse iterator (from end to start) filling up a batch until either
  313. // (a) the iterator reaches the start or (b) the iterator has added a 1000 keys (this avoids the
  314. // batch from growing too large)
  315. func (store dbStore) reverseBatchDelete(batch dbm.Batch, start, end []byte) ([]byte, error) {
  316. iter, err := store.db.ReverseIterator(start, end)
  317. if err != nil {
  318. return end, fmt.Errorf("iterator error: %w", err)
  319. }
  320. defer iter.Close()
  321. size := 0
  322. for ; iter.Valid(); iter.Next() {
  323. if err := batch.Delete(iter.Key()); err != nil {
  324. return end, fmt.Errorf("pruning error at key %X: %w", iter.Key(), err)
  325. }
  326. // avoid batches growing too large by capping them
  327. size++
  328. if size == 1000 {
  329. return iter.Key(), iter.Error()
  330. }
  331. }
  332. return start, iter.Error()
  333. }
  334. //------------------------------------------------------------------------
  335. // ABCIResponsesResultsHash returns the root hash of a Merkle tree of
  336. // ResponseDeliverTx responses (see ABCIResults.Hash)
  337. //
  338. // See merkle.SimpleHashFromByteSlices
  339. func ABCIResponsesResultsHash(ar *tmstate.ABCIResponses) []byte {
  340. return types.NewResults(ar.DeliverTxs).Hash()
  341. }
  342. // LoadABCIResponses loads the ABCIResponses for the given height from the
  343. // database. If not found, ErrNoABCIResponsesForHeight is returned.
  344. //
  345. // This is useful for recovering from crashes where we called app.Commit and
  346. // before we called s.Save(). It can also be used to produce Merkle proofs of
  347. // the result of txs.
  348. func (store dbStore) LoadABCIResponses(height int64) (*tmstate.ABCIResponses, error) {
  349. buf, err := store.db.Get(abciResponsesKey(height))
  350. if err != nil {
  351. return nil, err
  352. }
  353. if len(buf) == 0 {
  354. return nil, ErrNoABCIResponsesForHeight{height}
  355. }
  356. abciResponses := new(tmstate.ABCIResponses)
  357. err = abciResponses.Unmarshal(buf)
  358. if err != nil {
  359. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  360. tmos.Exit(fmt.Sprintf(`LoadABCIResponses: Data has been corrupted or its spec has
  361. changed: %v\n`, err))
  362. }
  363. // TODO: ensure that buf is completely read.
  364. return abciResponses, nil
  365. }
  366. // SaveABCIResponses persists the ABCIResponses to the database.
  367. // This is useful in case we crash after app.Commit and before s.Save().
  368. // Responses are indexed by height so they can also be loaded later to produce
  369. // Merkle proofs.
  370. //
  371. // Exposed for testing.
  372. func (store dbStore) SaveABCIResponses(height int64, abciResponses *tmstate.ABCIResponses) error {
  373. return store.saveABCIResponses(height, abciResponses)
  374. }
  375. func (store dbStore) saveABCIResponses(height int64, abciResponses *tmstate.ABCIResponses) error {
  376. var dtxs []*abci.ResponseDeliverTx
  377. // strip nil values,
  378. for _, tx := range abciResponses.DeliverTxs {
  379. if tx != nil {
  380. dtxs = append(dtxs, tx)
  381. }
  382. }
  383. abciResponses.DeliverTxs = dtxs
  384. bz, err := abciResponses.Marshal()
  385. if err != nil {
  386. return err
  387. }
  388. return store.db.SetSync(abciResponsesKey(height), bz)
  389. }
  390. // SaveValidatorSets is used to save the validator set over multiple heights.
  391. // It is exposed so that a backfill operation during state sync can populate
  392. // the store with the necessary amount of validator sets to verify any evidence
  393. // it may encounter.
  394. func (store dbStore) SaveValidatorSets(lowerHeight, upperHeight int64, vals *types.ValidatorSet) error {
  395. batch := store.db.NewBatch()
  396. defer batch.Close()
  397. // batch together all the validator sets from lowerHeight to upperHeight
  398. for height := lowerHeight; height <= upperHeight; height++ {
  399. if err := store.saveValidatorsInfo(height, lowerHeight, vals, batch); err != nil {
  400. return err
  401. }
  402. }
  403. return batch.WriteSync()
  404. }
  405. //-----------------------------------------------------------------------------
  406. // LoadValidators loads the ValidatorSet for a given height.
  407. // Returns ErrNoValSetForHeight if the validator set can't be found for this height.
  408. func (store dbStore) LoadValidators(height int64) (*types.ValidatorSet, error) {
  409. valInfo, err := loadValidatorsInfo(store.db, height)
  410. if err != nil {
  411. return nil, ErrNoValSetForHeight{height}
  412. }
  413. if valInfo.ValidatorSet == nil {
  414. lastStoredHeight := lastStoredHeightFor(height, valInfo.LastHeightChanged)
  415. valInfo2, err := loadValidatorsInfo(store.db, lastStoredHeight)
  416. if err != nil || valInfo2.ValidatorSet == nil {
  417. return nil,
  418. fmt.Errorf("couldn't find validators at height %d (height %d was originally requested): %w",
  419. lastStoredHeight,
  420. height,
  421. err,
  422. )
  423. }
  424. vs, err := types.ValidatorSetFromProto(valInfo2.ValidatorSet)
  425. if err != nil {
  426. return nil, err
  427. }
  428. vs.IncrementProposerPriority(tmmath.SafeConvertInt32(height - lastStoredHeight)) // mutate
  429. vi2, err := vs.ToProto()
  430. if err != nil {
  431. return nil, err
  432. }
  433. valInfo2.ValidatorSet = vi2
  434. valInfo = valInfo2
  435. }
  436. vip, err := types.ValidatorSetFromProto(valInfo.ValidatorSet)
  437. if err != nil {
  438. return nil, err
  439. }
  440. return vip, nil
  441. }
  442. func lastStoredHeightFor(height, lastHeightChanged int64) int64 {
  443. checkpointHeight := height - height%valSetCheckpointInterval
  444. return tmmath.MaxInt64(checkpointHeight, lastHeightChanged)
  445. }
  446. // CONTRACT: Returned ValidatorsInfo can be mutated.
  447. func loadValidatorsInfo(db dbm.DB, height int64) (*tmstate.ValidatorsInfo, error) {
  448. buf, err := db.Get(validatorsKey(height))
  449. if err != nil {
  450. return nil, err
  451. }
  452. if len(buf) == 0 {
  453. return nil, errors.New("value retrieved from db is empty")
  454. }
  455. v := new(tmstate.ValidatorsInfo)
  456. err = v.Unmarshal(buf)
  457. if err != nil {
  458. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  459. tmos.Exit(fmt.Sprintf(`LoadValidators: Data has been corrupted or its spec has changed:
  460. %v\n`, err))
  461. }
  462. // TODO: ensure that buf is completely read.
  463. return v, nil
  464. }
  465. // saveValidatorsInfo persists the validator set.
  466. //
  467. // `height` is the effective height for which the validator is responsible for
  468. // signing. It should be called from s.Save(), right before the state itself is
  469. // persisted.
  470. func (store dbStore) saveValidatorsInfo(
  471. height, lastHeightChanged int64,
  472. valSet *types.ValidatorSet,
  473. batch dbm.Batch,
  474. ) error {
  475. if lastHeightChanged > height {
  476. return errors.New("lastHeightChanged cannot be greater than ValidatorsInfo height")
  477. }
  478. valInfo := &tmstate.ValidatorsInfo{
  479. LastHeightChanged: lastHeightChanged,
  480. }
  481. // Only persist validator set if it was updated or checkpoint height (see
  482. // valSetCheckpointInterval) is reached.
  483. if height == lastHeightChanged || height%valSetCheckpointInterval == 0 {
  484. pv, err := valSet.ToProto()
  485. if err != nil {
  486. return err
  487. }
  488. valInfo.ValidatorSet = pv
  489. }
  490. bz, err := valInfo.Marshal()
  491. if err != nil {
  492. return err
  493. }
  494. return batch.Set(validatorsKey(height), bz)
  495. }
  496. //-----------------------------------------------------------------------------
  497. // ConsensusParamsInfo represents the latest consensus params, or the last height it changed
  498. // Allocate empty Consensus params at compile time to avoid multiple allocations during runtime
  499. var (
  500. empty = types.ConsensusParams{}
  501. emptypb = tmproto.ConsensusParams{}
  502. )
  503. // LoadConsensusParams loads the ConsensusParams for a given height.
  504. func (store dbStore) LoadConsensusParams(height int64) (types.ConsensusParams, error) {
  505. paramsInfo, err := store.loadConsensusParamsInfo(height)
  506. if err != nil {
  507. return empty, fmt.Errorf("could not find consensus params for height #%d: %w", height, err)
  508. }
  509. if paramsInfo.ConsensusParams.Equal(&emptypb) {
  510. paramsInfo2, err := store.loadConsensusParamsInfo(paramsInfo.LastHeightChanged)
  511. if err != nil {
  512. return empty, fmt.Errorf(
  513. "couldn't find consensus params at height %d (height %d was originally requested): %w",
  514. paramsInfo.LastHeightChanged,
  515. height,
  516. err,
  517. )
  518. }
  519. paramsInfo = paramsInfo2
  520. }
  521. return types.ConsensusParamsFromProto(paramsInfo.ConsensusParams), nil
  522. }
  523. func (store dbStore) loadConsensusParamsInfo(height int64) (*tmstate.ConsensusParamsInfo, error) {
  524. buf, err := store.db.Get(consensusParamsKey(height))
  525. if err != nil {
  526. return nil, err
  527. }
  528. if len(buf) == 0 {
  529. return nil, errors.New("value retrieved from db is empty")
  530. }
  531. paramsInfo := new(tmstate.ConsensusParamsInfo)
  532. if err = paramsInfo.Unmarshal(buf); err != nil {
  533. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  534. tmos.Exit(fmt.Sprintf(`LoadConsensusParams: Data has been corrupted or its spec has changed:
  535. %v\n`, err))
  536. }
  537. // TODO: ensure that buf is completely read.
  538. return paramsInfo, nil
  539. }
  540. // saveConsensusParamsInfo persists the consensus params for the next block to disk.
  541. // It should be called from s.Save(), right before the state itself is persisted.
  542. // If the consensus params did not change after processing the latest block,
  543. // only the last height for which they changed is persisted.
  544. func (store dbStore) saveConsensusParamsInfo(
  545. nextHeight, changeHeight int64,
  546. params types.ConsensusParams,
  547. batch dbm.Batch,
  548. ) error {
  549. paramsInfo := &tmstate.ConsensusParamsInfo{
  550. LastHeightChanged: changeHeight,
  551. }
  552. if changeHeight == nextHeight {
  553. paramsInfo.ConsensusParams = params.ToProto()
  554. }
  555. bz, err := paramsInfo.Marshal()
  556. if err != nil {
  557. return err
  558. }
  559. err = batch.Set(consensusParamsKey(nextHeight), bz)
  560. if err != nil {
  561. return err
  562. }
  563. return nil
  564. }