You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

631 lines
17 KiB

blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
10 years ago
  1. package store
  2. import (
  3. "bytes"
  4. "fmt"
  5. "strconv"
  6. "github.com/gogo/protobuf/proto"
  7. "github.com/google/orderedcode"
  8. dbm "github.com/tendermint/tm-db"
  9. tmproto "github.com/tendermint/tendermint/proto/tendermint/types"
  10. "github.com/tendermint/tendermint/types"
  11. )
  12. /*
  13. BlockStore is a simple low level store for blocks.
  14. There are three types of information stored:
  15. - BlockMeta: Meta information about each block
  16. - Block part: Parts of each block, aggregated w/ PartSet
  17. - Commit: The commit part of each block, for gossiping precommit votes
  18. Currently the precommit signatures are duplicated in the Block parts as
  19. well as the Commit. In the future this may change, perhaps by moving
  20. the Commit data outside the Block. (TODO)
  21. The store can be assumed to contain all contiguous blocks between base and height (inclusive).
  22. // NOTE: BlockStore methods will panic if they encounter errors
  23. // deserializing loaded data, indicating probable corruption on disk.
  24. */
  25. type BlockStore struct {
  26. db dbm.DB
  27. }
  28. // NewBlockStore returns a new BlockStore with the given DB,
  29. // initialized to the last height that was committed to the DB.
  30. func NewBlockStore(db dbm.DB) *BlockStore {
  31. return &BlockStore{db}
  32. }
  33. // Base returns the first known contiguous block height, or 0 for empty block stores.
  34. func (bs *BlockStore) Base() int64 {
  35. iter, err := bs.db.Iterator(
  36. blockMetaKey(1),
  37. blockMetaKey(1<<63-1),
  38. )
  39. if err != nil {
  40. panic(err)
  41. }
  42. defer iter.Close()
  43. if iter.Valid() {
  44. height, err := decodeBlockMetaKey(iter.Key())
  45. if err == nil {
  46. return height
  47. }
  48. }
  49. if err := iter.Error(); err != nil {
  50. panic(err)
  51. }
  52. return 0
  53. }
  54. // Height returns the last known contiguous block height, or 0 for empty block stores.
  55. func (bs *BlockStore) Height() int64 {
  56. iter, err := bs.db.ReverseIterator(
  57. blockMetaKey(1),
  58. blockMetaKey(1<<63-1),
  59. )
  60. if err != nil {
  61. panic(err)
  62. }
  63. defer iter.Close()
  64. if iter.Valid() {
  65. height, err := decodeBlockMetaKey(iter.Key())
  66. if err == nil {
  67. return height
  68. }
  69. }
  70. if err := iter.Error(); err != nil {
  71. panic(err)
  72. }
  73. return 0
  74. }
  75. // Size returns the number of blocks in the block store.
  76. func (bs *BlockStore) Size() int64 {
  77. height := bs.Height()
  78. if height == 0 {
  79. return 0
  80. }
  81. return height + 1 - bs.Base()
  82. }
  83. // LoadBase atomically loads the base block meta, or returns nil if no base is found.
  84. func (bs *BlockStore) LoadBaseMeta() *types.BlockMeta {
  85. iter, err := bs.db.Iterator(
  86. blockMetaKey(1),
  87. blockMetaKey(1<<63-1),
  88. )
  89. if err != nil {
  90. return nil
  91. }
  92. defer iter.Close()
  93. if iter.Valid() {
  94. var pbbm = new(tmproto.BlockMeta)
  95. err = proto.Unmarshal(iter.Value(), pbbm)
  96. if err != nil {
  97. panic(fmt.Errorf("unmarshal to tmproto.BlockMeta: %w", err))
  98. }
  99. blockMeta, err := types.BlockMetaFromProto(pbbm)
  100. if err != nil {
  101. panic(fmt.Errorf("error from proto blockMeta: %w", err))
  102. }
  103. return blockMeta
  104. }
  105. return nil
  106. }
  107. // LoadBlock returns the block with the given height.
  108. // If no block is found for that height, it returns nil.
  109. func (bs *BlockStore) LoadBlock(height int64) *types.Block {
  110. var blockMeta = bs.LoadBlockMeta(height)
  111. if blockMeta == nil {
  112. return nil
  113. }
  114. pbb := new(tmproto.Block)
  115. buf := []byte{}
  116. for i := 0; i < int(blockMeta.BlockID.PartSetHeader.Total); i++ {
  117. part := bs.LoadBlockPart(height, i)
  118. // If the part is missing (e.g. since it has been deleted after we
  119. // loaded the block meta) we consider the whole block to be missing.
  120. if part == nil {
  121. return nil
  122. }
  123. buf = append(buf, part.Bytes...)
  124. }
  125. err := proto.Unmarshal(buf, pbb)
  126. if err != nil {
  127. // NOTE: The existence of meta should imply the existence of the
  128. // block. So, make sure meta is only saved after blocks are saved.
  129. panic(fmt.Sprintf("Error reading block: %v", err))
  130. }
  131. block, err := types.BlockFromProto(pbb)
  132. if err != nil {
  133. panic(fmt.Errorf("error from proto block: %w", err))
  134. }
  135. return block
  136. }
  137. // LoadBlockByHash returns the block with the given hash.
  138. // If no block is found for that hash, it returns nil.
  139. // Panics if it fails to parse height associated with the given hash.
  140. func (bs *BlockStore) LoadBlockByHash(hash []byte) *types.Block {
  141. bz, err := bs.db.Get(blockHashKey(hash))
  142. if err != nil {
  143. panic(err)
  144. }
  145. if len(bz) == 0 {
  146. return nil
  147. }
  148. s := string(bz)
  149. height, err := strconv.ParseInt(s, 10, 64)
  150. if err != nil {
  151. panic(fmt.Sprintf("failed to extract height from %s: %v", s, err))
  152. }
  153. return bs.LoadBlock(height)
  154. }
  155. // LoadBlockPart returns the Part at the given index
  156. // from the block at the given height.
  157. // If no part is found for the given height and index, it returns nil.
  158. func (bs *BlockStore) LoadBlockPart(height int64, index int) *types.Part {
  159. var pbpart = new(tmproto.Part)
  160. bz, err := bs.db.Get(blockPartKey(height, index))
  161. if err != nil {
  162. panic(err)
  163. }
  164. if len(bz) == 0 {
  165. return nil
  166. }
  167. err = proto.Unmarshal(bz, pbpart)
  168. if err != nil {
  169. panic(fmt.Errorf("unmarshal to tmproto.Part failed: %w", err))
  170. }
  171. part, err := types.PartFromProto(pbpart)
  172. if err != nil {
  173. panic(fmt.Sprintf("Error reading block part: %v", err))
  174. }
  175. return part
  176. }
  177. // LoadBlockMeta returns the BlockMeta for the given height.
  178. // If no block is found for the given height, it returns nil.
  179. func (bs *BlockStore) LoadBlockMeta(height int64) *types.BlockMeta {
  180. var pbbm = new(tmproto.BlockMeta)
  181. bz, err := bs.db.Get(blockMetaKey(height))
  182. if err != nil {
  183. panic(err)
  184. }
  185. if len(bz) == 0 {
  186. return nil
  187. }
  188. err = proto.Unmarshal(bz, pbbm)
  189. if err != nil {
  190. panic(fmt.Errorf("unmarshal to tmproto.BlockMeta: %w", err))
  191. }
  192. blockMeta, err := types.BlockMetaFromProto(pbbm)
  193. if err != nil {
  194. panic(fmt.Errorf("error from proto blockMeta: %w", err))
  195. }
  196. return blockMeta
  197. }
  198. // LoadBlockCommit returns the Commit for the given height.
  199. // This commit consists of the +2/3 and other Precommit-votes for block at `height`,
  200. // and it comes from the block.LastCommit for `height+1`.
  201. // If no commit is found for the given height, it returns nil.
  202. func (bs *BlockStore) LoadBlockCommit(height int64) *types.Commit {
  203. var pbc = new(tmproto.Commit)
  204. bz, err := bs.db.Get(blockCommitKey(height))
  205. if err != nil {
  206. panic(err)
  207. }
  208. if len(bz) == 0 {
  209. return nil
  210. }
  211. err = proto.Unmarshal(bz, pbc)
  212. if err != nil {
  213. panic(fmt.Errorf("error reading block commit: %w", err))
  214. }
  215. commit, err := types.CommitFromProto(pbc)
  216. if err != nil {
  217. panic(fmt.Sprintf("Error reading block commit: %v", err))
  218. }
  219. return commit
  220. }
  221. // LoadSeenCommit returns the last locally seen Commit before being
  222. // cannonicalized. This is useful when we've seen a commit, but there
  223. // has not yet been a new block at `height + 1` that includes this
  224. // commit in its block.LastCommit.
  225. func (bs *BlockStore) LoadSeenCommit() *types.Commit {
  226. var pbc = new(tmproto.Commit)
  227. bz, err := bs.db.Get(seenCommitKey())
  228. if err != nil {
  229. panic(err)
  230. }
  231. if len(bz) == 0 {
  232. return nil
  233. }
  234. err = proto.Unmarshal(bz, pbc)
  235. if err != nil {
  236. panic(fmt.Sprintf("error reading block seen commit: %v", err))
  237. }
  238. commit, err := types.CommitFromProto(pbc)
  239. if err != nil {
  240. panic(fmt.Errorf("error from proto commit: %w", err))
  241. }
  242. return commit
  243. }
  244. // PruneBlocks removes block up to (but not including) a height. It returns the number of blocks pruned.
  245. func (bs *BlockStore) PruneBlocks(height int64) (uint64, error) {
  246. if height <= 0 {
  247. return 0, fmt.Errorf("height must be greater than 0")
  248. }
  249. if height > bs.Height() {
  250. return 0, fmt.Errorf("height must be equal to or less than the latest height %d", bs.Height())
  251. }
  252. // when removing the block meta, use the hash to remove the hash key at the same time
  253. removeBlockHash := func(key, value []byte, batch dbm.Batch) error {
  254. // unmarshal block meta
  255. var pbbm = new(tmproto.BlockMeta)
  256. err := proto.Unmarshal(value, pbbm)
  257. if err != nil {
  258. return fmt.Errorf("unmarshal to tmproto.BlockMeta: %w", err)
  259. }
  260. blockMeta, err := types.BlockMetaFromProto(pbbm)
  261. if err != nil {
  262. return fmt.Errorf("error from proto blockMeta: %w", err)
  263. }
  264. // delete the hash key corresponding to the block meta's hash
  265. if err := batch.Delete(blockHashKey(blockMeta.BlockID.Hash)); err != nil {
  266. return fmt.Errorf("failed to delete hash key: %X: %w", blockHashKey(blockMeta.BlockID.Hash), err)
  267. }
  268. return nil
  269. }
  270. // remove block meta first as this is used to indicate whether the block exists.
  271. // For this reason, we also use ony block meta as a measure of the amount of blocks pruned
  272. pruned, err := bs.pruneRange(blockMetaKey(0), blockMetaKey(height), removeBlockHash)
  273. if err != nil {
  274. return pruned, err
  275. }
  276. if _, err := bs.pruneRange(blockPartKey(0, 0), blockPartKey(height, 0), nil); err != nil {
  277. return pruned, err
  278. }
  279. if _, err := bs.pruneRange(blockCommitKey(0), blockCommitKey(height), nil); err != nil {
  280. return pruned, err
  281. }
  282. return pruned, nil
  283. }
  284. // pruneRange is a generic function for deleting a range of values based on the lowest
  285. // height up to but excluding retainHeight. For each key/value pair, an optional hook can be
  286. // executed before the deletion itself is made. pruneRange will use batch delete to delete
  287. // keys in batches of at most 1000 keys.
  288. func (bs *BlockStore) pruneRange(
  289. start []byte,
  290. end []byte,
  291. preDeletionHook func(key, value []byte, batch dbm.Batch) error,
  292. ) (uint64, error) {
  293. var (
  294. err error
  295. pruned uint64
  296. totalPruned uint64
  297. )
  298. batch := bs.db.NewBatch()
  299. defer batch.Close()
  300. pruned, start, err = bs.batchDelete(batch, start, end, preDeletionHook)
  301. if err != nil {
  302. return totalPruned, err
  303. }
  304. // loop until we have finished iterating over all the keys by writing, opening a new batch
  305. // and incrementing through the next range of keys.
  306. for !bytes.Equal(start, end) {
  307. if err := batch.Write(); err != nil {
  308. return totalPruned, err
  309. }
  310. totalPruned += pruned
  311. if err := batch.Close(); err != nil {
  312. return totalPruned, err
  313. }
  314. batch = bs.db.NewBatch()
  315. pruned, start, err = bs.batchDelete(batch, start, end, preDeletionHook)
  316. if err != nil {
  317. return totalPruned, err
  318. }
  319. }
  320. // once we looped over all keys we do a final flush to disk
  321. if err := batch.WriteSync(); err != nil {
  322. return totalPruned, err
  323. }
  324. totalPruned += pruned
  325. return totalPruned, nil
  326. }
  327. // batchDelete runs an iterator over a set of keys, first preforming a pre deletion hook before adding it to the batch.
  328. // The function ends when either 1000 keys have been added to the batch or the iterator has reached the end.
  329. func (bs *BlockStore) batchDelete(
  330. batch dbm.Batch,
  331. start, end []byte,
  332. preDeletionHook func(key, value []byte, batch dbm.Batch) error,
  333. ) (uint64, []byte, error) {
  334. var pruned uint64
  335. iter, err := bs.db.Iterator(start, end)
  336. if err != nil {
  337. return pruned, start, err
  338. }
  339. defer iter.Close()
  340. for ; iter.Valid(); iter.Next() {
  341. key := iter.Key()
  342. if preDeletionHook != nil {
  343. if err := preDeletionHook(key, iter.Value(), batch); err != nil {
  344. return 0, start, fmt.Errorf("pruning error at key %X: %w", iter.Key(), err)
  345. }
  346. }
  347. if err := batch.Delete(key); err != nil {
  348. return 0, start, fmt.Errorf("pruning error at key %X: %w", iter.Key(), err)
  349. }
  350. pruned++
  351. if pruned == 1000 {
  352. return pruned, iter.Key(), iter.Error()
  353. }
  354. }
  355. return pruned, end, iter.Error()
  356. }
  357. // SaveBlock persists the given block, blockParts, and seenCommit to the underlying db.
  358. // blockParts: Must be parts of the block
  359. // seenCommit: The +2/3 precommits that were seen which committed at height.
  360. // If all the nodes restart after committing a block,
  361. // we need this to reload the precommits to catch-up nodes to the
  362. // most recent height. Otherwise they'd stall at H-1.
  363. func (bs *BlockStore) SaveBlock(block *types.Block, blockParts *types.PartSet, seenCommit *types.Commit) {
  364. if block == nil {
  365. panic("BlockStore can only save a non-nil block")
  366. }
  367. batch := bs.db.NewBatch()
  368. height := block.Height
  369. hash := block.Hash()
  370. if g, w := height, bs.Height()+1; bs.Base() > 0 && g != w {
  371. panic(fmt.Sprintf("BlockStore can only save contiguous blocks. Wanted %v, got %v", w, g))
  372. }
  373. if !blockParts.IsComplete() {
  374. panic("BlockStore can only save complete block part sets")
  375. }
  376. // Save block parts. This must be done before the block meta, since callers
  377. // typically load the block meta first as an indication that the block exists
  378. // and then go on to load block parts - we must make sure the block is
  379. // complete as soon as the block meta is written.
  380. for i := 0; i < int(blockParts.Total()); i++ {
  381. part := blockParts.GetPart(i)
  382. bs.saveBlockPart(height, i, part, batch)
  383. }
  384. blockMeta := types.NewBlockMeta(block, blockParts)
  385. pbm := blockMeta.ToProto()
  386. if pbm == nil {
  387. panic("nil blockmeta")
  388. }
  389. metaBytes := mustEncode(pbm)
  390. if err := batch.Set(blockMetaKey(height), metaBytes); err != nil {
  391. panic(err)
  392. }
  393. if err := batch.Set(blockHashKey(hash), []byte(fmt.Sprintf("%d", height))); err != nil {
  394. panic(err)
  395. }
  396. pbc := block.LastCommit.ToProto()
  397. blockCommitBytes := mustEncode(pbc)
  398. if err := batch.Set(blockCommitKey(height-1), blockCommitBytes); err != nil {
  399. panic(err)
  400. }
  401. // Save seen commit (seen +2/3 precommits for block)
  402. pbsc := seenCommit.ToProto()
  403. seenCommitBytes := mustEncode(pbsc)
  404. if err := batch.Set(seenCommitKey(), seenCommitBytes); err != nil {
  405. panic(err)
  406. }
  407. if err := batch.WriteSync(); err != nil {
  408. panic(err)
  409. }
  410. if err := batch.Close(); err != nil {
  411. panic(err)
  412. }
  413. }
  414. func (bs *BlockStore) saveBlockPart(height int64, index int, part *types.Part, batch dbm.Batch) {
  415. pbp, err := part.ToProto()
  416. if err != nil {
  417. panic(fmt.Errorf("unable to make part into proto: %w", err))
  418. }
  419. partBytes := mustEncode(pbp)
  420. if err := batch.Set(blockPartKey(height, index), partBytes); err != nil {
  421. panic(err)
  422. }
  423. }
  424. // SaveSeenCommit saves a seen commit, used by e.g. the state sync reactor when bootstrapping node.
  425. func (bs *BlockStore) SaveSeenCommit(height int64, seenCommit *types.Commit) error {
  426. pbc := seenCommit.ToProto()
  427. seenCommitBytes, err := proto.Marshal(pbc)
  428. if err != nil {
  429. return fmt.Errorf("unable to marshal commit: %w", err)
  430. }
  431. return bs.db.Set(seenCommitKey(), seenCommitBytes)
  432. }
  433. func (bs *BlockStore) SaveSignedHeader(sh *types.SignedHeader, blockID types.BlockID) error {
  434. // first check that the block store doesn't already have the block
  435. bz, err := bs.db.Get(blockMetaKey(sh.Height))
  436. if err != nil {
  437. return err
  438. }
  439. if bz != nil {
  440. return fmt.Errorf("block at height %d already saved", sh.Height)
  441. }
  442. // FIXME: saving signed headers although necessary for proving evidence,
  443. // doesn't have complete parity with block meta's thus block size and num
  444. // txs are filled with negative numbers. We should aim to find a solution to
  445. // this.
  446. blockMeta := &types.BlockMeta{
  447. BlockID: blockID,
  448. BlockSize: -1,
  449. Header: *sh.Header,
  450. NumTxs: -1,
  451. }
  452. batch := bs.db.NewBatch()
  453. pbm := blockMeta.ToProto()
  454. metaBytes := mustEncode(pbm)
  455. if err := batch.Set(blockMetaKey(sh.Height), metaBytes); err != nil {
  456. return fmt.Errorf("unable to save block meta: %w", err)
  457. }
  458. pbc := sh.Commit.ToProto()
  459. blockCommitBytes := mustEncode(pbc)
  460. if err := batch.Set(blockCommitKey(sh.Height), blockCommitBytes); err != nil {
  461. return fmt.Errorf("unable to save commit: %w", err)
  462. }
  463. if err := batch.WriteSync(); err != nil {
  464. return err
  465. }
  466. return batch.Close()
  467. }
  468. //---------------------------------- KEY ENCODING -----------------------------------------
  469. // key prefixes
  470. const (
  471. // prefixes are unique across all tm db's
  472. prefixBlockMeta = int64(0)
  473. prefixBlockPart = int64(1)
  474. prefixBlockCommit = int64(2)
  475. prefixSeenCommit = int64(3)
  476. prefixBlockHash = int64(4)
  477. )
  478. func blockMetaKey(height int64) []byte {
  479. key, err := orderedcode.Append(nil, prefixBlockMeta, height)
  480. if err != nil {
  481. panic(err)
  482. }
  483. return key
  484. }
  485. func decodeBlockMetaKey(key []byte) (height int64, err error) {
  486. var prefix int64
  487. remaining, err := orderedcode.Parse(string(key), &prefix, &height)
  488. if err != nil {
  489. return
  490. }
  491. if len(remaining) != 0 {
  492. return -1, fmt.Errorf("expected complete key but got remainder: %s", remaining)
  493. }
  494. if prefix != prefixBlockMeta {
  495. return -1, fmt.Errorf("incorrect prefix. Expected %v, got %v", prefixBlockMeta, prefix)
  496. }
  497. return
  498. }
  499. func blockPartKey(height int64, partIndex int) []byte {
  500. key, err := orderedcode.Append(nil, prefixBlockPart, height, int64(partIndex))
  501. if err != nil {
  502. panic(err)
  503. }
  504. return key
  505. }
  506. func blockCommitKey(height int64) []byte {
  507. key, err := orderedcode.Append(nil, prefixBlockCommit, height)
  508. if err != nil {
  509. panic(err)
  510. }
  511. return key
  512. }
  513. func seenCommitKey() []byte {
  514. key, err := orderedcode.Append(nil, prefixSeenCommit)
  515. if err != nil {
  516. panic(err)
  517. }
  518. return key
  519. }
  520. func blockHashKey(hash []byte) []byte {
  521. key, err := orderedcode.Append(nil, prefixBlockHash, string(hash))
  522. if err != nil {
  523. panic(err)
  524. }
  525. return key
  526. }
  527. //-----------------------------------------------------------------------------
  528. // mustEncode proto encodes a proto.message and panics if fails
  529. func mustEncode(pb proto.Message) []byte {
  530. bz, err := proto.Marshal(pb)
  531. if err != nil {
  532. panic(fmt.Errorf("unable to marshal: %w", err))
  533. }
  534. return bz
  535. }