You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

509 lines
15 KiB

7 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
7 years ago
7 years ago
7 years ago
7 years ago
7 years ago
  1. package state
  2. import (
  3. "fmt"
  4. "github.com/gogo/protobuf/proto"
  5. dbm "github.com/tendermint/tm-db"
  6. abci "github.com/tendermint/tendermint/abci/types"
  7. "github.com/tendermint/tendermint/crypto/merkle"
  8. tmmath "github.com/tendermint/tendermint/libs/math"
  9. tmos "github.com/tendermint/tendermint/libs/os"
  10. tmstate "github.com/tendermint/tendermint/proto/tendermint/state"
  11. tmproto "github.com/tendermint/tendermint/proto/tendermint/types"
  12. "github.com/tendermint/tendermint/types"
  13. )
  14. const (
  15. // persist validators every valSetCheckpointInterval blocks to avoid
  16. // LoadValidators taking too much time.
  17. // https://github.com/tendermint/tendermint/pull/3438
  18. // 100000 results in ~ 100ms to get 100 validators (see BenchmarkLoadValidators)
  19. valSetCheckpointInterval = 100000
  20. )
  21. //------------------------------------------------------------------------
  22. func calcValidatorsKey(height int64) []byte {
  23. return []byte(fmt.Sprintf("validatorsKey:%v", height))
  24. }
  25. func calcConsensusParamsKey(height int64) []byte {
  26. return []byte(fmt.Sprintf("consensusParamsKey:%v", height))
  27. }
  28. func calcABCIResponsesKey(height int64) []byte {
  29. return []byte(fmt.Sprintf("abciResponsesKey:%v", height))
  30. }
  31. // LoadStateFromDBOrGenesisFile loads the most recent state from the database,
  32. // or creates a new one from the given genesisFilePath and persists the result
  33. // to the database.
  34. func LoadStateFromDBOrGenesisFile(stateDB dbm.DB, genesisFilePath string) (State, error) {
  35. state := LoadState(stateDB)
  36. if state.IsEmpty() {
  37. var err error
  38. state, err = MakeGenesisStateFromFile(genesisFilePath)
  39. if err != nil {
  40. return state, err
  41. }
  42. SaveState(stateDB, state)
  43. }
  44. return state, nil
  45. }
  46. // LoadStateFromDBOrGenesisDoc loads the most recent state from the database,
  47. // or creates a new one from the given genesisDoc and persists the result
  48. // to the database.
  49. func LoadStateFromDBOrGenesisDoc(stateDB dbm.DB, genesisDoc *types.GenesisDoc) (State, error) {
  50. state := LoadState(stateDB)
  51. if state.IsEmpty() {
  52. var err error
  53. state, err = MakeGenesisState(genesisDoc)
  54. if err != nil {
  55. return state, err
  56. }
  57. SaveState(stateDB, state)
  58. }
  59. return state, nil
  60. }
  61. // LoadState loads the State from the database.
  62. func LoadState(db dbm.DB) State {
  63. return loadState(db, stateKey)
  64. }
  65. func loadState(db dbm.DB, key []byte) (state State) {
  66. buf, err := db.Get(key)
  67. if err != nil {
  68. panic(err)
  69. }
  70. if len(buf) == 0 {
  71. return state
  72. }
  73. sp := new(tmstate.State)
  74. err = proto.Unmarshal(buf, sp)
  75. if err != nil {
  76. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  77. tmos.Exit(fmt.Sprintf(`LoadState: Data has been corrupted or its spec has changed:
  78. %v\n`, err))
  79. }
  80. sm, err := StateFromProto(sp)
  81. if err != nil {
  82. panic(err)
  83. }
  84. return *sm
  85. }
  86. // SaveState persists the State, the ValidatorsInfo, and the ConsensusParamsInfo to the database.
  87. // This flushes the writes (e.g. calls SetSync).
  88. func SaveState(db dbm.DB, state State) {
  89. saveState(db, state, stateKey)
  90. }
  91. func saveState(db dbm.DB, state State, key []byte) {
  92. nextHeight := state.LastBlockHeight + 1
  93. // If first block, save validators for block 1.
  94. if nextHeight == 1 {
  95. // This extra logic due to Tendermint validator set changes being delayed 1 block.
  96. // It may get overwritten due to InitChain validator updates.
  97. lastHeightVoteChanged := int64(1)
  98. saveValidatorsInfo(db, nextHeight, lastHeightVoteChanged, state.Validators)
  99. }
  100. // Save next validators.
  101. saveValidatorsInfo(db, nextHeight+1, state.LastHeightValidatorsChanged, state.NextValidators)
  102. // Save next consensus params.
  103. saveConsensusParamsInfo(db, nextHeight, state.LastHeightConsensusParamsChanged, state.ConsensusParams)
  104. err := db.SetSync(key, state.Bytes())
  105. if err != nil {
  106. panic(err)
  107. }
  108. }
  109. // BootstrapState saves a new state, used e.g. by state sync when starting from non-zero height.
  110. func BootstrapState(db dbm.DB, state State) error {
  111. height := state.LastBlockHeight
  112. saveValidatorsInfo(db, height, height, state.LastValidators)
  113. saveValidatorsInfo(db, height+1, height+1, state.Validators)
  114. saveValidatorsInfo(db, height+2, height+2, state.NextValidators)
  115. saveConsensusParamsInfo(db, height+1, height+1, state.ConsensusParams)
  116. return db.SetSync(stateKey, state.Bytes())
  117. }
  118. // PruneStates deletes states between the given heights (including from, excluding to). It is not
  119. // guaranteed to delete all states, since the last checkpointed state and states being pointed to by
  120. // e.g. `LastHeightChanged` must remain. The state at to must also exist.
  121. //
  122. // The from parameter is necessary since we can't do a key scan in a performant way due to the key
  123. // encoding not preserving ordering: https://github.com/tendermint/tendermint/issues/4567
  124. // This will cause some old states to be left behind when doing incremental partial prunes,
  125. // specifically older checkpoints and LastHeightChanged targets.
  126. func PruneStates(db dbm.DB, from int64, to int64) error {
  127. if from <= 0 || to <= 0 {
  128. return fmt.Errorf("from height %v and to height %v must be greater than 0", from, to)
  129. }
  130. if from >= to {
  131. return fmt.Errorf("from height %v must be lower than to height %v", from, to)
  132. }
  133. valInfo := loadValidatorsInfo(db, to)
  134. if valInfo == nil {
  135. return fmt.Errorf("validators at height %v not found", to)
  136. }
  137. paramsInfo := loadConsensusParamsInfo(db, to)
  138. if paramsInfo == nil {
  139. return fmt.Errorf("consensus params at height %v not found", to)
  140. }
  141. keepVals := make(map[int64]bool)
  142. if valInfo.ValidatorSet == nil {
  143. keepVals[valInfo.LastHeightChanged] = true
  144. keepVals[lastStoredHeightFor(to, valInfo.LastHeightChanged)] = true // keep last checkpoint too
  145. }
  146. keepParams := make(map[int64]bool)
  147. if paramsInfo.ConsensusParams.Equal(&tmproto.ConsensusParams{}) {
  148. keepParams[paramsInfo.LastHeightChanged] = true
  149. }
  150. batch := db.NewBatch()
  151. defer batch.Close()
  152. pruned := uint64(0)
  153. var err error
  154. // We have to delete in reverse order, to avoid deleting previous heights that have validator
  155. // sets and consensus params that we may need to retrieve.
  156. for h := to - 1; h >= from; h-- {
  157. // For heights we keep, we must make sure they have the full validator set or consensus
  158. // params, otherwise they will panic if they're retrieved directly (instead of
  159. // indirectly via a LastHeightChanged pointer).
  160. if keepVals[h] {
  161. v := loadValidatorsInfo(db, h)
  162. if v.ValidatorSet == nil {
  163. vip, err := LoadValidators(db, h)
  164. if err != nil {
  165. return err
  166. }
  167. pvi, err := vip.ToProto()
  168. if err != nil {
  169. return err
  170. }
  171. v.ValidatorSet = pvi
  172. v.LastHeightChanged = h
  173. bz, err := v.Marshal()
  174. if err != nil {
  175. return err
  176. }
  177. batch.Set(calcValidatorsKey(h), bz)
  178. }
  179. } else {
  180. batch.Delete(calcValidatorsKey(h))
  181. }
  182. if keepParams[h] {
  183. p := loadConsensusParamsInfo(db, h)
  184. if p.ConsensusParams.Equal(&tmproto.ConsensusParams{}) {
  185. p.ConsensusParams, err = LoadConsensusParams(db, h)
  186. if err != nil {
  187. return err
  188. }
  189. p.LastHeightChanged = h
  190. bz, err := p.Marshal()
  191. if err != nil {
  192. return err
  193. }
  194. batch.Set(calcConsensusParamsKey(h), bz)
  195. }
  196. } else {
  197. batch.Delete(calcConsensusParamsKey(h))
  198. }
  199. batch.Delete(calcABCIResponsesKey(h))
  200. pruned++
  201. // avoid batches growing too large by flushing to database regularly
  202. if pruned%1000 == 0 && pruned > 0 {
  203. err := batch.Write()
  204. if err != nil {
  205. return err
  206. }
  207. batch.Close()
  208. batch = db.NewBatch()
  209. defer batch.Close()
  210. }
  211. }
  212. err = batch.WriteSync()
  213. if err != nil {
  214. return err
  215. }
  216. return nil
  217. }
  218. //------------------------------------------------------------------------
  219. // ABCIResponsesResultsHash returns the root hash of a Merkle tree with 3 leafs:
  220. // 1) proto encoded ResponseBeginBlock.Events
  221. // 2) root hash of a Merkle tree of ResponseDeliverTx responses (see ABCIResults.Hash)
  222. // 3) proto encoded ResponseEndBlock.Events
  223. //
  224. // See merkle.SimpleHashFromByteSlices
  225. func ABCIResponsesResultsHash(ar *tmstate.ABCIResponses) []byte {
  226. // proto-encode BeginBlock events.
  227. bbeBytes, err := proto.Marshal(&abci.ResponseBeginBlock{
  228. Events: ar.BeginBlock.Events,
  229. })
  230. if err != nil {
  231. panic(err)
  232. }
  233. // Build a Merkle tree of proto-encoded DeliverTx results and get a hash.
  234. results := types.NewResults(ar.DeliverTxs)
  235. // proto-encode EndBlock events.
  236. ebeBytes, err := proto.Marshal(&abci.ResponseEndBlock{
  237. Events: ar.EndBlock.Events,
  238. })
  239. if err != nil {
  240. panic(err)
  241. }
  242. // Build a Merkle tree out of the above 3 binary slices.
  243. return merkle.HashFromByteSlices([][]byte{bbeBytes, results.Hash(), ebeBytes})
  244. }
  245. // LoadABCIResponses loads the ABCIResponses for the given height from the
  246. // database. If not found, ErrNoABCIResponsesForHeight is returned.
  247. //
  248. // This is useful for recovering from crashes where we called app.Commit and
  249. // before we called s.Save(). It can also be used to produce Merkle proofs of
  250. // the result of txs.
  251. func LoadABCIResponses(db dbm.DB, height int64) (*tmstate.ABCIResponses, error) {
  252. buf, err := db.Get(calcABCIResponsesKey(height))
  253. if err != nil {
  254. return nil, err
  255. }
  256. if len(buf) == 0 {
  257. return nil, ErrNoABCIResponsesForHeight{height}
  258. }
  259. abciResponses := new(tmstate.ABCIResponses)
  260. err = abciResponses.Unmarshal(buf)
  261. if err != nil {
  262. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  263. tmos.Exit(fmt.Sprintf(`LoadABCIResponses: Data has been corrupted or its spec has
  264. changed: %v\n`, err))
  265. }
  266. // TODO: ensure that buf is completely read.
  267. return abciResponses, nil
  268. }
  269. // SaveABCIResponses persists the ABCIResponses to the database.
  270. // This is useful in case we crash after app.Commit and before s.Save().
  271. // Responses are indexed by height so they can also be loaded later to produce
  272. // Merkle proofs.
  273. //
  274. // Exposed for testing.
  275. func SaveABCIResponses(db dbm.DB, height int64, abciResponses *tmstate.ABCIResponses) {
  276. var dtxs []*abci.ResponseDeliverTx
  277. //strip nil values,
  278. for _, tx := range abciResponses.DeliverTxs {
  279. if tx != nil {
  280. dtxs = append(dtxs, tx)
  281. }
  282. }
  283. abciResponses.DeliverTxs = dtxs
  284. bz, err := abciResponses.Marshal()
  285. if err != nil {
  286. panic(err)
  287. }
  288. db.SetSync(calcABCIResponsesKey(height), bz)
  289. }
  290. //-----------------------------------------------------------------------------
  291. // LoadValidators loads the ValidatorSet for a given height.
  292. // Returns ErrNoValSetForHeight if the validator set can't be found for this height.
  293. func LoadValidators(db dbm.DB, height int64) (*types.ValidatorSet, error) {
  294. valInfo := loadValidatorsInfo(db, height)
  295. if valInfo == nil {
  296. return nil, ErrNoValSetForHeight{height}
  297. }
  298. if valInfo.ValidatorSet == nil {
  299. lastStoredHeight := lastStoredHeightFor(height, valInfo.LastHeightChanged)
  300. valInfo2 := loadValidatorsInfo(db, lastStoredHeight)
  301. if valInfo2 == nil || valInfo2.ValidatorSet == nil {
  302. panic(
  303. fmt.Sprintf("Couldn't find validators at height %d (height %d was originally requested)",
  304. lastStoredHeight,
  305. height,
  306. ),
  307. )
  308. }
  309. vs, err := types.ValidatorSetFromProto(valInfo2.ValidatorSet)
  310. if err != nil {
  311. return nil, err
  312. }
  313. vs.IncrementProposerPriority(tmmath.SafeConvertInt32(height - lastStoredHeight)) // mutate
  314. vi2, err := vs.ToProto()
  315. if err != nil {
  316. return nil, err
  317. }
  318. valInfo2.ValidatorSet = vi2
  319. valInfo = valInfo2
  320. }
  321. vip, err := types.ValidatorSetFromProto(valInfo.ValidatorSet)
  322. if err != nil {
  323. return nil, err
  324. }
  325. return vip, nil
  326. }
  327. func lastStoredHeightFor(height, lastHeightChanged int64) int64 {
  328. checkpointHeight := height - height%valSetCheckpointInterval
  329. return tmmath.MaxInt64(checkpointHeight, lastHeightChanged)
  330. }
  331. // CONTRACT: Returned ValidatorsInfo can be mutated.
  332. func loadValidatorsInfo(db dbm.DB, height int64) *tmstate.ValidatorsInfo {
  333. buf, err := db.Get(calcValidatorsKey(height))
  334. if err != nil {
  335. panic(err)
  336. }
  337. if len(buf) == 0 {
  338. return nil
  339. }
  340. v := new(tmstate.ValidatorsInfo)
  341. err = v.Unmarshal(buf)
  342. if err != nil {
  343. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  344. tmos.Exit(fmt.Sprintf(`LoadValidators: Data has been corrupted or its spec has changed:
  345. %v\n`, err))
  346. }
  347. // TODO: ensure that buf is completely read.
  348. return v
  349. }
  350. // saveValidatorsInfo persists the validator set.
  351. //
  352. // `height` is the effective height for which the validator is responsible for
  353. // signing. It should be called from s.Save(), right before the state itself is
  354. // persisted.
  355. func saveValidatorsInfo(db dbm.DB, height, lastHeightChanged int64, valSet *types.ValidatorSet) {
  356. if lastHeightChanged > height {
  357. panic("LastHeightChanged cannot be greater than ValidatorsInfo height")
  358. }
  359. valInfo := &tmstate.ValidatorsInfo{
  360. LastHeightChanged: lastHeightChanged,
  361. }
  362. // Only persist validator set if it was updated or checkpoint height (see
  363. // valSetCheckpointInterval) is reached.
  364. if height == lastHeightChanged || height%valSetCheckpointInterval == 0 {
  365. pv, err := valSet.ToProto()
  366. if err != nil {
  367. panic(err)
  368. }
  369. valInfo.ValidatorSet = pv
  370. }
  371. bz, err := valInfo.Marshal()
  372. if err != nil {
  373. panic(err)
  374. }
  375. db.Set(calcValidatorsKey(height), bz)
  376. }
  377. //-----------------------------------------------------------------------------
  378. // ConsensusParamsInfo represents the latest consensus params, or the last height it changed
  379. // LoadConsensusParams loads the ConsensusParams for a given height.
  380. func LoadConsensusParams(db dbm.DB, height int64) (tmproto.ConsensusParams, error) {
  381. empty := tmproto.ConsensusParams{}
  382. paramsInfo := loadConsensusParamsInfo(db, height)
  383. if paramsInfo == nil {
  384. return empty, ErrNoConsensusParamsForHeight{height}
  385. }
  386. if paramsInfo.ConsensusParams.Equal(&empty) {
  387. paramsInfo2 := loadConsensusParamsInfo(db, paramsInfo.LastHeightChanged)
  388. if paramsInfo2 == nil {
  389. panic(
  390. fmt.Sprintf(
  391. "Couldn't find consensus params at height %d as last changed from height %d",
  392. paramsInfo.LastHeightChanged,
  393. height,
  394. ),
  395. )
  396. }
  397. paramsInfo = paramsInfo2
  398. }
  399. return paramsInfo.ConsensusParams, nil
  400. }
  401. func loadConsensusParamsInfo(db dbm.DB, height int64) *tmstate.ConsensusParamsInfo {
  402. buf, err := db.Get(calcConsensusParamsKey(height))
  403. if err != nil {
  404. panic(err)
  405. }
  406. if len(buf) == 0 {
  407. return nil
  408. }
  409. paramsInfo := new(tmstate.ConsensusParamsInfo)
  410. if err = paramsInfo.Unmarshal(buf); err != nil {
  411. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  412. tmos.Exit(fmt.Sprintf(`LoadConsensusParams: Data has been corrupted or its spec has changed:
  413. %v\n`, err))
  414. }
  415. // TODO: ensure that buf is completely read.
  416. return paramsInfo
  417. }
  418. // saveConsensusParamsInfo persists the consensus params for the next block to disk.
  419. // It should be called from s.Save(), right before the state itself is persisted.
  420. // If the consensus params did not change after processing the latest block,
  421. // only the last height for which they changed is persisted.
  422. func saveConsensusParamsInfo(db dbm.DB, nextHeight, changeHeight int64, params tmproto.ConsensusParams) {
  423. paramsInfo := &tmstate.ConsensusParamsInfo{
  424. LastHeightChanged: changeHeight,
  425. }
  426. if changeHeight == nextHeight {
  427. paramsInfo.ConsensusParams = params
  428. }
  429. bz, err := paramsInfo.Marshal()
  430. if err != nil {
  431. panic(err)
  432. }
  433. db.Set(calcConsensusParamsKey(nextHeight), bz)
  434. }