You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

323 lines
11 KiB

state: add more tests for block validation (#3674) * Expose priv validators for use in testing * Generalize block header validation test past height 1 * Remove ineffectual assignment * Remove redundant SaveState call * Reorder comment for clarity * Use the block executor ApplyBlock function instead of implementing a stripped-down version of it * Remove commented-out code * Remove unnecessary test The required tests already appear to be implemented (implicitly) through the TestValidateBlockHeader test. * Allow for catching of specific error types during TestValidateBlockCommit * Make return error testable * Clean up and add TestValidateBlockCommit code * Fix formatting * Extract function to create a new mock test app * Update comment for clarity * Fix comment * Add skeleton code for evidence-related test * Allow for addressing priv val by address * Generalize test beyond a single validator * Generalize TestValidateBlockEvidence past first height * Reorder code to clearly separate tests and utility code * Use a common constant for stop height for testing in state/validation_test.go * Refactor errors to resemble existing conventions * Fix formatting * Extract common helper functions Having the tests littered with helper functions makes them less easily readable imho, so I've pulled them out into a separate file. This also makes it easier to see what helper functions are available during testing, so we minimize the chance of duplication when writing new tests. * Remove unused parameter * Remove unused parameters * Add field keys * Remove unused height constant * Fix typo * Fix incorrect return error * Add field keys * Use separate package for tests This refactors all of the state package's tests into a state_test package, so as to keep any usage of the state package's internal methods explicit. Any internal methods/constants used by tests are now explicitly exported in state/export_test.go * Refactor: extract helper function to make, validate, execute and commit a block * Rename state function to makeState * Remove redundant constant for number of validators * Refactor mock evidence registration into TestMain * Remove extraneous nVals variable * Replace function-level TODOs with file-level TODO and explanation * Remove extraneous comment * Fix linting issues brought up by GolangCI (pulled in from latest merge from develop)
5 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
  1. package state_test
  2. import (
  3. "context"
  4. "fmt"
  5. "math/rand"
  6. "os"
  7. "testing"
  8. "github.com/stretchr/testify/assert"
  9. "github.com/stretchr/testify/require"
  10. dbm "github.com/tendermint/tm-db"
  11. abci "github.com/tendermint/tendermint/abci/types"
  12. "github.com/tendermint/tendermint/config"
  13. "github.com/tendermint/tendermint/crypto"
  14. "github.com/tendermint/tendermint/crypto/ed25519"
  15. sm "github.com/tendermint/tendermint/internal/state"
  16. "github.com/tendermint/tendermint/internal/test/factory"
  17. tmrand "github.com/tendermint/tendermint/libs/rand"
  18. tmstate "github.com/tendermint/tendermint/proto/tendermint/state"
  19. "github.com/tendermint/tendermint/types"
  20. )
  21. const (
  22. // make sure this is the same as in state/store.go
  23. valSetCheckpointInterval = 100000
  24. )
  25. func TestStoreBootstrap(t *testing.T) {
  26. stateDB := dbm.NewMemDB()
  27. stateStore := sm.NewStore(stateDB)
  28. ctx, cancel := context.WithCancel(context.Background())
  29. defer cancel()
  30. val, _, err := factory.Validator(ctx, 10+int64(rand.Uint32()))
  31. require.NoError(t, err)
  32. val2, _, err := factory.Validator(ctx, 10+int64(rand.Uint32()))
  33. require.NoError(t, err)
  34. val3, _, err := factory.Validator(ctx, 10+int64(rand.Uint32()))
  35. require.NoError(t, err)
  36. vals := types.NewValidatorSet([]*types.Validator{val, val2, val3})
  37. bootstrapState := makeRandomStateFromValidatorSet(vals, 100, 100)
  38. require.NoError(t, stateStore.Bootstrap(bootstrapState))
  39. // bootstrap should also save the previous validator
  40. _, err = stateStore.LoadValidators(99)
  41. require.NoError(t, err)
  42. _, err = stateStore.LoadValidators(100)
  43. require.NoError(t, err)
  44. _, err = stateStore.LoadValidators(101)
  45. require.NoError(t, err)
  46. state, err := stateStore.Load()
  47. require.NoError(t, err)
  48. require.Equal(t, bootstrapState, state)
  49. }
  50. func TestStoreLoadValidators(t *testing.T) {
  51. stateDB := dbm.NewMemDB()
  52. stateStore := sm.NewStore(stateDB)
  53. ctx, cancel := context.WithCancel(context.Background())
  54. defer cancel()
  55. val, _, err := factory.Validator(ctx, 10+int64(rand.Uint32()))
  56. require.NoError(t, err)
  57. val2, _, err := factory.Validator(ctx, 10+int64(rand.Uint32()))
  58. require.NoError(t, err)
  59. val3, _, err := factory.Validator(ctx, 10+int64(rand.Uint32()))
  60. require.NoError(t, err)
  61. vals := types.NewValidatorSet([]*types.Validator{val, val2, val3})
  62. // 1) LoadValidators loads validators using a height where they were last changed
  63. // Note that only the next validators at height h + 1 are saved
  64. require.NoError(t, stateStore.Save(makeRandomStateFromValidatorSet(vals, 1, 1)))
  65. require.NoError(t, stateStore.Save(makeRandomStateFromValidatorSet(vals.CopyIncrementProposerPriority(1), 2, 1)))
  66. loadedVals, err := stateStore.LoadValidators(3)
  67. require.NoError(t, err)
  68. require.Equal(t, vals.CopyIncrementProposerPriority(3), loadedVals)
  69. // 2) LoadValidators loads validators using a checkpoint height
  70. // add a validator set at the checkpoint
  71. err = stateStore.Save(makeRandomStateFromValidatorSet(vals, valSetCheckpointInterval, 1))
  72. require.NoError(t, err)
  73. // check that a request will go back to the last checkpoint
  74. _, err = stateStore.LoadValidators(valSetCheckpointInterval + 1)
  75. require.Error(t, err)
  76. require.Equal(t, fmt.Sprintf("couldn't find validators at height %d (height %d was originally requested): "+
  77. "value retrieved from db is empty",
  78. valSetCheckpointInterval, valSetCheckpointInterval+1), err.Error())
  79. // now save a validator set at that checkpoint
  80. err = stateStore.Save(makeRandomStateFromValidatorSet(vals, valSetCheckpointInterval-1, 1))
  81. require.NoError(t, err)
  82. loadedVals, err = stateStore.LoadValidators(valSetCheckpointInterval)
  83. require.NoError(t, err)
  84. // validator set gets updated with the one given hence we expect it to equal next validators (with an increment of one)
  85. // as opposed to being equal to an increment of 100000 - 1 (if we didn't save at the checkpoint)
  86. require.Equal(t, vals.CopyIncrementProposerPriority(2), loadedVals)
  87. require.NotEqual(t, vals.CopyIncrementProposerPriority(valSetCheckpointInterval), loadedVals)
  88. }
  89. // This benchmarks the speed of loading validators from different heights if there is no validator set change.
  90. // NOTE: This isn't too indicative of validator retrieval speed as the db is always (regardless of height) only
  91. // performing two operations: 1) retrieve validator info at height x, which has a last validator set change of 1
  92. // and 2) retrieve the validator set at the aforementioned height 1.
  93. func BenchmarkLoadValidators(b *testing.B) {
  94. const valSetSize = 100
  95. cfg, err := config.ResetTestRoot(b.TempDir(), "state_")
  96. require.NoError(b, err)
  97. defer os.RemoveAll(cfg.RootDir)
  98. dbType := dbm.BackendType(cfg.DBBackend)
  99. stateDB, err := dbm.NewDB("state", dbType, cfg.DBDir())
  100. require.NoError(b, err)
  101. stateStore := sm.NewStore(stateDB)
  102. state, err := sm.MakeGenesisStateFromFile(cfg.GenesisFile())
  103. if err != nil {
  104. b.Fatal(err)
  105. }
  106. state.Validators = genValSet(valSetSize)
  107. state.NextValidators = state.Validators.CopyIncrementProposerPriority(1)
  108. err = stateStore.Save(state)
  109. require.NoError(b, err)
  110. b.ResetTimer()
  111. for i := 10; i < 10000000000; i *= 10 { // 10, 100, 1000, ...
  112. i := i
  113. err = stateStore.Save(makeRandomStateFromValidatorSet(state.NextValidators,
  114. int64(i)-1, state.LastHeightValidatorsChanged))
  115. if err != nil {
  116. b.Fatalf("error saving store: %v", err)
  117. }
  118. b.Run(fmt.Sprintf("height=%d", i), func(b *testing.B) {
  119. for n := 0; n < b.N; n++ {
  120. _, err := stateStore.LoadValidators(int64(i))
  121. if err != nil {
  122. b.Fatal(err)
  123. }
  124. }
  125. })
  126. }
  127. }
  128. func TestStoreLoadConsensusParams(t *testing.T) {
  129. ctx, cancel := context.WithCancel(context.Background())
  130. defer cancel()
  131. stateDB := dbm.NewMemDB()
  132. stateStore := sm.NewStore(stateDB)
  133. err := stateStore.Save(makeRandomStateFromConsensusParams(ctx, t, types.DefaultConsensusParams(), 1, 1))
  134. require.NoError(t, err)
  135. params, err := stateStore.LoadConsensusParams(1)
  136. require.NoError(t, err)
  137. require.Equal(t, types.DefaultConsensusParams(), &params)
  138. // we give the state store different params but say that the height hasn't changed, hence
  139. // it should save a pointer to the params at height 1
  140. differentParams := types.DefaultConsensusParams()
  141. differentParams.Block.MaxBytes = 20000
  142. err = stateStore.Save(makeRandomStateFromConsensusParams(ctx, t, differentParams, 10, 1))
  143. require.NoError(t, err)
  144. res, err := stateStore.LoadConsensusParams(10)
  145. require.NoError(t, err)
  146. require.Equal(t, res, params)
  147. require.NotEqual(t, res, differentParams)
  148. }
  149. func TestPruneStates(t *testing.T) {
  150. testcases := map[string]struct {
  151. startHeight int64
  152. endHeight int64
  153. pruneHeight int64
  154. expectErr bool
  155. remainingValSetHeight int64
  156. remainingParamsHeight int64
  157. }{
  158. "error when prune height is 0": {1, 100, 0, true, 0, 0},
  159. "error when prune height is negative": {1, 100, -10, true, 0, 0},
  160. "error when prune height does not exist": {1, 100, 101, true, 0, 0},
  161. "prune all": {1, 100, 100, false, 93, 95},
  162. "prune from non 1 height": {10, 50, 40, false, 33, 35},
  163. "prune some": {1, 10, 8, false, 3, 5},
  164. // we test this because we flush to disk every 1000 "states"
  165. "prune more than 1000 state": {1, 1010, 1010, false, 1003, 1005},
  166. "prune across checkpoint": {99900, 100002, 100002, false, 100000, 99995},
  167. }
  168. for name, tc := range testcases {
  169. tc := tc
  170. t.Run(name, func(t *testing.T) {
  171. db := dbm.NewMemDB()
  172. stateStore := sm.NewStore(db)
  173. pk := ed25519.GenPrivKey().PubKey()
  174. // Generate a bunch of state data. Validators change for heights ending with 3, and
  175. // parameters when ending with 5.
  176. validator := &types.Validator{Address: tmrand.Bytes(crypto.AddressSize), VotingPower: 100, PubKey: pk}
  177. validatorSet := &types.ValidatorSet{
  178. Validators: []*types.Validator{validator},
  179. Proposer: validator,
  180. }
  181. valsChanged := int64(0)
  182. paramsChanged := int64(0)
  183. for h := tc.startHeight; h <= tc.endHeight; h++ {
  184. if valsChanged == 0 || h%10 == 2 {
  185. valsChanged = h + 1 // Have to add 1, since NextValidators is what's stored
  186. }
  187. if paramsChanged == 0 || h%10 == 5 {
  188. paramsChanged = h
  189. }
  190. state := sm.State{
  191. InitialHeight: 1,
  192. LastBlockHeight: h - 1,
  193. Validators: validatorSet,
  194. NextValidators: validatorSet,
  195. ConsensusParams: types.ConsensusParams{
  196. Block: types.BlockParams{MaxBytes: 10e6},
  197. },
  198. LastHeightValidatorsChanged: valsChanged,
  199. LastHeightConsensusParamsChanged: paramsChanged,
  200. }
  201. if state.LastBlockHeight >= 1 {
  202. state.LastValidators = state.Validators
  203. }
  204. err := stateStore.Save(state)
  205. require.NoError(t, err)
  206. err = stateStore.SaveABCIResponses(h, &tmstate.ABCIResponses{
  207. FinalizeBlock: &abci.ResponseFinalizeBlock{
  208. Txs: []*abci.ResponseDeliverTx{
  209. {Data: []byte{1}},
  210. {Data: []byte{2}},
  211. {Data: []byte{3}},
  212. },
  213. },
  214. })
  215. require.NoError(t, err)
  216. }
  217. // Test assertions
  218. err := stateStore.PruneStates(tc.pruneHeight)
  219. if tc.expectErr {
  220. require.Error(t, err)
  221. return
  222. }
  223. require.NoError(t, err)
  224. for h := tc.pruneHeight; h <= tc.endHeight; h++ {
  225. vals, err := stateStore.LoadValidators(h)
  226. require.NoError(t, err, h)
  227. require.NotNil(t, vals, h)
  228. params, err := stateStore.LoadConsensusParams(h)
  229. require.NoError(t, err, h)
  230. require.NotNil(t, params, h)
  231. abci, err := stateStore.LoadABCIResponses(h)
  232. require.NoError(t, err, h)
  233. require.NotNil(t, abci, h)
  234. }
  235. emptyParams := types.ConsensusParams{}
  236. for h := tc.startHeight; h < tc.pruneHeight; h++ {
  237. vals, err := stateStore.LoadValidators(h)
  238. if h == tc.remainingValSetHeight {
  239. require.NoError(t, err, h)
  240. require.NotNil(t, vals, h)
  241. } else {
  242. require.Error(t, err, h)
  243. require.Nil(t, vals, h)
  244. }
  245. params, err := stateStore.LoadConsensusParams(h)
  246. if h == tc.remainingParamsHeight {
  247. require.NoError(t, err, h)
  248. require.NotEqual(t, emptyParams, params, h)
  249. } else {
  250. require.Error(t, err, h)
  251. require.Equal(t, emptyParams, params, h)
  252. }
  253. abci, err := stateStore.LoadABCIResponses(h)
  254. require.Error(t, err, h)
  255. require.Nil(t, abci, h)
  256. }
  257. })
  258. }
  259. }
  260. func TestABCIResponsesResultsHash(t *testing.T) {
  261. responses := &tmstate.ABCIResponses{
  262. FinalizeBlock: &abci.ResponseFinalizeBlock{
  263. Txs: []*abci.ResponseDeliverTx{
  264. {Code: 32, Data: []byte("Hello"), Log: "Huh?"},
  265. },
  266. },
  267. }
  268. root := sm.ABCIResponsesResultsHash(responses)
  269. // root should be Merkle tree root of FinalizeBlock tx responses
  270. results := types.NewResults(responses.FinalizeBlock.Txs)
  271. assert.Equal(t, root, results.Hash())
  272. // test we can prove first tx in FinalizeBlock
  273. proof := results.ProveResult(0)
  274. bz, err := results[0].Marshal()
  275. require.NoError(t, err)
  276. assert.NoError(t, proof.Verify(root, bz))
  277. }