You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

534 lines
15 KiB

blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
5 years ago
blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
5 years ago
blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
blockchain: add v2 reactor (#4361) The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase. ## Commits: * Blockchainv 2 reactor: + I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase. + The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. * fixes after merge * reorder iIO interface methodset * change iO -> IO * panic before send nil block * rename switchToConsensus -> trySwitchToConsensus * rename tdState -> tmState * Update blockchain/v2/reactor.go Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> * remove peer when it sends a block unsolicited * check for not ready in markReceived * fix error * fix the pcFinished event * typo fix * add documentation for processor fields * simplify time.Since * try and make the linter happy * some doc updates * fix channel diagram * Update adr-043-blockchain-riri-org.md * panic on nil switch * liting fixes * account for nil block in bBlockResponseMessage * panic on duplicate block enqueued by processor * linting * goimport reactor_test.go Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com> Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
  1. package v2
  2. import (
  3. "net"
  4. "os"
  5. "sort"
  6. "sync"
  7. "testing"
  8. "time"
  9. "github.com/pkg/errors"
  10. "github.com/stretchr/testify/assert"
  11. dbm "github.com/tendermint/tm-db"
  12. abci "github.com/tendermint/tendermint/abci/types"
  13. "github.com/tendermint/tendermint/behaviour"
  14. cfg "github.com/tendermint/tendermint/config"
  15. "github.com/tendermint/tendermint/libs/log"
  16. "github.com/tendermint/tendermint/libs/service"
  17. "github.com/tendermint/tendermint/mock"
  18. "github.com/tendermint/tendermint/p2p"
  19. "github.com/tendermint/tendermint/p2p/conn"
  20. "github.com/tendermint/tendermint/proxy"
  21. sm "github.com/tendermint/tendermint/state"
  22. "github.com/tendermint/tendermint/store"
  23. "github.com/tendermint/tendermint/types"
  24. tmtime "github.com/tendermint/tendermint/types/time"
  25. )
  26. type mockPeer struct {
  27. service.Service
  28. id p2p.ID
  29. }
  30. func (mp mockPeer) FlushStop() {}
  31. func (mp mockPeer) ID() p2p.ID { return mp.id }
  32. func (mp mockPeer) RemoteIP() net.IP { return net.IP{} }
  33. func (mp mockPeer) RemoteAddr() net.Addr { return &net.TCPAddr{IP: mp.RemoteIP(), Port: 8800} }
  34. func (mp mockPeer) IsOutbound() bool { return true }
  35. func (mp mockPeer) IsPersistent() bool { return true }
  36. func (mp mockPeer) CloseConn() error { return nil }
  37. func (mp mockPeer) NodeInfo() p2p.NodeInfo {
  38. return p2p.DefaultNodeInfo{
  39. DefaultNodeID: "",
  40. ListenAddr: "",
  41. }
  42. }
  43. func (mp mockPeer) Status() conn.ConnectionStatus { return conn.ConnectionStatus{} }
  44. func (mp mockPeer) SocketAddr() *p2p.NetAddress { return &p2p.NetAddress{} }
  45. func (mp mockPeer) Send(byte, []byte) bool { return true }
  46. func (mp mockPeer) TrySend(byte, []byte) bool { return true }
  47. func (mp mockPeer) Set(string, interface{}) {}
  48. func (mp mockPeer) Get(string) interface{} { return struct{}{} }
  49. //nolint:unused
  50. type mockBlockStore struct {
  51. blocks map[int64]*types.Block
  52. }
  53. func (ml *mockBlockStore) Height() int64 {
  54. return int64(len(ml.blocks))
  55. }
  56. func (ml *mockBlockStore) LoadBlock(height int64) *types.Block {
  57. return ml.blocks[height]
  58. }
  59. func (ml *mockBlockStore) SaveBlock(block *types.Block, part *types.PartSet, commit *types.Commit) {
  60. ml.blocks[block.Height] = block
  61. }
  62. type mockBlockApplier struct {
  63. }
  64. // XXX: Add whitelist/blacklist?
  65. func (mba *mockBlockApplier) ApplyBlock(
  66. state sm.State, blockID types.BlockID, block *types.Block,
  67. ) (sm.State, int64, error) {
  68. state.LastBlockHeight++
  69. return state, 0, nil
  70. }
  71. type mockSwitchIo struct {
  72. mtx sync.Mutex
  73. switchedToConsensus bool
  74. numStatusResponse int
  75. numBlockResponse int
  76. numNoBlockResponse int
  77. }
  78. func (sio *mockSwitchIo) sendBlockRequest(peerID p2p.ID, height int64) error {
  79. return nil
  80. }
  81. func (sio *mockSwitchIo) sendStatusResponse(height int64, peerID p2p.ID) error {
  82. sio.mtx.Lock()
  83. defer sio.mtx.Unlock()
  84. sio.numStatusResponse++
  85. return nil
  86. }
  87. func (sio *mockSwitchIo) sendBlockToPeer(block *types.Block, peerID p2p.ID) error {
  88. sio.mtx.Lock()
  89. defer sio.mtx.Unlock()
  90. sio.numBlockResponse++
  91. return nil
  92. }
  93. func (sio *mockSwitchIo) sendBlockNotFound(height int64, peerID p2p.ID) error {
  94. sio.mtx.Lock()
  95. defer sio.mtx.Unlock()
  96. sio.numNoBlockResponse++
  97. return nil
  98. }
  99. func (sio *mockSwitchIo) trySwitchToConsensus(state sm.State, blocksSynced int) {
  100. sio.mtx.Lock()
  101. defer sio.mtx.Unlock()
  102. sio.switchedToConsensus = true
  103. }
  104. func (sio *mockSwitchIo) broadcastStatusRequest(base int64, height int64) {
  105. }
  106. type testReactorParams struct {
  107. logger log.Logger
  108. genDoc *types.GenesisDoc
  109. privVals []types.PrivValidator
  110. startHeight int64
  111. bufferSize int
  112. mockA bool
  113. }
  114. func newTestReactor(p testReactorParams) *BlockchainReactor {
  115. store, state, _ := newReactorStore(p.genDoc, p.privVals, p.startHeight)
  116. reporter := behaviour.NewMockReporter()
  117. var appl blockApplier
  118. if p.mockA {
  119. appl = &mockBlockApplier{}
  120. } else {
  121. app := &testApp{}
  122. cc := proxy.NewLocalClientCreator(app)
  123. proxyApp := proxy.NewAppConns(cc)
  124. err := proxyApp.Start()
  125. if err != nil {
  126. panic(errors.Wrap(err, "error start app"))
  127. }
  128. db := dbm.NewMemDB()
  129. appl = sm.NewBlockExecutor(db, p.logger, proxyApp.Consensus(), mock.Mempool{}, sm.MockEvidencePool{})
  130. sm.SaveState(db, state)
  131. }
  132. r := newReactor(state, store, reporter, appl, p.bufferSize)
  133. logger := log.TestingLogger()
  134. r.SetLogger(logger.With("module", "blockchain"))
  135. return r
  136. }
  137. // This test is left here and not deleted to retain the termination cases for
  138. // future improvement in [#4482](https://github.com/tendermint/tendermint/issues/4482).
  139. // func TestReactorTerminationScenarios(t *testing.T) {
  140. // config := cfg.ResetTestRoot("blockchain_reactor_v2_test")
  141. // defer os.RemoveAll(config.RootDir)
  142. // genDoc, privVals := randGenesisDoc(config.ChainID(), 1, false, 30)
  143. // refStore, _, _ := newReactorStore(genDoc, privVals, 20)
  144. // params := testReactorParams{
  145. // logger: log.TestingLogger(),
  146. // genDoc: genDoc,
  147. // privVals: privVals,
  148. // startHeight: 10,
  149. // bufferSize: 100,
  150. // mockA: true,
  151. // }
  152. // type testEvent struct {
  153. // evType string
  154. // peer string
  155. // height int64
  156. // }
  157. // tests := []struct {
  158. // name string
  159. // params testReactorParams
  160. // msgs []testEvent
  161. // }{
  162. // {
  163. // name: "simple termination on max peer height - one peer",
  164. // params: params,
  165. // msgs: []testEvent{
  166. // {evType: "AddPeer", peer: "P1"},
  167. // {evType: "ReceiveS", peer: "P1", height: 13},
  168. // {evType: "BlockReq"},
  169. // {evType: "ReceiveB", peer: "P1", height: 11},
  170. // {evType: "BlockReq"},
  171. // {evType: "BlockReq"},
  172. // {evType: "ReceiveB", peer: "P1", height: 12},
  173. // {evType: "Process"},
  174. // {evType: "ReceiveB", peer: "P1", height: 13},
  175. // {evType: "Process"},
  176. // },
  177. // },
  178. // {
  179. // name: "simple termination on max peer height - two peers",
  180. // params: params,
  181. // msgs: []testEvent{
  182. // {evType: "AddPeer", peer: "P1"},
  183. // {evType: "AddPeer", peer: "P2"},
  184. // {evType: "ReceiveS", peer: "P1", height: 13},
  185. // {evType: "ReceiveS", peer: "P2", height: 15},
  186. // {evType: "BlockReq"},
  187. // {evType: "BlockReq"},
  188. // {evType: "ReceiveB", peer: "P1", height: 11},
  189. // {evType: "ReceiveB", peer: "P2", height: 12},
  190. // {evType: "Process"},
  191. // {evType: "BlockReq"},
  192. // {evType: "BlockReq"},
  193. // {evType: "ReceiveB", peer: "P1", height: 13},
  194. // {evType: "Process"},
  195. // {evType: "ReceiveB", peer: "P2", height: 14},
  196. // {evType: "Process"},
  197. // {evType: "BlockReq"},
  198. // {evType: "ReceiveB", peer: "P2", height: 15},
  199. // {evType: "Process"},
  200. // },
  201. // },
  202. // {
  203. // name: "termination on max peer height - two peers, noBlock error",
  204. // params: params,
  205. // msgs: []testEvent{
  206. // {evType: "AddPeer", peer: "P1"},
  207. // {evType: "AddPeer", peer: "P2"},
  208. // {evType: "ReceiveS", peer: "P1", height: 13},
  209. // {evType: "ReceiveS", peer: "P2", height: 15},
  210. // {evType: "BlockReq"},
  211. // {evType: "BlockReq"},
  212. // {evType: "ReceiveNB", peer: "P1", height: 11},
  213. // {evType: "BlockReq"},
  214. // {evType: "ReceiveB", peer: "P2", height: 12},
  215. // {evType: "ReceiveB", peer: "P2", height: 11},
  216. // {evType: "Process"},
  217. // {evType: "BlockReq"},
  218. // {evType: "BlockReq"},
  219. // {evType: "ReceiveB", peer: "P2", height: 13},
  220. // {evType: "Process"},
  221. // {evType: "ReceiveB", peer: "P2", height: 14},
  222. // {evType: "Process"},
  223. // {evType: "BlockReq"},
  224. // {evType: "ReceiveB", peer: "P2", height: 15},
  225. // {evType: "Process"},
  226. // },
  227. // },
  228. // {
  229. // name: "termination on max peer height - two peers, remove one peer",
  230. // params: params,
  231. // msgs: []testEvent{
  232. // {evType: "AddPeer", peer: "P1"},
  233. // {evType: "AddPeer", peer: "P2"},
  234. // {evType: "ReceiveS", peer: "P1", height: 13},
  235. // {evType: "ReceiveS", peer: "P2", height: 15},
  236. // {evType: "BlockReq"},
  237. // {evType: "BlockReq"},
  238. // {evType: "RemovePeer", peer: "P1"},
  239. // {evType: "BlockReq"},
  240. // {evType: "ReceiveB", peer: "P2", height: 12},
  241. // {evType: "ReceiveB", peer: "P2", height: 11},
  242. // {evType: "Process"},
  243. // {evType: "BlockReq"},
  244. // {evType: "BlockReq"},
  245. // {evType: "ReceiveB", peer: "P2", height: 13},
  246. // {evType: "Process"},
  247. // {evType: "ReceiveB", peer: "P2", height: 14},
  248. // {evType: "Process"},
  249. // {evType: "BlockReq"},
  250. // {evType: "ReceiveB", peer: "P2", height: 15},
  251. // {evType: "Process"},
  252. // },
  253. // },
  254. // }
  255. // for _, tt := range tests {
  256. // tt := tt
  257. // t.Run(tt.name, func(t *testing.T) {
  258. // reactor := newTestReactor(params)
  259. // reactor.Start()
  260. // reactor.reporter = behaviour.NewMockReporter()
  261. // mockSwitch := &mockSwitchIo{switchedToConsensus: false}
  262. // reactor.io = mockSwitch
  263. // // time for go routines to start
  264. // time.Sleep(time.Millisecond)
  265. // for _, step := range tt.msgs {
  266. // switch step.evType {
  267. // case "AddPeer":
  268. // reactor.scheduler.send(bcAddNewPeer{peerID: p2p.ID(step.peer)})
  269. // case "RemovePeer":
  270. // reactor.scheduler.send(bcRemovePeer{peerID: p2p.ID(step.peer)})
  271. // case "ReceiveS":
  272. // reactor.scheduler.send(bcStatusResponse{
  273. // peerID: p2p.ID(step.peer),
  274. // height: step.height,
  275. // time: time.Now(),
  276. // })
  277. // case "ReceiveB":
  278. // reactor.scheduler.send(bcBlockResponse{
  279. // peerID: p2p.ID(step.peer),
  280. // block: refStore.LoadBlock(step.height),
  281. // size: 10,
  282. // time: time.Now(),
  283. // })
  284. // case "ReceiveNB":
  285. // reactor.scheduler.send(bcNoBlockResponse{
  286. // peerID: p2p.ID(step.peer),
  287. // height: step.height,
  288. // time: time.Now(),
  289. // })
  290. // case "BlockReq":
  291. // reactor.scheduler.send(rTrySchedule{time: time.Now()})
  292. // case "Process":
  293. // reactor.processor.send(rProcessBlock{})
  294. // }
  295. // // give time for messages to propagate between routines
  296. // time.Sleep(time.Millisecond)
  297. // }
  298. // // time for processor to finish and reactor to switch to consensus
  299. // time.Sleep(20 * time.Millisecond)
  300. // assert.True(t, mockSwitch.hasSwitchedToConsensus())
  301. // reactor.Stop()
  302. // })
  303. // }
  304. // }
  305. func TestReactorHelperMode(t *testing.T) {
  306. var (
  307. channelID = byte(0x40)
  308. )
  309. config := cfg.ResetTestRoot("blockchain_reactor_v2_test")
  310. defer os.RemoveAll(config.RootDir)
  311. genDoc, privVals := randGenesisDoc(config.ChainID(), 1, false, 30)
  312. params := testReactorParams{
  313. logger: log.TestingLogger(),
  314. genDoc: genDoc,
  315. privVals: privVals,
  316. startHeight: 20,
  317. bufferSize: 100,
  318. mockA: true,
  319. }
  320. type testEvent struct {
  321. peer string
  322. event interface{}
  323. }
  324. tests := []struct {
  325. name string
  326. params testReactorParams
  327. msgs []testEvent
  328. }{
  329. {
  330. name: "status request",
  331. params: params,
  332. msgs: []testEvent{
  333. {"P1", bcStatusRequestMessage{}},
  334. {"P1", bcBlockRequestMessage{Height: 13}},
  335. {"P1", bcBlockRequestMessage{Height: 20}},
  336. {"P1", bcBlockRequestMessage{Height: 22}},
  337. },
  338. },
  339. }
  340. for _, tt := range tests {
  341. tt := tt
  342. t.Run(tt.name, func(t *testing.T) {
  343. reactor := newTestReactor(params)
  344. reactor.Start()
  345. mockSwitch := &mockSwitchIo{switchedToConsensus: false}
  346. reactor.io = mockSwitch
  347. for i := 0; i < len(tt.msgs); i++ {
  348. step := tt.msgs[i]
  349. switch ev := step.event.(type) {
  350. case bcStatusRequestMessage:
  351. old := mockSwitch.numStatusResponse
  352. reactor.Receive(channelID, mockPeer{id: p2p.ID(step.peer)}, cdc.MustMarshalBinaryBare(ev))
  353. assert.Equal(t, old+1, mockSwitch.numStatusResponse)
  354. case bcBlockRequestMessage:
  355. if ev.Height > params.startHeight {
  356. old := mockSwitch.numNoBlockResponse
  357. reactor.Receive(channelID, mockPeer{id: p2p.ID(step.peer)}, cdc.MustMarshalBinaryBare(ev))
  358. assert.Equal(t, old+1, mockSwitch.numNoBlockResponse)
  359. } else {
  360. old := mockSwitch.numBlockResponse
  361. reactor.Receive(channelID, mockPeer{id: p2p.ID(step.peer)}, cdc.MustMarshalBinaryBare(ev))
  362. assert.Equal(t, old+1, mockSwitch.numBlockResponse)
  363. }
  364. }
  365. }
  366. reactor.Stop()
  367. })
  368. }
  369. }
  370. func TestReactorSetSwitchNil(t *testing.T) {
  371. config := cfg.ResetTestRoot("blockchain_reactor_v2_test")
  372. defer os.RemoveAll(config.RootDir)
  373. genDoc, privVals := randGenesisDoc(config.ChainID(), 1, false, 30)
  374. reactor := newTestReactor(testReactorParams{
  375. logger: log.TestingLogger(),
  376. genDoc: genDoc,
  377. privVals: privVals,
  378. })
  379. reactor.SetSwitch(nil)
  380. assert.Nil(t, reactor.Switch)
  381. assert.Nil(t, reactor.io)
  382. }
  383. //----------------------------------------------
  384. // utility funcs
  385. func makeTxs(height int64) (txs []types.Tx) {
  386. for i := 0; i < 10; i++ {
  387. txs = append(txs, types.Tx([]byte{byte(height), byte(i)}))
  388. }
  389. return txs
  390. }
  391. func makeBlock(height int64, state sm.State, lastCommit *types.Commit) *types.Block {
  392. block, _ := state.MakeBlock(height, makeTxs(height), lastCommit, nil, state.Validators.GetProposer().Address)
  393. return block
  394. }
  395. type testApp struct {
  396. abci.BaseApplication
  397. }
  398. func randGenesisDoc(chainID string, numValidators int, randPower bool, minPower int64) (
  399. *types.GenesisDoc, []types.PrivValidator) {
  400. validators := make([]types.GenesisValidator, numValidators)
  401. privValidators := make([]types.PrivValidator, numValidators)
  402. for i := 0; i < numValidators; i++ {
  403. val, privVal := types.RandValidator(randPower, minPower)
  404. validators[i] = types.GenesisValidator{
  405. PubKey: val.PubKey,
  406. Power: val.VotingPower,
  407. }
  408. privValidators[i] = privVal
  409. }
  410. sort.Sort(types.PrivValidatorsByAddress(privValidators))
  411. return &types.GenesisDoc{
  412. GenesisTime: tmtime.Now(),
  413. ChainID: chainID,
  414. Validators: validators,
  415. }, privValidators
  416. }
  417. // Why are we importing the entire blockExecutor dependency graph here
  418. // when we have the facilities to
  419. func newReactorStore(
  420. genDoc *types.GenesisDoc,
  421. privVals []types.PrivValidator,
  422. maxBlockHeight int64) (*store.BlockStore, sm.State, *sm.BlockExecutor) {
  423. if len(privVals) != 1 {
  424. panic("only support one validator")
  425. }
  426. app := &testApp{}
  427. cc := proxy.NewLocalClientCreator(app)
  428. proxyApp := proxy.NewAppConns(cc)
  429. err := proxyApp.Start()
  430. if err != nil {
  431. panic(errors.Wrap(err, "error start app"))
  432. }
  433. stateDB := dbm.NewMemDB()
  434. blockStore := store.NewBlockStore(dbm.NewMemDB())
  435. state, err := sm.LoadStateFromDBOrGenesisDoc(stateDB, genDoc)
  436. if err != nil {
  437. panic(errors.Wrap(err, "error constructing state from genesis file"))
  438. }
  439. db := dbm.NewMemDB()
  440. blockExec := sm.NewBlockExecutor(db, log.TestingLogger(), proxyApp.Consensus(),
  441. mock.Mempool{}, sm.MockEvidencePool{})
  442. sm.SaveState(db, state)
  443. // add blocks in
  444. for blockHeight := int64(1); blockHeight <= maxBlockHeight; blockHeight++ {
  445. lastCommit := types.NewCommit(blockHeight-1, 0, types.BlockID{}, nil)
  446. if blockHeight > 1 {
  447. lastBlockMeta := blockStore.LoadBlockMeta(blockHeight - 1)
  448. lastBlock := blockStore.LoadBlock(blockHeight - 1)
  449. vote, err := types.MakeVote(
  450. lastBlock.Header.Height,
  451. lastBlockMeta.BlockID,
  452. state.Validators,
  453. privVals[0],
  454. lastBlock.Header.ChainID,
  455. time.Now(),
  456. )
  457. if err != nil {
  458. panic(err)
  459. }
  460. lastCommit = types.NewCommit(vote.Height, vote.Round,
  461. lastBlockMeta.BlockID, []types.CommitSig{vote.CommitSig()})
  462. }
  463. thisBlock := makeBlock(blockHeight, state, lastCommit)
  464. thisParts := thisBlock.MakePartSet(types.BlockPartSizeBytes)
  465. blockID := types.BlockID{Hash: thisBlock.Hash(), PartsHeader: thisParts.Header()}
  466. state, _, err = blockExec.ApplyBlock(state, blockID, thisBlock)
  467. if err != nil {
  468. panic(errors.Wrap(err, "error apply block"))
  469. }
  470. blockStore.SaveBlock(thisBlock, thisParts, lastCommit)
  471. }
  472. return blockStore, state, blockExec
  473. }