Closes#4603
Commands used (VIM):
```
:args `rg -l errors.Wrap`
:argdo normal @q | update
```
where q is a macros rewriting the `errors.Wrap` to `fmt.Errorf`.
Integrates the blockchain v2 reactor with state sync, fixes#4765. This mostly involves deferring fast syncing until after state sync completes. I tried a few different approaches, this was the least effort:
* `Reactor.events` is `nil` if no fast sync is in progress, in which case events are not dispatched - most importantly `AddPeer`.
* Accept status messages from unknown peers in the scheduler and register them as ready. On fast sync startup, broadcast status requests to all existing peers.
* When switching from state sync, first send a `bcResetState` message to the processor and scheduler to update their states - most importantly the initial block height.
* When fast sync completes, shut down event loop, scheduler and processor, and set `events` channel to `nil`.
Predominantly following the discussions regarding the conventions of using mocks, I have decided to revert back to the previous state where mocks were specialized and stored in the separate packages that used them rather then have a generalized mock in the evidence package.
This also was a problem as the state package were running tests too slow and occasionally timing out unnecessarily.
For the replay file I renamed mockEvidencePool to emptyEvidencePool to illustrate that it was intentionally like this and not give the impression that testing software was being used in production
Closes: #4786
Fixes#828. Adds state sync, as outlined in [ADR-053](https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-053-state-sync-prototype.md). See related PRs in Cosmos SDK (https://github.com/cosmos/cosmos-sdk/pull/5803) and Gaia (https://github.com/cosmos/gaia/pull/327).
This is split out of the previous PR #4645, and branched off of the ABCI interface in #4704.
* Adds a new P2P reactor which exchanges snapshots with peers, and bootstraps an empty local node from remote snapshots when requested.
* Adds a new configuration section `[statesync]` that enables state sync and configures the light client. Also enables `statesync:info` logging by default.
* Integrates state sync into node startup. Does not support the v2 blockchain reactor, since it needs some reorganization to defer startup.
<!-- < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < ☺
v ✰ Thanks for creating a PR! ✰
v Before smashing the submit button please review the checkboxes.
v If a checkbox is n/a - please still include it but + a little note why
☺ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -->
Fixes an issue reported in https://github.com/tendermint/tendermint/issues/4595#issuecomment-612667441.
Not sure if this is sufficient to fully remove the reactor, but it fixes the immediate problem.
______
For contributor use:
- [x] Wrote tests
- [x] ~Updated CHANGELOG_PENDING.md~
- [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
- [x] ~Updated relevant documentation (`docs/`) and code comments~
- [x] Re-reviewed `Files changed` in the Github PR explorer
* Added BlockStore.DeleteBlock()
* Added initial block pruner prototype
* wip
* Added BlockStore.PruneBlocks()
* Added consensus setting for block pruning
* Added BlockStore base
* Error on replay if base does not have blocks
* Handle missing blocks when sending VoteSetMaj23Message
* Error message tweak
* Properly update blockstore state
* Error message fix again
* blockchain: ignore peer missing blocks
* Added FIXME
* Added test for block replay with truncated history
* Handle peer base in blockchain reactor
* Improved replay error handling
* Added tests for Store.PruneBlocks()
* Fix non-RPC handling of truncated block history
* Panic on missing block meta in needProofBlock()
* Updated changelog
* Handle truncated block history in RPC layer
* Added info about earliest block in /status RPC
* Reorder height and base in blockchain reactor messages
* Updated changelog
* Fix tests
* Appease linter
* Minor review fixes
* Non-empty BlockStores should always have base > 0
* Update code to assume base > 0 invariant
* Added blockstore tests for pruning to 0
* Make sure we don't prune below the current base
* Added BlockStore.Size()
* config: added retain_blocks recommendations
* Update v1 blockchain reactor to handle blockstore base
* Added state database pruning
* Propagate errors on missing validator sets
* Comment tweaks
* Improved error message
Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>
* use ABCI field ResponseCommit.retain_height instead of retain-blocks config option
* remove State.RetainHeight, return value instead
* fix minor issues
* rename pruneHeights() to pruneBlocks()
* noop to fix GitHub borkage
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
* format: add format cmd & goimport repo
- replaced format command
- added goimports to format command
- ran goimports
Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>
* fix outliers & undo proto file changes
* test functions take time.Now and other minor changes
* updated remaining test files
* Update validation_test.go
* fix typo
* go fmt
* import time
Co-authored-by: Marko <marbar3778@yahoo.com>
The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor. This PR replaces #4067 which got far too large and messy after a failed attempt to rebase.
## Commits:
* Blockchainv 2 reactor:
+ I cleaner copy of the work done in #4067 which fell too far behind and was a nightmare to rebase.
+ The work includes the reactor which ties together all the seperate routines involved in the design of the blockchain v2 refactor.
* fixes after merge
* reorder iIO interface methodset
* change iO -> IO
* panic before send nil block
* rename switchToConsensus -> trySwitchToConsensus
* rename tdState -> tmState
* Update blockchain/v2/reactor.go
Co-Authored-By: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com>
* remove peer when it sends a block unsolicited
* check for not ready in markReceived
* fix error
* fix the pcFinished event
* typo fix
* add documentation for processor fields
* simplify time.Since
* try and make the linter happy
* some doc updates
* fix channel diagram
* Update adr-043-blockchain-riri-org.md
* panic on nil switch
* liting fixes
* account for nil block in bBlockResponseMessage
* panic on duplicate block enqueued by processor
* linting
* goimport reactor_test.go
Co-authored-by: Bot from GolangCI <42910462+golangcibot@users.noreply.github.com>
Co-authored-by: Anca Zamfir <ancazamfir@users.noreply.github.com>
Co-authored-by: Marko <marbar3778@yahoo.com>
Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
Marko
Erik Grinaker
Anton Kaliaev
Callum Waters
Shivani Joshi
Sean Braithwaite