* p2p: fix false-positive error logging when stopping connections
This changeset fixes two types of false-positive errors occurring during
connection shutdown.
The first occurs when the process invokes FlushStop() or Stop() on a
connection. While the previous behavior did properly wait for the sendRoutine
to finish, it did not notify the recvRoutine that the connection was shutting
down. This would cause the recvRouting to receive and error when reading and
log this error. The changeset fixes this by notifying the recvRoutine that
the connection is shutting down.
The second occurs when the connection is terminated (gracefully) by the other side.
The recvRoutine would get an EOF error during the read, log it, and stop the connection
with an error. The changeset detects EOF and gracefully shuts down the connection.
* bring back the comment about flushing
* add changelog entry
* listen for quitRecvRoutine too
* we have to call stopForError
Otherwise peer won't be removed from the peer set and maybe readded
later.
* go routines in blockchain reactor
* Added reference to the go routine diagram
* Initial commit
* cleanup
* Undo testing_logger change, committed by mistake
* Fix the test loggers
* pulled some fsm code into pool.go
* added pool tests
* changes to the design
added block requests under peer
moved the request trigger in the reactor poolRoutine, triggered now by a ticker
in general moved everything required for making block requests smarter in the poolRoutine
added a simple map of heights to keep track of what will need to be requested next
added a few more tests
* send errors to FSM in a different channel than blocks
send errors (RemovePeer) from switch on a different channel than the
one receiving blocks
renamed channels
added more pool tests
* more pool tests
* lint errors
* more tests
* more tests
* switch fast sync to new implementation
* fixed data race in tests
* cleanup
* finished fsm tests
* address golangci comments :)
* address golangci comments :)
* Added timeout on next block needed to advance
* updating docs and cleanup
* fix issue in test from previous cleanup
* cleanup
* Added termination scenarios, tests and more cleanup
* small fixes to adr, comments and cleanup
* Fix bug in sendRequest()
If we tried to send a request to a peer not present in the switch, a
missing continue statement caused the request to be blackholed in a peer
that was removed and never retried.
While this bug was manifesting, the reactor kept asking for other
blocks that would be stored and never consumed. Added the number of
unconsumed blocks in the math for requesting blocks ahead of current
processing height so eventually there will be no more blocks requested
until the already received ones are consumed.
* remove bpPeer's didTimeout field
* Use distinct err codes for peer timeout and FSM timeouts
* Don't allow peers to update with lower height
* review comments from Ethan and Zarko
* some cleanup, renaming, comments
* Move block execution in separate goroutine
* Remove pool's numPending
* review comments
* fix lint, remove old blockchain reactor and duplicates in fsm tests
* small reorg around peer after review comments
* add the reactor spec
* verify block only once
* review comments
* change to int for max number of pending requests
* cleanup and godoc
* Add configuration flag fast sync version
* golangci fixes
* fix config template
* move both reactor versions under blockchain
* cleanup, golint, renaming stuff
* updated documentation, fixed more golint warnings
* integrate with behavior package
* sync with master
* gofmt
* add changelog_pending entry
* move to improvments
* suggestion to changelog entry
* Remove db from tendemrint in favor of tendermint/tm-cmn
- remove db from `libs`
- update dependancy, there have been no breaking changes in the updated deps
- https://github.com/grpc/grpc-go/releases
- https://github.com/golang/protobuf/releases
Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>
* changelog add
* gofmt
* more gofmt
* Release branch for v0.32.1
Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>
* add links to changelog
* wording change
* Apply suggestions from code review
Comment from PR
Co-Authored-By: Ethan Buchman <ethan@coinculture.info>
* add link bump abci Version
* include doc change for abci/app
* moved abci change to features as it doesnt break the abci
* pr comments (#3803)
* pr comments
* abci changelog change
* Marko/update release1 (#3806)
* pr comments
* remove empty space
* more minor cleanup of libs
Remove unused `version.go`, `assert.go` and `libs/circle.yml`
* Update types/vote_set_test.go
Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>
* spelling change
- The removed functions are not used in Iavl, Cosmos-sdk and tendermint repos
- Code-hygenie `whoop whoop`
Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>
* Remove file heap.go
- cmn.Heap is not being used in cosmos-sdk, iavl nor tendermint repo.
Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>
* changelog entry
* Update CHANGELOG_PENDING.md
closes#2432
* change invocation of NewNode across
* custom reactor name are prefixed with CUSTOM_
* upgate changelog pending
* improve comments
* node: refactor NewNode to use functional options
Calling ensurePeers outside of ensurePeersRoutine can lead to nodes
disconnecting from us due to "sent next PEX request too soon" error.
Solution is to just dial addrs we got from src instead of calling
ensurePeers.
Refs #2093Fixes#3338
As per #2127, this refactors the RequestCheckTx ProtoBuf struct to allow for a flag indicating whether a query is a recheck or not (and allows for possible future, more nuanced states).
In order to pass this extended information through to the ABCI app, the proxy.AppConnMempool (and, for consistency, the proxy.AppConnConsensus) interface seems to need to be refactored along with abcicli.Client.
And, as per this comment, I've made the following modification to the protobuf definition for the RequestCheckTx structure:
enum CheckTxType {
New = 0;
Recheck = 1;
}
message RequestCheckTx {
bytes tx = 1;
CheckTxType type = 2;
}
* Refactor ABCI CheckTx to notify of recheck
As per #2127, this refactors the `RequestCheckTx` ProtoBuf struct to allow for:
1. a flag indicating whether a query is a recheck or not (and allows for
possible future, more nuanced states)
2. an `additional_data` bytes array to provide information for those more
nuanced states.
In order to pass this extended information through to the ABCI app, the
`proxy.AppConnMempool` (and, for consistency, the
`proxy.AppConnConsensus`) interface seems to need to be refactored.
Commits:
* Fix linting issue
* Add CHANGELOG_PENDING entry
* Remove extraneous explicit initialization
* Update ABCI spec doc to include new CheckTx params
* Rename method param for consistency
* Rename CheckTxType enum values and remove additional_data param
* changelog and version
* Add section on ABCI changes
* Update ABCI upgrade section to include events examples
* update upgrading
* more upgrading and changelog
* update changelog from pending
* refer to #rpc_changes
* minor word changes
* Refactor signature of Application.CheckTx
* Refactor signature of Application.DeliverTx
* Refactor example variable names for clarity and consistency
* Rename method variables for consistency
* Rename method variables for consistency
* add a changelog entry
* update docs
* node: fix a bug where `nil` is recorded as node's address
Solution
AddOurAddress when we know it
sw.NetAddress is nil in createAddrBookAndSetOnSwitch
it's set by n.transport.Listen function, which is called during start
Fixes#3716
* use addr instead of n.sw.NetAddress
* add both ExternalAddress and ListenAddress as our addresses
## PR
This PR introduces a fundamental breaking change to the structure of ABCI response and tx tags and the way they're processed. Namely, the SDK can support more complex and aggregated events for distribution and slashing. In addition, block responses can include duplicate keys in events.
Implement new Event type. An event has a type and a list of KV pairs (ie. list-of-lists). Typical events may look like:
"rewards": [{"amount": "5000uatom", "validator": "...", "recipient": "..."}]
"sender": [{"address": "...", "balance": "100uatom"}]
The events are indexed by {even.type}.{even.attribute[i].key}/.... In this case a client would subscribe or query for rewards.recipient='...'
ABCI response types and related types now include Events []Event instead of Tags []cmn.KVPair.
PubSub logic now publishes/matches against map[string][]string instead of map[string]string to support duplicate keys in response events (from #1385). A match is successful if the value is found in the slice of strings.
closes: #1859closes: #2905
## Commits:
* Implement Event ABCI type and updates responses to use events
* Update messages_test.go
* Update kvstore.go
* Update event_bus.go
* Update subscription.go
* Update pubsub.go
* Update kvstore.go
* Update query logic to handle slice of strings in events
* Update Empty#Matches and unit tests
* Update pubsub logic
* Update EventBus#Publish
* Update kv tx indexer
* Update godocs
* Update ResultEvent to use slice of strings; update RPC
* Update more tests
* Update abci.md
* Check for key in validateAndStringifyEvents
* Fix KV indexer to skip empty keys
* Fix linting errors
* Update CHANGELOG_PENDING.md
* Update docs/spec/abci/abci.md
Co-Authored-By: Federico Kunze <31522760+fedekunze@users.noreply.github.com>
* Update abci/types/types.proto
Co-Authored-By: Ethan Buchman <ethan@coinculture.info>
* Update docs/spec/abci/abci.md
Co-Authored-By: Ethan Buchman <ethan@coinculture.info>
* Update libs/pubsub/query/query.go
Co-Authored-By: Ethan Buchman <ethan@coinculture.info>
* Update match function to match if ANY value matches
* Implement TestSubscribeDuplicateKeys
* Update TestMatches to include multi-key test cases
* Update events.go
* Update Query interface godoc
* Update match godoc
* Add godoc for matchValue
* DRY-up tx indexing
* Return error from PublishWithEvents in EventBus#Publish
* Update PublishEventNewBlockHeader to return an error
* Fix build
* Update events doc in ABCI
* Update ABCI events godoc
* Implement TestEventBusPublishEventTxDuplicateKeys
* Update TestSubscribeDuplicateKeys to be table-driven
* Remove mod file
* Remove markdown from events godoc
* Implement TestTxSearchDeprecatedIndexing test
* Update to using go mod from dep
* Remove references to make get_vendor_deps
* Specify go version
* Set GO111MODULE=on and add -mod=readonly
* Fix exported env
* switch to using go1.12 everywhere
* Fix test scripts
* Typo:
* Prepend GO111MODULE=on
* remove dep cache
* Revert "remove dep cache"
This reverts commit 45117bda
Signed-off-by: Ismail Khoffi <Ismail.Khoffi@gmail.com>
* bring back the dependency cache and change it to cache modules instead
of vendored deps; also:
- bump version for dependency cache
- bump version on pkg-cache (includes modules directory)
Signed-off-by: Ismail Khoffi <Ismail.Khoffi@gmail.com>
* remove some more traces of dep:
- remove Gopkg.(toml | lock)
- update contributing guidlines
- set global default in circleci (GO111MODULE=on)
Signed-off-by: Ismail Khoffi <Ismail.Khoffi@gmail.com>
* global var failed for `test_cover` with
`go: unknown environment setting GO111MODULE=true`
although the var was `GO111MODULE: on`
Signed-off-by: Ismail Khoffi <Ismail.Khoffi@gmail.com>
* Changelog pending entry
Signed-off-by: Ismail Khoffi <Ismail.Khoffi@gmail.com>
* Add bbolt dependency to go.mod
Signed-off-by: Ismail Khoffi <Ismail.Khoffi@gmail.com>
* move -mod=readonly to build flags
BoltDB's accessors will return slices that are only valid for the
lifetime of the transaction. This adds copies where required to prevent
hard to debug crashes (among other things).
Fixes#3521
The function NewNetAddressStringWithOptionalID is from a time when peer
IDs were optional. They're not anymore. So this should be renamed to
NewNetAddressString and should ensure the ID is provided.
* update changelog
* use NewNetAddress in transport tests
* use NewNetAddress in TestTransportMultiplexAcceptMultiple
So we can have an error code when rpc fails.
* Update CHANGELOG_PENDING.md
Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>
* wrap correct error
* format code
* update changelog
* Update changelog
- less detail about internal stuff like the `mempool`
- focus BUG FIX entries more on what was fixed rather than how
* minor cleanup.
- remove entry for GenesisDocProvider, it's being undone
* minor fix
* Allow testnet hostnames to be overridden
This allows one to specify the `--hostname` flag multiple times, each
time providing an additional custom hostname for a respective peer
(validator or non-validator). This overrides any of the
`--hostname-prefix` or `--starting-ip-address` flags.
The string array approach is taken instead of the string slice approach
(see the pflag docs:
https://godoc.org/github.com/spf13/pflag#StringArray) because the string
slice approach (a comma-separated string) doesn't allow for cleaner
multi-line BASH scripts - where this feature is intended to be used.
* Reorder conditional for clarity with simpler earlier return
* Allow for specifying peer hostname suffix
* Quote values in help strings for greater clarity
* Fix command switch
* Add CHANGELOG_PENDING entry for PR
* Allow for unique monikers
The current approach to generating monikers for testnet nodes assigns
the local hostname of the machine on which the testnet config was
generated to all nodes. This results in the same moniker for each and
every node.
This commit makes use of the supplied `--hostname-prefix` and
`--hostname-suffix`, or `--hostname` parameters to generate unique
monikers for each node. Alternatively, another parameter
(`--random-monikers`) allows one to forcibly override all of the other
options with random hexadecimal strings.
* Update CHANGELOG_PENDING entry for new command line switch
* fix peer state init to late
Peer does not have a state yet. We set it in AddPeer.
We need an new interface before mconnection is started
* pex message to soon
fix reconnection pex send too fast,
error is caused lastReceivedRequests is still
not deleted when a peer reconnected
* add test case for initpeer
* add prove case
* remove potentially infinite loop
* Update consensus/reactor.go
Co-Authored-By: guagualvcha <baifudong@lancai.cn>
* Update consensus/reactor_test.go
Co-Authored-By: guagualvcha <baifudong@lancai.cn>
* document Reactor interface better
* refactor TestReactorReceiveDoesNotPanicIfAddPeerHasntBeenCalledYet
* fix merge conflicts
* blockchain: remove peer's ID from the pool in InitPeer
Refs #3338
* pex: resetPeersRequestsInfo both upon InitPeer and RemovePeer
* ensure RemovePeer is always called before InitPeer
by removing the peer from the switch last (after we've stopped it and
removed from all reactors)
* add some comments for ConsensusReactor#InitPeer
* fix pex reactor
* format code
* fix spelling
* update changelog
* remove unused methods
* do not clear lastReceivedRequests upon error
only in RemovePeer
* call InitPeer before we start the peer!
* add a comment to InitPeer
* write a test
* use waitUntilSwitchHasAtLeastNPeers func
* bring back timeouts
* Test to ensure Receive panics if InitPeer has not been called
* p2p: initial implementation of peer behaviour
* [p2p] re-use newMockPeer
* p2p: add inline docs for peer_behaviour interface
* [p2p] Align PeerBehaviour interface (#3558)
* [p2p] make switchedPeerHebaviour private
* [p2p] make storePeerHebaviour private
* [p2p] Add CHANGELOG_PENDING entry for PeerBehaviour
* [p2p] Adjustment naming for PeerBehaviour
* [p2p] Add coarse lock around storedPeerBehaviour
* [p2p]: Fix non-pointer methods in storedPeerBehaviour
+ Structs with embeded locks must specify all methods with pointer
receivers to avoid creating a copy of the embeded lock.
* [p2p] Thorough refactoring based on comments in #3552
+ Decouple PeerBehaviour interface from Peer by parametrizing
methods with `p2p.ID` instead of `p2p.Peer`
+ Setter methods wrapped in a write lock
+ Getter methods wrapped in a read lock
+ Getter methods on storedPeerBehaviour now take a `p2p.ID`
+ Getter methods return a copy of underlying stored behaviours
+ Add doc strings to public types and methods
* [p2p] make structs public
* [p2p] Test empty StoredPeerBehaviour
* [p2p] typo fix
* [p2p] add TestStoredPeerBehaviourConcurrency
+ Add a test which uses StoredPeerBehaviour in multiple goroutines
to ensure thread-safety.
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour_test.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* Update p2p/peer_behaviour.go
Co-Authored-By: brapse <brapse@gmail.com>
* [p2p] field ordering convention
* p2p: peer behaviour refactor
+ Change naming of reporting behaviour to `Report`
+ Remove the responsibility of distinguishing between the categories
of good and bad behaviour and instead focus on reporting behaviour.
* p2p: rename PeerReporter -> PeerBehaviourReporter
* libs/db: conditional compilation
For cleveldb: go build -tags cleveldb
For boltdb: go build -tags boltdb
Fixes#3611
* document db_backend param better
* remove deprecated LevelDBBackend
* update changelog
* add missing lines
* add new line
* fix TestRemoteDB
* add a line about boltdb tag
* Revert "remove deprecated LevelDBBackend"
This reverts commit 1aa85453f7.
* make PR non breaking
* change DEPRECATED label format
https://stackoverflow.com/a/36360323/820520
Anton Kaliaev
Anca Zamfir
Jun Kimura
zjubfd
Marko
Roman Useinov
Ashley Vega
Jay Namsayin
Thane Thomson
Ivan Kushmantsev
Ethan Buchman
needkane
Alexander Bezobchuk
Jack Zampolin
Yawning Angel
Haochuan Guo
Sean Braithwaite
Carlos Flores
yutianwu