You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

1068 lines
37 KiB

deps: update gogo/protobuf version from v1.2.1 to v1.3.0 (#3947) * update gogo/protobuf version from v1.2.1 to v1.3.0 Also errcheck from v1.1.0 to v1.2.0 See full diff in - https://github.com/gogo/protobuf/compare/v1.2.1...v1.3.0 - https://github.com/kisielk/errcheck/compare/v1.1.0...v1.2.0 Changelog: Tested versions: go 1.12.9 protoc 3.7.1 Improvements: plugin/stringer - Handle repeated and/or nullable types a bit better now. plugin/size - Remove the loop in sovXXX by using bit twiddling. Thanks: https://github.com/apelisse plugin/marshalto - Implemented a reverse marshal strategy which allows for faster marshalling. This now avoids a recursive (and repeated) call to Size(). Thanks: https://github.com/apelisse plugin/compare - Added support for for oneof types. Bug fixes: protoc-gen-gogo/generator - Fix assignment to entry in nil map. Thanks: https://github.com/tgulacsi protoc-gen-gogo/generator - Allows plugins to call RecordTypeUse without panicking. Thanks: https://github.com/fedenusy proto/extensions - Fixed set extension regression. We did not clear the extensions before setting. io/uint32 - fix uint32reader bug that causes ReadMsg to recreate buffer when lengths are the same. Thanks: https://github.com/SebiSujar proto/table_merge: Fix merge of non-nullable slices. Thanks: https://github.com/euroelessar Upstream commits: merged in golang/protobuf commit 318d17de72747ed1c16502681db4b2bb709a92d0 - Add UnimplementedServer for server interface merged in golang/protobuf commit b85cd75de734650db18a99a943fe351d41387800 - protoc-gen-go/grpc: inline errUnimplemented function merged in golang/protobuf commit d3c38a4eb4970272b87a425ae00ccc4548e2f9bb - protoc-gen-go/grpc: use status and code packages only if needed merged in golang/protobuf commit e91709a02e0e8ff8b86b7aa913fdc9ae9498e825 - fix indentation in jsonpb with Any messages merged in golang/protobuf commit 8d0c54c1246661d9a51ca0ba455d22116d485eaa - protoc-gen-go: generate XXX_OneofWrappers instead of XXX_OneofFuncs Misc: extensions.md - Markdown update. Thanks: https://github.com/TennyZhuang Readme.md - Added user. go/protoc update - Updated to go1.12.x and protoc 3.7.1 Makefile update - fix go vet shadow tool reference test/mixbench - Update mixbench tool. Expose runnable benchmarks via flags. * update certstrap * update golangci-lint from v1.13.2 to v1.17.1 * update gox as well * test * comment out golangci temporary * comment out go-deadlock deps
5 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
6 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
6 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
6 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
6 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
6 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
6 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
6 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
deps: update gogo/protobuf version from v1.2.1 to v1.3.0 (#3947) * update gogo/protobuf version from v1.2.1 to v1.3.0 Also errcheck from v1.1.0 to v1.2.0 See full diff in - https://github.com/gogo/protobuf/compare/v1.2.1...v1.3.0 - https://github.com/kisielk/errcheck/compare/v1.1.0...v1.2.0 Changelog: Tested versions: go 1.12.9 protoc 3.7.1 Improvements: plugin/stringer - Handle repeated and/or nullable types a bit better now. plugin/size - Remove the loop in sovXXX by using bit twiddling. Thanks: https://github.com/apelisse plugin/marshalto - Implemented a reverse marshal strategy which allows for faster marshalling. This now avoids a recursive (and repeated) call to Size(). Thanks: https://github.com/apelisse plugin/compare - Added support for for oneof types. Bug fixes: protoc-gen-gogo/generator - Fix assignment to entry in nil map. Thanks: https://github.com/tgulacsi protoc-gen-gogo/generator - Allows plugins to call RecordTypeUse without panicking. Thanks: https://github.com/fedenusy proto/extensions - Fixed set extension regression. We did not clear the extensions before setting. io/uint32 - fix uint32reader bug that causes ReadMsg to recreate buffer when lengths are the same. Thanks: https://github.com/SebiSujar proto/table_merge: Fix merge of non-nullable slices. Thanks: https://github.com/euroelessar Upstream commits: merged in golang/protobuf commit 318d17de72747ed1c16502681db4b2bb709a92d0 - Add UnimplementedServer for server interface merged in golang/protobuf commit b85cd75de734650db18a99a943fe351d41387800 - protoc-gen-go/grpc: inline errUnimplemented function merged in golang/protobuf commit d3c38a4eb4970272b87a425ae00ccc4548e2f9bb - protoc-gen-go/grpc: use status and code packages only if needed merged in golang/protobuf commit e91709a02e0e8ff8b86b7aa913fdc9ae9498e825 - fix indentation in jsonpb with Any messages merged in golang/protobuf commit 8d0c54c1246661d9a51ca0ba455d22116d485eaa - protoc-gen-go: generate XXX_OneofWrappers instead of XXX_OneofFuncs Misc: extensions.md - Markdown update. Thanks: https://github.com/TennyZhuang Readme.md - Added user. go/protoc update - Updated to go1.12.x and protoc 3.7.1 Makefile update - fix go vet shadow tool reference test/mixbench - Update mixbench tool. Expose runnable benchmarks via flags. * update certstrap * update golangci-lint from v1.13.2 to v1.17.1 * update gox as well * test * comment out golangci temporary * comment out go-deadlock deps
5 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
6 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
6 years ago
  1. package config
  2. import (
  3. "encoding/hex"
  4. "errors"
  5. "fmt"
  6. "net/http"
  7. "os"
  8. "path/filepath"
  9. "time"
  10. )
  11. const (
  12. // FuzzModeDrop is a mode in which we randomly drop reads/writes, connections or sleep
  13. FuzzModeDrop = iota
  14. // FuzzModeDelay is a mode in which we randomly sleep
  15. FuzzModeDelay
  16. // LogFormatPlain is a format for colored text
  17. LogFormatPlain = "plain"
  18. // LogFormatJSON is a format for json output
  19. LogFormatJSON = "json"
  20. )
  21. // NOTE: Most of the structs & relevant comments + the
  22. // default configuration options were used to manually
  23. // generate the config.toml. Please reflect any changes
  24. // made here in the defaultConfigTemplate constant in
  25. // config/toml.go
  26. // NOTE: libs/cli must know to look in the config dir!
  27. var (
  28. DefaultTendermintDir = ".tendermint"
  29. defaultConfigDir = "config"
  30. defaultDataDir = "data"
  31. defaultConfigFileName = "config.toml"
  32. defaultGenesisJSONName = "genesis.json"
  33. defaultPrivValKeyName = "priv_validator_key.json"
  34. defaultPrivValStateName = "priv_validator_state.json"
  35. defaultNodeKeyName = "node_key.json"
  36. defaultAddrBookName = "addrbook.json"
  37. defaultConfigFilePath = filepath.Join(defaultConfigDir, defaultConfigFileName)
  38. defaultGenesisJSONPath = filepath.Join(defaultConfigDir, defaultGenesisJSONName)
  39. defaultPrivValKeyPath = filepath.Join(defaultConfigDir, defaultPrivValKeyName)
  40. defaultPrivValStatePath = filepath.Join(defaultDataDir, defaultPrivValStateName)
  41. defaultNodeKeyPath = filepath.Join(defaultConfigDir, defaultNodeKeyName)
  42. defaultAddrBookPath = filepath.Join(defaultConfigDir, defaultAddrBookName)
  43. )
  44. // Config defines the top level configuration for a Tendermint node
  45. type Config struct {
  46. // Top level options use an anonymous struct
  47. BaseConfig `mapstructure:",squash"`
  48. // Options for services
  49. RPC *RPCConfig `mapstructure:"rpc"`
  50. P2P *P2PConfig `mapstructure:"p2p"`
  51. Mempool *MempoolConfig `mapstructure:"mempool"`
  52. StateSync *StateSyncConfig `mapstructure:"statesync"`
  53. FastSync *FastSyncConfig `mapstructure:"fastsync"`
  54. Consensus *ConsensusConfig `mapstructure:"consensus"`
  55. TxIndex *TxIndexConfig `mapstructure:"tx-index"`
  56. Instrumentation *InstrumentationConfig `mapstructure:"instrumentation"`
  57. }
  58. // DefaultConfig returns a default configuration for a Tendermint node
  59. func DefaultConfig() *Config {
  60. return &Config{
  61. BaseConfig: DefaultBaseConfig(),
  62. RPC: DefaultRPCConfig(),
  63. P2P: DefaultP2PConfig(),
  64. Mempool: DefaultMempoolConfig(),
  65. StateSync: DefaultStateSyncConfig(),
  66. FastSync: DefaultFastSyncConfig(),
  67. Consensus: DefaultConsensusConfig(),
  68. TxIndex: DefaultTxIndexConfig(),
  69. Instrumentation: DefaultInstrumentationConfig(),
  70. }
  71. }
  72. // TestConfig returns a configuration that can be used for testing
  73. func TestConfig() *Config {
  74. return &Config{
  75. BaseConfig: TestBaseConfig(),
  76. RPC: TestRPCConfig(),
  77. P2P: TestP2PConfig(),
  78. Mempool: TestMempoolConfig(),
  79. StateSync: TestStateSyncConfig(),
  80. FastSync: TestFastSyncConfig(),
  81. Consensus: TestConsensusConfig(),
  82. TxIndex: TestTxIndexConfig(),
  83. Instrumentation: TestInstrumentationConfig(),
  84. }
  85. }
  86. // SetRoot sets the RootDir for all Config structs
  87. func (cfg *Config) SetRoot(root string) *Config {
  88. cfg.BaseConfig.RootDir = root
  89. cfg.RPC.RootDir = root
  90. cfg.P2P.RootDir = root
  91. cfg.Mempool.RootDir = root
  92. cfg.Consensus.RootDir = root
  93. return cfg
  94. }
  95. // ValidateBasic performs basic validation (checking param bounds, etc.) and
  96. // returns an error if any check fails.
  97. func (cfg *Config) ValidateBasic() error {
  98. if err := cfg.BaseConfig.ValidateBasic(); err != nil {
  99. return err
  100. }
  101. if err := cfg.RPC.ValidateBasic(); err != nil {
  102. return fmt.Errorf("error in [rpc] section: %w", err)
  103. }
  104. if err := cfg.P2P.ValidateBasic(); err != nil {
  105. return fmt.Errorf("error in [p2p] section: %w", err)
  106. }
  107. if err := cfg.Mempool.ValidateBasic(); err != nil {
  108. return fmt.Errorf("error in [mempool] section: %w", err)
  109. }
  110. if err := cfg.StateSync.ValidateBasic(); err != nil {
  111. return fmt.Errorf("error in [statesync] section: %w", err)
  112. }
  113. if err := cfg.FastSync.ValidateBasic(); err != nil {
  114. return fmt.Errorf("error in [fastsync] section: %w", err)
  115. }
  116. if err := cfg.Consensus.ValidateBasic(); err != nil {
  117. return fmt.Errorf("error in [consensus] section: %w", err)
  118. }
  119. if err := cfg.Instrumentation.ValidateBasic(); err != nil {
  120. return fmt.Errorf("error in [instrumentation] section: %w", err)
  121. }
  122. return nil
  123. }
  124. //-----------------------------------------------------------------------------
  125. // BaseConfig
  126. // BaseConfig defines the base configuration for a Tendermint node
  127. type BaseConfig struct { //nolint: maligned
  128. // chainID is unexposed and immutable but here for convenience
  129. chainID string
  130. // The root directory for all data.
  131. // This should be set in viper so it can unmarshal into this struct
  132. RootDir string `mapstructure:"home"`
  133. // TCP or UNIX socket address of the ABCI application,
  134. // or the name of an ABCI application compiled in with the Tendermint binary
  135. ProxyApp string `mapstructure:"proxy-app"`
  136. // A custom human readable name for this node
  137. Moniker string `mapstructure:"moniker"`
  138. // If this node is many blocks behind the tip of the chain, FastSync
  139. // allows them to catchup quickly by downloading blocks in parallel
  140. // and verifying their commits
  141. FastSyncMode bool `mapstructure:"fast-sync"`
  142. // Database backend: goleveldb | cleveldb | boltdb | rocksdb
  143. // * goleveldb (github.com/syndtr/goleveldb - most popular implementation)
  144. // - pure go
  145. // - stable
  146. // * cleveldb (uses levigo wrapper)
  147. // - fast
  148. // - requires gcc
  149. // - use cleveldb build tag (go build -tags cleveldb)
  150. // * boltdb (uses etcd's fork of bolt - github.com/etcd-io/bbolt)
  151. // - EXPERIMENTAL
  152. // - may be faster is some use-cases (random reads - indexer)
  153. // - use boltdb build tag (go build -tags boltdb)
  154. // * rocksdb (uses github.com/tecbot/gorocksdb)
  155. // - EXPERIMENTAL
  156. // - requires gcc
  157. // - use rocksdb build tag (go build -tags rocksdb)
  158. // * badgerdb (uses github.com/dgraph-io/badger)
  159. // - EXPERIMENTAL
  160. // - use badgerdb build tag (go build -tags badgerdb)
  161. DBBackend string `mapstructure:"db-backend"`
  162. // Database directory
  163. DBPath string `mapstructure:"db-dir"`
  164. // Output level for logging
  165. LogLevel string `mapstructure:"log-level"`
  166. // Output format: 'plain' (colored text) or 'json'
  167. LogFormat string `mapstructure:"log-format"`
  168. // Path to the JSON file containing the initial validator set and other meta data
  169. Genesis string `mapstructure:"genesis-file"`
  170. // Path to the JSON file containing the private key to use as a validator in the consensus protocol
  171. PrivValidatorKey string `mapstructure:"priv-validator-key-file"`
  172. // Path to the JSON file containing the last sign state of a validator
  173. PrivValidatorState string `mapstructure:"priv-validator-state-file"`
  174. // TCP or UNIX socket address for Tendermint to listen on for
  175. // connections from an external PrivValidator process
  176. PrivValidatorListenAddr string `mapstructure:"priv-validator-laddr"`
  177. // A JSON file containing the private key to use for p2p authenticated encryption
  178. NodeKey string `mapstructure:"node-key-file"`
  179. // Mechanism to connect to the ABCI application: socket | grpc
  180. ABCI string `mapstructure:"abci"`
  181. // If true, query the ABCI app on connecting to a new peer
  182. // so the app can decide if we should keep the connection or not
  183. FilterPeers bool `mapstructure:"filter-peers"` // false
  184. }
  185. // DefaultBaseConfig returns a default base configuration for a Tendermint node
  186. func DefaultBaseConfig() BaseConfig {
  187. return BaseConfig{
  188. Genesis: defaultGenesisJSONPath,
  189. PrivValidatorKey: defaultPrivValKeyPath,
  190. PrivValidatorState: defaultPrivValStatePath,
  191. NodeKey: defaultNodeKeyPath,
  192. Moniker: defaultMoniker,
  193. ProxyApp: "tcp://127.0.0.1:26658",
  194. ABCI: "socket",
  195. LogLevel: DefaultPackageLogLevels(),
  196. LogFormat: LogFormatPlain,
  197. FastSyncMode: true,
  198. FilterPeers: false,
  199. DBBackend: "goleveldb",
  200. DBPath: "data",
  201. }
  202. }
  203. // TestBaseConfig returns a base configuration for testing a Tendermint node
  204. func TestBaseConfig() BaseConfig {
  205. cfg := DefaultBaseConfig()
  206. cfg.chainID = "tendermint_test"
  207. cfg.ProxyApp = "kvstore"
  208. cfg.FastSyncMode = false
  209. cfg.DBBackend = "memdb"
  210. return cfg
  211. }
  212. func (cfg BaseConfig) ChainID() string {
  213. return cfg.chainID
  214. }
  215. // GenesisFile returns the full path to the genesis.json file
  216. func (cfg BaseConfig) GenesisFile() string {
  217. return rootify(cfg.Genesis, cfg.RootDir)
  218. }
  219. // PrivValidatorKeyFile returns the full path to the priv_validator_key.json file
  220. func (cfg BaseConfig) PrivValidatorKeyFile() string {
  221. return rootify(cfg.PrivValidatorKey, cfg.RootDir)
  222. }
  223. // PrivValidatorFile returns the full path to the priv_validator_state.json file
  224. func (cfg BaseConfig) PrivValidatorStateFile() string {
  225. return rootify(cfg.PrivValidatorState, cfg.RootDir)
  226. }
  227. // NodeKeyFile returns the full path to the node_key.json file
  228. func (cfg BaseConfig) NodeKeyFile() string {
  229. return rootify(cfg.NodeKey, cfg.RootDir)
  230. }
  231. // DBDir returns the full path to the database directory
  232. func (cfg BaseConfig) DBDir() string {
  233. return rootify(cfg.DBPath, cfg.RootDir)
  234. }
  235. // ValidateBasic performs basic validation (checking param bounds, etc.) and
  236. // returns an error if any check fails.
  237. func (cfg BaseConfig) ValidateBasic() error {
  238. switch cfg.LogFormat {
  239. case LogFormatPlain, LogFormatJSON:
  240. default:
  241. return errors.New("unknown log format (must be 'plain' or 'json')")
  242. }
  243. return nil
  244. }
  245. // DefaultLogLevel returns a default log level of "error"
  246. func DefaultLogLevel() string {
  247. return "error"
  248. }
  249. // DefaultPackageLogLevels returns a default log level setting so all packages
  250. // log at "error", while the `state` and `main` packages log at "info"
  251. func DefaultPackageLogLevels() string {
  252. return fmt.Sprintf("main:info,state:info,statesync:info,*:%s", DefaultLogLevel())
  253. }
  254. //-----------------------------------------------------------------------------
  255. // RPCConfig
  256. // RPCConfig defines the configuration options for the Tendermint RPC server
  257. type RPCConfig struct {
  258. RootDir string `mapstructure:"home"`
  259. // TCP or UNIX socket address for the RPC server to listen on
  260. ListenAddress string `mapstructure:"laddr"`
  261. // A list of origins a cross-domain request can be executed from.
  262. // If the special '*' value is present in the list, all origins will be allowed.
  263. // An origin may contain a wildcard (*) to replace 0 or more characters (i.e.: http://*.domain.com).
  264. // Only one wildcard can be used per origin.
  265. CORSAllowedOrigins []string `mapstructure:"cors-allowed-origins"`
  266. // A list of methods the client is allowed to use with cross-domain requests.
  267. CORSAllowedMethods []string `mapstructure:"cors-allowed-methods"`
  268. // A list of non simple headers the client is allowed to use with cross-domain requests.
  269. CORSAllowedHeaders []string `mapstructure:"cors-allowed-headers"`
  270. // TCP or UNIX socket address for the gRPC server to listen on
  271. // NOTE: This server only supports /broadcast_tx_commit
  272. GRPCListenAddress string `mapstructure:"grpc-laddr"`
  273. // Maximum number of simultaneous connections.
  274. // Does not include RPC (HTTP&WebSocket) connections. See max-open-connections
  275. // If you want to accept a larger number than the default, make sure
  276. // you increase your OS limits.
  277. // 0 - unlimited.
  278. GRPCMaxOpenConnections int `mapstructure:"grpc-max-open-connections"`
  279. // Activate unsafe RPC commands like /dial-persistent-peers and /unsafe-flush-mempool
  280. Unsafe bool `mapstructure:"unsafe"`
  281. // Maximum number of simultaneous connections (including WebSocket).
  282. // Does not include gRPC connections. See grpc-max-open-connections
  283. // If you want to accept a larger number than the default, make sure
  284. // you increase your OS limits.
  285. // 0 - unlimited.
  286. // Should be < {ulimit -Sn} - {MaxNumInboundPeers} - {MaxNumOutboundPeers} - {N of wal, db and other open files}
  287. // 1024 - 40 - 10 - 50 = 924 = ~900
  288. MaxOpenConnections int `mapstructure:"max-open-connections"`
  289. // Maximum number of unique clientIDs that can /subscribe
  290. // If you're using /broadcast_tx_commit, set to the estimated maximum number
  291. // of broadcast_tx_commit calls per block.
  292. MaxSubscriptionClients int `mapstructure:"max-subscription-clients"`
  293. // Maximum number of unique queries a given client can /subscribe to
  294. // If you're using GRPC (or Local RPC client) and /broadcast_tx_commit, set
  295. // to the estimated maximum number of broadcast_tx_commit calls per block.
  296. MaxSubscriptionsPerClient int `mapstructure:"max-subscriptions-per-client"`
  297. // How long to wait for a tx to be committed during /broadcast_tx_commit
  298. // WARNING: Using a value larger than 10s will result in increasing the
  299. // global HTTP write timeout, which applies to all connections and endpoints.
  300. // See https://github.com/tendermint/tendermint/issues/3435
  301. TimeoutBroadcastTxCommit time.Duration `mapstructure:"timeout-broadcast-tx-commit"`
  302. // Maximum size of request body, in bytes
  303. MaxBodyBytes int64 `mapstructure:"max-body-bytes"`
  304. // Maximum size of request header, in bytes
  305. MaxHeaderBytes int `mapstructure:"max-header-bytes"`
  306. // The path to a file containing certificate that is used to create the HTTPS server.
  307. // Migth be either absolute path or path related to tendermint's config directory.
  308. //
  309. // If the certificate is signed by a certificate authority,
  310. // the certFile should be the concatenation of the server's certificate, any intermediates,
  311. // and the CA's certificate.
  312. //
  313. // NOTE: both tls-cert-file and tls-key-file must be present for Tendermint to create HTTPS server.
  314. // Otherwise, HTTP server is run.
  315. TLSCertFile string `mapstructure:"tls-cert-file"`
  316. // The path to a file containing matching private key that is used to create the HTTPS server.
  317. // Migth be either absolute path or path related to tendermint's config directory.
  318. //
  319. // NOTE: both tls-cert-file and tls-key-file must be present for Tendermint to create HTTPS server.
  320. // Otherwise, HTTP server is run.
  321. TLSKeyFile string `mapstructure:"tls-key-file"`
  322. // pprof listen address (https://golang.org/pkg/net/http/pprof)
  323. PprofListenAddress string `mapstructure:"pprof-laddr"`
  324. }
  325. // DefaultRPCConfig returns a default configuration for the RPC server
  326. func DefaultRPCConfig() *RPCConfig {
  327. return &RPCConfig{
  328. ListenAddress: "tcp://127.0.0.1:26657",
  329. CORSAllowedOrigins: []string{},
  330. CORSAllowedMethods: []string{http.MethodHead, http.MethodGet, http.MethodPost},
  331. CORSAllowedHeaders: []string{"Origin", "Accept", "Content-Type", "X-Requested-With", "X-Server-Time"},
  332. GRPCListenAddress: "",
  333. GRPCMaxOpenConnections: 900,
  334. Unsafe: false,
  335. MaxOpenConnections: 900,
  336. MaxSubscriptionClients: 100,
  337. MaxSubscriptionsPerClient: 5,
  338. TimeoutBroadcastTxCommit: 10 * time.Second,
  339. MaxBodyBytes: int64(1000000), // 1MB
  340. MaxHeaderBytes: 1 << 20, // same as the net/http default
  341. TLSCertFile: "",
  342. TLSKeyFile: "",
  343. }
  344. }
  345. // TestRPCConfig returns a configuration for testing the RPC server
  346. func TestRPCConfig() *RPCConfig {
  347. cfg := DefaultRPCConfig()
  348. cfg.ListenAddress = "tcp://127.0.0.1:36657"
  349. cfg.GRPCListenAddress = "tcp://127.0.0.1:36658"
  350. cfg.Unsafe = true
  351. return cfg
  352. }
  353. // ValidateBasic performs basic validation (checking param bounds, etc.) and
  354. // returns an error if any check fails.
  355. func (cfg *RPCConfig) ValidateBasic() error {
  356. if cfg.GRPCMaxOpenConnections < 0 {
  357. return errors.New("grpc-max-open-connections can't be negative")
  358. }
  359. if cfg.MaxOpenConnections < 0 {
  360. return errors.New("max-open-connections can't be negative")
  361. }
  362. if cfg.MaxSubscriptionClients < 0 {
  363. return errors.New("max-subscription-clients can't be negative")
  364. }
  365. if cfg.MaxSubscriptionsPerClient < 0 {
  366. return errors.New("max-subscriptions-per-client can't be negative")
  367. }
  368. if cfg.TimeoutBroadcastTxCommit < 0 {
  369. return errors.New("timeout-broadcast-tx-commit can't be negative")
  370. }
  371. if cfg.MaxBodyBytes < 0 {
  372. return errors.New("max-body-bytes can't be negative")
  373. }
  374. if cfg.MaxHeaderBytes < 0 {
  375. return errors.New("max-header-bytes can't be negative")
  376. }
  377. return nil
  378. }
  379. // IsCorsEnabled returns true if cross-origin resource sharing is enabled.
  380. func (cfg *RPCConfig) IsCorsEnabled() bool {
  381. return len(cfg.CORSAllowedOrigins) != 0
  382. }
  383. func (cfg RPCConfig) KeyFile() string {
  384. path := cfg.TLSKeyFile
  385. if filepath.IsAbs(path) {
  386. return path
  387. }
  388. return rootify(filepath.Join(defaultConfigDir, path), cfg.RootDir)
  389. }
  390. func (cfg RPCConfig) CertFile() string {
  391. path := cfg.TLSCertFile
  392. if filepath.IsAbs(path) {
  393. return path
  394. }
  395. return rootify(filepath.Join(defaultConfigDir, path), cfg.RootDir)
  396. }
  397. func (cfg RPCConfig) IsTLSEnabled() bool {
  398. return cfg.TLSCertFile != "" && cfg.TLSKeyFile != ""
  399. }
  400. //-----------------------------------------------------------------------------
  401. // P2PConfig
  402. // P2PConfig defines the configuration options for the Tendermint peer-to-peer networking layer
  403. type P2PConfig struct { //nolint: maligned
  404. RootDir string `mapstructure:"home"`
  405. // Address to listen for incoming connections
  406. ListenAddress string `mapstructure:"laddr"`
  407. // Address to advertise to peers for them to dial
  408. ExternalAddress string `mapstructure:"external-address"`
  409. // Comma separated list of seed nodes to connect to
  410. // We only use these if we can’t connect to peers in the addrbook
  411. Seeds string `mapstructure:"seeds"`
  412. // Comma separated list of nodes to keep persistent connections to
  413. PersistentPeers string `mapstructure:"persistent-peers"`
  414. // UPNP port forwarding
  415. UPNP bool `mapstructure:"upnp"`
  416. // Path to address book
  417. AddrBook string `mapstructure:"addr-book-file"`
  418. // Set true for strict address routability rules
  419. // Set false for private or local networks
  420. AddrBookStrict bool `mapstructure:"addr-book-strict"`
  421. // Maximum number of inbound peers
  422. MaxNumInboundPeers int `mapstructure:"max-num-inbound-peers"`
  423. // Maximum number of outbound peers to connect to, excluding persistent peers
  424. MaxNumOutboundPeers int `mapstructure:"max-num-outbound-peers"`
  425. // List of node IDs, to which a connection will be (re)established ignoring any existing limits
  426. UnconditionalPeerIDs string `mapstructure:"unconditional-peer-ids"`
  427. // Maximum pause when redialing a persistent peer (if zero, exponential backoff is used)
  428. PersistentPeersMaxDialPeriod time.Duration `mapstructure:"persistent-peers-max-dial-period"`
  429. // Time to wait before flushing messages out on the connection
  430. FlushThrottleTimeout time.Duration `mapstructure:"flush-throttle-timeout"`
  431. // Maximum size of a message packet payload, in bytes
  432. MaxPacketMsgPayloadSize int `mapstructure:"max-packet-msg-payload-size"`
  433. // Rate at which packets can be sent, in bytes/second
  434. SendRate int64 `mapstructure:"send-rate"`
  435. // Rate at which packets can be received, in bytes/second
  436. RecvRate int64 `mapstructure:"recv-rate"`
  437. // Set true to enable the peer-exchange reactor
  438. PexReactor bool `mapstructure:"pex"`
  439. // Seed mode, in which node constantly crawls the network and looks for
  440. // peers. If another node asks it for addresses, it responds and disconnects.
  441. //
  442. // Does not work if the peer-exchange reactor is disabled.
  443. SeedMode bool `mapstructure:"seed-mode"`
  444. // Comma separated list of peer IDs to keep private (will not be gossiped to
  445. // other peers)
  446. PrivatePeerIDs string `mapstructure:"private-peer-ids"`
  447. // Toggle to disable guard against peers connecting from the same ip.
  448. AllowDuplicateIP bool `mapstructure:"allow-duplicate-ip"`
  449. // Peer connection configuration.
  450. HandshakeTimeout time.Duration `mapstructure:"handshake-timeout"`
  451. DialTimeout time.Duration `mapstructure:"dial-timeout"`
  452. // Testing params.
  453. // Force dial to fail
  454. TestDialFail bool `mapstructure:"test-dial-fail"`
  455. }
  456. // DefaultP2PConfig returns a default configuration for the peer-to-peer layer
  457. func DefaultP2PConfig() *P2PConfig {
  458. return &P2PConfig{
  459. ListenAddress: "tcp://0.0.0.0:26656",
  460. ExternalAddress: "",
  461. UPNP: false,
  462. AddrBook: defaultAddrBookPath,
  463. AddrBookStrict: true,
  464. MaxNumInboundPeers: 40,
  465. MaxNumOutboundPeers: 10,
  466. PersistentPeersMaxDialPeriod: 0 * time.Second,
  467. FlushThrottleTimeout: 100 * time.Millisecond,
  468. MaxPacketMsgPayloadSize: 1024, // 1 kB
  469. SendRate: 5120000, // 5 mB/s
  470. RecvRate: 5120000, // 5 mB/s
  471. PexReactor: true,
  472. SeedMode: false,
  473. AllowDuplicateIP: false,
  474. HandshakeTimeout: 20 * time.Second,
  475. DialTimeout: 3 * time.Second,
  476. TestDialFail: false,
  477. }
  478. }
  479. // TestP2PConfig returns a configuration for testing the peer-to-peer layer
  480. func TestP2PConfig() *P2PConfig {
  481. cfg := DefaultP2PConfig()
  482. cfg.ListenAddress = "tcp://127.0.0.1:36656"
  483. cfg.FlushThrottleTimeout = 10 * time.Millisecond
  484. cfg.AllowDuplicateIP = true
  485. return cfg
  486. }
  487. // AddrBookFile returns the full path to the address book
  488. func (cfg *P2PConfig) AddrBookFile() string {
  489. return rootify(cfg.AddrBook, cfg.RootDir)
  490. }
  491. // ValidateBasic performs basic validation (checking param bounds, etc.) and
  492. // returns an error if any check fails.
  493. func (cfg *P2PConfig) ValidateBasic() error {
  494. if cfg.MaxNumInboundPeers < 0 {
  495. return errors.New("max-num-inbound-peers can't be negative")
  496. }
  497. if cfg.MaxNumOutboundPeers < 0 {
  498. return errors.New("max-num-outbound-peers can't be negative")
  499. }
  500. if cfg.FlushThrottleTimeout < 0 {
  501. return errors.New("flush-throttle-timeout can't be negative")
  502. }
  503. if cfg.PersistentPeersMaxDialPeriod < 0 {
  504. return errors.New("persistent-peers-max-dial-period can't be negative")
  505. }
  506. if cfg.MaxPacketMsgPayloadSize < 0 {
  507. return errors.New("max-packet-msg-payload-size can't be negative")
  508. }
  509. if cfg.SendRate < 0 {
  510. return errors.New("send-rate can't be negative")
  511. }
  512. if cfg.RecvRate < 0 {
  513. return errors.New("recv-rate can't be negative")
  514. }
  515. return nil
  516. }
  517. //-----------------------------------------------------------------------------
  518. // MempoolConfig
  519. // MempoolConfig defines the configuration options for the Tendermint mempool
  520. type MempoolConfig struct {
  521. RootDir string `mapstructure:"home"`
  522. Recheck bool `mapstructure:"recheck"`
  523. Broadcast bool `mapstructure:"broadcast"`
  524. WalPath string `mapstructure:"wal-dir"`
  525. // Maximum number of transactions in the mempool
  526. Size int `mapstructure:"size"`
  527. // Limit the total size of all txs in the mempool.
  528. // This only accounts for raw transactions (e.g. given 1MB transactions and
  529. // max-txs-bytes=5MB, mempool will only accept 5 transactions).
  530. MaxTxsBytes int64 `mapstructure:"max-txs-bytes"`
  531. // Size of the cache (used to filter transactions we saw earlier) in transactions
  532. CacheSize int `mapstructure:"cache-size"`
  533. // Maximum size of a single transaction
  534. // NOTE: the max size of a tx transmitted over the network is {max-tx-bytes}.
  535. MaxTxBytes int `mapstructure:"max-tx-bytes"`
  536. // Maximum size of a batch of transactions to send to a peer
  537. // Including space needed by encoding (one varint per transaction).
  538. MaxBatchBytes int `mapstructure:"max-batch-bytes"`
  539. }
  540. // DefaultMempoolConfig returns a default configuration for the Tendermint mempool
  541. func DefaultMempoolConfig() *MempoolConfig {
  542. return &MempoolConfig{
  543. Recheck: true,
  544. Broadcast: true,
  545. WalPath: "",
  546. // Each signature verification takes .5ms, Size reduced until we implement
  547. // ABCI Recheck
  548. Size: 5000,
  549. MaxTxsBytes: 1024 * 1024 * 1024, // 1GB
  550. CacheSize: 10000,
  551. MaxTxBytes: 1024 * 1024, // 1MB
  552. MaxBatchBytes: 10 * 1024 * 1024, // 10MB
  553. }
  554. }
  555. // TestMempoolConfig returns a configuration for testing the Tendermint mempool
  556. func TestMempoolConfig() *MempoolConfig {
  557. cfg := DefaultMempoolConfig()
  558. cfg.CacheSize = 1000
  559. return cfg
  560. }
  561. // WalDir returns the full path to the mempool's write-ahead log
  562. func (cfg *MempoolConfig) WalDir() string {
  563. return rootify(cfg.WalPath, cfg.RootDir)
  564. }
  565. // WalEnabled returns true if the WAL is enabled.
  566. func (cfg *MempoolConfig) WalEnabled() bool {
  567. return cfg.WalPath != ""
  568. }
  569. // ValidateBasic performs basic validation (checking param bounds, etc.) and
  570. // returns an error if any check fails.
  571. func (cfg *MempoolConfig) ValidateBasic() error {
  572. if cfg.Size < 0 {
  573. return errors.New("size can't be negative")
  574. }
  575. if cfg.MaxTxsBytes < 0 {
  576. return errors.New("max-txs-bytes can't be negative")
  577. }
  578. if cfg.CacheSize < 0 {
  579. return errors.New("cache-size can't be negative")
  580. }
  581. if cfg.MaxTxBytes < 0 {
  582. return errors.New("max-tx-bytes can't be negative")
  583. }
  584. if cfg.MaxBatchBytes < 0 {
  585. return errors.New("max-batch-bytes can't be negative")
  586. }
  587. if cfg.MaxBatchBytes <= cfg.MaxTxBytes {
  588. return errors.New("max-batch-bytes can't be less or equal to max-tx-bytes")
  589. }
  590. return nil
  591. }
  592. //-----------------------------------------------------------------------------
  593. // StateSyncConfig
  594. // StateSyncConfig defines the configuration for the Tendermint state sync service
  595. type StateSyncConfig struct {
  596. Enable bool `mapstructure:"enable"`
  597. TempDir string `mapstructure:"temp-dir"`
  598. RPCServers []string `mapstructure:"rpc-servers"`
  599. TrustPeriod time.Duration `mapstructure:"trust-period"`
  600. TrustHeight int64 `mapstructure:"trust-height"`
  601. TrustHash string `mapstructure:"trust-hash"`
  602. DiscoveryTime time.Duration `mapstructure:"discovery-time"`
  603. }
  604. func (cfg *StateSyncConfig) TrustHashBytes() []byte {
  605. // validated in ValidateBasic, so we can safely panic here
  606. bytes, err := hex.DecodeString(cfg.TrustHash)
  607. if err != nil {
  608. panic(err)
  609. }
  610. return bytes
  611. }
  612. // DefaultStateSyncConfig returns a default configuration for the state sync service
  613. func DefaultStateSyncConfig() *StateSyncConfig {
  614. return &StateSyncConfig{
  615. TrustPeriod: 168 * time.Hour,
  616. DiscoveryTime: 15 * time.Second,
  617. }
  618. }
  619. // TestFastSyncConfig returns a default configuration for the state sync service
  620. func TestStateSyncConfig() *StateSyncConfig {
  621. return DefaultStateSyncConfig()
  622. }
  623. // ValidateBasic performs basic validation.
  624. func (cfg *StateSyncConfig) ValidateBasic() error {
  625. if cfg.Enable {
  626. if len(cfg.RPCServers) == 0 {
  627. return errors.New("rpc-servers is required")
  628. }
  629. if len(cfg.RPCServers) < 2 {
  630. return errors.New("at least two rpc-servers entries is required")
  631. }
  632. for _, server := range cfg.RPCServers {
  633. if len(server) == 0 {
  634. return errors.New("found empty rpc-servers entry")
  635. }
  636. }
  637. if cfg.TrustPeriod <= 0 {
  638. return errors.New("trusted-period is required")
  639. }
  640. if cfg.TrustHeight <= 0 {
  641. return errors.New("trusted-height is required")
  642. }
  643. if len(cfg.TrustHash) == 0 {
  644. return errors.New("trusted-hash is required")
  645. }
  646. _, err := hex.DecodeString(cfg.TrustHash)
  647. if err != nil {
  648. return fmt.Errorf("invalid trusted-hash: %w", err)
  649. }
  650. }
  651. return nil
  652. }
  653. //-----------------------------------------------------------------------------
  654. // FastSyncConfig
  655. // FastSyncConfig defines the configuration for the Tendermint fast sync service
  656. type FastSyncConfig struct {
  657. Version string `mapstructure:"version"`
  658. }
  659. // DefaultFastSyncConfig returns a default configuration for the fast sync service
  660. func DefaultFastSyncConfig() *FastSyncConfig {
  661. return &FastSyncConfig{
  662. Version: "v0",
  663. }
  664. }
  665. // TestFastSyncConfig returns a default configuration for the fast sync.
  666. func TestFastSyncConfig() *FastSyncConfig {
  667. return DefaultFastSyncConfig()
  668. }
  669. // ValidateBasic performs basic validation.
  670. func (cfg *FastSyncConfig) ValidateBasic() error {
  671. switch cfg.Version {
  672. case "v0":
  673. return nil
  674. case "v2":
  675. return nil
  676. default:
  677. return fmt.Errorf("unknown fastsync version %s", cfg.Version)
  678. }
  679. }
  680. //-----------------------------------------------------------------------------
  681. // ConsensusConfig
  682. // ConsensusConfig defines the configuration for the Tendermint consensus service,
  683. // including timeouts and details about the WAL and the block structure.
  684. type ConsensusConfig struct {
  685. RootDir string `mapstructure:"home"`
  686. WalPath string `mapstructure:"wal-file"`
  687. walFile string // overrides WalPath if set
  688. // How long we wait for a proposal block before prevoting nil
  689. TimeoutPropose time.Duration `mapstructure:"timeout-propose"`
  690. // How much timeout-propose increases with each round
  691. TimeoutProposeDelta time.Duration `mapstructure:"timeout-propose-delta"`
  692. // How long we wait after receiving +2/3 prevotes for “anything” (ie. not a single block or nil)
  693. TimeoutPrevote time.Duration `mapstructure:"timeout-prevote"`
  694. // How much the timeout-prevote increases with each round
  695. TimeoutPrevoteDelta time.Duration `mapstructure:"timeout-prevote-delta"`
  696. // How long we wait after receiving +2/3 precommits for “anything” (ie. not a single block or nil)
  697. TimeoutPrecommit time.Duration `mapstructure:"timeout-precommit"`
  698. // How much the timeout-precommit increases with each round
  699. TimeoutPrecommitDelta time.Duration `mapstructure:"timeout-precommit-delta"`
  700. // How long we wait after committing a block, before starting on the new
  701. // height (this gives us a chance to receive some more precommits, even
  702. // though we already have +2/3).
  703. // NOTE: when modifying, make sure to update time-iota-ms genesis parameter
  704. TimeoutCommit time.Duration `mapstructure:"timeout-commit"`
  705. // Make progress as soon as we have all the precommits (as if TimeoutCommit = 0)
  706. SkipTimeoutCommit bool `mapstructure:"skip-timeout-commit"`
  707. // EmptyBlocks mode and possible interval between empty blocks
  708. CreateEmptyBlocks bool `mapstructure:"create-empty-blocks"`
  709. CreateEmptyBlocksInterval time.Duration `mapstructure:"create-empty-blocks-interval"`
  710. // Reactor sleep duration parameters
  711. PeerGossipSleepDuration time.Duration `mapstructure:"peer-gossip-sleep-duration"`
  712. PeerQueryMaj23SleepDuration time.Duration `mapstructure:"peer-query-maj23-sleep-duration"`
  713. DoubleSignCheckHeight int64 `mapstructure:"double-sign-check-height"`
  714. }
  715. // DefaultConsensusConfig returns a default configuration for the consensus service
  716. func DefaultConsensusConfig() *ConsensusConfig {
  717. return &ConsensusConfig{
  718. WalPath: filepath.Join(defaultDataDir, "cs.wal", "wal"),
  719. TimeoutPropose: 3000 * time.Millisecond,
  720. TimeoutProposeDelta: 500 * time.Millisecond,
  721. TimeoutPrevote: 1000 * time.Millisecond,
  722. TimeoutPrevoteDelta: 500 * time.Millisecond,
  723. TimeoutPrecommit: 1000 * time.Millisecond,
  724. TimeoutPrecommitDelta: 500 * time.Millisecond,
  725. TimeoutCommit: 1000 * time.Millisecond,
  726. SkipTimeoutCommit: false,
  727. CreateEmptyBlocks: true,
  728. CreateEmptyBlocksInterval: 0 * time.Second,
  729. PeerGossipSleepDuration: 100 * time.Millisecond,
  730. PeerQueryMaj23SleepDuration: 2000 * time.Millisecond,
  731. DoubleSignCheckHeight: int64(0),
  732. }
  733. }
  734. // TestConsensusConfig returns a configuration for testing the consensus service
  735. func TestConsensusConfig() *ConsensusConfig {
  736. cfg := DefaultConsensusConfig()
  737. cfg.TimeoutPropose = 40 * time.Millisecond
  738. cfg.TimeoutProposeDelta = 1 * time.Millisecond
  739. cfg.TimeoutPrevote = 10 * time.Millisecond
  740. cfg.TimeoutPrevoteDelta = 1 * time.Millisecond
  741. cfg.TimeoutPrecommit = 10 * time.Millisecond
  742. cfg.TimeoutPrecommitDelta = 1 * time.Millisecond
  743. // NOTE: when modifying, make sure to update time_iota_ms (testGenesisFmt) in toml.go
  744. cfg.TimeoutCommit = 10 * time.Millisecond
  745. cfg.SkipTimeoutCommit = true
  746. cfg.PeerGossipSleepDuration = 5 * time.Millisecond
  747. cfg.PeerQueryMaj23SleepDuration = 250 * time.Millisecond
  748. cfg.DoubleSignCheckHeight = int64(0)
  749. return cfg
  750. }
  751. // WaitForTxs returns true if the consensus should wait for transactions before entering the propose step
  752. func (cfg *ConsensusConfig) WaitForTxs() bool {
  753. return !cfg.CreateEmptyBlocks || cfg.CreateEmptyBlocksInterval > 0
  754. }
  755. // Propose returns the amount of time to wait for a proposal
  756. func (cfg *ConsensusConfig) Propose(round int32) time.Duration {
  757. return time.Duration(
  758. cfg.TimeoutPropose.Nanoseconds()+cfg.TimeoutProposeDelta.Nanoseconds()*int64(round),
  759. ) * time.Nanosecond
  760. }
  761. // Prevote returns the amount of time to wait for straggler votes after receiving any +2/3 prevotes
  762. func (cfg *ConsensusConfig) Prevote(round int32) time.Duration {
  763. return time.Duration(
  764. cfg.TimeoutPrevote.Nanoseconds()+cfg.TimeoutPrevoteDelta.Nanoseconds()*int64(round),
  765. ) * time.Nanosecond
  766. }
  767. // Precommit returns the amount of time to wait for straggler votes after receiving any +2/3 precommits
  768. func (cfg *ConsensusConfig) Precommit(round int32) time.Duration {
  769. return time.Duration(
  770. cfg.TimeoutPrecommit.Nanoseconds()+cfg.TimeoutPrecommitDelta.Nanoseconds()*int64(round),
  771. ) * time.Nanosecond
  772. }
  773. // Commit returns the amount of time to wait for straggler votes after receiving +2/3 precommits
  774. // for a single block (ie. a commit).
  775. func (cfg *ConsensusConfig) Commit(t time.Time) time.Time {
  776. return t.Add(cfg.TimeoutCommit)
  777. }
  778. // WalFile returns the full path to the write-ahead log file
  779. func (cfg *ConsensusConfig) WalFile() string {
  780. if cfg.walFile != "" {
  781. return cfg.walFile
  782. }
  783. return rootify(cfg.WalPath, cfg.RootDir)
  784. }
  785. // SetWalFile sets the path to the write-ahead log file
  786. func (cfg *ConsensusConfig) SetWalFile(walFile string) {
  787. cfg.walFile = walFile
  788. }
  789. // ValidateBasic performs basic validation (checking param bounds, etc.) and
  790. // returns an error if any check fails.
  791. func (cfg *ConsensusConfig) ValidateBasic() error {
  792. if cfg.TimeoutPropose < 0 {
  793. return errors.New("timeout-propose can't be negative")
  794. }
  795. if cfg.TimeoutProposeDelta < 0 {
  796. return errors.New("timeout-propose-delta can't be negative")
  797. }
  798. if cfg.TimeoutPrevote < 0 {
  799. return errors.New("timeout-prevote can't be negative")
  800. }
  801. if cfg.TimeoutPrevoteDelta < 0 {
  802. return errors.New("timeout-prevote-delta can't be negative")
  803. }
  804. if cfg.TimeoutPrecommit < 0 {
  805. return errors.New("timeout-precommit can't be negative")
  806. }
  807. if cfg.TimeoutPrecommitDelta < 0 {
  808. return errors.New("timeout-precommit-delta can't be negative")
  809. }
  810. if cfg.TimeoutCommit < 0 {
  811. return errors.New("timeout-commit can't be negative")
  812. }
  813. if cfg.CreateEmptyBlocksInterval < 0 {
  814. return errors.New("create-empty-blocks-interval can't be negative")
  815. }
  816. if cfg.PeerGossipSleepDuration < 0 {
  817. return errors.New("peer-gossip-sleep-duration can't be negative")
  818. }
  819. if cfg.PeerQueryMaj23SleepDuration < 0 {
  820. return errors.New("peer-query-maj23-sleep-duration can't be negative")
  821. }
  822. if cfg.DoubleSignCheckHeight < 0 {
  823. return errors.New("double-sign-check-height can't be negative")
  824. }
  825. return nil
  826. }
  827. //-----------------------------------------------------------------------------
  828. // TxIndexConfig
  829. // Remember that Event has the following structure:
  830. // type: [
  831. // key: value,
  832. // ...
  833. // ]
  834. //
  835. // CompositeKeys are constructed by `type.key`
  836. // TxIndexConfig defines the configuration for the transaction indexer,
  837. // including composite keys to index.
  838. type TxIndexConfig struct {
  839. // What indexer to use for transactions
  840. //
  841. // Options:
  842. // 1) "null"
  843. // 2) "kv" (default) - the simplest possible indexer,
  844. // backed by key-value storage (defaults to levelDB; see DBBackend).
  845. Indexer string `mapstructure:"indexer"`
  846. }
  847. // DefaultTxIndexConfig returns a default configuration for the transaction indexer.
  848. func DefaultTxIndexConfig() *TxIndexConfig {
  849. return &TxIndexConfig{
  850. Indexer: "kv",
  851. }
  852. }
  853. // TestTxIndexConfig returns a default configuration for the transaction indexer.
  854. func TestTxIndexConfig() *TxIndexConfig {
  855. return DefaultTxIndexConfig()
  856. }
  857. //-----------------------------------------------------------------------------
  858. // InstrumentationConfig
  859. // InstrumentationConfig defines the configuration for metrics reporting.
  860. type InstrumentationConfig struct {
  861. // When true, Prometheus metrics are served under /metrics on
  862. // PrometheusListenAddr.
  863. // Check out the documentation for the list of available metrics.
  864. Prometheus bool `mapstructure:"prometheus"`
  865. // Address to listen for Prometheus collector(s) connections.
  866. PrometheusListenAddr string `mapstructure:"prometheus-listen-addr"`
  867. // Maximum number of simultaneous connections.
  868. // If you want to accept a larger number than the default, make sure
  869. // you increase your OS limits.
  870. // 0 - unlimited.
  871. MaxOpenConnections int `mapstructure:"max-open-connections"`
  872. // Instrumentation namespace.
  873. Namespace string `mapstructure:"namespace"`
  874. }
  875. // DefaultInstrumentationConfig returns a default configuration for metrics
  876. // reporting.
  877. func DefaultInstrumentationConfig() *InstrumentationConfig {
  878. return &InstrumentationConfig{
  879. Prometheus: false,
  880. PrometheusListenAddr: ":26660",
  881. MaxOpenConnections: 3,
  882. Namespace: "tendermint",
  883. }
  884. }
  885. // TestInstrumentationConfig returns a default configuration for metrics
  886. // reporting.
  887. func TestInstrumentationConfig() *InstrumentationConfig {
  888. return DefaultInstrumentationConfig()
  889. }
  890. // ValidateBasic performs basic validation (checking param bounds, etc.) and
  891. // returns an error if any check fails.
  892. func (cfg *InstrumentationConfig) ValidateBasic() error {
  893. if cfg.MaxOpenConnections < 0 {
  894. return errors.New("max-open-connections can't be negative")
  895. }
  896. return nil
  897. }
  898. //-----------------------------------------------------------------------------
  899. // Utils
  900. // helper function to make config creation independent of root dir
  901. func rootify(path, root string) string {
  902. if filepath.IsAbs(path) {
  903. return path
  904. }
  905. return filepath.Join(root, path)
  906. }
  907. //-----------------------------------------------------------------------------
  908. // Moniker
  909. var defaultMoniker = getDefaultMoniker()
  910. // getDefaultMoniker returns a default moniker, which is the host name. If runtime
  911. // fails to get the host name, "anonymous" will be returned.
  912. func getDefaultMoniker() string {
  913. moniker, err := os.Hostname()
  914. if err != nil {
  915. moniker = "anonymous"
  916. }
  917. return moniker
  918. }