You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

1372 lines
48 KiB

deps: update gogo/protobuf version from v1.2.1 to v1.3.0 (#3947) * update gogo/protobuf version from v1.2.1 to v1.3.0 Also errcheck from v1.1.0 to v1.2.0 See full diff in - https://github.com/gogo/protobuf/compare/v1.2.1...v1.3.0 - https://github.com/kisielk/errcheck/compare/v1.1.0...v1.2.0 Changelog: Tested versions: go 1.12.9 protoc 3.7.1 Improvements: plugin/stringer - Handle repeated and/or nullable types a bit better now. plugin/size - Remove the loop in sovXXX by using bit twiddling. Thanks: https://github.com/apelisse plugin/marshalto - Implemented a reverse marshal strategy which allows for faster marshalling. This now avoids a recursive (and repeated) call to Size(). Thanks: https://github.com/apelisse plugin/compare - Added support for for oneof types. Bug fixes: protoc-gen-gogo/generator - Fix assignment to entry in nil map. Thanks: https://github.com/tgulacsi protoc-gen-gogo/generator - Allows plugins to call RecordTypeUse without panicking. Thanks: https://github.com/fedenusy proto/extensions - Fixed set extension regression. We did not clear the extensions before setting. io/uint32 - fix uint32reader bug that causes ReadMsg to recreate buffer when lengths are the same. Thanks: https://github.com/SebiSujar proto/table_merge: Fix merge of non-nullable slices. Thanks: https://github.com/euroelessar Upstream commits: merged in golang/protobuf commit 318d17de72747ed1c16502681db4b2bb709a92d0 - Add UnimplementedServer for server interface merged in golang/protobuf commit b85cd75de734650db18a99a943fe351d41387800 - protoc-gen-go/grpc: inline errUnimplemented function merged in golang/protobuf commit d3c38a4eb4970272b87a425ae00ccc4548e2f9bb - protoc-gen-go/grpc: use status and code packages only if needed merged in golang/protobuf commit e91709a02e0e8ff8b86b7aa913fdc9ae9498e825 - fix indentation in jsonpb with Any messages merged in golang/protobuf commit 8d0c54c1246661d9a51ca0ba455d22116d485eaa - protoc-gen-go: generate XXX_OneofWrappers instead of XXX_OneofFuncs Misc: extensions.md - Markdown update. Thanks: https://github.com/TennyZhuang Readme.md - Added user. go/protoc update - Updated to go1.12.x and protoc 3.7.1 Makefile update - fix go vet shadow tool reference test/mixbench - Update mixbench tool. Expose runnable benchmarks via flags. * update certstrap * update golangci-lint from v1.13.2 to v1.17.1 * update gox as well * test * comment out golangci temporary * comment out go-deadlock deps
5 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
5 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
deps: update gogo/protobuf version from v1.2.1 to v1.3.0 (#3947) * update gogo/protobuf version from v1.2.1 to v1.3.0 Also errcheck from v1.1.0 to v1.2.0 See full diff in - https://github.com/gogo/protobuf/compare/v1.2.1...v1.3.0 - https://github.com/kisielk/errcheck/compare/v1.1.0...v1.2.0 Changelog: Tested versions: go 1.12.9 protoc 3.7.1 Improvements: plugin/stringer - Handle repeated and/or nullable types a bit better now. plugin/size - Remove the loop in sovXXX by using bit twiddling. Thanks: https://github.com/apelisse plugin/marshalto - Implemented a reverse marshal strategy which allows for faster marshalling. This now avoids a recursive (and repeated) call to Size(). Thanks: https://github.com/apelisse plugin/compare - Added support for for oneof types. Bug fixes: protoc-gen-gogo/generator - Fix assignment to entry in nil map. Thanks: https://github.com/tgulacsi protoc-gen-gogo/generator - Allows plugins to call RecordTypeUse without panicking. Thanks: https://github.com/fedenusy proto/extensions - Fixed set extension regression. We did not clear the extensions before setting. io/uint32 - fix uint32reader bug that causes ReadMsg to recreate buffer when lengths are the same. Thanks: https://github.com/SebiSujar proto/table_merge: Fix merge of non-nullable slices. Thanks: https://github.com/euroelessar Upstream commits: merged in golang/protobuf commit 318d17de72747ed1c16502681db4b2bb709a92d0 - Add UnimplementedServer for server interface merged in golang/protobuf commit b85cd75de734650db18a99a943fe351d41387800 - protoc-gen-go/grpc: inline errUnimplemented function merged in golang/protobuf commit d3c38a4eb4970272b87a425ae00ccc4548e2f9bb - protoc-gen-go/grpc: use status and code packages only if needed merged in golang/protobuf commit e91709a02e0e8ff8b86b7aa913fdc9ae9498e825 - fix indentation in jsonpb with Any messages merged in golang/protobuf commit 8d0c54c1246661d9a51ca0ba455d22116d485eaa - protoc-gen-go: generate XXX_OneofWrappers instead of XXX_OneofFuncs Misc: extensions.md - Markdown update. Thanks: https://github.com/TennyZhuang Readme.md - Added user. go/protoc update - Updated to go1.12.x and protoc 3.7.1 Makefile update - fix go vet shadow tool reference test/mixbench - Update mixbench tool. Expose runnable benchmarks via flags. * update certstrap * update golangci-lint from v1.13.2 to v1.17.1 * update gox as well * test * comment out golangci temporary * comment out go-deadlock deps
5 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
limit number of /subscribe clients and queries per client (#3269) * limit number of /subscribe clients and queries per client Add the following config variables (under [rpc] section): * max_subscription_clients * max_subscriptions_per_client * timeout_broadcast_tx_commit Fixes #2826 new HTTPClient interface for subscriptions finalize HTTPClient events interface remove EventSubscriber fix data race ``` WARNING: DATA RACE Read at 0x00c000a36060 by goroutine 129: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0 Previous write at 0x00c000a36060 by goroutine 132: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 129 (running) created at: github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe() /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7 github.com/tendermint/tendermint/rpc/client.WaitForOneEvent() /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 Goroutine 132 (running) created at: testing.(*T).Run() /usr/local/go/src/testing/testing.go:878 +0x659 github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync() /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186 testing.tRunner() /usr/local/go/src/testing/testing.go:827 +0x162 ================== ``` lite client works (tested manually) godoc comments httpclient: do not close the out channel use TimeoutBroadcastTxCommit no timeout for unsubscribe but 1s Local (5s HTTP) timeout for resubscribe format code change Subscribe#out cap to 1 and replace config vars with RPCConfig TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout rpc: Context as first parameter to all functions reformat code fixes after my own review fixes after Ethan's review add test stubs fix config.toml * fixes after manual testing - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes Tendermint resources (pubsub) - rpc: better error in Subscribe and BroadcastTxCommit - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed * fixes after Ismail's review * Update rpc/grpc/grpc_test.go Co-Authored-By: melekes <anton.kalyaev@gmail.com>
6 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
5 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
5 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
5 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
5 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
5 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
5 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
5 years ago
blockchain: Reorg reactor (#3561) * go routines in blockchain reactor * Added reference to the go routine diagram * Initial commit * cleanup * Undo testing_logger change, committed by mistake * Fix the test loggers * pulled some fsm code into pool.go * added pool tests * changes to the design added block requests under peer moved the request trigger in the reactor poolRoutine, triggered now by a ticker in general moved everything required for making block requests smarter in the poolRoutine added a simple map of heights to keep track of what will need to be requested next added a few more tests * send errors to FSM in a different channel than blocks send errors (RemovePeer) from switch on a different channel than the one receiving blocks renamed channels added more pool tests * more pool tests * lint errors * more tests * more tests * switch fast sync to new implementation * fixed data race in tests * cleanup * finished fsm tests * address golangci comments :) * address golangci comments :) * Added timeout on next block needed to advance * updating docs and cleanup * fix issue in test from previous cleanup * cleanup * Added termination scenarios, tests and more cleanup * small fixes to adr, comments and cleanup * Fix bug in sendRequest() If we tried to send a request to a peer not present in the switch, a missing continue statement caused the request to be blackholed in a peer that was removed and never retried. While this bug was manifesting, the reactor kept asking for other blocks that would be stored and never consumed. Added the number of unconsumed blocks in the math for requesting blocks ahead of current processing height so eventually there will be no more blocks requested until the already received ones are consumed. * remove bpPeer's didTimeout field * Use distinct err codes for peer timeout and FSM timeouts * Don't allow peers to update with lower height * review comments from Ethan and Zarko * some cleanup, renaming, comments * Move block execution in separate goroutine * Remove pool's numPending * review comments * fix lint, remove old blockchain reactor and duplicates in fsm tests * small reorg around peer after review comments * add the reactor spec * verify block only once * review comments * change to int for max number of pending requests * cleanup and godoc * Add configuration flag fast sync version * golangci fixes * fix config template * move both reactor versions under blockchain * cleanup, golint, renaming stuff * updated documentation, fixed more golint warnings * integrate with behavior package * sync with master * gofmt * add changelog_pending entry * move to improvments * suggestion to changelog entry
5 years ago
  1. package config
  2. import (
  3. "encoding/hex"
  4. "errors"
  5. "fmt"
  6. "io/ioutil"
  7. "net/http"
  8. "os"
  9. "path/filepath"
  10. "time"
  11. tmjson "github.com/tendermint/tendermint/libs/json"
  12. "github.com/tendermint/tendermint/libs/log"
  13. tmos "github.com/tendermint/tendermint/libs/os"
  14. "github.com/tendermint/tendermint/types"
  15. )
  16. const (
  17. // FuzzModeDrop is a mode in which we randomly drop reads/writes, connections or sleep
  18. FuzzModeDrop = iota
  19. // FuzzModeDelay is a mode in which we randomly sleep
  20. FuzzModeDelay
  21. // DefaultLogLevel defines a default log level as INFO.
  22. DefaultLogLevel = "info"
  23. ModeFull = "full"
  24. ModeValidator = "validator"
  25. ModeSeed = "seed"
  26. BlockSyncV0 = "v0"
  27. BlockSyncV2 = "v2"
  28. MempoolV0 = "v0"
  29. MempoolV1 = "v1"
  30. )
  31. // NOTE: Most of the structs & relevant comments + the
  32. // default configuration options were used to manually
  33. // generate the config.toml. Please reflect any changes
  34. // made here in the defaultConfigTemplate constant in
  35. // config/toml.go
  36. // NOTE: libs/cli must know to look in the config dir!
  37. var (
  38. DefaultTendermintDir = ".tendermint"
  39. defaultConfigDir = "config"
  40. defaultDataDir = "data"
  41. defaultConfigFileName = "config.toml"
  42. defaultGenesisJSONName = "genesis.json"
  43. defaultMode = ModeFull
  44. defaultPrivValKeyName = "priv_validator_key.json"
  45. defaultPrivValStateName = "priv_validator_state.json"
  46. defaultNodeKeyName = "node_key.json"
  47. defaultAddrBookName = "addrbook.json"
  48. defaultConfigFilePath = filepath.Join(defaultConfigDir, defaultConfigFileName)
  49. defaultGenesisJSONPath = filepath.Join(defaultConfigDir, defaultGenesisJSONName)
  50. defaultPrivValKeyPath = filepath.Join(defaultConfigDir, defaultPrivValKeyName)
  51. defaultPrivValStatePath = filepath.Join(defaultDataDir, defaultPrivValStateName)
  52. defaultNodeKeyPath = filepath.Join(defaultConfigDir, defaultNodeKeyName)
  53. defaultAddrBookPath = filepath.Join(defaultConfigDir, defaultAddrBookName)
  54. minSubscriptionBufferSize = 100
  55. defaultSubscriptionBufferSize = 200
  56. )
  57. // Config defines the top level configuration for a Tendermint node
  58. type Config struct {
  59. // Top level options use an anonymous struct
  60. BaseConfig `mapstructure:",squash"`
  61. // Options for services
  62. RPC *RPCConfig `mapstructure:"rpc"`
  63. P2P *P2PConfig `mapstructure:"p2p"`
  64. Mempool *MempoolConfig `mapstructure:"mempool"`
  65. StateSync *StateSyncConfig `mapstructure:"statesync"`
  66. BlockSync *BlockSyncConfig `mapstructure:"blocksync"`
  67. Consensus *ConsensusConfig `mapstructure:"consensus"`
  68. TxIndex *TxIndexConfig `mapstructure:"tx-index"`
  69. Instrumentation *InstrumentationConfig `mapstructure:"instrumentation"`
  70. PrivValidator *PrivValidatorConfig `mapstructure:"priv-validator"`
  71. }
  72. // DefaultConfig returns a default configuration for a Tendermint node
  73. func DefaultConfig() *Config {
  74. return &Config{
  75. BaseConfig: DefaultBaseConfig(),
  76. RPC: DefaultRPCConfig(),
  77. P2P: DefaultP2PConfig(),
  78. Mempool: DefaultMempoolConfig(),
  79. StateSync: DefaultStateSyncConfig(),
  80. BlockSync: DefaultBlockSyncConfig(),
  81. Consensus: DefaultConsensusConfig(),
  82. TxIndex: DefaultTxIndexConfig(),
  83. Instrumentation: DefaultInstrumentationConfig(),
  84. PrivValidator: DefaultPrivValidatorConfig(),
  85. }
  86. }
  87. // DefaultValidatorConfig returns default config with mode as validator
  88. func DefaultValidatorConfig() *Config {
  89. cfg := DefaultConfig()
  90. cfg.Mode = ModeValidator
  91. return cfg
  92. }
  93. // TestConfig returns a configuration that can be used for testing
  94. func TestConfig() *Config {
  95. return &Config{
  96. BaseConfig: TestBaseConfig(),
  97. RPC: TestRPCConfig(),
  98. P2P: TestP2PConfig(),
  99. Mempool: TestMempoolConfig(),
  100. StateSync: TestStateSyncConfig(),
  101. BlockSync: TestBlockSyncConfig(),
  102. Consensus: TestConsensusConfig(),
  103. TxIndex: TestTxIndexConfig(),
  104. Instrumentation: TestInstrumentationConfig(),
  105. PrivValidator: DefaultPrivValidatorConfig(),
  106. }
  107. }
  108. // SetRoot sets the RootDir for all Config structs
  109. func (cfg *Config) SetRoot(root string) *Config {
  110. cfg.BaseConfig.RootDir = root
  111. cfg.RPC.RootDir = root
  112. cfg.P2P.RootDir = root
  113. cfg.Mempool.RootDir = root
  114. cfg.Consensus.RootDir = root
  115. cfg.PrivValidator.RootDir = root
  116. return cfg
  117. }
  118. // ValidateBasic performs basic validation (checking param bounds, etc.) and
  119. // returns an error if any check fails.
  120. func (cfg *Config) ValidateBasic() error {
  121. if err := cfg.BaseConfig.ValidateBasic(); err != nil {
  122. return err
  123. }
  124. if err := cfg.RPC.ValidateBasic(); err != nil {
  125. return fmt.Errorf("error in [rpc] section: %w", err)
  126. }
  127. if err := cfg.P2P.ValidateBasic(); err != nil {
  128. return fmt.Errorf("error in [p2p] section: %w", err)
  129. }
  130. if err := cfg.Mempool.ValidateBasic(); err != nil {
  131. return fmt.Errorf("error in [mempool] section: %w", err)
  132. }
  133. if err := cfg.StateSync.ValidateBasic(); err != nil {
  134. return fmt.Errorf("error in [statesync] section: %w", err)
  135. }
  136. if err := cfg.BlockSync.ValidateBasic(); err != nil {
  137. return fmt.Errorf("error in [blocksync] section: %w", err)
  138. }
  139. if err := cfg.Consensus.ValidateBasic(); err != nil {
  140. return fmt.Errorf("error in [consensus] section: %w", err)
  141. }
  142. if err := cfg.Instrumentation.ValidateBasic(); err != nil {
  143. return fmt.Errorf("error in [instrumentation] section: %w", err)
  144. }
  145. return nil
  146. }
  147. //-----------------------------------------------------------------------------
  148. // BaseConfig
  149. // BaseConfig defines the base configuration for a Tendermint node
  150. type BaseConfig struct { //nolint: maligned
  151. // chainID is unexposed and immutable but here for convenience
  152. chainID string
  153. // The root directory for all data.
  154. // This should be set in viper so it can unmarshal into this struct
  155. RootDir string `mapstructure:"home"`
  156. // TCP or UNIX socket address of the ABCI application,
  157. // or the name of an ABCI application compiled in with the Tendermint binary
  158. ProxyApp string `mapstructure:"proxy-app"`
  159. // A custom human readable name for this node
  160. Moniker string `mapstructure:"moniker"`
  161. // Mode of Node: full | validator | seed
  162. // * validator
  163. // - all reactors
  164. // - with priv_validator_key.json, priv_validator_state.json
  165. // * full
  166. // - all reactors
  167. // - No priv_validator_key.json, priv_validator_state.json
  168. // * seed
  169. // - only P2P, PEX Reactor
  170. // - No priv_validator_key.json, priv_validator_state.json
  171. Mode string `mapstructure:"mode"`
  172. // Database backend: goleveldb | cleveldb | boltdb | rocksdb
  173. // * goleveldb (github.com/syndtr/goleveldb - most popular implementation)
  174. // - pure go
  175. // - stable
  176. // * cleveldb (uses levigo wrapper)
  177. // - fast
  178. // - requires gcc
  179. // - use cleveldb build tag (go build -tags cleveldb)
  180. // * boltdb (uses etcd's fork of bolt - github.com/etcd-io/bbolt)
  181. // - EXPERIMENTAL
  182. // - may be faster is some use-cases (random reads - indexer)
  183. // - use boltdb build tag (go build -tags boltdb)
  184. // * rocksdb (uses github.com/tecbot/gorocksdb)
  185. // - EXPERIMENTAL
  186. // - requires gcc
  187. // - use rocksdb build tag (go build -tags rocksdb)
  188. // * badgerdb (uses github.com/dgraph-io/badger)
  189. // - EXPERIMENTAL
  190. // - use badgerdb build tag (go build -tags badgerdb)
  191. DBBackend string `mapstructure:"db-backend"`
  192. // Database directory
  193. DBPath string `mapstructure:"db-dir"`
  194. // Output level for logging
  195. LogLevel string `mapstructure:"log-level"`
  196. // Output format: 'plain' (colored text) or 'json'
  197. LogFormat string `mapstructure:"log-format"`
  198. // Path to the JSON file containing the initial validator set and other meta data
  199. Genesis string `mapstructure:"genesis-file"`
  200. // A JSON file containing the private key to use for p2p authenticated encryption
  201. NodeKey string `mapstructure:"node-key-file"`
  202. // Mechanism to connect to the ABCI application: socket | grpc
  203. ABCI string `mapstructure:"abci"`
  204. // If true, query the ABCI app on connecting to a new peer
  205. // so the app can decide if we should keep the connection or not
  206. FilterPeers bool `mapstructure:"filter-peers"` // false
  207. Other map[string]interface{} `mapstructure:",remain"`
  208. }
  209. // DefaultBaseConfig returns a default base configuration for a Tendermint node
  210. func DefaultBaseConfig() BaseConfig {
  211. return BaseConfig{
  212. Genesis: defaultGenesisJSONPath,
  213. NodeKey: defaultNodeKeyPath,
  214. Mode: defaultMode,
  215. Moniker: defaultMoniker,
  216. ProxyApp: "tcp://127.0.0.1:26658",
  217. ABCI: "socket",
  218. LogLevel: DefaultLogLevel,
  219. LogFormat: log.LogFormatPlain,
  220. FilterPeers: false,
  221. DBBackend: "goleveldb",
  222. DBPath: "data",
  223. }
  224. }
  225. // TestBaseConfig returns a base configuration for testing a Tendermint node
  226. func TestBaseConfig() BaseConfig {
  227. cfg := DefaultBaseConfig()
  228. cfg.chainID = "tendermint_test"
  229. cfg.Mode = ModeValidator
  230. cfg.ProxyApp = "kvstore"
  231. cfg.DBBackend = "memdb"
  232. return cfg
  233. }
  234. func (cfg BaseConfig) ChainID() string {
  235. return cfg.chainID
  236. }
  237. // GenesisFile returns the full path to the genesis.json file
  238. func (cfg BaseConfig) GenesisFile() string {
  239. return rootify(cfg.Genesis, cfg.RootDir)
  240. }
  241. // NodeKeyFile returns the full path to the node_key.json file
  242. func (cfg BaseConfig) NodeKeyFile() string {
  243. return rootify(cfg.NodeKey, cfg.RootDir)
  244. }
  245. // LoadNodeKey loads NodeKey located in filePath.
  246. func (cfg BaseConfig) LoadNodeKeyID() (types.NodeID, error) {
  247. jsonBytes, err := ioutil.ReadFile(cfg.NodeKeyFile())
  248. if err != nil {
  249. return "", err
  250. }
  251. nodeKey := types.NodeKey{}
  252. err = tmjson.Unmarshal(jsonBytes, &nodeKey)
  253. if err != nil {
  254. return "", err
  255. }
  256. nodeKey.ID = types.NodeIDFromPubKey(nodeKey.PubKey())
  257. return nodeKey.ID, nil
  258. }
  259. // LoadOrGenNodeKey attempts to load the NodeKey from the given filePath. If
  260. // the file does not exist, it generates and saves a new NodeKey.
  261. func (cfg BaseConfig) LoadOrGenNodeKeyID() (types.NodeID, error) {
  262. if tmos.FileExists(cfg.NodeKeyFile()) {
  263. nodeKey, err := cfg.LoadNodeKeyID()
  264. if err != nil {
  265. return "", err
  266. }
  267. return nodeKey, nil
  268. }
  269. nodeKey := types.GenNodeKey()
  270. if err := nodeKey.SaveAs(cfg.NodeKeyFile()); err != nil {
  271. return "", err
  272. }
  273. return nodeKey.ID, nil
  274. }
  275. // DBDir returns the full path to the database directory
  276. func (cfg BaseConfig) DBDir() string {
  277. return rootify(cfg.DBPath, cfg.RootDir)
  278. }
  279. // ValidateBasic performs basic validation (checking param bounds, etc.) and
  280. // returns an error if any check fails.
  281. func (cfg BaseConfig) ValidateBasic() error {
  282. switch cfg.LogFormat {
  283. case log.LogFormatJSON, log.LogFormatText, log.LogFormatPlain:
  284. default:
  285. return errors.New("unknown log format (must be 'plain', 'text' or 'json')")
  286. }
  287. switch cfg.Mode {
  288. case ModeFull, ModeValidator, ModeSeed:
  289. case "":
  290. return errors.New("no mode has been set")
  291. default:
  292. return fmt.Errorf("unknown mode: %v", cfg.Mode)
  293. }
  294. // TODO (https://github.com/tendermint/tendermint/issues/6908) remove this check after the v0.35 release cycle.
  295. // This check was added to give users an upgrade prompt to use the new
  296. // configuration option in v0.35. In future release cycles they should no longer
  297. // be using this configuration parameter so the check can be removed.
  298. // The cfg.Other field can likely be removed at the same time if it is not referenced
  299. // elsewhere as it was added to service this check.
  300. if fs, ok := cfg.Other["fastsync"]; ok {
  301. if _, ok := fs.(map[string]interface{}); ok {
  302. return fmt.Errorf("a configuration section named 'fastsync' was found in the " +
  303. "configuration file. The 'fastsync' section has been renamed to " +
  304. "'blocksync', please update the 'fastsync' field in your configuration file to 'blocksync'")
  305. }
  306. }
  307. if fs, ok := cfg.Other["fast-sync"]; ok {
  308. if fs != "" {
  309. return fmt.Errorf("a parameter named 'fast-sync' was found in the " +
  310. "configuration file. The parameter to enable or disable quickly syncing with a blockchain" +
  311. "has moved to the [blocksync] section of the configuration file as blocksync.enable. " +
  312. "Please move the 'fast-sync' field in your configuration file to 'blocksync.enable'")
  313. }
  314. }
  315. return nil
  316. }
  317. //-----------------------------------------------------------------------------
  318. // PrivValidatorConfig
  319. // PrivValidatorConfig defines the configuration parameters for running a validator
  320. type PrivValidatorConfig struct {
  321. RootDir string `mapstructure:"home"`
  322. // Path to the JSON file containing the private key to use as a validator in the consensus protocol
  323. Key string `mapstructure:"key-file"`
  324. // Path to the JSON file containing the last sign state of a validator
  325. State string `mapstructure:"state-file"`
  326. // TCP or UNIX socket address for Tendermint to listen on for
  327. // connections from an external PrivValidator process
  328. ListenAddr string `mapstructure:"laddr"`
  329. // Client certificate generated while creating needed files for secure connection.
  330. // If a remote validator address is provided but no certificate, the connection will be insecure
  331. ClientCertificate string `mapstructure:"client-certificate-file"`
  332. // Client key generated while creating certificates for secure connection
  333. ClientKey string `mapstructure:"client-key-file"`
  334. // Path Root Certificate Authority used to sign both client and server certificates
  335. RootCA string `mapstructure:"root-ca-file"`
  336. }
  337. // DefaultBaseConfig returns a default private validator configuration
  338. // for a Tendermint node.
  339. func DefaultPrivValidatorConfig() *PrivValidatorConfig {
  340. return &PrivValidatorConfig{
  341. Key: defaultPrivValKeyPath,
  342. State: defaultPrivValStatePath,
  343. }
  344. }
  345. // ClientKeyFile returns the full path to the priv_validator_key.json file
  346. func (cfg *PrivValidatorConfig) ClientKeyFile() string {
  347. return rootify(cfg.ClientKey, cfg.RootDir)
  348. }
  349. // ClientCertificateFile returns the full path to the priv_validator_key.json file
  350. func (cfg *PrivValidatorConfig) ClientCertificateFile() string {
  351. return rootify(cfg.ClientCertificate, cfg.RootDir)
  352. }
  353. // CertificateAuthorityFile returns the full path to the priv_validator_key.json file
  354. func (cfg *PrivValidatorConfig) RootCAFile() string {
  355. return rootify(cfg.RootCA, cfg.RootDir)
  356. }
  357. // KeyFile returns the full path to the priv_validator_key.json file
  358. func (cfg *PrivValidatorConfig) KeyFile() string {
  359. return rootify(cfg.Key, cfg.RootDir)
  360. }
  361. // StateFile returns the full path to the priv_validator_state.json file
  362. func (cfg *PrivValidatorConfig) StateFile() string {
  363. return rootify(cfg.State, cfg.RootDir)
  364. }
  365. func (cfg *PrivValidatorConfig) AreSecurityOptionsPresent() bool {
  366. switch {
  367. case cfg.RootCA == "":
  368. return false
  369. case cfg.ClientKey == "":
  370. return false
  371. case cfg.ClientCertificate == "":
  372. return false
  373. default:
  374. return true
  375. }
  376. }
  377. //-----------------------------------------------------------------------------
  378. // RPCConfig
  379. // RPCConfig defines the configuration options for the Tendermint RPC server
  380. type RPCConfig struct {
  381. RootDir string `mapstructure:"home"`
  382. // TCP or UNIX socket address for the RPC server to listen on
  383. ListenAddress string `mapstructure:"laddr"`
  384. // A list of origins a cross-domain request can be executed from.
  385. // If the special '*' value is present in the list, all origins will be allowed.
  386. // An origin may contain a wildcard (*) to replace 0 or more characters (i.e.: http://*.domain.com).
  387. // Only one wildcard can be used per origin.
  388. CORSAllowedOrigins []string `mapstructure:"cors-allowed-origins"`
  389. // A list of methods the client is allowed to use with cross-domain requests.
  390. CORSAllowedMethods []string `mapstructure:"cors-allowed-methods"`
  391. // A list of non simple headers the client is allowed to use with cross-domain requests.
  392. CORSAllowedHeaders []string `mapstructure:"cors-allowed-headers"`
  393. // TCP or UNIX socket address for the gRPC server to listen on
  394. // NOTE: This server only supports /broadcast_tx_commit
  395. // Deprecated: gRPC in the RPC layer of Tendermint will be removed in 0.36.
  396. GRPCListenAddress string `mapstructure:"grpc-laddr"`
  397. // Maximum number of simultaneous connections.
  398. // Does not include RPC (HTTP&WebSocket) connections. See max-open-connections
  399. // If you want to accept a larger number than the default, make sure
  400. // you increase your OS limits.
  401. // 0 - unlimited.
  402. // Deprecated: gRPC in the RPC layer of Tendermint will be removed in 0.36.
  403. GRPCMaxOpenConnections int `mapstructure:"grpc-max-open-connections"`
  404. // Activate unsafe RPC commands like /dial-persistent-peers and /unsafe-flush-mempool
  405. Unsafe bool `mapstructure:"unsafe"`
  406. // Maximum number of simultaneous connections (including WebSocket).
  407. // Does not include gRPC connections. See grpc-max-open-connections
  408. // If you want to accept a larger number than the default, make sure
  409. // you increase your OS limits.
  410. // 0 - unlimited.
  411. // Should be < {ulimit -Sn} - {MaxNumInboundPeers} - {MaxNumOutboundPeers} - {N of wal, db and other open files}
  412. // 1024 - 40 - 10 - 50 = 924 = ~900
  413. MaxOpenConnections int `mapstructure:"max-open-connections"`
  414. // Maximum number of unique clientIDs that can /subscribe
  415. // If you're using /broadcast_tx_commit, set to the estimated maximum number
  416. // of broadcast_tx_commit calls per block.
  417. MaxSubscriptionClients int `mapstructure:"max-subscription-clients"`
  418. // Maximum number of unique queries a given client can /subscribe to
  419. // If you're using GRPC (or Local RPC client) and /broadcast_tx_commit, set
  420. // to the estimated maximum number of broadcast_tx_commit calls per block.
  421. MaxSubscriptionsPerClient int `mapstructure:"max-subscriptions-per-client"`
  422. // The number of events that can be buffered per subscription before
  423. // returning `ErrOutOfCapacity`.
  424. SubscriptionBufferSize int `mapstructure:"experimental-subscription-buffer-size"`
  425. // The maximum number of responses that can be buffered per WebSocket
  426. // client. If clients cannot read from the WebSocket endpoint fast enough,
  427. // they will be disconnected, so increasing this parameter may reduce the
  428. // chances of them being disconnected (but will cause the node to use more
  429. // memory).
  430. //
  431. // Must be at least the same as `SubscriptionBufferSize`, otherwise
  432. // connections may be dropped unnecessarily.
  433. WebSocketWriteBufferSize int `mapstructure:"experimental-websocket-write-buffer-size"`
  434. // If a WebSocket client cannot read fast enough, at present we may
  435. // silently drop events instead of generating an error or disconnecting the
  436. // client.
  437. //
  438. // Enabling this parameter will cause the WebSocket connection to be closed
  439. // instead if it cannot read fast enough, allowing for greater
  440. // predictability in subscription behavior.
  441. CloseOnSlowClient bool `mapstructure:"experimental-close-on-slow-client"`
  442. // How long to wait for a tx to be committed during /broadcast_tx_commit
  443. // WARNING: Using a value larger than 10s will result in increasing the
  444. // global HTTP write timeout, which applies to all connections and endpoints.
  445. // See https://github.com/tendermint/tendermint/issues/3435
  446. TimeoutBroadcastTxCommit time.Duration `mapstructure:"timeout-broadcast-tx-commit"`
  447. // Maximum size of request body, in bytes
  448. MaxBodyBytes int64 `mapstructure:"max-body-bytes"`
  449. // Maximum size of request header, in bytes
  450. MaxHeaderBytes int `mapstructure:"max-header-bytes"`
  451. // The path to a file containing certificate that is used to create the HTTPS server.
  452. // Might be either absolute path or path related to Tendermint's config directory.
  453. //
  454. // If the certificate is signed by a certificate authority,
  455. // the certFile should be the concatenation of the server's certificate, any intermediates,
  456. // and the CA's certificate.
  457. //
  458. // NOTE: both tls-cert-file and tls-key-file must be present for Tendermint to create HTTPS server.
  459. // Otherwise, HTTP server is run.
  460. TLSCertFile string `mapstructure:"tls-cert-file"`
  461. // The path to a file containing matching private key that is used to create the HTTPS server.
  462. // Might be either absolute path or path related to tendermint's config directory.
  463. //
  464. // NOTE: both tls-cert-file and tls-key-file must be present for Tendermint to create HTTPS server.
  465. // Otherwise, HTTP server is run.
  466. TLSKeyFile string `mapstructure:"tls-key-file"`
  467. // pprof listen address (https://golang.org/pkg/net/http/pprof)
  468. PprofListenAddress string `mapstructure:"pprof-laddr"`
  469. }
  470. // DefaultRPCConfig returns a default configuration for the RPC server
  471. func DefaultRPCConfig() *RPCConfig {
  472. return &RPCConfig{
  473. ListenAddress: "tcp://127.0.0.1:26657",
  474. CORSAllowedOrigins: []string{},
  475. CORSAllowedMethods: []string{http.MethodHead, http.MethodGet, http.MethodPost},
  476. CORSAllowedHeaders: []string{"Origin", "Accept", "Content-Type", "X-Requested-With", "X-Server-Time"},
  477. GRPCListenAddress: "",
  478. GRPCMaxOpenConnections: 900,
  479. Unsafe: false,
  480. MaxOpenConnections: 900,
  481. MaxSubscriptionClients: 100,
  482. MaxSubscriptionsPerClient: 5,
  483. SubscriptionBufferSize: defaultSubscriptionBufferSize,
  484. TimeoutBroadcastTxCommit: 10 * time.Second,
  485. WebSocketWriteBufferSize: defaultSubscriptionBufferSize,
  486. MaxBodyBytes: int64(1000000), // 1MB
  487. MaxHeaderBytes: 1 << 20, // same as the net/http default
  488. TLSCertFile: "",
  489. TLSKeyFile: "",
  490. }
  491. }
  492. // TestRPCConfig returns a configuration for testing the RPC server
  493. func TestRPCConfig() *RPCConfig {
  494. cfg := DefaultRPCConfig()
  495. cfg.ListenAddress = "tcp://127.0.0.1:36657"
  496. cfg.GRPCListenAddress = "tcp://127.0.0.1:36658"
  497. cfg.Unsafe = true
  498. return cfg
  499. }
  500. // ValidateBasic performs basic validation (checking param bounds, etc.) and
  501. // returns an error if any check fails.
  502. func (cfg *RPCConfig) ValidateBasic() error {
  503. if cfg.GRPCMaxOpenConnections < 0 {
  504. return errors.New("grpc-max-open-connections can't be negative")
  505. }
  506. if cfg.MaxOpenConnections < 0 {
  507. return errors.New("max-open-connections can't be negative")
  508. }
  509. if cfg.MaxSubscriptionClients < 0 {
  510. return errors.New("max-subscription-clients can't be negative")
  511. }
  512. if cfg.MaxSubscriptionsPerClient < 0 {
  513. return errors.New("max-subscriptions-per-client can't be negative")
  514. }
  515. if cfg.SubscriptionBufferSize < minSubscriptionBufferSize {
  516. return fmt.Errorf(
  517. "experimental-subscription-buffer-size must be >= %d",
  518. minSubscriptionBufferSize,
  519. )
  520. }
  521. if cfg.WebSocketWriteBufferSize < cfg.SubscriptionBufferSize {
  522. return fmt.Errorf(
  523. "experimental-websocket-write-buffer-size must be >= experimental-subscription-buffer-size (%d)",
  524. cfg.SubscriptionBufferSize,
  525. )
  526. }
  527. if cfg.TimeoutBroadcastTxCommit < 0 {
  528. return errors.New("timeout-broadcast-tx-commit can't be negative")
  529. }
  530. if cfg.MaxBodyBytes < 0 {
  531. return errors.New("max-body-bytes can't be negative")
  532. }
  533. if cfg.MaxHeaderBytes < 0 {
  534. return errors.New("max-header-bytes can't be negative")
  535. }
  536. return nil
  537. }
  538. // IsCorsEnabled returns true if cross-origin resource sharing is enabled.
  539. func (cfg *RPCConfig) IsCorsEnabled() bool {
  540. return len(cfg.CORSAllowedOrigins) != 0
  541. }
  542. func (cfg RPCConfig) KeyFile() string {
  543. path := cfg.TLSKeyFile
  544. if filepath.IsAbs(path) {
  545. return path
  546. }
  547. return rootify(filepath.Join(defaultConfigDir, path), cfg.RootDir)
  548. }
  549. func (cfg RPCConfig) CertFile() string {
  550. path := cfg.TLSCertFile
  551. if filepath.IsAbs(path) {
  552. return path
  553. }
  554. return rootify(filepath.Join(defaultConfigDir, path), cfg.RootDir)
  555. }
  556. func (cfg RPCConfig) IsTLSEnabled() bool {
  557. return cfg.TLSCertFile != "" && cfg.TLSKeyFile != ""
  558. }
  559. //-----------------------------------------------------------------------------
  560. // P2PConfig
  561. // P2PConfig defines the configuration options for the Tendermint peer-to-peer networking layer
  562. type P2PConfig struct { //nolint: maligned
  563. RootDir string `mapstructure:"home"`
  564. // Address to listen for incoming connections
  565. ListenAddress string `mapstructure:"laddr"`
  566. // Address to advertise to peers for them to dial
  567. ExternalAddress string `mapstructure:"external-address"`
  568. // Comma separated list of seed nodes to connect to
  569. // We only use these if we can’t connect to peers in the addrbook
  570. // NOTE: not used by the new PEX reactor. Please use BootstrapPeers instead.
  571. // TODO: Remove once p2p refactor is complete
  572. // ref: https://github.com/tendermint/tendermint/issues/5670
  573. Seeds string `mapstructure:"seeds"`
  574. // Comma separated list of peers to be added to the peer store
  575. // on startup. Either BootstrapPeers or PersistentPeers are
  576. // needed for peer discovery
  577. BootstrapPeers string `mapstructure:"bootstrap-peers"`
  578. // Comma separated list of nodes to keep persistent connections to
  579. PersistentPeers string `mapstructure:"persistent-peers"`
  580. // UPNP port forwarding
  581. UPNP bool `mapstructure:"upnp"`
  582. // Path to address book
  583. AddrBook string `mapstructure:"addr-book-file"`
  584. // Set true for strict address routability rules
  585. // Set false for private or local networks
  586. AddrBookStrict bool `mapstructure:"addr-book-strict"`
  587. // Maximum number of inbound peers
  588. //
  589. // TODO: Remove once p2p refactor is complete in favor of MaxConnections.
  590. // ref: https://github.com/tendermint/tendermint/issues/5670
  591. MaxNumInboundPeers int `mapstructure:"max-num-inbound-peers"`
  592. // Maximum number of outbound peers to connect to, excluding persistent peers.
  593. //
  594. // TODO: Remove once p2p refactor is complete in favor of MaxConnections.
  595. // ref: https://github.com/tendermint/tendermint/issues/5670
  596. MaxNumOutboundPeers int `mapstructure:"max-num-outbound-peers"`
  597. // MaxConnections defines the maximum number of connected peers (inbound and
  598. // outbound).
  599. MaxConnections uint16 `mapstructure:"max-connections"`
  600. // MaxIncomingConnectionAttempts rate limits the number of incoming connection
  601. // attempts per IP address.
  602. MaxIncomingConnectionAttempts uint `mapstructure:"max-incoming-connection-attempts"`
  603. // List of node IDs, to which a connection will be (re)established ignoring any existing limits
  604. UnconditionalPeerIDs string `mapstructure:"unconditional-peer-ids"`
  605. // Maximum pause when redialing a persistent peer (if zero, exponential backoff is used)
  606. PersistentPeersMaxDialPeriod time.Duration `mapstructure:"persistent-peers-max-dial-period"`
  607. // Time to wait before flushing messages out on the connection
  608. FlushThrottleTimeout time.Duration `mapstructure:"flush-throttle-timeout"`
  609. // Maximum size of a message packet payload, in bytes
  610. MaxPacketMsgPayloadSize int `mapstructure:"max-packet-msg-payload-size"`
  611. // Rate at which packets can be sent, in bytes/second
  612. SendRate int64 `mapstructure:"send-rate"`
  613. // Rate at which packets can be received, in bytes/second
  614. RecvRate int64 `mapstructure:"recv-rate"`
  615. // Set true to enable the peer-exchange reactor
  616. PexReactor bool `mapstructure:"pex"`
  617. // Comma separated list of peer IDs to keep private (will not be gossiped to
  618. // other peers)
  619. PrivatePeerIDs string `mapstructure:"private-peer-ids"`
  620. // Toggle to disable guard against peers connecting from the same ip.
  621. AllowDuplicateIP bool `mapstructure:"allow-duplicate-ip"`
  622. // Peer connection configuration.
  623. HandshakeTimeout time.Duration `mapstructure:"handshake-timeout"`
  624. DialTimeout time.Duration `mapstructure:"dial-timeout"`
  625. // Testing params.
  626. // Force dial to fail
  627. TestDialFail bool `mapstructure:"test-dial-fail"`
  628. // UseLegacy enables the "legacy" P2P implementation and
  629. // disables the newer default implementation. This flag will
  630. // be removed in a future release.
  631. UseLegacy bool `mapstructure:"use-legacy"`
  632. // Makes it possible to configure which queue backend the p2p
  633. // layer uses. Options are: "fifo", "priority" and "wdrr",
  634. // with the default being "priority".
  635. QueueType string `mapstructure:"queue-type"`
  636. }
  637. // DefaultP2PConfig returns a default configuration for the peer-to-peer layer
  638. func DefaultP2PConfig() *P2PConfig {
  639. return &P2PConfig{
  640. ListenAddress: "tcp://0.0.0.0:26656",
  641. ExternalAddress: "",
  642. UPNP: false,
  643. AddrBook: defaultAddrBookPath,
  644. AddrBookStrict: true,
  645. MaxNumInboundPeers: 40,
  646. MaxNumOutboundPeers: 10,
  647. MaxConnections: 64,
  648. MaxIncomingConnectionAttempts: 100,
  649. PersistentPeersMaxDialPeriod: 0 * time.Second,
  650. FlushThrottleTimeout: 100 * time.Millisecond,
  651. // The MTU (Maximum Transmission Unit) for Ethernet is 1500 bytes.
  652. // The IP header and the TCP header take up 20 bytes each at least (unless
  653. // optional header fields are used) and thus the max for (non-Jumbo frame)
  654. // Ethernet is 1500 - 20 -20 = 1460
  655. // Source: https://stackoverflow.com/a/3074427/820520
  656. MaxPacketMsgPayloadSize: 1400,
  657. SendRate: 5120000, // 5 mB/s
  658. RecvRate: 5120000, // 5 mB/s
  659. PexReactor: true,
  660. AllowDuplicateIP: false,
  661. HandshakeTimeout: 20 * time.Second,
  662. DialTimeout: 3 * time.Second,
  663. TestDialFail: false,
  664. QueueType: "priority",
  665. UseLegacy: false,
  666. }
  667. }
  668. // TestP2PConfig returns a configuration for testing the peer-to-peer layer
  669. func TestP2PConfig() *P2PConfig {
  670. cfg := DefaultP2PConfig()
  671. cfg.ListenAddress = "tcp://127.0.0.1:36656"
  672. cfg.FlushThrottleTimeout = 10 * time.Millisecond
  673. cfg.AllowDuplicateIP = true
  674. return cfg
  675. }
  676. // AddrBookFile returns the full path to the address book
  677. func (cfg *P2PConfig) AddrBookFile() string {
  678. return rootify(cfg.AddrBook, cfg.RootDir)
  679. }
  680. // ValidateBasic performs basic validation (checking param bounds, etc.) and
  681. // returns an error if any check fails.
  682. func (cfg *P2PConfig) ValidateBasic() error {
  683. if cfg.MaxNumInboundPeers < 0 {
  684. return errors.New("max-num-inbound-peers can't be negative")
  685. }
  686. if cfg.MaxNumOutboundPeers < 0 {
  687. return errors.New("max-num-outbound-peers can't be negative")
  688. }
  689. if cfg.FlushThrottleTimeout < 0 {
  690. return errors.New("flush-throttle-timeout can't be negative")
  691. }
  692. if cfg.PersistentPeersMaxDialPeriod < 0 {
  693. return errors.New("persistent-peers-max-dial-period can't be negative")
  694. }
  695. if cfg.MaxPacketMsgPayloadSize < 0 {
  696. return errors.New("max-packet-msg-payload-size can't be negative")
  697. }
  698. if cfg.SendRate < 0 {
  699. return errors.New("send-rate can't be negative")
  700. }
  701. if cfg.RecvRate < 0 {
  702. return errors.New("recv-rate can't be negative")
  703. }
  704. return nil
  705. }
  706. //-----------------------------------------------------------------------------
  707. // MempoolConfig
  708. // MempoolConfig defines the configuration options for the Tendermint mempool.
  709. type MempoolConfig struct {
  710. Version string `mapstructure:"version"`
  711. RootDir string `mapstructure:"home"`
  712. Recheck bool `mapstructure:"recheck"`
  713. Broadcast bool `mapstructure:"broadcast"`
  714. // Maximum number of transactions in the mempool
  715. Size int `mapstructure:"size"`
  716. // Limit the total size of all txs in the mempool.
  717. // This only accounts for raw transactions (e.g. given 1MB transactions and
  718. // max-txs-bytes=5MB, mempool will only accept 5 transactions).
  719. MaxTxsBytes int64 `mapstructure:"max-txs-bytes"`
  720. // Size of the cache (used to filter transactions we saw earlier) in transactions
  721. CacheSize int `mapstructure:"cache-size"`
  722. // Do not remove invalid transactions from the cache (default: false)
  723. // Set to true if it's not possible for any invalid transaction to become
  724. // valid again in the future.
  725. KeepInvalidTxsInCache bool `mapstructure:"keep-invalid-txs-in-cache"`
  726. // Maximum size of a single transaction
  727. // NOTE: the max size of a tx transmitted over the network is {max-tx-bytes}.
  728. MaxTxBytes int `mapstructure:"max-tx-bytes"`
  729. // Maximum size of a batch of transactions to send to a peer
  730. // Including space needed by encoding (one varint per transaction).
  731. // XXX: Unused due to https://github.com/tendermint/tendermint/issues/5796
  732. MaxBatchBytes int `mapstructure:"max-batch-bytes"`
  733. // TTLDuration, if non-zero, defines the maximum amount of time a transaction
  734. // can exist for in the mempool.
  735. //
  736. // Note, if TTLNumBlocks is also defined, a transaction will be removed if it
  737. // has existed in the mempool at least TTLNumBlocks number of blocks or if it's
  738. // insertion time into the mempool is beyond TTLDuration.
  739. TTLDuration time.Duration `mapstructure:"ttl-duration"`
  740. // TTLNumBlocks, if non-zero, defines the maximum number of blocks a transaction
  741. // can exist for in the mempool.
  742. //
  743. // Note, if TTLDuration is also defined, a transaction will be removed if it
  744. // has existed in the mempool at least TTLNumBlocks number of blocks or if
  745. // it's insertion time into the mempool is beyond TTLDuration.
  746. TTLNumBlocks int64 `mapstructure:"ttl-num-blocks"`
  747. }
  748. // DefaultMempoolConfig returns a default configuration for the Tendermint mempool.
  749. func DefaultMempoolConfig() *MempoolConfig {
  750. return &MempoolConfig{
  751. Version: MempoolV1,
  752. Recheck: true,
  753. Broadcast: true,
  754. // Each signature verification takes .5ms, Size reduced until we implement
  755. // ABCI Recheck
  756. Size: 5000,
  757. MaxTxsBytes: 1024 * 1024 * 1024, // 1GB
  758. CacheSize: 10000,
  759. MaxTxBytes: 1024 * 1024, // 1MB
  760. TTLDuration: 0 * time.Second,
  761. TTLNumBlocks: 0,
  762. }
  763. }
  764. // TestMempoolConfig returns a configuration for testing the Tendermint mempool
  765. func TestMempoolConfig() *MempoolConfig {
  766. cfg := DefaultMempoolConfig()
  767. cfg.CacheSize = 1000
  768. return cfg
  769. }
  770. // ValidateBasic performs basic validation (checking param bounds, etc.) and
  771. // returns an error if any check fails.
  772. func (cfg *MempoolConfig) ValidateBasic() error {
  773. if cfg.Size < 0 {
  774. return errors.New("size can't be negative")
  775. }
  776. if cfg.MaxTxsBytes < 0 {
  777. return errors.New("max-txs-bytes can't be negative")
  778. }
  779. if cfg.CacheSize < 0 {
  780. return errors.New("cache-size can't be negative")
  781. }
  782. if cfg.MaxTxBytes < 0 {
  783. return errors.New("max-tx-bytes can't be negative")
  784. }
  785. if cfg.TTLDuration < 0 {
  786. return errors.New("ttl-duration can't be negative")
  787. }
  788. if cfg.TTLNumBlocks < 0 {
  789. return errors.New("ttl-num-blocks can't be negative")
  790. }
  791. return nil
  792. }
  793. //-----------------------------------------------------------------------------
  794. // StateSyncConfig
  795. // StateSyncConfig defines the configuration for the Tendermint state sync service
  796. type StateSyncConfig struct {
  797. // State sync rapidly bootstraps a new node by discovering, fetching, and restoring a
  798. // state machine snapshot from peers instead of fetching and replaying historical
  799. // blocks. Requires some peers in the network to take and serve state machine
  800. // snapshots. State sync is not attempted if the node has any local state
  801. // (LastBlockHeight > 0). The node will have a truncated block history, starting from
  802. // the height of the snapshot.
  803. Enable bool `mapstructure:"enable"`
  804. // State sync uses light client verification to verify state. This can be done either
  805. // through the P2P layer or the RPC layer. Set this to true to use the P2P layer. If
  806. // false (default), the RPC layer will be used.
  807. UseP2P bool `mapstructure:"use-p2p"`
  808. // If using RPC, at least two addresses need to be provided. They should be compatible
  809. // with net.Dial, for example: "host.example.com:2125".
  810. RPCServers []string `mapstructure:"rpc-servers"`
  811. // The hash and height of a trusted block. Must be within the trust-period.
  812. TrustHeight int64 `mapstructure:"trust-height"`
  813. TrustHash string `mapstructure:"trust-hash"`
  814. // The trust period should be set so that Tendermint can detect and gossip
  815. // misbehavior before it is considered expired. For chains based on the Cosmos SDK,
  816. // one day less than the unbonding period should suffice.
  817. TrustPeriod time.Duration `mapstructure:"trust-period"`
  818. // Time to spend discovering snapshots before initiating a restore.
  819. DiscoveryTime time.Duration `mapstructure:"discovery-time"`
  820. // Temporary directory for state sync snapshot chunks, defaults to os.TempDir().
  821. // The synchronizer will create a new, randomly named directory within this directory
  822. // and remove it when the sync is complete.
  823. TempDir string `mapstructure:"temp-dir"`
  824. // The timeout duration before re-requesting a chunk, possibly from a different
  825. // peer (default: 15 seconds).
  826. ChunkRequestTimeout time.Duration `mapstructure:"chunk-request-timeout"`
  827. // The number of concurrent chunk and block fetchers to run (default: 4).
  828. Fetchers int32 `mapstructure:"fetchers"`
  829. }
  830. func (cfg *StateSyncConfig) TrustHashBytes() []byte {
  831. // validated in ValidateBasic, so we can safely panic here
  832. bytes, err := hex.DecodeString(cfg.TrustHash)
  833. if err != nil {
  834. panic(err)
  835. }
  836. return bytes
  837. }
  838. // DefaultStateSyncConfig returns a default configuration for the state sync service
  839. func DefaultStateSyncConfig() *StateSyncConfig {
  840. return &StateSyncConfig{
  841. TrustPeriod: 168 * time.Hour,
  842. DiscoveryTime: 15 * time.Second,
  843. ChunkRequestTimeout: 15 * time.Second,
  844. Fetchers: 4,
  845. }
  846. }
  847. // TestStateSyncConfig returns a default configuration for the state sync service
  848. func TestStateSyncConfig() *StateSyncConfig {
  849. return DefaultStateSyncConfig()
  850. }
  851. // ValidateBasic performs basic validation.
  852. func (cfg *StateSyncConfig) ValidateBasic() error {
  853. if !cfg.Enable {
  854. return nil
  855. }
  856. // If we're not using the P2P stack then we need to validate the
  857. // RPCServers
  858. if !cfg.UseP2P {
  859. if len(cfg.RPCServers) < 2 {
  860. return errors.New("at least two rpc-servers must be specified")
  861. }
  862. for _, server := range cfg.RPCServers {
  863. if server == "" {
  864. return errors.New("found empty rpc-servers entry")
  865. }
  866. }
  867. }
  868. if cfg.DiscoveryTime != 0 && cfg.DiscoveryTime < 5*time.Second {
  869. return errors.New("discovery time must be 0s or greater than five seconds")
  870. }
  871. if cfg.TrustPeriod <= 0 {
  872. return errors.New("trusted-period is required")
  873. }
  874. if cfg.TrustHeight <= 0 {
  875. return errors.New("trusted-height is required")
  876. }
  877. if len(cfg.TrustHash) == 0 {
  878. return errors.New("trusted-hash is required")
  879. }
  880. _, err := hex.DecodeString(cfg.TrustHash)
  881. if err != nil {
  882. return fmt.Errorf("invalid trusted-hash: %w", err)
  883. }
  884. if cfg.ChunkRequestTimeout < 5*time.Second {
  885. return errors.New("chunk-request-timeout must be at least 5 seconds")
  886. }
  887. if cfg.Fetchers <= 0 {
  888. return errors.New("fetchers is required")
  889. }
  890. return nil
  891. }
  892. //-----------------------------------------------------------------------------
  893. // BlockSyncConfig (formerly known as FastSync) defines the configuration for the Tendermint block sync service
  894. // If this node is many blocks behind the tip of the chain, BlockSync
  895. // allows them to catchup quickly by downloading blocks in parallel
  896. // and verifying their commits.
  897. type BlockSyncConfig struct {
  898. Enable bool `mapstructure:"enable"`
  899. Version string `mapstructure:"version"`
  900. }
  901. // DefaultBlockSyncConfig returns a default configuration for the block sync service
  902. func DefaultBlockSyncConfig() *BlockSyncConfig {
  903. return &BlockSyncConfig{
  904. Enable: true,
  905. Version: BlockSyncV0,
  906. }
  907. }
  908. // TestBlockSyncConfig returns a default configuration for the block sync.
  909. func TestBlockSyncConfig() *BlockSyncConfig {
  910. return DefaultBlockSyncConfig()
  911. }
  912. // ValidateBasic performs basic validation.
  913. func (cfg *BlockSyncConfig) ValidateBasic() error {
  914. switch cfg.Version {
  915. case BlockSyncV0:
  916. return nil
  917. case BlockSyncV2:
  918. return errors.New("blocksync version v2 is no longer supported. Please use v0")
  919. default:
  920. return fmt.Errorf("unknown blocksync version %s", cfg.Version)
  921. }
  922. }
  923. //-----------------------------------------------------------------------------
  924. // ConsensusConfig
  925. // ConsensusConfig defines the configuration for the Tendermint consensus service,
  926. // including timeouts and details about the WAL and the block structure.
  927. type ConsensusConfig struct {
  928. RootDir string `mapstructure:"home"`
  929. WalPath string `mapstructure:"wal-file"`
  930. walFile string // overrides WalPath if set
  931. // TODO: remove timeout configs, these should be global not local
  932. // How long we wait for a proposal block before prevoting nil
  933. TimeoutPropose time.Duration `mapstructure:"timeout-propose"`
  934. // How much timeout-propose increases with each round
  935. TimeoutProposeDelta time.Duration `mapstructure:"timeout-propose-delta"`
  936. // How long we wait after receiving +2/3 prevotes for “anything” (ie. not a single block or nil)
  937. TimeoutPrevote time.Duration `mapstructure:"timeout-prevote"`
  938. // How much the timeout-prevote increases with each round
  939. TimeoutPrevoteDelta time.Duration `mapstructure:"timeout-prevote-delta"`
  940. // How long we wait after receiving +2/3 precommits for “anything” (ie. not a single block or nil)
  941. TimeoutPrecommit time.Duration `mapstructure:"timeout-precommit"`
  942. // How much the timeout-precommit increases with each round
  943. TimeoutPrecommitDelta time.Duration `mapstructure:"timeout-precommit-delta"`
  944. // How long we wait after committing a block, before starting on the new
  945. // height (this gives us a chance to receive some more precommits, even
  946. // though we already have +2/3).
  947. TimeoutCommit time.Duration `mapstructure:"timeout-commit"`
  948. // Make progress as soon as we have all the precommits (as if TimeoutCommit = 0)
  949. SkipTimeoutCommit bool `mapstructure:"skip-timeout-commit"`
  950. // EmptyBlocks mode and possible interval between empty blocks
  951. CreateEmptyBlocks bool `mapstructure:"create-empty-blocks"`
  952. CreateEmptyBlocksInterval time.Duration `mapstructure:"create-empty-blocks-interval"`
  953. // Reactor sleep duration parameters
  954. PeerGossipSleepDuration time.Duration `mapstructure:"peer-gossip-sleep-duration"`
  955. PeerQueryMaj23SleepDuration time.Duration `mapstructure:"peer-query-maj23-sleep-duration"`
  956. DoubleSignCheckHeight int64 `mapstructure:"double-sign-check-height"`
  957. }
  958. // DefaultConsensusConfig returns a default configuration for the consensus service
  959. func DefaultConsensusConfig() *ConsensusConfig {
  960. return &ConsensusConfig{
  961. WalPath: filepath.Join(defaultDataDir, "cs.wal", "wal"),
  962. TimeoutPropose: 3000 * time.Millisecond,
  963. TimeoutProposeDelta: 500 * time.Millisecond,
  964. TimeoutPrevote: 1000 * time.Millisecond,
  965. TimeoutPrevoteDelta: 500 * time.Millisecond,
  966. TimeoutPrecommit: 1000 * time.Millisecond,
  967. TimeoutPrecommitDelta: 500 * time.Millisecond,
  968. TimeoutCommit: 1000 * time.Millisecond,
  969. SkipTimeoutCommit: false,
  970. CreateEmptyBlocks: true,
  971. CreateEmptyBlocksInterval: 0 * time.Second,
  972. PeerGossipSleepDuration: 100 * time.Millisecond,
  973. PeerQueryMaj23SleepDuration: 2000 * time.Millisecond,
  974. DoubleSignCheckHeight: int64(0),
  975. }
  976. }
  977. // TestConsensusConfig returns a configuration for testing the consensus service
  978. func TestConsensusConfig() *ConsensusConfig {
  979. cfg := DefaultConsensusConfig()
  980. cfg.TimeoutPropose = 40 * time.Millisecond
  981. cfg.TimeoutProposeDelta = 1 * time.Millisecond
  982. cfg.TimeoutPrevote = 10 * time.Millisecond
  983. cfg.TimeoutPrevoteDelta = 1 * time.Millisecond
  984. cfg.TimeoutPrecommit = 10 * time.Millisecond
  985. cfg.TimeoutPrecommitDelta = 1 * time.Millisecond
  986. cfg.TimeoutCommit = 10 * time.Millisecond
  987. cfg.SkipTimeoutCommit = true
  988. cfg.PeerGossipSleepDuration = 5 * time.Millisecond
  989. cfg.PeerQueryMaj23SleepDuration = 250 * time.Millisecond
  990. cfg.DoubleSignCheckHeight = int64(0)
  991. return cfg
  992. }
  993. // WaitForTxs returns true if the consensus should wait for transactions before entering the propose step
  994. func (cfg *ConsensusConfig) WaitForTxs() bool {
  995. return !cfg.CreateEmptyBlocks || cfg.CreateEmptyBlocksInterval > 0
  996. }
  997. // Propose returns the amount of time to wait for a proposal
  998. func (cfg *ConsensusConfig) Propose(round int32) time.Duration {
  999. return time.Duration(
  1000. cfg.TimeoutPropose.Nanoseconds()+cfg.TimeoutProposeDelta.Nanoseconds()*int64(round),
  1001. ) * time.Nanosecond
  1002. }
  1003. // Prevote returns the amount of time to wait for straggler votes after receiving any +2/3 prevotes
  1004. func (cfg *ConsensusConfig) Prevote(round int32) time.Duration {
  1005. return time.Duration(
  1006. cfg.TimeoutPrevote.Nanoseconds()+cfg.TimeoutPrevoteDelta.Nanoseconds()*int64(round),
  1007. ) * time.Nanosecond
  1008. }
  1009. // Precommit returns the amount of time to wait for straggler votes after receiving any +2/3 precommits
  1010. func (cfg *ConsensusConfig) Precommit(round int32) time.Duration {
  1011. return time.Duration(
  1012. cfg.TimeoutPrecommit.Nanoseconds()+cfg.TimeoutPrecommitDelta.Nanoseconds()*int64(round),
  1013. ) * time.Nanosecond
  1014. }
  1015. // Commit returns the amount of time to wait for straggler votes after receiving +2/3 precommits
  1016. // for a single block (ie. a commit).
  1017. func (cfg *ConsensusConfig) Commit(t time.Time) time.Time {
  1018. return t.Add(cfg.TimeoutCommit)
  1019. }
  1020. // WalFile returns the full path to the write-ahead log file
  1021. func (cfg *ConsensusConfig) WalFile() string {
  1022. if cfg.walFile != "" {
  1023. return cfg.walFile
  1024. }
  1025. return rootify(cfg.WalPath, cfg.RootDir)
  1026. }
  1027. // SetWalFile sets the path to the write-ahead log file
  1028. func (cfg *ConsensusConfig) SetWalFile(walFile string) {
  1029. cfg.walFile = walFile
  1030. }
  1031. // ValidateBasic performs basic validation (checking param bounds, etc.) and
  1032. // returns an error if any check fails.
  1033. func (cfg *ConsensusConfig) ValidateBasic() error {
  1034. if cfg.TimeoutPropose < 0 {
  1035. return errors.New("timeout-propose can't be negative")
  1036. }
  1037. if cfg.TimeoutProposeDelta < 0 {
  1038. return errors.New("timeout-propose-delta can't be negative")
  1039. }
  1040. if cfg.TimeoutPrevote < 0 {
  1041. return errors.New("timeout-prevote can't be negative")
  1042. }
  1043. if cfg.TimeoutPrevoteDelta < 0 {
  1044. return errors.New("timeout-prevote-delta can't be negative")
  1045. }
  1046. if cfg.TimeoutPrecommit < 0 {
  1047. return errors.New("timeout-precommit can't be negative")
  1048. }
  1049. if cfg.TimeoutPrecommitDelta < 0 {
  1050. return errors.New("timeout-precommit-delta can't be negative")
  1051. }
  1052. if cfg.TimeoutCommit < 0 {
  1053. return errors.New("timeout-commit can't be negative")
  1054. }
  1055. if cfg.CreateEmptyBlocksInterval < 0 {
  1056. return errors.New("create-empty-blocks-interval can't be negative")
  1057. }
  1058. if cfg.PeerGossipSleepDuration < 0 {
  1059. return errors.New("peer-gossip-sleep-duration can't be negative")
  1060. }
  1061. if cfg.PeerQueryMaj23SleepDuration < 0 {
  1062. return errors.New("peer-query-maj23-sleep-duration can't be negative")
  1063. }
  1064. if cfg.DoubleSignCheckHeight < 0 {
  1065. return errors.New("double-sign-check-height can't be negative")
  1066. }
  1067. return nil
  1068. }
  1069. //-----------------------------------------------------------------------------
  1070. // TxIndexConfig
  1071. // Remember that Event has the following structure:
  1072. // type: [
  1073. // key: value,
  1074. // ...
  1075. // ]
  1076. //
  1077. // CompositeKeys are constructed by `type.key`
  1078. // TxIndexConfig defines the configuration for the transaction indexer,
  1079. // including composite keys to index.
  1080. type TxIndexConfig struct {
  1081. // The backend database list to back the indexer.
  1082. // If list contains `null`, meaning no indexer service will be used.
  1083. //
  1084. // Options:
  1085. // 1) "null" - no indexer services.
  1086. // 2) "kv" (default) - the simplest possible indexer,
  1087. // backed by key-value storage (defaults to levelDB; see DBBackend).
  1088. // 3) "psql" - the indexer services backed by PostgreSQL.
  1089. Indexer []string `mapstructure:"indexer"`
  1090. // The PostgreSQL connection configuration, the connection format:
  1091. // postgresql://<user>:<password>@<host>:<port>/<db>?<opts>
  1092. PsqlConn string `mapstructure:"psql-conn"`
  1093. }
  1094. // DefaultTxIndexConfig returns a default configuration for the transaction indexer.
  1095. func DefaultTxIndexConfig() *TxIndexConfig {
  1096. return &TxIndexConfig{
  1097. Indexer: []string{"kv"},
  1098. }
  1099. }
  1100. // TestTxIndexConfig returns a default configuration for the transaction indexer.
  1101. func TestTxIndexConfig() *TxIndexConfig {
  1102. return DefaultTxIndexConfig()
  1103. }
  1104. //-----------------------------------------------------------------------------
  1105. // InstrumentationConfig
  1106. // InstrumentationConfig defines the configuration for metrics reporting.
  1107. type InstrumentationConfig struct {
  1108. // When true, Prometheus metrics are served under /metrics on
  1109. // PrometheusListenAddr.
  1110. // Check out the documentation for the list of available metrics.
  1111. Prometheus bool `mapstructure:"prometheus"`
  1112. // Address to listen for Prometheus collector(s) connections.
  1113. PrometheusListenAddr string `mapstructure:"prometheus-listen-addr"`
  1114. // Maximum number of simultaneous connections.
  1115. // If you want to accept a larger number than the default, make sure
  1116. // you increase your OS limits.
  1117. // 0 - unlimited.
  1118. MaxOpenConnections int `mapstructure:"max-open-connections"`
  1119. // Instrumentation namespace.
  1120. Namespace string `mapstructure:"namespace"`
  1121. }
  1122. // DefaultInstrumentationConfig returns a default configuration for metrics
  1123. // reporting.
  1124. func DefaultInstrumentationConfig() *InstrumentationConfig {
  1125. return &InstrumentationConfig{
  1126. Prometheus: false,
  1127. PrometheusListenAddr: ":26660",
  1128. MaxOpenConnections: 3,
  1129. Namespace: "tendermint",
  1130. }
  1131. }
  1132. // TestInstrumentationConfig returns a default configuration for metrics
  1133. // reporting.
  1134. func TestInstrumentationConfig() *InstrumentationConfig {
  1135. return DefaultInstrumentationConfig()
  1136. }
  1137. // ValidateBasic performs basic validation (checking param bounds, etc.) and
  1138. // returns an error if any check fails.
  1139. func (cfg *InstrumentationConfig) ValidateBasic() error {
  1140. if cfg.MaxOpenConnections < 0 {
  1141. return errors.New("max-open-connections can't be negative")
  1142. }
  1143. return nil
  1144. }
  1145. //-----------------------------------------------------------------------------
  1146. // Utils
  1147. // helper function to make config creation independent of root dir
  1148. func rootify(path, root string) string {
  1149. if filepath.IsAbs(path) {
  1150. return path
  1151. }
  1152. return filepath.Join(root, path)
  1153. }
  1154. //-----------------------------------------------------------------------------
  1155. // Moniker
  1156. var defaultMoniker = getDefaultMoniker()
  1157. // getDefaultMoniker returns a default moniker, which is the host name. If runtime
  1158. // fails to get the host name, "anonymous" will be returned.
  1159. func getDefaultMoniker() string {
  1160. moniker, err := os.Hostname()
  1161. if err != nil {
  1162. moniker = "anonymous"
  1163. }
  1164. return moniker
  1165. }