diff --git a/.circleci/config.yml b/.circleci/config.yml index 55a3da4f9..0de4a1791 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -92,6 +92,7 @@ jobs: command: | export PATH="$GOBIN:$PATH" make get_tools + make get_dev_tools - run: name: dependencies command: | diff --git a/CHANGELOG.md b/CHANGELOG.md index eb22e8b84..bfdb9a50e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,24 @@ # Changelog +## v0.26.2 + +*November 15th, 2018* + +Special thanks to external contributors on this release: @hleb-albau, @zhuzeyu + +Friendly reminder, we have a [bug bounty program](https://hackerone.com/tendermint). + +### FEATURES: + +- [rpc] [\#2582](https://github.com/tendermint/tendermint/issues/2582) Enable CORS on RPC API (@hleb-albau) + +### BUG FIXES: + +- [abci] [\#2748](https://github.com/tendermint/tendermint/issues/2748) Unlock mutex in localClient so even when app panics (e.g. during CheckTx), consensus continue working +- [abci] [\#2748](https://github.com/tendermint/tendermint/issues/2748) Fix DATA RACE in localClient +- [amino] [\#2822](https://github.com/tendermint/tendermint/issues/2822) Update to v0.14.1 to support compiling on 32-bit platforms +- [rpc] [\#2748](https://github.com/tendermint/tendermint/issues/2748) Drain channel before calling Unsubscribe(All) in `/broadcast_tx_commit` + ## v0.26.1 *November 11, 2018* @@ -134,6 +153,8 @@ increasing attention to backwards compatibility. Thanks for bearing with us! - [abci] [\#2557](https://github.com/tendermint/tendermint/issues/2557) Add `Codespace` field to `Response{CheckTx, DeliverTx, Query}` - [abci] [\#2662](https://github.com/tendermint/tendermint/issues/2662) Add `BlockVersion` and `P2PVersion` to `RequestInfo` - [crypto/merkle] [\#2298](https://github.com/tendermint/tendermint/issues/2298) General Merkle Proof scheme for chaining various types of Merkle trees together +- [docs/architecture] [\#1181](https://github.com/tendermint/tendermint/issues/1181) S +plit immutable and mutable parts of priv_validator.json ### IMPROVEMENTS: - Additional Metrics diff --git a/CHANGELOG_PENDING.md b/CHANGELOG_PENDING.md index ea3b97599..6494867d7 100644 --- a/CHANGELOG_PENDING.md +++ b/CHANGELOG_PENDING.md @@ -1,12 +1,13 @@ # Pending -## v0.26.2 +## v0.26.3 -*TBA* +*TBD* Special thanks to external contributors on this release: -Friendly reminder, we have a [bug bounty program](https://hackerone.com/tendermint). +Friendly reminder, we have a [bug bounty +program](https://hackerone.com/tendermint). ### BREAKING CHANGES: @@ -20,9 +21,9 @@ Friendly reminder, we have a [bug bounty program](https://hackerone.com/tendermi * P2P Protocol + ### FEATURES: ### IMPROVEMENTS: ### BUG FIXES: - diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index d47c0f15e..7088fca48 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -6,7 +6,7 @@ This code of conduct applies to all projects run by the Tendermint/COSMOS team a # Conduct -## Contact: adrian@tendermint.com +## Contact: conduct@tendermint.com * We are committed to providing a friendly, safe and welcoming environment for all, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, nationality, or other similar characteristic. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 3500732f5..a0057aae5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -27,8 +27,8 @@ Of course, replace `ebuchman` with your git handle. To pull in updates from the origin repo, run - * `git fetch upstream` - * `git rebase upstream/master` (or whatever branch you want) + * `git fetch upstream` + * `git rebase upstream/master` (or whatever branch you want) Please don't make Pull Requests to `master`. @@ -50,6 +50,11 @@ as apps, tools, and the core, should use dep. Run `dep status` to get a list of vendor dependencies that may not be up-to-date. +When updating dependencies, please only update the particular dependencies you +need. Instead of running `dep ensure -update`, which will update anything, +specify exactly the dependency you want to update, eg. +`dep ensure -update github.com/tendermint/go-amino`. + ## Vagrant If you are a [Vagrant](https://www.vagrantup.com/) user, you can get started @@ -73,34 +78,41 @@ tested by circle using `go test -v -race ./...`. If not, they will need a `circle.yml`. Ideally, every repo has a `Makefile` that defines `make test` and includes its continuous integration status using a badge in the `README.md`. -## Branching Model and Release +## Changelog -User-facing repos should adhere to the branching model: http://nvie.com/posts/a-successful-git-branching-model/. -That is, these repos should be well versioned, and any merge to master requires a version bump and tagged release. +## Branching Model and Release -Libraries need not follow the model strictly, but would be wise to, -especially `go-p2p` and `go-rpc`, as their versions are referenced in tendermint core. +All repos should adhere to the branching model: http://nvie.com/posts/a-successful-git-branching-model/. +This means that all pull-requests should be made against develop. Any merge to +master constitutes a tagged release. ### Development Procedure: - the latest state of development is on `develop` - `develop` must never fail `make test` -- no --force onto `develop` (except when reverting a broken commit, which should seldom happen) +- never --force onto `develop` (except when reverting a broken commit, which should seldom happen) - create a development branch either on github.com/tendermint/tendermint, or your fork (using `git remote add origin`) -- before submitting a pull request, begin `git rebase` on top of `develop` +- make changes and update the `CHANGELOG_PENDING.md` to record your change +- before submitting a pull request, run `git rebase` on top of the latest `develop` ### Pull Merge Procedure: -- ensure pull branch is rebased on develop +- ensure pull branch is based on a recent develop - run `make test` to ensure that all tests pass - merge pull request -- the `unstable` branch may be used to aggregate pull merges before testing once -- push master may request that pull requests be rebased on top of `unstable` +- the `unstable` branch may be used to aggregate pull merges before fixing tests ### Release Procedure: - start on `develop` - run integration tests (see `test_integrations` in Makefile) -- prepare changelog/release issue +- prepare changelog: + - copy `CHANGELOG_PENDING.md` to `CHANGELOG.md` + - run `python ./scripts/linkify_changelog.py CHANGELOG.md` to add links for + all issues + - run `bash ./scripts/authors.sh` to get a list of authors since the latest + release, and add the github aliases of external contributors to the top of + the changelog. To lookup an alias from an email, try `bash + ./scripts/authors.sh ` - bump versions -- push to release-vX.X.X to run the extended integration tests on the CI +- push to release/vX.X.X to run the extended integration tests on the CI - merge to master - merge master back to develop diff --git a/Gopkg.lock b/Gopkg.lock index 35542bf62..8695205e7 100644 --- a/Gopkg.lock +++ b/Gopkg.lock @@ -128,6 +128,14 @@ revision = "ea4d1f681babbce9545c9c5f3d5194a789c89f5b" version = "v1.2.0" +[[projects]] + digest = "1:b0c25f00bad20d783d259af2af8666969e2fc343fa0dc9efe52936bbd67fb758" + name = "github.com/rs/cors" + packages = ["."] + pruneopts = "UT" + revision = "9a47f48565a795472d43519dd49aac781f3034fb" + version = "v1.6.0" + [[projects]] digest = "1:ea40c24cdbacd054a6ae9de03e62c5f252479b96c716375aace5c120d68647c8" name = "github.com/hashicorp/hcl" @@ -358,12 +366,12 @@ revision = "e5840949ff4fff0c56f9b6a541e22b63581ea9df" [[projects]] - digest = "1:10b3a599325740c84a7c81f3f3cb2e1fdb70b3ea01b7fa28495567a2519df431" + digest = "1:ad9c4c1a4e7875330b1f62906f2830f043a23edb5db997e3a5ac5d3e6eadf80a" name = "github.com/tendermint/go-amino" packages = ["."] pruneopts = "UT" - revision = "6dcc6ddc143e116455c94b25c1004c99e0d0ca12" - version = "v0.14.0" + revision = "dc14acf9ef15f85828bfbc561ed9dd9d2a284885" + version = "v0.14.1" [[projects]] digest = "1:72b71e3a29775e5752ed7a8012052a3dee165e27ec18cedddae5288058f09acf" diff --git a/Gopkg.toml b/Gopkg.toml index 47418bef3..1ab6a18e9 100644 --- a/Gopkg.toml +++ b/Gopkg.toml @@ -40,6 +40,10 @@ name = "github.com/gorilla/websocket" version = "=1.2.0" +[[constraint]] + name = "github.com/rs/cors" + version = "1.6.0" + [[constraint]] name = "github.com/pkg/errors" version = "=0.8.0" @@ -58,7 +62,7 @@ [[constraint]] name = "github.com/tendermint/go-amino" - version = "v0.14.0" + version = "v0.14.1" [[constraint]] name = "google.golang.org/grpc" diff --git a/Makefile b/Makefile index 4390b1fbb..294a319b3 100644 --- a/Makefile +++ b/Makefile @@ -17,7 +17,6 @@ all: check build test install check: check_tools get_vendor_deps - ######################################## ### Build Tendermint @@ -79,6 +78,8 @@ check_tools: get_tools: @echo "--> Installing tools" ./scripts/get_tools.sh + +get_dev_tools: @echo "--> Downloading linters (this may take awhile)" $(GOPATH)/src/github.com/alecthomas/gometalinter/scripts/install.sh -b $(GOBIN) @@ -327,4 +328,4 @@ build-slate: # To avoid unintended conflicts with file names, always add to .PHONY # unless there is a reason not to. # https://www.gnu.org/software/make/manual/html_node/Phony-Targets.html -.PHONY: check build build_race build_abci dist install install_abci check_dep check_tools get_tools update_tools get_vendor_deps draw_deps get_protoc protoc_abci protoc_libs gen_certs clean_certs grpc_dbserver test_cover test_apps test_persistence test_p2p test test_race test_integrations test_release test100 vagrant_test fmt rpc-docs build-linux localnet-start localnet-stop build-docker build-docker-localnode sentry-start sentry-config sentry-stop build-slate protoc_grpc protoc_all +.PHONY: check build build_race build_abci dist install install_abci check_dep check_tools get_tools get_dev_tools update_tools get_vendor_deps draw_deps get_protoc protoc_abci protoc_libs gen_certs clean_certs grpc_dbserver test_cover test_apps test_persistence test_p2p test test_race test_integrations test_release test100 vagrant_test fmt rpc-docs build-linux localnet-start localnet-stop build-docker build-docker-localnode sentry-start sentry-config sentry-stop build-slate protoc_grpc protoc_all diff --git a/README.md b/README.md index 328557ae3..7c386ec3a 100644 --- a/README.md +++ b/README.md @@ -41,7 +41,7 @@ please [contact us](mailto:partners@tendermint.com) and [join the chat](https:// ## Security To report a security vulnerability, see our [bug bounty -program](https://tendermint.com/security). +program](https://hackerone.com/tendermint) For examples of the kinds of bugs we're looking for, see [SECURITY.md](SECURITY.md) @@ -51,14 +51,18 @@ Requirement|Notes ---|--- Go version | Go1.10 or higher -## Install +## Documentation + +Complete documentation can be found on the [website](https://tendermint.com/docs/). + +### Install See the [install instructions](/docs/introduction/install.md) -## Quick Start +### Quick Start -- [Single node](/docs/tendermint-core/using-tendermint.md) -- [Local cluster using docker-compose](/networks/local) +- [Single node](/docs/introduction/quick-start.md) +- [Local cluster using docker-compose](/docs/networks/docker-compose.md) - [Remote cluster using terraform and ansible](/docs/networks/terraform-and-ansible.md) - [Join the Cosmos testnet](https://cosmos.network/testnet) @@ -91,6 +95,7 @@ Additional documentation is found [here](/docs/tools). ### Research +* [The latest gossip on BFT consensus](https://arxiv.org/abs/1807.04938) * [Master's Thesis on Tendermint](https://atrium.lib.uoguelph.ca/xmlui/handle/10214/9769) * [Original Whitepaper](https://tendermint.com/static/docs/tendermint.pdf) * [Blog](https://blog.cosmos.network/tendermint/home) diff --git a/SECURITY.md b/SECURITY.md index 8b9793782..8a373a290 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -1,7 +1,8 @@ # Security As part of our [Coordinated Vulnerability Disclosure -Policy](https://tendermint.com/security), we operate a bug bounty. +Policy](https://tendermint.com/security), we operate a [bug +bounty](https://hackerone.com/tendermint). See the policy for more details on submissions and rewards. Here is a list of examples of the kinds of bugs we're most interested in: diff --git a/abci/client/client.go b/abci/client/client.go index 558588107..e1eea5d4e 100644 --- a/abci/client/client.go +++ b/abci/client/client.go @@ -105,8 +105,8 @@ func (reqRes *ReqRes) SetCallback(cb func(res *types.Response)) { return } - defer reqRes.mtx.Unlock() reqRes.cb = cb + reqRes.mtx.Unlock() } func (reqRes *ReqRes) GetCallback() func(*types.Response) { diff --git a/abci/client/grpc_client.go b/abci/client/grpc_client.go index 4f37b17b6..d5cd233a3 100644 --- a/abci/client/grpc_client.go +++ b/abci/client/grpc_client.go @@ -111,8 +111,8 @@ func (cli *grpcClient) Error() error { // NOTE: callback may get internally generated flush responses. func (cli *grpcClient) SetResponseCallback(resCb Callback) { cli.mtx.Lock() - defer cli.mtx.Unlock() cli.resCb = resCb + cli.mtx.Unlock() } //---------------------------------------- diff --git a/abci/client/local_client.go b/abci/client/local_client.go index 3ac3b6afa..d0e50c330 100644 --- a/abci/client/local_client.go +++ b/abci/client/local_client.go @@ -9,8 +9,13 @@ import ( var _ Client = (*localClient)(nil) +// NOTE: use defer to unlock mutex because Application might panic (e.g., in +// case of malicious tx or query). It only makes sense for publicly exposed +// methods like CheckTx (/broadcast_tx_* RPC endpoint) or Query (/abci_query +// RPC endpoint), but defers are used everywhere for the sake of consistency. type localClient struct { cmn.BaseService + mtx *sync.Mutex types.Application Callback @@ -30,8 +35,8 @@ func NewLocalClient(mtx *sync.Mutex, app types.Application) *localClient { func (app *localClient) SetResponseCallback(cb Callback) { app.mtx.Lock() - defer app.mtx.Unlock() app.Callback = cb + app.mtx.Unlock() } // TODO: change types.Application to include Error()? @@ -45,6 +50,9 @@ func (app *localClient) FlushAsync() *ReqRes { } func (app *localClient) EchoAsync(msg string) *ReqRes { + app.mtx.Lock() + defer app.mtx.Unlock() + return app.callback( types.ToRequestEcho(msg), types.ToResponseEcho(msg), @@ -53,8 +61,9 @@ func (app *localClient) EchoAsync(msg string) *ReqRes { func (app *localClient) InfoAsync(req types.RequestInfo) *ReqRes { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.Info(req) - app.mtx.Unlock() return app.callback( types.ToRequestInfo(req), types.ToResponseInfo(res), @@ -63,8 +72,9 @@ func (app *localClient) InfoAsync(req types.RequestInfo) *ReqRes { func (app *localClient) SetOptionAsync(req types.RequestSetOption) *ReqRes { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.SetOption(req) - app.mtx.Unlock() return app.callback( types.ToRequestSetOption(req), types.ToResponseSetOption(res), @@ -73,8 +83,9 @@ func (app *localClient) SetOptionAsync(req types.RequestSetOption) *ReqRes { func (app *localClient) DeliverTxAsync(tx []byte) *ReqRes { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.DeliverTx(tx) - app.mtx.Unlock() return app.callback( types.ToRequestDeliverTx(tx), types.ToResponseDeliverTx(res), @@ -83,8 +94,9 @@ func (app *localClient) DeliverTxAsync(tx []byte) *ReqRes { func (app *localClient) CheckTxAsync(tx []byte) *ReqRes { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.CheckTx(tx) - app.mtx.Unlock() return app.callback( types.ToRequestCheckTx(tx), types.ToResponseCheckTx(res), @@ -93,8 +105,9 @@ func (app *localClient) CheckTxAsync(tx []byte) *ReqRes { func (app *localClient) QueryAsync(req types.RequestQuery) *ReqRes { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.Query(req) - app.mtx.Unlock() return app.callback( types.ToRequestQuery(req), types.ToResponseQuery(res), @@ -103,8 +116,9 @@ func (app *localClient) QueryAsync(req types.RequestQuery) *ReqRes { func (app *localClient) CommitAsync() *ReqRes { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.Commit() - app.mtx.Unlock() return app.callback( types.ToRequestCommit(), types.ToResponseCommit(res), @@ -113,19 +127,20 @@ func (app *localClient) CommitAsync() *ReqRes { func (app *localClient) InitChainAsync(req types.RequestInitChain) *ReqRes { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.InitChain(req) - reqRes := app.callback( + return app.callback( types.ToRequestInitChain(req), types.ToResponseInitChain(res), ) - app.mtx.Unlock() - return reqRes } func (app *localClient) BeginBlockAsync(req types.RequestBeginBlock) *ReqRes { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.BeginBlock(req) - app.mtx.Unlock() return app.callback( types.ToRequestBeginBlock(req), types.ToResponseBeginBlock(res), @@ -134,8 +149,9 @@ func (app *localClient) BeginBlockAsync(req types.RequestBeginBlock) *ReqRes { func (app *localClient) EndBlockAsync(req types.RequestEndBlock) *ReqRes { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.EndBlock(req) - app.mtx.Unlock() return app.callback( types.ToRequestEndBlock(req), types.ToResponseEndBlock(res), @@ -154,64 +170,73 @@ func (app *localClient) EchoSync(msg string) (*types.ResponseEcho, error) { func (app *localClient) InfoSync(req types.RequestInfo) (*types.ResponseInfo, error) { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.Info(req) - app.mtx.Unlock() return &res, nil } func (app *localClient) SetOptionSync(req types.RequestSetOption) (*types.ResponseSetOption, error) { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.SetOption(req) - app.mtx.Unlock() return &res, nil } func (app *localClient) DeliverTxSync(tx []byte) (*types.ResponseDeliverTx, error) { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.DeliverTx(tx) - app.mtx.Unlock() return &res, nil } func (app *localClient) CheckTxSync(tx []byte) (*types.ResponseCheckTx, error) { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.CheckTx(tx) - app.mtx.Unlock() return &res, nil } func (app *localClient) QuerySync(req types.RequestQuery) (*types.ResponseQuery, error) { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.Query(req) - app.mtx.Unlock() return &res, nil } func (app *localClient) CommitSync() (*types.ResponseCommit, error) { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.Commit() - app.mtx.Unlock() return &res, nil } func (app *localClient) InitChainSync(req types.RequestInitChain) (*types.ResponseInitChain, error) { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.InitChain(req) - app.mtx.Unlock() return &res, nil } func (app *localClient) BeginBlockSync(req types.RequestBeginBlock) (*types.ResponseBeginBlock, error) { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.BeginBlock(req) - app.mtx.Unlock() return &res, nil } func (app *localClient) EndBlockSync(req types.RequestEndBlock) (*types.ResponseEndBlock, error) { app.mtx.Lock() + defer app.mtx.Unlock() + res := app.Application.EndBlock(req) - app.mtx.Unlock() return &res, nil } diff --git a/abci/client/socket_client.go b/abci/client/socket_client.go index affea1a9e..531d12bca 100644 --- a/abci/client/socket_client.go +++ b/abci/client/socket_client.go @@ -118,8 +118,8 @@ func (cli *socketClient) Error() error { // NOTE: callback may get internally generated flush responses. func (cli *socketClient) SetResponseCallback(resCb Callback) { cli.mtx.Lock() - defer cli.mtx.Unlock() cli.resCb = resCb + cli.mtx.Unlock() } //---------------------------------------- diff --git a/config/config.go b/config/config.go index fa6182114..ea6582c09 100644 --- a/config/config.go +++ b/config/config.go @@ -242,6 +242,18 @@ type RPCConfig struct { // TCP or UNIX socket address for the RPC server to listen on ListenAddress string `mapstructure:"laddr"` + // A list of origins a cross-domain request can be executed from. + // If the special '*' value is present in the list, all origins will be allowed. + // An origin may contain a wildcard (*) to replace 0 or more characters (i.e.: http://*.domain.com). + // Only one wildcard can be used per origin. + CORSAllowedOrigins []string `mapstructure:"cors_allowed_origins"` + + // A list of methods the client is allowed to use with cross-domain requests. + CORSAllowedMethods []string `mapstructure:"cors_allowed_methods"` + + // A list of non simple headers the client is allowed to use with cross-domain requests. + CORSAllowedHeaders []string `mapstructure:"cors_allowed_headers"` + // TCP or UNIX socket address for the gRPC server to listen on // NOTE: This server only supports /broadcast_tx_commit GRPCListenAddress string `mapstructure:"grpc_laddr"` @@ -269,8 +281,10 @@ type RPCConfig struct { // DefaultRPCConfig returns a default configuration for the RPC server func DefaultRPCConfig() *RPCConfig { return &RPCConfig{ - ListenAddress: "tcp://0.0.0.0:26657", - + ListenAddress: "tcp://0.0.0.0:26657", + CORSAllowedOrigins: []string{}, + CORSAllowedMethods: []string{"HEAD", "GET", "POST"}, + CORSAllowedHeaders: []string{"Origin", "Accept", "Content-Type", "X-Requested-With", "X-Server-Time"}, GRPCListenAddress: "", GRPCMaxOpenConnections: 900, @@ -300,6 +314,11 @@ func (cfg *RPCConfig) ValidateBasic() error { return nil } +// IsCorsEnabled returns true if cross-origin resource sharing is enabled. +func (cfg *RPCConfig) IsCorsEnabled() bool { + return len(cfg.CORSAllowedOrigins) != 0 +} + //----------------------------------------------------------------------------- // P2PConfig diff --git a/config/toml.go b/config/toml.go index d73b9c81d..89be3783d 100644 --- a/config/toml.go +++ b/config/toml.go @@ -119,6 +119,17 @@ filter_peers = {{ .BaseConfig.FilterPeers }} # TCP or UNIX socket address for the RPC server to listen on laddr = "{{ .RPC.ListenAddress }}" +# A list of origins a cross-domain request can be executed from +# Default value '[]' disables cors support +# Use '["*"]' to allow any origin +cors_allowed_origins = "{{ .RPC.CORSAllowedOrigins }}" + +# A list of methods the client is allowed to use with cross-domain requests +cors_allowed_methods = "{{ .RPC.CORSAllowedMethods }}" + +# A list of non simple headers the client is allowed to use with cross-domain requests +cors_allowed_headers = "{{ .RPC.CORSAllowedHeaders }}" + # TCP or UNIX socket address for the gRPC server to listen on # NOTE: This server only supports /broadcast_tx_commit grpc_laddr = "{{ .RPC.GRPCListenAddress }}" diff --git a/consensus/reactor.go b/consensus/reactor.go index 12e8e0f14..1768a8f08 100644 --- a/consensus/reactor.go +++ b/consensus/reactor.go @@ -183,7 +183,11 @@ func (conR *ConsensusReactor) RemovePeer(peer p2p.Peer, reason interface{}) { return } // TODO - //peer.Get(PeerStateKey).(*PeerState).Disconnect() + // ps, ok := peer.Get(PeerStateKey).(*PeerState) + // if !ok { + // panic(fmt.Sprintf("Peer %v has no state", peer)) + // } + // ps.Disconnect() } // Receive implements Reactor @@ -214,7 +218,10 @@ func (conR *ConsensusReactor) Receive(chID byte, src p2p.Peer, msgBytes []byte) conR.Logger.Debug("Receive", "src", src, "chId", chID, "msg", msg) // Get peer states - ps := src.Get(types.PeerStateKey).(*PeerState) + ps, ok := src.Get(types.PeerStateKey).(*PeerState) + if !ok { + panic(fmt.Sprintf("Peer %v has no state", src)) + } switch chID { case StateChannel: @@ -293,9 +300,9 @@ func (conR *ConsensusReactor) Receive(chID byte, src p2p.Peer, msgBytes []byte) switch msg := msg.(type) { case *VoteMessage: cs := conR.conS - cs.mtx.Lock() + cs.mtx.RLock() height, valSize, lastCommitSize := cs.Height, cs.Validators.Size(), cs.LastCommit.Size() - cs.mtx.Unlock() + cs.mtx.RUnlock() ps.EnsureVoteBitArrays(height, valSize) ps.EnsureVoteBitArrays(height-1, lastCommitSize) ps.SetHasVote(msg.Vote) @@ -428,7 +435,10 @@ func (conR *ConsensusReactor) broadcastHasVoteMessage(vote *types.Vote) { /* // TODO: Make this broadcast more selective. for _, peer := range conR.Switch.Peers().List() { - ps := peer.Get(PeerStateKey).(*PeerState) + ps, ok := peer.Get(PeerStateKey).(*PeerState) + if !ok { + panic(fmt.Sprintf("Peer %v has no state", peer)) + } prs := ps.GetRoundState() if prs.Height == vote.Height { // TODO: Also filter on round? @@ -826,7 +836,10 @@ func (conR *ConsensusReactor) peerStatsRoutine() { continue } // Get peer state - ps := peer.Get(types.PeerStateKey).(*PeerState) + ps, ok := peer.Get(types.PeerStateKey).(*PeerState) + if !ok { + panic(fmt.Sprintf("Peer %v has no state", peer)) + } switch msg.Msg.(type) { case *VoteMessage: if numVotes := ps.RecordVote(); numVotes%votesToContributeToBecomeGoodPeer == 0 { @@ -859,7 +872,10 @@ func (conR *ConsensusReactor) StringIndented(indent string) string { s := "ConsensusReactor{\n" s += indent + " " + conR.conS.StringIndented(indent+" ") + "\n" for _, peer := range conR.Switch.Peers().List() { - ps := peer.Get(types.PeerStateKey).(*PeerState) + ps, ok := peer.Get(types.PeerStateKey).(*PeerState) + if !ok { + panic(fmt.Sprintf("Peer %v has no state", peer)) + } s += indent + " " + ps.StringIndented(indent+" ") + "\n" } s += indent + "}" diff --git a/consensus/replay_file.go b/consensus/replay_file.go index 685eb71f2..a326e70ef 100644 --- a/consensus/replay_file.go +++ b/consensus/replay_file.go @@ -58,7 +58,18 @@ func (cs *ConsensusState) ReplayFile(file string, console bool) error { if err != nil { return errors.Errorf("failed to subscribe %s to %v", subscriber, types.EventQueryNewRoundStep) } - defer cs.eventBus.Unsubscribe(ctx, subscriber, types.EventQueryNewRoundStep) + defer func() { + // drain newStepCh to make sure we don't block + LOOP: + for { + select { + case <-newStepCh: + default: + break LOOP + } + } + cs.eventBus.Unsubscribe(ctx, subscriber, types.EventQueryNewRoundStep) + }() // just open the file for reading, no need to use wal fp, err := os.OpenFile(file, os.O_RDONLY, 0600) @@ -221,7 +232,18 @@ func (pb *playback) replayConsoleLoop() int { if err != nil { cmn.Exit(fmt.Sprintf("failed to subscribe %s to %v", subscriber, types.EventQueryNewRoundStep)) } - defer pb.cs.eventBus.Unsubscribe(ctx, subscriber, types.EventQueryNewRoundStep) + defer func() { + // drain newStepCh to make sure we don't block + LOOP: + for { + select { + case <-newStepCh: + default: + break LOOP + } + } + pb.cs.eventBus.Unsubscribe(ctx, subscriber, types.EventQueryNewRoundStep) + }() if len(tokens) == 1 { if err := pb.replayReset(1, newStepCh); err != nil { diff --git a/consensus/state.go b/consensus/state.go index 8c2e292c8..e8603011f 100644 --- a/consensus/state.go +++ b/consensus/state.go @@ -207,18 +207,16 @@ func (cs *ConsensusState) GetState() sm.State { // GetLastHeight returns the last height committed. // If there were no blocks, returns 0. func (cs *ConsensusState) GetLastHeight() int64 { - cs.mtx.Lock() - defer cs.mtx.Unlock() - + cs.mtx.RLock() + defer cs.mtx.RUnlock() return cs.RoundState.Height - 1 } // GetRoundState returns a shallow copy of the internal consensus state. func (cs *ConsensusState) GetRoundState() *cstypes.RoundState { cs.mtx.RLock() - defer cs.mtx.RUnlock() - rs := cs.RoundState // copy + cs.mtx.RUnlock() return &rs } @@ -226,7 +224,6 @@ func (cs *ConsensusState) GetRoundState() *cstypes.RoundState { func (cs *ConsensusState) GetRoundStateJSON() ([]byte, error) { cs.mtx.RLock() defer cs.mtx.RUnlock() - return cdc.MarshalJSON(cs.RoundState) } @@ -234,7 +231,6 @@ func (cs *ConsensusState) GetRoundStateJSON() ([]byte, error) { func (cs *ConsensusState) GetRoundStateSimpleJSON() ([]byte, error) { cs.mtx.RLock() defer cs.mtx.RUnlock() - return cdc.MarshalJSON(cs.RoundState.RoundStateSimple()) } @@ -248,15 +244,15 @@ func (cs *ConsensusState) GetValidators() (int64, []*types.Validator) { // SetPrivValidator sets the private validator account for signing votes. func (cs *ConsensusState) SetPrivValidator(priv types.PrivValidator) { cs.mtx.Lock() - defer cs.mtx.Unlock() cs.privValidator = priv + cs.mtx.Unlock() } // SetTimeoutTicker sets the local timer. It may be useful to overwrite for testing. func (cs *ConsensusState) SetTimeoutTicker(timeoutTicker TimeoutTicker) { cs.mtx.Lock() - defer cs.mtx.Unlock() cs.timeoutTicker = timeoutTicker + cs.mtx.Unlock() } // LoadCommit loads the commit for a given height. diff --git a/docs/app-dev/ecosystem.json b/docs/app-dev/ecosystem.json index 1c2bc2b25..f66ed28b6 100644 --- a/docs/app-dev/ecosystem.json +++ b/docs/app-dev/ecosystem.json @@ -175,35 +175,5 @@ "language": "Javascript", "author": "Dennis McKinnon" } - ], - "deploymentTools": [ - { - "name": "mintnet-kubernetes", - "url": "https://github.com/tendermint/tools", - "technology": "Docker and Kubernetes", - "author": "Tendermint", - "description": "Deploy a Tendermint test network using Google's kubernetes" - }, - { - "name": "terraforce", - "url": "https://github.com/tendermint/tools", - "technology": "Terraform", - "author": "Tendermint", - "description": "Terraform + our custom terraforce tool; deploy a production Tendermint network with load balancing over multiple AWS availability zones" - }, - { - "name": "ansible-tendermint", - "url": "https://github.com/tendermint/tools", - "technology": "Ansible", - "author": "Tendermint", - "description": "Ansible playbooks + Tendermint" - }, - { - "name": "brooklyn-tendermint", - "url": "https://github.com/cloudsoft/brooklyn-tendermint", - "technology": "Clocker for Apache Brooklyn ", - "author": "Cloudsoft", - "description": "Deploy a tendermint test network in docker containers " - } ] } diff --git a/docs/app-dev/ecosystem.md b/docs/app-dev/ecosystem.md index 7960e6c0d..e51ca430a 100644 --- a/docs/app-dev/ecosystem.md +++ b/docs/app-dev/ecosystem.md @@ -9,13 +9,3 @@ We thank the community for their contributions thus far and welcome the addition of new projects. A pull request can be submitted to [this file](https://github.com/tendermint/tendermint/blob/master/docs/app-dev/ecosystem.json) to include your project. - -## Other Tools - -See [deploy testnets](./deploy-testnets) for information about all -the tools built by Tendermint. We have Kubernetes, Ansible, and -Terraform integrations. - -For upgrading from older to newer versions of tendermint and to migrate -your chain data, see [tm-migrator](https://github.com/hxzqlh/tm-tools) -written by @hxzqlh. diff --git a/docs/architecture/adr-034-priv-validator-file-structure.md b/docs/architecture/adr-034-priv-validator-file-structure.md new file mode 100644 index 000000000..83160bfb8 --- /dev/null +++ b/docs/architecture/adr-034-priv-validator-file-structure.md @@ -0,0 +1,72 @@ +# ADR 034: PrivValidator file structure + +## Changelog + +03-11-2018: Initial Draft + +## Context + +For now, the PrivValidator file `priv_validator.json` contains mutable and immutable parts. +Even in an insecure mode which does not encrypt private key on disk, it is reasonable to separate +the mutable part and immutable part. + +References: +[#1181](https://github.com/tendermint/tendermint/issues/1181) +[#2657](https://github.com/tendermint/tendermint/issues/2657) +[#2313](https://github.com/tendermint/tendermint/issues/2313) + +## Proposed Solution + +We can split mutable and immutable parts with two structs: +```go +// FilePVKey stores the immutable part of PrivValidator +type FilePVKey struct { + Address types.Address `json:"address"` + PubKey crypto.PubKey `json:"pub_key"` + PrivKey crypto.PrivKey `json:"priv_key"` + + filePath string +} + +// FilePVState stores the mutable part of PrivValidator +type FilePVLastSignState struct { + Height int64 `json:"height"` + Round int `json:"round"` + Step int8 `json:"step"` + Signature []byte `json:"signature,omitempty"` + SignBytes cmn.HexBytes `json:"signbytes,omitempty"` + + filePath string + mtx sync.Mutex +} +``` + +Then we can combine `FilePVKey` with `FilePVLastSignState` and will get the original `FilePV`. + +```go +type FilePV struct { + Key FilePVKey + LastSignState FilePVLastSignState +} +``` + +As discussed, `FilePV` should be located in `config`, and `FilePVLastSignState` should be stored in `data`. The +store path of each file should be specified in `config.yml`. + +What we need to do next is changing the methods of `FilePV`. + +## Status + +Draft. + +## Consequences + +### Positive + +- separate the mutable and immutable of PrivValidator + +### Negative + +- need to add more config for file path + +### Neutral diff --git a/docs/tendermint-core/configuration.md b/docs/tendermint-core/configuration.md index 8b3c3c22f..7052ca507 100644 --- a/docs/tendermint-core/configuration.md +++ b/docs/tendermint-core/configuration.md @@ -68,6 +68,17 @@ filter_peers = false # TCP or UNIX socket address for the RPC server to listen on laddr = "tcp://0.0.0.0:26657" +# A list of origins a cross-domain request can be executed from +# Default value '[]' disables cors support +# Use '["*"]' to allow any origin +cors_allowed_origins = "[]" + +# A list of methods the client is allowed to use with cross-domain requests +cors_allowed_methods = "[HEAD GET POST]" + +# A list of non simple headers the client is allowed to use with cross-domain requests +cors_allowed_headers = "[Origin Accept Content-Type X-Requested-With X-Server-Time]" + # TCP or UNIX socket address for the gRPC server to listen on # NOTE: This server only supports /broadcast_tx_commit grpc_laddr = "" diff --git a/docs/tendermint-core/using-tendermint.md b/docs/tendermint-core/using-tendermint.md index 5ee18361f..c99150427 100644 --- a/docs/tendermint-core/using-tendermint.md +++ b/docs/tendermint-core/using-tendermint.md @@ -60,42 +60,34 @@ definition](https://github.com/tendermint/tendermint/blob/master/types/genesis.g ``` { - "genesis_time": "2018-07-09T22:43:06.255718641Z", - "chain_id": "chain-IAkWsK", - "validators": [ - { - "pub_key": { - "type": "tendermint/PubKeyEd25519", - "value": "oX8HhKsErMluxI0QWNSR8djQMSupDvHdAYrHwP7n73k=" - }, - "power": "1", - "name": "node0" - }, - { - "pub_key": { - "type": "tendermint/PubKeyEd25519", - "value": "UZNSJA9zmeFQj36Rs296lY+WFQ4Rt6s7snPpuKypl5I=" - }, - "power": "1", - "name": "node1" + "genesis_time": "2018-11-13T18:11:50.277637Z", + "chain_id": "test-chain-s4ui7D", + "consensus_params": { + "block_size": { + "max_bytes": "22020096", + "max_gas": "-1" }, - { - "pub_key": { - "type": "tendermint/PubKeyEd25519", - "value": "i9GrM6/MHB4zjCelMZBUYHNXYIzl4n0RkDCVmmLhS/o=" - }, - "power": "1", - "name": "node2" + "evidence": { + "max_age": "100000" }, + "validator": { + "pub_key_types": [ + "ed25519" + ] + } + }, + "validators": [ { + "address": "39C04A480B54AB258A45355A5E48ADDED9956C65", "pub_key": { "type": "tendermint/PubKeyEd25519", - "value": "0qq7954l87trEqbQV9c7d1gurnjTGMxreXc848ZZ5aw=" + "value": "DMEMMj1+thrkUCGocbvvKzXeaAtRslvX9MWtB+smuIA=" }, - "power": "1", - "name": "node3" + "power": "10", + "name": "" } - ] + ], + "app_hash": "" } ``` diff --git a/evidence/reactor.go b/evidence/reactor.go index 32753b2b9..48092fdff 100644 --- a/evidence/reactor.go +++ b/evidence/reactor.go @@ -160,12 +160,15 @@ func (evR *EvidenceReactor) broadcastEvidenceRoutine(peer p2p.Peer) { // Returns the message to send the peer, or nil if the evidence is invalid for the peer. // If message is nil, return true if we should sleep and try again. func (evR EvidenceReactor) checkSendEvidenceMessage(peer p2p.Peer, ev types.Evidence) (msg EvidenceMessage, retry bool) { - // make sure the peer is up to date evHeight := ev.Height() peerState, ok := peer.Get(types.PeerStateKey).(PeerState) - if !ok { - evR.Logger.Info("Found peer without PeerState", "peer", peer) + if !ok { + // Peer does not have a state yet. We set it in the consensus reactor, but + // when we add peer in Switch, the order we call reactors#AddPeer is + // different every time due to us using a map. Sometimes other reactors + // will be initialized before the consensus reactor. We should wait a few + // milliseconds and retry. return nil, true } diff --git a/evidence/reactor_test.go b/evidence/reactor_test.go index 69dcdec57..1c4e731ab 100644 --- a/evidence/reactor_test.go +++ b/evidence/reactor_test.go @@ -165,6 +165,16 @@ func TestReactorSelectiveBroadcast(t *testing.T) { // make reactors from statedb reactors := makeAndConnectEvidenceReactors(config, []dbm.DB{stateDB1, stateDB2}) + + // set the peer height on each reactor + for _, r := range reactors { + for _, peer := range r.Switch.Peers().List() { + ps := peerState{height1} + peer.Set(types.PeerStateKey, ps) + } + } + + // update the first reactor peer's height to be very small peer := reactors[0].Switch.Peers().List()[0] ps := peerState{height2} peer.Set(types.PeerStateKey, ps) diff --git a/libs/pubsub/pubsub.go b/libs/pubsub/pubsub.go index 18f098d87..b4e392bbd 100644 --- a/libs/pubsub/pubsub.go +++ b/libs/pubsub/pubsub.go @@ -30,9 +30,15 @@ // // s.Subscribe(ctx, sub, qry, out) // defer func() { -// for range out { -// // drain out to make sure we don't block -// } +// // drain out to make sure we don't block +// LOOP: +// for { +// select { +// case <-out: +// default: +// break LOOP +// } +// } // s.UnsubscribeAll(ctx, sub) // }() // for msg := range out { diff --git a/mempool/mempool.go b/mempool/mempool.go index b84eb4a68..0bdb47140 100644 --- a/mempool/mempool.go +++ b/mempool/mempool.go @@ -300,6 +300,7 @@ func (mem *Mempool) TxsWaitChan() <-chan struct{} { // CONTRACT: Either cb will get called, or err returned. func (mem *Mempool) CheckTx(tx types.Tx, cb func(*abci.Response)) (err error) { mem.proxyMtx.Lock() + // use defer to unlock mutex because application (*local client*) might panic defer mem.proxyMtx.Unlock() if mem.Size() >= mem.config.Size { diff --git a/mempool/reactor.go b/mempool/reactor.go index 96988be78..072f96675 100644 --- a/mempool/reactor.go +++ b/mempool/reactor.go @@ -133,16 +133,23 @@ func (memR *MempoolReactor) broadcastTxRoutine(peer p2p.Peer) { } memTx := next.Value.(*mempoolTx) + // make sure the peer is up to date - height := memTx.Height() - if peerState_i := peer.Get(types.PeerStateKey); peerState_i != nil { - peerState := peerState_i.(PeerState) - peerHeight := peerState.GetHeight() - if peerHeight < height-1 { // Allow for a lag of 1 block - time.Sleep(peerCatchupSleepIntervalMS * time.Millisecond) - continue - } + peerState, ok := peer.Get(types.PeerStateKey).(PeerState) + if !ok { + // Peer does not have a state yet. We set it in the consensus reactor, but + // when we add peer in Switch, the order we call reactors#AddPeer is + // different every time due to us using a map. Sometimes other reactors + // will be initialized before the consensus reactor. We should wait a few + // milliseconds and retry. + time.Sleep(peerCatchupSleepIntervalMS * time.Millisecond) + continue } + if peerState.GetHeight() < memTx.Height()-1 { // Allow for a lag of 1 block + time.Sleep(peerCatchupSleepIntervalMS * time.Millisecond) + continue + } + // send memTx msg := &TxMessage{Tx: memTx.tx} success := peer.Send(MempoolChannel, cdc.MustMarshalBinaryBare(msg)) diff --git a/mempool/reactor_test.go b/mempool/reactor_test.go index 8ac400b0a..ad9ad8b40 100644 --- a/mempool/reactor_test.go +++ b/mempool/reactor_test.go @@ -21,6 +21,14 @@ import ( "github.com/tendermint/tendermint/types" ) +type peerState struct { + height int64 +} + +func (ps peerState) GetHeight() int64 { + return ps.height +} + // mempoolLogger is a TestingLogger which uses a different // color for each validator ("validator" key must exist). func mempoolLogger() log.Logger { @@ -107,6 +115,11 @@ func TestReactorBroadcastTxMessage(t *testing.T) { r.Stop() } }() + for _, r := range reactors { + for _, peer := range r.Switch.Peers().List() { + peer.Set(types.PeerStateKey, peerState{1}) + } + } // send a bunch of txs to the first reactor's mempool // and wait for them all to be received in the others diff --git a/node/node.go b/node/node.go index 0652a392f..796bbc2a8 100644 --- a/node/node.go +++ b/node/node.go @@ -13,8 +13,9 @@ import ( "github.com/prometheus/client_golang/prometheus" "github.com/prometheus/client_golang/prometheus/promhttp" + "github.com/rs/cors" - amino "github.com/tendermint/go-amino" + "github.com/tendermint/go-amino" abci "github.com/tendermint/tendermint/abci/types" bc "github.com/tendermint/tendermint/blockchain" cfg "github.com/tendermint/tendermint/config" @@ -651,9 +652,20 @@ func (n *Node) startRPC() ([]net.Listener, error) { wm.SetLogger(rpcLogger.With("protocol", "websocket")) mux.HandleFunc("/websocket", wm.WebsocketHandler) rpcserver.RegisterRPCFuncs(mux, rpccore.Routes, coreCodec, rpcLogger) + + var rootHandler http.Handler = mux + if n.config.RPC.IsCorsEnabled() { + corsMiddleware := cors.New(cors.Options{ + AllowedOrigins: n.config.RPC.CORSAllowedOrigins, + AllowedMethods: n.config.RPC.CORSAllowedMethods, + AllowedHeaders: n.config.RPC.CORSAllowedHeaders, + }) + rootHandler = corsMiddleware.Handler(mux) + } + listener, err := rpcserver.StartHTTPServer( listenAddr, - mux, + rootHandler, rpcLogger, rpcserver.Config{MaxOpenConnections: n.config.RPC.MaxOpenConnections}, ) diff --git a/p2p/pex/addrbook.go b/p2p/pex/addrbook.go index 405a4628b..e2fcc0436 100644 --- a/p2p/pex/addrbook.go +++ b/p2p/pex/addrbook.go @@ -162,10 +162,10 @@ func (a *addrBook) FilePath() string { // AddOurAddress one of our addresses. func (a *addrBook) AddOurAddress(addr *p2p.NetAddress) { - a.mtx.Lock() - defer a.mtx.Unlock() a.Logger.Info("Add our address to book", "addr", addr) + a.mtx.Lock() a.ourAddrs[addr.String()] = struct{}{} + a.mtx.Unlock() } // OurAddress returns true if it is our address. @@ -178,10 +178,10 @@ func (a *addrBook) OurAddress(addr *p2p.NetAddress) bool { func (a *addrBook) AddPrivateIDs(IDs []string) { a.mtx.Lock() - defer a.mtx.Unlock() for _, id := range IDs { a.privateIDs[p2p.ID(id)] = struct{}{} } + a.mtx.Unlock() } // AddAddress implements AddrBook @@ -202,7 +202,7 @@ func (a *addrBook) RemoveAddress(addr *p2p.NetAddress) { if ka == nil { return } - a.Logger.Info("Remove address from book", "addr", ka.Addr, "ID", ka.ID()) + a.Logger.Info("Remove address from book", "addr", addr) a.removeFromAllBuckets(ka) } @@ -217,8 +217,8 @@ func (a *addrBook) IsGood(addr *p2p.NetAddress) bool { // HasAddress returns true if the address is in the book. func (a *addrBook) HasAddress(addr *p2p.NetAddress) bool { a.mtx.Lock() - defer a.mtx.Unlock() ka := a.addrLookup[addr.ID] + a.mtx.Unlock() return ka != nil } @@ -461,13 +461,12 @@ ADDRS_LOOP: // ListOfKnownAddresses returns the new and old addresses. func (a *addrBook) ListOfKnownAddresses() []*knownAddress { - a.mtx.Lock() - defer a.mtx.Unlock() - addrs := []*knownAddress{} + a.mtx.Lock() for _, addr := range a.addrLookup { addrs = append(addrs, addr.copy()) } + a.mtx.Unlock() return addrs } diff --git a/rpc/client/helpers.go b/rpc/client/helpers.go index 7e64d1164..2e80a3063 100644 --- a/rpc/client/helpers.go +++ b/rpc/client/helpers.go @@ -69,7 +69,18 @@ func WaitForOneEvent(c EventsClient, evtTyp string, timeout time.Duration) (type } // make sure to unregister after the test is over - defer c.UnsubscribeAll(ctx, subscriber) + defer func() { + // drain evts to make sure we don't block + LOOP: + for { + select { + case <-evts: + default: + break LOOP + } + } + c.UnsubscribeAll(ctx, subscriber) + }() select { case evt := <-evts: diff --git a/rpc/client/rpc_test.go b/rpc/client/rpc_test.go index 602525b51..217971fda 100644 --- a/rpc/client/rpc_test.go +++ b/rpc/client/rpc_test.go @@ -2,6 +2,7 @@ package client_test import ( "fmt" + "net/http" "strings" "testing" @@ -11,7 +12,7 @@ import ( abci "github.com/tendermint/tendermint/abci/types" "github.com/tendermint/tendermint/rpc/client" - rpctest "github.com/tendermint/tendermint/rpc/test" + "github.com/tendermint/tendermint/rpc/test" "github.com/tendermint/tendermint/types" ) @@ -32,6 +33,21 @@ func GetClients() []client.Client { } } +func TestCorsEnabled(t *testing.T) { + origin := rpctest.GetConfig().RPC.CORSAllowedOrigins[0] + remote := strings.Replace(rpctest.GetConfig().RPC.ListenAddress, "tcp", "http", -1) + + req, err := http.NewRequest("GET", remote, nil) + require.Nil(t, err, "%+v", err) + req.Header.Set("Origin", origin) + c := &http.Client{} + resp, err := c.Do(req) + defer resp.Body.Close() + + require.Nil(t, err, "%+v", err) + assert.Equal(t, resp.Header.Get("Access-Control-Allow-Origin"), origin) +} + // Make sure status is correct (we connect properly) func TestStatus(t *testing.T) { for i, c := range GetClients() { diff --git a/rpc/core/consensus.go b/rpc/core/consensus.go index 1c2619d5c..146628859 100644 --- a/rpc/core/consensus.go +++ b/rpc/core/consensus.go @@ -193,7 +193,10 @@ func DumpConsensusState() (*ctypes.ResultDumpConsensusState, error) { peers := p2pPeers.Peers().List() peerStates := make([]ctypes.PeerStateInfo, len(peers)) for i, peer := range peers { - peerState := peer.Get(types.PeerStateKey).(*cm.PeerState) + peerState, ok := peer.Get(types.PeerStateKey).(*cm.PeerState) + if !ok { // peer does not have a state yet + continue + } peerStateJSON, err := peerState.ToJSON() if err != nil { return nil, err diff --git a/rpc/core/doc.go b/rpc/core/doc.go index 5378dde24..ec79c8e17 100644 --- a/rpc/core/doc.go +++ b/rpc/core/doc.go @@ -12,7 +12,10 @@ See it here: https://github.com/tendermint/tendermint/tree/master/rpc/lib ## Configuration -Set the `laddr` config parameter under `[rpc]` table in the `$TMHOME/config/config.toml` file or the `--rpc.laddr` command-line flag to the desired protocol://host:port setting. Default: `tcp://0.0.0.0:26657`. +RPC can be configured by tuning parameters under `[rpc]` table in the `$TMHOME/config/config.toml` file or by using the `--rpc.X` command-line flags. + +Default rpc listen address is `tcp://0.0.0.0:26657`. To set another address, set the `laddr` config parameter to desired value. +CORS (Cross-Origin Resource Sharing) can be enabled by setting `cors_allowed_origins`, `cors_allowed_methods`, `cors_allowed_headers` config parameters. ## Arguments diff --git a/rpc/core/mempool.go b/rpc/core/mempool.go index c015363af..598774921 100644 --- a/rpc/core/mempool.go +++ b/rpc/core/mempool.go @@ -8,7 +8,6 @@ import ( "github.com/pkg/errors" abci "github.com/tendermint/tendermint/abci/types" - cmn "github.com/tendermint/tendermint/libs/common" ctypes "github.com/tendermint/tendermint/rpc/core/types" "github.com/tendermint/tendermint/types" ) @@ -51,7 +50,7 @@ import ( func BroadcastTxAsync(tx types.Tx) (*ctypes.ResultBroadcastTx, error) { err := mempool.CheckTx(tx, nil) if err != nil { - return nil, fmt.Errorf("Error broadcasting transaction: %v", err) + return nil, err } return &ctypes.ResultBroadcastTx{Hash: tx.Hash()}, nil } @@ -94,7 +93,7 @@ func BroadcastTxSync(tx types.Tx) (*ctypes.ResultBroadcastTx, error) { resCh <- res }) if err != nil { - return nil, fmt.Errorf("Error broadcasting transaction: %v", err) + return nil, err } res := <-resCh r := res.GetCheckTx() @@ -106,8 +105,9 @@ func BroadcastTxSync(tx types.Tx) (*ctypes.ResultBroadcastTx, error) { }, nil } -// CONTRACT: only returns error if mempool.BroadcastTx errs (ie. problem with the app) -// or if we timeout waiting for tx to commit. +// CONTRACT: only returns error if mempool.CheckTx() errs or if we timeout +// waiting for tx to commit. +// // If CheckTx or DeliverTx fail, no error will be returned, but the returned result // will contain a non-OK ABCI code. // @@ -150,20 +150,31 @@ func BroadcastTxSync(tx types.Tx) (*ctypes.ResultBroadcastTx, error) { // |-----------+------+---------+----------+-----------------| // | tx | Tx | nil | true | The transaction | func BroadcastTxCommit(tx types.Tx) (*ctypes.ResultBroadcastTxCommit, error) { - // subscribe to tx being committed in block + // Subscribe to tx being committed in block. ctx, cancel := context.WithTimeout(context.Background(), subscribeTimeout) defer cancel() - deliverTxResCh := make(chan interface{}) + deliverTxResCh := make(chan interface{}, 1) q := types.EventQueryTxFor(tx) err := eventBus.Subscribe(ctx, "mempool", q, deliverTxResCh) if err != nil { err = errors.Wrap(err, "failed to subscribe to tx") - logger.Error("Error on broadcastTxCommit", "err", err) - return nil, fmt.Errorf("Error on broadcastTxCommit: %v", err) + logger.Error("Error on broadcast_tx_commit", "err", err) + return nil, err } - defer eventBus.Unsubscribe(context.Background(), "mempool", q) + defer func() { + // drain deliverTxResCh to make sure we don't block + LOOP: + for { + select { + case <-deliverTxResCh: + default: + break LOOP + } + } + eventBus.Unsubscribe(context.Background(), "mempool", q) + }() - // broadcast the tx and register checktx callback + // Broadcast tx and wait for CheckTx result checkTxResCh := make(chan *abci.Response, 1) err = mempool.CheckTx(tx, func(res *abci.Response) { checkTxResCh <- res @@ -172,40 +183,35 @@ func BroadcastTxCommit(tx types.Tx) (*ctypes.ResultBroadcastTxCommit, error) { logger.Error("Error on broadcastTxCommit", "err", err) return nil, fmt.Errorf("Error on broadcastTxCommit: %v", err) } - checkTxRes := <-checkTxResCh - checkTxR := checkTxRes.GetCheckTx() - if checkTxR.Code != abci.CodeTypeOK { - // CheckTx failed! + checkTxResMsg := <-checkTxResCh + checkTxRes := checkTxResMsg.GetCheckTx() + if checkTxRes.Code != abci.CodeTypeOK { return &ctypes.ResultBroadcastTxCommit{ - CheckTx: *checkTxR, + CheckTx: *checkTxRes, DeliverTx: abci.ResponseDeliverTx{}, Hash: tx.Hash(), }, nil } - // Wait for the tx to be included in a block, - // timeout after something reasonable. - // TODO: configurable? - timer := time.NewTimer(60 * 2 * time.Second) + // Wait for the tx to be included in a block or timeout. + var deliverTxTimeout = 10 * time.Second // TODO: configurable? select { - case deliverTxResMsg := <-deliverTxResCh: + case deliverTxResMsg := <-deliverTxResCh: // The tx was included in a block. deliverTxRes := deliverTxResMsg.(types.EventDataTx) - // The tx was included in a block. - deliverTxR := deliverTxRes.Result - logger.Info("DeliverTx passed ", "tx", cmn.HexBytes(tx), "response", deliverTxR) return &ctypes.ResultBroadcastTxCommit{ - CheckTx: *checkTxR, - DeliverTx: deliverTxR, + CheckTx: *checkTxRes, + DeliverTx: deliverTxRes.Result, Hash: tx.Hash(), Height: deliverTxRes.Height, }, nil - case <-timer.C: - logger.Error("failed to include tx") + case <-time.After(deliverTxTimeout): + err = errors.New("Timed out waiting for tx to be included in a block") + logger.Error("Error on broadcastTxCommit", "err", err) return &ctypes.ResultBroadcastTxCommit{ - CheckTx: *checkTxR, + CheckTx: *checkTxRes, DeliverTx: abci.ResponseDeliverTx{}, Hash: tx.Hash(), - }, fmt.Errorf("Timed out waiting for transaction to be included in a block") + }, err } } diff --git a/rpc/lib/server/http_server.go b/rpc/lib/server/http_server.go index 6de376c29..8cacaeefb 100644 --- a/rpc/lib/server/http_server.go +++ b/rpc/lib/server/http_server.go @@ -151,11 +151,6 @@ func RecoverAndLogHandler(handler http.Handler, logger log.Logger) http.Handler rww := &ResponseWriterWrapper{-1, w} begin := time.Now() - // Common headers - origin := r.Header.Get("Origin") - rww.Header().Set("Access-Control-Allow-Origin", origin) - rww.Header().Set("Access-Control-Allow-Credentials", "true") - rww.Header().Set("Access-Control-Expose-Headers", "X-Server-Time") rww.Header().Set("X-Server-Time", fmt.Sprintf("%v", begin.Unix())) defer func() { diff --git a/rpc/test/helpers.go b/rpc/test/helpers.go index 0a9cd9847..e68ec1490 100644 --- a/rpc/test/helpers.go +++ b/rpc/test/helpers.go @@ -84,6 +84,7 @@ func GetConfig() *cfg.Config { tm, rpc, grpc := makeAddrs() globalConfig.P2P.ListenAddress = tm globalConfig.RPC.ListenAddress = rpc + globalConfig.RPC.CORSAllowedOrigins = []string{"https://tendermint.com/"} globalConfig.RPC.GRPCListenAddress = grpc globalConfig.TxIndex.IndexTags = "app.creator,tx.height" // see kvstore application } diff --git a/scripts/install/install_tendermint_arm.sh b/scripts/install/install_tendermint_arm.sh new file mode 100644 index 000000000..2e8d50aef --- /dev/null +++ b/scripts/install/install_tendermint_arm.sh @@ -0,0 +1,48 @@ +#!/usr/bin/env bash + +# XXX: this script is intended to be run from +# a fresh Digital Ocean droplet with Ubuntu + +# upon its completion, you must either reset +# your terminal or run `source ~/.profile` + +# as written, this script will install +# tendermint core from master branch +REPO=github.com/tendermint/tendermint + +# change this to a specific release or branch +BRANCH=master + +GO_VERSION=1.11.2 + +sudo apt-get update -y + +# get and unpack golang +curl -O https://storage.googleapis.com/golang/go$GO_VERSION.linux-armv6l.tar.gz +tar -xvf go$GO_VERSION.linux-armv6l.tar.gz + +# move go folder and add go binary to path +sudo mv go /usr/local +echo "export PATH=\$PATH:/usr/local/go/bin" >> ~/.profile + +# create the go directory, set GOPATH, and put it on PATH +mkdir go +echo "export GOPATH=$HOME/go" >> ~/.profile +echo "export PATH=\$PATH:\$GOPATH/bin" >> ~/.profile +source ~/.profile + +# get the code and move into repo +go get $REPO +cd "$GOPATH/src/$REPO" + +# build & install +git checkout $BRANCH +# XXX: uncomment if branch isn't master +# git fetch origin $BRANCH +make get_tools +make get_vendor_deps +make install + +# the binary is located in $GOPATH/bin +# run `source ~/.profile` or reset your terminal +# to persist the changes diff --git a/scripts/install/install_tendermint_bsd.sh b/scripts/install/install_tendermint_bsd.sh index 5b30eab31..0f7ef9b5e 100644 --- a/scripts/install/install_tendermint_bsd.sh +++ b/scripts/install/install_tendermint_bsd.sh @@ -14,6 +14,9 @@ # change this to a specific release or branch set BRANCH=master +set REPO=github.com/tendermint/tendermint + +set GO_VERSION=1.11.2 sudo pkg update @@ -21,8 +24,8 @@ sudo pkg install -y gmake sudo pkg install -y git # get and unpack golang -curl -O https://storage.googleapis.com/golang/go1.11.freebsd-amd64.tar.gz -tar -xvf go1.11.freebsd-amd64.tar.gz +curl -O https://storage.googleapis.com/golang/go$GO_VERSION.freebsd-amd64.tar.gz +tar -xvf go$GO_VERSION.freebsd-amd64.tar.gz # move go folder and add go binary to path sudo mv go /usr/local @@ -38,7 +41,6 @@ echo "set path=($path $GOPATH/bin)" >> ~/.tcshrc source ~/.tcshrc # get the code and move into repo -set REPO=github.com/tendermint/tendermint go get $REPO cd "$GOPATH/src/$REPO" diff --git a/scripts/install/install_tendermint_ubuntu.sh b/scripts/install/install_tendermint_ubuntu.sh index 29e975088..91ca1598d 100644 --- a/scripts/install/install_tendermint_ubuntu.sh +++ b/scripts/install/install_tendermint_ubuntu.sh @@ -13,20 +13,22 @@ REPO=github.com/tendermint/tendermint # change this to a specific release or branch BRANCH=master +GO_VERSION=1.11.2 + sudo apt-get update -y sudo apt-get install -y make # get and unpack golang -curl -O https://storage.googleapis.com/golang/go1.11.linux-amd64.tar.gz -tar -xvf go1.11.linux-amd64.tar.gz +curl -O https://storage.googleapis.com/golang/go$GO_VERSION.linux-amd64.tar.gz +tar -xvf go$GO_VERSION.linux-amd64.tar.gz # move go folder and add go binary to path sudo mv go /usr/local echo "export PATH=\$PATH:/usr/local/go/bin" >> ~/.profile -# create the goApps directory, set GOPATH, and put it on PATH -mkdir goApps -echo "export GOPATH=$HOME/goApps" >> ~/.profile +# create the go directory, set GOPATH, and put it on PATH +mkdir go +echo "export GOPATH=$HOME/go" >> ~/.profile echo "export PATH=\$PATH:\$GOPATH/bin" >> ~/.profile source ~/.profile diff --git a/state/store.go b/state/store.go index 7a0ef255a..086dcdf5a 100644 --- a/state/store.go +++ b/state/store.go @@ -96,7 +96,7 @@ func saveState(db dbm.DB, state State, key []byte) { saveValidatorsInfo(db, nextHeight+1, state.LastHeightValidatorsChanged, state.NextValidators) // Save next consensus params. saveConsensusParamsInfo(db, nextHeight, state.LastHeightConsensusParamsChanged, state.ConsensusParams) - db.SetSync(stateKey, state.Bytes()) + db.SetSync(key, state.Bytes()) } //------------------------------------------------------------------------ diff --git a/types/genesis.go b/types/genesis.go index 8684eb33b..54b81e9e2 100644 --- a/types/genesis.go +++ b/types/genesis.go @@ -19,6 +19,9 @@ const ( //------------------------------------------------------------ // core types for a genesis definition +// NOTE: any changes to the genesis definition should +// be reflected in the documentation: +// docs/tendermint-core/using-tendermint.md // GenesisValidator is an initial validator. type GenesisValidator struct { diff --git a/version/version.go b/version/version.go index b7a72a7f7..aae545129 100644 --- a/version/version.go +++ b/version/version.go @@ -18,7 +18,7 @@ const ( // TMCoreSemVer is the current version of Tendermint Core. // It's the Semantic Version of the software. // Must be a string because scripts like dist.sh read this file. - TMCoreSemVer = "0.26.1" + TMCoreSemVer = "0.26.2" // ABCISemVer is the semantic version of the ABCI library ABCISemVer = "0.15.0"