Release/v0.29.2pull/3282/head v0.29.2
@ -0,0 +1,61 @@ | |||
run: | |||
deadline: 1m | |||
linters: | |||
enable-all: true | |||
disable: | |||
- gocyclo | |||
- golint | |||
- maligned | |||
- errcheck | |||
- staticcheck | |||
- dupl | |||
- ineffassign | |||
- interfacer | |||
- unconvert | |||
- goconst | |||
- unparam | |||
- nakedret | |||
- lll | |||
- gochecknoglobals | |||
- govet | |||
- gocritic | |||
- gosec | |||
- gochecknoinits | |||
- scopelint | |||
- stylecheck | |||
# linters-settings: | |||
# govet: | |||
# check-shadowing: true | |||
# golint: | |||
# min-confidence: 0 | |||
# gocyclo: | |||
# min-complexity: 10 | |||
# maligned: | |||
# suggest-new: true | |||
# dupl: | |||
# threshold: 100 | |||
# goconst: | |||
# min-len: 2 | |||
# min-occurrences: 2 | |||
# depguard: | |||
# list-type: blacklist | |||
# packages: | |||
# # logging is allowed only by logutils.Log, logrus | |||
# # is allowed to use only in logutils package | |||
# - github.com/sirupsen/logrus | |||
# misspell: | |||
# locale: US | |||
# lll: | |||
# line-length: 140 | |||
# goimports: | |||
# local-prefixes: github.com/golangci/golangci-lint | |||
# gocritic: | |||
# enabled-tags: | |||
# - performance | |||
# - style | |||
# - experimental | |||
# disabled-checks: | |||
# - wrapperFunc | |||
# - commentFormatting # https://github.com/go-critic/go-critic/issues/755 |
@ -0,0 +1,158 @@ | |||
## Design goals | |||
The design goals for Tendermint (and the SDK and related libraries) are: | |||
* Simplicity and Legibility | |||
* Parallel performance, namely ability to utilize multicore architecture | |||
* Ability to evolve the codebase bug-free | |||
* Debuggability | |||
* Complete correctness that considers all edge cases, esp in concurrency | |||
* Future-proof modular architecture, message protocol, APIs, and encapsulation | |||
### Justification | |||
Legibility is key to maintaining bug-free software as it evolves toward more | |||
optimizations, more ease of debugging, and additional features. | |||
It is too easy to introduce bugs over time by replacing lines of code with | |||
those that may panic, which means ideally locks are unlocked by defer | |||
statements. | |||
For example, | |||
```go | |||
func (obj *MyObj) something() { | |||
mtx.Lock() | |||
obj.something = other | |||
mtx.Unlock() | |||
} | |||
``` | |||
It is too easy to refactor the codebase in the future to replace `other` with | |||
`other.String()` for example, and this may introduce a bug that causes a | |||
deadlock. So as much as reasonably possible, we need to be using defer | |||
statements, even though it introduces additional overhead. | |||
If it is necessary to optimize the unlocking of mutex locks, the solution is | |||
more modularity via smaller functions, so that defer'd unlocks are scoped | |||
within a smaller function. | |||
Similarly, idiomatic for-loops should always be preferred over those that use | |||
custom counters, because it is too easy to evolve the body of a for-loop to | |||
become more complicated over time, and it becomes more and more difficult to | |||
assess the correctness of such a for-loop by visual inspection. | |||
### On performance | |||
It doesn't matter whether there are alternative implementations that are 2x or | |||
3x more performant, when the software doesn't work, deadlocks, or if bugs | |||
cannot be debugged. By taking advantage of multicore concurrency, the | |||
Tendermint implementation will at least be an order of magnitude within the | |||
range of what is theoretically possible. The design philosophy of Tendermint, | |||
and the choice of Go as implementation language, is designed to make Tendermint | |||
implementation the standard specification for concurrent BFT software. | |||
By focusing on the message protocols (e.g. ABCI, p2p messages), and | |||
encapsulation e.g. IAVL module, (relatively) independent reactors, we are both | |||
implementing a standard implementation to be used as the specification for | |||
future implementations in more optimizable languages like Rust, Java, and C++; | |||
as well as creating sufficiently performant software. Tendermint Core will | |||
never be as fast as future implementations of the Tendermint Spec, because Go | |||
isn't designed to be as fast as possible. The advantage of using Go is that we | |||
can develop the whole stack of modular components **faster** than in other | |||
languages. | |||
Furthermore, the real bottleneck is in the application layer, and it isn't | |||
necessary to support more than a sufficiently decentralized set of validators | |||
(e.g. 100 ~ 300 validators is sufficient, with delegated bonded PoS). | |||
Instead of optimizing Tendermint performance down to the metal, lets focus on | |||
optimizing on other matters, namely ability to push feature complete software | |||
that works well enough, can be debugged and maintained, and can serve as a spec | |||
for future implementations. | |||
### On encapsulation | |||
In order to create maintainable, forward-optimizable software, it is critical | |||
to develop well-encapsulated objects that have well understood properties, and | |||
to re-use these easy-to-use-correctly components as building blocks for further | |||
encapsulated meta-objects. | |||
For example, mutexes are cheap enough for Tendermint's design goals when there | |||
isn't goroutine contention, so it is encouraged to create concurrency safe | |||
structures with struct-level mutexes. If they are used in the context of | |||
non-concurrent logic, then the performance is good enough. If they are used in | |||
the context of concurrent logic, then it will still perform correctly. | |||
Examples of this design principle can be seen in the types.ValidatorSet struct, | |||
and the cmn.Rand struct. It's one single struct declaration that can be used | |||
in both concurrent and non-concurrent logic, and due to its well encapsulation, | |||
it's easy to get the usage of the mutex right. | |||
#### example: cmn.Rand: | |||
`The default Source is safe for concurrent use by multiple goroutines, but | |||
Sources created by NewSource are not`. The reason why the default | |||
package-level source is safe for concurrent use is because it is protected (see | |||
`lockedSource` in https://golang.org/src/math/rand/rand.go). | |||
But we shouldn't rely on the global source, we should be creating our own | |||
Rand/Source instances and using them, especially for determinism in testing. | |||
So it is reasonable to have cmn.Rand be protected by a mutex. Whether we want | |||
our own implementation of Rand is another question, but the answer there is | |||
also in the affirmative. Sometimes you want to know where Rand is being used | |||
in your code, so it becomes a simple matter of dropping in a log statement to | |||
inject inspectability into Rand usage. Also, it is nice to be able to extend | |||
the functionality of Rand with custom methods. For these reasons, and for the | |||
reasons which is outlined in this design philosophy document, we should | |||
continue to use the cmn.Rand object, with mutex protection. | |||
Another key aspect of good encapsulation is the choice of exposed vs unexposed | |||
methods. It should be clear to the reader of the code, which methods are | |||
intended to be used in what context, and what safe usage is. Part of this is | |||
solved by hiding methods via unexported methods. Another part of this is | |||
naming conventions on the methods (e.g. underscores) with good documentation, | |||
and code organization. If there are too many exposed methods and it isn't | |||
clear what methods have what side effects, then there is something wrong about | |||
the design of abstractions that should be revisited. | |||
### On concurrency | |||
In order for Tendermint to remain relevant in the years to come, it is vital | |||
for Tendermint to take advantage of multicore architectures. Due to the nature | |||
of the problem, namely consensus across a concurrent p2p gossip network, and to | |||
handle RPC requests for a large number of consuming subscribers, it is | |||
unavoidable for Tendermint development to require expertise in concurrency | |||
design, especially when it comes to the reactor design, and also for RPC | |||
request handling. | |||
## Guidelines | |||
Here are some guidelines for designing for (sufficient) performance and concurrency: | |||
* Mutex locks are cheap enough when there isn't contention. | |||
* Do not optimize code without analytical or observed proof that it is in a hot path. | |||
* Don't over-use channels when mutex locks w/ encapsulation are sufficient. | |||
* The need to drain channels are often a hint of unconsidered edge cases. | |||
* The creation of O(N) one-off goroutines is generally technical debt that | |||
needs to get addressed sooner than later. Avoid creating too many | |||
goroutines as a patch around incomplete concurrency design, or at least be | |||
aware of the debt and do not invest in the debt. On the other hand, Tendermint | |||
is designed to have a limited number of peers (e.g. 10 or 20), so the creation | |||
of O(C) goroutines per O(P) peers is still O(C\*P=constant). | |||
* Use defer statements to unlock as much as possible. If you want to unlock sooner, | |||
try to create more modular functions that do make use of defer statements. | |||
## Matras | |||
* Premature optimization kills | |||
* Readability is paramount | |||
* Beautiful is better than fast. | |||
* In the face of ambiguity, refuse the temptation to guess. | |||
* In the face of bugs, refuse the temptation to cover the bug. | |||
* There should be one-- and preferably only one --obvious way to do it. |
@ -0,0 +1,24 @@ | |||
// +build libsecp256k1 | |||
package secp256k1 | |||
import ( | |||
"github.com/ethereum/go-ethereum/crypto/secp256k1" | |||
"github.com/tendermint/tendermint/crypto" | |||
) | |||
// Sign creates an ECDSA signature on curve Secp256k1, using SHA256 on the msg. | |||
func (privKey PrivKeySecp256k1) Sign(msg []byte) ([]byte, error) { | |||
rsv, err := secp256k1.Sign(crypto.Sha256(msg), privKey[:]) | |||
if err != nil { | |||
return nil, err | |||
} | |||
// we do not need v in r||s||v: | |||
rs := rsv[:len(rsv)-1] | |||
return rs, nil | |||
} | |||
func (pubKey PubKeySecp256k1) VerifyBytes(msg []byte, sig []byte) bool { | |||
return secp256k1.VerifySignature(pubKey[:], crypto.Sha256(msg), sig) | |||
} |
@ -0,0 +1,71 @@ | |||
// +build !libsecp256k1 | |||
package secp256k1 | |||
import ( | |||
"math/big" | |||
secp256k1 "github.com/btcsuite/btcd/btcec" | |||
"github.com/tendermint/tendermint/crypto" | |||
) | |||
// used to reject malleable signatures | |||
// see: | |||
// - https://github.com/ethereum/go-ethereum/blob/f9401ae011ddf7f8d2d95020b7446c17f8d98dc1/crypto/signature_nocgo.go#L90-L93 | |||
// - https://github.com/ethereum/go-ethereum/blob/f9401ae011ddf7f8d2d95020b7446c17f8d98dc1/crypto/crypto.go#L39 | |||
var secp256k1N, _ = new(big.Int).SetString("fffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141", 16) | |||
var secp256k1halfN = new(big.Int).Div(secp256k1N, big.NewInt(2)) | |||
// Sign creates an ECDSA signature on curve Secp256k1, using SHA256 on the msg. | |||
// The returned signature will be of the form R || S (in lower-S form). | |||
func (privKey PrivKeySecp256k1) Sign(msg []byte) ([]byte, error) { | |||
priv, _ := secp256k1.PrivKeyFromBytes(secp256k1.S256(), privKey[:]) | |||
sig, err := priv.Sign(crypto.Sha256(msg)) | |||
if err != nil { | |||
return nil, err | |||
} | |||
sigBytes := serializeSig(sig) | |||
return sigBytes, nil | |||
} | |||
// VerifyBytes verifies a signature of the form R || S. | |||
// It rejects signatures which are not in lower-S form. | |||
func (pubKey PubKeySecp256k1) VerifyBytes(msg []byte, sigStr []byte) bool { | |||
if len(sigStr) != 64 { | |||
return false | |||
} | |||
pub, err := secp256k1.ParsePubKey(pubKey[:], secp256k1.S256()) | |||
if err != nil { | |||
return false | |||
} | |||
// parse the signature: | |||
signature := signatureFromBytes(sigStr) | |||
// Reject malleable signatures. libsecp256k1 does this check but btcec doesn't. | |||
// see: https://github.com/ethereum/go-ethereum/blob/f9401ae011ddf7f8d2d95020b7446c17f8d98dc1/crypto/signature_nocgo.go#L90-L93 | |||
if signature.S.Cmp(secp256k1halfN) > 0 { | |||
return false | |||
} | |||
return signature.Verify(crypto.Sha256(msg), pub) | |||
} | |||
// Read Signature struct from R || S. Caller needs to ensure | |||
// that len(sigStr) == 64. | |||
func signatureFromBytes(sigStr []byte) *secp256k1.Signature { | |||
return &secp256k1.Signature{ | |||
new(big.Int).SetBytes(sigStr[:32]), | |||
new(big.Int).SetBytes(sigStr[32:64]), | |||
} | |||
} | |||
// Serialize signature to R || S. | |||
// R, S are padded to 32 bytes respectively. | |||
func serializeSig(sig *secp256k1.Signature) []byte { | |||
rBytes := sig.R.Bytes() | |||
sBytes := sig.S.Bytes() | |||
sigBytes := make([]byte, 64) | |||
// 0 pad the byte arrays from the left if they aren't big enough. | |||
copy(sigBytes[32-len(rBytes):32], rBytes) | |||
copy(sigBytes[64-len(sBytes):64], sBytes) | |||
return sigBytes | |||
} |
@ -0,0 +1,39 @@ | |||
// +build !libsecp256k1 | |||
package secp256k1 | |||
import ( | |||
"testing" | |||
secp256k1 "github.com/btcsuite/btcd/btcec" | |||
"github.com/stretchr/testify/require" | |||
) | |||
// Ensure that signature verification works, and that | |||
// non-canonical signatures fail. | |||
// Note: run with CGO_ENABLED=0 or go test -tags !cgo. | |||
func TestSignatureVerificationAndRejectUpperS(t *testing.T) { | |||
msg := []byte("We have lingered long enough on the shores of the cosmic ocean.") | |||
for i := 0; i < 500; i++ { | |||
priv := GenPrivKey() | |||
sigStr, err := priv.Sign(msg) | |||
require.NoError(t, err) | |||
sig := signatureFromBytes(sigStr) | |||
require.False(t, sig.S.Cmp(secp256k1halfN) > 0) | |||
pub := priv.PubKey() | |||
require.True(t, pub.VerifyBytes(msg, sigStr)) | |||
// malleate: | |||
sig.S.Sub(secp256k1.S256().CurveParams.N, sig.S) | |||
require.True(t, sig.S.Cmp(secp256k1halfN) > 0) | |||
malSigStr := serializeSig(sig) | |||
require.False(t, pub.VerifyBytes(msg, malSigStr), | |||
"VerifyBytes incorrect with malleated & invalid S. sig=%v, key=%v", | |||
sig, | |||
priv, | |||
) | |||
} | |||
} |
@ -1,4 +1,7 @@ | |||
# Overview | |||
Tendermint comes with some tools for [benchmarking](./benchmarking.md) | |||
and [monitoring](./monitoring.md). | |||
Tendermint comes with some tools for: | |||
* [Benchmarking](./benchmarking.md) | |||
* [Monitoring](./monitoring.md) | |||
* [Validation of remote signers](./remote-signer-validation.md) |
@ -0,0 +1,146 @@ | |||
# tm-signer-harness | |||
Located under the `tools/tm-signer-harness` folder in the [Tendermint | |||
repository](https://github.com/tendermint/tendermint). | |||
The Tendermint remote signer test harness facilitates integration testing | |||
between Tendermint and remote signers such as | |||
[KMS](https://github.com/tendermint/kms). Such remote signers allow for signing | |||
of important Tendermint messages using | |||
[HSMs](https://en.wikipedia.org/wiki/Hardware_security_module), providing | |||
additional security. | |||
When executed, `tm-signer-harness`: | |||
1. Runs a listener (either TCP or Unix sockets). | |||
2. Waits for a connection from the remote signer. | |||
3. Upon connection from the remote signer, executes a number of automated tests | |||
to ensure compatibility. | |||
4. Upon successful validation, the harness process exits with a 0 exit code. | |||
Upon validation failure, it exits with a particular exit code related to the | |||
error. | |||
## Prerequisites | |||
Requires the same prerequisites as for building | |||
[Tendermint](https://github.com/tendermint/tendermint). | |||
## Building | |||
From the `tools/tm-signer-harness` directory in your Tendermint source | |||
repository, simply run: | |||
```bash | |||
make | |||
# To have global access to this executable | |||
make install | |||
``` | |||
## Docker Image | |||
To build a Docker image containing the `tm-signer-harness`, also from the | |||
`tools/tm-signer-harness` directory of your Tendermint source repo, simply run: | |||
```bash | |||
make docker-image | |||
``` | |||
## Running against KMS | |||
As an example of how to use `tm-signer-harness`, the following instructions show | |||
you how to execute its tests against [KMS](https://github.com/tendermint/kms). | |||
For this example, we will make use of the **software signing module in KMS**, as | |||
the hardware signing module requires a physical | |||
[YubiHSM](https://www.yubico.com/products/yubihsm/) device. | |||
### Step 1: Install KMS on your local machine | |||
See the [KMS repo](https://github.com/tendermint/kms) for details on how to set | |||
KMS up on your local machine. | |||
If you have [Rust](https://www.rust-lang.org/) installed on your local machine, | |||
you can simply install KMS by: | |||
```bash | |||
cargo install tmkms | |||
``` | |||
### Step 2: Make keys for KMS | |||
The KMS software signing module needs a key with which to sign messages. In our | |||
example, we will simply export a signing key from our local Tendermint instance. | |||
```bash | |||
# Will generate all necessary Tendermint configuration files, including: | |||
# - ~/.tendermint/config/priv_validator_key.json | |||
# - ~/.tendermint/data/priv_validator_state.json | |||
tendermint init | |||
# Extract the signing key from our local Tendermint instance | |||
tm-signer-harness extract_key \ # Use the "extract_key" command | |||
-tmhome ~/.tendermint \ # Where to find the Tendermint home directory | |||
-output ./signing.key # Where to write the key | |||
``` | |||
Also, because we want KMS to connect to `tm-signer-harness`, we will need to | |||
provide a secret connection key from KMS' side: | |||
```bash | |||
tmkms keygen secret_connection.key | |||
``` | |||
### Step 3: Configure and run KMS | |||
KMS needs some configuration to tell it to use the softer signing module as well | |||
as the `signing.key` file we just generated. Save the following to a file called | |||
`tmkms.toml`: | |||
```toml | |||
[[validator]] | |||
addr = "tcp://127.0.0.1:61219" # This is where we will find tm-signer-harness. | |||
chain_id = "test-chain-0XwP5E" # The Tendermint chain ID for which KMS will be signing (found in ~/.tendermint/config/genesis.json). | |||
reconnect = true # true is the default | |||
secret_key = "./secret_connection.key" # Where to find our secret connection key. | |||
[[providers.softsign]] | |||
id = "test-chain-0XwP5E" # The Tendermint chain ID for which KMS will be signing (same as validator.chain_id above). | |||
path = "./signing.key" # The signing key we extracted earlier. | |||
``` | |||
Then run KMS with this configuration: | |||
```bash | |||
tmkms start -c tmkms.toml | |||
``` | |||
This will start KMS, which will repeatedly try to connect to | |||
`tcp://127.0.0.1:61219` until it is successful. | |||
### Step 4: Run tm-signer-harness | |||
Now we get to run the signer test harness: | |||
```bash | |||
tm-signer-harness run \ # The "run" command executes the tests | |||
-addr tcp://127.0.0.1:61219 \ # The address we promised KMS earlier | |||
-tmhome ~/.tendermint # Where to find our Tendermint configuration/data files. | |||
``` | |||
If the current version of Tendermint and KMS are compatible, `tm-signer-harness` | |||
should now exit with a 0 exit code. If they are somehow not compatible, it | |||
should exit with a meaningful non-zero exit code (see the exit codes below). | |||
### Step 5: Shut down KMS | |||
Simply hit Ctrl+Break on your KMS instance (or use the `kill` command in Linux) | |||
to terminate it gracefully. | |||
## Exit Code Meanings | |||
The following list shows the various exit codes from `tm-signer-harness` and | |||
their meanings: | |||
| Exit Code | Description | | |||
| --- | --- | | |||
| 0 | Success! | | |||
| 1 | Invalid command line parameters supplied to `tm-signer-harness` | | |||
| 2 | Maximum number of accept retries reached (the `-accept-retries` parameter) | | |||
| 3 | Failed to load `${TMHOME}/config/genesis.json` | | |||
| 4 | Failed to create listener specified by `-addr` parameter | | |||
| 5 | Failed to start listener | | |||
| 6 | Interrupted by `SIGINT` (e.g. when hitting Ctrl+Break or Ctrl+C) | | |||
| 7 | Other unknown error | | |||
| 8 | Test 1 failed: public key mismatch | | |||
| 9 | Test 2 failed: signing of proposals failed | | |||
| 10 | Test 3 failed: signing of votes failed | |