diff --git a/CHANGELOG.md b/CHANGELOG.md index 967fdef65..0b38873f5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,6 +24,12 @@ BUG FIXES: - Graceful handling/recovery for apps that have non-determinism or fail to halt - Graceful handling/recovery for violations of safety, or liveness +## 0.19.1 (TBD) + +BUG FIXES + +- [spec] Document address format and pubkey encoding + ## 0.19.0 (April 13th, 2018) BREAKING: @@ -42,6 +48,7 @@ genesis/priv_validator/node_key JSON files. FEATURES: - [cmd] added `gen_node_key` command + ## 0.18.0 (April 6th, 2018) BREAKING: diff --git a/README.md b/README.md index 1a4cc00cb..89c28e9c2 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,9 @@ _NOTE: This is alpha software. Please contact us if you intend to run it in prod Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state transition machine - written in any programming language - and securely replicates it on many machines. -For more information, from introduction to install to application development, [Read The Docs](https://tendermint.readthedocs.io/en/master/). +For more information, from introduction to installation and application development, [Read The Docs](https://tendermint.readthedocs.io/en/master/). + +For protocol details, see [the specification](./docs/specification/new-spec). ## Minimum requirements @@ -46,7 +48,8 @@ For more details (or if it fails), [read the docs](https://tendermint.readthedoc ### Tendermint Core -All resources involving the use of, building application on, or developing for, tendermint, can be found at [Read The Docs](https://tendermint.readthedocs.io/en/master/). Additional information about some - and eventually all - of the sub-projects below, can be found at Read The Docs. +To use Tendermint, build apps on it, or develop it, [Read The Docs](https://tendermint.readthedocs.io/en/master/). +Additional information about some - and eventually all - of the sub-projects below, can be found at Read The Docs. ### Sub-projects @@ -61,8 +64,8 @@ All resources involving the use of, building application on, or developing for, ### Applications -* [Ethermint](http://github.com/tendermint/ethermint); Ethereum on Tendermint * [Cosmos SDK](http://github.com/cosmos/cosmos-sdk); a cryptocurrency application framework +* [Ethermint](http://github.com/tendermint/ethermint); Ethereum on Tendermint * [Many more](https://tendermint.readthedocs.io/en/master/ecosystem.html#abci-applications) ### More diff --git a/docs/specification/new-spec/abci.md b/docs/specification/new-spec/abci.md index 75eed6b58..813aac787 100644 --- a/docs/specification/new-spec/abci.md +++ b/docs/specification/new-spec/abci.md @@ -33,6 +33,9 @@ For Ed25519 pubkeys, the Amino prefix is always "1624DE6220". For example, the 3 Amino encoded as `1624DE622076852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85` +(Note: in old versions of Tendermint (pre-v0.19.0), the pubkey is just prefixed with a +single type byte, so for ED25519 we'd have `pub_key = 0x1 | pub`) + The `power` is the new voting power for the validator, with the following rules: diff --git a/docs/specification/new-spec/pre-amino.md b/docs/specification/new-spec/pre-amino.md new file mode 100644 index 000000000..edddeff46 --- /dev/null +++ b/docs/specification/new-spec/pre-amino.md @@ -0,0 +1,246 @@ +# Tendermint Encoding (Pre-Amino) + +## PubKeys and Addresses + +PubKeys are prefixed with a type-byte, followed by the raw bytes of the public +key. + +Two keys are supported with the following type bytes: + +``` +TypeByteEd25519 = 0x1 +TypeByteSecp256k1 = 0x2 +``` + +``` +// TypeByte: 0x1 +type PubKeyEd25519 [32]byte + +func (pub PubKeyEd25519) Encode() []byte { + return 0x1 | pub +} + +func (pub PubKeyEd25519) Address() []byte { + // NOTE: the length (0x0120) is also included + return RIPEMD160(0x1 | 0x0120 | pub) +} + +// TypeByte: 0x2 +// NOTE: OpenSSL compressed pubkey (x-cord with 0x2 or 0x3) +type PubKeySecp256k1 [33]byte + +func (pub PubKeySecp256k1) Encode() []byte { + return 0x2 | pub +} + +func (pub PubKeySecp256k1) Address() []byte { + return RIPEMD160(SHA256(pub)) +} +``` + +See https://github.com/tendermint/go-crypto/blob/v0.5.0/pub_key.go for more. + +## Binary Serialization (go-wire) + +Tendermint aims to encode data structures in a manner similar to how the corresponding Go structs +are laid out in memory. +Variable length items are length-prefixed. +While the encoding was inspired by Go, it is easily implemented in other languages as well, given its intuitive design. + +XXX: This is changing to use real varints and 4-byte-prefixes. +See https://github.com/tendermint/go-wire/tree/sdk2. + +### Fixed Length Integers + +Fixed length integers are encoded in Big-Endian using the specified number of bytes. +So `uint8` and `int8` use one byte, `uint16` and `int16` use two bytes, +`uint32` and `int32` use 3 bytes, and `uint64` and `int64` use 4 bytes. + +Negative integers are encoded via twos-complement. + +Examples: + +```go +encode(uint8(6)) == [0x06] +encode(uint32(6)) == [0x00, 0x00, 0x00, 0x06] + +encode(int8(-6)) == [0xFA] +encode(int32(-6)) == [0xFF, 0xFF, 0xFF, 0xFA] +``` + +### Variable Length Integers + +Variable length integers are encoded as length-prefixed Big-Endian integers. +The length-prefix consists of a single byte and corresponds to the length of the encoded integer. + +Negative integers are encoded by flipping the leading bit of the length-prefix to a `1`. + +Zero is encoded as `0x00`. It is not length-prefixed. + +Examples: + +```go +encode(uint(6)) == [0x01, 0x06] +encode(uint(70000)) == [0x03, 0x01, 0x11, 0x70] + +encode(int(-6)) == [0xF1, 0x06] +encode(int(-70000)) == [0xF3, 0x01, 0x11, 0x70] + +encode(int(0)) == [0x00] +``` + +### Strings + +An encoded string is length-prefixed followed by the underlying bytes of the string. +The length-prefix is itself encoded as an `int`. + +The empty string is encoded as `0x00`. It is not length-prefixed. + +Examples: + +```go +encode("") == [0x00] +encode("a") == [0x01, 0x01, 0x61] +encode("hello") == [0x01, 0x05, 0x68, 0x65, 0x6C, 0x6C, 0x6F] +encode("¥") == [0x01, 0x02, 0xC2, 0xA5] +``` + +### Arrays (fixed length) + +An encoded fix-lengthed array is the concatenation of the encoding of its elements. +There is no length-prefix. + +Examples: + +```go +encode([4]int8{1, 2, 3, 4}) == [0x01, 0x02, 0x03, 0x04] +encode([4]int16{1, 2, 3, 4}) == [0x00, 0x01, 0x00, 0x02, 0x00, 0x03, 0x00, 0x04] +encode([4]int{1, 2, 3, 4}) == [0x01, 0x01, 0x01, 0x02, 0x01, 0x03, 0x01, 0x04] +encode([2]string{"abc", "efg"}) == [0x01, 0x03, 0x61, 0x62, 0x63, 0x01, 0x03, 0x65, 0x66, 0x67] +``` + +### Slices (variable length) + +An encoded variable-length array is length-prefixed followed by the concatenation of the encoding of +its elements. +The length-prefix is itself encoded as an `int`. + +An empty slice is encoded as `0x00`. It is not length-prefixed. + +Examples: + +```go +encode([]int8{}) == [0x00] +encode([]int8{1, 2, 3, 4}) == [0x01, 0x04, 0x01, 0x02, 0x03, 0x04] +encode([]int16{1, 2, 3, 4}) == [0x01, 0x04, 0x00, 0x01, 0x00, 0x02, 0x00, 0x03, 0x00, 0x04] +encode([]int{1, 2, 3, 4}) == [0x01, 0x04, 0x01, 0x01, 0x01, 0x02, 0x01, 0x03, 0x01, 0x4] +encode([]string{"abc", "efg"}) == [0x01, 0x02, 0x01, 0x03, 0x61, 0x62, 0x63, 0x01, 0x03, 0x65, 0x66, 0x67] +``` + +### BitArray + +BitArray is encoded as an `int` of the number of bits, and with an array of `uint64` to encode +value of each array element. + +```go +type BitArray struct { + Bits int + Elems []uint64 +} +``` + +### Time + +Time is encoded as an `int64` of the number of nanoseconds since January 1, 1970, +rounded to the nearest millisecond. + +Times before then are invalid. + +Examples: + +```go +encode(time.Time("Jan 1 00:00:00 UTC 1970")) == [0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00] +encode(time.Time("Jan 1 00:00:01 UTC 1970")) == [0x00, 0x00, 0x00, 0x00, 0x3B, 0x9A, 0xCA, 0x00] // 1,000,000,000 ns +encode(time.Time("Mon Jan 2 15:04:05 -0700 MST 2006")) == [0x0F, 0xC4, 0xBB, 0xC1, 0x53, 0x03, 0x12, 0x00] +``` + +### Structs + +An encoded struct is the concatenation of the encoding of its elements. +There is no length-prefix. + +Examples: + +```go +type MyStruct struct{ + A int + B string + C time.Time +} +encode(MyStruct{4, "hello", time.Time("Mon Jan 2 15:04:05 -0700 MST 2006")}) == + [0x01, 0x04, 0x01, 0x05, 0x68, 0x65, 0x6C, 0x6C, 0x6F, 0x0F, 0xC4, 0xBB, 0xC1, 0x53, 0x03, 0x12, 0x00] +``` + +## Merkle Trees + +Simple Merkle trees are used in numerous places in Tendermint to compute a cryptographic digest of a data structure. + +RIPEMD160 is always used as the hashing function. + +The function `SimpleMerkleRoot` is a simple recursive function defined as follows: + +```go +func SimpleMerkleRoot(hashes [][]byte) []byte{ + switch len(hashes) { + case 0: + return nil + case 1: + return hashes[0] + default: + left := SimpleMerkleRoot(hashes[:(len(hashes)+1)/2]) + right := SimpleMerkleRoot(hashes[(len(hashes)+1)/2:]) + return RIPEMD160(append(left, right)) + } +} +``` + +Note: we abuse notion and call `SimpleMerkleRoot` with arguments of type `struct` or type `[]struct`. +For `struct` arguments, we compute a `[][]byte` by sorting elements of the `struct` according to +field name and then hashing them. +For `[]struct` arguments, we compute a `[][]byte` by hashing the individual `struct` elements. + +## JSON (TMJSON) + +Signed messages (eg. votes, proposals) in the consensus are encoded in TMJSON, rather than TMBIN. +TMJSON is JSON where `[]byte` are encoded as uppercase hex, rather than base64. + +When signing, the elements of a message are sorted by key and the sorted message is embedded in an +outer JSON that includes a `chain_id` field. +We call this encoding the CanonicalSignBytes. For instance, CanonicalSignBytes for a vote would look +like: + +```json +{"chain_id":"my-chain-id","vote":{"block_id":{"hash":DEADBEEF,"parts":{"hash":BEEFDEAD,"total":3}},"height":3,"round":2,"timestamp":1234567890, "type":2} +``` + +Note how the fields within each level are sorted. + +## Other + +### MakeParts + +Encode an object using TMBIN and slice it into parts. + +```go +MakeParts(object, partSize) +``` + +### Part + +```go +type Part struct { + Index int + Bytes byte[] + Proof byte[] +} +``` diff --git a/docs/specification/new-spec/state.md b/docs/specification/new-spec/state.md index abd32edba..c7de007f3 100644 --- a/docs/specification/new-spec/state.md +++ b/docs/specification/new-spec/state.md @@ -74,9 +74,6 @@ func TotalVotingPower(vals []Validators) int64{ } ``` -### PubKey - -TODO: ### ConsensusParams diff --git a/version/version.go b/version/version.go index b0123eb1f..f07690271 100644 --- a/version/version.go +++ b/version/version.go @@ -1,8 +1,11 @@ package version -const Maj = "0" -const Min = "19" -const Fix = "0" +// Version components +const ( + Maj = "0" + Min = "19" + Fix = "0" +) var ( // Version is the current version of Tendermint