diff --git a/docs/specification/new-spec/README.md b/docs/specification/new-spec/README.md new file mode 100644 index 000000000..18ad457ec --- /dev/null +++ b/docs/specification/new-spec/README.md @@ -0,0 +1,17 @@ +# Tendermint Specification + +This is a markdown specification of the Tendermint blockchain. + +It defines the base data structures used in the blockchain and how they are validated. + +It contains the following components: + +- [Encoding and Digests](encoding.md) +- [Blockchain](blockchain.md) +- [State](state.md) + +TODO: + +- Light Client +- P2P +- Reactor protocols (consensus, mempool, blockchain, pex) diff --git a/docs/specification/new-spec/blockchain.md b/docs/specification/new-spec/blockchain.md index c99fc639b..de78b81df 100644 --- a/docs/specification/new-spec/blockchain.md +++ b/docs/specification/new-spec/blockchain.md @@ -68,9 +68,13 @@ parts. ``` type BlockID struct { - HeaderHash []byte - NumParts int32 - PartsHash []byte + Hash []byte + Parts PartsHeader +} + +type PartsHeader struct { + Hash []byte + Total int32 } ``` @@ -141,7 +145,7 @@ TODO # Validation -Here we describe the validation rules for every element in block. +Here we describe the validation rules for every element in a block. Blocks which do not satisfy these rules are considered invalid. We abuse notation by using something that looks like Go, supplemented with English. @@ -150,11 +154,10 @@ A statement such as `x == y` is an assertion - if it fails, the item is invalid. We refer to certain globally available objects: `block` is the block under consideration, `prevBlock` is the `block` at the previous height, -`state` keeps track of the validator set -and the consensus parameters as they are updated by the application, -and `app` is the set of responses returned by the application during the -execution of the `prevBlock`. Elements of an object are accessed as expected, -ie. `block.Header`. See the definitions of `state` and `app` elsewhere. +and `state` keeps track of the validator set, the consensus parameters +and other results from the application. +Elements of an object are accessed as expected, +ie. `block.Header`. See [here](state.md) for the definition of `state`. ## Header @@ -180,7 +183,7 @@ The height is an incrementing integer. The first block has `block.Header.Height ### Time The median of the timestamps of the valid votes in the block.LastCommit. -Corresponds to the number of milliseconds since January 1, 1970. +Corresponds to the number of nanoseconds, with millisecond resolution, since January 1, 1970. Increments with every new block. ### NumTxs @@ -223,11 +226,13 @@ The first block has `block.Header.TotalTxs = block.Header.NumberTxs`. ### LastBlockID ``` -parts := MakeBlockParts(block, state.ConsensusParams.BlockGossip.BlockPartSize) +parts := MakeParts(block, state.ConsensusParams.BlockGossip.BlockPartSize) block.HeaderLastBlockID == BlockID{ SimpleMerkleRoot(block.Header), - len(parts), - SimpleMerkleRoot(parts), + PartsHeader{ + SimpleMerkleRoot(parts), + len(parts), + }, } ``` @@ -239,7 +244,7 @@ The first block has `block.Header.LastBlockID == BlockID{}`. ### ResultsHash ``` -block.ResultsHash == SimpleMerkleRoot(app.Results) +block.ResultsHash == SimpleMerkleRoot(state.LastResults) ``` Simple Merkle root of the results of the transactions in the previous block. @@ -249,7 +254,7 @@ The first block has `block.Header.ResultsHash == []byte{}`. ### AppHash ``` -block.AppHash == app.AppHash +block.AppHash == state.AppHash ``` Arbitrary byte array returned by the application after executing and commiting the previous block. @@ -281,7 +286,7 @@ May be updated by the application. block.Header.Proposer in state.Validators ``` -Original proposer of the block. +Original proposer of the block. Must be a current validator. NOTE: this field can only be further verified by real-time participants in the consensus. This is because the same block can be proposed in multiple rounds for the same height @@ -323,13 +328,13 @@ for i, vote := range block.LastCommit{ vote.Round == block.LastCommit.Round() vote.BlockID == block.LastBlockID - pubkey, votingPower := state.LastValidators.Get(i) - VerifyVoteSignature(block.ChainID, vote, pubkey) + val := state.LastValidators[i] + vote.Verify(block.ChainID, val.PubKey) == true - talliedVotingPower += votingPower + talliedVotingPower += val.VotingPower } -talliedVotingPower > (2/3) * state.LastValidators.TotalVotingPower() +talliedVotingPower > (2/3) * TotalVotingPower(state.LastValidators) ``` Includes one (possibly nil) vote for every current validator. @@ -340,6 +345,24 @@ All votes must have a valid signature from the corresponding validator. The sum total of the voting power of the validators that voted must be greater than 2/3 of the total voting power of the complete validator set. +### Vote + +A vote is a signed message broadcast in the consensus for a particular block at a particular height and round. +When stored in the blockchain or propagated over the network, votes are encoded in TMBIN. +For signing, votes are encoded in JSON, and the ChainID is included, in the form of the `CanonicalSignBytes`. + +We define a method `Verify` that returns `true` if the signature verifies against the pubkey for the CanonicalSignBytes +using the given ChainID: + +``` +func (v Vote) Verify(chainID string, pubKey PubKey) bool { + return pubKey.Verify(v.Signature, CanonicalSignBytes(chainID, v)) +} +``` + +where `pubKey.Verify` performs the approprioate digital signature verification of the `pubKey` +against the given signature and message bytes. + ## Evidence @@ -352,3 +375,15 @@ Every piece of evidence contains two conflicting votes from a single validator t was active at the height indicated in the votes. The votes must not be too old. + +# Execution + +Once a block is validated, it can be executed against the state. + +The state follows the recursive equation: + +``` +app = NewABCIApp +state(1) = InitialState +state(h+1) <- Execute(state(h), app, block(h)) +``` diff --git a/docs/specification/new-spec/encoding.md b/docs/specification/new-spec/encoding.md index 584b0253e..a7482e6cb 100644 --- a/docs/specification/new-spec/encoding.md +++ b/docs/specification/new-spec/encoding.md @@ -1,6 +1,6 @@ # Tendermint Encoding -## Serialization +## Binary Serialization (TMBIN) Tendermint aims to encode data structures in a manner similar to how the corresponding Go structs are laid out in memory. Variable length items are length-prefixed. @@ -128,7 +128,7 @@ encode(MyStruct{4, "hello", time.Time("Mon Jan 2 15:04:05 -0700 MST 2006")}) == ## Merkle Trees -Merkle trees are used in numerous places in Tendermint to compute a cryptographic digest of a data structure. +Simple Merkle trees are used in numerous places in Tendermint to compute a cryptographic digest of a data structure. RIPEMD160 is always used as the hashing function. @@ -152,3 +152,27 @@ func SimpleMerkleRoot(hashes [][]byte) []byte{ Note we abuse notion and call `SimpleMerkleRoot` with arguments of type `struct` or type `[]struct`. For `struct` arguments, we compute a `[][]byte` by sorting elements of the `struct` according to field name and then hashing them. For `[]struct` arguments, we compute a `[][]byte` by hashing the individual `struct` elements. + +## JSON (TMJSON) + +Signed messages (eg. votes, proposals) in the consensus are encoded in TMJSON, rather than TMBIN. +TMJSON is JSON where `[]byte` are encoded as uppercase hex, rather than base64. + +When signing, the elements of a message are sorted by key and the sorted message is embedded in an outer JSON that includes a `chain_id` field. +We call this encoding the CanonicalSignBytes. For instance, CanonicalSignBytes for a vote would look like: + +``` +{"chain_id":"my-chain-id","vote":{"block_id":{"hash":DEADBEEF,"parts":{"hash":BEEFDEAD,"total":3}},"height":3,"round":2,"timestamp":1234567890, "type":2} +``` + +Note how the fields within each level are sorted. + +## Other + +### MakeParts + +TMBIN encode an object and slice it into parts. + +``` +MakeParts(object, partSize) +``` diff --git a/docs/specification/new-spec/state.md b/docs/specification/new-spec/state.md new file mode 100644 index 000000000..1d7900273 --- /dev/null +++ b/docs/specification/new-spec/state.md @@ -0,0 +1,104 @@ +# Tendermint State + +## State + +The state contains information whose cryptographic digest is included in block headers, +and thus is necessary for validating new blocks. +For instance, the Merkle root of the results from executing the previous block, or the Merkle root of the current validators. +While neither the results of transactions now the validators are ever included in the blockchain itself, +the Merkle roots are, and hence we need a separate data structure to track them. + +``` +type State struct { + LastResults []Result + AppHash []byte + + Validators []Validator + LastValidators []Validator + + ConsensusParams ConsensusParams +} +``` + +### Result + +``` +type Result struct { + Code uint32 + Data []byte + Tags []KVPair +} + +type KVPair struct { + Key []byte + Value []byte +} +``` + +`Result` is the result of executing a transaction against the application. +It returns a result code, an arbitrary byte array (ie. a return value), +and a list of key-value pairs ordered by key. The key-value pairs, or tags, +can be used to index transactions according to their "effects", which are +represented in the tags. + +### Validator + +A validator is an active participant in the consensus with a public key and a voting power. +Validator's also contain an address which is derived from the PubKey: + +``` +type Validator struct { + Address []byte + PubKey PubKey + VotingPower int64 +} +``` + +The `state.Validators` and `state.LastValidators` must always by sorted by validator address, +so that there is a canonical order for computing the SimpleMerkleRoot. + +We also define a `TotalVotingPower` function, to return the total voting power: + +``` +func TotalVotingPower(vals []Validators) int64{ + sum := 0 + for v := range vals{ + sum += v.VotingPower + } + return sum +} +``` + +### PubKey + +TODO: + +### ConsensusParams + +TODO: + +## Execution + +We define an `Execute` function that takes a state and a block, +executes the block against the application, and returns an updated state. + +``` +Execute(s State, app ABCIApp, block Block) State { + abciResponses := app.ApplyBlock(block) + + return State{ + LastResults: abciResponses.DeliverTxResults, + AppHash: abciResponses.AppHash, + Validators: UpdateValidators(state.Validators, abciResponses.ValidatorChanges), + LastValidators: state.Validators, + ConsensusParams: UpdateConsensusParams(state.ConsensusParams, abci.Responses.ConsensusParamChanges), + } +} + +type ABCIResponses struct { + DeliverTxResults []Result + ValidatorChanges []Validator + ConsensusParamChanges ConsensusParams + AppHash []byte +} +```