state

docs/specification/new-spec/README.md

@@ -0,0 +1,17 @@
+# Tendermint Specification
+
+This is a markdown specification of the Tendermint blockchain.
+
+It defines the base data structures used in the blockchain and how they are validated.
+
+It contains the following components:
+
+- [Encoding and Digests](encoding.md)
+- [Blockchain](blockchain.md)
+- [State](state.md)
+
+TODO:
+
+- Light Client
+- P2P
+- Reactor protocols (consensus, mempool, blockchain, pex)

docs/specification/new-spec/blockchain.md

@@ -68,9 +68,13 @@ parts.
 
 ```
 type BlockID struct {
-    HeaderHash  []byte
-    NumParts    int32
-    PartsHash   []byte
+    Hash []byte
+    Parts PartsHeader
+}
+
+type PartsHeader struct {
+    Hash []byte
+    Total int32
 }
 ```
 
@@ -141,7 +145,7 @@ TODO
 
 # Validation
 
-Here we describe the validation rules for every element in block.
+Here we describe the validation rules for every element in a block.
 Blocks which do not satisfy these rules are considered invalid.
 
 We abuse notation by using something that looks like Go, supplemented with English.
@@ -150,11 +154,10 @@ A statement such as `x == y` is an assertion - if it fails, the item is invalid.
 We refer to certain globally available objects:
 `block` is the block under consideration,
 `prevBlock` is the `block` at the previous height,
-`state` keeps track of the validator set
-and the consensus parameters as they are updated by the application,
-and `app` is the set of responses returned by the application during the
-execution of the `prevBlock`. Elements of an object are accessed as expected,
-ie. `block.Header`. See the definitions of `state` and `app` elsewhere.
+and `state` keeps track of the validator set, the consensus parameters
+and other results from the application.
+Elements of an object are accessed as expected,
+ie. `block.Header`. See [here](state.md) for the definition of `state`.
 
 ## Header
 
@@ -180,7 +183,7 @@ The height is an incrementing integer. The first block has `block.Header.Height
 ### Time
 
 The median of the timestamps of the valid votes in the block.LastCommit.
-Corresponds to the number of milliseconds since January 1, 1970.
+Corresponds to the number of nanoseconds, with millisecond resolution, since January 1, 1970.
 Increments with every new block.
 
 ### NumTxs
@@ -223,11 +226,13 @@ The first block has `block.Header.TotalTxs = block.Header.NumberTxs`.
 ### LastBlockID
 
 ```
-parts := MakeBlockParts(block, state.ConsensusParams.BlockGossip.BlockPartSize)
+parts := MakeParts(block, state.ConsensusParams.BlockGossip.BlockPartSize)
 block.HeaderLastBlockID == BlockID{
     SimpleMerkleRoot(block.Header),
-    len(parts),
-    SimpleMerkleRoot(parts),
+    PartsHeader{
+        SimpleMerkleRoot(parts),
+        len(parts),
+    },
 }
 ```
 
@@ -239,7 +244,7 @@ The first block has `block.Header.LastBlockID == BlockID{}`.
 ### ResultsHash
 
 ```
-block.ResultsHash == SimpleMerkleRoot(app.Results)
+block.ResultsHash == SimpleMerkleRoot(state.LastResults)
 ```
 
 Simple Merkle root of the results of the transactions in the previous block.
@@ -249,7 +254,7 @@ The first block has `block.Header.ResultsHash == []byte{}`.
 ### AppHash
 
 ```
-block.AppHash == app.AppHash
+block.AppHash == state.AppHash
 ```
 
 Arbitrary byte array returned by the application after executing and commiting the previous block.
@@ -281,7 +286,7 @@ May be updated by the application.
 block.Header.Proposer in state.Validators
 ```
 
-Original proposer of the block.
+Original proposer of the block. Must be a current validator.
 
 NOTE: this field can only be further verified by real-time participants in the consensus.
 This is because the same block can be proposed in multiple rounds for the same height
@@ -323,13 +328,13 @@ for i, vote := range block.LastCommit{
     vote.Round == block.LastCommit.Round()
     vote.BlockID == block.LastBlockID
 
-    pubkey, votingPower := state.LastValidators.Get(i)
-    VerifyVoteSignature(block.ChainID, vote, pubkey)
+    val := state.LastValidators[i]
+    vote.Verify(block.ChainID, val.PubKey) == true
 
-    talliedVotingPower += votingPower
+    talliedVotingPower += val.VotingPower
 }
 
-talliedVotingPower > (2/3) * state.LastValidators.TotalVotingPower()
+talliedVotingPower > (2/3) * TotalVotingPower(state.LastValidators)
 ```
 
 Includes one (possibly nil) vote for every current validator.
@@ -340,6 +345,24 @@ All votes must have a valid signature from the corresponding validator.
 The sum total of the voting power of the validators that voted
 must be greater than 2/3 of the total voting power of the complete validator set.
 
+### Vote
+
+A vote is a signed message broadcast in the consensus for a particular block at a particular height and round.
+When stored in the blockchain or propagated over the network, votes are encoded in TMBIN.
+For signing, votes are encoded in JSON, and the ChainID is included, in the form of the `CanonicalSignBytes`.
+
+We define a method `Verify` that returns `true` if the signature verifies against the pubkey for the CanonicalSignBytes
+using the given ChainID:
+
+```
+func (v Vote) Verify(chainID string, pubKey PubKey) bool {
+    return pubKey.Verify(v.Signature, CanonicalSignBytes(chainID, v))
+}
+```
+
+where `pubKey.Verify` performs the approprioate digital signature verification of the `pubKey`
+against the given signature and message bytes.
+
 ## Evidence
 
 
@@ -352,3 +375,15 @@ Every piece of evidence contains two conflicting votes from a single validator t
 was active at the height indicated in the votes.
 The votes must not be too old.
 
+
+# Execution
+
+Once a block is validated, it can be executed against the state.
+
+The state follows the recursive equation:
+
+```
+app = NewABCIApp
+state(1) = InitialState
+state(h+1) <- Execute(state(h), app, block(h))
+```

docs/specification/new-spec/encoding.md

@@ -1,6 +1,6 @@
 # Tendermint Encoding
 
-## Serialization
+## Binary Serialization (TMBIN)
 
 Tendermint aims to encode data structures in a manner similar to how the corresponding Go structs are laid out in memory.
 Variable length items are length-prefixed.
@@ -128,7 +128,7 @@ encode(MyStruct{4, "hello", time.Time("Mon Jan 2 15:04:05 -0700 MST 2006")}) ==
 
 ## Merkle Trees
 
-Merkle trees are used in numerous places in Tendermint to compute a cryptographic digest of a data structure.
+Simple Merkle trees are used in numerous places in Tendermint to compute a cryptographic digest of a data structure.
 
 RIPEMD160 is always used as the hashing function.
 
@@ -152,3 +152,27 @@ func SimpleMerkleRoot(hashes [][]byte) []byte{
 Note we abuse notion and call `SimpleMerkleRoot` with arguments of type `struct` or type `[]struct`.
 For `struct` arguments, we compute a `[][]byte` by sorting elements of the `struct` according to field name and then hashing them.
 For `[]struct` arguments, we compute a `[][]byte` by hashing the individual `struct` elements.
+
+## JSON (TMJSON)
+
+Signed messages (eg. votes, proposals) in the consensus are encoded in TMJSON, rather than TMBIN.
+TMJSON is JSON where `[]byte` are encoded as uppercase hex, rather than base64.
+
+When signing, the elements of a message are sorted by key and the sorted message is embedded in an outer JSON that includes a `chain_id` field.
+We call this encoding the CanonicalSignBytes. For instance, CanonicalSignBytes for a vote would look like:
+
+```
+{"chain_id":"my-chain-id","vote":{"block_id":{"hash":DEADBEEF,"parts":{"hash":BEEFDEAD,"total":3}},"height":3,"round":2,"timestamp":1234567890, "type":2}
+```
+
+Note how the fields within each level are sorted.
+
+## Other
+
+### MakeParts
+
+TMBIN encode an object and slice it into parts.
+
+```
+MakeParts(object, partSize)
+```

docs/specification/new-spec/state.md

@@ -0,0 +1,104 @@
+# Tendermint State
+
+## State
+
+The state contains information whose cryptographic digest is included in block headers,
+and thus is necessary for validating new blocks.
+For instance, the Merkle root of the results from executing the previous block, or the Merkle root of the current validators.
+While neither the results of transactions now the validators are ever included in the blockchain itself,
+the Merkle roots are, and hence we need a separate data structure to track them.
+
+```
+type State struct {
+    LastResults []Result
+    AppHash []byte
+
+    Validators []Validator
+    LastValidators []Validator
+
+    ConsensusParams ConsensusParams
+}
+```
+
+### Result
+
+```
+type Result struct {
+    Code uint32
+    Data []byte
+    Tags []KVPair
+}
+
+type KVPair struct {
+    Key     []byte
+    Value   []byte
+}
+```
+
+`Result` is the result of executing a transaction against the application.
+It returns a result code, an arbitrary byte array (ie. a return value),
+and a list of key-value pairs ordered by key. The key-value pairs, or tags,
+can be used to index transactions according to their "effects", which are
+represented in the tags.
+
+### Validator
+
+A validator is an active participant in the consensus with a public key and a voting power.
+Validator's also contain an address which is derived from the PubKey:
+
+```
+type Validator struct {
+    Address     []byte
+    PubKey      PubKey
+    VotingPower int64
+}
+```
+
+The `state.Validators` and `state.LastValidators` must always by sorted by validator address,
+so that there is a canonical order for computing the SimpleMerkleRoot.
+
+We also define a `TotalVotingPower` function, to return the total voting power:
+
+```
+func TotalVotingPower(vals []Validators) int64{
+    sum := 0
+    for v := range vals{
+        sum += v.VotingPower
+    }
+    return sum
+}
+```
+
+### PubKey
+
+TODO:
+
+### ConsensusParams
+
+TODO:
+
+## Execution
+
+We define an `Execute` function that takes a state and a block,
+executes the block against the application, and returns an updated state.
+
+```
+Execute(s State, app ABCIApp, block Block) State {
+    abciResponses := app.ApplyBlock(block)
+
+    return State{
+        LastResults: abciResponses.DeliverTxResults,
+        AppHash: abciResponses.AppHash,
+        Validators: UpdateValidators(state.Validators, abciResponses.ValidatorChanges),
+        LastValidators: state.Validators,
+        ConsensusParams: UpdateConsensusParams(state.ConsensusParams, abci.Responses.ConsensusParamChanges),
+    }
+}
+
+type ABCIResponses struct {
+    DeliverTxResults        []Result
+    ValidatorChanges        []Validator
+    ConsensusParamChanges   ConsensusParams
+    AppHash                 []byte
+}
+```