Merge pull request #1039 from tendermint/add_consensus_spec

Spec of consensus/gossip protocol

docs/specification/new-spec/consensus.md

@@ -0,0 +1,197 @@
+# Tendermint Consensus
+
+Tendermint consensus is a distributed protocol executed by validator processes to agree on
+the next block to be added to the Tendermint blockchain. The protocol proceeds in rounds, where 
+each round is a try to reach agreement on the next block. A round starts by having a dedicated 
+process (called proposer) suggesting to other processes what should be the next block with 
+the `ProposalMessage`.
+The processes respond by voting for a block with `VoteMessage` (there are two kinds of vote messages, prevote 
+and precommit votes). Note that a proposal message is just a suggestion what the next block should be; a 
+validator might vote with a `VoteMessage` for a different block. If in some round, enough number 
+of processes vote for the same block, then this block is committed and later added to the blockchain. 
+`ProposalMessage` and `VoteMessage` are signed by the private key of the validator.
+The internals of the protocol and how it ensures safety and liveness properties are 
+explained [here](https://github.com/tendermint/spec).
+
+For efficiency reasons, validators in Tendermint consensus protocol do not agree directly on the block
+as the block size is big, i.e., they don't embed the block inside `Proposal` and `VoteMessage`. Instead, they 
+reach agreement on the `BlockID` (see `BlockID` definition in [Blockchain](blockchain.md) section) 
+that uniquely identifies each block. The block itself is disseminated to validator processes using
+peer-to-peer gossiping protocol. It starts by having a proposer first splitting a block into a 
+number of block parts, that are then gossiped between processes using `BlockPartMessage`. 
+
+Validators in Tendermint communicate by peer-to-peer gossiping protocol. Each validator is connected 
+only to a subset of processes called peers. By the gossiping protocol, a validator send to its peers 
+all needed  information (`ProposalMessage`, `VoteMessage` and `BlockPartMessage`) so they can 
+reach agreement on some block, and also obtain the content of the chosen block (block parts). As part of
+the gossiping protocol, processes also send auxiliary messages that inform peers about the 
+executed steps of the core consensus algorithm (`NewRoundStepMessage` and `CommitStepMessage`), and 
+also messages that inform peers what votes the process has seen (`HasVoteMessage`,
+`VoteSetMaj23Message` and `VoteSetBitsMessage`). These messages are then used in the gossiping protocol
+to determine what messages a process should send to its peers.  
+ 
+We now describe the content of each message exchanged during Tendermint consensus protocol.
+
+## ProposalMessage
+ProposalMessage is sent when a new block is proposed. It is a suggestion of what the 
+next block in the blockchain should be.
+```
+type ProposalMessage struct {
+	Proposal Proposal
+}
+```
+### Proposal
+Proposal contains height and round for which this proposal is made, BlockID as a unique identifier of proposed
+block, timestamp, and two fields (POLRound and POLBlockID) that are needed for termination of the consensus.
+The message is signed by the validator private key.  
+
+```
+type Proposal struct {
+	Height           int64            
+	Round            int              
+	Timestamp        Time        
+	BlockID          BlockID      
+	POLRound         int              
+	POLBlockID       BlockID          
+	Signature        Signature 
+}
+```
+
+NOTE: In the current version of the Tendermint, the consensus value in proposal is represented with
+PartSetHeader, and with BlockID in vote message. It should be aligned as suggested in this spec as 
+BlockID contains PartSetHeader.   
+
+## VoteMessage
+VoteMessage is sent to vote for some block (or to inform others that a process does not vote in the current round).
+Vote is defined in [Blockchain](blockchain.md) section and contains validator's information (validator address 
+and index), height and round for which the vote is sent, vote type, blockID if process vote for some 
+block (`nil` otherwise) and a timestamp when the vote is sent. The message is signed by the validator private key.
+```
+type VoteMessage struct {
+	Vote Vote
+}
+```
+
+## BlockPartMessage
+BlockPartMessage is sent when gossipping a piece of the proposed block. It contains height, round 
+and the block part. 
+
+```
+type BlockPartMessage struct {
+	Height int64
+	Round  int
+	Part   Part
+}
+```
+
+## ProposalHeartbeatMessage 
+ProposalHeartbeatMessage is sent to signal that a node is alive and waiting for transactions 
+to be able to create a next block proposal.
+
+```
+type ProposalHeartbeatMessage struct {
+    Heartbeat Heartbeat
+}
+```
+
+### Heartbeat
+Heartbeat contains validator information (address and index),
+height, round and sequence number. It is signed by the private key of the validator. 
+
+```
+type Heartbeat struct {
+	ValidatorAddress []byte       
+	ValidatorIndex   int              
+	Height           int64            
+	Round            int              
+	Sequence         int              
+	Signature        Signature 
+}
+```
+
+## NewRoundStepMessage
+NewRoundStepMessage is sent for every step transition during the core consensus algorithm execution. It is 
+used in the gossip part of the Tendermint protocol to inform peers about a current height/round/step 
+a process is in.
+
+```
+type NewRoundStepMessage struct {
+	Height                int64
+	Round                 int
+	Step                  RoundStepType
+	SecondsSinceStartTime int
+	LastCommitRound       int
+}
+```
+
+## CommitStepMessage
+CommitStepMessage is sent when an agreement on some block is reached. It contains height for which agreement
+is reached, block parts header that describes the decided block and is used to obtain all block parts,
+and a bit array of the block parts a process currently has, so its peers can know what parts 
+it is missing so they can send them. 
+
+```
+type CommitStepMessage struct {
+	Height           int64
+	BlockID          BlockID
+	BlockParts       BitArray
+}
+```
+
+TODO: We use BlockID instead of BlockPartsHeader (in current implementation) for symmetry. 
+
+## ProposalPOLMessage
+ProposalPOLMessage is sent when a previous block is re-proposed.
+It is used to inform peers in what round the process learned for this block (ProposalPOLRound),
+and what prevotes for the re-proposed block the process has. 
+			
+```
+type ProposalPOLMessage struct {
+	Height           int64
+	ProposalPOLRound int
+	ProposalPOL      BitArray
+}
+```
+
+
+## HasVoteMessage
+HasVoteMessage is sent to indicate that a particular vote has been received. It contains height, 
+round, vote type and the index of the validator that is the originator of the corresponding vote. 
+
+```
+type HasVoteMessage struct {
+	Height int64
+	Round  int
+	Type   byte
+	Index  int
+}
+```
+
+## VoteSetMaj23Message
+VoteSetMaj23Message is sent to indicate that a process has seen +2/3 votes for some BlockID.
+It contains height, round, vote type and the BlockID. 
+
+```
+type VoteSetMaj23Message struct {
+	Height  int64
+	Round   int
+	Type    byte
+	BlockID BlockID
+}
+```
+
+## VoteSetBitsMessage
+VoteSetBitsMessage is sent to communicate the bit-array of votes a process has seen for a given 
+BlockID. It contains height, round, vote type, BlockID and a bit array of 
+the votes a process has.
+
+```
+type VoteSetBitsMessage struct {
+	Height  int64
+	Round   int
+	Type    byte
+	BlockID BlockID
+	Votes   BitArray
+}
+```
+

docs/specification/new-spec/encoding.md

@@ -93,6 +93,17 @@ encode([]int{1, 2, 3, 4})       == [0x01, 0x04, 0x01, 0x01, 0x01, 0x02, 0x01, 0x
 encode([]string{"abc", "efg"})  == [0x01, 0x02, 0x01, 0x03, 0x61, 0x62, 0x63, 0x01, 0x03, 0x65, 0x66, 0x67]
 ```
 
+### BitArray
+BitArray is encoded as an `int` of the number of bits, and with an array of `uint64` to encode 
+value of each array element.
+
+```
+type BitArray struct {
+    Bits  int      
+    Elems []uint64 
+}
+```
+
 ### Time
 
 Time is encoded as an `int64` of the number of nanoseconds since January 1, 1970,
@@ -176,3 +187,13 @@ TMBIN encode an object and slice it into parts.
 ```
 MakeParts(object, partSize)
 ```
+
+### Part
+
+```
+type Part struct {
+	Index int                
+	Bytes byte[]         
+	Proof byte[]             
+}
+```