Merge pull request #1137 from tendermint/docs-consolidate

WIP: docs consolidation

docs/specification/new-spec/README.md

@@ -1,7 +1,8 @@
 # Tendermint Specification
 
 This is a markdown specification of the Tendermint blockchain.
-It defines the base data structures used in the blockchain and how they are validated.
+It defines the base data structures, how they are validated,
+and how they are communicated over the network.
 
 XXX: this spec is a work in progress and not yet complete - see github
 [isses](https://github.com/tendermint/tendermint/issues) and
@@ -14,12 +15,25 @@ please submit them to our [bug bounty](https://tendermint.com/security)!
 
 ## Contents
 
+### Data Structures
+
 - [Overview](#overview)
 - [Encoding and Digests](encoding.md)
 - [Blockchain](blockchain.md)
 - [State](state.md)
-- [Consensus](consensus.md)
-- [P2P](p2p/node.md)
+
+### P2P and Network Protocols
+
+- [The Base P2P Layer](p2p/README.md): multiplex the protocols ("reactors") on authenticated and encrypted TCP conns
+- [Peer Exchange (PEX)](pex/README.md): gossip known peer addresses so peers can find eachother
+- [Block Sync](block_sync/README.md): gossip blocks so peers can catch up quickly
+- [Consensus](consensus/README.md): gossip votes and block parts so new blocks can be committed
+- [Mempool](mempool/README.md): gossip transactions so they get included in blocks
+- [Evidence](evidence/README.md): TODO
+
+### More
+- [Light Client](light_client/README.md): TODO
+- [Persistence](persistence/README.md): TODO
 
 ## Overview
 
@@ -60,10 +74,3 @@ Also note that information like the transaction results and the validator set ar
 directly included in the block - only their cryptographic digests (Merkle roots) are.
 Hence, verification of a block requires a separate data structure to store this information.
 We call this the `State`. Block verification also requires access to the previous block.
-
-## TODO
-
-- Light Client
-- P2P
-- Reactor protocols (consensus, mempool, blockchain, pex)
-

docs/specification/new-spec/blockchain.md

@@ -10,7 +10,7 @@ The Tendermint blockchains consists of a short list of basic data types:
 ## Block
 
 A block consists of a header, a list of transactions, a list of votes (the commit),
-and a list of evidence if malfeasance (ie. signing conflicting votes).
+and a list of evidence of malfeasance (ie. signing conflicting votes).
 
 ```go
 type Block struct {
@@ -366,10 +366,10 @@ against the given signature and message bytes.
 
 ## Evidence
 
+TODO
 
 ```
-
-
+TODO
 ```
 
 Every piece of evidence contains two conflicting votes from a single validator that
@@ -384,7 +384,35 @@ Once a block is validated, it can be executed against the state.
 The state follows the recursive equation:
 
 ```go
-app = NewABCIApp
 state(1) = InitialState
-state(h+1) <- Execute(state(h), app, block(h))
+state(h+1) <- Execute(state(h), ABCIApp, block(h))
+```
+
+Where `InitialState` includes the initial consensus parameters and validator set,
+and `ABCIApp` is an ABCI application that can return results and changes to the validator
+set (TODO). Execute is defined as:
+
+```go
+Execute(s State, app ABCIApp, block Block) State {
+    TODO: just spell out ApplyBlock here
+    and remove ABCIResponses struct.
+    abciResponses := app.ApplyBlock(block)
+
+    return State{
+        LastResults: abciResponses.DeliverTxResults,
+        AppHash: abciResponses.AppHash,
+        Validators: UpdateValidators(state.Validators, abciResponses.ValidatorChanges),
+        LastValidators: state.Validators,
+        ConsensusParams: UpdateConsensusParams(state.ConsensusParams, abci.Responses.ConsensusParamChanges),
+    }
+}
+
+type ABCIResponses struct {
+    DeliverTxResults        []Result
+    ValidatorChanges        []Validator
+    ConsensusParamChanges   ConsensusParams
+    AppHash                 []byte
+}
 ```
+
+

docs/specification/new-spec/reactors/block_sync/reactor.md

@@ -0,0 +1,49 @@
+# Blockchain Reactor
+
+The Blockchain Reactor's high level responsibility is to enable peers who are
+far behind the current state of the consensus to quickly catch up by downloading
+many blocks in parallel, verifying their commits, and executing them against the
+ABCI application.
+
+Tendermint full nodes run the Blockchain Reactor as a service to provide blocks
+to new nodes. New nodes run the Blockchain Reactor in "fast_sync" mode,
+where they actively make requests for more blocks until they sync up.
+Once caught up, "fast_sync" mode is disabled and the node switches to
+using the Consensus Reactor. , and turn on the Consensus Reactor.
+
+## Message Types
+
+```go
+const (
+    msgTypeBlockRequest    = byte(0x10)
+    msgTypeBlockResponse   = byte(0x11)
+    msgTypeNoBlockResponse = byte(0x12)
+    msgTypeStatusResponse  = byte(0x20)
+    msgTypeStatusRequest   = byte(0x21)
+)
+```
+
+```go
+type bcBlockRequestMessage struct {
+    Height int64
+}
+
+type bcNoBlockResponseMessage struct {
+    Height int64
+}
+
+type bcBlockResponseMessage struct {
+    Block Block
+}
+
+type bcStatusRequestMessage struct {
+    Height int64
+
+type bcStatusResponseMessage struct {
+    Height int64
+}
+```
+
+## Protocol
+
+TODO

docs/specification/new-spec/state.md

@@ -3,10 +3,15 @@
 ## State
 
 The state contains information whose cryptographic digest is included in block headers, and thus is
-necessary for validating new blocks. For instance, the Merkle root of the results from executing the
-previous block, or the Merkle root of the current validators. While neither the results of
-transactions now the validators are ever included in the blockchain itself, the Merkle roots are,
-and hence we need a separate data structure to track them.
+necessary for validating new blocks. For instance, the set of validators and the results of
+transactions are never included in blocks, but their Merkle roots are - the state keeps track of them.
+
+Note that the `State` object itself is an implementation detail, since it is never
+included in a block or gossipped over the network, and we never compute
+its hash. However, the types it contains are part of the specification, since
+their Merkle roots are included in blocks.
+
+For details on an implementation of `State` with persistence, see TODO
 
 ```go
 type State struct {
@@ -77,28 +82,3 @@ TODO:
 
 TODO:
 
-## Execution
-
-We define an `Execute` function that takes a state and a block,
-executes the block against the application, and returns an updated state.
-
-```go
-Execute(s State, app ABCIApp, block Block) State {
-    abciResponses := app.ApplyBlock(block)
-
-    return State{
-        LastResults: abciResponses.DeliverTxResults,
-        AppHash: abciResponses.AppHash,
-        Validators: UpdateValidators(state.Validators, abciResponses.ValidatorChanges),
-        LastValidators: state.Validators,
-        ConsensusParams: UpdateConsensusParams(state.ConsensusParams, abci.Responses.ConsensusParamChanges),
-    }
-}
-
-type ABCIResponses struct {
-    DeliverTxResults        []Result
-    ValidatorChanges        []Validator
-    ConsensusParamChanges   ConsensusParams
-    AppHash                 []byte
-}
-```