docs: add docs from website

8 years ago · 6fef4b080e
--- a/docs/getting-started/#02-first-app.md
+++ b/docs/getting-started/#02-first-app.md
@ -0,0 +1,256 @@
 # First Tendermint App

 As a general purpose blockchain engine, Tendermint is agnostic to the application you want to run.
 So, to run a complete blockchain that does something useful, you must start two programs:
 one is Tendermint Core, the other is your application, which can be written in any programming language.
 Recall from [the intro to ABCI](/intro/abci-overview) that Tendermint Core handles all the p2p and consensus stuff,
 and just forwards transactions to the application when they need to be validated, or when they're ready to be committed to a block.

 In this guide, we show you some examples of how to run an application using Tendermint.

 ## Install

 First, make sure you have [installed Tendermint](/download).
 The first apps we will work with are written in Go. 
 To install them, you need to [install Go](https://golang.org/doc/install) and 
 [put `$GOPATH/bin` in your `$PATH`](https://github.com/tendermint/tendermint/wiki/Setting-GOPATH). 

 Then run

 ```
 go get -u github.com/tendermint/abci/cmd/...
 ```

 If there is an error, download the `glide` tool to pin the dependencies:

 ```
 go get github.com/Masterminds/glide
 cd $GOPATH/src/github.com/tendermint/abci
 glide install
 go install ./cmd/...
 ```

 Now you should have two apps installed: 

 ```
 dummy --help
 counter --help
 ```

 Both of these applications are in Go. 
 But we also want to run an application in another language - 
 in this case, we'll run a Javascript version of the `counter`.
 To run it, you'll need to [install node](https://nodejs.org/en/download/).

 You'll also need to fetch the relevant repository, from https://github.com/tendermint/js-abci.
 Since I keep all my code under the `$GOPATH`, I just `go get github.com/tendermint/js-abci &> /dev/null`.
 Then `cd` into the `example` directory within that repository and run `npm install`.
 For instance, if you used `go get`, 

 ```
 cd $GOPATH/src/github.com/tendermint/js-abci/example
 npm install
 ```

 Now, let's run some apps!

 ## A First Example - Dummy

 The dummy app is a [Merkle tree](https://en.wikipedia.org/wiki/Merkle_tree) that just stores all transactions.
 If the transaction contains an `=`, eg. `key=value`, 
 then the `value` is stored under the `key` in the Merkle tree.
 Otherwise, the full transaction bytes are stored as the key and the value.

 Let's start a dummy application. 

 ```
 dummy
 ```

 In another terminal, we can start Tendermint.
 If you have never run Tendermint before, use:

 ```
 tendermint init 
 tendermint node
 ```

 If you have used Tendermint, you may want to reset the data for a new blockchain by running `tendermint unsafe_reset_all`.
 Then you can run `tendermint node` to start Tendermint, and connect to the app.
 For more details, see [the guide on using Tendermint](/docs/guides/using-tendermint).

 You should see Tendermint making blocks! 
 We can get the status of our Tendermint node as follows:

 ```
 curl -s localhost:46657/status
 ```

 The `-s` just silences `curl`. For nicer output, pipe the result into a tool like [jq](https://stedolan.github.io/jq/) 
 or [jsonpp](https://github.com/jmhodges/jsonpp).

 Now let's send some transactions to the dummy.

 ```
 curl -s 'localhost:46657/broadcast_tx_commit?tx="abcd"'
 ```

 Note the single quote (`'`) around the url, which ensures that the double quotes (`"`) are not escaped by bash.
 This command sent a transaction with bytes `abcd`, so `abcd` will be stored as both the key and the value in the Merkle tree.
 The response should look something like:

 ```
 {"jsonrpc":"2.0","id":"","result":[98,{"check_tx":{},"deliver_tx":{}}],"error":""}
 ```

 The `98` is a type-byte, and can be ignored (it's useful for serializing and deserializing arbitrary json).
 Otherwise, this result is empty - there's nothing to report on and everything is OK.

 We can confirm that our transaction worked and the value got stored by querying the app:

 ```
 curl -s 'localhost:46657/abci_query?data="abcd"&path=""&prove=false'
 ```

 The `path` and `prove` arguments can be ignored for now, and in a future release can be left out.
 The result should look like:


 ```
 {"jsonrpc":"2.0","id":"","result":[112,{"response":{"value":"61626364","log":"exists"}}],"error":""}
 ```

 Again, the `112` is the type-byte. Note the `value` in the result (`61626364`); this is the hex-encoding of the ASCII of `abcd`.
 You can verify this in a python shell by running `"61626364".decode('hex')`.
 Stay tuned for a future release that makes this output more human-readable ;). 

 Now let's try setting a different key and value:

 ```
 curl -s 'localhost:46657/broadcast_tx_commit?tx="name=satoshi"'
 ```

 Now if we query for `name`, we should get `satoshi`, or `7361746F736869` in hex:

 ```
 curl -s 'localhost:46657/abci_query?data="name"&path=""&prove=false'
 ```

 Try some other transactions and queries to make sure everything is working!

 ## Another Example - Counter

 Now that we've got the hang of it, let's try another application, the "counter" app.

 The counter app doesn't use a Merkle tree, it just counts how many times we've sent a transaction,
 or committed the state. 

 This application has two modes: `serial=off` and `serial=on`.

 When `serial=on`, transactions must be a big-endian encoded incrementing integer, starting at 0.

 If `serial=off`, there are no restrictions on transactions.

 In a live blockchain, transactions collect in memory before they are committed into blocks.
 To avoid wasting resources on invalid transactions,
 ABCI provides the `CheckTx` message,
 which application developers can use to accept or reject transactions,
 before they are stored in memory or gossipped to other peers.

 In this instance of the counter app, with `serial=on`, `CheckTx` only allows transactions whose integer is greater than the last committed one.

 Let's kill the previous instance of tendermint and the dummy application, and start the counter app.
 We can enable `serial=on` with a flag:

 ```
 counter --serial
 ```

 In another window, reset and start Tendermint:

 ```
 tendermint unsafe_reset_all
 tendermint node
 ```

 Once again, you can see the blocks streaming by. Let's send some transactions.
 Since we have set `serial=on`, the first transaction must be the number `0`:

 ```
 curl localhost:46657/broadcast_tx_commit?tx=0x00
 ```

 Note the empty, hence successful, response.
 The next transaction must be the number `1`. If instead, we try to send a `5`, we get an error:

 ```
 > curl localhost:46657/broadcast_tx_commit?tx=0x05
 {"jsonrpc":"2.0","id":"","result":[98,{"check_tx":{},"deliver_tx":{"code":3,"log":"Invalid nonce. Expected 1, got 5"}}],"error":""}
 ```

 But if we send a `1`, it works again:

 ```
 > curl localhost:46657/broadcast_tx_commit?tx=0x01
 {"jsonrpc":"2.0","id":"","result":[98,{"check_tx":{},"deliver_tx":{}}],"error":""}
 ```

 For more details on the `broadcast_tx` API, 
 see [the guide on using Tendermint](/docs/guides/using-tendermint).

 ## Example in Another Language - CounterJS

 The ultimate flexibility in Tendermint comes from being able to easily write the application in any language.
 While we already used the implementation written in Go, 
 let's now try the Counter application written in Javascript!

 Kill the previous `counter` and `tendermint` processes.
 Change directory to the location of the `github.com/tendermint/js-abci`.
 If you fetched the repository with `go get`, it would be 

 ```
 cd $GOPATH/src/github.com/tendermint/js-abci
 ```

 Now run the app:

 ```
 node example/app.js
 ```

 In another window, reset and start `tendermint`:

 ```
 tendermint unsafe_reset_all
 tendermint node
 ```

 Once again, you should see blocks streaming by - but now, our application is written in javascript!
 Try sending some transasctions, like before - the results should be the same:

 ```
 curl localhost:46657/broadcast_tx_commit?tx=0x00 # ok
 curl localhost:46657/broadcast_tx_commit?tx=0x05 # invalid nonce
 curl localhost:46657/broadcast_tx_commit?tx=0x01 # ok
 ```

 Neat, eh?

 ## A More Interesting Example - Basecoin

 Before concluding, we'd like to introduce you to our star application, [Basecoin](https://github.com/tendermint/basecoin).
 Unlike the `dummy` and `counter`, which are strictly for example purposes, 
 `basecoin` is designed to be actually useful - it's a general purpose framework for building cryptocurrencies.

 The default `basecoin` application is a multi-asset cryptocurrency that supports inter-blockchain communication.
 For more details on how basecoin works and how to use it, see our [basecoin guide](https://github.com/tendermint/basecoin/blob/develop/docs/guide/basecoin-basics.md)


 ## Next Step

 In this tutorial you learned how to run applications using Tendermint on a single node.
 You saw how applications could be written in different languages, 
 and how to send transactions and query for the latest state.
 But the true power of Tendermint comes from its ability to securely and efficiently run an application 
 across a distributed network of nodes, while keeping them all in sync using its state-of-the-art consensus protocol.
 This is the subject of the next tutorial, where we show you [how to deploy Tendermint networks](/docs/getting-started/deploy-testnet).
--- a/docs/getting-started/#03-deploy-testnet.md
+++ b/docs/getting-started/#03-deploy-testnet.md
@ -0,0 +1,41 @@
 # Deploy a Testnet

 Now that we've seen how ABCI works, and even played with a few applications on a single validator node,
 it's time to deploy a test network to four validator nodes.
 For this deployment, we'll use the `basecoin` application.

 ## Manual Deployments

 It's relatively easy to setup a Tendermint cluster manually.
 The only requirements for a particular Tendermint node are a private key for the validator,
 stored as `priv_validator.json`, and a list of the public keys of all validators, stored as `genesis.json`.
 These files should be stored in `~/.tendermint`, or wherever the `$TMROOT` variable might be set to.

 Here are the steps to setting up a testnet manually:

 1) Provision nodes on your cloud provider of choice
 2) Install Tendermint and the application of interest on all nodes
 3) Generate a private key for each validator using `tendermint gen_validator`
 4) Compile a list of public keys for each validator into a `genesis.json` file.
 5) Run `tendermint node --p2p.seeds=< seed addresses >` on each node, where `< seed addresses >` is a
 comma separated list of the IP:PORT combination for each node. The default port for Tendermint is `46656`.
 Thus, if the IP addresses of your nodes were `192.168.0.1, 192.168.0.2, 192.168.0.3, 192.168.0.4`,
 the command would look like: `tendermint node --p2p.seeds=192.168.0.1:46656,192.168.0.2:46656,192.168.0.3:46656,192.168.0.4:46656`.

 After a few seconds, all the nodes should connect to eachother and start making blocks!
 For more information, see the Tendermint Networks section of [the guide to using Tendermint](/docs/guides/using-tendermint).

 ## Automated Deployments

 While the manual deployment is easy enough, an automated deployment is always better.
 For this, we have the [mintnet-kubernetes tool](https://github.com/tendermint/mintnet-kubernetes),
 which allows us to automate the deployment of a Tendermint network on an already provisioned kubernetes cluster.

 For more details, see the [mintnet-kubernetes repo](https://github.com/tendermint/mintnet-kubernetes),
 and check out [Google Cloud Platform](https://cloud.google.com/) for simple provisioning of kubernetes clusters.

 TODO: a better tutorial here, and Ansible/Terraform.

 ## Next Steps

 Done trying out the testnet? Continue [onwards](/docs/getting-started/next-steps).
--- a/docs/getting-started/#04-next-steps.md
+++ b/docs/getting-started/#04-next-steps.md
@ -0,0 +1,21 @@
 # Next Steps

 By now you've seen how to run a simple example ABCI application on a local Tendermint node
 and on a remote Tendermint cluster. 

 To learn more about building ABCI applications and integrating with Tendermint, see the [Developer Guides](/docs/guides/app-development).
 To learn more about running the Tendermint software, see the [Using Tendermint Guide](/docs/guides/using-tendermint).

 To learn more about Tendermint's various pieces, checkout the [Documentation](/docs).
 For a deeper dive, see [this thesis](https://atrium.lib.uoguelph.ca/xmlui/handle/10214/9769). 
 There is also the [original whitepaper](/static/docs/tendermint.pdf), though it is now quite outdated.

 The Tendermint [Software Ecosystem](/ecosystem) contains many example applications and related software built by the Tendermint team and others. Check it out for some inspiration!

 For details on how the software has changed, and what changes are in store, see the [Changelog](/docs/changelog) and the [Roadmap](/docs/roadmap).

 See our [Community](/community) page for more ways to collaborate.

 You can also [get in touch with the team](/contact).

 Most importantly, enjoy!
--- a/docs/guides/abci-cli.md
+++ b/docs/guides/abci-cli.md
@ -0,0 +1,220 @@
 # Using the abci-cli

 To facilitate testing and debugging of ABCI servers and simple apps,
 we built a CLI, the `abci-cli`, for sending ABCI messages from the command line.

 ##  Install

 Make sure you [have Go installed](https://golang.org/doc/install) and [put `$GOPATH/bin` in your `$PATH`](https://github.com/tendermint/tendermint/wiki/Setting-GOPATH).

 Next, install the `abci-cli` tool and example applications:

 ```
 go get -u github.com/tendermint/abci/cmd/...
 ```

 If this fails, you may need to use `glide` to get vendored dependencies:

 ```
 go get github.com/Masterminds/glide
 cd $GOPATH/src/github.com/tendermint/abci
 glide install
 go install ./cmd/...
 ```

 Now run `abci-cli --help` to see the list of commands:

 ```
 COMMANDS:
   batch        Run a batch of ABCI commands against an application
   console      Start an interactive console for multiple commands
   echo         Have the application echo a message
   info         Get some info about the application
   set_option   Set an option on the application
   deliver_tx    Append a new tx to application
   check_tx     Validate a tx
   commit       Get application Merkle root hash
   help, h      Shows a list of commands or help for one command

 GLOBAL OPTIONS:
   --address "tcp://127.0.0.1:46658"    address of application socket
   --help, -h                           show help
   --version, -v                        print the version
 ```

 ## First Example - Dummy

 The `abci-cli` tool lets us send ABCI messages to our application, to help build and debug them.

 The most important messages are `deliver_tx`, `check_tx`, and `commit`,
 but there are others for convenience, configuration, and information purposes.

 Let's start a dummy application. The dummy just stores transactions in a merkle tree:

 ```
 dummy
 ```

 In another terminal, run

 ```
 abci-cli echo hello
 abci-cli info
 ```

 The application should echo `hello` and give you some information about itself.

 An ABCI application must provide two things:

  - a socket server
  - a handler for ABCI messages

 When we run the `abci-cli` tool we open a new connection to the application's socket server,
 send the given ABCI message, and wait for a response.

 The server may be generic for a particular language, and we provide a [reference implementation 
 in Golang](https://github.com/tendermint/abci/tree/master/server). 
 See the [list of other ABCI implementations](https://tendermint.com/ecosystem) 
 for servers in other languages.

 The handler is specific to the application, and may be arbitrary,
 so long as it is deterministic and conforms to the ABCI interface specification.

 So when we run `abci-cli info`, we open a new connection to the ABCI server, which calls the `Info()` method on the application, which tells us the number of transactions in our Merkle tree.

 Now, since every command opens a new connection, we provide the `abci-cli console` and `abci-cli batch` commands,
 to allow multiple ABCI messages to be sent over a single connection.

 Running `abci-cli console` should drop you in an interactive console for speaking ABCI messages to your application.

 Try running these commands:

 ```
 > echo hello
 -> data: hello

 > info
 -> data: {"size":0}

 > commit
 -> data: 0x

 > deliver_tx "abc"
 -> code: OK

 > info
 -> data: {"size":1}

 > commit
 -> data: 0x750502FC7E84BBD788ED589624F06CFA871845D1

 > query "abc"
 -> code: OK
 -> data: {"index":0,"value":"abc","exists":true}

 > deliver_tx "def=xyz"
 -> code: OK

 > commit
 -> data: 0x76393B8A182E450286B0694C629ECB51B286EFD5

 > query "def"
 -> code: OK
 -> data: {"index":1,"value":"xyz","exists":true}
 ```

 Note that if we do `deliver_tx "abc"` it will store `(abc, abc)`,
 but if we do `deliver_tx "abc=efg"` it will store `(abc, efg)`.

 Similarly, you could put the commands in a file and run `abci-cli --verbose batch < myfile`.


 ## Another Example - Counter

 Now that we've got the hang of it, let's try another application, the "counter" app.

 The counter app doesn't use a Merkle tree, it just counts how many times we've sent a transaction,
 asked for a hash, or committed the state. The result of `commit` is just the number of transactions sent.

 This application has two modes: `serial=off` and `serial=on`.

 When `serial=on`, transactions must be a big-endian encoded incrementing integer, starting at 0.

 If `serial=off`, there are no restrictions on transactions.

 We can toggle the value of `serial` using the `set_option` ABCI message.

 When `serial=on`, some transactions are invalid.
 In a live blockchain, transactions collect in memory before they are committed into blocks.
 To avoid wasting resources on invalid transactions,
 ABCI provides the `check_tx` message,
 which application developers can use to accept or reject transactions,
 before they are stored in memory or gossipped to other peers.

 In this instance of the counter app, `check_tx` only allows transactions whose integer is greater than the last committed one.

 Let's kill the console and the dummy application, and start the counter app:

 ```
 counter
 ```

 In another window, start the `abci-cli console`:

 ```
 > set_option serial on
 -> data: serial=on

 > check_tx 0x00
 -> code: OK

 > check_tx 0xff
 -> code: OK

 > deliver_tx 0x00
 -> code: OK

 > check_tx 0x00
 -> code: BadNonce
 -> log: Invalid nonce. Expected >= 1, got 0

 > deliver_tx 0x01
 -> code: OK

 > deliver_tx 0x04
 -> code: BadNonce
 -> log: Invalid nonce. Expected 2, got 4

 > info
 -> data: {"hashes":0,"txs":2}
 ```

 This is a very simple application, but between `counter` and `dummy`, its easy to see how you can build out arbitrary application states on top of the ABCI.
 In the near future, `erisdb` of Eris Industries will also run atop ABCI, bringing with it Ethereum-like accounts, the Ethereum virtual-machine, Eris's permissioning scheme, and native contracts extensions.

 But the ultimate flexibility comes from being able to write the application easily in any language.

 We have implemented the counter in a number of languages (see the example directory).

 To run the Node JS version, `cd` to `example/js` and run

 ```
 node app.js
 ```

 (you'll have to kill the other counter application process).
 In another window, run the console and those previous ABCI commands.
 You should get the same results as for the Go version.

 Want to write the counter app in your favorite language?! We'd be happy to add you to our [ecosystem](https://tendermint.com/ecosystem)! We're also offering [bounties](https://tendermint.com/bounties) for implementations in new languages!

 ## Notes

 The `abci-cli` is designed strictly for testing and debugging. 
 In a real deployment, the role of sending messages is taken by Tendermint,
 which connects to the app using three separate connections, 
 each with its own pattern of messages. 

 For more information, see the [application developers guide](/docs/guides/app-development).
 For examples of running an ABCI app with Tendermint, see the [introductory guide](/docs/getting-started/first-abci-app).

--- a/docs/guides/app-architecture.md
+++ b/docs/guides/app-architecture.md
@ -0,0 +1,70 @@
 # Application Architecture Guide

 ## Overview

 A blockchain application is more than the consensus engine and the transaction logic (eg. smart contracts, business logic) as implemented in the ABCI app.  There are also (mobile, web, desktop) clients that will need to connect and make use of the app.  We will assume for now that you have a well designed transactions and database model, but maybe this will be the topic of another article.  This article is more interested in various ways of setting up the "plumbing" and connecting these pieces, and demonstrating some evolving best practices.

 ## Security

 A very important aspect when constructing a blockchain is security.  The consensus model can be DoSed (no consensus possible) by corrupting 1/3 of the validators and exploited (writing arbitrary blocks) by corrupting 2/3 of the validators. So, while the security is not that of the "weakest link", you should take care that the "average link" is sufficiently hardened.

 One big attack surface on the validators is the communication between the ABCI app and the tendermint core.  This should be highly protected. Ideally, the app and the core are running on the same machine, so no external agent can target the communication channel.  You can use unix sockets (with permissions preventing access from other users), or even compile the two apps into one binary if the ABCI app is also writen in go (@ebuchman says this is possible). If you are unable to do that due to language support, then the ABCI app should bind a TCP connection to localhost (127.0.0.1), which is less efficient and secure, but still not reachable from outside.  If you must run the ABCI app and tendermint core on separate machines, make sure you have a secure communication channel (ssh tunnel?)

 Now assuming, you have linked together your app and the core securely, you must also make sure no one can get on the machine it is hosted on.  At this point it is basic network security. Run on a secure operating system (SELinux?). Limit who has access to the machine (user accounts, but also where the physical machine is hosted).  Turn off all services except for ssh, which should only be accessible by some well-guarded public/private key pairs (no password). And maybe even firewall off access to the ports used by the validators, so only known validators can connect.

 There was also a suggestion on slack from @jhon about compiling everything together with a unikernel for more security, such as [Mirage](https://mirage.io) or [UNIK](https://github.com/emc-advanced-dev/unik).

 ## Connecting your client to the blockchain

 ### Tendermint Core RPC

 I believe this was the original design from @ebuchman.  The concept is that the ABCI app is completely hidden from the outside world and only communicated through a tested and secured [interface exposed by the tendermint core](/docs/specs/rpc). This interface exposes a lot of data on the block header and consensus process, which is quite useful for externally verifying the system.  It also includes 3(!) methods to broadcast a transaction (propose it for the blockchain, and possibly await a response).  And one method to query app-specific data from the ABCI application.

 Pros:
 * Server code already written
 * Access to block headers to validate merkle proofs (nice for light clients)
 * Basic read/write functionality is supported

 Cons:
 * Limited interface to app.  All queries must be serialized into []byte (less expressive than JSON over HTTP) and there is no way to push data from ABCI app to the client (eg. notify me if account X receives a transaction)

 ### Custom ABCI server

 This was proposed by @wolfposd on slack and demonstrated by [TMChat](https://github.com/wolfposd/TMChat), a sample app. The concept is to write a custom server for your app (with typical REST API/websockets/etc for easy use by a mobile app). This custom server is in the same binary as the ABCI app and data store, so can easily react to complex events there that involve understanding the data format (send a message if my balance drops below 500). All "writes" sent to this server are proxied via websocket/JSON-RPC to tendermint core.  When they come back as deliver_tx over ABCI, they will be written to the data store. For "reads", we can do any queries we wish that are supported by our architecture, using any web technology that is useful. The general architecture is shown in the following diagram:

 <img alt="Application Architecture" src="../assets/images/tm-app-example.png">

 Pros:
 * Separates application logic from blockchain logic
 * Allows much richer, more flexible client-facing API
 * Allows pub-sub, watching certain fields, etc.

 Cons:
 * Access to ABCI app can be dangerous (be VERY careful not to write unless it comes from the validator node)
 * No direct access to the blockchain headers to verify tx
 * You must write your own API (but maybe that's a pro...)

 ### Hybrid solutions

 Likely the least secure but most versatile.  The client can access both the tendermint node for all blockchain info, as well as a custom app server, for complex queries and pub-sub on the abci app.

 Pros:
 * All from both above solutions

 Cons:
 * Even more complexity
 * Even more attack vectors (less security)

 ## Scalability

 Read replica using non-validating nodes?  They could forward transactions to the validators (fewer connections, more security), and locally allow all queries in any of the above configurations. Thus, while transaction-processing speed is limited by the speed of the abci app and the number of validators, one should be able to scale our read performance to quite an extent (until the replication process drains too many resources from the validator nodes).

 ## Example Code

 * [TMChat](https://github.com/wolfposd/TMChat)

 TODO: more!

 ## Contributions

 Many thanks to Ethan Frey for writing this page.
--- a/docs/guides/app-development.md
+++ b/docs/guides/app-development.md
@ -0,0 +1,157 @@
 # Application Development Guide

 ## ABCI Design

 The purpose of ABCI is to provide a clean interface between state transition machines on one computer and the mechanics of their replication across multiple computers. The former we call 'application logic' and the latter the 'consensus engine'. Application logic validates transactions and optionally executes transactions against some persistent state. A consensus engine ensures all transactions are replicated in the same order on every machine. We call each machine in a consensus engine a 'validator', and each validator runs the same transactions through the same application logic. In particular, we are interested in blockchain-style consensus engines, where transactions are committed in hash-linked blocks.

 The ABCI design has a few distinct components:

 - message protocol
    - pairs of request and response messages
    - consensus makes requests, application responds
    - defined using protobuf
 - server/client
    - consensus engine runs the client
    - application runs the server
    - two implementations:
        - async raw bytes
        - grpc
 - blockchain protocol
    - abci is connection oriented
    - Tendermint Core maintains three connections:
        - [mempool connection](#mempool-connection): for checking if transactions should be relayed before they are committed. only uses `CheckTx`
        - [consensus connection](#consensus-connection): for executing transactions that have been committed. Message sequence is, for every block, `BeginBlock, [DeliverTx, ...], EndBlock, Commit`
        - [query connection](#query-connection): for querying the application state.  only uses Query and Info

 <img src="../assets/images/abci.png">

 The mempool and consensus logic act as clients, and each maintains an open ABCI connection with the application, which hosts an ABCI server. Shown are the request and response types sent on each connection.

 ## Message Protocol

 The message protocol consists of pairs of requests and responses. Some messages have no fields, while others may include byte-arrays, strings, or integers. See the `message Request` and `message Response` definitions in [the protobuf definition file](https://github.com/tendermint/abci/blob/master/types/types.proto), and the [protobuf documentation](https://developers.google.com/protocol-buffers/docs/overview) for more details.

 For each request, a server should respond with the corresponding response, where order of requests is preserved in the order of responses.

 ## Server

 To use ABCI in your programming language of choice, there must be a ABCI server in that language.
 Tendermint supports two kinds of implementation of the server:

 - Asynchronous, raw socket server (Tendermint Socket Protocol, also known as TSP or Teaspoon)
 - GRPC

 Both can be tested using the `abci-cli` by setting the `--abci` flag appropriately (ie. to `socket` or `grpc`).

 See examples, in various stages of maintenance, in [Go](https://github.com/tendermint/abci/tree/master/server), [JavaScript](https://github.com/tendermint/js-abci), [Python](https://github.com/tendermint/abci/tree/master/example/python3/abci), [C++](https://github.com/mdyring/cpp-tmsp), and [Java](https://github.com/jTendermint/jabci).

 ### GRPC

 If GRPC is available in your language, this is the easiest approach,
 though it will have significant performance overhead.

 To get started with GRPC, copy in the [protobuf file](https://github.com/tendermint/abci/blob/master/types/types.proto) and compile it using the GRPC plugin for your language.
 For instance, for golang, the command is `protoc --go_out=plugins=grpc:. types.proto`. See the [grpc documentation for more details](http://www.grpc.io/docs/). `protoc` will autogenerate all the necessary code for ABCI client and server in your language, including whatever interface your application must satisfy to be used by the ABCI server for handling requests.

 ### TSP

 If GRPC is not available in your language, or you require higher performance, or otherwise enjoy programming, you may implement your own ABCI server
 using the Tendermint Socket Protocol, known affectionaltely as Teaspoon.
 The first step is still to auto-generate the relevant data types and codec in your language using `protoc`.
 Messages coming over the socket are Protobuf3 encoded, but additionally length-prefixed to facilitate use as a streaming protocol. Protobuf3 doesn't have an official length-prefix standard, so we use our own. The first byte in the prefix represents the length of the Big Endian encoded length. The remaining bytes in the prefix are the Big Endian encoded length.

 For example, if the Protobuf3 encoded ABCI message is 0xDEADBEEF (4 bytes), the length-prefixed message is 0x0104DEADBEEF. If the Protobuf3 encoded ABCI message is 65535 bytes long, the length-prefixed message would be like 0x02FFFF....

 Note this prefixing does not apply for grpc.

 An ABCI server must also be able to support multiple connections, as Tendermint uses three connections.

 ## Client

 There are currently two use-cases for an ABCI client.
 One is a testing tool, as in the `abci-cli`, which allows ABCI requests to be sent via command line.
 The other is a consensus engine, such as Tendermint Core, which makes requests to the application every time a new transaction is received or a block is committed.

 It is unlikely that you will need to implement a client. For details of our client, see [here](https://github.com/tendermint/abci/tree/master/client).

 ## Blockchain Protocol

 In ABCI, a transaction is simply an arbitrary length byte-array.
 It is the application's responsibility to define the transaction codec as they please,
 and to use it for both CheckTx and DeliverTx.

 Note that there are two distinct means for running transactions, corresponding to stages of 'awareness'
 of the transaction in the network. The first stage is when a transaction is received by a validator from a client into the so-called mempool or transaction pool - this is where we use CheckTx. The second is when the transaction is successfully committed on more than 2/3 of validators - where we use DeliverTx. In the former case, it may not be necessary to run all the state transitions associated with the transaction, as the transaction may not ultimately be committed until some much later time, when the result of its execution will be different.
 For instance, an Ethereum ABCI app would check signatures and amounts in CheckTx, but would not actually execute any contract code until the DeliverTx, so as to avoid executing state transitions that have not been finalized.

 To formalize the distinction further, two explicit ABCI connections are made between Tendermint Core and the application: the mempool connection and the consensus connection. We also make a third connection, the query connection, to query the local state of the app.

 ### Mempool Connection

 The mempool connection is used *only* for CheckTx requests.
 Transactions are run using CheckTx in the same order they were received by the validator.
 If the CheckTx returns `OK`, the transaction is kept in memory and relayed to other peers in the same order it was received. Otherwise, it is discarded.

 CheckTx requests run concurrently with block processing;
 so they should run against a copy of the main application state which is reset after every block.
 This copy is necessary to track transitions made by a sequence of CheckTx requests before they are included in a block. When a block is committed, the application must ensure to reset the mempool state to the latest committed state. Tendermint Core will then filter through all transactions in the mempool, removing any that were included in the block, and re-run the rest using CheckTx against the post-Commit mempool state.

 ### Consensus Connection

 The consensus connection is used only when a new block is committed, and communicates all information from the block in a series of requests:  `BeginBlock, [DeliverTx, ...], EndBlock, Commit`.
 That is, when a block is committed in the consensus, we send a list of DeliverTx requests (one for each transaction) sandwiched by BeginBlock and EndBlock requests, and followed by a Commit.

 #### DeliverTx

 DeliverTx is the workhorse of the blockchain. Tendermint sends the DeliverTx requests asynchronously but in order,
 and relies on the underlying socket protocol (ie. TCP) to ensure they are received by the app in order. They have already been ordered in the global consensus by the Tendermint protocol.

 DeliverTx returns a abci.Result, which includes a Code, Data, and Log. The code may be non-zero (non-OK), meaning the corresponding transaction should have been rejected by the mempool,
 but may have been included in a block by a Byzantine proposer.

 The block header will be updated (TODO) to include some commitment to the results of DeliverTx, be it a bitarray of non-OK transactions, or a merkle root of the data returned by the DeliverTx requests, or both.

 #### Commit

 Once all processing of the block is complete, Tendermint sends the Commit request and blocks waiting
 for a response. While the mempool may run concurrently with block processing (the BeginBlock, DeliverTxs, and EndBlock), it is locked for the Commit request so that its state can be safely reset during Commit. This means the app *MUST NOT* do any blocking communication with the mempool (ie. broadcast_tx) during Commit, or there will be deadlock. Note also that all remaining transactions in the mempool are replayed on the mempool connection (CheckTx) following a commit.

 The Commit response includes a byte array, which is the deterministic state root of the application. It is included in the header of the next block. It can be used to provide easily verified Merkle-proofs of the state of the application.

 It is expected that the app will persist state to disk on Commit. The option to have all transactions replayed from some previous block is the job of the [Handshake](#handshake).

 #### BeginBlock

 The BeginBlock request can be used to run some code at the beginning of every block. It also allows Tendermint to send the current block hash and header to the application, before it sends any of the transactions.

 The app should remember the latest height and header (ie. from which it has run a successful Commit) so that it can tell Tendermint where to pick up from when it restarts. See information on the Handshake, below.

 #### EndBlock

 The EndBlock request can be used to run some code at the end of every block. Additionally, the response may contain a list of validators, which can be used to update the validator set. To add a new validator or update an existing one, simply include them in the list returned in the EndBlock response. To remove one, include it in the list with a `power` equal to `0`. Tendermint core will take care of updating the validator set. Note validator set changes are only available in v0.8.0 and up.

 ### Query Connection

 This connection is used to query the application without engaging consensus. It's exposed over the tendermint core rpc, so clients can query the app without exposing a server on the app itself, but they must serialize each query as a single byte array. Additionally, certain "standardized" queries may be used to inform local decisions, for instance about which peers to connect to.

 Tendermint Core currently uses the Query connection to filter peers upon connecting, according to IP address or public key. For instance, returning non-OK ABCI response to either of the following queries will cause Tendermint to not connect to the corresponding peer:

 - `p2p/filter/addr/<addr>`, where `<addr>` is an IP address.
 - `p2p/filter/pubkey/<pubkey>`, where `<pubkey>` is the hex-encoded ED25519 key of the node (not it's validator key)

 Note: these query formats are subject to change!

 ### Handshake

 When the app or tendermint restarts, they need to sync to a common height.
 When an ABCI connection is first established, Tendermint will call `Info` on the Query connection.
 The response should contain the LastBlockHeight and LastBlockAppHash
 - the former is the last block for the which the app ran Commit successfully,
 the latter is the response from that Commit.

 Using this information, Tendermint will determine what needs to be replayed, if anything, against the app,
 to ensure both Tendermint and the app are synced to the latest block height.

 If the app returns a LastBlockHeight of 0, Tendermint will just replay all blocks.

 Note this functionality is only available in v0.8.0 and up.
--- a/docs/guides/install-from-source.md
+++ b/docs/guides/install-from-source.md
@ -0,0 +1,100 @@
 # Install from Source

 This page provides instructions on installing Tendermint from source.
 To download pre-built binaries, see the [Download page](/download).

 ## Install Go

 Make sure you have [installed Go](/docs/guides/install-go) and set the `GOPATH`.

 ## Install Tendermint

 You should be able to install the latest with a simple 

 ```
 go get github.com/tendermint/tendermint/cmd/tendermint
 ```

 Run `tendermint --help` for more.

 If the installation failed, a dependency may been updated and become incompatible with the latest Tendermint master branch.
 We solve this using the `glide` tool for dependency management.

 Fist, install `glide`:

 ```
 go get github.com/Masterminds/glide
 ```

 Now we can fetch the correct versions of each dependency by running:

 ```
 cd $GOPATH/src/github.com/tendermint/tendermint
 glide install
 go install ./cmd/tendermint
 ```

 Note that even though `go get` originally failed, 
 the repository was still cloned to the correct location in the `$GOPATH`.

 The latest Tendermint Core version is now installed. 

 ### Reinstall

 If you already have Tendermint installed, and you make updates,
 simply 

 ```
 cd $GOPATH/src/github.com/tendermint/tendermint
 go install ./cmd/tendermint
 ```

 To upgrade, there are a few options:

 - set a new `$GOPATH` and run `go get github.com/tendermint/tendermint/cmd/tendermint`. This makes a fresh copy of everything for the new version. 
 - run `go get -u github.com/tendermint/tendermint/cmd/tendermint`, where the `-u` fetches the latest updates for the repository and its dependencies
 - fetch and checkout the latest master branch in `$GOPATH/src/github.com/tendermint/tendermint`, and then run `glide install && go install ./cmd/tendermint` as above.

 Note the first two options should usually work, but may fail.
 If they do, use `glide`, as above: 

 ```
 cd $GOPATH/src/github.com/tendermint/tendermint
 glide install
 go install ./cmd/tendermint
 ```

 Since the third option just uses `glide` right away, it should always work.


 ### Troubleshooting

 If `go get` failing bothers you, fetch the code using `git`:

 ```
 mkdir -p $GOPATH/src/github.com/tendermint
 git clone https://github.com/tendermint/tendermint $GOPATH/src/github.com/tendermint/tendermint
 cd $GOPATH/src/github.com/tendermint/tendermint
 glide install
 go install ./cmd/tendermint
 ```

 ### Run

 To start a one-node blockchain with a simple in-process application: 

 ```
 tendermint init
 tendermint node --proxy_app=dummy
 ```

 See the 
 [App Development](/docs/guides/app-development)
 guide for more details on building applications,
 and the 
 [Using Tendermint](/docs/guides/using-tendermint) 
 guide for more details about using the `tendermint` program.

 ## Next Step

 Learn how to [create your first ABCI app](/docs/getting-started/first-abci-app).
--- a/docs/guides/using-tendermint.md
+++ b/docs/guides/using-tendermint.md
@ -0,0 +1,314 @@
 # Using Tendermint

 This is a guide to using the `tendermint` program from the command line.
 It assumes only that you [have tendermint installed](/download) and have some rudimentary idea
 of what Tendermint and ABCI are.

 You can see the help menu with `tendermint --help`, and the version number with `tendermint version`.

 ## Directory Root

 The default directory for blockchain data is `~/.tendermint`. Override this by setting the `TMROOT` environment variable.

 ## Initialize

 Initialize the root directory by running:

 ```
 tendermint init
 ```

 This will create a new private key (`priv_validator.json`), and a genesis file (`genesis.json`) containing the associated public key.
 This is all that's necessary to run a local testnet with one validator.

 For more elaborate initialization, see our quick and dirty deployment tool, [mintnet](https://github.com/tendermint/mintnet).


 ## Run

 To run a tendermint node, use

 ```
 tendermint node
 ```

 By default, Tendermint will try to connect to a abci appliction on [127.0.0.1:46658](127.0.0.1:46658).
 If you have the `dummy` ABCI app installed, run it in another window.
 If you don't, kill tendermint and run an in-process version with

 ```
 tendermint node --proxy_app=dummy
 ```

 After a few seconds you should see blocks start streaming in.
 Note that blocks are produced regularly, even if there are no transactions.
 Hopefully we will change this.

 Tendermint supports in-process versions of the dummy, counter, and nil apps that ship as examples in the [ABCI repository](https://github.com/tendermint/abci).
 It's easy to compile your own app in-process with tendermint if it's written in Go.
 If your app is not written in Go, simply run it in another process,
 and use the `--proxy_app` flag to specify the address of the socket it is listening on, for instance


 ```
 tendermint node --proxy_app=/var/run/abci.sock
 ```


 ## Transactions

 To send a transaction, use `curl` to make requests to the Tendermint RPC server:

 ```
 curl http://localhost:46657/broadcast_tx_commit?tx=\"abcd\"
 ```

 For handling responses, we recommend you [install the `jsonpp` tool](http://jmhodges.github.io/jsonpp/) to pretty print the JSON

 We can see the chain's status at the `/status` end-point:

 ```
 curl http://localhost:46657/status |  jsonpp
 ```

 and the `latest_app_hash` in particular:

 ```
 curl http://localhost:46657/status |  jsonpp | grep app_hash
 ```

 Visit [http://localhost:46657](http://localhost:46657) in your browser to see the list of other endpoints.
 Some take no arguments (like `/status`), while others specify the argument name and use `_` as a placeholder.

 ## Reset

 WARNING: UNSAFE. Only do this in development and only if you can afford to lose all blockchain data!

 To reset a blockchain, stop the node, remove the `~/.tendermint/data` directory and run

 ```
 tendermint unsafe_reset_priv_validator
 ```

 This final step is necessary to reset the `priv_validator.json`,
 which otherwise prevents you from making conflicting votes in the consensus
 (something that could get you in trouble if you do it on a real blockchain).
 If you don't reset the priv_validator, your fresh new blockchain will not make any blocks.

 ## Configuration

 Tendermint uses a `config.toml` for configutation. For details, see [the documentation](/docs/specs/configuration).

 Notable options include the socket address of the application (`proxy_app`),
 the listenting address of the tendermint peer (`p2p.laddr`),
 and the listening address of the rpc server (`rpc.laddr`).

 Some fields from the config file can be overwritten with flags.

 ## Broadcast API

 Earlier, we used the `broadcast_tx_commit` endpoint to send a transaction.
 When a transaction is sent to a tendermint node,
 it will run via `CheckTx` against the application.
 If it passes `CheckTx`, it will be included in the mempool,
 broadcast to other peers, and eventually included in a block.

 Since there are multiple phases to processing a transaction, we offer multiple endpoints to broadcast a transaction:

 ```
 /broadcast_tx_async
 /broadcast_tx_sync
 /broadcast_tx_commit
 ```

 These correspond to no-processing, processing through the mempool, and processing through a block, respectively.
 That is, `broadcast_tx_async`, will return right away without waiting to hear if the transaction is even valid,
 while `broadcast_tx_sync` will return with the result of running the transaction through `CheckTx`.
 Using `broadcast_tx_commit` will wait until the transaction is committed in a block or until some timeout is reached,
 but will return right away if the transaction does not pass `CheckTx`.
 The return value for `broadcast_tx_commit` includes two fields, `check_tx` and `deliver_tx`, pertaining to the result of running
 the transaction through those ABCI messages.

 The benefit of using `broadcast_tx_commit` is that the request returns after the transaction is committed (ie. included in a block),
 but that can take on the order of a second.
 For a quick result, use `broadcast_tx_sync`,
 but the transaction will not be committed until later, and by that point its effect on the state may change.

 We are working to improve the API and add more ways to track the status of a transaction and get its results.

 ## Tendermint Networks

 When `tendermint init` is run, both a `genesis.json` and `priv_validator.json` are created in `~/.tendermint`.
 The `genesis.json` might look like:

 ```
 {
 	"app_hash": "",
 	"chain_id": "test-chain-HZw6TB",
 	"genesis_time": "0001-01-01T00:00:00.000Z",
 	"validators": [
 		{
 			"amount": 10,
 			"name": "",
 			"pub_key": [
 				1,
 				"5770B4DD55B3E08B7F5711C48B516347D8C33F47C30C226315D21AA64E0DFF2E"
 			]
 		}
 	]
 }
 ```

 And the `priv_validator.json`:

 ```
 {
 	"address": "4F4D895F882A18E1D1FC608D102601DA8D3570E5",
 	"last_height": 0,
 	"last_round": 0,
 	"last_signature": null,
 	"last_signbytes": "",
 	"last_step": 0,
 	"priv_key": [
 		1,
 		"F9FA3CD435BDAE54D0BCA8F1BC289D718C23D855C6DB21E8543F5E4F457E62805770B4DD55B3E08B7F5711C48B516347D8C33F47C30C226315D21AA64E0DFF2E"
 	],
 	"pub_key": [
 		1,
 		"5770B4DD55B3E08B7F5711C48B516347D8C33F47C30C226315D21AA64E0DFF2E"
 	]
 }
 ```

 The `priv_validator.json` actually contains a private key, and should be kept absolutely secret,
 but for now we work with the plain text.
 Note the `last_` fields, which prevent us from signing conflicting messages.

 Note also that the `pub_key` (the public key) in the `priv_validator.json` is also present in the `genesis.json`.

 The genesis file contains the list of public keys which may participate in the consensus,
 and their corresponding voting power.
 More than 2/3 of the voting power must be active (ie. the corresponding private keys must be producing signatures)
 for the consensus to make progress.
 In our case, the genesis file contains the public key of our `priv_validator.json`,
 so a tendermint node started with the default root directory will be able to make new blocks,
 as we've already seen.

 If we want to add more nodes to the network, we have two choices:
 we can add a new validator node, who will also participate in the consensus
 by proposing blocks and voting on them,
 or we can add a new non-validator node, who will not participate directly,
 but will verify and keep up with the consensus protocol.

 ### Peers

 To connect to peers on start-up, specify them in the `config.toml` or on the command line.

 For instance,

 ```
 tendermint node --p2p.seeds "1.2.3.4:46656,5.6.7.8:46656"
 ```

 Alternatively, you can use the `/dial_seeds` endpoint of the RPC to specify peers for a running node to connect to:

 ```
 curl --data-urlencode "seeds=[\"1.2.3.4:46656\",\"5.6.7.8:46656\"]" localhost:46657/dial_seeds
 ```

 Additionally, the peer-exchange protocol can be enabled using the `--pex` flag,
 though this feature is still under development (beware of dragons!).
 If `--pex` is enabled, peers will gossip about known peers and form a more resilient network.


 ### Adding a Non-Validator

 Adding a non-validator is simple. Just copy the original `genesis.json` to `~/.tendermint` on the new machine
 and start the node, specifying seeds as necessary.
 If no seeds are specified, the node won't make any blocks, because it's not a validator,
 and it won't hear about any blocks, because it's not connected to the other peer.

 ### Adding a Validator

 The easiest way to add new validators is to do it in the `genesis.json`, before starting the network.
 For instance, we could make a new `priv_validator.json`, and copy it's `pub_key` into the above genesis.

 We can generate a new `priv_validator.json` with the command:

 ```
 tendermint gen_validator
 ```

 Now we can update our genesis file. For instance, if the new `priv_validator.json` looks like:

 ```
 {
        "address": "AC379688105901436A34A65F185C115B8BB277A1",
        "last_height": 0,
        "last_round": 0,
        "last_signature": null,
        "last_signbytes": "",
        "last_step": 0,
        "priv_key": [
                1,
                "0D2ED337D748ADF79BE28559B9E59EBE1ABBA0BAFE6D65FCB9797985329B950C8F2B5AACAACC9FCE41881349743B0CFDE190DF0177744568D4E82A18F0B7DF94"
        ],
        "pub_key": [
                1,
                "8F2B5AACAACC9FCE41881349743B0CFDE190DF0177744568D4E82A18F0B7DF94"
        ]
 }
 ```

 then the new `genesis.json` will be:

 ```
 {
 	"app_hash": "",
 	"chain_id": "test-chain-HZw6TB",
 	"genesis_time": "0001-01-01T00:00:00.000Z",
 	"validators": [
 		{
 			"amount": 10,
 			"name": "",
 			"pub_key": [
 				1,
 				"5770B4DD55B3E08B7F5711C48B516347D8C33F47C30C226315D21AA64E0DFF2E"
 			]
 		},
 		{
 			"amount": 10,
 			"name": "",
 			"pub_key": [
 				1,
 				"8F2B5AACAACC9FCE41881349743B0CFDE190DF0177744568D4E82A18F0B7DF94"
 			]
 		}
 	]
 }
 ```

 Update the `genesis.json` in `~/.tendermint`. Copy the genesis file and the new `priv_validator.json`
 to the `~/.tendermint` on a new machine.

 Now run `tendermint node` on both machines, and use either `--p2p.seeds` or the `/dial_seeds` to get them to peer up.
 They should start making blocks, and will only continue to do so so long as both of them are online.

 To make a Tendermint network that can tolerate one of the validators failing, you need at least four validator nodes.

 Updating validators in a live network is supported as of v0.8, but it must be explicitly programmed by the application developer.
 See the [application developers guide](/docs/guides/app-development#Handshake) for more details.

 ### Local Network

 To run a network locally, say on a single machine, you must change the `_laddr` fields in the `config.toml` (or using the flags)
 so that the listening addresses of the various sockets don't conflict.
 Additionally, you must set `addrbook_strict=false` in the `config.toml`,
 otherwise Tendermint's p2p library will deny making connections to peers with the same IP address.

 ## More

 Got a couple nodes talking to each other using the dummy app?
 Try a more sophisticated app like [Ethermint](https://github.com/tendermint/ethermint),
 or learn more about building your own in the [Application Developer's Guide](/docs/guides/app-development).

--- a/docs/index.md
+++ b/docs/index.md
@ -0,0 +1,28 @@
 # Documentation

 If you're new here, start with the [Tendermint Intro](/intro).

 To start building an ABCI application and integrating with Tendermint, 
 see the [App Development](/docs/guides/app-development)
 and [App Architecture](/docs/guides/app-architecture) guides.

 To learn more about running the Tendermint software, see the [Using Tendermint Guide](/docs/guides/using-tendermint).

 To learn more about Tendermint's various pieces, check out the [Documentation](/docs).
 For a deeper dive, see [this thesis](https://atrium.lib.uoguelph.ca/xmlui/handle/10214/9769). 
 There is also the [original whitepaper](https://tendermint.com/static/docs/tendermint.pdf), though it is now quite outdated.
 You might also be interested in the [Cosmos Whitepaper](https://cosmos.network/whitepaper),
 which describes Tendermint, ABCI, and how to build a scalable, heterogeneous, cryptocurrency network.

 For details on how the software has changed, and what changes are in store, see the [Changelog](https://github.com/tendermint/tendermint/blob/master/CHANGELOG.md) and the [Roadmap](https://github.com/tendermint/tendermint/blob/master/roadmap.md).

 If you're interested in contributing, see our [Contributor Guidelines](https://github.com/tendermint/tendermint/blob/master/CONTRIBUTING.md)

 The Tendermint [Software Ecosystem](/ecosystem) contains many example applications and related software built by the Tendermint team and others. 
 Check it out for some motivation and inspiration!

 See our [Community](/community) page for more ways to collaborate.

 You can also [get in touch with the team](/contact).

 Most importantly, enjoy!
--- a/docs/specs/block-structure.md
+++ b/docs/specs/block-structure.md
@ -0,0 +1,176 @@
 # Block Structure

 The tendermint consensus engine records all agreements by a supermajority of
 nodes into a blockchain, which is replicated among all nodes.  This blockchain
 is accessible via various rpc endpoints, mainly `/block?height=` to get the full
 block, as well as `/blockchain?minHeight=_&maxHeight=_` to get a list of headers.
 But what exactly is stored in these blocks?

 ### Block

 A [Block](/docs/specs/tendermint-types#Block) contains:

 * a [Header](#header) contains merkle hashes for various chain states
 * the [Data]((/docs/specs/tendermint-types#Data) is all transactions which are to be processed
 * the [LastCommit](#commit) > 2/3 signatures for the last block

 The signatures returned along with block `H` are those validating block `H-1`.
 This can be a little confusing, but we must also consider that the
 [Header](/docs/specs/tendermint-types#Header) also contains the `LastCommitHash`.
 It would be impossible for a Header to include the commits that sign it, as it
 would cause an infinite loop here. But when we get block `H`, we find
 `Header.LastCommitHash`, which must match the hash of `LastCommit`.

 ### Header

 The [Header](/docs/specs/tendermint-types#Header) contains lots of information (follow
 link for up-to-date info).  Notably, it maintains the `Height`, the `LastBlockID`
 (to make it a chain), and hashes of the data, the app state, and the validator set.
 This is important as the only item that is signed by the validators is the `Header`,
 and all other data must be validated against one of the merkle hashes in the `Header`.

 The `DataHash` can provide a nice check on the [Data](/docs/specs/tendermint-types#Data)
 returned in this same block. If you are subscribed to new blocks, via tendermint RPC, in order to display or process the new transactions
 you should at least validate that the `DataHash` is valid.
 If it is important to verify autheniticity, you must wait for the `LastCommit` from the next block to make sure the block header (including `DataHash`) was properly signed.

 The `ValidatorHash` contains a hash of the current
 [Validators](/docs/specs/tendermint-types#Validator). Tracking all changes in the
 validator set is complex, but a client can quickly compare this hash
 with the [hash of the currently known validators](/docs/specs/tendermint-types#ValidatorSet.Hash)
 to see if there have been changes.

 The `AppHash` serves as the basis for validating any merkle proofs that come
 from the [ABCI application](https://github.com/tendermint/abci). It represents
 the state of the actual application, rather that the state of the blockchain
 itself. This means it's necessary in order to perform any business logic,
 such as verifying and account balance.

 **Note** After the transactions are committed to a block, they still need to
 be processed in a separate step, which happens between the blocks. If you
 find a given transaction in the block at height `H`, the effects of running
 that transaction will be first visible in the `AppHash` from the block
 header at height `H+1`.

 Like the `LastCommit` issue, this is a requirement of the
 immutability of the block chain, as the application only applies transactions
 *after* they are commited to the chain.

 ### Commit

 The [Commit](/docs/specs/tendermint-types#Commit) contains a set of
 [Votes](/docs/specs/tendermint-types#Vote) that were made by the validator set to
 reach consensus on this block. This is the key to the security in any PoS
 system, and actually no data that cannot be traced back to a block header
 with a valid set of Votes can be trusted. Thus, getting the Commit data
 and verifying the votes is extremely important.

 As mentioned above, in order to find the `precommit votes` for block header `H`,
 we need to query block `H+1`.  Then we need to check the votes, make sure they
 really are for that block, and properly formatted. Much of this code is implemented
 in Go in the [light-client](https://github.com/tendermint/light-client) package,
 especially [Node.SignedHeader](https://github.com/tendermint/light-client/blob/develop/rpc/node.go#L117).
 If you look at the code, you will notice that we need to provide the `chainID`
 of the blockchain in order to properly calculate the votes.  This is to protect
 anyone from swapping votes between chains to fake (or frame) a validator.
 Also note that this `chainID` is in the `genesis.json` from _Tendermint_,
 not the `genesis.json` from the basecoin app ([that is a different chainID...](https://github.com/tendermint/basecoin/issues/32)).

 Once we have those votes,
 and we calculated the proper [sign bytes](/docs/specs/tendermint-types#Vote.WriteSignBytes)
 using the chainID and a [nice helper function](/docs/specs/tendermint-types#SignBytes),
 we can verify them.  The light client is responsible for maintaining a set of
 validators that we trust.  Each vote only stores the validators `Address`, as well
 as the `Signature`. Assuming we have a local copy of the trusted validator set,
 we can look up the `Public Key` of the validator given its `Address`, then
 verify that the `Signature` matches the `SignBytes` and `Public Key`.
 Then we sum up the total voting power of all validators, whose votes fulfilled
 all these stringent requirements. If the total number of voting power for a single block is greater
 than 2/3 of all voting power, then we can finally trust the
 block header, the AppHash, and the proof we got from the ABCI application.

 To make this a bit more concrete, you can take a look at a
 [StaticCertifier](https://github.com/tendermint/light-client/blob/develop/rpc/certifier.go#L23)
 to see how this logic works, given a static set of validators. And you can see
 an example of how one can perform the entire chain of validation in the
 proxy server [proof call](https://github.com/tendermint/light-client/blob/develop/proxy/viewer.go#L61)
 or the [test code for auditing](https://github.com/tendermint/light-client/blob/develop/rpc/tests/node_test.go#L102).

 #### Vote Sign Bytes
 The `sign-bytes` of a vote is produced by taking a [`stable-json`](https://github.com/substack/json-stable-stringify)-like deterministic JSON [`wire`](/docs/specs/wire-protocol) encoding of the vote (excluding the `Signature` field), and wrapping it with `{"chain_id":"my_chain","vote":...}`.

 For example, a precommit vote might have the following `sign-bytes`:

 ```json
 {"chain_id":"my_chain","vote":{"block_hash":"611801F57B4CE378DF1A3FFF1216656E89209A99","block_parts_header":{"hash":"B46697379DBE0774CC2C3B656083F07CA7E0F9CE","total":123},"height":1234,"round":1,"type":2}}
 ```

 ### Block Hash

 The [block hash](/docs/specs/tendermint-types#Block.Hash) is the [Simple Tree hash](Merkle-Trees#simple-tree-with-dictionaries) of the fields of the block `Header` encoded as a list of `KVPair`s.

 ### Transaction

 A transaction is any sequence of bytes.  It is up to your [ABCI](https://github.com/tendermint/abci) application to accept or reject transactions.

 ### BlockID

 Many of these data structures refer to the [BlockID](/docs/specs/tendermint-types#BlockID),
 which is the `BlockHash` (hash of the block header, also referred to by the next block)
 along with the `PartSetHeader`.  The `PartSetHeader` is explained below and is used internally
 to orchestrate the p2p propogation.  For clients, it is basically opaque bytes,
 but they must match for all votes.

 ### PartSetHeader

 The [PartSetHeader](/docs/specs/tendermint-types#PartSetHeader) contains the total number of pieces in a [PartSet](/docs/specs/tendermint-types#PartSet), and the Merkle root hash of those pieces.

 ### PartSet

 PartSet is used to split a byteslice of data into parts (pieces) for transmission.
 By splitting data into smaller parts and computing a Merkle root hash on the list,
 you can verify that a part is legitimately part of the complete data, and the
 part can be forwarded to other peers before all the parts are known.  In short,
 it's a fast way to securely propagate a large chunk of data (like a block) over a gossip network.

 PartSet was inspired by the LibSwift project.

 Usage:

 ```Go
 data := RandBytes(2 << 20) // Something large

 partSet := NewPartSetFromData(data)
 partSet.Total()     // Total number of 4KB parts
 partSet.Count()     // Equal to the Total, since we already have all the parts
 partSet.Hash()      // The Merkle root hash
 partSet.BitArray()  // A BitArray of partSet.Total() 1's

 header := partSet.Header() // Send this to the peer
 header.Total        // Total number of parts
 header.Hash         // The merkle root hash

 // Now we'll reconstruct the data from the parts
 partSet2 := NewPartSetFromHeader(header)
 partSet2.Total()    // Same total as partSet.Total()
 partSet2.Count()    // Zero, since this PartSet doesn't have any parts yet.
 partSet2.Hash()     // Same hash as in partSet.Hash()
 partSet2.BitArray() // A BitArray of partSet.Total() 0's

 // In a gossip network the parts would arrive in arbitrary order, perhaps
 // in response to explicit requests for parts, or optimistically in response
 // to the receiving peer's partSet.BitArray().
 for !partSet2.IsComplete() {
    part := receivePartFromGossipNetwork()
    added, err := partSet2.AddPart(part)
    if err != nil {
    // A wrong part,
        // the merkle trail does not hash to partSet2.Hash()
    } else if !added {
        // A duplicate part already received
    }
 }

 data2, _ := ioutil.ReadAll(partSet2.GetReader())
 bytes.Equal(data, data2) // true
 ```
--- a/docs/specs/byzantine-consensus-algorithm.md
+++ b/docs/specs/byzantine-consensus-algorithm.md
@ -0,0 +1,195 @@
 # Byzantine Consensus Algorithm

 _The draft 0.6 whitepaper is outdated. The new algorithm is detailed below.  See [revisions](#revisions)_

 ## Terms
 - The network is composed of optionally connected _nodes_.  Nodes directly connected to a particular node are called _peers_.
 - The consensus process in deciding the next block (at some _height_ `H`) is composed of one or many _rounds_.
 - `NewHeight`, `Propose`, `Prevote`, `Precommit`, and `Commit` represent state machine states of a round. (aka `RoundStep` or just "step").
 - A node is said to be _at_ a given height, round, and step, or at `(H,R,S)`, or at `(H,R)` in short to omit the step.
 - To _prevote_ or _precommit_ something means to broadcast a [prevote vote](/docs/specs/block-structure#vote) or [precommit vote](/docs/specs/block-structure#precommit-vote) for something.
 - A vote _at_ `(H,R)` is a vote signed with the bytes for `H` and `R` included in its [`sign-bytes`](/docs/specs/block-structure#vote-sign-bytes).
 - _+2/3_ is short for "more than 2/3"
 - _1/3+_ is short for "1/3 or more"
 - A set of +2/3 of prevotes for a particular block or `<nil>` at `(H,R)` is called a _proof-of-lock-change_ or _PoLC_ for short.

 ## State Machine Overview

 At each height of the blockchain a round-based protocol is run to determine
 the next block. Each round is composed of three _steps_ (`Propose`, `Prevote`, and
 `Precommit`), along with two special steps `Commit` and `NewHeight`.

 In the optimal scenario, the order of steps is:
 ```
 NewHeight -> (Propose -> Prevote -> Precommit)+ -> Commit -> NewHeight ->...
 ```

 The sequence `(Propose -> Prevote -> Precommit)` is called a _round_. There may be more than one round required to commit a block at a given height.  Examples for why more rounds may be required include:

 - The designated proposer was not online.
 - The block proposed by the designated proposer was not valid.
 - The block proposed by the designated proposer did not propagate in time.
 - The block proposed was valid, but +2/3 of prevotes for the proposed block were not received in time for enough validator nodes by the time they reached the `Precommit` step.  Even though +2/3 of prevotes are necessary to progress to the next step, at least one validator may have voted `<nil>` or maliciously voted for something else.
 - The block proposed was valid, and +2/3 of prevotes were received for enough nodes, but +2/3 of precommits for the proposed block were not received for enough validator nodes.

 Some of these problems are resolved by moving onto the next round & proposer.  Others are resolved by increasing certain round timeout parameters over each successive round.

 ## State Machine Diagram
 ```
                            +-------------------------------------+
                            v                                     |(Wait til `CommmitTime+timeoutCommit`)
                      +-----------+                         +-----+-----+
         +----------> |  Propose  +--------------+          | NewHeight |
         |            +-----------+              |          +-----------+
         |                                       |                ^
         |(Else, after timeoutPrecommit)         v                |
   +-----+-----+                           +-----------+          |
   | Precommit |  <------------------------+  Prevote  |          |
   +-----+-----+                           +-----------+          |
         |(When +2/3 Precommits for block found)                  |
         v                                                        |
   +--------------------------------------------------------------------+
   |  Commit                                                            |
   |                                                                    |
   |  * Set CommitTime = now;                                           |
   |  * Wait for block, then stage/save/commit block;                   |
   +--------------------------------------------------------------------+
 ```

 ## Background Gossip

 A node may not have a corresponding validator private key, but it nevertheless plays an active role in the consensus process by relaying relevant meta-data, proposals, blocks, and votes to its peers.  A node that has the private keys of an active validator and is engaged in signing votes is called a _validator-node_.  All nodes (not just validator-nodes) have an associated state (the current height, round, and step) and work to make progress.

 Between two nodes there exists a `Connection`, and multiplexed on top of this connection are fairly throttled `Channel`s of information.  An epidemic gossip protocol is implemented among some of these channels to bring peers up to speed on the most recent state of consensus.  For example,

 - Nodes gossip `PartSet` parts of the current round's proposer's proposed block.  A LibSwift inspired algorithm is used to quickly broadcast blocks across the gossip network.
 - Nodes gossip prevote/precommit votes.  A node NODE_A that is ahead of NODE_B can send NODE_B prevotes or precommits for NODE_B's current (or future) round to enable it to progress forward.
 - Nodes gossip prevotes for the proposed PoLC (proof-of-lock-change) round if one is proposed.
 - Nodes gossip to nodes lagging in blockchain height with block [commits](/docs/specs/block-structure/commit) for older blocks.
 - Nodes opportunistically gossip `HasVote` messages to hint peers what votes it already has.
 - Nodes broadcast their current state to all neighboring peers. (but is not gossiped further)

 There's more, but let's not get ahead of ourselves here.

 ## Proposals

 A proposal is signed and published by the designated proposer at each round.  The proposer is chosen by a deterministic and non-choking round robin selection algorithm that selects proposers in proportion to their voting power. (see [implementation](https://github.com/tendermint/tendermint/blob/develop/types/validator_set.go#L49))

 A proposal at `(H,R)` is composed of a block and an optional latest `PoLC-Round < R` which is included iff the proposer knows of one.  This hints the network to allow nodes to unlock (when safe) to ensure the liveness property.

 ## State Machine Spec

 ### Propose Step (height:H,round:R)
 Upon entering `Propose`:
 - The designated proposer proposes a block at `(H,R)`.

 The `Propose` step ends:
 - After `timeoutProposeR` after entering `Propose`.                --> goto `Prevote(H,R)`
 - After receiving proposal block and all prevotes at `PoLC-Round`. --> goto `Prevote(H,R)`
 - After [common exit conditions](#common-exit-conditions)

 ### Prevote Step (height:H,round:R)
 Upon entering `Prevote`, each validator broadcasts its prevote vote.

 - First, if the validator is locked on a block since `LastLockRound` but now has a PoLC for something else at round `PoLC-Round` where `LastLockRound < PoLC-Round < R`, then it unlocks.
 - If the validator is still locked on a block, it prevotes that.
 - Else, if the proposed block from `Propose(H,R)` is good, it prevotes that.
 - Else, if the proposal is invalid or wasn't received on time, it prevotes `<nil>`.

 The `Prevote` step ends:
 - After +2/3 prevotes for a particular block or `<nil>`.           --> goto `Precommit(H,R)`
 - After `timeoutPrevote` after receiving any +2/3 prevotes.        --> goto `Precommit(H,R)`
 - After [common exit conditions](#common-exit-conditions)

 ### Precommit Step (height:H,round:R)
 Upon entering `Precommit`, each validator broadcasts its precommit vote.
 - If the validator has a PoLC at `(H,R)` for a particular block `B`, it (re)locks (or changes lock to) and precommits `B` and sets `LastLockRound = R`.
 - Else, if the validator has a PoLC at `(H,R)` for `<nil>`, it unlocks and precommits `<nil>`.
 - Else, it keeps the lock unchanged and precommits `<nil>`.

 A precommit for `<nil>` means "I didn’t see a PoLC for this round, but I did get +2/3 prevotes and waited a bit".

 The Precommit step ends:
 - After +2/3 precommits for `<nil>`.                               --> goto `Propose(H,R+1)`
 - After `timeoutPrecommit` after receiving any +2/3 precommits.    --> goto `Propose(H,R+1)`
 - After [common exit conditions](#common-exit-conditions)

 #### common exit conditions
 - After +2/3 precommits for a particular block.                    --> goto `Commit(H)`
 - After any +2/3 prevotes received at `(H,R+x)`.                   --> goto `Prevote(H,R+x)`
 - After any +2/3 precommits received at `(H,R+x)`.                 --> goto `Precommit(H,R+x)`

 ### Commit Step (height:H)
 - Set `CommitTime = now()`
 - Wait until block is received.                                    --> goto `NewHeight(H+1)`

 ### NewHeight Step (height:H)
 - Move `Precommits` to `LastCommit` and increment height.
 - Set `StartTime = CommitTime+timeoutCommit`
 - Wait until `StartTime` to receive straggler commits.             --> goto `Propose(H,0)`

 ## Proofs

 ### Proof of Safety
 Assume that at most -1/3 of the voting power of validators is byzantine.  If a validator commits block `B` at round `R`, it's because it saw +2/3 of precommits at round `R`. This implies that 1/3+ of honest nodes are still locked at round `R' > R`.  These locked validators will remain locked until they see a PoLC at `R' > R`, but this won't happen because 1/3+ are locked and honest, so at most -2/3 are available to vote for anything other than `B`.

 ### Proof of Liveness
 If 1/3+ honest validators are locked on two different blocks from different rounds, a proposers' `PoLC-Round` will eventually cause nodes locked from the earlier round to unlock.  Eventually, the designated proposer will be one that is aware of a PoLC at the later round.  Also, `timeoutProposalR` increments with round `R`, while the size of a proposal are capped, so eventually the network is able to "fully gossip" the whole proposal (e.g. the block & PoLC).

 ### Proof of Fork Accountability
 Define the JSet (justification-vote-set) at height `H` of a validator `V1` to be all the votes signed by the validator at `H` along with justification PoLC prevotes for each lock change.  For example, if `V1` signed the following precommits: `Precommit(B1 @ round 0)`, `Precommit(<nil> @ round 1)`, `Precommit(B2 @ round 4)` (note that no precommits were signed for rounds 2 and 3, and that's ok), `Precommit(B1 @ round 0)` must be justified by a PoLC at round 0, and `Precommit(B2 @ round 4)` must be justified by a PoLC at round 4; but the precommit for `<nil>` at round 1 is not a lock-change by definition so the JSet for `V1` need not include any prevotes at round 1, 2, or 3 (unless `V1` happened to have prevoted for those rounds).

 Further, define the JSet at height `H` of a set of validators `VSet` to be the union of the JSets for each validator in `VSet`.  For a given commit by honest validators at round `R` for block `B` we can construct a JSet to justify the commit for `B` at `R`.
 We say that a JSet _justifies_ a commit at `(H,R)` if all the committers (validators in the commit-set) are each justified in the JSet with no duplicitous vote signatures (by the committers).

 - **Lemma**: When a fork is detected by the existence of two conflicting [commits](/docs/specs/validators#commiting-a-block), the union of the JSets for both commits (if they can be compiled) must include double-signing by at least 1/3+ of the validator set. **Proof**: The commit cannot be at the same round, because that would immediately imply double-signing by 1/3+.  Take the union of the JSets of both commits.  If there is no double-signing by at least 1/3+ of the validator set in the union, then no honest validator could have precommitted any different block after the first commit.  Yet, +2/3 did.  Reductio ad absurdum.

 As a corollary, when there is a fork, an external process can determine the blame by requiring each validator to justify all of its round votes.  Either we will find 1/3+ who cannot justify at least one of their votes, and/or, we will find 1/3+ who had double-signed.

 ### Alternative algorithm

 Alternatively, we can take the JSet of a commit to be the "full commit".  That is, if light clients and validators do not consider a block to be committed unless the JSet of the commit is also known, then we get the desirable property that if there ever is a fork (e.g. there are two conflicting "full commits"), then 1/3+ of the validators are immediately punishable for double-signing.

 There are many ways to ensure that the gossip network efficiently share the JSet of a commit.  One solution is to add a new message type that tells peers that this node has (or does not have) a +2/3 majority for B (or <nil>) at (H,R), and a bitarray of which votes contributed towards that majority.  Peers can react by responding with appropriate votes.

 We will implement such an algorithm for the next iteration of the Tendermint consensus protocol.

 Other potential improvements include adding more data in votes such as the last known PoLC round that caused a lock change, and the last voted round/step (or, we may require that validators not skip any votes).  This may make JSet verification/gossip logic easier to implement.

 ### Censorship Attacks

 Due to the definition of a block [commit](/docs/specs/validators#commiting-a-block), any 1/3+ coalition of validators can halt the blockchain by not broadcasting their votes.  Such a coalition can also censor particular transactions by rejecting blocks that include these transactions, though this would result in a significant proportion of block proposals to be rejected, which would slow down the rate of block commits of the blockchain, reducing its utility and value.  The malicious coalition might also broadcast votes in a trickle so as to grind blockchain block commits to a near halt, or engage in any combination of these attacks.

 If a global active adversary were also involved, it can partition the network in such a way that it may appear that the wrong subset of validators were responsible for the slowdown.  This is not just a limitation of Tendermint, but rather a limitation of all consensus protocols whose network is potentially controlled by an active adversary.

 ### Overcoming Forks and Censorship Attacks

 For these types of attacks, a subset of the validators through external means
 should coordinate to sign a reorg-proposal that chooses a fork (and any evidence
 thereof) and the initial subset of validators with their signatures. Validators
 who sign such a reorg-proposal forego its collateral on all other forks.
 Clients should verify the signatures on the reorg-proposal, verify any evidence,
 and make a judgement or prompt the end-user for a decision.  For example, a
 phone wallet app may prompt the user with a security warning, while a
 refrigerator may accept any reorg-proposal signed by +½ of the original
 validators.

 No non-synchronous Byzantine fault-tolerant algorithm can come to consensus when
 ⅓+ of validators are dishonest, yet a fork assumes that ⅓+ of validators have
 already been dishonest by double-signing or lock-changing without justification.
 So, signing the reorg-proposal is a coordination problem that cannot be solved
 by any non-synchronous protocol (i.e. automatically, and without making
 assumptions about the reliability of the underlying network). It must be
 provided by means external to the weakly-synchronous Tendermint consensus
 algorithm.  For now, we leave the problem of reorg-proposal coordination to
 human coordination via internet media.  Validators must take care to ensure that
 there are no significant network partitions, to avoid situations where two
 conflicting reorg-proposals are signed.

 Assuming that the external coordination medium and protocol is robust, it follows that forks are less of a concern than [censorship attacks](#censorship-attacks).

 ### Revisions

 #### 0.6 -> 0.7 (current)
 1. Reduced the minimum number of signature steps from 3 to 2 by removing the "commit" vote and step.
 2. The protocol is more asynchronous: instead of each round taking a predetermined duration of time, each step of a round progresses after +2/3 of the step's votes are found and a timeout is reached, or immediately after +2/3 of matching votes (e.g. a PoLC for prevotes, or a commit for precommits).
--- a/docs/specs/configuration.md
+++ b/docs/specs/configuration.md
@ -0,0 +1,36 @@
 # Configuration

 TendermintCore can be configured via a TOML file in `$TMHOME/config.toml`.
 Some of these parameters can be overridden by command-line flags.

 ### Config parameters

 The main config parameters are defined [here](https://github.com/tendermint/tendermint/blob/master/config/config.go).

 * `abci`: ABCI transport (socket | grpc). _Default_: `socket`
 * `db_backend`: Database backend for the blockchain and TendermintCore state.  `leveldb` or `memdb`.  _Default_: `"leveldb"`
 * `db_dir`: Database dir.  _Default_: `"$TMHOME/data"`
 * `fast_sync`: Whether to sync faster from the block pool.  _Default_: `true`
 * `genesis_file`: The location of the genesis file.  _Default_: `"$TMHOME/genesis.json"`
 * `log_level`: _Default_: `"state:info,*:error"`
 * `moniker`: Name of this node.  _Default_: `"anonymous"`
 * `priv_validator_file`: Validator private key file.  _Default_: `"$TMHOME/priv_validator.json"`
 * `prof_laddr`: Profile listen address. _Default_: `""`
 * `proxy_app`: The ABCI app endpoint.  _Default_: `"tcp://127.0.0.1:46658"`

 * `consensus.max_block_size_txs`: Maximum number of block txs.  _Default_: `10000`
 * `consensus.timeout_*`: Various consensus timeout parameters **TODO**
 * `consensus.wal_file`: Consensus state WAL.  _Default_: `"$TMHOME/data/cswal"`
 * `consensus.wal_light`: Whether to use light-mode for Consensus state WAL.  _Default_: `false`

 * `mempool.*`: Various mempool parameters **TODO**

 * `p2p.addr_book_file`: Peer address book.  _Default_: `"$TMHOME/addrbook.json"`.  **NOT USED**
 * `p2p.laddr`: Node listen address. (0.0.0.0:0 means any interface, any port). _Default_: `"0.0.0.0:46656"`
 * `p2p.pex`: Enable Peer-Exchange (dev feature). _Default_: `false`
 * `p2p.seeds`: Comma delimited host:port seed nodes.  _Default_: `""`
 * `p2p.skip_upnp`: Skip UPNP detection.  _Default_: `false`

 * `rpc.grpc_laddr`: GRPC listen address (BroadcastTx only). Port required. _Default_: `""`
 * `rpc.laddr`: RPC listen address. Port required. _Default_: `"0.0.0.0:46657"`
 * `rpc.unsafe`: Enabled unsafe rpc methods. _Default_: `true`
--- a/docs/specs/fast-sync.md
+++ b/docs/specs/fast-sync.md
@ -0,0 +1,13 @@
 # Fast Sync

 ## Background

 In a proof of work blockchain, syncing with the chain is the same process as staying up-to-date with the consensus: download blocks, and look for the one with the most total work. In proof-of-stake, the consensus process is more complex, as it involves rounds of communication between the nodes to determine what block should be committed next. Using this process to sync up with the blockchain from scratch can take a very long time. It's much faster to just download blocks and check the merkle tree of validators than to run the real-time consensus gossip protocol. 

 ## Fast Sync

 To support faster syncing, tendermint offers a `fast-sync` mode, which is enabled by default, and can be toggled in the `config.toml` or via `--fast_sync=false`. 

 In this mode, the tendermint daemon will sync hundreds of times faster than if it used the real-time consensus process. Once caught up, the daemon will switch out of fast sync and into the normal consensus mode. After running for some time, the node is considered `caught up` if it has at least one peer and it's height is at least as high as the max reported peer height. See [the IsCaughtUp method](https://github.com/tendermint/tendermint/blob/master/blockchain/pool.go#L128).

 If we're lagging sufficiently, we should go back to fast syncing, but this is an open issue: https://github.com/tendermint/tendermint/issues/129
--- a/docs/specs/genesis.md
+++ b/docs/specs/genesis.md
@ -0,0 +1,60 @@
 # Genesis

 The genesis.json file in `$TMROOT` defines the initial TendermintCore state upon genesis of the blockchain ([see definition](https://github.com/tendermint/tendermint/blob/master/types/genesis.go)).

 NOTE: This does not (yet) specify the application state (e.g. initial distribution of tokens). Currently we leave it up to the application to load the initial application genesis state.  In the future, we may include genesis SetOption messages that get passed from TendermintCore to the app upon genesis.

 ### Fields

 * `genesis_time`: Official time of blockchain start.
 * `chain_id`: ID of the blockchain.  This must be unique for every blockchain.  If your testnet blockchains do not have unique chain IDs, you will have a bad time.
 * `validators`:
  * `pub_key`: The first element specifies the pub_key type. 1 == Ed25519.  The second element are the pubkey bytes.
  * `amount`: The validator's voting power.
  * `name`: Name of the validator (optional).
 * `app_hash`: The expected application hash (as returned by the `Commit` ABCI message) upon genesis.  If the app's hash does not match, a warning message is printed.

 ### Sample genesis.json

 This example is from the Basecoin mintnet example ([link to file](https://github.com/tendermint/mintnet/blob/master/examples/basecoin/mach1/core/genesis.json)).
 ```json
 {
  "genesis_time": "2016-02-05T06:02:31.526Z",
  "chain_id": "chain-tTH4mi",
  "validators": [
    {
      "pub_key": [
        1,
        "9BC5112CB9614D91CE423FA8744885126CD9D08D9FC9D1F42E552D662BAA411E"
      ],
      "amount": 1,
      "name": "mach1"
    },
    {
      "pub_key": [
        1,
        "F46A5543D51F31660D9F59653B4F96061A740FF7433E0DC1ECBC30BE8494DE06"
      ],
      "amount": 1,
      "name": "mach2"
    },
    {
      "pub_key": [
        1,
        "0E7B423C1635FD07C0FC3603B736D5D27953C1C6CA865BB9392CD79DE1A682BB"
      ],
      "amount": 1,
      "name": "mach3"
    },
    {
      "pub_key": [
        1,
        "4F49237B9A32EB50682EDD83C48CE9CDB1D02A7CFDADCFF6EC8C1FAADB358879"
      ],
      "amount": 1,
      "name": "mach4"
    }
  ],
  "app_hash": "15005165891224E721CB664D15CB972240F5703F"
 }
 ```
--- a/docs/specs/light-client-protocol.md
+++ b/docs/specs/light-client-protocol.md
@ -0,0 +1,18 @@
 # Light Client Protocol

 Light clients are an important part of the complete blockchain system for most applications.  Tendermint provides unique speed and security properties for light client applications.

 See our developing [light-client repository](github.com/tendermint/light-client).

 ## Overview

 The objective of the light client protocol is to get a [commit](/docs/specs/validators#committing-a-block) for a recent [block hash](/docs/specs/block-structure#block-hash) where the commit includes a majority of signatures from the last known validator set.  From there, all the application state is verifiable with [merkle proofs](/docs/specs/merkle-trees#iavl-tree).

 ### Syncing the Validator Set
 TODO

 ## Properties

 - You get the full collateralized security benefits of Tendermint;  No need to wait for confirmations.
 - You get the full speed benefits of Tendermint;  Transactions commit instantly.
 - You can get the most recent version of the application state non-interactively (without committing anything to the blockchain).  For example, this means that you can get the most recent value of a name from the name-registry without worrying about fork censorship attacks, without posting a commit and waiting for confirmations.  It's fast, secure, and free!
--- a/docs/specs/merkle.md
+++ b/docs/specs/merkle.md
@ -0,0 +1,45 @@
 # Merkle

 For an overview of Merkle trees, see [wikipedia](https://en.wikipedia.org/wiki/Merkle_tree).

 There are two types of Merkle trees used in Tendermint.

 - [`IAVL+ Tree`](#iavl-tree): An immutable self-balancing binary tree for persistent application state
 - [`Simple Tree`](#simple-tree): A simple compact binary tree for a static list of items

 ## IAVL+ Tree

 The purpose of this data structure is to provide persistent storage for key-value pairs (e.g. account state, name-registrar data, and per-contract data) such that a deterministic merkle root hash can be computed.  The tree is balanced using a variant of the [AVL algorithm](http://en.wikipedia.org/wiki/AVL_tree) so all operations are O(log(n)).

 Nodes of this tree are immutable and indexed by its hash.  Thus any node serves as an immutable snapshot which lets us stage uncommitted transactions from the mempool cheaply, and we can instantly roll back to the last committed state to process transactions of a newly committed block (which may not be the same set of transactions as those from the mempool).

 In an AVL tree, the heights of the two child subtrees of any node differ by at most one.  Whenever this condition is violated upon an update, the tree is rebalanced by creating O(log(n)) new nodes that point to unmodified nodes of the old tree.  In the original AVL algorithm, inner nodes can also hold key-value pairs.  The AVL+ algorithm (note the plus) modifies the AVL algorithm to keep all values on leaf nodes, while only using branch-nodes to store keys.  This simplifies the algorithm while minimizing the size of merkle proofs

 In Ethereum, the analog is the [Patricia trie](http://en.wikipedia.org/wiki/Radix_tree).  There are tradeoffs.  Keys do not need to be hashed prior to insertion in IAVL+ trees, so this provides faster iteration in the key space which may benefit some applications.  The logic is simpler to implement, requiring only two types of nodes -- inner nodes and leaf nodes.  The IAVL+ tree is a binary tree, so merkle proofs are much shorter than the base 16 Patricia trie.  On the other hand, while IAVL+ trees provide a deterministic merkle root hash, it depends on the order of updates.  In practice this shouldn't be a problem, since you can efficiently encode the tree structure when serializing the tree contents.

 ## Simple Tree

 For merkelizing smaller static lists, use the Simple Tree.  The transactions and validation signatures of a block are hashed using this simple merkle tree logic.

 If the number of items is not a power of two, the tree will not be full and some leaf nodes will be at different levels. Simple Tree tries to keep both sides of the tree the same size, but the left side may be one greater.

 ```
    Simple Tree with 6 items           Simple Tree with 7 items 
                                                         
               *                                  *             
              / \                                / \            
            /     \                            /     \          
          /         \                        /         \        
        /             \                    /             \      
       *               *                  *               *     
      / \             / \                / \             / \    
     /   \           /   \              /   \           /   \   
    /     \         /     \            /     \         /     \  
   *       h2      *       h5         *       *       *       h6
  / \             / \                / \     / \     / \        
 h0  h1          h3  h4             h0  h1  h2  h3  h4  h5      
 ```

 ### Simple Tree with Dictionaries

 The Simple Tree is used to merkelize a list of items, so to merkelize a (short) dictionary of key-value pairs, encode the dictionary as an ordered list of `KVPair` structs.  The block hash is such a hash derived from all the fields of the block `Header`.  The state hash is similarly derived.
--- a/docs/specs/rpc.md
+++ b/docs/specs/rpc.md
@ -0,0 +1,156 @@
 # RPC

 Tendermint supports the following RPC protocols:

 * URI over HTTP
 * JSONRPC over HTTP
 * JSONRPC over websockets

 Tendermint RPC is build using [our own RPC library](https://github.com/tendermint/tendermint/tree/master/rpc/lib). Documentation and tests for that library could be found at `tendermint/rpc/lib` directory.

 ### Configuration

 Set the `laddr` config parameter under `[rpc]` table in the $TMHOME/config.toml file or the `--rpc.laddr` command-line flag to the desired protocol://host:port setting.  Default: `tcp://0.0.0.0:46657`.

 ### Arguments

 Arguments which expect strings or byte arrays may be passed as quoted strings, like `"abc"` or as `0x`-prefixed strings, like `0x616263`.

 ### URI/HTTP

 Example request:
 ```bash
 curl -s 'http://localhost:46657/broadcast_tx_sync?tx="abc"' | jq .
 ```

 Response:
 ```json
 {
  "error": "",
  "result": {
    "hash": "2B8EC32BA2579B3B8606E42C06DE2F7AFA2556EF",
    "log": "",
    "data": "",
    "code": 0
  },
  "id": "",
  "jsonrpc": "2.0"
 }
 ```

 The first entry in the result-array (`96`) is the method this response correlates with. `96` refers to "ResultTypeBroadcastTx", see [responses.go](https://github.com/tendermint/tendermint/blob/master/rpc/core/types/responses.go) for a complete overview.

 ### JSONRPC/HTTP

 JSONRPC requests can be POST'd to the root RPC endpoint via HTTP (e.g. `http://localhost:46657/`).

 Example request:
 ```json
 {
  "method": "broadcast_tx_sync",
  "jsonrpc": "2.0",
  "params": [ "abc" ],
  "id": "dontcare"
 }
 ```

 ### JSONRPC/websockets

 JSONRPC requests can be made via websocket.  The websocket endpoint is at `/websocket`, e.g. `http://localhost:46657/websocket`.  Asynchronous RPC functions like event `subscribe` and `unsubscribe` are only available via websockets.

 ### Endpoints

 An HTTP Get request to the root RPC endpoint (e.g. `http://localhost:46657`) shows a list of available endpoints.

 ```
 Available endpoints:
 http://localhost:46657/abci_info
 http://localhost:46657/dump_consensus_state
 http://localhost:46657/genesis
 http://localhost:46657/net_info
 http://localhost:46657/num_unconfirmed_txs
 http://localhost:46657/status
 http://localhost:46657/unconfirmed_txs
 http://localhost:46657/unsafe_flush_mempool
 http://localhost:46657/unsafe_stop_cpu_profiler
 http://localhost:46657/validators

 Endpoints that require arguments:
 http://localhost:46657/abci_query?path=_&data=_&prove=_
 http://localhost:46657/block?height=_
 http://localhost:46657/blockchain?minHeight=_&maxHeight=_
 http://localhost:46657/broadcast_tx_async?tx=_
 http://localhost:46657/broadcast_tx_commit?tx=_
 http://localhost:46657/broadcast_tx_sync?tx=_
 http://localhost:46657/commit?height=_
 http://localhost:46657/dial_seeds?seeds=_
 http://localhost:46657/subscribe?event=_
 http://localhost:46657/tx?hash=_&prove=_
 http://localhost:46657/unsafe_start_cpu_profiler?filename=_
 http://localhost:46657/unsafe_write_heap_profile?filename=_
 http://localhost:46657/unsubscribe?event=_
 ```

 ### tx

 Returns a transaction matching the given transaction hash.

 **Parameters**

 1. hash - the transaction hash
 2. prove - include a proof of the transaction inclusion in the block in the result (optional, default: false)

 **Returns**

 - `proof`: the `types.TxProof` object
 - `tx`: `[]byte` - the transaction
 - `tx_result`: the `abci.Result` object
 - `index`: `int` - index of the transaction
 - `height`: `int` - height of the block where this transaction was in

 **Example**

 ```bash
 curl -s 'http://localhost:46657/broadcast_tx_commit?tx="abc"' | jq .
 # {
 #   "error": "",
 #   "result": {
 #     "hash": "2B8EC32BA2579B3B8606E42C06DE2F7AFA2556EF",
 #     "log": "",
 #     "data": "",
 #     "code": 0
 #   },
 #   "id": "",
 #   "jsonrpc": "2.0"
 # }

 curl -s 'http://localhost:46657/tx?hash=0x2B8EC32BA2579B3B8606E42C06DE2F7AFA2556EF' | jq .
 # {
 #   "error": "",
 #   "result": {
 #     "proof": {
 #       "Proof": {
 #         "aunts": []
 #       },
 #       "Data": "YWJjZA==",
 #       "RootHash": "2B8EC32BA2579B3B8606E42C06DE2F7AFA2556EF",
 #       "Total": 1,
 #       "Index": 0
 #     },
 #     "tx": "YWJjZA==",
 #     "tx_result": {
 #       "log": "",
 #       "data": "",
 #       "code": 0
 #     },
 #     "index": 0,
 #     "height": 52
 #   },
 #   "id": "",
 #   "jsonrpc": "2.0"
 # }
 ```

 ### More Examples

 See the various bash tests using curl in `test/`, and examples using the `Go` API in `rpc/client/`.
--- a/docs/specs/secure-p2p.md
+++ b/docs/specs/secure-p2p.md
@ -0,0 +1,33 @@
 # Secure P2P

 The Tendermint p2p protocol uses an authenticated encryption scheme based on the [Station-to-Station Protocol](https://en.wikipedia.org/wiki/Station-to-Station_protocol). The implementation uses [golang's](https://godoc.org/golang.org/x/crypto/nacl/box) [nacl box](http://nacl.cr.yp.to/box.html) for the actual authenticated encryption algorithm.

 Each peer generates an ED25519 key-pair to use as a persistent (long-term) id.

 When two peers establish a TCP connection, they first each generate an ephemeral ED25519 key-pair to use for this session, and send each other their respective ephemeral public keys. This happens in the clear.

 They then each compute the shared secret. The shared secret is the multiplication of the peer's ephemeral private key by the other peer's ephemeral public key. The result is the same for both peers by the magic of [elliptic curves](https://en.wikipedia.org/wiki/Elliptic_curve_cryptography). The shared secret is used as the symmetric key for the encryption algorithm.

 The two ephemeral public keys are sorted to establish a canonical order. Then a 24-byte nonce is generated by concatenating the public keys and hashing them with Ripemd160. Note Ripemd160 produces 20byte hashes, so the nonce ends with four 0s.

 The nonce is used to seed the encryption - it is critical that the same nonce never be used twice with the same private key. For convenience, the last bit of the nonce is flipped, giving us two nonces: one for encrypting our own messages, one for decrypting our peer's.  Which ever peer has the higher public key uses the "bit-flipped" nonce for encryption.

 Now, a challenge is generated by concatenating the ephemeral public keys and taking the SHA256 hash.

 Each peer signs the challenge with their persistent private key, and sends the other peer an AuthSigMsg, containing their persistent public key and the signature. On receiving an AuthSigMsg, the peer verifies the signature.

 The peers are now authenticated.

 All future communications can now be encrypted using the shared secret and the generated nonces, where each nonce is incremented by one each time it is used. The communications maintain Perfect Forward Secrecy, as the persistent key pair was not used for generating secrets - only for authenticating.

 Caveat
 ------
 This system is still vulnerable to a Man-In-The-Middle attack if the persistent public key of the remote node is not known in advance. The only way to mitigate this is with a public key authentication system, such as the Web-of-Trust or Certificate Authorities. In our case, we can use the blockchain itself as a certificate authority to ensure that we are connected to at least one validator.

 Links
 ------

 - [Implementation](https://github.com/tendermint/go-p2p/blob/master/secret_connection.go#L49)
 - [Original STS paper by Whitfield Diffie, Paul C. van Oorschot and Michael J. Wiener](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.216.6107&rep=rep1&type=pdf)
 - [Further work on secret handshakes](https://dominictarr.github.io/secret-handshake-paper/shs.pdf)

--- a/docs/specs/tendermint-types.md
+++ b/docs/specs/tendermint-types.md
@ -0,0 +1,4 @@


 # types
 see the [godoc version](https://godoc.org/github.com/tendermint/tendermint/types)
--- a/docs/specs/validators.md
+++ b/docs/specs/validators.md
@ -0,0 +1,21 @@
 # Validators

 Validators are responsible for committing new blocks in the blockchain.
 These validators participate in the consensus protocol by broadcasting _votes_ which contain cryptographic signatures signed by each validator's public key.

 Some Proof-of-Stake consensus algorithms aim to create a "completely" decentralized system where all stakeholders (even those who are not always available online) participate in the committing of blocks.  Tendermint has a different approach to block creation.  Validators are expected to be online, and the set of validators is permissioned/curated by some external process.  Proof-of-stake is not required, but can be implemented on top of Tendermint consensus.  That is, validators may be required to post collateral on-chain, off-chain, or may not be required to post any collateral at all.

 Validators have a cryptographic key-pair and an associated amount of "voting power".  Voting power need not be the same.

 ## Becoming a Validator

 There are two ways to become validator.

 1. They can be pre-established in the [genesis state](/docs/specs/genesis)
 2. The [ABCI app responds to the EndBlock message](https://github.com/tendermint/abci) with changes to the existing validator set.

 ## Committing a Block

 _+2/3 is short for "more than 2/3"_

 A block is committed when +2/3 of the validator set sign [precommit votes](/docs/specs/block-structure#vote) for that block at the same [round](/docs/specs/consensus).  The +2/3 set of precommit votes is called a [_commit_](/docs/specs/block-structure#commit).  While any +2/3 set of precommits for the same block at the same height&round can serve as validation, the canonical commit is included in the next block (see [LastCommit](/docs/specs/block-structure).
--- a/docs/specs/wire-protocol.md
+++ b/docs/specs/wire-protocol.md
@ -0,0 +1,119 @@
 # Wire Protocol

 The [Tendermint wire protocol](https://github.com/tendermint/go-wire) encodes data in [c-style binary](#binary) and [JSON](#json) form.

 ## Supported types

 - Primitive types
  - `uint8` (aka `byte`), `uint16`, `uint32`, `uint64`
  - `int8`, `int16`, `int32`, `int64`
  - `uint`, `int`: variable length (un)signed integers
  - `string`, `[]byte`
  - `time`
 - Derived types
  - structs
  - var-length arrays of a particular type
  - fixed-length arrays of a particular type
  - interfaces: registered union types preceded by a `type byte`
  - pointers

 ## Binary

 **Fixed-length primitive types** are encoded with 1,2,3, or 4 big-endian bytes.
  - `uint8` (aka `byte`), `uint16`, `uint32`, `uint64`: takes 1,2,3, and 4 bytes respectively
  - `int8`, `int16`, `int32`, `int64`: takes 1,2,3, and 4 bytes respectively
  - `time`: `int64` representation of nanoseconds since epoch

 **Variable-length integers** are encoded with a single leading byte representing the length of the following big-endian bytes.  For signed negative integers, the most significant bit of the leading byte is a 1.

  - `uint`: 1-byte length prefixed variable-size (0 ~ 255 bytes) unsigned integers
  - `int`: 1-byte length prefixed variable-size (0 ~ 127 bytes) signed integers

 NOTE: While the number 0 (zero) is encoded with a single byte `x00`, the number 1 (one) takes two bytes to represent: `x0101`.  This isn't the most efficient representation, but the rules are easier to remember.

 | number       | binary `uint` | binary `int`  |
 | ------------ | ------------- | ------------- |
 | 0            | `x00`         | `x00`         |
 | 1            | `x0101`       | `x0101`       |
 | 2            | `x0102`       | `x0102`       |
 | 256          | `x020100`     | `x020100`     |
 | 2^(127*8)-1  | `x7FFFFF...`  | `x7FFFFF...`  |
 | 2^(127*8)    | `x800100...`  | overflow      |
 | 2^(255*8)-1  | `xFFFFFF...`  | overflow      |
 | -1           | n/a           | `x8101`       |
 | -2           | n/a           | `x8102`       |
 | -256         | n/a           | `x820100`     |

 **Structures** are encoded by encoding the field values in order of declaration.

 ```go
 type Foo struct {
    MyString string
    MyUint32 uint32
 }
 var foo = Foo{"626172", math.MaxUint32}

 /* The binary representation of foo:
 0103626172FFFFFFFF
 0103:               `int` encoded length of string, here 3
     626172:         3 bytes of string "bar"
           FFFFFFFF: 4 bytes of uint32 MaxUint32
 */
 ```

 **Variable-length arrays** are encoded with a leading `int` denoting the length of the array followed by the binary representation of the items.  **Fixed-length arrays** are similar but aren't preceded by the leading `int`.

 ```go
 foos := []Foo{foo, foo}

 /* The binary representation of foos:
 01020103626172FFFFFFFF0103626172FFFFFFFF
 0102:                                     `int` encoded length of array, here 2
     0103626172FFFFFFFF:                   the first `foo`
                       0103626172FFFFFFFF: the second `foo`
 */

 foos := [2]Foo{foo, foo} // fixed-length array

 /* The binary representation of foos:
 0103626172FFFFFFFF0103626172FFFFFFFF
 0103626172FFFFFFFF:                   the first `foo`
                   0103626172FFFFFFFF: the second `foo`
 */
 ```

 **Interfaces** can represent one of any number of concrete types.  The concrete types of an interface must first be declared with their corresponding `type byte`.  An interface is then encoded with the leading `type byte`, then the binary encoding of the underlying concrete type.

 NOTE: The byte `x00` is reserved for the `nil` interface value and `nil` pointer values.

 ```go
 type Animal interface{}
 type Dog uint32
 type Cat string

 RegisterInterface(
    struct{ Animal }{},          // Convenience for referencing the 'Animal' interface
    ConcreteType{Dog(0),  0x01}, // Register the byte 0x01 to denote a Dog
    ConcreteType{Cat(""), 0x02}, // Register the byte 0x02 to denote a Cat
 )

 var animal Animal = Dog(02)

 /* The binary representation of animal:
 010102
 01:     the type byte for a `Dog`
   0102: the bytes of Dog(02)
 */
 ```

 **Pointers** are encoded with a single leading byte `x00` for `nil` pointers, otherwise encoded with a leading byte `x01` followed by the binary encoding of the value pointed to.

 NOTE: It's easy to convert pointer types into interface types, since the `type byte` `x00` is always `nil`.

 ## JSON

 The JSON codec is compatible with the [`binary`](#binary) codec, and is fairly intuitive if you're already familiar with golang's JSON encoding.  Some quirks are noted below:

 - variable-length and fixed-length bytes are encoded as uppercase hexadecimal strings
 - interface values are encoded as an array of two items: `[type_byte, concrete_value]`
 - times are encoded as rfc2822 strings