Block Structure

The tendermint consensus engine records all agreements by a supermajority of nodes into a blockchain, which is replicated among all nodes. This blockchain is accessible via various rpc endpoints, mainly /block?height= to get the full block, as well as /blockchain?minHeight=_&maxHeight=_ to get a list of headers. But what exactly is stored in these blocks?

Block

A Block contains:

a Header contains merkle hashes for various chain states
the Data is all transactions which are to be processed
the LastCommit > 2/3 signatures for the last block

The signatures returned along with block H are those validating block H-1. This can be a little confusing, but we must also consider that the Header also contains the LastCommitHash. It would be impossible for a Header to include the commits that sign it, as it would cause an infinite loop here. But when we get block H, we find Header.LastCommitHash, which must match the hash of LastCommit.

Header

The Header contains lots of information (follow link for up-to-date info). Notably, it maintains the Height, the LastBlockID (to make it a chain), and hashes of the data, the app state, and the validator set. This is important as the only item that is signed by the validators is the Header, and all other data must be validated against one of the merkle hashes in the Header.

The DataHash can provide a nice check on the Data returned in this same block. If you are subscribed to new blocks, via tendermint RPC, in order to display or process the new transactions you should at least validate that the DataHash is valid. If it is important to verify autheniticity, you must wait for the LastCommit from the next block to make sure the block header (including DataHash) was properly signed.

The ValidatorHash contains a hash of the current Validators. Tracking all changes in the validator set is complex, but a client can quickly compare this hash with the hash of the currently known validators to see if there have been changes.

The AppHash serves as the basis for validating any merkle proofs that come from the ABCI application. It represents the state of the actual application, rather that the state of the blockchain itself. This means it's necessary in order to perform any business logic, such as verifying and account balance.

Note After the transactions are committed to a block, they still need to be processed in a separate step, which happens between the blocks. If you find a given transaction in the block at height H, the effects of running that transaction will be first visible in the AppHash from the block header at height H+1.

Like the LastCommit issue, this is a requirement of the immutability of the block chain, as the application only applies transactions after they are commited to the chain.

Commit

The Commit contains a set of Votes that were made by the validator set to reach consensus on this block. This is the key to the security in any PoS system, and actually no data that cannot be traced back to a block header with a valid set of Votes can be trusted. Thus, getting the Commit data and verifying the votes is extremely important.

As mentioned above, in order to find the precommit votes for block header H, we need to query block H+1. Then we need to check the votes, make sure they really are for that block, and properly formatted. Much of this code is implemented in Go in the light-client package. If you look at the code, you will notice that we need to provide the chainID of the blockchain in order to properly calculate the votes. This is to protect anyone from swapping votes between chains to fake (or frame) a validator. Also note that this chainID is in the genesis.json from Tendermint, not the genesis.json from the basecoin app (that is a different chainID...).

Once we have those votes, and we calculated the proper sign bytes using the chainID and a nice helper function, we can verify them. The light client is responsible for maintaining a set of validators that we trust. Each vote only stores the validators Address, as well as the Signature. Assuming we have a local copy of the trusted validator set, we can look up the Public Key of the validator given its Address, then verify that the Signature matches the SignBytes and Public Key. Then we sum up the total voting power of all validators, whose votes fulfilled all these stringent requirements. If the total number of voting power for a single block is greater than 2/3 of all voting power, then we can finally trust the block header, the AppHash, and the proof we got from the ABCI application.

Vote Sign Bytes

The sign-bytes of a vote is produced by taking a stable-json-like deterministic JSON wire encoding of the vote (excluding the Signature field), and wrapping it with {"chain_id":"my_chain","vote":...}.

For example, a precommit vote might have the following sign-bytes:

{"chain_id":"my_chain","vote":{"block_hash":"611801F57B4CE378DF1A3FFF1216656E89209A99","block_parts_header":{"hash":"B46697379DBE0774CC2C3B656083F07CA7E0F9CE","total":123},"height":1234,"round":1,"type":2}}

Block Hash

The block hash is the Simple Tree hash of the fields of the block Header encoded as a list of KVPairs.

Transaction

A transaction is any sequence of bytes. It is up to your ABCI application to accept or reject transactions.

BlockID

Many of these data structures refer to the BlockID, which is the BlockHash (hash of the block header, also referred to by the next block) along with the PartSetHeader. The PartSetHeader is explained below and is used internally to orchestrate the p2p propogation. For clients, it is basically opaque bytes, but they must match for all votes.

PartSetHeader

The PartSetHeader contains the total number of pieces in a PartSet, and the Merkle root hash of those pieces.

PartSet

PartSet is used to split a byteslice of data into parts (pieces) for transmission. By splitting data into smaller parts and computing a Merkle root hash on the list, you can verify that a part is legitimately part of the complete data, and the part can be forwarded to other peers before all the parts are known. In short, it's a fast way to securely propagate a large chunk of data (like a block) over a gossip network.

PartSet was inspired by the LibSwift project.

Usage:

data := RandBytes(2 << 20) // Something large

partSet := NewPartSetFromData(data)
partSet.Total()     // Total number of 4KB parts
partSet.Count()     // Equal to the Total, since we already have all the parts
partSet.Hash()      // The Merkle root hash
partSet.BitArray()  // A BitArray of partSet.Total() 1's

header := partSet.Header() // Send this to the peer
header.Total        // Total number of parts
header.Hash         // The merkle root hash

// Now we'll reconstruct the data from the parts
partSet2 := NewPartSetFromHeader(header)
partSet2.Total()    // Same total as partSet.Total()
partSet2.Count()    // Zero, since this PartSet doesn't have any parts yet.
partSet2.Hash()     // Same hash as in partSet.Hash()
partSet2.BitArray() // A BitArray of partSet.Total() 0's

// In a gossip network the parts would arrive in arbitrary order, perhaps
// in response to explicit requests for parts, or optimistically in response
// to the receiving peer's partSet.BitArray().
for !partSet2.IsComplete() {
    part := receivePartFromGossipNetwork()
    added, err := partSet2.AddPart(part)
    if err != nil {
    // A wrong part,
        // the merkle trail does not hash to partSet2.Hash()
    } else if !added {
        // A duplicate part already received
    }
}

data2, _ := ioutil.ReadAll(partSet2.GetReader())
bytes.Equal(data, data2) // true

8.9 KiB Raw Blame History