You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

220 lines
9.3 KiB

  1. Block Structure
  2. ===============
  3. The tendermint consensus engine records all agreements by a
  4. supermajority of nodes into a blockchain, which is replicated among all
  5. nodes. This blockchain is accessible via various rpc endpoints, mainly
  6. ``/block?height=`` to get the full block, as well as
  7. ``/blockchain?minHeight=_&maxHeight=_`` to get a list of headers. But
  8. what exactly is stored in these blocks?
  9. Block
  10. ~~~~~
  11. A
  12. `Block <https://godoc.org/github.com/tendermint/tendermint/types#Block>`__
  13. contains:
  14. - a `Header <#header>`__ contains merkle hashes for various chain
  15. states
  16. - the
  17. `Data <https://godoc.org/github.com/tendermint/tendermint/types#Data>`__
  18. is all transactions which are to be processed
  19. - the `LastCommit <#commit>`__ > 2/3 signatures for the last block
  20. The signatures returned along with block ``H`` are those validating
  21. block ``H-1``. This can be a little confusing, but we must also consider
  22. that the ``Header`` also contains the ``LastCommitHash``. It would be
  23. impossible for a Header to include the commits that sign it, as it would
  24. cause an infinite loop here. But when we get block ``H``, we find
  25. ``Header.LastCommitHash``, which must match the hash of ``LastCommit``.
  26. Header
  27. ~~~~~~
  28. The
  29. `Header <https://godoc.org/github.com/tendermint/tendermint/types#Header>`__
  30. contains lots of information (follow link for up-to-date info). Notably,
  31. it maintains the ``Height``, the ``LastBlockID`` (to make it a chain),
  32. and hashes of the data, the app state, and the validator set. This is
  33. important as the only item that is signed by the validators is the
  34. ``Header``, and all other data must be validated against one of the
  35. merkle hashes in the ``Header``.
  36. The ``DataHash`` can provide a nice check on the
  37. `Data <https://godoc.org/github.com/tendermint/tendermint/types#Data>`__
  38. returned in this same block. If you are subscribed to new blocks, via
  39. tendermint RPC, in order to display or process the new transactions you
  40. should at least validate that the ``DataHash`` is valid. If it is
  41. important to verify autheniticity, you must wait for the ``LastCommit``
  42. from the next block to make sure the block header (including
  43. ``DataHash``) was properly signed.
  44. The ``ValidatorHash`` contains a hash of the current
  45. `Validators <https://godoc.org/github.com/tendermint/tendermint/types#Validator>`__.
  46. Tracking all changes in the validator set is complex, but a client can
  47. quickly compare this hash with the `hash of the currently known
  48. validators <https://godoc.org/github.com/tendermint/tendermint/types#ValidatorSet.Hash>`__
  49. to see if there have been changes.
  50. The ``AppHash`` serves as the basis for validating any merkle proofs
  51. that come from the `ABCI
  52. application <https://github.com/tendermint/abci>`__. It represents the
  53. state of the actual application, rather that the state of the blockchain
  54. itself. This means it's necessary in order to perform any business
  55. logic, such as verifying and account balance.
  56. **Note** After the transactions are committed to a block, they still
  57. need to be processed in a separate step, which happens between the
  58. blocks. If you find a given transaction in the block at height ``H``,
  59. the effects of running that transaction will be first visible in the
  60. ``AppHash`` from the block header at height ``H+1``.
  61. Like the ``LastCommit`` issue, this is a requirement of the immutability
  62. of the block chain, as the application only applies transactions *after*
  63. they are commited to the chain.
  64. Commit
  65. ~~~~~~
  66. The
  67. `Commit <https://godoc.org/github.com/tendermint/tendermint/types#Commit>`__
  68. contains a set of
  69. `Votes <https://godoc.org/github.com/tendermint/tendermint/types#Vote>`__
  70. that were made by the validator set to reach consensus on this block.
  71. This is the key to the security in any PoS system, and actually no data
  72. that cannot be traced back to a block header with a valid set of Votes
  73. can be trusted. Thus, getting the Commit data and verifying the votes is
  74. extremely important.
  75. As mentioned above, in order to find the ``precommit votes`` for block
  76. header ``H``, we need to query block ``H+1``. Then we need to check the
  77. votes, make sure they really are for that block, and properly formatted.
  78. Much of this code is implemented in Go in the
  79. `light-client <https://github.com/tendermint/light-client>`__ package.
  80. If you look at the code, you will notice that we need to provide the
  81. ``chainID`` of the blockchain in order to properly calculate the votes.
  82. This is to protect anyone from swapping votes between chains to fake (or
  83. frame) a validator. Also note that this ``chainID`` is in the
  84. ``genesis.json`` from *Tendermint*, not the ``genesis.json`` from the
  85. basecoin app (`that is a different
  86. chainID... <https://github.com/tendermint/basecoin/issues/32>`__).
  87. Once we have those votes, and we calculated the proper `sign
  88. bytes <https://godoc.org/github.com/tendermint/tendermint/types#Vote.WriteSignBytes>`__
  89. using the chainID and a `nice helper
  90. function <https://godoc.org/github.com/tendermint/tendermint/types#SignBytes>`__,
  91. we can verify them. The light client is responsible for maintaining a
  92. set of validators that we trust. Each vote only stores the validators
  93. ``Address``, as well as the ``Signature``. Assuming we have a local copy
  94. of the trusted validator set, we can look up the ``Public Key`` of the
  95. validator given its ``Address``, then verify that the ``Signature``
  96. matches the ``SignBytes`` and ``Public Key``. Then we sum up the total
  97. voting power of all validators, whose votes fulfilled all these
  98. stringent requirements. If the total number of voting power for a single
  99. block is greater than 2/3 of all voting power, then we can finally trust
  100. the block header, the AppHash, and the proof we got from the ABCI
  101. application.
  102. Vote Sign Bytes
  103. ^^^^^^^^^^^^^^^
  104. The ``sign-bytes`` of a vote is produced by taking a
  105. `stable-json <https://github.com/substack/json-stable-stringify>`__-like
  106. deterministic JSON `wire <./wire-protocol.html>`__ encoding of
  107. the vote (excluding the ``Signature`` field), and wrapping it with
  108. ``{"chain_id":"my_chain","vote":...}``.
  109. For example, a precommit vote might have the following ``sign-bytes``:
  110. .. code:: json
  111. {"chain_id":"my_chain","vote":{"block_hash":"611801F57B4CE378DF1A3FFF1216656E89209A99","block_parts_header":{"hash":"B46697379DBE0774CC2C3B656083F07CA7E0F9CE","total":123},"height":1234,"round":1,"type":2}}
  112. Block Hash
  113. ~~~~~~~~~~
  114. The `block
  115. hash <https://godoc.org/github.com/tendermint/tendermint/types#Block.Hash>`__
  116. is the `Simple Tree hash <Merkle-Trees#simple-tree-with-dictionaries>`__
  117. of the fields of the block ``Header`` encoded as a list of
  118. ``KVPair``\ s.
  119. Transaction
  120. ~~~~~~~~~~~
  121. A transaction is any sequence of bytes. It is up to your
  122. `ABCI <https://github.com/tendermint/abci>`__ application to accept or
  123. reject transactions.
  124. BlockID
  125. ~~~~~~~
  126. Many of these data structures refer to the
  127. `BlockID <https://godoc.org/github.com/tendermint/tendermint/types#BlockID>`__,
  128. which is the ``BlockHash`` (hash of the block header, also referred to
  129. by the next block) along with the ``PartSetHeader``. The
  130. ``PartSetHeader`` is explained below and is used internally to
  131. orchestrate the p2p propogation. For clients, it is basically opaque
  132. bytes, but they must match for all votes.
  133. PartSetHeader
  134. ~~~~~~~~~~~~~
  135. The
  136. `PartSetHeader <https://godoc.org/github.com/tendermint/tendermint/types#PartSetHeader>`__
  137. contains the total number of pieces in a
  138. `PartSet <https://godoc.org/github.com/tendermint/tendermint/types#PartSet>`__,
  139. and the Merkle root hash of those pieces.
  140. PartSet
  141. ~~~~~~~
  142. PartSet is used to split a byteslice of data into parts (pieces) for
  143. transmission. By splitting data into smaller parts and computing a
  144. Merkle root hash on the list, you can verify that a part is legitimately
  145. part of the complete data, and the part can be forwarded to other peers
  146. before all the parts are known. In short, it's a fast way to securely
  147. propagate a large chunk of data (like a block) over a gossip network.
  148. PartSet was inspired by the LibSwift project.
  149. Usage:
  150. .. code:: go
  151. data := RandBytes(2 << 20) // Something large
  152. partSet := NewPartSetFromData(data)
  153. partSet.Total() // Total number of 4KB parts
  154. partSet.Count() // Equal to the Total, since we already have all the parts
  155. partSet.Hash() // The Merkle root hash
  156. partSet.BitArray() // A BitArray of partSet.Total() 1's
  157. header := partSet.Header() // Send this to the peer
  158. header.Total // Total number of parts
  159. header.Hash // The merkle root hash
  160. // Now we'll reconstruct the data from the parts
  161. partSet2 := NewPartSetFromHeader(header)
  162. partSet2.Total() // Same total as partSet.Total()
  163. partSet2.Count() // Zero, since this PartSet doesn't have any parts yet.
  164. partSet2.Hash() // Same hash as in partSet.Hash()
  165. partSet2.BitArray() // A BitArray of partSet.Total() 0's
  166. // In a gossip network the parts would arrive in arbitrary order, perhaps
  167. // in response to explicit requests for parts, or optimistically in response
  168. // to the receiving peer's partSet.BitArray().
  169. for !partSet2.IsComplete() {
  170. part := receivePartFromGossipNetwork()
  171. added, err := partSet2.AddPart(part)
  172. if err != nil {
  173. // A wrong part,
  174. // the merkle trail does not hash to partSet2.Hash()
  175. } else if !added {
  176. // A duplicate part already received
  177. }
  178. }
  179. data2, _ := ioutil.ReadAll(partSet2.GetReader())
  180. bytes.Equal(data, data2) // true