You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

630 lines
22 KiB

  1. Application Development Guide
  2. =============================
  3. ABCI Design
  4. -----------
  5. The purpose of ABCI is to provide a clean interface between state
  6. transition machines on one computer and the mechanics of their
  7. replication across multiple computers. The former we call 'application
  8. logic' and the latter the 'consensus engine'. Application logic
  9. validates transactions and optionally executes transactions against some
  10. persistent state. A consensus engine ensures all transactions are
  11. replicated in the same order on every machine. We call each machine in a
  12. consensus engine a 'validator', and each validator runs the same
  13. transactions through the same application logic. In particular, we are
  14. interested in blockchain-style consensus engines, where transactions are
  15. committed in hash-linked blocks.
  16. The ABCI design has a few distinct components:
  17. - message protocol
  18. - pairs of request and response messages
  19. - consensus makes requests, application responds
  20. - defined using protobuf
  21. - server/client
  22. - consensus engine runs the client
  23. - application runs the server
  24. - two implementations:
  25. - async raw bytes
  26. - grpc
  27. - blockchain protocol
  28. - abci is connection oriented
  29. - Tendermint Core maintains three connections:
  30. - `mempool connection <#mempool-connection>`__: for checking if
  31. transactions should be relayed before they are committed; only
  32. uses ``CheckTx``
  33. - `consensus connection <#consensus-connection>`__: for executing
  34. transactions that have been committed. Message sequence is -
  35. for every block -
  36. ``BeginBlock, [DeliverTx, ...], EndBlock, Commit``
  37. - `query connection <#query-connection>`__: for querying the
  38. application state; only uses Query and Info
  39. The mempool and consensus logic act as clients, and each maintains an
  40. open ABCI connection with the application, which hosts an ABCI server.
  41. Shown are the request and response types sent on each connection.
  42. Message Protocol
  43. ----------------
  44. The message protocol consists of pairs of requests and responses. Some
  45. messages have no fields, while others may include byte-arrays, strings,
  46. or integers. See the ``message Request`` and ``message Response``
  47. definitions in `the protobuf definition
  48. file <https://github.com/tendermint/abci/blob/master/types/types.proto>`__,
  49. and the `protobuf
  50. documentation <https://developers.google.com/protocol-buffers/docs/overview>`__
  51. for more details.
  52. For each request, a server should respond with the corresponding
  53. response, where order of requests is preserved in the order of
  54. responses.
  55. Server
  56. ------
  57. To use ABCI in your programming language of choice, there must be a ABCI
  58. server in that language. Tendermint supports two kinds of implementation
  59. of the server:
  60. - Asynchronous, raw socket server (Tendermint Socket Protocol, also
  61. known as TSP or Teaspoon)
  62. - GRPC
  63. Both can be tested using the ``abci-cli`` by setting the ``--abci`` flag
  64. appropriately (ie. to ``socket`` or ``grpc``).
  65. See examples, in various stages of maintenance, in
  66. `Go <https://github.com/tendermint/abci/tree/master/server>`__,
  67. `JavaScript <https://github.com/tendermint/js-abci>`__,
  68. `Python <https://github.com/tendermint/abci/tree/master/example/python3/abci>`__,
  69. `C++ <https://github.com/mdyring/cpp-tmsp>`__, and
  70. `Java <https://github.com/jTendermint/jabci>`__.
  71. GRPC
  72. ~~~~
  73. If GRPC is available in your language, this is the easiest approach,
  74. though it will have significant performance overhead.
  75. To get started with GRPC, copy in the `protobuf
  76. file <https://github.com/tendermint/abci/blob/master/types/types.proto>`__
  77. and compile it using the GRPC plugin for your language. For instance,
  78. for golang, the command is
  79. ``protoc --go_out=plugins=grpc:. types.proto``. See the `grpc
  80. documentation for more details <http://www.grpc.io/docs/>`__. ``protoc``
  81. will autogenerate all the necessary code for ABCI client and server in
  82. your language, including whatever interface your application must
  83. satisfy to be used by the ABCI server for handling requests.
  84. TSP
  85. ~~~
  86. If GRPC is not available in your language, or you require higher
  87. performance, or otherwise enjoy programming, you may implement your own
  88. ABCI server using the Tendermint Socket Protocol, known affectionately
  89. as Teaspoon. The first step is still to auto-generate the relevant data
  90. types and codec in your language using ``protoc``. Messages coming over
  91. the socket are Protobuf3 encoded, but additionally length-prefixed to
  92. facilitate use as a streaming protocol. Protobuf3 doesn't have an
  93. official length-prefix standard, so we use our own. The first byte in
  94. the prefix represents the length of the Big Endian encoded length. The
  95. remaining bytes in the prefix are the Big Endian encoded length.
  96. For example, if the Protobuf3 encoded ABCI message is 0xDEADBEEF (4
  97. bytes), the length-prefixed message is 0x0104DEADBEEF. If the Protobuf3
  98. encoded ABCI message is 65535 bytes long, the length-prefixed message
  99. would be like 0x02FFFF....
  100. Note this prefixing does not apply for grpc.
  101. An ABCI server must also be able to support multiple connections, as
  102. Tendermint uses three connections.
  103. Client
  104. ------
  105. There are currently two use-cases for an ABCI client. One is a testing
  106. tool, as in the ``abci-cli``, which allows ABCI requests to be sent via
  107. command line. The other is a consensus engine, such as Tendermint Core,
  108. which makes requests to the application every time a new transaction is
  109. received or a block is committed.
  110. It is unlikely that you will need to implement a client. For details of
  111. our client, see
  112. `here <https://github.com/tendermint/abci/tree/master/client>`__.
  113. Most of the examples below are from `dummy application
  114. <https://github.com/tendermint/abci/blob/master/example/dummy/dummy.go>`__,
  115. which is a part of the abci repo. `persistent_dummy application
  116. <https://github.com/tendermint/abci/blob/master/example/dummy/persistent_dummy.go>`__
  117. is used to show ``BeginBlock``, ``EndBlock`` and ``InitChain``
  118. example implementations.
  119. Blockchain Protocol
  120. -------------------
  121. In ABCI, a transaction is simply an arbitrary length byte-array. It is
  122. the application's responsibility to define the transaction codec as they
  123. please, and to use it for both CheckTx and DeliverTx.
  124. Note that there are two distinct means for running transactions,
  125. corresponding to stages of 'awareness' of the transaction in the
  126. network. The first stage is when a transaction is received by a
  127. validator from a client into the so-called mempool or transaction pool -
  128. this is where we use CheckTx. The second is when the transaction is
  129. successfully committed on more than 2/3 of validators - where we use
  130. DeliverTx. In the former case, it may not be necessary to run all the
  131. state transitions associated with the transaction, as the transaction
  132. may not ultimately be committed until some much later time, when the
  133. result of its execution will be different. For instance, an Ethereum
  134. ABCI app would check signatures and amounts in CheckTx, but would not
  135. actually execute any contract code until the DeliverTx, so as to avoid
  136. executing state transitions that have not been finalized.
  137. To formalize the distinction further, two explicit ABCI connections are
  138. made between Tendermint Core and the application: the mempool connection
  139. and the consensus connection. We also make a third connection, the query
  140. connection, to query the local state of the app.
  141. Mempool Connection
  142. ~~~~~~~~~~~~~~~~~~
  143. The mempool connection is used *only* for CheckTx requests. Transactions
  144. are run using CheckTx in the same order they were received by the
  145. validator. If the CheckTx returns ``OK``, the transaction is kept in
  146. memory and relayed to other peers in the same order it was received.
  147. Otherwise, it is discarded.
  148. CheckTx requests run concurrently with block processing; so they should
  149. run against a copy of the main application state which is reset after
  150. every block. This copy is necessary to track transitions made by a
  151. sequence of CheckTx requests before they are included in a block. When a
  152. block is committed, the application must ensure to reset the mempool
  153. state to the latest committed state. Tendermint Core will then filter
  154. through all transactions in the mempool, removing any that were included
  155. in the block, and re-run the rest using CheckTx against the post-Commit
  156. mempool state.
  157. .. container:: toggle
  158. .. container:: header
  159. **Show/Hide Go Example**
  160. .. code-block:: go
  161. func (app *DummyApplication) CheckTx(tx []byte) types.Result {
  162. return types.OK
  163. }
  164. .. container:: toggle
  165. .. container:: header
  166. **Show/Hide Java Example**
  167. .. code-block:: java
  168. ResponseCheckTx requestCheckTx(RequestCheckTx req) {
  169. byte[] transaction = req.getTx().toByteArray();
  170. // validate transaction
  171. if (notValid) {
  172. return ResponseCheckTx.newBuilder().setCode(CodeType.BadNonce).setLog("invalid tx").build();
  173. } else {
  174. return ResponseCheckTx.newBuilder().setCode(CodeType.OK).build();
  175. }
  176. }
  177. Consensus Connection
  178. ~~~~~~~~~~~~~~~~~~~~
  179. The consensus connection is used only when a new block is committed, and
  180. communicates all information from the block in a series of requests:
  181. ``BeginBlock, [DeliverTx, ...], EndBlock, Commit``. That is, when a
  182. block is committed in the consensus, we send a list of DeliverTx
  183. requests (one for each transaction) sandwiched by BeginBlock and
  184. EndBlock requests, and followed by a Commit.
  185. DeliverTx
  186. ^^^^^^^^^
  187. DeliverTx is the workhorse of the blockchain. Tendermint sends the
  188. DeliverTx requests asynchronously but in order, and relies on the
  189. underlying socket protocol (ie. TCP) to ensure they are received by the
  190. app in order. They have already been ordered in the global consensus by
  191. the Tendermint protocol.
  192. DeliverTx returns a abci.Result, which includes a Code, Data, and Log.
  193. The code may be non-zero (non-OK), meaning the corresponding transaction
  194. should have been rejected by the mempool, but may have been included in
  195. a block by a Byzantine proposer.
  196. The block header will be updated (TODO) to include some commitment to
  197. the results of DeliverTx, be it a bitarray of non-OK transactions, or a
  198. merkle root of the data returned by the DeliverTx requests, or both.
  199. .. container:: toggle
  200. .. container:: header
  201. **Show/Hide Go Example**
  202. .. code-block:: go
  203. // tx is either "key=value" or just arbitrary bytes
  204. func (app *DummyApplication) DeliverTx(tx []byte) types.Result {
  205. parts := strings.Split(string(tx), "=")
  206. if len(parts) == 2 {
  207. app.state.Set([]byte(parts[0]), []byte(parts[1]))
  208. } else {
  209. app.state.Set(tx, tx)
  210. }
  211. return types.OK
  212. }
  213. .. container:: toggle
  214. .. container:: header
  215. **Show/Hide Java Example**
  216. .. code-block:: java
  217. /**
  218. * Using Protobuf types from the protoc compiler, we always start with a byte[]
  219. */
  220. ResponseDeliverTx deliverTx(RequestDeliverTx request) {
  221. byte[] transaction = request.getTx().toByteArray();
  222. // validate your transaction
  223. if (notValid) {
  224. return ResponseDeliverTx.newBuilder().setCode(CodeType.BadNonce).setLog("transaction was invalid").build();
  225. } else {
  226. ResponseDeliverTx.newBuilder().setCode(CodeType.OK).build();
  227. }
  228. }
  229. Commit
  230. ^^^^^^
  231. Once all processing of the block is complete, Tendermint sends the
  232. Commit request and blocks waiting for a response. While the mempool may
  233. run concurrently with block processing (the BeginBlock, DeliverTxs, and
  234. EndBlock), it is locked for the Commit request so that its state can be
  235. safely reset during Commit. This means the app *MUST NOT* do any
  236. blocking communication with the mempool (ie. broadcast\_tx) during
  237. Commit, or there will be deadlock. Note also that all remaining
  238. transactions in the mempool are replayed on the mempool connection
  239. (CheckTx) following a commit.
  240. The app should respond to the Commit request with a byte array, which is the deterministic
  241. state root of the application. It is included in the header of the next
  242. block. It can be used to provide easily verified Merkle-proofs of the
  243. state of the application.
  244. It is expected that the app will persist state to disk on Commit. The
  245. option to have all transactions replayed from some previous block is the
  246. job of the `Handshake <#handshake>`__.
  247. .. container:: toggle
  248. .. container:: header
  249. **Show/Hide Go Example**
  250. .. code-block:: go
  251. func (app *DummyApplication) Commit() types.Result {
  252. hash := app.state.Hash()
  253. return types.NewResultOK(hash, "")
  254. }
  255. .. container:: toggle
  256. .. container:: header
  257. **Show/Hide Java Example**
  258. .. code-block:: java
  259. ResponseCommit requestCommit(RequestCommit requestCommit) {
  260. // update the internal app-state
  261. byte[] newAppState = calculateAppState();
  262. // and return it to the node
  263. return ResponseCommit.newBuilder().setCode(CodeType.OK).setData(ByteString.copyFrom(newAppState)).build();
  264. }
  265. BeginBlock
  266. ^^^^^^^^^^
  267. The BeginBlock request can be used to run some code at the beginning of
  268. every block. It also allows Tendermint to send the current block hash
  269. and header to the application, before it sends any of the transactions.
  270. The app should remember the latest height and header (ie. from which it
  271. has run a successful Commit) so that it can tell Tendermint where to
  272. pick up from when it restarts. See information on the Handshake, below.
  273. .. container:: toggle
  274. .. container:: header
  275. **Show/Hide Go Example**
  276. .. code-block:: go
  277. // Track the block hash and header information
  278. func (app *PersistentDummyApplication) BeginBlock(params types.RequestBeginBlock) {
  279. // update latest block info
  280. app.blockHeader = params.Header
  281. // reset valset changes
  282. app.changes = make([]*types.Validator, 0)
  283. }
  284. .. container:: toggle
  285. .. container:: header
  286. **Show/Hide Java Example**
  287. .. code-block:: java
  288. /*
  289. * all types come from protobuf definition
  290. */
  291. ResponseBeginBlock requestBeginBlock(RequestBeginBlock req) {
  292. Header header = req.getHeader();
  293. byte[] prevAppHash = header.getAppHash().toByteArray();
  294. long prevHeight = header.getHeight();
  295. long numTxs = header.getNumTxs();
  296. // run your pre-block logic. Maybe prepare a state snapshot, message components, etc
  297. return ResponseBeginBlock.newBuilder().build();
  298. }
  299. EndBlock
  300. ^^^^^^^^
  301. The EndBlock request can be used to run some code at the end of every block.
  302. Additionally, the response may contain a list of validators, which can be used
  303. to update the validator set. To add a new validator or update an existing one,
  304. simply include them in the list returned in the EndBlock response. To remove
  305. one, include it in the list with a ``power`` equal to ``0``. Tendermint core
  306. will take care of updating the validator set. Note the change in voting power
  307. must be strictly less than 1/3 per block if you want a light client to be able
  308. to prove the transition externally. See the `light client docs
  309. <https://godoc.org/github.com/tendermint/tendermint/lite#hdr-How_We_Track_Validators>`__
  310. for details on how it tracks validators.
  311. .. container:: toggle
  312. .. container:: header
  313. **Show/Hide Go Example**
  314. .. code-block:: go
  315. // Update the validator set
  316. func (app *PersistentDummyApplication) EndBlock(req types.RequestEndBlock) types.ResponseEndBlock {
  317. return types.ResponseEndBlock{ValidatorUpdates: app.ValUpdates}
  318. }
  319. .. container:: toggle
  320. .. container:: header
  321. **Show/Hide Java Example**
  322. .. code-block:: java
  323. /*
  324. * Assume that one validator changes. The new validator has a power of 10
  325. */
  326. ResponseEndBlock requestEndBlock(RequestEndBlock req) {
  327. final long currentHeight = req.getHeight();
  328. final byte[] validatorPubKey = getValPubKey();
  329. ResponseEndBlock.Builder builder = ResponseEndBlock.newBuilder();
  330. builder.addDiffs(1, Types.Validator.newBuilder().setPower(10L).setPubKey(ByteString.copyFrom(validatorPubKey)).build());
  331. return builder.build();
  332. }
  333. Query Connection
  334. ~~~~~~~~~~~~~~~~
  335. This connection is used to query the application without engaging
  336. consensus. It's exposed over the tendermint core rpc, so clients can
  337. query the app without exposing a server on the app itself, but they must
  338. serialize each query as a single byte array. Additionally, certain
  339. "standardized" queries may be used to inform local decisions, for
  340. instance about which peers to connect to.
  341. Tendermint Core currently uses the Query connection to filter peers upon
  342. connecting, according to IP address or public key. For instance,
  343. returning non-OK ABCI response to either of the following queries will
  344. cause Tendermint to not connect to the corresponding peer:
  345. - ``p2p/filter/addr/<addr>``, where ``<addr>`` is an IP address.
  346. - ``p2p/filter/pubkey/<pubkey>``, where ``<pubkey>`` is the hex-encoded
  347. ED25519 key of the node (not it's validator key)
  348. Note: these query formats are subject to change!
  349. .. container:: toggle
  350. .. container:: header
  351. **Show/Hide Go Example**
  352. .. code-block:: go
  353. func (app *DummyApplication) Query(reqQuery types.RequestQuery) (resQuery types.ResponseQuery) {
  354. if reqQuery.Prove {
  355. value, proof, exists := app.state.Proof(reqQuery.Data)
  356. resQuery.Index = -1 // TODO make Proof return index
  357. resQuery.Key = reqQuery.Data
  358. resQuery.Value = value
  359. resQuery.Proof = proof
  360. if exists {
  361. resQuery.Log = "exists"
  362. } else {
  363. resQuery.Log = "does not exist"
  364. }
  365. return
  366. } else {
  367. index, value, exists := app.state.Get(reqQuery.Data)
  368. resQuery.Index = int64(index)
  369. resQuery.Value = value
  370. if exists {
  371. resQuery.Log = "exists"
  372. } else {
  373. resQuery.Log = "does not exist"
  374. }
  375. return
  376. }
  377. }
  378. .. container:: toggle
  379. .. container:: header
  380. **Show/Hide Java Example**
  381. .. code-block:: java
  382. ResponseQuery requestQuery(RequestQuery req) {
  383. final boolean isProveQuery = req.getProve();
  384. final ResponseQuery.Builder responseBuilder = ResponseQuery.newBuilder();
  385. if (isProveQuery) {
  386. com.app.example.ProofResult proofResult = generateProof(req.getData().toByteArray());
  387. final byte[] proofAsByteArray = proofResult.getAsByteArray();
  388. responseBuilder.setProof(ByteString.copyFrom(proofAsByteArray));
  389. responseBuilder.setKey(req.getData());
  390. responseBuilder.setValue(ByteString.copyFrom(proofResult.getData()));
  391. responseBuilder.setLog(result.getLogValue());
  392. } else {
  393. byte[] queryData = req.getData().toByteArray();
  394. final com.app.example.QueryResult result = generateQueryResult(queryData);
  395. responseBuilder.setIndex(result.getIndex());
  396. responseBuilder.setValue(ByteString.copyFrom(result.getValue()));
  397. responseBuilder.setLog(result.getLogValue());
  398. }
  399. return responseBuilder.build();
  400. }
  401. Handshake
  402. ~~~~~~~~~
  403. When the app or tendermint restarts, they need to sync to a common
  404. height. When an ABCI connection is first established, Tendermint will
  405. call ``Info`` on the Query connection. The response should contain the
  406. LastBlockHeight and LastBlockAppHash - the former is the last block for
  407. which the app ran Commit successfully, the latter is the response
  408. from that Commit.
  409. Using this information, Tendermint will determine what needs to be
  410. replayed, if anything, against the app, to ensure both Tendermint and
  411. the app are synced to the latest block height.
  412. If the app returns a LastBlockHeight of 0, Tendermint will just replay
  413. all blocks.
  414. .. container:: toggle
  415. .. container:: header
  416. **Show/Hide Go Example**
  417. .. code-block:: go
  418. func (app *DummyApplication) Info(req types.RequestInfo) (resInfo types.ResponseInfo) {
  419. return types.ResponseInfo{Data: cmn.Fmt("{\"size\":%v}", app.state.Size())}
  420. }
  421. .. container:: toggle
  422. .. container:: header
  423. **Show/Hide Java Example**
  424. .. code-block:: java
  425. ResponseInfo requestInfo(RequestInfo req) {
  426. final byte[] lastAppHash = getLastAppHash();
  427. final long lastHeight = getLastHeight();
  428. return ResponseInfo.newBuilder().setLastBlockAppHash(ByteString.copyFrom(lastAppHash)).setLastBlockHeight(lastHeight).build();
  429. }
  430. Genesis
  431. ~~~~~~~
  432. ``InitChain`` will be called once upon the genesis. ``params`` includes the
  433. initial validator set. Later on, it may be extended to take parts of the
  434. consensus params.
  435. .. container:: toggle
  436. .. container:: header
  437. **Show/Hide Go Example**
  438. .. code-block:: go
  439. // Save the validators in the merkle tree
  440. func (app *PersistentDummyApplication) InitChain(params types.RequestInitChain) {
  441. for _, v := range params.Validators {
  442. r := app.updateValidator(v)
  443. if r.IsErr() {
  444. app.logger.Error("Error updating validators", "r", r)
  445. }
  446. }
  447. }
  448. .. container:: toggle
  449. .. container:: header
  450. **Show/Hide Java Example**
  451. .. code-block:: java
  452. /*
  453. * all types come from protobuf definition
  454. */
  455. ResponseInitChain requestInitChain(RequestInitChain req) {
  456. final int validatorsCount = req.getValidatorsCount();
  457. final List<Types.Validator> validatorsList = req.getValidatorsList();
  458. validatorsList.forEach((validator) -> {
  459. long power = validator.getPower();
  460. byte[] validatorPubKey = validator.getPubKey().toByteArray();
  461. // do somehing for validator setup in app
  462. });
  463. return ResponseInitChain.newBuilder().build();
  464. }