From c939e155a6aab1bfab36479a652d2d8a34a2ba29 Mon Sep 17 00:00:00 2001 From: William Banfield <4561443+williambanfield@users.noreply.github.com> Date: Tue, 24 Aug 2021 14:27:19 -0400 Subject: [PATCH] abci: clarify connection use in-process (#337) * abci: clarify connection use in-process * Update abci.md * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger * invert abci explanations * lint++ * lint++ * lint++ * lint++ Co-authored-by: M. J. Fromberger --- rust-spec/lightclient/verification/README.md | 2 +- spec/abci/abci.md | 296 ++++++++++--------- 2 files changed, 154 insertions(+), 144 deletions(-) diff --git a/rust-spec/lightclient/verification/README.md b/rust-spec/lightclient/verification/README.md index 94f62e3b4..bc5deb2db 100644 --- a/rust-spec/lightclient/verification/README.md +++ b/rust-spec/lightclient/verification/README.md @@ -1,3 +1,3 @@ # Verification -Deprecated see [spec/light-client/verification](/spec/light-client/verification/README.md) +Deprecated see [spec/light-client/verification](../../../spec/light-client/verification/README.md) diff --git a/spec/abci/abci.md b/spec/abci/abci.md index e756b0e53..925624962 100644 --- a/spec/abci/abci.md +++ b/spec/abci/abci.md @@ -5,27 +5,37 @@ title: Method and Types # Methods and Types -## Overview +## Connections -The ABCI message types are defined in a [protobuf -file](https://github.com/tendermint/tendermint/blob/master/proto/tendermint/abci/types.proto). +ABCI applications can run either within the _same_ process as the Tendermint +state-machine replication engine, or as a _separate_ process from the state-machine +replication engine. When run within the same process, Tendermint will call the ABCI +application methods directly as Go method calls. -ABCI methods are split across four separate ABCI _connections_: +When Tendermint and the ABCI application are run as separate processes, Tendermint +opens four connections to the application for ABCI methods. The connections each +handle a subset of the ABCI method calls. These subsets are defined as follows: -- Consensus connection: `InitChain`, `BeginBlock`, `DeliverTx`, `EndBlock`, `Commit` -- Mempool connection: `CheckTx` -- Info connection: `Info`, `Query` -- Snapshot connection: `ListSnapshots`, `LoadSnapshotChunk`, `OfferSnapshot`, `ApplySnapshotChunk` +#### **Consensus** connection -The consensus connection is driven by a consensus protocol and is responsible -for block execution. +* Driven by a consensus protocol and is responsible for block execution. +* Handles the `InitChain`, `BeginBlock`, `DeliverTx`, `EndBlock`, and `Commit` method +calls. -The mempool connection is for validating new transactions, before they're -shared or included in a block. +#### **Mempool** connection -The info connection is for initialization and for queries from the user. +* For validating new transactions, before they're shared or included in a block. +* Handles the `CheckTx` calls. -The snapshot connection is for serving and restoring [state sync snapshots](apps.md#state-sync). +#### **Info** connection + +* For initialization and for queries from the user. +* Handles the `Info` and `Query` calls. + +#### **Snapshot** connection + +* For serving and restoring [state sync snapshots](apps.md#state-sync). +* Handles the `ListSnapshots`, `LoadSnapshotChunk`, `OfferSnapshot`, and `ApplySnapshotChunk` calls. Additionally, there is a `Flush` method that is called on every connection, and an `Echo` method that is just for debugging. @@ -163,20 +173,20 @@ non-determinism must be fixed and the nodes restarted. Sources of non-determinism in applications may include: -- Hardware failures - - Cosmic rays, overheating, etc. -- Node-dependent state - - Random numbers - - Time -- Underspecification - - Library version changes - - Race conditions - - Floating point numbers - - JSON serialization - - Iterating through hash-tables/maps/dictionaries -- External Sources - - Filesystem - - Network calls (eg. some external REST API service) +* Hardware failures + * Cosmic rays, overheating, etc. +* Node-dependent state + * Random numbers + * Time +* Underspecification + * Library version changes + * Race conditions + * Floating point numbers + * JSON serialization + * Iterating through hash-tables/maps/dictionaries +* External Sources + * Filesystem + * Network calls (eg. some external REST API service) See [#56](https://github.com/tendermint/abci/issues/56) for original discussion. @@ -220,17 +230,17 @@ via light client. ### Echo -- **Request**: - - `Message (string)`: A string to echo back -- **Response**: - - `Message (string)`: The input string -- **Usage**: - - Echo a string to test an abci client/server implementation +* **Request**: + * `Message (string)`: A string to echo back +* **Response**: + * `Message (string)`: The input string +* **Usage**: + * Echo a string to test an abci client/server implementation ### Flush -- **Usage**: - - Signals that messages queued on the client should be flushed to +* **Usage**: + * Signals that messages queued on the client should be flushed to the server. It is called periodically by the client implementation to ensure asynchronous requests are actually sent, and is called immediately to make a synchronous request, @@ -238,7 +248,7 @@ via light client. ### Info -- **Request**: +* **Request**: | Name | Type | Description | Field Number | |---------------|--------|------------------------------------------|--------------| @@ -247,7 +257,7 @@ via light client. | p2p_version | uint64 | The Tendermint P2P Protocol version | 3 | | abci_version | string | The Tendermint ABCI semantic version | 4 | -- **Response**: +* **Response**: | Name | Type | Description | Field Number | |---------------------|--------|--------------------------------------------------|--------------| @@ -257,12 +267,12 @@ via light client. | last_block_height | int64 | Latest block for which the app has called Commit | 4 | | last_block_app_hash | bytes | Latest result of Commit | 5 | -- **Usage**: - - Return information about the application state. - - Used to sync Tendermint with the application during a handshake +* **Usage**: + * Return information about the application state. + * Used to sync Tendermint with the application during a handshake that happens on startup. - - The returned `app_version` will be included in the Header of every block. - - Tendermint expects `last_block_app_hash` and `last_block_height` to + * The returned `app_version` will be included in the Header of every block. + * Tendermint expects `last_block_app_hash` and `last_block_height` to be updated during `Commit`, ensuring that `Commit` is never called twice for the same block height. @@ -270,7 +280,7 @@ via light client. ### InitChain -- **Request**: +* **Request**: | Name | Type | Description | Field Number | |------------------|--------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------|--------------| @@ -281,7 +291,7 @@ via light client. | app_state_bytes | bytes | Serialized initial application state. JSON bytes. | 5 | | initial_height | int64 | Height of the initial block (typically `1`). | 6 | -- **Response**: +* **Response**: | Name | Type | Description | Field Number | |------------------|----------------------------------------------|-------------------------------------------------|--------------| @@ -289,19 +299,19 @@ via light client. | validators | repeated [ValidatorUpdate](#validatorupdate) | Initial validator set (optional). | 2 | | app_hash | bytes | Initial application hash. | 3 | -- **Usage**: - - Called once upon genesis. - - If ResponseInitChain.Validators is empty, the initial validator set will be the RequestInitChain.Validators - - If ResponseInitChain.Validators is not empty, it will be the initial +* **Usage**: + * Called once upon genesis. + * If ResponseInitChain.Validators is empty, the initial validator set will be the RequestInitChain.Validators + * If ResponseInitChain.Validators is not empty, it will be the initial validator set (regardless of what is in RequestInitChain.Validators). - - This allows the app to decide if it wants to accept the initial validator + * This allows the app to decide if it wants to accept the initial validator set proposed by tendermint (ie. in the genesis file), or if it wants to use a different one (perhaps computed based on some application specific information in the genesis file). ### Query -- **Request**: +* **Request**: | Name | Type | Description | Field Number | |--------|--------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| @@ -310,7 +320,7 @@ via light client. | height | int64 | The block height for which you want the query (default=0 returns data for the latest committed block). Note that this is the height of the block containing the application's Merkle root hash, which represents the state as it was after committing the block at Height-1 | 3 | | prove | bool | Return Merkle proof with response if possible | 4 | -- **Response**: +* **Response**: | Name | Type | Description | Field Number | |-----------|-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| @@ -324,15 +334,15 @@ via light client. | height | int64 | The block height from which data was derived. Note that this is the height of the block containing the application's Merkle root hash, which represents the state as it was after committing the block at Height-1 | 9 | | codespace | string | Namespace for the `code`. | 10 | -- **Usage**: - - Query for data from the application at current or past height. - - Optionally return Merkle proof. - - Merkle proof includes self-describing `type` field to support many types +* **Usage**: + * Query for data from the application at current or past height. + * Optionally return Merkle proof. + * Merkle proof includes self-describing `type` field to support many types of Merkle trees and encoding formats. ### BeginBlock -- **Request**: +* **Request**: | Name | Type | Description | Field Number | |----------------------|---------------------------------------------|-------------------------------------------------------------------------------------------------------------------|--------------| @@ -341,31 +351,31 @@ via light client. | last_commit_info | [LastCommitInfo](#lastcommitinfo) | Info about the last commit, including the round, and the list of validators and which ones signed the last block. | 3 | | byzantine_validators | repeated [Evidence](#evidence) | List of evidence of validators that acted maliciously. | 4 | -- **Response**: +* **Response**: | Name | Type | Description | Field Number | |--------|---------------------------|-------------------------------------|--------------| | events | repeated [Event](#events) | ype & Key-Value events for indexing | 1 | -- **Usage**: - - Signals the beginning of a new block. Called prior to +* **Usage**: + * Signals the beginning of a new block. Called prior to any DeliverTxs. - - The header contains the height, timestamp, and more - it exactly matches the + * The header contains the height, timestamp, and more - it exactly matches the Tendermint block header. We may seek to generalize this in the future. - - The `LastCommitInfo` and `ByzantineValidators` can be used to determine + * The `LastCommitInfo` and `ByzantineValidators` can be used to determine rewards and punishments for the validators. NOTE validators here do not include pubkeys. ### CheckTx -- **Request**: +* **Request**: | Name | Type | Description | Field Number | |------|-------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| | tx | bytes | The request transaction bytes | 1 | | type | CheckTxType | What type of `CheckTx` request is this? At present, there are two possible values: `CheckTx_New` (the default, which says that a full check is required), and `CheckTx_Recheck` (when the mempool is initiating a normal recheck of a transaction). | 2 | -- **Response**: +* **Response**: | Name | Type | Description | Field Number | |------------|---------------------------|-----------------------------------------------------------------------|--------------| @@ -380,28 +390,28 @@ via light client. | sender | string | The transaction's sender (e.g. the signer) | 9 | | priority | int64 | The transaction's priority (for mempool ordering) | 10 | -- **Usage**: +* **Usage**: - - Technically optional - not involved in processing blocks. - - Guardian of the mempool: every node runs CheckTx before letting a + * Technically optional - not involved in processing blocks. + * Guardian of the mempool: every node runs CheckTx before letting a transaction into its local mempool. - - The transaction may come from an external user or another node - - CheckTx need not execute the transaction in full, but rather a light-weight + * The transaction may come from an external user or another node + * CheckTx need not execute the transaction in full, but rather a light-weight yet stateful validation, like checking signatures and account balances, but not running code in a virtual machine. - - Transactions where `ResponseCheckTx.Code != 0` will be rejected - they will not be broadcast to + * Transactions where `ResponseCheckTx.Code != 0` will be rejected - they will not be broadcast to other nodes or included in a proposal block. - - Tendermint attributes no other value to the response code + * Tendermint attributes no other value to the response code ### DeliverTx -- **Request**: +* **Request**: | Name | Type | Description | Field Number | |------|-------|--------------------------------|--------------| | tx | bytes | The request transaction bytes. | 1 | -- **Response**: +* **Response**: | Name | Type | Description | Field Number | |------------|---------------------------|-----------------------------------------------------------------------|--------------| @@ -414,20 +424,20 @@ via light client. | events | repeated [Event](#events) | Type & Key-Value events for indexing transactions (eg. by account). | 7 | | codespace | string | Namespace for the `code`. | 8 | -- **Usage**: - - The workhorse of the application - non-optional. - - Execute the transaction in full. - - `ResponseDeliverTx.Code == 0` only if the transaction is fully valid. +* **Usage**: + * The workhorse of the application * non-optional. + * Execute the transaction in full. + * `ResponseDeliverTx.Code == 0` only if the transaction is fully valid. ### EndBlock -- **Request**: +* **Request**: | Name | Type | Description | Field Number | |--------|-------|------------------------------------|--------------| | height | int64 | Height of the block just executed. | 1 | -- **Response**: +* **Response**: | Name | Type | Description | Field Number | |-------------------------|----------------------------------------------|-----------------------------------------------------------------|--------------| @@ -435,44 +445,44 @@ via light client. | consensus_param_updates | [ConsensusParams](#consensusparams) | Changes to consensus-critical time, size, and other parameters. | 2 | | events | repeated [Event](#events) | Type & Key-Value events for indexing | 3 | -- **Usage**: - - Signals the end of a block. - - Called after all transactions, prior to each Commit. - - Validator updates returned by block `H` impact blocks `H+1`, `H+2`, and +* **Usage**: + * Signals the end of a block. + * Called after all transactions, prior to each Commit. + * Validator updates returned by block `H` impact blocks `H+1`, `H+2`, and `H+3`, but only effects changes on the validator set of `H+2`: - - `H+1`: NextValidatorsHash - - `H+2`: ValidatorsHash (and thus the validator set) - - `H+3`: LastCommitInfo (ie. the last validator set) - - Consensus params returned for block `H` apply for block `H+1` + * `H+1`: NextValidatorsHash + * `H+2`: ValidatorsHash (and thus the validator set) + * `H+3`: LastCommitInfo (ie. the last validator set) + * Consensus params returned for block `H` apply for block `H+1` ### Commit -- **Request**: +* **Request**: | Name | Type | Description | Field Number | |--------|-------|------------------------------------|--------------| Empty request meant to signal to the app it can write state transitions to state. -- **Response**: +* **Response**: | Name | Type | Description | Field Number | |---------------|-------|------------------------------------------------------------------------|--------------| | data | bytes | The Merkle root hash of the application state. | 2 | | retain_height | int64 | Blocks below this height may be removed. Defaults to `0` (retain all). | 3 | -- **Usage**: - - Persist the application state. - - Return an (optional) Merkle root hash of the application state - - `ResponseCommit.Data` is included as the `Header.AppHash` in the next block - - it may be empty - - Later calls to `Query` can return proofs about the application state anchored +* **Usage**: + * Persist the application state. + * Return an (optional) Merkle root hash of the application state + * `ResponseCommit.Data` is included as the `Header.AppHash` in the next block + * it may be empty + * Later calls to `Query` can return proofs about the application state anchored in this Merkle root hash - - Note developers can return whatever they want here (could be nothing, or a - constant string, etc.), so long as it is deterministic - it must not be a + * Note developers can return whatever they want here (could be nothing, or a + constant string, etc.), so long as it is deterministic * it must not be a function of anything that did not come from the BeginBlock/DeliverTx/EndBlock methods. - - Use `RetainHeight` with caution! If all nodes in the network remove historical + * Use `RetainHeight` with caution! If all nodes in the network remove historical blocks then this data is permanently lost, and no new nodes will be able to join the network and bootstrap. Historical blocks may also be required for other purposes, e.g. auditing, replay of non-persisted heights, light client @@ -480,26 +490,26 @@ via light client. ### ListSnapshots -- **Request**: +* **Request**: | Name | Type | Description | Field Number | |--------|-------|------------------------------------|--------------| Empty request asking the application for a list of snapshots. -- **Response**: +* **Response**: | Name | Type | Description | Field Number | |-----------|--------------------------------|--------------------------------|--------------| | snapshots | repeated [Snapshot](#snapshot) | List of local state snapshots. | 1 | -- **Usage**: - - Used during state sync to discover available snapshots on peers. - - See `Snapshot` data type for details. +* **Usage**: + * Used during state sync to discover available snapshots on peers. + * See `Snapshot` data type for details. ### LoadSnapshotChunk -- **Request**: +* **Request**: | Name | Type | Description | Field Number | |--------|--------|-----------------------------------------------------------------------|--------------| @@ -507,25 +517,25 @@ via light client. | format | uint32 | The application-specific format of the snapshot the chunk belongs to. | 2 | | chunk | uint32 | The chunk index, starting from `0` for the initial chunk. | 3 | -- **Response**: +* **Response**: | Name | Type | Description | Field Number | |-------|-------|-------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| | chunk | bytes | The binary chunk contents, in an arbitray format. Chunk messages cannot be larger than 16 MB _including metadata_, so 10 MB is a good starting point. | 1 | -- **Usage**: - - Used during state sync to retrieve snapshot chunks from peers. +* **Usage**: + * Used during state sync to retrieve snapshot chunks from peers. ### OfferSnapshot -- **Request**: +* **Request**: | Name | Type | Description | Field Number | |----------|-----------------------|--------------------------------------------------------------------------|--------------| | snapshot | [Snapshot](#snapshot) | The snapshot offered for restoration. | 1 | | app_hash | bytes | The light client-verified app hash for this height, from the blockchain. | 2 | -- **Response**: +* **Response**: | Name | Type | Description | Field Number | |--------|-------------------|-----------------------------------|--------------| @@ -544,21 +554,21 @@ via light client. } ``` -- **Usage**: - - `OfferSnapshot` is called when bootstrapping a node using state sync. The application may +* **Usage**: + * `OfferSnapshot` is called when bootstrapping a node using state sync. The application may accept or reject snapshots as appropriate. Upon accepting, Tendermint will retrieve and apply snapshot chunks via `ApplySnapshotChunk`. The application may also choose to reject a snapshot in the chunk response, in which case it should be prepared to accept further `OfferSnapshot` calls. - - Only `AppHash` can be trusted, as it has been verified by the light client. Any other data + * Only `AppHash` can be trusted, as it has been verified by the light client. Any other data can be spoofed by adversaries, so applications should employ additional verification schemes to avoid denial-of-service attacks. The verified `AppHash` is automatically checked against the restored application at the end of snapshot restoration. - - For more information, see the `Snapshot` data type or the [state sync section](apps.md#state-sync). + * For more information, see the `Snapshot` data type or the [state sync section](apps.md#state-sync). ### ApplySnapshotChunk -- **Request**: +* **Request**: | Name | Type | Description | Field Number | |--------|--------|-----------------------------------------------------------------------------|--------------| @@ -566,7 +576,7 @@ via light client. | chunk | bytes | The binary chunk contents, as returned by `LoadSnapshotChunk`. | 2 | | sender | string | The P2P ID of the node who sent this chunk. | 3 | -- **Response**: +* **Response**: | Name | Type | Description | Field Number | |----------------|---------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| @@ -585,16 +595,16 @@ via light client. } ``` -- **Usage**: - - The application can choose to refetch chunks and/or ban P2P peers as appropriate. Tendermint +* **Usage**: + * The application can choose to refetch chunks and/or ban P2P peers as appropriate. Tendermint will not do this unless instructed by the application. - - The application may want to verify each chunk, e.g. by attaching chunk hashes in + * The application may want to verify each chunk, e.g. by attaching chunk hashes in `Snapshot.Metadata` and/or incrementally verifying contents against `AppHash`. - - When all chunks have been accepted, Tendermint will make an ABCI `Info` call to verify that + * When all chunks have been accepted, Tendermint will make an ABCI `Info` call to verify that `LastBlockAppHash` and `LastBlockHeight` matches the expected values, and record the `AppVersion` in the node state. It then switches to fast sync or consensus and joins the network. - - If Tendermint is unable to retrieve the next chunk after some time (e.g. because no suitable + * If Tendermint is unable to retrieve the next chunk after some time (e.g. because no suitable peers are available), it will reject the snapshot and try a different one via `OfferSnapshot`. The application should be prepared to reset and accept it or abort as appropriate. @@ -604,48 +614,48 @@ The data types not listed below are the same as the [core data structures](../co ### Validator -- **Fields**: +* **Fields**: | Name | Type | Description | Field Number | |---------|-------|---------------------------------------------------------------------|--------------| | address | bytes | Address of the validator (the first 20 bytes of SHA256(public key)) | 1 | | power | int64 | Voting power of the validator | 3 | -- **Usage**: - - Validator identified by address - - Used in RequestBeginBlock as part of VoteInfo - - Does not include PubKey to avoid sending potentially large quantum pubkeys +* **Usage**: + * Validator identified by address + * Used in RequestBeginBlock as part of VoteInfo + * Does not include PubKey to avoid sending potentially large quantum pubkeys over the ABCI ### ValidatorUpdate -- **Fields**: +* **Fields**: | Name | Type | Description | Field Number | |---------|--------------------------------------------------|-------------------------------|--------------| | pub_key | [Public Key](../core/data_structures.md#pub_key) | Public key of the validator | 1 | | power | int64 | Voting power of the validator | 2 | -- **Usage**: - - Validator identified by PubKey - - Used to tell Tendermint to update the validator set +* **Usage**: + * Validator identified by PubKey + * Used to tell Tendermint to update the validator set ### VoteInfo -- **Fields**: +* **Fields**: | Name | Type | Description | Field Number | |-------------------|-------------------------|--------------------------------------------------------------|--------------| | validator | [Validator](#validator) | A validator | 1 | | signed_last_block | bool | Indicates whether or not the validator signed the last block | 2 | -- **Usage**: - - Indicates whether a validator signed the last block, allowing for rewards +* **Usage**: + * Indicates whether a validator signed the last block, allowing for rewards based on validator availability ### Evidence -- **Fields**: +* **Fields**: | Name | Type | Description | Field Number | |--------------------|--------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------|--------------| @@ -657,7 +667,7 @@ The data types not listed below are the same as the [core data structures](../co #### EvidenceType -- **Fields** +* **Fields** EvidenceType is an enum with the listed fields: @@ -669,7 +679,7 @@ The data types not listed below are the same as the [core data structures](../co ### LastCommitInfo -- **Fields**: +* **Fields**: | Name | Type | Description | Field Number | |-------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|--------------| @@ -678,7 +688,7 @@ The data types not listed below are the same as the [core data structures](../co ### ConsensusParams -- **Fields**: +* **Fields**: | Name | Type | Description | Field Number | |-----------|---------------------------------------------------------------|------------------------------------------------------------------------------|--------------| @@ -689,7 +699,7 @@ The data types not listed below are the same as the [core data structures](../co ### ProofOps -- **Fields**: +* **Fields**: | Name | Type | Description | Field Number | |------|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| @@ -697,7 +707,7 @@ The data types not listed below are the same as the [core data structures](../co ### ProofOp -- **Fields**: +* **Fields**: | Name | Type | Description | Field Number | |------|--------|------------------------------------------------|--------------| @@ -707,7 +717,7 @@ The data types not listed below are the same as the [core data structures](../co ### Snapshot -- **Fields**: +* **Fields**: | Name | Type | Description | Field Number | |----------|--------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| @@ -717,8 +727,8 @@ The data types not listed below are the same as the [core data structures](../co | hash | bytes | TAn arbitrary snapshot hash. Must be equal only for identical snapshots across nodes. Tendermint does not interpret the hash, it only compares them. | 3 | | metadata | bytes | Arbitrary application metadata, for example chunk hashes or other verification data. | 3 | -- **Usage**: - - Used for state sync snapshots, see [separate section](apps.md#state-sync) for details. - - A snapshot is considered identical across nodes only if _all_ fields are equal (including +* **Usage**: + * Used for state sync snapshots, see [separate section](apps.md#state-sync) for details. + * A snapshot is considered identical across nodes only if _all_ fields are equal (including `Metadata`). Chunks may be retrieved from all nodes that have the same snapshot. - - When sent across the network, a snapshot message can be at most 4 MB. + * When sent across the network, a snapshot message can be at most 4 MB.