Browse Source
Merge pull request #2343 from tendermint/release/v0.24.0
Major spec update to prepare v0.24.0 for releasepull/2349/head
Ethan Buchman
6 years ago
committed by
GitHub
GitHub
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
16 changed files with 1013 additions and 358 deletions
Split View
Diff Options
-
+97 -0CHANGELOG.md
-
+1 -59CHANGELOG_PENDING.md
-
+5 -5abci/version/version.go
-
+11 -2docs/app-dev/abci-spec.md
-
+5 -0docs/app-dev/app-development.md
-
+19 -0docs/spec/abci/README.md
-
+372 -0docs/spec/abci/abci.md
-
+207 -0docs/spec/abci/apps.md
-
+104 -0docs/spec/abci/client-server.md
-
+109 -79docs/spec/blockchain/blockchain.md
-
+7 -10docs/spec/blockchain/encoding.md
-
+66 -16docs/spec/blockchain/state.md
-
+1 -183docs/spec/software/abci.md
-
+1 -1docs/tendermint-core/using-tendermint.md
-
+2 -0types/time/time.go
-
+6 -3version/version.go
+ 97
- 0
CHANGELOG.md
View File
+ 1
- 59
CHANGELOG_PENDING.md
View File
| @ -1,80 +1,22 @@ | |||
| # Pending | |||
| Special thanks to external contributors with PRs included in this release: ackratos, james-ray, bradyjoestar, | |||
| peerlink, Ahmah2009, bluele, b00f | |||
| Special thanks to external contributors with PRs included in this release: | |||
| BREAKING CHANGES: | |||
| * CLI/RPC/Config | |||
| - [config] \#2169 Replace MaxNumPeers with MaxNumInboundPeers and MaxNumOutboundPeers | |||
| - [config] \#2300 Reduce default mempool size from 100k to 5k, until ABCI rechecking is implemented. | |||
| - [rpc] \#1815 `/commit` returns a `signed_header` field instead of everything being top-level | |||
| * Apps | |||
| - [abci] Added address of the original proposer of the block to Header | |||
| - [abci] Change ABCI Header to match Tendermint exactly | |||
| - [abci] \#2159 Update use of `Validator` (see | |||
| [ADR-018](https://github.com/tendermint/tendermint/blob/develop/docs/architecture/adr-018-ABCI-Validators.md)): | |||
| - Remove PubKey from `Validator` (so it's just Address and Power) | |||
| - Introduce `ValidatorUpdate` (with just PubKey and Power) | |||
| - InitChain and EndBlock use ValidatorUpdate | |||
| - Update field names and types in BeginBlock | |||
| - [state] \#1815 Validator set changes are now delayed by one block | |||
| - updates returned in ResponseEndBlock for block H will be included in RequestBeginBlock for block H+2 | |||
| * Go API | |||
| - [lite] \#1815 Complete refactor of the package | |||
| - [node] \#2212 NewNode now accepts a `*p2p.NodeKey` (@bradyjoestar) | |||
| - [libs/common] \#2199 Remove Fmt, in favor of fmt.Sprintf | |||
| - [libs/common] SplitAndTrim was deleted | |||
| - [libs/common] \#2274 Remove unused Math functions like MaxInt, MaxInt64, | |||
| MinInt, MinInt64 (@Ahmah2009) | |||
| - [libs/clist] Panics if list extends beyond MaxLength | |||
| - [crypto] \#2205 Rename AminoRoute variables to no longer be prefixed by signature type. | |||
| * Blockchain Protocol | |||
| - [state] \#1815 Validator set changes are now delayed by one block (!) | |||
| - Add NextValidatorSet to State, changes on-disk representation of state | |||
| - [state] \#2184 Enforce ConsensusParams.BlockSize.MaxBytes (See | |||
| [ADR-020](https://github.com/tendermint/tendermint/blob/develop/docs/architecture/adr-020-block-size.md)). | |||
| - Remove ConsensusParams.BlockSize.MaxTxs | |||
| - Introduce maximum sizes for all components of a block, including ChainID | |||
| - [types] Updates to the block Header: | |||
| - \#1815 NextValidatorsHash - hash of the validator set for the next block, | |||
| so the current validators actually sign over the hash for the new | |||
| validators | |||
| - \#2106 ProposerAddress - address of the block's original proposer | |||
| - [consensus] \#2203 Implement BFT time | |||
| - Timestamp in block must be monotonic and equal the median of timestamps in block's LastCommit | |||
| - [crypto] \#2239 Secp256k1 signature changes (See | |||
| [ADR-014](https://github.com/tendermint/tendermint/blob/develop/docs/architecture/adr-014-secp-malleability.md)): | |||
| - format changed from DER to `r || s`, both little endian encoded as 32 bytes. | |||
| - malleability removed by requiring `s` to be in canonical form. | |||
| * P2P Protocol | |||
| - [p2p] \#2263 Update secret connection to use a little endian encoded nonce | |||
| - [blockchain] \#2213 Fix Amino routes for blockchain reactor messages | |||
| (@peerlink) | |||
| FEATURES: | |||
| - [types] \#2015 Allow genesis file to have 0 validators (@b00f) | |||
| - Initial validator set can be determined by the app in ResponseInitChain | |||
| - [rpc] \#2161 New event `ValidatorSetUpdates` for when the validator set changes | |||
| - [crypto/multisig] \#2164 Introduce multisig pubkey and signature format | |||
| - [libs/db] \#2293 Allow passing options through when creating instances of leveldb dbs | |||
| IMPROVEMENTS: | |||
| - [docs] Lint documentation with `write-good` and `stop-words`. | |||
| - [scripts] \#2196 Added json2wal tool, which is supposed to help our users restore (@bradyjoestar) | |||
| corrupted WAL files and compose test WAL files (@bradyjoestar) | |||
| - [mempool] \#2234 Now stores txs by hash inside of the cache, to mitigate memory leakage | |||
| - [mempool] \#2166 Set explicit capacity for map when updating txs (@bluele) | |||
| BUG FIXES: | |||
| - [config] \#2284 Replace `db_path` with `db_dir` from automatically generated configuration files. | |||
| - [mempool] \#2188 Fix OOM issue from cache map and list getting out of sync | |||
| - [state] \#2051 KV store index supports searching by `tx.height` (@ackratos) | |||
| - [rpc] \#2327 `/dial_peers` does not try to dial existing peers | |||
| - [node] \#2323 Filter empty strings from config lists (@james-ray) | |||
| - [abci/client] \#2236 Fix closing GRPC connection (@bradyjoestar) | |||
+ 5
- 5
abci/version/version.go
View File
| @ -1,9 +1,9 @@ | |||
| package version | |||
| // NOTE: we should probably be versioning the ABCI and the abci-cli separately | |||
| import ( | |||
| "github.com/tendermint/tendermint/version" | |||
| ) | |||
| const Maj = "0" | |||
| const Min = "12" | |||
| const Fix = "0" | |||
| // TODO: eliminate this after some version refactor | |||
| const Version = "0.12.0" | |||
| const Version = version.ABCIVersion | |||
+ 11
- 2
docs/app-dev/abci-spec.md
View File
+ 5
- 0
docs/app-dev/app-development.md
View File
+ 19
- 0
docs/spec/abci/README.md
View File
| @ -0,0 +1,19 @@ | |||
| # ABCI | |||
| ABCI is the interface between Tendermint (a state-machine replication engine) | |||
| and an application (the actual state machine). It consists of a set of | |||
| *methods*, where each method has a corresponding `Request` and `Response` | |||
| message type. Tendermint calls the ABCI methods on the ABCI application by sending the `Request*` | |||
| messages and receiving the `Response*` messages in return. | |||
| All message types are defined in a [protobuf file](https://github.com/tendermint/tendermint/blob/develop/abci/types/types.proto). | |||
| This allows Tendermint to run applications written in any programming language. | |||
| This specification is split as follows: | |||
| - [Methods and Types](abci.md) - complete details on all ABCI methods and | |||
| message types | |||
| - [Applications](apps.md) - how to manage ABCI application state and other | |||
| details about building ABCI applications | |||
| - [Client and Server](client-server.md) - for those looking to implement their | |||
| own ABCI application servers | |||
+ 372
- 0
docs/spec/abci/abci.md
View File
| @ -0,0 +1,372 @@ | |||
| # ABCI Methods and Types | |||
| ## Overview | |||
| The ABCI message types are defined in a [protobuf | |||
| file](https://github.com/tendermint/tendermint/blob/develop/abci/types/types.proto). | |||
| ABCI methods are split across 3 separate ABCI *connections*: | |||
| - `Consensus Connection: InitChain, BeginBlock, DeliverTx, EndBlock, Commit` | |||
| - `Mempool Connection: CheckTx` | |||
| - `Info Connection: Info, SetOption, Query` | |||
| The `Consensus Connection` is driven by a consensus protocol and is responsible | |||
| for block exection. | |||
| The `Mempool Connection` is for validating new transactions, before they're | |||
| shared or included in a block. | |||
| The `Info Connection` is for initialization and for queries from the user. | |||
| Additionally, there is a `Flush` method that is called on every connection, | |||
| and an `Echo` method that is just for debugging. | |||
| ## Errors | |||
| Some methods (`Echo, Info, InitChain, BeginBlock, EndBlock, Commit`), | |||
| don't return errors because an error would indicate a critical failure | |||
| in the application and there's nothing Tendermint can do. The problem | |||
| should be addressed and both Tendermint and the application restarted. | |||
| All other methods (`SetOption, Query, CheckTx, DeliverTx`) return an | |||
| application-specific response `Code uint32`, where only `0` is reserved | |||
| for `OK`. | |||
| ## Tags | |||
| Some methods (`CheckTx, BeginBlock, DeliverTx, EndBlock`) | |||
| include a `Tags` field in their `Response*`. Each tag is key-value pair denoting | |||
| something about what happened during the methods execution. | |||
| Tags can be used to index transactions and blocks according to what happened | |||
| during their execution. | |||
| Keys and values in tags must be UTF-8 encoded strings (e.g. | |||
| "account.owner": "Bob", "balance": "100.0", | |||
| "time": "2018-01-02T12:30:00Z") | |||
| ## Determinism | |||
| Some methods (`SetOption, Query, CheckTx, DeliverTx`) return | |||
| non-deterministic data in the form of `Info` and `Log`. The `Log` is | |||
| intended for the literal output from the application's logger, while the | |||
| `Info` is any additional info that should be returned. | |||
| All other fields in the `Response*` of all methods must be strictly deterministic. | |||
| For this reason, it is recommended that applications not be exposed to any | |||
| external user or process except via the ABCI connections to a consensus engine | |||
| like Tendermint Core. | |||
| ## Block Execution | |||
| The first time a new blockchain is started, Tendermint calls | |||
| `InitChain`. From then on, the follow sequence of methods is executed for each | |||
| block: | |||
| `BeginBlock, [DeliverTx], EndBlock, Commit` | |||
| where one `DeliverTx` is called for each transaction in the block. | |||
| The result is an updated application state. | |||
| Cryptographic commitments to the results of DeliverTx, EndBlock, and | |||
| Commit are included in the header of the next block. | |||
| ## Messages | |||
| ### Echo | |||
| - **Request**: | |||
| - `Message (string)`: A string to echo back | |||
| - **Response**: | |||
| - `Message (string)`: The input string | |||
| - **Usage**: | |||
| - Echo a string to test an abci client/server implementation | |||
| ### Flush | |||
| - **Usage**: | |||
| - Signals that messages queued on the client should be flushed to | |||
| the server. It is called periodically by the client | |||
| implementation to ensure asynchronous requests are actually | |||
| sent, and is called immediately to make a synchronous request, | |||
| which returns when the Flush response comes back. | |||
| ### Info | |||
| - **Request**: | |||
| - `Version (string)`: The Tendermint version | |||
| - **Response**: | |||
| - `Data (string)`: Some arbitrary information | |||
| - `Version (Version)`: Version information | |||
| - `LastBlockHeight (int64)`: Latest block for which the app has | |||
| called Commit | |||
| - `LastBlockAppHash ([]byte)`: Latest result of Commit | |||
| - **Usage**: | |||
| - Return information about the application state. | |||
| - Used to sync Tendermint with the application during a handshake | |||
| that happens on startup. | |||
| - Tendermint expects `LastBlockAppHash` and `LastBlockHeight` to | |||
| be updated during `Commit`, ensuring that `Commit` is never | |||
| called twice for the same block height. | |||
| ### SetOption | |||
| - **Request**: | |||
| - `Key (string)`: Key to set | |||
| - `Value (string)`: Value to set for key | |||
| - **Response**: | |||
| - `Code (uint32)`: Response code | |||
| - `Log (string)`: The output of the application's logger. May | |||
| be non-deterministic. | |||
| - `Info (string)`: Additional information. May | |||
| be non-deterministic. | |||
| - **Usage**: | |||
| - Set non-consensus critical application specific options. | |||
| - e.g. Key="min-fee", Value="100fermion" could set the minimum fee | |||
| required for CheckTx (but not DeliverTx - that would be | |||
| consensus critical). | |||
| ### InitChain | |||
| - **Request**: | |||
| - `Time (google.protobuf.Timestamp)`: Genesis time. | |||
| - `ChainID (string)`: ID of the blockchain. | |||
| - `ConsensusParams (ConsensusParams)`: Initial consensus-critical parameters. | |||
| - `Validators ([]ValidatorUpdate)`: Initial genesis validators. | |||
| - `AppStateBytes ([]byte)`: Serialized initial application state. Amino-encoded JSON bytes. | |||
| - **Response**: | |||
| - `ConsensusParams (ConsensusParams)`: Initial | |||
| consensus-critical parameters. | |||
| - `Validators ([]ValidatorUpdate)`: Initial validator set (if non empty). | |||
| - **Usage**: | |||
| - Called once upon genesis. | |||
| - If ResponseInitChain.Validators is empty, the initial validator set will be the RequestInitChain.Validators | |||
| - If ResponseInitChain.Validators is not empty, the initial validator set will be the | |||
| ResponseInitChain.Validators (regardless of what is in RequestInitChain.Validators). | |||
| - This allows the app to decide if it wants to accept the initial validator | |||
| set proposed by tendermint (ie. in the genesis file), or if it wants to use | |||
| a different one (perhaps computed based on some application specific | |||
| information in the genesis file). | |||
| ### Query | |||
| - **Request**: | |||
| - `Data ([]byte)`: Raw query bytes. Can be used with or in lieu | |||
| of Path. | |||
| - `Path (string)`: Path of request, like an HTTP GET path. Can be | |||
| used with or in liue of Data. | |||
| - Apps MUST interpret '/store' as a query by key on the | |||
| underlying store. The key SHOULD be specified in the Data field. | |||
| - Apps SHOULD allow queries over specific types like | |||
| '/accounts/...' or '/votes/...' | |||
| - `Height (int64)`: The block height for which you want the query | |||
| (default=0 returns data for the latest committed block). Note | |||
| that this is the height of the block containing the | |||
| application's Merkle root hash, which represents the state as it | |||
| was after committing the block at Height-1 | |||
| - `Prove (bool)`: Return Merkle proof with response if possible | |||
| - **Response**: | |||
| - `Code (uint32)`: Response code. | |||
| - `Log (string)`: The output of the application's logger. May | |||
| be non-deterministic. | |||
| - `Info (string)`: Additional information. May | |||
| be non-deterministic. | |||
| - `Index (int64)`: The index of the key in the tree. | |||
| - `Key ([]byte)`: The key of the matching data. | |||
| - `Value ([]byte)`: The value of the matching data. | |||
| - `Proof ([]byte)`: Proof for the data, if requested. | |||
| - `Height (int64)`: The block height from which data was derived. | |||
| Note that this is the height of the block containing the | |||
| application's Merkle root hash, which represents the state as it | |||
| was after committing the block at Height-1 | |||
| - **Usage**: | |||
| - Query for data from the application at current or past height. | |||
| - Optionally return Merkle proof. | |||
| ### BeginBlock | |||
| - **Request**: | |||
| - `Hash ([]byte)`: The block's hash. This can be derived from the | |||
| block header. | |||
| - `Header (struct{})`: The block header. | |||
| - `LastCommitInfo (LastCommitInfo)`: Info about the last commit, including the | |||
| round, and the list of validators and which ones signed the last block. | |||
| - `ByzantineValidators ([]Evidence)`: List of evidence of | |||
| validators that acted maliciously. | |||
| - **Response**: | |||
| - `Tags ([]cmn.KVPair)`: Key-Value tags for filtering and indexing | |||
| - **Usage**: | |||
| - Signals the beginning of a new block. Called prior to | |||
| any DeliverTxs. | |||
| - The header contains the height, timestamp, and more - it exactly matches the | |||
| Tendermint block header. We may seek to generalize this in the future. | |||
| - The `LastCommitInfo` and `ByzantineValidators` can be used to determine | |||
| rewards and punishments for the validators. NOTE validators here do not | |||
| include pubkeys. | |||
| ### CheckTx | |||
| - **Request**: | |||
| - `Tx ([]byte)`: The request transaction bytes | |||
| - **Response**: | |||
| - `Code (uint32)`: Response code | |||
| - `Data ([]byte)`: Result bytes, if any. | |||
| - `Log (string)`: The output of the application's logger. May | |||
| be non-deterministic. | |||
| - `Info (string)`: Additional information. May | |||
| be non-deterministic. | |||
| - `GasWanted (int64)`: Amount of gas request for transaction. | |||
| - `GasUsed (int64)`: Amount of gas consumed by transaction. | |||
| - `Tags ([]cmn.KVPair)`: Key-Value tags for filtering and indexing | |||
| transactions (eg. by account). | |||
| - **Usage**: | |||
| - Technically optional - not involved in processing blocks. | |||
| - Guardian of the mempool: every node runs CheckTx before letting a | |||
| transaction into its local mempool. | |||
| - The transaction may come from an external user or another node | |||
| - CheckTx need not execute the transaction in full, but rather a light-weight | |||
| yet stateful validation, like checking signatures and account balances, but | |||
| not running code in a virtual machine. | |||
| - Transactions where `ResponseCheckTx.Code != 0` will be rejected - they will not be broadcast to | |||
| other nodes or included in a proposal block. | |||
| - Tendermint attributes no other value to the response code | |||
| ### DeliverTx | |||
| - **Request**: | |||
| - `Tx ([]byte)`: The request transaction bytes. | |||
| - **Response**: | |||
| - `Code (uint32)`: Response code. | |||
| - `Data ([]byte)`: Result bytes, if any. | |||
| - `Log (string)`: The output of the application's logger. May | |||
| be non-deterministic. | |||
| - `Info (string)`: Additional information. May | |||
| be non-deterministic. | |||
| - `GasWanted (int64)`: Amount of gas requested for transaction. | |||
| - `GasUsed (int64)`: Amount of gas consumed by transaction. | |||
| - `Tags ([]cmn.KVPair)`: Key-Value tags for filtering and indexing | |||
| transactions (eg. by account). | |||
| - **Usage**: | |||
| - The workhorse of the application - non-optional. | |||
| - Execute the transaction in full. | |||
| - `ResponseDeliverTx.Code == 0` only if the transaction is fully valid. | |||
| ### EndBlock | |||
| - **Request**: | |||
| - `Height (int64)`: Height of the block just executed. | |||
| - **Response**: | |||
| - `ValidatorUpdates ([]ValidatorUpdate)`: Changes to validator set (set | |||
| voting power to 0 to remove). | |||
| - `ConsensusParamUpdates (ConsensusParams)`: Changes to | |||
| consensus-critical time, size, and other parameters. | |||
| - `Tags ([]cmn.KVPair)`: Key-Value tags for filtering and indexing | |||
| - **Usage**: | |||
| - Signals the end of a block. | |||
| - Called after all transactions, prior to each Commit. | |||
| - Validator updates returned by block `H` impact blocks `H+1`, `H+2`, and | |||
| `H+3`, but only effects changes on the validator set of `H+2`: | |||
| - `H+1`: NextValidatorsHash | |||
| - `H+2`: ValidatorsHash (and thus the validator set) | |||
| - `H+3`: LastCommitInfo (ie. the last validator set) | |||
| - Consensus params returned for block `H` apply for block `H+1` | |||
| ### Commit | |||
| - **Response**: | |||
| - `Data ([]byte)`: The Merkle root hash | |||
| - **Usage**: | |||
| - Persist the application state. | |||
| - Return a Merkle root hash of the application state. | |||
| - It's critical that all application instances return the | |||
| same hash. If not, they will not be able to agree on the next | |||
| block, because the hash is included in the next block! | |||
| ## Data Types | |||
| ### Header | |||
| - **Fields**: | |||
| - `ChainID (string)`: ID of the blockchain | |||
| - `Height (int64)`: Height of the block in the chain | |||
| - `Time (google.protobuf.Timestamp)`: Time of the block. It is the proposer's | |||
| local time when block was created. | |||
| - `NumTxs (int32)`: Number of transactions in the block | |||
| - `TotalTxs (int64)`: Total number of transactions in the blockchain until | |||
| now | |||
| - `LastBlockID (BlockID)`: Hash of the previous (parent) block | |||
| - `LastCommitHash ([]byte)`: Hash of the previous block's commit | |||
| - `ValidatorsHash ([]byte)`: Hash of the validator set for this block | |||
| - `NextValidatorsHash ([]byte)`: Hash of the validator set for the next block | |||
| - `ConsensusHash ([]byte)`: Hash of the consensus parameters for this block | |||
| - `AppHash ([]byte)`: Data returned by the last call to `Commit` - typically the | |||
| Merkle root of the application state after executing the previous block's | |||
| transactions | |||
| - `LastResultsHash ([]byte)`: Hash of the ABCI results returned by the last block | |||
| - `EvidenceHash ([]byte)`: Hash of the evidence included in this block | |||
| - `ProposerAddress ([]byte)`: Original proposer for the block | |||
| - **Usage**: | |||
| - Provided in RequestBeginBlock | |||
| - Provides important context about the current state of the blockchain - | |||
| especially height and time. | |||
| - Provides the proposer of the current block, for use in proposer-based | |||
| reward mechanisms. | |||
| ### Validator | |||
| - **Fields**: | |||
| - `Address ([]byte)`: Address of the validator (hash of the public key) | |||
| - `Power (int64)`: Voting power of the validator | |||
| - **Usage**: | |||
| - Validator identified by address | |||
| - Used in RequestBeginBlock as part of VoteInfo | |||
| - Does not include PubKey to avoid sending potentially large quantum pubkeys | |||
| over the ABCI | |||
| ### ValidatorUpdate | |||
| - **Fields**: | |||
| - `PubKey (PubKey)`: Public key of the validator | |||
| - `Power (int64)`: Voting power of the validator | |||
| - **Usage**: | |||
| - Validator identified by PubKey | |||
| - Used to tell Tendermint to update the validator set | |||
| ### VoteInfo | |||
| - **Fields**: | |||
| - `Validator (Validator)`: A validator | |||
| - `SignedLastBlock (bool)`: Indicates whether or not the validator signed | |||
| the last block | |||
| - **Usage**: | |||
| - Indicates whether a validator signed the last block, allowing for rewards | |||
| based on validator availability | |||
| ### PubKey | |||
| - **Fields**: | |||
| - `Type (string)`: Type of the public key. A simple string like `"ed25519"`. | |||
| In the future, may indicate a serialization algorithm to parse the `Data`, | |||
| for instance `"amino"`. | |||
| - `Data ([]byte)`: Public key data. For a simple public key, it's just the | |||
| raw bytes. If the `Type` indicates an encoding algorithm, this is the | |||
| encoded public key. | |||
| - **Usage**: | |||
| - A generic and extensible typed public key | |||
| ### Evidence | |||
| - **Fields**: | |||
| - `Type (string)`: Type of the evidence. A hierarchical path like | |||
| "duplicate/vote". | |||
| - `Validator (Validator`: The offending validator | |||
| - `Height (int64)`: Height when the offense was committed | |||
| - `Time (google.protobuf.Timestamp)`: Time of the block at height `Height`. | |||
| It is the proposer's local time when block was created. | |||
| - `TotalVotingPower (int64)`: Total voting power of the validator set at | |||
| height `Height` | |||
| ### LastCommitInfo | |||
| - **Fields**: | |||
| - `Round (int32)`: Commit round. | |||
| - `Votes ([]VoteInfo)`: List of validators addresses in the last validator set | |||
| with their voting power and whether or not they signed a vote. | |||
+ 207
- 0
docs/spec/abci/apps.md
View File
| @ -0,0 +1,207 @@ | |||
| # ABCI Applications | |||
| Please ensure you've first read the spec for [ABCI Methods and Types](abci.md) | |||
| Here we cover the following components of ABCI applications: | |||
| - [State](#State) - the interplay between ABCI connections and application state | |||
| and the differences between `CheckTx` and `DeliverTx`. | |||
| - [Validator Set Updates](#Validator-Set-Updates) - how validator sets are | |||
| changed during `InitChain` and `EndBlock` | |||
| - [Query](#Query) - standards for using the `Query` method | |||
| - [Crash Recovery](#Crash-Recovery) - handshake protocol to synchronize | |||
| Tendermint and the application on startup. | |||
| ## State | |||
| Since Tendermint maintains multiple concurrent ABCI connections, it is typical | |||
| for an application to maintain a distinct state for each, and for the states to | |||
| be sycnronized during `Commit`. | |||
| ### Commit | |||
| Before `Commit` is called, Tendermint locks and flushes the mempool so that no new messages will | |||
| be received on the mempool connection. This provides an opportunity to safely update all three | |||
| states to the latest committed state at once. | |||
| When `Commit` completes, it unlocks the mempool. | |||
| Note that it is not possible to send transactions to Tendermint during `Commit` - if your app | |||
| tries to send a `/broadcast_tx` to Tendermint during Commit, it will deadlock. | |||
| ### Consensus Connection | |||
| The Consensus Connection should maintain a `DeliverTxState` - | |||
| the working state for block execution. It should be updated by the calls to | |||
| `BeginBlock`, `DeliverTx`, and `EndBlock` during block execution and committed to | |||
| disk as the "latest committed state" during `Commit`. | |||
| Updates made to the DeliverTxState by each method call must be readable by each subsequent method - | |||
| ie. the updates are linearizeable. | |||
| ### Mempool Connection | |||
| The Mempool Connection should maintain a `CheckTxState` - | |||
| to process pending transactions in the mempool that have | |||
| not yet been committed. It should be initialized to the latest committed state | |||
| at the end of every `Commit`. Note it may be updated concurrently with the | |||
| DeliverTxState. | |||
| Before calling `Commit`, Tendermint will lock and flush the mempool, | |||
| ensuring that all existing CheckTx are responded to and no new ones can | |||
| begin. | |||
| After `Commit`, CheckTx is run again on all transactions that remain in the | |||
| node's local mempool after filtering those included in the block. To prevent the | |||
| mempool from rechecking all transactions every time a block is committed, set | |||
| the configuration option `mempool.recheck=false`. | |||
| Finally, the mempool will unlock and new transactions can be processed through CheckTx again. | |||
| Note that CheckTx doesn't have to check everything that affects transaction validity; the | |||
| expensive things can be skipped. In fact, CheckTx doesn't have to check | |||
| anything; it might say that any transaction is a valid transaction. | |||
| Unlike DeliverTx, CheckTx is just there as | |||
| a sort of weak filter to keep invalid transactions out of the blockchain. It's | |||
| weak, because a Byzantine node doesn't care about CheckTx; it can propose a | |||
| block full of invalid transactions if it wants. | |||
| ### Info Connection | |||
| The Mempool Connection should maintain a `QueryState` for answering queries from the user, | |||
| and for initialization when Tendermint first starts up. | |||
| It should always contain the latest committed state associated with the | |||
| latest commited block. | |||
| QueryState should be set to the latest `DeliverTxState` at the end of every `Commit`, | |||
| ie. after the full block has been processed and the state committed to disk. | |||
| Otherwise it should never be modified. | |||
| ## Validator Updates | |||
| ### EndBlock | |||
| Updates to the Tendermint validator set can be made by returning | |||
| `ValidatorUpdate` objects in the `ResponseEndBlock`: | |||
| ``` | |||
| message ValidatorUpdate { | |||
| PubKey pub_key | |||
| int64 power | |||
| } | |||
| message PubKey { | |||
| string type | |||
| bytes data | |||
| } | |||
| ``` | |||
| The `pub_key` currently supports only one type: | |||
| - `type = "ed25519" and`data = <raw 32-byte public key>` | |||
| The `power` is the new voting power for the validator, with the | |||
| following rules: | |||
| - power must be non-negative | |||
| - if power is 0, the validator must already exist, and will be removed from the | |||
| validator set | |||
| - if power is non-0: | |||
| - if the validator does not already exist, it will be added to the validator | |||
| set with the given power | |||
| - if the validator does already exist, its power will be adjusted to the given power | |||
| ### InitChain | |||
| ResponseInitChain has the option to return a list of validators. | |||
| If the list is not empty, Tendermint will adopt it for the validator set. | |||
| This way the application can determine the initial validator set for the | |||
| blockchain. | |||
| ResponseInitChain also includes ConsensusParams, but these are presently | |||
| ignored. | |||
| ## Query | |||
| Query is a generic message type with lots of flexibility to enable diverse sets | |||
| of queries from applications. Tendermint has no requirements from the Query | |||
| message for normal operation - that is, the ABCI app developer need not implement Query functionality if they do not wish too. | |||
| That said, Tendermint makes a number of queries to support some optional | |||
| features. These are: | |||
| ### Peer Filtering | |||
| When Tendermint connects to a peer, it sends two queries to the ABCI application | |||
| using the following paths, with no additional data: | |||
| - `/p2p/filter/addr/<IP:PORT>`, where `<IP:PORT>` denote the IP address and | |||
| the port of the connection | |||
| - `p2p/filter/id/<ID>`, where `<ID>` is the peer node ID (ie. the | |||
| pubkey.Address() for the peer's PubKey) | |||
| If either of these queries return a non-zero ABCI code, Tendermint will refuse | |||
| to connect to the peer. | |||
| ## Crash Recovery | |||
| On startup, Tendermint calls the `Info` method on the Info Connection to get the latest | |||
| committed state of the app. The app MUST return information consistent with the | |||
| last block it succesfully completed Commit for. | |||
| If the app succesfully committed block H but not H+1, then `last_block_height = H` and `last_block_app_hash = <hash returned by Commit for block H>`. If the app | |||
| failed during the Commit of block H, then `last_block_height = H-1` and | |||
| `last_block_app_hash = <hash returned by Commit for block H-1, which is the hash in the header of block H>`. | |||
| We now distinguish three heights, and describe how Tendermint syncs itself with | |||
| the app. | |||
| ``` | |||
| storeBlockHeight = height of the last block Tendermint saw a commit for | |||
| stateBlockHeight = height of the last block for which Tendermint completed all | |||
| block processing and saved all ABCI results to disk | |||
| appBlockHeight = height of the last block for which ABCI app succesfully | |||
| completely Commit | |||
| ``` | |||
| Note we always have `storeBlockHeight >= stateBlockHeight` and `storeBlockHeight >= appBlockHeight` | |||
| Note also we never call Commit on an ABCI app twice for the same height. | |||
| The procedure is as follows. | |||
| First, some simeple start conditions: | |||
| If `appBlockHeight == 0`, then call InitChain. | |||
| If `storeBlockHeight == 0`, we're done. | |||
| Now, some sanity checks: | |||
| If `storeBlockHeight < appBlockHeight`, error | |||
| If `storeBlockHeight < stateBlockHeight`, panic | |||
| If `storeBlockHeight > stateBlockHeight+1`, panic | |||
| Now, the meat: | |||
| If `storeBlockHeight == stateBlockHeight && appBlockHeight < storeBlockHeight`, | |||
| replay all blocks in full from `appBlockHeight` to `storeBlockHeight`. | |||
| This happens if we completed processing the block, but the app forgot its height. | |||
| If `storeBlockHeight == stateBlockHeight && appBlockHeight == storeBlockHeight`, we're done | |||
| This happens if we crashed at an opportune spot. | |||
| If `storeBlockHeight == stateBlockHeight+1` | |||
| This happens if we started processing the block but didn't finish. | |||
| If `appBlockHeight < stateBlockHeight` | |||
| replay all blocks in full from `appBlockHeight` to `storeBlockHeight-1`, | |||
| and replay the block at `storeBlockHeight` using the WAL. | |||
| This happens if the app forgot the last block it committed. | |||
| If `appBlockHeight == stateBlockHeight`, | |||
| replay the last block (storeBlockHeight) in full. | |||
| This happens if we crashed before the app finished Commit | |||
| If appBlockHeight == storeBlockHeight { | |||
| update the state using the saved ABCI responses but dont run the block against the real app. | |||
| This happens if we crashed after the app finished Commit but before Tendermint saved the state. | |||
+ 104
- 0
docs/spec/abci/client-server.md
View File
| @ -0,0 +1,104 @@ | |||
| # ABCI Client and Server | |||
| This section is for those looking to implement their own ABCI Server, perhaps in | |||
| a new programming language. | |||
| You are expected to have read [ABCI Methods and Types](abci.md) and [ABCI | |||
| Applications](apps.md). | |||
| See additional details in the [ABCI | |||
| readme](https://github.com/tendermint/tendermint/blob/develop/abci/README.md)(TODO: deduplicate | |||
| those details). | |||
| ## Message Protocol | |||
| The message protocol consists of pairs of requests and responses defined in the | |||
| [protobuf file](https://github.com/tendermint/tendermint/blob/develop/abci/types/types.proto). | |||
| Some messages have no fields, while others may include byte-arrays, strings, integers, | |||
| or custom protobuf types. | |||
| For more details on protobuf, see the [documentation](https://developers.google.com/protocol-buffers/docs/overview). | |||
| For each request, a server should respond with the corresponding | |||
| response, where the order of requests is preserved in the order of | |||
| responses. | |||
| ## Server | |||
| To use ABCI in your programming language of choice, there must be a ABCI | |||
| server in that language. Tendermint supports two kinds of implementation | |||
| of the server: | |||
| - Asynchronous, raw socket server (Tendermint Socket Protocol, also | |||
| known as TSP or Teaspoon) | |||
| - GRPC | |||
| Both can be tested using the `abci-cli` by setting the `--abci` flag | |||
| appropriately (ie. to `socket` or `grpc`). | |||
| See examples, in various stages of maintenance, in | |||
| [Go](https://github.com/tendermint/tendermint/tree/develop/abci/server), | |||
| [JavaScript](https://github.com/tendermint/js-abci), | |||
| [Python](https://github.com/tendermint/tendermint/tree/develop/abci/example/python3/abci), | |||
| [C++](https://github.com/mdyring/cpp-tmsp), and | |||
| [Java](https://github.com/jTendermint/jabci). | |||
| ### GRPC | |||
| If GRPC is available in your language, this is the easiest approach, | |||
| though it will have significant performance overhead. | |||
| To get started with GRPC, copy in the [protobuf | |||
| file](https://github.com/tendermint/tendermint/blob/develop/abci/types/types.proto) | |||
| and compile it using the GRPC plugin for your language. For instance, | |||
| for golang, the command is `protoc --go_out=plugins=grpc:. types.proto`. | |||
| See the [grpc documentation for more details](http://www.grpc.io/docs/). | |||
| `protoc` will autogenerate all the necessary code for ABCI client and | |||
| server in your language, including whatever interface your application | |||
| must satisfy to be used by the ABCI server for handling requests. | |||
| ### TSP | |||
| If GRPC is not available in your language, or you require higher | |||
| performance, or otherwise enjoy programming, you may implement your own | |||
| ABCI server using the Tendermint Socket Protocol, known affectionately | |||
| as Teaspoon. The first step is still to auto-generate the relevant data | |||
| types and codec in your language using `protoc`. Messages coming over | |||
| the socket are proto3 encoded, but additionally length-prefixed to | |||
| facilitate use as a streaming protocol. proto3 doesn't have an | |||
| official length-prefix standard, so we use our own. The first byte in | |||
| the prefix represents the length of the Big Endian encoded length. The | |||
| remaining bytes in the prefix are the Big Endian encoded length. | |||
| For example, if the proto3 encoded ABCI message is 0xDEADBEEF (4 | |||
| bytes), the length-prefixed message is 0x0104DEADBEEF. If the proto3 | |||
| encoded ABCI message is 65535 bytes long, the length-prefixed message | |||
| would be like 0x02FFFF.... | |||
| Note this prefixing does not apply for grpc. | |||
| An ABCI server must also be able to support multiple connections, as | |||
| Tendermint uses three connections. | |||
| ### Async vs Sync | |||
| The main ABCI server (ie. non-GRPC) provides ordered asynchronous messages. | |||
| This is useful for DeliverTx and CheckTx, since it allows Tendermint to forward | |||
| transactions to the app before it's finished processing previous ones. | |||
| Thus, DeliverTx and CheckTx messages are sent asycnhronously, while all other | |||
| messages are sent synchronously. | |||
| ## Client | |||
| There are currently two use-cases for an ABCI client. One is a testing | |||
| tool, as in the `abci-cli`, which allows ABCI requests to be sent via | |||
| command line. The other is a consensus engine, such as Tendermint Core, | |||
| which makes requests to the application every time a new transaction is | |||
| received or a block is committed. | |||
| It is unlikely that you will need to implement a client. For details of | |||
| our client, see | |||
| [here](https://github.com/tendermint/tendermint/tree/develop/abci/client). | |||
+ 109
- 79
docs/spec/blockchain/blockchain.md
View File
+ 7
- 10
docs/spec/blockchain/encoding.md
View File
+ 66
- 16
docs/spec/blockchain/state.md
View File
+ 1
- 183
docs/spec/software/abci.md
View File
| @ -1,185 +1,3 @@ | |||
| # Application Blockchain Interface (ABCI) | |||
| ABCI is the interface between Tendermint (a state-machine replication engine) | |||
| and an application (the actual state machine). | |||
| The ABCI message types are defined in a [protobuf | |||
| file](https://github.com/tendermint/tendermint/blob/develop/abci/types/types.proto). | |||
| For full details on the ABCI message types and protocol, see the [ABCI | |||
| specification](https://github.com/tendermint/tendermint/blob/develop/docs/app-dev/abci-spec.md). | |||
| Be sure to read the specification if you're trying to build an ABCI app! | |||
| For additional details on server implementation, see the [ABCI | |||
| readme](https://github.com/tendermint/tendermint/blob/develop/abci/README.md). | |||
| Here we provide some more details around the use of ABCI by Tendermint and | |||
| clarify common "gotchas". | |||
| ## ABCI connections | |||
| Tendermint opens 3 ABCI connections to the app: one for Consensus, one for | |||
| Mempool, one for Queries. | |||
| ## Async vs Sync | |||
| The main ABCI server (ie. non-GRPC) provides ordered asynchronous messages. | |||
| This is useful for DeliverTx and CheckTx, since it allows Tendermint to forward | |||
| transactions to the app before it's finished processing previous ones. | |||
| Thus, DeliverTx and CheckTx messages are sent asycnhronously, while all other | |||
| messages are sent synchronously. | |||
| ## CheckTx and Commit | |||
| It is typical to hold three distinct states in an ABCI app: CheckTxState, DeliverTxState, | |||
| QueryState. The QueryState contains the latest committed state for a block. | |||
| The CheckTxState and DeliverTxState may be updated concurrently with one another. | |||
| Before Commit is called, Tendermint locks and flushes the mempool so that no new changes will happen | |||
| to CheckTxState. When Commit completes, it unlocks the mempool. | |||
| Thus, during Commit, it is safe to reset the QueryState and the CheckTxState to the latest DeliverTxState | |||
| (ie. the new state from executing all the txs in the block). | |||
| Note, however, that it is not possible to send transactions to Tendermint during Commit - if your app | |||
| tries to send a `/broadcast_tx` to Tendermint during Commit, it will deadlock. | |||
| ## EndBlock Validator Updates | |||
| Updates to the Tendermint validator set can be made by returning `Validator` | |||
| objects in the `ResponseBeginBlock`: | |||
| ``` | |||
| message Validator { | |||
| PubKey pub_key | |||
| int64 power | |||
| } | |||
| message PubKey { | |||
| string type | |||
| bytes data | |||
| } | |||
| ``` | |||
| The `pub_key` currently supports two types: | |||
| - `type = "ed25519" and`data = <raw 32-byte public key>` | |||
| - `type = "secp256k1" and `data = <33-byte OpenSSL compressed public key>` | |||
| If the address is provided, it must match the address of the pubkey, as | |||
| specified [here](/docs/spec/blockchain/encoding.md#Addresses) | |||
| (Note: In the v0.19 series, the `pub_key` is the [Amino encoded public | |||
| key](/docs/spec/blockchain/encoding.md#public-key-cryptography). | |||
| For Ed25519 pubkeys, the Amino prefix is always "1624DE6220". For example, the 32-byte Ed25519 pubkey | |||
| `76852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85` would be | |||
| Amino encoded as | |||
| `1624DE622076852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85`) | |||
| (Note: In old versions of Tendermint (pre-v0.19.0), the pubkey is just prefixed with a | |||
| single type byte, so for ED25519 we'd have `pub_key = 0x1 | pub`) | |||
| The `power` is the new voting power for the validator, with the | |||
| following rules: | |||
| - power must be non-negative | |||
| - if power is 0, the validator must already exist, and will be removed from the | |||
| validator set | |||
| - if power is non-0: | |||
| - if the validator does not already exist, it will be added to the validator | |||
| set with the given power | |||
| - if the validator does already exist, its power will be adjusted to the given power | |||
| ## InitChain Validator Updates | |||
| ResponseInitChain has the option to return a list of validators. | |||
| If the list is not empty, Tendermint will adopt it for the validator set. | |||
| This way the application can determine the initial validator set for the | |||
| blockchain. | |||
| ResponseInitChain also includes ConsensusParams, but these are presently | |||
| ignored. | |||
| ## Query | |||
| Query is a generic message type with lots of flexibility to enable diverse sets | |||
| of queries from applications. Tendermint has no requirements from the Query | |||
| message for normal operation - that is, the ABCI app developer need not implement Query functionality if they do not wish too. | |||
| That said, Tendermint makes a number of queries to support some optional | |||
| features. These are: | |||
| ### Peer Filtering | |||
| When Tendermint connects to a peer, it sends two queries to the ABCI application | |||
| using the following paths, with no additional data: | |||
| - `/p2p/filter/addr/<IP:PORT>`, where `<IP:PORT>` denote the IP address and | |||
| the port of the connection | |||
| - `p2p/filter/id/<ID>`, where `<ID>` is the peer node ID (ie. the | |||
| pubkey.Address() for the peer's PubKey) | |||
| If either of these queries return a non-zero ABCI code, Tendermint will refuse | |||
| to connect to the peer. | |||
| ## Info and the Handshake/Replay | |||
| On startup, Tendermint calls Info on the Query connection to get the latest | |||
| committed state of the app. The app MUST return information consistent with the | |||
| last block it succesfully completed Commit for. | |||
| If the app succesfully committed block H but not H+1, then `last_block_height = H` and `last_block_app_hash = <hash returned by Commit for block H>`. If the app | |||
| failed during the Commit of block H, then `last_block_height = H-1` and | |||
| `last_block_app_hash = <hash returned by Commit for block H-1, which is the hash in the header of block H>`. | |||
| We now distinguish three heights, and describe how Tendermint syncs itself with | |||
| the app. | |||
| ``` | |||
| storeBlockHeight = height of the last block Tendermint saw a commit for | |||
| stateBlockHeight = height of the last block for which Tendermint completed all | |||
| block processing and saved all ABCI results to disk | |||
| appBlockHeight = height of the last block for which ABCI app succesfully | |||
| completely Commit | |||
| ``` | |||
| Note we always have `storeBlockHeight >= stateBlockHeight` and `storeBlockHeight >= appBlockHeight` | |||
| Note also we never call Commit on an ABCI app twice for the same height. | |||
| The procedure is as follows. | |||
| First, some simeple start conditions: | |||
| If `appBlockHeight == 0`, then call InitChain. | |||
| If `storeBlockHeight == 0`, we're done. | |||
| Now, some sanity checks: | |||
| If `storeBlockHeight < appBlockHeight`, error | |||
| If `storeBlockHeight < stateBlockHeight`, panic | |||
| If `storeBlockHeight > stateBlockHeight+1`, panic | |||
| Now, the meat: | |||
| If `storeBlockHeight == stateBlockHeight && appBlockHeight < storeBlockHeight`, | |||
| replay all blocks in full from `appBlockHeight` to `storeBlockHeight`. | |||
| This happens if we completed processing the block, but the app forgot its height. | |||
| If `storeBlockHeight == stateBlockHeight && appBlockHeight == storeBlockHeight`, we're done | |||
| This happens if we crashed at an opportune spot. | |||
| If `storeBlockHeight == stateBlockHeight+1` | |||
| This happens if we started processing the block but didn't finish. | |||
| If `appBlockHeight < stateBlockHeight` | |||
| replay all blocks in full from `appBlockHeight` to `storeBlockHeight-1`, | |||
| and replay the block at `storeBlockHeight` using the WAL. | |||
| This happens if the app forgot the last block it committed. | |||
| If `appBlockHeight == stateBlockHeight`, | |||
| replay the last block (storeBlockHeight) in full. | |||
| This happens if we crashed before the app finished Commit | |||
| If appBlockHeight == storeBlockHeight { | |||
| update the state using the saved ABCI responses but dont run the block against the real app. | |||
| This happens if we crashed after the app finished Commit but before Tendermint saved the state. | |||
| This page has [moved](../spec/abci/apps.md). | |||