Browse Source
Merge pull request #2343 from tendermint/release/v0.24.0
Major spec update to prepare v0.24.0 for releasepull/2349/head
Ethan Buchman
6 years ago
committed by
GitHub
GitHub
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
16 changed files with 1013 additions and 358 deletions
Unified View
Diff Options
-
+97 -0CHANGELOG.md
-
+1 -59CHANGELOG_PENDING.md
-
+5 -5abci/version/version.go
-
+11 -2docs/app-dev/abci-spec.md
-
+5 -0docs/app-dev/app-development.md
-
+19 -0docs/spec/abci/README.md
-
+372 -0docs/spec/abci/abci.md
-
+207 -0docs/spec/abci/apps.md
-
+104 -0docs/spec/abci/client-server.md
-
+109 -79docs/spec/blockchain/blockchain.md
-
+7 -10docs/spec/blockchain/encoding.md
-
+66 -16docs/spec/blockchain/state.md
-
+1 -183docs/spec/software/abci.md
-
+1 -1docs/tendermint-core/using-tendermint.md
-
+2 -0types/time/time.go
-
+6 -3version/version.go
+ 97
- 0
CHANGELOG.md
View File
+ 1
- 59
CHANGELOG_PENDING.md
View File
| @ -1,80 +1,22 @@ | |||||
| # Pending | # Pending | ||||
| Special thanks to external contributors with PRs included in this release: ackratos, james-ray, bradyjoestar, | |||||
| peerlink, Ahmah2009, bluele, b00f | |||||
| Special thanks to external contributors with PRs included in this release: | |||||
| BREAKING CHANGES: | BREAKING CHANGES: | ||||
| * CLI/RPC/Config | * CLI/RPC/Config | ||||
| - [config] \#2169 Replace MaxNumPeers with MaxNumInboundPeers and MaxNumOutboundPeers | |||||
| - [config] \#2300 Reduce default mempool size from 100k to 5k, until ABCI rechecking is implemented. | |||||
| - [rpc] \#1815 `/commit` returns a `signed_header` field instead of everything being top-level | |||||
| * Apps | * Apps | ||||
| - [abci] Added address of the original proposer of the block to Header | |||||
| - [abci] Change ABCI Header to match Tendermint exactly | |||||
| - [abci] \#2159 Update use of `Validator` (see | |||||
| [ADR-018](https://github.com/tendermint/tendermint/blob/develop/docs/architecture/adr-018-ABCI-Validators.md)): | |||||
| - Remove PubKey from `Validator` (so it's just Address and Power) | |||||
| - Introduce `ValidatorUpdate` (with just PubKey and Power) | |||||
| - InitChain and EndBlock use ValidatorUpdate | |||||
| - Update field names and types in BeginBlock | |||||
| - [state] \#1815 Validator set changes are now delayed by one block | |||||
| - updates returned in ResponseEndBlock for block H will be included in RequestBeginBlock for block H+2 | |||||
| * Go API | * Go API | ||||
| - [lite] \#1815 Complete refactor of the package | |||||
| - [node] \#2212 NewNode now accepts a `*p2p.NodeKey` (@bradyjoestar) | |||||
| - [libs/common] \#2199 Remove Fmt, in favor of fmt.Sprintf | |||||
| - [libs/common] SplitAndTrim was deleted | |||||
| - [libs/common] \#2274 Remove unused Math functions like MaxInt, MaxInt64, | |||||
| MinInt, MinInt64 (@Ahmah2009) | |||||
| - [libs/clist] Panics if list extends beyond MaxLength | |||||
| - [crypto] \#2205 Rename AminoRoute variables to no longer be prefixed by signature type. | |||||
| * Blockchain Protocol | * Blockchain Protocol | ||||
| - [state] \#1815 Validator set changes are now delayed by one block (!) | |||||
| - Add NextValidatorSet to State, changes on-disk representation of state | |||||
| - [state] \#2184 Enforce ConsensusParams.BlockSize.MaxBytes (See | |||||
| [ADR-020](https://github.com/tendermint/tendermint/blob/develop/docs/architecture/adr-020-block-size.md)). | |||||
| - Remove ConsensusParams.BlockSize.MaxTxs | |||||
| - Introduce maximum sizes for all components of a block, including ChainID | |||||
| - [types] Updates to the block Header: | |||||
| - \#1815 NextValidatorsHash - hash of the validator set for the next block, | |||||
| so the current validators actually sign over the hash for the new | |||||
| validators | |||||
| - \#2106 ProposerAddress - address of the block's original proposer | |||||
| - [consensus] \#2203 Implement BFT time | |||||
| - Timestamp in block must be monotonic and equal the median of timestamps in block's LastCommit | |||||
| - [crypto] \#2239 Secp256k1 signature changes (See | |||||
| [ADR-014](https://github.com/tendermint/tendermint/blob/develop/docs/architecture/adr-014-secp-malleability.md)): | |||||
| - format changed from DER to `r || s`, both little endian encoded as 32 bytes. | |||||
| - malleability removed by requiring `s` to be in canonical form. | |||||
| * P2P Protocol | * P2P Protocol | ||||
| - [p2p] \#2263 Update secret connection to use a little endian encoded nonce | |||||
| - [blockchain] \#2213 Fix Amino routes for blockchain reactor messages | |||||
| (@peerlink) | |||||
| FEATURES: | FEATURES: | ||||
| - [types] \#2015 Allow genesis file to have 0 validators (@b00f) | |||||
| - Initial validator set can be determined by the app in ResponseInitChain | |||||
| - [rpc] \#2161 New event `ValidatorSetUpdates` for when the validator set changes | |||||
| - [crypto/multisig] \#2164 Introduce multisig pubkey and signature format | |||||
| - [libs/db] \#2293 Allow passing options through when creating instances of leveldb dbs | |||||
| IMPROVEMENTS: | IMPROVEMENTS: | ||||
| - [docs] Lint documentation with `write-good` and `stop-words`. | |||||
| - [scripts] \#2196 Added json2wal tool, which is supposed to help our users restore (@bradyjoestar) | |||||
| corrupted WAL files and compose test WAL files (@bradyjoestar) | |||||
| - [mempool] \#2234 Now stores txs by hash inside of the cache, to mitigate memory leakage | |||||
| - [mempool] \#2166 Set explicit capacity for map when updating txs (@bluele) | |||||
| BUG FIXES: | BUG FIXES: | ||||
| - [config] \#2284 Replace `db_path` with `db_dir` from automatically generated configuration files. | |||||
| - [mempool] \#2188 Fix OOM issue from cache map and list getting out of sync | |||||
| - [state] \#2051 KV store index supports searching by `tx.height` (@ackratos) | |||||
| - [rpc] \#2327 `/dial_peers` does not try to dial existing peers | |||||
| - [node] \#2323 Filter empty strings from config lists (@james-ray) | |||||
| - [abci/client] \#2236 Fix closing GRPC connection (@bradyjoestar) | |||||
+ 5
- 5
abci/version/version.go
View File
| @ -1,9 +1,9 @@ | |||||
| package version | package version | ||||
| // NOTE: we should probably be versioning the ABCI and the abci-cli separately | |||||
| import ( | |||||
| "github.com/tendermint/tendermint/version" | |||||
| ) | |||||
| const Maj = "0" | |||||
| const Min = "12" | |||||
| const Fix = "0" | |||||
| // TODO: eliminate this after some version refactor | |||||
| const Version = "0.12.0" | |||||
| const Version = version.ABCIVersion | |||||
+ 11
- 2
docs/app-dev/abci-spec.md
View File
+ 5
- 0
docs/app-dev/app-development.md
View File
+ 19
- 0
docs/spec/abci/README.md
View File
| @ -0,0 +1,19 @@ | |||||
| # ABCI | |||||
| ABCI is the interface between Tendermint (a state-machine replication engine) | |||||
| and an application (the actual state machine). It consists of a set of | |||||
| *methods*, where each method has a corresponding `Request` and `Response` | |||||
| message type. Tendermint calls the ABCI methods on the ABCI application by sending the `Request*` | |||||
| messages and receiving the `Response*` messages in return. | |||||
| All message types are defined in a [protobuf file](https://github.com/tendermint/tendermint/blob/develop/abci/types/types.proto). | |||||
| This allows Tendermint to run applications written in any programming language. | |||||
| This specification is split as follows: | |||||
| - [Methods and Types](abci.md) - complete details on all ABCI methods and | |||||
| message types | |||||
| - [Applications](apps.md) - how to manage ABCI application state and other | |||||
| details about building ABCI applications | |||||
| - [Client and Server](client-server.md) - for those looking to implement their | |||||
| own ABCI application servers | |||||
+ 372
- 0
docs/spec/abci/abci.md
View File
| @ -0,0 +1,372 @@ | |||||
| # ABCI Methods and Types | |||||
| ## Overview | |||||
| The ABCI message types are defined in a [protobuf | |||||
| file](https://github.com/tendermint/tendermint/blob/develop/abci/types/types.proto). | |||||
| ABCI methods are split across 3 separate ABCI *connections*: | |||||
| - `Consensus Connection: InitChain, BeginBlock, DeliverTx, EndBlock, Commit` | |||||
| - `Mempool Connection: CheckTx` | |||||
| - `Info Connection: Info, SetOption, Query` | |||||
| The `Consensus Connection` is driven by a consensus protocol and is responsible | |||||
| for block exection. | |||||
| The `Mempool Connection` is for validating new transactions, before they're | |||||
| shared or included in a block. | |||||
| The `Info Connection` is for initialization and for queries from the user. | |||||
| Additionally, there is a `Flush` method that is called on every connection, | |||||
| and an `Echo` method that is just for debugging. | |||||
| ## Errors | |||||
| Some methods (`Echo, Info, InitChain, BeginBlock, EndBlock, Commit`), | |||||
| don't return errors because an error would indicate a critical failure | |||||
| in the application and there's nothing Tendermint can do. The problem | |||||
| should be addressed and both Tendermint and the application restarted. | |||||
| All other methods (`SetOption, Query, CheckTx, DeliverTx`) return an | |||||
| application-specific response `Code uint32`, where only `0` is reserved | |||||
| for `OK`. | |||||
| ## Tags | |||||
| Some methods (`CheckTx, BeginBlock, DeliverTx, EndBlock`) | |||||
| include a `Tags` field in their `Response*`. Each tag is key-value pair denoting | |||||
| something about what happened during the methods execution. | |||||
| Tags can be used to index transactions and blocks according to what happened | |||||
| during their execution. | |||||
| Keys and values in tags must be UTF-8 encoded strings (e.g. | |||||
| "account.owner": "Bob", "balance": "100.0", | |||||
| "time": "2018-01-02T12:30:00Z") | |||||
| ## Determinism | |||||
| Some methods (`SetOption, Query, CheckTx, DeliverTx`) return | |||||
| non-deterministic data in the form of `Info` and `Log`. The `Log` is | |||||
| intended for the literal output from the application's logger, while the | |||||
| `Info` is any additional info that should be returned. | |||||
| All other fields in the `Response*` of all methods must be strictly deterministic. | |||||
| For this reason, it is recommended that applications not be exposed to any | |||||
| external user or process except via the ABCI connections to a consensus engine | |||||
| like Tendermint Core. | |||||
| ## Block Execution | |||||
| The first time a new blockchain is started, Tendermint calls | |||||
| `InitChain`. From then on, the follow sequence of methods is executed for each | |||||
| block: | |||||
| `BeginBlock, [DeliverTx], EndBlock, Commit` | |||||
| where one `DeliverTx` is called for each transaction in the block. | |||||
| The result is an updated application state. | |||||
| Cryptographic commitments to the results of DeliverTx, EndBlock, and | |||||
| Commit are included in the header of the next block. | |||||
| ## Messages | |||||
| ### Echo | |||||
| - **Request**: | |||||
| - `Message (string)`: A string to echo back | |||||
| - **Response**: | |||||
| - `Message (string)`: The input string | |||||
| - **Usage**: | |||||
| - Echo a string to test an abci client/server implementation | |||||
| ### Flush | |||||
| - **Usage**: | |||||
| - Signals that messages queued on the client should be flushed to | |||||
| the server. It is called periodically by the client | |||||
| implementation to ensure asynchronous requests are actually | |||||
| sent, and is called immediately to make a synchronous request, | |||||
| which returns when the Flush response comes back. | |||||
| ### Info | |||||
| - **Request**: | |||||
| - `Version (string)`: The Tendermint version | |||||
| - **Response**: | |||||
| - `Data (string)`: Some arbitrary information | |||||
| - `Version (Version)`: Version information | |||||
| - `LastBlockHeight (int64)`: Latest block for which the app has | |||||
| called Commit | |||||
| - `LastBlockAppHash ([]byte)`: Latest result of Commit | |||||
| - **Usage**: | |||||
| - Return information about the application state. | |||||
| - Used to sync Tendermint with the application during a handshake | |||||
| that happens on startup. | |||||
| - Tendermint expects `LastBlockAppHash` and `LastBlockHeight` to | |||||
| be updated during `Commit`, ensuring that `Commit` is never | |||||
| called twice for the same block height. | |||||
| ### SetOption | |||||
| - **Request**: | |||||
| - `Key (string)`: Key to set | |||||
| - `Value (string)`: Value to set for key | |||||
| - **Response**: | |||||
| - `Code (uint32)`: Response code | |||||
| - `Log (string)`: The output of the application's logger. May | |||||
| be non-deterministic. | |||||
| - `Info (string)`: Additional information. May | |||||
| be non-deterministic. | |||||
| - **Usage**: | |||||
| - Set non-consensus critical application specific options. | |||||
| - e.g. Key="min-fee", Value="100fermion" could set the minimum fee | |||||
| required for CheckTx (but not DeliverTx - that would be | |||||
| consensus critical). | |||||
| ### InitChain | |||||
| - **Request**: | |||||
| - `Time (google.protobuf.Timestamp)`: Genesis time. | |||||
| - `ChainID (string)`: ID of the blockchain. | |||||
| - `ConsensusParams (ConsensusParams)`: Initial consensus-critical parameters. | |||||
| - `Validators ([]ValidatorUpdate)`: Initial genesis validators. | |||||
| - `AppStateBytes ([]byte)`: Serialized initial application state. Amino-encoded JSON bytes. | |||||
| - **Response**: | |||||
| - `ConsensusParams (ConsensusParams)`: Initial | |||||
| consensus-critical parameters. | |||||
| - `Validators ([]ValidatorUpdate)`: Initial validator set (if non empty). | |||||
| - **Usage**: | |||||
| - Called once upon genesis. | |||||
| - If ResponseInitChain.Validators is empty, the initial validator set will be the RequestInitChain.Validators | |||||
| - If ResponseInitChain.Validators is not empty, the initial validator set will be the | |||||
| ResponseInitChain.Validators (regardless of what is in RequestInitChain.Validators). | |||||
| - This allows the app to decide if it wants to accept the initial validator | |||||
| set proposed by tendermint (ie. in the genesis file), or if it wants to use | |||||
| a different one (perhaps computed based on some application specific | |||||
| information in the genesis file). | |||||
| ### Query | |||||
| - **Request**: | |||||
| - `Data ([]byte)`: Raw query bytes. Can be used with or in lieu | |||||
| of Path. | |||||
| - `Path (string)`: Path of request, like an HTTP GET path. Can be | |||||
| used with or in liue of Data. | |||||
| - Apps MUST interpret '/store' as a query by key on the | |||||
| underlying store. The key SHOULD be specified in the Data field. | |||||
| - Apps SHOULD allow queries over specific types like | |||||
| '/accounts/...' or '/votes/...' | |||||
| - `Height (int64)`: The block height for which you want the query | |||||
| (default=0 returns data for the latest committed block). Note | |||||
| that this is the height of the block containing the | |||||
| application's Merkle root hash, which represents the state as it | |||||
| was after committing the block at Height-1 | |||||
| - `Prove (bool)`: Return Merkle proof with response if possible | |||||
| - **Response**: | |||||
| - `Code (uint32)`: Response code. | |||||
| - `Log (string)`: The output of the application's logger. May | |||||
| be non-deterministic. | |||||
| - `Info (string)`: Additional information. May | |||||
| be non-deterministic. | |||||
| - `Index (int64)`: The index of the key in the tree. | |||||
| - `Key ([]byte)`: The key of the matching data. | |||||
| - `Value ([]byte)`: The value of the matching data. | |||||
| - `Proof ([]byte)`: Proof for the data, if requested. | |||||
| - `Height (int64)`: The block height from which data was derived. | |||||
| Note that this is the height of the block containing the | |||||
| application's Merkle root hash, which represents the state as it | |||||
| was after committing the block at Height-1 | |||||
| - **Usage**: | |||||
| - Query for data from the application at current or past height. | |||||
| - Optionally return Merkle proof. | |||||
| ### BeginBlock | |||||
| - **Request**: | |||||
| - `Hash ([]byte)`: The block's hash. This can be derived from the | |||||
| block header. | |||||
| - `Header (struct{})`: The block header. | |||||
| - `LastCommitInfo (LastCommitInfo)`: Info about the last commit, including the | |||||
| round, and the list of validators and which ones signed the last block. | |||||
| - `ByzantineValidators ([]Evidence)`: List of evidence of | |||||
| validators that acted maliciously. | |||||
| - **Response**: | |||||
| - `Tags ([]cmn.KVPair)`: Key-Value tags for filtering and indexing | |||||
| - **Usage**: | |||||
| - Signals the beginning of a new block. Called prior to | |||||
| any DeliverTxs. | |||||
| - The header contains the height, timestamp, and more - it exactly matches the | |||||
| Tendermint block header. We may seek to generalize this in the future. | |||||
| - The `LastCommitInfo` and `ByzantineValidators` can be used to determine | |||||
| rewards and punishments for the validators. NOTE validators here do not | |||||
| include pubkeys. | |||||
| ### CheckTx | |||||
| - **Request**: | |||||
| - `Tx ([]byte)`: The request transaction bytes | |||||
| - **Response**: | |||||
| - `Code (uint32)`: Response code | |||||
| - `Data ([]byte)`: Result bytes, if any. | |||||
| - `Log (string)`: The output of the application's logger. May | |||||
| be non-deterministic. | |||||
| - `Info (string)`: Additional information. May | |||||
| be non-deterministic. | |||||
| - `GasWanted (int64)`: Amount of gas request for transaction. | |||||
| - `GasUsed (int64)`: Amount of gas consumed by transaction. | |||||
| - `Tags ([]cmn.KVPair)`: Key-Value tags for filtering and indexing | |||||
| transactions (eg. by account). | |||||
| - **Usage**: | |||||
| - Technically optional - not involved in processing blocks. | |||||
| - Guardian of the mempool: every node runs CheckTx before letting a | |||||
| transaction into its local mempool. | |||||
| - The transaction may come from an external user or another node | |||||
| - CheckTx need not execute the transaction in full, but rather a light-weight | |||||
| yet stateful validation, like checking signatures and account balances, but | |||||
| not running code in a virtual machine. | |||||
| - Transactions where `ResponseCheckTx.Code != 0` will be rejected - they will not be broadcast to | |||||
| other nodes or included in a proposal block. | |||||
| - Tendermint attributes no other value to the response code | |||||
| ### DeliverTx | |||||
| - **Request**: | |||||
| - `Tx ([]byte)`: The request transaction bytes. | |||||
| - **Response**: | |||||
| - `Code (uint32)`: Response code. | |||||
| - `Data ([]byte)`: Result bytes, if any. | |||||
| - `Log (string)`: The output of the application's logger. May | |||||
| be non-deterministic. | |||||
| - `Info (string)`: Additional information. May | |||||
| be non-deterministic. | |||||
| - `GasWanted (int64)`: Amount of gas requested for transaction. | |||||
| - `GasUsed (int64)`: Amount of gas consumed by transaction. | |||||
| - `Tags ([]cmn.KVPair)`: Key-Value tags for filtering and indexing | |||||
| transactions (eg. by account). | |||||
| - **Usage**: | |||||
| - The workhorse of the application - non-optional. | |||||
| - Execute the transaction in full. | |||||
| - `ResponseDeliverTx.Code == 0` only if the transaction is fully valid. | |||||
| ### EndBlock | |||||
| - **Request**: | |||||
| - `Height (int64)`: Height of the block just executed. | |||||
| - **Response**: | |||||
| - `ValidatorUpdates ([]ValidatorUpdate)`: Changes to validator set (set | |||||
| voting power to 0 to remove). | |||||
| - `ConsensusParamUpdates (ConsensusParams)`: Changes to | |||||
| consensus-critical time, size, and other parameters. | |||||
| - `Tags ([]cmn.KVPair)`: Key-Value tags for filtering and indexing | |||||
| - **Usage**: | |||||
| - Signals the end of a block. | |||||
| - Called after all transactions, prior to each Commit. | |||||
| - Validator updates returned by block `H` impact blocks `H+1`, `H+2`, and | |||||
| `H+3`, but only effects changes on the validator set of `H+2`: | |||||
| - `H+1`: NextValidatorsHash | |||||
| - `H+2`: ValidatorsHash (and thus the validator set) | |||||
| - `H+3`: LastCommitInfo (ie. the last validator set) | |||||
| - Consensus params returned for block `H` apply for block `H+1` | |||||
| ### Commit | |||||
| - **Response**: | |||||
| - `Data ([]byte)`: The Merkle root hash | |||||
| - **Usage**: | |||||
| - Persist the application state. | |||||
| - Return a Merkle root hash of the application state. | |||||
| - It's critical that all application instances return the | |||||
| same hash. If not, they will not be able to agree on the next | |||||
| block, because the hash is included in the next block! | |||||
| ## Data Types | |||||
| ### Header | |||||
| - **Fields**: | |||||
| - `ChainID (string)`: ID of the blockchain | |||||
| - `Height (int64)`: Height of the block in the chain | |||||
| - `Time (google.protobuf.Timestamp)`: Time of the block. It is the proposer's | |||||
| local time when block was created. | |||||
| - `NumTxs (int32)`: Number of transactions in the block | |||||
| - `TotalTxs (int64)`: Total number of transactions in the blockchain until | |||||
| now | |||||
| - `LastBlockID (BlockID)`: Hash of the previous (parent) block | |||||
| - `LastCommitHash ([]byte)`: Hash of the previous block's commit | |||||
| - `ValidatorsHash ([]byte)`: Hash of the validator set for this block | |||||
| - `NextValidatorsHash ([]byte)`: Hash of the validator set for the next block | |||||
| - `ConsensusHash ([]byte)`: Hash of the consensus parameters for this block | |||||
| - `AppHash ([]byte)`: Data returned by the last call to `Commit` - typically the | |||||
| Merkle root of the application state after executing the previous block's | |||||
| transactions | |||||
| - `LastResultsHash ([]byte)`: Hash of the ABCI results returned by the last block | |||||
| - `EvidenceHash ([]byte)`: Hash of the evidence included in this block | |||||
| - `ProposerAddress ([]byte)`: Original proposer for the block | |||||
| - **Usage**: | |||||
| - Provided in RequestBeginBlock | |||||
| - Provides important context about the current state of the blockchain - | |||||
| especially height and time. | |||||
| - Provides the proposer of the current block, for use in proposer-based | |||||
| reward mechanisms. | |||||
| ### Validator | |||||
| - **Fields**: | |||||
| - `Address ([]byte)`: Address of the validator (hash of the public key) | |||||
| - `Power (int64)`: Voting power of the validator | |||||
| - **Usage**: | |||||
| - Validator identified by address | |||||
| - Used in RequestBeginBlock as part of VoteInfo | |||||
| - Does not include PubKey to avoid sending potentially large quantum pubkeys | |||||
| over the ABCI | |||||
| ### ValidatorUpdate | |||||
| - **Fields**: | |||||
| - `PubKey (PubKey)`: Public key of the validator | |||||
| - `Power (int64)`: Voting power of the validator | |||||
| - **Usage**: | |||||
| - Validator identified by PubKey | |||||
| - Used to tell Tendermint to update the validator set | |||||
| ### VoteInfo | |||||
| - **Fields**: | |||||
| - `Validator (Validator)`: A validator | |||||
| - `SignedLastBlock (bool)`: Indicates whether or not the validator signed | |||||
| the last block | |||||
| - **Usage**: | |||||
| - Indicates whether a validator signed the last block, allowing for rewards | |||||
| based on validator availability | |||||
| ### PubKey | |||||
| - **Fields**: | |||||
| - `Type (string)`: Type of the public key. A simple string like `"ed25519"`. | |||||
| In the future, may indicate a serialization algorithm to parse the `Data`, | |||||
| for instance `"amino"`. | |||||
| - `Data ([]byte)`: Public key data. For a simple public key, it's just the | |||||
| raw bytes. If the `Type` indicates an encoding algorithm, this is the | |||||
| encoded public key. | |||||
| - **Usage**: | |||||
| - A generic and extensible typed public key | |||||
| ### Evidence | |||||
| - **Fields**: | |||||
| - `Type (string)`: Type of the evidence. A hierarchical path like | |||||
| "duplicate/vote". | |||||
| - `Validator (Validator`: The offending validator | |||||
| - `Height (int64)`: Height when the offense was committed | |||||
| - `Time (google.protobuf.Timestamp)`: Time of the block at height `Height`. | |||||
| It is the proposer's local time when block was created. | |||||
| - `TotalVotingPower (int64)`: Total voting power of the validator set at | |||||
| height `Height` | |||||
| ### LastCommitInfo | |||||
| - **Fields**: | |||||
| - `Round (int32)`: Commit round. | |||||
| - `Votes ([]VoteInfo)`: List of validators addresses in the last validator set | |||||
| with their voting power and whether or not they signed a vote. | |||||
+ 207
- 0
docs/spec/abci/apps.md
View File
| @ -0,0 +1,207 @@ | |||||
| # ABCI Applications | |||||
| Please ensure you've first read the spec for [ABCI Methods and Types](abci.md) | |||||
| Here we cover the following components of ABCI applications: | |||||
| - [State](#State) - the interplay between ABCI connections and application state | |||||
| and the differences between `CheckTx` and `DeliverTx`. | |||||
| - [Validator Set Updates](#Validator-Set-Updates) - how validator sets are | |||||
| changed during `InitChain` and `EndBlock` | |||||
| - [Query](#Query) - standards for using the `Query` method | |||||
| - [Crash Recovery](#Crash-Recovery) - handshake protocol to synchronize | |||||
| Tendermint and the application on startup. | |||||
| ## State | |||||
| Since Tendermint maintains multiple concurrent ABCI connections, it is typical | |||||
| for an application to maintain a distinct state for each, and for the states to | |||||
| be sycnronized during `Commit`. | |||||
| ### Commit | |||||
| Before `Commit` is called, Tendermint locks and flushes the mempool so that no new messages will | |||||
| be received on the mempool connection. This provides an opportunity to safely update all three | |||||
| states to the latest committed state at once. | |||||
| When `Commit` completes, it unlocks the mempool. | |||||
| Note that it is not possible to send transactions to Tendermint during `Commit` - if your app | |||||
| tries to send a `/broadcast_tx` to Tendermint during Commit, it will deadlock. | |||||
| ### Consensus Connection | |||||
| The Consensus Connection should maintain a `DeliverTxState` - | |||||
| the working state for block execution. It should be updated by the calls to | |||||
| `BeginBlock`, `DeliverTx`, and `EndBlock` during block execution and committed to | |||||
| disk as the "latest committed state" during `Commit`. | |||||
| Updates made to the DeliverTxState by each method call must be readable by each subsequent method - | |||||
| ie. the updates are linearizeable. | |||||
| ### Mempool Connection | |||||
| The Mempool Connection should maintain a `CheckTxState` - | |||||
| to process pending transactions in the mempool that have | |||||
| not yet been committed. It should be initialized to the latest committed state | |||||
| at the end of every `Commit`. Note it may be updated concurrently with the | |||||
| DeliverTxState. | |||||
| Before calling `Commit`, Tendermint will lock and flush the mempool, | |||||
| ensuring that all existing CheckTx are responded to and no new ones can | |||||
| begin. | |||||
| After `Commit`, CheckTx is run again on all transactions that remain in the | |||||
| node's local mempool after filtering those included in the block. To prevent the | |||||
| mempool from rechecking all transactions every time a block is committed, set | |||||
| the configuration option `mempool.recheck=false`. | |||||
| Finally, the mempool will unlock and new transactions can be processed through CheckTx again. | |||||
| Note that CheckTx doesn't have to check everything that affects transaction validity; the | |||||
| expensive things can be skipped. In fact, CheckTx doesn't have to check | |||||
| anything; it might say that any transaction is a valid transaction. | |||||
| Unlike DeliverTx, CheckTx is just there as | |||||
| a sort of weak filter to keep invalid transactions out of the blockchain. It's | |||||
| weak, because a Byzantine node doesn't care about CheckTx; it can propose a | |||||
| block full of invalid transactions if it wants. | |||||
| ### Info Connection | |||||
| The Mempool Connection should maintain a `QueryState` for answering queries from the user, | |||||
| and for initialization when Tendermint first starts up. | |||||
| It should always contain the latest committed state associated with the | |||||
| latest commited block. | |||||
| QueryState should be set to the latest `DeliverTxState` at the end of every `Commit`, | |||||
| ie. after the full block has been processed and the state committed to disk. | |||||
| Otherwise it should never be modified. | |||||
| ## Validator Updates | |||||
| ### EndBlock | |||||
| Updates to the Tendermint validator set can be made by returning | |||||
| `ValidatorUpdate` objects in the `ResponseEndBlock`: | |||||
| ``` | |||||
| message ValidatorUpdate { | |||||
| PubKey pub_key | |||||
| int64 power | |||||
| } | |||||
| message PubKey { | |||||
| string type | |||||
| bytes data | |||||
| } | |||||
| ``` | |||||
| The `pub_key` currently supports only one type: | |||||
| - `type = "ed25519" and`data = <raw 32-byte public key>` | |||||
| The `power` is the new voting power for the validator, with the | |||||
| following rules: | |||||
| - power must be non-negative | |||||
| - if power is 0, the validator must already exist, and will be removed from the | |||||
| validator set | |||||
| - if power is non-0: | |||||
| - if the validator does not already exist, it will be added to the validator | |||||
| set with the given power | |||||
| - if the validator does already exist, its power will be adjusted to the given power | |||||
| ### InitChain | |||||
| ResponseInitChain has the option to return a list of validators. | |||||
| If the list is not empty, Tendermint will adopt it for the validator set. | |||||
| This way the application can determine the initial validator set for the | |||||
| blockchain. | |||||
| ResponseInitChain also includes ConsensusParams, but these are presently | |||||
| ignored. | |||||
| ## Query | |||||
| Query is a generic message type with lots of flexibility to enable diverse sets | |||||
| of queries from applications. Tendermint has no requirements from the Query | |||||
| message for normal operation - that is, the ABCI app developer need not implement Query functionality if they do not wish too. | |||||
| That said, Tendermint makes a number of queries to support some optional | |||||
| features. These are: | |||||
| ### Peer Filtering | |||||
| When Tendermint connects to a peer, it sends two queries to the ABCI application | |||||
| using the following paths, with no additional data: | |||||
| - `/p2p/filter/addr/<IP:PORT>`, where `<IP:PORT>` denote the IP address and | |||||
| the port of the connection | |||||
| - `p2p/filter/id/<ID>`, where `<ID>` is the peer node ID (ie. the | |||||
| pubkey.Address() for the peer's PubKey) | |||||
| If either of these queries return a non-zero ABCI code, Tendermint will refuse | |||||
| to connect to the peer. | |||||
| ## Crash Recovery | |||||
| On startup, Tendermint calls the `Info` method on the Info Connection to get the latest | |||||
| committed state of the app. The app MUST return information consistent with the | |||||
| last block it succesfully completed Commit for. | |||||
| If the app succesfully committed block H but not H+1, then `last_block_height = H` and `last_block_app_hash = <hash returned by Commit for block H>`. If the app | |||||
| failed during the Commit of block H, then `last_block_height = H-1` and | |||||
| `last_block_app_hash = <hash returned by Commit for block H-1, which is the hash in the header of block H>`. | |||||
| We now distinguish three heights, and describe how Tendermint syncs itself with | |||||
| the app. | |||||
| ``` | |||||
| storeBlockHeight = height of the last block Tendermint saw a commit for | |||||
| stateBlockHeight = height of the last block for which Tendermint completed all | |||||
| block processing and saved all ABCI results to disk | |||||
| appBlockHeight = height of the last block for which ABCI app succesfully | |||||
| completely Commit | |||||
| ``` | |||||
| Note we always have `storeBlockHeight >= stateBlockHeight` and `storeBlockHeight >= appBlockHeight` | |||||
| Note also we never call Commit on an ABCI app twice for the same height. | |||||
| The procedure is as follows. | |||||
| First, some simeple start conditions: | |||||
| If `appBlockHeight == 0`, then call InitChain. | |||||
| If `storeBlockHeight == 0`, we're done. | |||||
| Now, some sanity checks: | |||||
| If `storeBlockHeight < appBlockHeight`, error | |||||
| If `storeBlockHeight < stateBlockHeight`, panic | |||||
| If `storeBlockHeight > stateBlockHeight+1`, panic | |||||
| Now, the meat: | |||||
| If `storeBlockHeight == stateBlockHeight && appBlockHeight < storeBlockHeight`, | |||||
| replay all blocks in full from `appBlockHeight` to `storeBlockHeight`. | |||||
| This happens if we completed processing the block, but the app forgot its height. | |||||
| If `storeBlockHeight == stateBlockHeight && appBlockHeight == storeBlockHeight`, we're done | |||||
| This happens if we crashed at an opportune spot. | |||||
| If `storeBlockHeight == stateBlockHeight+1` | |||||
| This happens if we started processing the block but didn't finish. | |||||
| If `appBlockHeight < stateBlockHeight` | |||||
| replay all blocks in full from `appBlockHeight` to `storeBlockHeight-1`, | |||||
| and replay the block at `storeBlockHeight` using the WAL. | |||||
| This happens if the app forgot the last block it committed. | |||||
| If `appBlockHeight == stateBlockHeight`, | |||||
| replay the last block (storeBlockHeight) in full. | |||||
| This happens if we crashed before the app finished Commit | |||||
| If appBlockHeight == storeBlockHeight { | |||||
| update the state using the saved ABCI responses but dont run the block against the real app. | |||||
| This happens if we crashed after the app finished Commit but before Tendermint saved the state. | |||||
+ 104
- 0
docs/spec/abci/client-server.md
View File
| @ -0,0 +1,104 @@ | |||||
| # ABCI Client and Server | |||||
| This section is for those looking to implement their own ABCI Server, perhaps in | |||||
| a new programming language. | |||||
| You are expected to have read [ABCI Methods and Types](abci.md) and [ABCI | |||||
| Applications](apps.md). | |||||
| See additional details in the [ABCI | |||||
| readme](https://github.com/tendermint/tendermint/blob/develop/abci/README.md)(TODO: deduplicate | |||||
| those details). | |||||
| ## Message Protocol | |||||
| The message protocol consists of pairs of requests and responses defined in the | |||||
| [protobuf file](https://github.com/tendermint/tendermint/blob/develop/abci/types/types.proto). | |||||
| Some messages have no fields, while others may include byte-arrays, strings, integers, | |||||
| or custom protobuf types. | |||||
| For more details on protobuf, see the [documentation](https://developers.google.com/protocol-buffers/docs/overview). | |||||
| For each request, a server should respond with the corresponding | |||||
| response, where the order of requests is preserved in the order of | |||||
| responses. | |||||
| ## Server | |||||
| To use ABCI in your programming language of choice, there must be a ABCI | |||||
| server in that language. Tendermint supports two kinds of implementation | |||||
| of the server: | |||||
| - Asynchronous, raw socket server (Tendermint Socket Protocol, also | |||||
| known as TSP or Teaspoon) | |||||
| - GRPC | |||||
| Both can be tested using the `abci-cli` by setting the `--abci` flag | |||||
| appropriately (ie. to `socket` or `grpc`). | |||||
| See examples, in various stages of maintenance, in | |||||
| [Go](https://github.com/tendermint/tendermint/tree/develop/abci/server), | |||||
| [JavaScript](https://github.com/tendermint/js-abci), | |||||
| [Python](https://github.com/tendermint/tendermint/tree/develop/abci/example/python3/abci), | |||||
| [C++](https://github.com/mdyring/cpp-tmsp), and | |||||
| [Java](https://github.com/jTendermint/jabci). | |||||
| ### GRPC | |||||
| If GRPC is available in your language, this is the easiest approach, | |||||
| though it will have significant performance overhead. | |||||
| To get started with GRPC, copy in the [protobuf | |||||
| file](https://github.com/tendermint/tendermint/blob/develop/abci/types/types.proto) | |||||
| and compile it using the GRPC plugin for your language. For instance, | |||||
| for golang, the command is `protoc --go_out=plugins=grpc:. types.proto`. | |||||
| See the [grpc documentation for more details](http://www.grpc.io/docs/). | |||||
| `protoc` will autogenerate all the necessary code for ABCI client and | |||||
| server in your language, including whatever interface your application | |||||
| must satisfy to be used by the ABCI server for handling requests. | |||||
| ### TSP | |||||
| If GRPC is not available in your language, or you require higher | |||||
| performance, or otherwise enjoy programming, you may implement your own | |||||
| ABCI server using the Tendermint Socket Protocol, known affectionately | |||||
| as Teaspoon. The first step is still to auto-generate the relevant data | |||||
| types and codec in your language using `protoc`. Messages coming over | |||||
| the socket are proto3 encoded, but additionally length-prefixed to | |||||
| facilitate use as a streaming protocol. proto3 doesn't have an | |||||
| official length-prefix standard, so we use our own. The first byte in | |||||
| the prefix represents the length of the Big Endian encoded length. The | |||||
| remaining bytes in the prefix are the Big Endian encoded length. | |||||
| For example, if the proto3 encoded ABCI message is 0xDEADBEEF (4 | |||||
| bytes), the length-prefixed message is 0x0104DEADBEEF. If the proto3 | |||||
| encoded ABCI message is 65535 bytes long, the length-prefixed message | |||||
| would be like 0x02FFFF.... | |||||
| Note this prefixing does not apply for grpc. | |||||
| An ABCI server must also be able to support multiple connections, as | |||||
| Tendermint uses three connections. | |||||
| ### Async vs Sync | |||||
| The main ABCI server (ie. non-GRPC) provides ordered asynchronous messages. | |||||
| This is useful for DeliverTx and CheckTx, since it allows Tendermint to forward | |||||
| transactions to the app before it's finished processing previous ones. | |||||
| Thus, DeliverTx and CheckTx messages are sent asycnhronously, while all other | |||||
| messages are sent synchronously. | |||||
| ## Client | |||||
| There are currently two use-cases for an ABCI client. One is a testing | |||||
| tool, as in the `abci-cli`, which allows ABCI requests to be sent via | |||||
| command line. The other is a consensus engine, such as Tendermint Core, | |||||
| which makes requests to the application every time a new transaction is | |||||
| received or a block is committed. | |||||
| It is unlikely that you will need to implement a client. For details of | |||||
| our client, see | |||||
| [here](https://github.com/tendermint/tendermint/tree/develop/abci/client). | |||||
+ 109
- 79
docs/spec/blockchain/blockchain.md
View File
+ 7
- 10
docs/spec/blockchain/encoding.md
View File
+ 66
- 16
docs/spec/blockchain/state.md
View File
+ 1
- 183
docs/spec/software/abci.md
View File
| @ -1,185 +1,3 @@ | |||||
| # Application Blockchain Interface (ABCI) | # Application Blockchain Interface (ABCI) | ||||
| ABCI is the interface between Tendermint (a state-machine replication engine) | |||||
| and an application (the actual state machine). | |||||
| The ABCI message types are defined in a [protobuf | |||||
| file](https://github.com/tendermint/tendermint/blob/develop/abci/types/types.proto). | |||||
| For full details on the ABCI message types and protocol, see the [ABCI | |||||
| specification](https://github.com/tendermint/tendermint/blob/develop/docs/app-dev/abci-spec.md). | |||||
| Be sure to read the specification if you're trying to build an ABCI app! | |||||
| For additional details on server implementation, see the [ABCI | |||||
| readme](https://github.com/tendermint/tendermint/blob/develop/abci/README.md). | |||||
| Here we provide some more details around the use of ABCI by Tendermint and | |||||
| clarify common "gotchas". | |||||
| ## ABCI connections | |||||
| Tendermint opens 3 ABCI connections to the app: one for Consensus, one for | |||||
| Mempool, one for Queries. | |||||
| ## Async vs Sync | |||||
| The main ABCI server (ie. non-GRPC) provides ordered asynchronous messages. | |||||
| This is useful for DeliverTx and CheckTx, since it allows Tendermint to forward | |||||
| transactions to the app before it's finished processing previous ones. | |||||
| Thus, DeliverTx and CheckTx messages are sent asycnhronously, while all other | |||||
| messages are sent synchronously. | |||||
| ## CheckTx and Commit | |||||
| It is typical to hold three distinct states in an ABCI app: CheckTxState, DeliverTxState, | |||||
| QueryState. The QueryState contains the latest committed state for a block. | |||||
| The CheckTxState and DeliverTxState may be updated concurrently with one another. | |||||
| Before Commit is called, Tendermint locks and flushes the mempool so that no new changes will happen | |||||
| to CheckTxState. When Commit completes, it unlocks the mempool. | |||||
| Thus, during Commit, it is safe to reset the QueryState and the CheckTxState to the latest DeliverTxState | |||||
| (ie. the new state from executing all the txs in the block). | |||||
| Note, however, that it is not possible to send transactions to Tendermint during Commit - if your app | |||||
| tries to send a `/broadcast_tx` to Tendermint during Commit, it will deadlock. | |||||
| ## EndBlock Validator Updates | |||||
| Updates to the Tendermint validator set can be made by returning `Validator` | |||||
| objects in the `ResponseBeginBlock`: | |||||
| ``` | |||||
| message Validator { | |||||
| PubKey pub_key | |||||
| int64 power | |||||
| } | |||||
| message PubKey { | |||||
| string type | |||||
| bytes data | |||||
| } | |||||
| ``` | |||||
| The `pub_key` currently supports two types: | |||||
| - `type = "ed25519" and`data = <raw 32-byte public key>` | |||||
| - `type = "secp256k1" and `data = <33-byte OpenSSL compressed public key>` | |||||
| If the address is provided, it must match the address of the pubkey, as | |||||
| specified [here](/docs/spec/blockchain/encoding.md#Addresses) | |||||
| (Note: In the v0.19 series, the `pub_key` is the [Amino encoded public | |||||
| key](/docs/spec/blockchain/encoding.md#public-key-cryptography). | |||||
| For Ed25519 pubkeys, the Amino prefix is always "1624DE6220". For example, the 32-byte Ed25519 pubkey | |||||
| `76852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85` would be | |||||
| Amino encoded as | |||||
| `1624DE622076852933A4686A721442E931A8415F62F5F1AEDF4910F1F252FB393F74C40C85`) | |||||
| (Note: In old versions of Tendermint (pre-v0.19.0), the pubkey is just prefixed with a | |||||
| single type byte, so for ED25519 we'd have `pub_key = 0x1 | pub`) | |||||
| The `power` is the new voting power for the validator, with the | |||||
| following rules: | |||||
| - power must be non-negative | |||||
| - if power is 0, the validator must already exist, and will be removed from the | |||||
| validator set | |||||
| - if power is non-0: | |||||
| - if the validator does not already exist, it will be added to the validator | |||||
| set with the given power | |||||
| - if the validator does already exist, its power will be adjusted to the given power | |||||
| ## InitChain Validator Updates | |||||
| ResponseInitChain has the option to return a list of validators. | |||||
| If the list is not empty, Tendermint will adopt it for the validator set. | |||||
| This way the application can determine the initial validator set for the | |||||
| blockchain. | |||||
| ResponseInitChain also includes ConsensusParams, but these are presently | |||||
| ignored. | |||||
| ## Query | |||||
| Query is a generic message type with lots of flexibility to enable diverse sets | |||||
| of queries from applications. Tendermint has no requirements from the Query | |||||
| message for normal operation - that is, the ABCI app developer need not implement Query functionality if they do not wish too. | |||||
| That said, Tendermint makes a number of queries to support some optional | |||||
| features. These are: | |||||
| ### Peer Filtering | |||||
| When Tendermint connects to a peer, it sends two queries to the ABCI application | |||||
| using the following paths, with no additional data: | |||||
| - `/p2p/filter/addr/<IP:PORT>`, where `<IP:PORT>` denote the IP address and | |||||
| the port of the connection | |||||
| - `p2p/filter/id/<ID>`, where `<ID>` is the peer node ID (ie. the | |||||
| pubkey.Address() for the peer's PubKey) | |||||
| If either of these queries return a non-zero ABCI code, Tendermint will refuse | |||||
| to connect to the peer. | |||||
| ## Info and the Handshake/Replay | |||||
| On startup, Tendermint calls Info on the Query connection to get the latest | |||||
| committed state of the app. The app MUST return information consistent with the | |||||
| last block it succesfully completed Commit for. | |||||
| If the app succesfully committed block H but not H+1, then `last_block_height = H` and `last_block_app_hash = <hash returned by Commit for block H>`. If the app | |||||
| failed during the Commit of block H, then `last_block_height = H-1` and | |||||
| `last_block_app_hash = <hash returned by Commit for block H-1, which is the hash in the header of block H>`. | |||||
| We now distinguish three heights, and describe how Tendermint syncs itself with | |||||
| the app. | |||||
| ``` | |||||
| storeBlockHeight = height of the last block Tendermint saw a commit for | |||||
| stateBlockHeight = height of the last block for which Tendermint completed all | |||||
| block processing and saved all ABCI results to disk | |||||
| appBlockHeight = height of the last block for which ABCI app succesfully | |||||
| completely Commit | |||||
| ``` | |||||
| Note we always have `storeBlockHeight >= stateBlockHeight` and `storeBlockHeight >= appBlockHeight` | |||||
| Note also we never call Commit on an ABCI app twice for the same height. | |||||
| The procedure is as follows. | |||||
| First, some simeple start conditions: | |||||
| If `appBlockHeight == 0`, then call InitChain. | |||||
| If `storeBlockHeight == 0`, we're done. | |||||
| Now, some sanity checks: | |||||
| If `storeBlockHeight < appBlockHeight`, error | |||||
| If `storeBlockHeight < stateBlockHeight`, panic | |||||
| If `storeBlockHeight > stateBlockHeight+1`, panic | |||||
| Now, the meat: | |||||
| If `storeBlockHeight == stateBlockHeight && appBlockHeight < storeBlockHeight`, | |||||
| replay all blocks in full from `appBlockHeight` to `storeBlockHeight`. | |||||
| This happens if we completed processing the block, but the app forgot its height. | |||||
| If `storeBlockHeight == stateBlockHeight && appBlockHeight == storeBlockHeight`, we're done | |||||
| This happens if we crashed at an opportune spot. | |||||
| If `storeBlockHeight == stateBlockHeight+1` | |||||
| This happens if we started processing the block but didn't finish. | |||||
| If `appBlockHeight < stateBlockHeight` | |||||
| replay all blocks in full from `appBlockHeight` to `storeBlockHeight-1`, | |||||
| and replay the block at `storeBlockHeight` using the WAL. | |||||
| This happens if the app forgot the last block it committed. | |||||
| If `appBlockHeight == stateBlockHeight`, | |||||
| replay the last block (storeBlockHeight) in full. | |||||
| This happens if we crashed before the app finished Commit | |||||
| If appBlockHeight == storeBlockHeight { | |||||
| update the state using the saved ABCI responses but dont run the block against the real app. | |||||
| This happens if we crashed after the app finished Commit but before Tendermint saved the state. | |||||
| This page has [moved](../spec/abci/apps.md). | |||||