diff --git a/docs/tendermint-core/consensus/proposer-based-timestamps.md b/docs/tendermint-core/consensus/proposer-based-timestamps.md index def42dc20..7f98f10d6 100644 --- a/docs/tendermint-core/consensus/proposer-based-timestamps.md +++ b/docs/tendermint-core/consensus/proposer-based-timestamps.md @@ -1,4 +1,6 @@ ---- order: 3 --- +--- +order: 3 +--- # PBTS @@ -6,7 +8,7 @@ algorithm added to Tendermint in the v0.36 release. It outlines the core functionality as well as the parameters and constraints of the this algorithm. -## Algorithm Overview +## Algorithm Overview The PBTS algorithm defines a way for a Tendermint blockchain to create block timestamps that are within a reasonable bound of the clocks of the validators on @@ -91,4 +93,3 @@ proposed by their peers to be valid either. * [The PBTS specification](https://github.com/tendermint/tendermint/blob/master/spec/consensus/proposer-based-timestamp/README.md) contains all of the details of the algorithm. - diff --git a/internal/statesync/reactor_test.go b/internal/statesync/reactor_test.go index dc7a73bf9..c1ac1a048 100644 --- a/internal/statesync/reactor_test.go +++ b/internal/statesync/reactor_test.go @@ -731,11 +731,15 @@ func handleLightBlockRequests( if requests%10 >= failureRate { lb, err := chain[int64(msg.Height)].ToProto() require.NoError(t, err) - sending <- p2p.Envelope{ + select { + case sending <- p2p.Envelope{ From: envelope.To, Message: &ssproto.LightBlockResponse{ LightBlock: lb, }, + }: + case <-ctx.Done(): + return } } else { switch errorCount % 3 { @@ -744,18 +748,26 @@ func handleLightBlockRequests( _, _, lb := mockLB(ctx, t, int64(msg.Height), factory.DefaultTestTime, factory.MakeBlockID(), vals, pv) differntLB, err := lb.ToProto() require.NoError(t, err) - sending <- p2p.Envelope{ + select { + case sending <- p2p.Envelope{ From: envelope.To, Message: &ssproto.LightBlockResponse{ LightBlock: differntLB, }, + }: + case <-ctx.Done(): + return } case 1: // send nil block i.e. pretend we don't have it - sending <- p2p.Envelope{ + select { + case sending <- p2p.Envelope{ From: envelope.To, Message: &ssproto.LightBlockResponse{ LightBlock: nil, }, + }: + case <-ctx.Done(): + return } case 2: // don't do anything } @@ -783,19 +795,23 @@ func handleConsensusParamsRequest( case <-ctx.Done(): return case envelope := <-receiving: - if ctx.Err() != nil { + msg, ok := envelope.Message.(*ssproto.ParamsRequest) + if !ok { + t.Errorf("message was %T which is not a params request", envelope.Message) return } - - t.Log("received consensus params request") - msg, ok := envelope.Message.(*ssproto.ParamsRequest) - require.True(t, ok) - sending <- p2p.Envelope{ + select { + case sending <- p2p.Envelope{ From: envelope.To, Message: &ssproto.ParamsResponse{ Height: msg.Height, ConsensusParams: paramsProto, }, + }: + case <-ctx.Done(): + return + case <-closeCh: + return } case <-closeCh: diff --git a/spec/abci++/README.md b/spec/abci++/README.md index f720dae4e..38feba9d7 100644 --- a/spec/abci++/README.md +++ b/spec/abci++/README.md @@ -25,8 +25,7 @@ This allows Tendermint to run with applications written in many programming lang This specification is split as follows: -- [Basic concepts and definitions](./abci++_basic_concepts_002_draft.md) - definitions and descriptions - of concepts that are needed to understand other parts of this sepcification. +- [Overview and basic concepts](./abci++_basic_concepts_002_draft.md) - interface's overview and concepts needed to understand other parts of this specification. - [Methods](./abci++_methods_002_draft.md) - complete details on all ABCI++ methods and message types. - [Requirements for the Application](./abci++_app_requirements_002_draft.md) - formal requirements diff --git a/spec/abci++/abci++_basic_concepts_002_draft.md b/spec/abci++/abci++_basic_concepts_002_draft.md index d2426f31c..22517fdbb 100644 --- a/spec/abci++/abci++_basic_concepts_002_draft.md +++ b/spec/abci++/abci++_basic_concepts_002_draft.md @@ -1,51 +1,259 @@ --- order: 1 -title: Basic concepts and definitions +title: Overview and basic concepts --- -# Basic concepts and definitions +## Outline +- [ABCI++ vs. ABCI](#abci-vs-abci) +- [Methods overview](#methods-overview) + - [Consensus methods](#consensus-methods) + - [Mempool methods](#mempool-methods) + - [Info methods](#info-methods) + - [State-sync methods](#state-sync-methods) +- [Next-block execution vs. same-block execution](#next-block-execution-vs-same-block-execution) + - [Tendermint timeouts](#tendermint-timeouts-in-same-block-execution) +- [Determinism](#determinism) +- [Errors](#errors) +- [Events](#events) +- [Evidence](#evidence) -## Connections +# Overview and basic concepts -ABCI++ applications can run either within the _same_ process as the Tendermint -state-machine replication engine, or as a _separate_ process from the state-machine -replication engine. When run within the same process, Tendermint will call the ABCI++ -application methods directly as Go method calls. +## ABCI++ vs. ABCI +[↑ Back to Outline](#outline) -When Tendermint and the ABCI++ application are run as separate processes, Tendermint -opens four connections to the application for ABCI++ methods. The connections each -handle a subset of the ABCI++ method calls. These subsets are defined as follows: +With ABCI, the application can only act at one phase in consensus, immediately after a block has been finalized. This restriction on the application prevents numerous features for the application, including many scalability improvements that are now better understood than when ABCI was first written. For example, many of the scalability proposals can be boiled down to "Make the miner / block proposers / validators do work, so the network does not have to". This includes optimizations such as tx-level signature aggregation, state transition proofs, etc. Furthermore, many new security properties cannot be achieved in the current paradigm, as the application cannot enforce validators to do more than just finalize txs. This includes features such as threshold cryptography, and guaranteed IBC connection attempts. -### **Consensus** connection +ABCI++ overcomes these limitations by allowing the application to intervene at three key places of the block execution. The new interface allows block proposers to perform application-dependent work in a block through the `PrepareProposal` method; validators to perform application-dependent work in a proposed block through the `ProcessProposal` method; and applications to require their validators do more than just validate blocks, e.g., validator guaranteed IBC connection attempts, through the `ExtendVote` and `VerifyVoteExtension` methods. Furthermore, ABCI++ renames {`BeginBlock`, [`DeliverTx`], `EndBlock`} to `FinalizeBlock`, as a simplified way to deliver a decided block to the Application. -* Driven by a consensus protocol and is responsible for block execution. -* Handles the `InitChain`, `PrepareProposal`, `ProcessProposal`, `ExtendVote`, - `VerifyVoteExtension`, and `FinalizeBlock` method calls. +## Methods overview +[↑ Back to Outline](#outline) -### **Mempool** connection +Methods can be classified into four categories: consensus, mempool, info, and state-sync. -* For validating new transactions, before they're shared or included in a block. -* Handles the `CheckTx` calls. +### Consensus/block execution methods -### **Info** connection +The first time a new blockchain is started, Tendermint calls +`InitChain`. From then on, method `FinalizeBlock` is executed at the end of each +block, resulting in an updated Application state. +During consensus execution of a block height, before method `FinalizeBlock` is +called, methods `PrepareProposal`, `ProcessProposal`, `ExtendVote`, and +`VerifyVoteExtension` may be called several times. +See [Tendermint's expected behavior](abci++_tmint_expected_behavior_002_draft.md) +for details on the possible call sequences of these methods. -* For initialization and for queries from the user. -* Handles the `Info` and `Query` calls. +* [**InitChain:**](./abci++_methods_002_draft.md#initchain) This method initializes the blockchain. Tendermint calls it once upon genesis. -### **Snapshot** connection +* [**PrepareProposal:**](./abci++_methods_002_draft.md#prepareproposal) It allows the block proposer to perform application-dependent work in a block before using it as its proposal. This enables, for instance, batch optimizations to a block, which has been empirically demonstrated to be a key component for scaling. Method `PrepareProposal` is called every time Tendermint is about to send +a proposal message, but no previous proposal has been locked at Tendermint level. +Tendermint gathers outstanding transactions from the mempool, generates a block header, and uses +them to create a block to propose. Then, it calls `RequestPrepareProposal` +with the newly created proposal, called _raw proposal_. The Application can +make changes to the raw proposal, such as modifying transactions, and returns +the (potentially) modified proposal, called _prepared proposal_ in the +`Response*` call. The logic modifying the raw proposal can be non-deterministic. -* For serving and restoring [state sync snapshots](../abci/apps.md#state-sync). -* Handles the `ListSnapshots`, `LoadSnapshotChunk`, `OfferSnapshot`, and `ApplySnapshotChunk` calls. +* [**ProcessProposal:**](./abci++_methods_002_draft.md#processproposal) It allows a validator to perform application-dependent work in a proposed block. This enables features such as allowing validators to reject a block according to whether the state machine deems it valid, and changing the block execution pipeline. Tendermint calls it when it receives a proposal and it is not locked on a block. The Application cannot +modify the proposal at this point but can reject it if it realizes it is invalid. +If that is the case, Tendermint will prevote `nil` on the proposal, which has +strong liveness implications for Tendermint. As a general rule, the Application +SHOULD accept a prepared proposal passed via `ProcessProposal`, even if a part of +the proposal is invalid (e.g., an invalid transaction); the Application can +ignore the invalid part of the prepared proposal at block execution time. -Additionally, there is a `Flush` method that is called on every connection, -and an `Echo` method that is just for debugging. +* [**ExtendVote:**](./abci++_methods_002_draft.md#extendvote) It allows applications to force their validators to do more than just validate within consensus. `ExtendVote` allows applications to include non-deterministic data, opaque to Tendermint, to precommit messages (the final round of voting). +The data, called _vote extension_, will also be made available to the +application in the next height, along with the vote it is extending, in the rounds +where the local process is the proposer. +The Application may also choose not to include any vote extension. +Tendermint calls it when is about to send a non-`nil` precommit message. ->**TODO** Figure out what to do with this. +* [**VerifyVoteExtension:**](./abci++_methods_002_draft.md#verifyvoteextension) It allows validators to validate the vote extension data attached to a precommit message. If the validation fails, the precommit message will be deemed invalid and ignored +by Tendermint. This has a negative impact on Tendermint's liveness, i.e., if vote extensions repeatedly cannot be verified by correct validators, Tendermint may not be able to finalize a block even if sufficiently many (+2/3) of the validators send precommit votes for that block. Thus, `VerifyVoteExtension` should be used with special care. +As a general rule, an Application that detects an invalid vote extension SHOULD +accept it in `ResponseVerifyVoteExtension` and ignore it in its own logic. Tendermint calls it when +a process receives a precommit message with a (possibly empty) vote extension. + +* [**FinalizeBlock:**](./abci++_methods_002_draft.md#finalizeblock) It delivers a decided block to the Application. The Application must execute the transactions in the block in order and update its state accordingly. Cryptographic commitments to the block and transaction results, via the corresponding +parameters in `ResponseFinalizeBlock`, are included in the header of the next block. Tendermint calls it when a new block is decided. + +### Mempool methods + +* [**CheckTx:**](./abci++_methods_002_draft.md#checktx) This method allows the Application to validate transactions against its current state, e.g., checking signatures and account balances. If a transaction passes the validation, then tendermint adds it to its local mempool, discarding it otherwise. Tendermint calls it when it receives a new transaction either coming from an external user or another node. Furthermore, Tendermint can be configured to re-call `CheckTx` on any decided transaction (after `FinalizeBlock`). + +### Info methods + +* [**Info:**](./abci++_methods_002_draft.md#info) Used to sync Tendermint with the Application during a handshake that happens on startup. + +* [**Query:**](./abci++_methods_002_draft.md#query) Clients can use this method to query the Application for information about the application state. + +### State-sync methods + +State sync allows new nodes to rapidly bootstrap by discovering, fetching, and applying +state machine snapshots instead of replaying historical blocks. For more details, see the +[state sync section](../p2p/messages/state-sync.md). + +New nodes will discover and request snapshots from other nodes in the P2P network. +A Tendermint node that receives a request for snapshots from a peer will call +`ListSnapshots` on its Application. The Application returns the list of locally avaiable snapshots. +Note that the list does not contain the actual snapshot but metadata about it: height at which the snapshot was taken, application-specific verification data and more (see [snapshot data type](./abci++_methods_002_draft.md#snapshot) for more details). After receiving a list of available snapshots from a peer, the new node can offer any of the snapshots in the list to its local Application via the `OfferSnapshot` method. The Application can check at this point the validity of the snapshot metadata. + +Snapshots may be quite large and are thus broken into smaller "chunks" that can be +assembled into the whole snapshot. Once the Application accepts a snapshot and +begins restoring it, Tendermint will fetch snapshot "chunks" from existing nodes. +The node providing "chunks" will fetch them from its local Application using +the `LoadSnapshotChunk` method. + +As the new node receives "chunks" it will apply them sequentially to the local +application with `ApplySnapshotChunk`. When all chunks have been applied, the +Application's `AppHash` is retrieved via an `Info` query. +To ensure that the sync proceeded correctly, Tendermint compares the local Application's `AppHash` to the `AppHash` stored on the blockchain (verified via +[light client verification](../light-client/verification/README.md)). + +In summary: + +* [**ListSnapshots:**](./abci++_methods_002_draft.md#listsnapshots) Used by nodes to discover available snapshots on peers. + +* [**LoadSnapshotChunk:**](./abci++_methods_002_draft.md#loadsnapshotchunk) Used by Tendermint to retrieve snapshot chunks from the application to send to peers. + +* [**OfferSnapshot:**](./abci++_methods_002_draft.md#offersnapshot) When a node receives a snapshot from a peer, Tendermint uses this method to offer the snapshot to the Application. + +* [**ApplySnapshotChunk:**](./abci++_methods_002_draft.md#applysnapshotchunk) Used by Tendermint to hand snapshot chunks to the Application. + +### Other methods + +Additionally, there is a [**Flush**](./abci++_methods_002_draft.md#flush) method that is called on every connection, +and an [**Echo**](./abci++_methods_002_draft.md#echo) method that is just for debugging. More details on managing state across connections can be found in the section on [ABCI Applications](../abci/apps.md). +## Next-block execution vs. same-block execution +[↑ Back to Outline](#outline) + +In the original ABCI protocol, the only moment when the Application had access to a +block was after it was decided. This led to a block execution model, called _next-block +execution_, where some fields hashed in a block header refer to the execution of the +previous block, namely: + +* the Merkle root of the Application's state +* the transaction results +* the consensus parameter updates +* the validator updates + +With ABCI++, an Application may decide to keep using the next-block execution model, by doing all its processing in `FinalizeBlock`; +however the new methods introduced, `PrepareProposal` and `ProcessProposal` allow +for a new execution model, called _same-block execution_. An Application implementing +this execution model, upon receiving a raw proposal via `RequestPrepareProposal` +and potentially modifying its transaction list, +fully executes the resulting prepared proposal as though it was the decided block. +The results of the block execution are used as follows: + +* The block execution may generate a set of events. The Application should store these events and return them back to Tendermint during the `FinalizeBlock` call if the block is finally decided. +* The Merkle root resulting from executing the prepared proposal is provided in + `ResponsePrepareProposal` and thus refers to the **current block**. Tendermint + will use it in the prepared proposal's header. +* likewise, the transaction results from executing the prepared proposal are + provided in `ResponsePrepareProposal` and refer to the transactions in the + **current block**. Tendermint will use them to calculate the results hash + in the prepared proposal's header. +* The consensus parameter updates and validator updates are also provided in + `ResponsePrepareProposal` and reflect the result of the prepared proposal's + execution. They come into force in height H+1 (as opposed to the H+2 rule + in next-block execution model). + +If the Application decides to keep the next-block execution model, it will not +provide any data in `ResponsePrepareProposal`, other than an optionally modified +transaction list. + +In the long term, the execution model will be set in a new boolean parameter +*same_block* in `ConsensusParams`. +It **must not** be changed once the blockchain has started unless the Application +developers _really_ know what they are doing. +However, modifying `ConsensusParams` structure cannot be done lightly if we are to +preserve blockchain compatibility. Therefore we need an interim solution until +soft upgrades are specified and implemented in Tendermint. This somewhat _unsafe_ +solution consists in Tendermint assuming same-block execution if the Application +fills the above mentioned fields in `ResponsePrepareProposal`. + +### Tendermint timeouts in same-block execution + +The new same-block execution mode requires the Application to fully execute the +prepared block at `PrepareProposal` time. This execution is synchronous, so +Tendermint cannot make progress until the Application returns from `PrepareProposal`. +This stands on Tendermint's critical path: if the Application takes a long time +executing the block, the default value of _TimeoutPropose_ might not be sufficient +to accommodate the long block execution time and non-proposer processes might time +out and prevote `nil`, thus starting a further round unnecessarily. + +The Application is the best suited to provide a value for _TimeoutPropose_ so +that the block execution time upon `PrepareProposal` fits well in the propose +timeout interval. + +Currently, the Application can override the value of _TimeoutPropose_ via the +`config.toml` file. In the future, `ConsensusParams` will have an extra field +with the current _TimeoutPropose_ value so that the Application can adapt it at every height. + +## Determinism +[↑ Back to Outline](#outline) + +ABCI++ applications must implement deterministic finite-state machines to be +securely replicated by the Tendermint consensus engine. This means block execution +over the Consensus Connection must be strictly deterministic: given the same +ordered set of transactions, all nodes will compute identical responses, for all +successive `FinalizeBlock` calls. This is critical because the +responses are included in the header of the next block, either via a Merkle root +or directly, so all nodes must agree on exactly what they are. + +For this reason, it is recommended that application state is not exposed to any +external user or process except via the ABCI connections to a consensus engine +like Tendermint Core. The Application must only change its state based on input +from block execution (`FinalizeBlock` calls), and not through +any other kind of request. This is the only way to ensure all nodes see the same +transactions and compute the same results. + +Some Applications may choose to execute the blocks that are about to be proposed +(via `PrepareProposal`), or those that the Application is asked to validate +(via `Processproposal`). However, the state changes caused by processing those +proposed blocks must never replace the previous state until `FinalizeBlock` confirms +the block decided. + +Additionally, vote extensions or the validation thereof (via `ExtendVote` or +`VerifyVoteExtension`) must _never_ have side effects on the current state. +They can only be used when their data is provided in a `RequestPrepareProposal` call. + +If there is some non-determinism in the state machine, consensus will eventually +fail as nodes disagree over the correct values for the block header. The +non-determinism must be fixed and the nodes restarted. + +Sources of non-determinism in applications may include: + +* Hardware failures + * Cosmic rays, overheating, etc. +* Node-dependent state + * Random numbers + * Time +* Underspecification + * Library version changes + * Race conditions + * Floating point numbers + * JSON or protobuf serialization + * Iterating through hash-tables/maps/dictionaries +* External Sources + * Filesystem + * Network calls (eg. some external REST API service) + +See [#56](https://github.com/tendermint/abci/issues/56) for original discussion. + +Note that some methods (`Query, CheckTx, FinalizeBlock`) return +explicitly non-deterministic data in the form of `Info` and `Log` fields. The `Log` is +intended for the literal output from the Application's logger, while the +`Info` is any additional info that should be returned. These are the only fields +that are not included in block header computations, so we don't need agreement +on them. All other fields in the `Response*` must be strictly deterministic. + ## Errors +[↑ Back to Outline](#outline) The `Query`, and `CheckTx` methods include a `Code` field in their `Response*`. The `Code` field is also included in type `TxResult`, used by @@ -75,8 +283,6 @@ The handling of non-zero response codes by Tendermint is described below ### `CheckTx` -The `CheckTx` ABCI++ method controls what transactions are considered for inclusion -in a block. When Tendermint receives a `ResponseCheckTx` with a non-zero `Code`, the associated transaction will not be added to Tendermint's mempool or it will be removed if it is already included. @@ -91,11 +297,11 @@ Tendermint consensus. ### `Query` -The `Query` ABCI++ method queries the Application for information about application state. When Tendermint receives a `ResponseQuery` with a non-zero `Code`, this code is returned directly to the client that initiated the query. ## Events +[↑ Back to Outline](#outline) Method `CheckTx` includes an `Events` field in its `Response*`. Method `FinalizeBlock` includes an `Events` field at the top level in its @@ -171,11 +377,12 @@ Example: } ``` -## EvidenceType +## Evidence +[↑ Back to Outline](#outline) Tendermint's security model relies on the use of "evidence". Evidence is proof of -malicious behaviour by a network participant. It is the responsibility of Tendermint -to detect such malicious behaviour. When malicious behavior is detected, Tendermint +malicious behavior by a network participant. It is the responsibility of Tendermint +to detect such malicious behavior. When malicious behavior is detected, Tendermint will gossip evidence of the behavior to other nodes and commit the evidence to the chain once it is verified by all validators. This evidence will then be passed on to the Application through ABCI++. It is the responsibility of the @@ -195,208 +402,3 @@ There are two forms of evidence: Duplicate Vote and Light Client Attack. More information can be found in either [data structures](../core/data_structures.md) or [accountability](../light-client/accountability/) -## Vote Extensions - -According to the Tendermint algorithm, a proposed block needs at least a predefined -number of precommit votes in order to be decided. Tendermint gathers all the valid -precommit votes for the decided block that it receives before the block is decided, -and then includes these votes in the proposed block for the next height whenever -the local process is the proposer of the round. - -When Tendermint's consensus is about to send a non-`nil` precommit message, it calls -method `ExtendVote`, which gives the Application the opportunity to include -non-deterministic data, opaque to Tendermint, that will be attached to the precommit -message. The data, called _vote extension_, will also be made available to the -application in the next height, along with the vote it is extending, in the rounds -where the local process is the proposer. - -The vote extension data is split into two parts, one signed by Tendermint as part -of the vote data structure, and the other (optionally) signed by the Application. -The Application may also choose not to include any vote extension. -When another process receives a precommit message with a vote extension, it calls -method `VerifyVoteExtension` so that the Application can validate the data received. -If the validation fails, the precommit message will be deemed invalid and ignored -by Tendermint. This has negative impact on Tendermint's liveness, i.e., if repeatedly vote extensions by correct validators cannot be verified by correct validators, Tendermint may not be able to finalize a block even if sufficiently many (+2/3) of the validators send precommit votes for that block. Thus, `VerifyVoteExtension` should only be used with special care. -As a general rule, an Application that detects an invalid vote extension SHOULD -accept it in `ResponseVerifyVoteExtension` and ignore it in its own logic. - -## Determinism - -ABCI++ applications must implement deterministic finite-state machines to be -securely replicated by the Tendermint consensus engine. This means block execution -over the Consensus Connection must be strictly deterministic: given the same -ordered set of requests, all nodes will compute identical responses, for all -successive `FinalizeBlock` calls. This is critical, because the -responses are included in the header of the next block, either via a Merkle root -or directly, so all nodes must agree on exactly what they are. - -For this reason, it is recommended that applications not be exposed to any -external user or process except via the ABCI connections to a consensus engine -like Tendermint Core. The Application must only change its state based on input -from block execution (`FinalizeBlock` calls), and not through -any other kind of request. This is the only way to ensure all nodes see the same -transactions and compute the same results. - -Some Applications may choose to execute the blocks that are about to be proposed -(via `PrepareProposal`), or those that the Application is asked to validate -(via `Processproposal`). However the state changes caused by processing those -proposed blocks must never replace the previous state until `FinalizeBlock` confirms -the block decided. - -Additionally, vote extensions or the validation thereof (via `ExtendVote` or -`VerifyVoteExtension`) must _never_ have side effects on the current state. -They can only be used when their data is included in a block. - -If there is some non-determinism in the state machine, consensus will eventually -fail as nodes disagree over the correct values for the block header. The -non-determinism must be fixed and the nodes restarted. - -Sources of non-determinism in applications may include: - -* Hardware failures - * Cosmic rays, overheating, etc. -* Node-dependent state - * Random numbers - * Time -* Underspecification - * Library version changes - * Race conditions - * Floating point numbers - * JSON or protobuf serialization - * Iterating through hash-tables/maps/dictionaries -* External Sources - * Filesystem - * Network calls (eg. some external REST API service) - -See [#56](https://github.com/tendermint/abci/issues/56) for original discussion. - -Note that some methods (`Query, CheckTx, FinalizeBlock`) return -explicitly non-deterministic data in the form of `Info` and `Log` fields. The `Log` is -intended for the literal output from the Application's logger, while the -`Info` is any additional info that should be returned. These are the only fields -that are not included in block header computations, so we don't need agreement -on them. All other fields in the `Response*` must be strictly deterministic. - -## Block Execution - -The first time a new blockchain is started, Tendermint calls -`InitChain`. From then on, method `FinalizeBlock` is executed at the end of each -block, resulting in an updated Application state. -During consensus execution of a block height, before method `FinalizeBlock` is -called, methods `PrepareProposal`, `ProcessProposal`, `ExtendVote`, and -`VerifyVoteExtension` may be called a number of times. -See [Tendermint's expected behavior](abci++_tmint_expected_behavior_002_draft.md) -for details on the possible call sequences of these methods. - -Method `PrepareProposal` is called every time Tendermint is about to send -a proposal message, but no previous proposal has been locked at Tendermint level. -Tendermint gathers outstanding transactions from the mempool -(see [PrepareProposal](#PrepareProposal)), generates a block header and uses -them to create a block to propose. Then, it calls `RequestPrepareProposal` -with the newly created proposal, called _raw proposal_. The Application can -make changes to the raw proposal, such as modifying transactions, and returns -the (potentially) modified proposal, called _prepared proposal_ in the -`Response*` call. The logic modifying the raw proposal can be non-deterministic. - -When Tendermint receives a prepared proposal it uses method `ProcessProposal` -to inform the Application of the proposal just received. The Application cannot -modify the proposal at this point but can reject it if it realises it is invalid. -If that is the case, Tendermint will prevote `nil` on the proposal, which has -strong liveness implications for Tendermint. As a general rule, the Application -SHOULD accept a prepared proposal passed via `ProcessProposal`, even if a part of -the proposal is invalid (e.g., an invalid transaction); the Application can later -ignore the invalid part of the prepared proposal at block execution time. - -Cryptographic commitments to the block and transaction results, via the corresponding -parameters in `FinalizeBlockResponse` are included in the header of the next block. - -## Next-block execution and same-block execution - -With ABCI++ predecessor, ABCI, the only moment when the Application had access to a -block was when it was decided. This led to a block execution model, called _next-block -execution_, where some fields hashed in a block header refer to the execution of the -previous block, namely: - -* the merkle root of the Application's state -* the transaction results -* the consensus parameter updates -* the validator updates - -With ABCI++, an Application may decide to keep using the next-block execution model; -however the new methods introduced, `PrepareProposal` and `ProcessProposal` allow -for a new execution model, called _same-block execution_. An Application implementing -this execution model, upon receiving a raw proposal via `RequestPrepareProposal` -and potentially modifying its transaction list, -fully executes the resulting prepared proposal as though it was the decided block. -The results of the block execution are used as follows: - -* the Application keeps the events generated and provides them if `FinalizeBlock` - is finally called on this prepared proposal. -* the merkle root resulting from executing the prepared proposal is provided in - `ResponsePrepareProposal` and thus refers to the **current block**. Tendermint - will use it in the prepared proposal's header. -* likewise, the transaction results from executing the prepared proposal are - provided in `ResponsePrepareProposal` and refer to the transactions in the - **current block**. Tendermint will use them to calculate the results hash - in the prepared proposal's header. -* the consensus parameter updates and validator updates are also provided in - `ResponsePrepareProposal` and reflect the result of the prepared proposal's - execution. They come into force in height H+1 (as opposed to the H+2 rule - in next-block execution model). - -If the Application decides to keep the next-block execution model, it will not -provide any data in `ResponsePrepareProposal`, other than an optionally modified -transaction list. - -In the long term, the execution model will be set in a new boolean parameter -*same_block* in `ConsensusParams`. -It should **not** be changed once the blockchain has started, unless the Application -developers _really_ know what they are doing. -However, modifying `ConsensusParams` structure cannot be done lightly if we are to -preserve blockchain compatibility. Therefore we need an interim solution until -soft upgrades are specified and implemented in Tendermint. This somewhat _unsafe_ -solution consists in Tendermint assuming same-block execution if the Application -fills the above mentioned fields in `ResponsePrepareProposal`. - -## Tendermint timeouts in same-block execution - -The new same-block execution mode requires the Application to fully execute the -prepared block at `PrepareProposal` time. This execution is synchronous, so -Tendermint cannot make progress until the Application returns from `PrepareProposal`. -This stands on Tendermint's critical path: if the Application takes a long time -executing the block, the default value of _TimeoutPropose_ might not be sufficient -to accomodate the long block execution time and non-proposer processes might time -out and prevote `nil`, thus starting a further round unnecessarily. - -The Application is the best suited to provide a value for _TimeoutPropose_ so -that the block execution time upon `PrepareProposal` fits well in the propose -timeout interval. - -Currently, the Application can override the value of _TimeoutPropose_ via the -`config.toml` file. In the future, `ConsensusParams` may have an extra field -with the current _TimeoutPropose_ value so that the Application has the possibility -to adapt it at every height. - -## State Sync - -State sync allows new nodes to rapidly bootstrap by discovering, fetching, and applying -state machine snapshots instead of replaying historical blocks. For more details, see the -[state sync section](../p2p/messages/state-sync.md). - -New nodes will discover and request snapshots from other nodes in the P2P network. -A Tendermint node that receives a request for snapshots from a peer will call -`ListSnapshots` on its Application to retrieve any local state snapshots. After receiving - snapshots from peers, the new node will offer each snapshot received from a peer -to its local Application via the `OfferSnapshot` method. - -Snapshots may be quite large and are thus broken into smaller "chunks" that can be -assembled into the whole snapshot. Once the Application accepts a snapshot and -begins restoring it, Tendermint will fetch snapshot "chunks" from existing nodes. -The node providing "chunks" will fetch them from its local Application using -the `LoadSnapshotChunk` method. - -As the new node receives "chunks" it will apply them sequentially to the local -application with `ApplySnapshotChunk`. When all chunks have been applied, the -Application's `AppHash` is retrieved via an `Info` query. The `AppHash` is then -compared to the blockchain's `AppHash` which is verified via -[light client verification](../light-client/verification/README.md). diff --git a/spec/abci++/abci++_methods_002_draft.md b/spec/abci++/abci++_methods_002_draft.md index 5f811a03f..316c16dbc 100644 --- a/spec/abci++/abci++_methods_002_draft.md +++ b/spec/abci++/abci++_methods_002_draft.md @@ -311,9 +311,9 @@ title: Methods and `RequestFinalizeBlock`. * The header contains the height, timestamp, and more - it exactly matches the Tendermint block header. - * `RequestPrepareProposal` contains a preliminary set of transactions `txs` that Tendermint considers to be a good block proposal, called _raw block_. The Application can modify this set via `ResponsePrepareProposal.tx_records` (see [TxRecord](#txrecord)). + * `RequestPrepareProposal` contains a preliminary set of transactions `txs` that Tendermint considers to be a good block proposal, called _raw proposal_. The Application can modify this set via `ResponsePrepareProposal.tx_records` (see [TxRecord](#txrecord)). * In this case, the Application should set `ResponsePrepareProposal.modified_tx` to true. - * The Application _can_ reorder, remove or add transactions to the raw block. Let `tx` be a transaction in `txs`: + * The Application _can_ reorder, remove or add transactions to the raw proposal. Let `tx` be a transaction in `txs`: * If the Application considers that `tx` should not be proposed in this block, e.g., there are other transactions with higher priority, then it should not include it in `tx_records`. In this case, Tendermint won't remove `tx` from the mempool. The Application should be extra-careful, as abusing this feature may cause transactions to stay forever in the mempool. * If the Application considers that a `tx` should not be included in the proposal and removed from the mempool, then the Application should include it in `tx_records` and _mark_ it as "REMOVE". In this case, Tendermint will remove `tx` from the mempool. * If the Application wants to add a new transaction, then the Application should include it in `tx_records` and _mark_ it as "ADD". In this case, Tendermint will add it to the mempool.