* add new transactions (not previously in the mempool) - `TxAction = ADDED`
* removed transactions (invalid) from the proposal and from the mempool - `TxAction = REMOVED`
* remove transactions from the proposal but not from the mempool (effectively _delaying_ them) - the
Application removes the transaction from the list
* modify transactions (e.g. aggregate them) - `TxAction = ADDED` followed by `TxAction = REMOVED`
* reorder transactions - the Application reorders transactions in the list
* **TODO**: include logic of `VerifyVoteExtension` here?
4. If the block is modified, the Application sets `ResponsePrepareProposal.modified` to true,
and includes the modified block in the return parameters (see the rules in section _Usage_).
The Application returns from the call.
@ -133,19 +136,22 @@ Note that, if _p_ has a non-`nil` _validValue_, Tendermint will use it as propos
* **Usage**:
* Contains a full proposed block.
* The parameters and types of `RequestProcessProposal` are the same as `RequestFinalizeBlock`.
* The parameters and types of `RequestProcessProposal` are the same as `RequestPrepareProposal`
and `RequestFinalizeBlock`.
* In "same-block execution" mode, the Application will fully execute the block
as though it was handling `RequestFinalizeBlock`.
However, any resulting state changes must be kept as _canditade state_,
and the Application should be ready to
backtrack/discard it in case the decided block is different.
* If `ResponseProcessProposal.accept` is false, Tendermint should assume the proposal received
* The header contains the height, timestamp, and more - it exactly matches the
Tendermint block header.
* If `ResponseProcessProposal.accept` is _false_, Tendermint assumes the proposal received
is not valid.
* The implementation of `ProcessProposal` MUST be deterministic. Moreover, the value of
`ResponseProcessProposal.accept` MUST *exclusively* depend on the parameters passed in
the call to `RequestProcessProposal`, and the last committed Application state
(see [Properties](#properties) section below).
* Moreover, application implementors SHOULD always set `ResponseProcessProposal.accept` to _true_
* Moreover, application implementors SHOULD always set `ResponseProcessProposal.accept` to _true_,
unless they _really_ know what the potential liveness implications of returning _false_ are.
>**TODO**: should `ResponseProcessProposal.accept` be of type `Result` rather than `bool`? (so we are able to extend the possible values in the future?)
@ -168,10 +174,7 @@ When a validator _p_ enters Tendermint consensus round _r_, height _h_, in which
not be able to reject the block, or force prevote/precommit `nil` afterwards.
3. If the returned value is
* _accept_, Tendermint prevotes on this proposal for round _r_, height _h_.
* _reject_, Tendermint will prevote $\bot$ for the proposal in round _r_, height _h_, which is a special value that processes can provote and precommit for,
and also decide. Processes can still vote `nil` according to the Tendermint algorithm. The role of $\bot$ is to protect Tendermint's liveness in cases
where the Application has a problem in its implementation of _ProcessProposal_ that, albeit deterministically, rejects too many proposals.
**TODO** Reword this when the "failure mode" discussion is settled.
* _reject_, Tendermint prevotes `nil`.
### ExtendVote
@ -234,14 +237,13 @@ In the cases when _p_'s Tendermint is to broadcast `precommit nil` messages (eit
| accept | bool | If false, Application is rejecting the vote extension | 1 |
* **Usage**:
* If `ResponseVerifyVoteExtension.accept` is false, Tendermint will flag the extension as _invalid_ but the
validity of the received Precommit message will not be influenced by this.
**TODO**: reword after deciding on failure mode
* If `ResponseVerifyVoteExtension.accept` is _false_, Tendermint will reject the whole received vote.
See the [Properties](#properties) section to understand the potential liveness implications of this.
* The implementation of `VerifyVoteExtension` MUST be deterministic. Moreover, the value of
`ResponseVerifyVoteExtension.accept` MUST *exclusively* depend on the parameters passed in
the call to `RequestVerifyVoteExtension`, and the last committed Application state
(see [Properties](#properties) section below).
* Moreover, application implementors SHOULD always set `ResponseVerifyVoteExtension.accept` to _true_
* Moreover, application implementors SHOULD always set `ResponseVerifyVoteExtension.accept` to _true_,
unless they _really_ know what the potential liveness implications of returning _false_ are.
@ -252,13 +254,14 @@ from this condition, but not sure), and _p_ receives a Precommit message for rou
>**TODO** [From Josef] We should understand the influence of equivocation on proposals in ABCI++
>
>* So far, the operation mode for ABCI has been mainly synchronous. The most notable exception is: checkTx (deliverTx is not really async).
> Calls to checkTx can afford to be async because checkTx provides no strong properties: checkTx returning Accept does not guarantee the Tx's validity when it makes it into a block.
>* AFAIU: The "join" part has two aims in the pseudocode (v4):
> * early collection and treatment of evidences. See comments above
> * influencing the precommit to be sent (whether _id(v)_ or `nil`)
>N.B.#1: If Byzantine proposer proposes both A and B in the same round, today we might only receive A (proposal) and not B (due to p2p implementation)
>
>* Several issues we need to fully understand
> * Changing ABCI from sync to async may have unintended side effects on both sides. We are introducing concurrency in the API that affects the logic of consensus
> * If App is late in providing the result, Tendermint may advance to r=1, thus forking a new ProcessProposal in parallel --> even more time to complete --> vicious circle
> * But, the main issue we'd need to overcome is the possibility that the value rejected by _ProcessProposal_ gets locked by $f + 1$ correct processes in that round
> * In this case (according to Tendermint algorithm's proof) no other value can be decided. Liveness breaks down!
> * About introducing $\bot$ here to fix liveness, I don't think it works (unlike at prevote step) as $\bot$ is considered "just another" value you can decide on,
> so it must follow the same rules to get locked and then decided (but the rejected value is already locked!)
>Sergio: some thoughts:
>
>**END TODO**
#### Separation of `VerifyHeader` and `ProcessProposal`
>**TODO** My interpretation of Dev's pseudo code is that it only makes sense if using the "fork-join" mechanism. So this discussion should be carried out after the one above
#### Byzantine proposer
[From Josef] We should understand the influence of equivocation on proposals in ABCI++
N.B.#1: If Byzantine proposer proposes both A and B in the same round, today we might only receive A (proposal) and not B
Sergio: preliminary thoughts:
[Thought 1] If Tendermint delivers all proposals from a Byzantine proposer _p_ in _h_ and _r_ to the App, we're vulnerable to DoS,
as _p_ can send as many as it wishes
* Assuming N.B.#1 above gets "fixed" (A and B get delivered by p2p)
* Assuming here an App in "same-block execution" mode, i.e., it fully executes the proposed block at _ProcessProposal_ time
So probably we are better off if we do not "fix" N.B.#1.
* or, a way to fix it would be to only deliver the first proposal to arrive to the Application, and use the others just to submit evidence
[Thought 2] _ProcessProposal_ is exposed to arbitrary input, yet it should still behave deterministically.
* In ABCI, this is also the case upon (BeginBlock, [DeliverTx], EndBlock, Commit). The difference is that, at the end of the block
we have the "protection" of the AppHash, and ResultsHash to guard against non-determinism.
* Here, arbitrary proposals that "uncover" non-determinism issues in the _ProcessProposal_ code compromise consensus's liveness
* Any ideas what we can do here? (I have the impression $\bot$ helps here only partially)
[Thought 3] In terms of the consensus's safety properties, as far as pure equivocation on proposals is concerned,
I think we are OK since Tendermint builds on a Byzantine failure model.
>[Thought 1] If Tendermint delivers all proposals from a Byzantine proposer _p_ in _h_ and _r_ to the App, we're vulnerable to DoS attacks,
>as _p_ can send as many as it wishes:
>
>* Assuming N.B.#1 above gets "fixed" (A and B get delivered by p2p)
>* Assuming the App fully executes the proposed block at _ProcessProposal_ time
>
>So, whenever N.B.#1 is "fixed" at p2p level, we need to ensure that only the the first proposal in round _r_, height _h_
>gets delivered to the Application. Any subsequent proposal for that round should just be used to submit evidence, but not given to the App.
>
>[Thought 2] In terms of the consensus's safety properties, as far as pure equivocation on proposals is concerned,
and `RequestVerifyVoteExtension` at height $h$ do not modify $s_{p,h-1}$.
* Property 9 [_all_, read-only]: $p$'s calls to `RequestPrepareProposal`, `RequestProcessProposal`,
`RequestExtendVote`, and `RequestVerifyVoteExtension` at height $h$ do not modify $s_{p,h-1}$.
* Property 10 [`ExtendVote`, `FinalizeBlock`, non-dependency]: for any correct process $p$,
and any vote extension $e$ that $q$ received at height $h$, the computation of
$s{p,h}$ does not depend on $e$.
>**TODO** We need other properties for `FinalizeBlock`, but postponing ATM as they are well
understood (basically, same a current ABCI).
The call to correct process $p$'s `RequestFinalizeBlock` at height $h$, with block $v_{p,h}$
passed as parameter, creates state $s_{p,h}$.
Additionally, $p$'s `FinalizeBlock` creates a set of transaction results $T_{p,h}$.
>**TODO** I have left out all the "events" as they don't have any impact in safety or liveness
>(same for consensus params, and validator set)
* Property 11 [`FinalizeBlock`, determinism-1]: For any correct process $p$,
the contents of $s_{p,h}$ exclusively depend on $s_{p,h}$ and $v_{p,h}$.
* Property 12 [`FinalizeBlock`, determinism-2]: For any correct process $p$,
the contents of $T_{p,h}$ exclusively depend on $s_{p,h}$ and $v_{p,h}$.
Note that Properties 11 and 12, combined with Agreement property of consensus ensure
the Application state evolves consistently at all correct processes.
Finally, notice that neither `PrepareProposal` nor `ExtendVote` have determinism-related properties associated.
Indeed, `PrepareProposal` is not required to be deterministic:
@ -598,8 +581,8 @@ Likewise, `ExtendVote` can also be non-deterministic:
The following sections use these definitions:
* We define the _common case_ as a run in which (a) the system behaves synchronously, and (b) there are no Byzantine processes.
The common case captures the conditions that hold most of the time, but not always.
* We define the _optimal case_ as a run in which (a) the system behaves synchronously, and (b) there are no Byzantine processes.
The optimal case captures the conditions that hold most of the time, but not always.
* We define the _suboptimal case_ as a run in which (a) the system behaves asynchronously in round 0, height h -- messages may be delayed,
timeouts may be triggered --, (b) it behaves synchronously in all subsequent rounds (_r>0_), and (c) there are no Byzantine processes.
The _suboptimal case_ captures a possible glitch in the network, or some sudden, sporadic performance issue in some validator
@ -612,7 +595,7 @@ The following sections use these definitions:
Given a block height _h_, process _p_'s Tendermint calls `RequestProcessProposal` depending on the case:
* In the common case, all Tendermint processes decide in round _r=0_. Let's call _v_ the block proposed by the proposer of round _r=0_, height _h_.
* In the optimal case, all Tendermint processes decide in round _r=0_. Let's call _v_ the block proposed by the proposer of round _r=0_, height _h_.
Process _p_'s Application will receive
* exactly one call to `RequestPrepareProposal` if _p_ is the proposer of round 0. If that is the case, _p_ will return _v_ in its call to
`ResponsePrepareProposal`
@ -655,26 +638,7 @@ The validity of every transaction in a block (from the App's point of view), as
## Failure modes
>**TODO**: this section is just holding parts we have moved from previous sections referring to the "continue" failure mode. This needs to be reworked.
>**TODO**: don't forget to extend the Header structure with "bool problem" when including the parts
common with the existing ABCI spec.
* The [Header](../core/data_structures.md#header) contains a boolan `problem` that indicates
if the block is the result of an Application rejection (i.e., $\bot$). This allows the
users to distinguish empty blocks because of lack of activity and empty blocks representing
$\bot$, i.e., a problem in the code of `PrepareProposal` and/or `ProcessProposal`.
In order to address this problem, we introduce a new mechanism: the $\bot$ block, which is an _empty_ block
which, if decided at a height, means too many processes are rejecting proposed values.
Operators, upon seeing $\bot$ blocks being committed, know instantly that there is a problem in the Application's
software and they can tackle it, while keeping Tendermint from getting stuck at a particular height
with high rounds and long timeouts.
The $\bot$ block is used as follows. A correct process $p$ will prevote $\bot$ (rather than `nil`)
if $p$'s `ProcessProposal` implementation rejects the block passed in `RequestProcessProposal`.
>**TODO** Is it worth explaining the failure modes? Since we're going for halt, and can't configure them.
Sergio Mena
3 years ago