zolfa
/
tendermint
1
0
Fork 0
Code Issues 0 Pull Requests 0 Projects 0 Releases 221 Wiki Activity
Browse Source

Merge a052819f9e into e4ae922c33

pull/8191/merge
Sergio Mena 3 years ago
committed by GitHub
parent
e4ae922c33 a052819f9e
commit
b012517325
No known key found for this signature in database GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 53 additions and 35 deletions
Unified View
Diff Options
Show Stats Download Patch File Download Diff File
  1. +34
    -27
      spec/abci++/abci++_app_requirements_002_draft.md
  2. +2
    -2
      spec/abci++/abci++_basic_concepts_002_draft.md
  3. +13
    -5
      spec/abci++/abci++_methods_002_draft.md
  4. +4
    -1
      spec/abci++/abci++_tmint_expected_behavior_002_draft.md

+ 34
- 27
spec/abci++/abci++_app_requirements_002_draft.md View File

@ -46,33 +46,43 @@ Full execution of blocks at `PrepareProposal` time stands on Tendermint's critic
Requirement 3 ensures the Application will set a value for _TimeoutPropose_ such that the time it takes Requirement 3 ensures the Application will set a value for _TimeoutPropose_ such that the time it takes
to fully execute blocks in `PrepareProposal` does not interfere with Tendermint's propose timer. to fully execute blocks in `PrepareProposal` does not interfere with Tendermint's propose timer.
* Requirement 4 [`PrepareProposal`, `ProcessProposal`, coherence]: For any two correct processes $p$ and $q$,
* Requirement 4 [`PrepareProposal`, tx-size] When $p$'s Application calls `ResponsePrepareProposal`, the
total size in bytes of the transactions returned does not exceed `RequestPrepareProposal.max_tx_bytes`.
Busy blockchains might seek to maximize the amount of transactions included in each block. Under those conditions,
Tendermint might choose to increase the transactions passed to the Application via `RequestPrepareProposal.txs`
beyond the `RequestPrepareProposal.max_tx_bytes` limit. The idea is that, if the Application drops some of
those transactions, it can still return a transaction list whose byte size is as close to
`RequestPrepareProposal.max_tx_bytes` as possible. Thus, Requirement 4 ensures that the size in bytes of the
transaction list returned by the application will never cause the resulting block to go beyond its byte limit.
* Requirement 5 [`PrepareProposal`, `ProcessProposal`, coherence]: For any two correct processes $p$ and $q$,
if $q$'s Tendermint calls `RequestProcessProposal` on $v'_p$, if $q$'s Tendermint calls `RequestProcessProposal` on $v'_p$,
$q$'s Application returns Accept in `ResponseProcessProposal`. $q$'s Application returns Accept in `ResponseProcessProposal`.
Requirement 4 makes sure that blocks proposed by correct processes _always_ pass the correct receiving process's
Requirement 5 makes sure that blocks proposed by correct processes _always_ pass the correct receiving process's
`ProcessProposal` check. `ProcessProposal` check.
On the other hand, if there is a deterministic bug in `PrepareProposal` or `ProcessProposal` (or in both), On the other hand, if there is a deterministic bug in `PrepareProposal` or `ProcessProposal` (or in both),
strictly speaking, this makes all processes that hit the bug byzantine. This is a problem in practice, strictly speaking, this makes all processes that hit the bug byzantine. This is a problem in practice,
as very often validators are running the Application from the same codebase, so potentially _all_ would as very often validators are running the Application from the same codebase, so potentially _all_ would
likely hit the bug at the same time. This would result in most (or all) processes prevoting `nil`, with the likely hit the bug at the same time. This would result in most (or all) processes prevoting `nil`, with the
serious consequences on Tendermint's liveness that this entails. Due to its criticality, Requirement 4 is a
serious consequences on Tendermint's liveness that this entails. Due to its criticality, Requirement 5 is a
target for extensive testing and automated verification. target for extensive testing and automated verification.
* Requirement 5 [`ProcessProposal`, determinism-1]: `ProcessProposal` is a (deterministic) function of the current
* Requirement 6 [`ProcessProposal`, determinism-1]: `ProcessProposal` is a (deterministic) function of the current
state and the block that is about to be applied. In other words, for any correct process $p$, and any arbitrary block $v'$, state and the block that is about to be applied. In other words, for any correct process $p$, and any arbitrary block $v'$,
if $p$'s Tendermint calls `RequestProcessProposal` on $v'$ at height $h$, if $p$'s Tendermint calls `RequestProcessProposal` on $v'$ at height $h$,
then $p$'s Application's acceptance or rejection **exclusively** depends on $v'$ and $s_{p,h-1}$. then $p$'s Application's acceptance or rejection **exclusively** depends on $v'$ and $s_{p,h-1}$.
* Requirement 6 [`ProcessProposal`, determinism-2]: For any two correct processes $p$ and $q$, and any arbitrary block $v'$,
* Requirement 7 [`ProcessProposal`, determinism-2]: For any two correct processes $p$ and $q$, and any arbitrary block $v'$,
if $p$'s (resp. $q$'s) Tendermint calls `RequestProcessProposal` on $v'$ at height $h$, if $p$'s (resp. $q$'s) Tendermint calls `RequestProcessProposal` on $v'$ at height $h$,
then $p$'s Application accepts $v'$ if and only if $q$'s Application accepts $v'$. then $p$'s Application accepts $v'$ if and only if $q$'s Application accepts $v'$.
Note that this requirement follows from Requirement 5 and the Agreement property of consensus.
Note that this requirement follows from Requirement 6 and the Agreement property of consensus.
Requirements 5 and 6 ensure that all correct processes will react in the same way to a proposed block, even
Requirements 6 and 7 ensure that all correct processes will react in the same way to a proposed block, even
if the proposer is Byzantine. However, `ProcessProposal` may contain a bug that renders the if the proposer is Byzantine. However, `ProcessProposal` may contain a bug that renders the
acceptance or rejection of the block non-deterministic, and therefore prevents processes hitting acceptance or rejection of the block non-deterministic, and therefore prevents processes hitting
the bug from fulfilling Requirements 5 or 6 (effectively making those processes Byzantine).
the bug from fulfilling Requirements 6 or 7 (effectively making those processes Byzantine).
In such a scenario, Tendermint's liveness cannot be guaranteed. In such a scenario, Tendermint's liveness cannot be guaranteed.
Again, this is a problem in practice if most validators are running the same software, as they are likely Again, this is a problem in practice if most validators are running the same software, as they are likely
to hit the bug at the same point. There is currently no clear solution to help with this situation, so to hit the bug at the same point. There is currently no clear solution to help with this situation, so
@ -85,43 +95,43 @@ is about to broadcast a non-`nil` precommit message, a correct process can only
Let $e^r_p$ be the vote extension that the Application of a correct process $p$ returns via `ResponseExtendVote` in round $r$, height $h$. Let $e^r_p$ be the vote extension that the Application of a correct process $p$ returns via `ResponseExtendVote` in round $r$, height $h$.
Let $w^r_p$ be the proposed block that $p$'s Tendermint passes to the Application via `RequestExtendVote` in round $r$, height $h$. Let $w^r_p$ be the proposed block that $p$'s Tendermint passes to the Application via `RequestExtendVote` in round $r$, height $h$.
* Requirement 7 [`ExtendVote`, `VerifyVoteExtension`, coherence]: For any two correct processes $p$ and $q$, if $q$ receives $e^r_p$
* Requirement 8 [`ExtendVote`, `VerifyVoteExtension`, coherence]: For any two correct processes $p$ and $q$, if $q$ receives $e^r_p$
from $p$ in height $h$, $q$'s Application returns Accept in `ResponseVerifyVoteExtension`. from $p$ in height $h$, $q$'s Application returns Accept in `ResponseVerifyVoteExtension`.
Requirement 7 constrains the creation and handling of vote extensions in a similar way as Requirement 4
Requirement 8 constrains the creation and handling of vote extensions in a similar way as Requirement 5
contrains the creation and handling of proposed blocks. contrains the creation and handling of proposed blocks.
Requirement 7 ensures that extensions created by correct processes _always_ pass the `VerifyVoteExtension`
Requirement 8 ensures that extensions created by correct processes _always_ pass the `VerifyVoteExtension`
checks performed by correct processes receiving those extensions. checks performed by correct processes receiving those extensions.
However, if there is a (deterministic) bug in `ExtendVote` or `VerifyVoteExtension` (or in both), However, if there is a (deterministic) bug in `ExtendVote` or `VerifyVoteExtension` (or in both),
we will face the same liveness issues as described for Requirement 4, as Precommit messages with invalid vote
we will face the same liveness issues as described for Requirement 5, as Precommit messages with invalid vote
extensions will be discarded. extensions will be discarded.
* Requirement 8 [`VerifyVoteExtension`, determinism-1]: `VerifyVoteExtension` is a (deterministic) function of
* Requirement 9 [`VerifyVoteExtension`, determinism-1]: `VerifyVoteExtension` is a (deterministic) function of
the current state, the vote extension received, and the prepared proposal that the extension refers to. the current state, the vote extension received, and the prepared proposal that the extension refers to.
In other words, for any correct process $p$, and any arbitrary vote extension $e$, and any arbitrary In other words, for any correct process $p$, and any arbitrary vote extension $e$, and any arbitrary
block $w$, if $p$'s (resp. $q$'s) Tendermint calls `RequestVerifyVoteExtension` on $e$ and $w$ at height $h$, block $w$, if $p$'s (resp. $q$'s) Tendermint calls `RequestVerifyVoteExtension` on $e$ and $w$ at height $h$,
then $p$'s Application's acceptance or rejection **exclusively** depends on $e$, $w$ and $s_{p,h-1}$. then $p$'s Application's acceptance or rejection **exclusively** depends on $e$, $w$ and $s_{p,h-1}$.
* Requirement 9 [`VerifyVoteExtension`, determinism-2]: For any two correct processes $p$ and $q$,
* Requirement 10 [`VerifyVoteExtension`, determinism-2]: For any two correct processes $p$ and $q$,
and any arbitrary vote extension $e$, and any arbitrary block $w$, and any arbitrary vote extension $e$, and any arbitrary block $w$,
if $p$'s (resp. $q$'s) Tendermint calls `RequestVerifyVoteExtension` on $e$ and $w$ at height $h$, if $p$'s (resp. $q$'s) Tendermint calls `RequestVerifyVoteExtension` on $e$ and $w$ at height $h$,
then $p$'s Application accepts $e$ if and only if $q$'s Application accepts $e$. then $p$'s Application accepts $e$ if and only if $q$'s Application accepts $e$.
Note that this requirement follows from Requirement 8 and the Agreement property of consensus.
Note that this requirement follows from Requirement 9 and the Agreement property of consensus.
Requirements 8 and 9 ensure that the validation of vote extensions will be deterministic at all
Requirements 9 and 10 ensure that the validation of vote extensions will be deterministic at all
correct processes. correct processes.
Requirements 8 and 9 protect against arbitrary vote extension data from Byzantine processes
similarly to Requirements 5 and 6 and proposed blocks.
Requirements 8 and 9 can be violated by a bug inducing non-determinism in
Requirements 9 and 10 protect against arbitrary vote extension data from Byzantine processes
similarly to Requirements 6 and 7 and proposed blocks.
Requirements 9 and 10 can be violated by a bug inducing non-determinism in
`VerifyVoteExtension`. In this case liveness can be compromised. `VerifyVoteExtension`. In this case liveness can be compromised.
Extra care should be put in the implementation of `ExtendVote` and `VerifyVoteExtension` and, Extra care should be put in the implementation of `ExtendVote` and `VerifyVoteExtension` and,
as a general rule, `VerifyVoteExtension` _should_ always accept the vote extension. as a general rule, `VerifyVoteExtension` _should_ always accept the vote extension.
* Requirement 10 [_all_, no-side-effects]: $p$'s calls to `RequestPrepareProposal`,
* Requirement 11 [_all_, no-side-effects]: $p$'s calls to `RequestPrepareProposal`,
`RequestProcessProposal`, `RequestExtendVote`, and `RequestVerifyVoteExtension` at height $h$ do `RequestProcessProposal`, `RequestExtendVote`, and `RequestVerifyVoteExtension` at height $h$ do
not modify $s_{p,h-1}$. not modify $s_{p,h-1}$.
* Requirement 11 [`ExtendVote`, `FinalizeBlock`, non-dependency]: for any correct process $p$,
* Requirement 12 [`ExtendVote`, `FinalizeBlock`, non-dependency]: for any correct process $p$,
and any vote extension $e$ that $p$ received at height $h$, the computation of and any vote extension $e$ that $p$ received at height $h$, the computation of
$s_{p,h}$ does not depend on $e$. $s_{p,h}$ does not depend on $e$.
@ -133,16 +143,13 @@ Additionally,
* in same-block execution mode, $p$'s `PrepareProposal` creates a set of transaction results $T_{p,h}$ * in same-block execution mode, $p$'s `PrepareProposal` creates a set of transaction results $T_{p,h}$
if $p$ was the proposer of $v_{p,h}$, otherwise `FinalizeBlock` creates $T_{p,h}$. if $p$ was the proposer of $v_{p,h}$, otherwise `FinalizeBlock` creates $T_{p,h}$.
>**TODO** I have left out all the "events" as they don't have any impact in safety or liveness
>(same for consensus params, and validator set)
* Requirement 12 [`FinalizeBlock`, determinism-1]: For any correct process $p$,
* Requirement 13 [`FinalizeBlock`, determinism-1]: For any correct process $p$,
$s_{p,h}$ exclusively depends on $s_{p,h-1}$ and $v_{p,h}$. $s_{p,h}$ exclusively depends on $s_{p,h-1}$ and $v_{p,h}$.
* Requirement 13 [`FinalizeBlock`, determinism-2]: For any correct process $p$,
* Requirement 14 [`FinalizeBlock`, determinism-2]: For any correct process $p$,
the contents of $T_{p,h}$ exclusively depend on $s_{p,h-1}$ and $v_{p,h}$. the contents of $T_{p,h}$ exclusively depend on $s_{p,h-1}$ and $v_{p,h}$.
Note that Requirements 12 and 13, combined with Agreement property of consensus ensure
Note that Requirements 13 and 14, combined with Agreement property of consensus ensure
the Application state evolves consistently at all correct processes. the Application state evolves consistently at all correct processes.
Finally, notice that neither `PrepareProposal` nor `ExtendVote` have determinism-related Finally, notice that neither `PrepareProposal` nor `ExtendVote` have determinism-related


+ 2
- 2
spec/abci++/abci++_basic_concepts_002_draft.md View File

@ -65,8 +65,8 @@ ignore the invalid part of the prepared proposal at block execution time.
The data, called _vote extension_, will also be made available to the The data, called _vote extension_, will also be made available to the
application in the next height, along with the vote it is extending, in the rounds application in the next height, along with the vote it is extending, in the rounds
where the local process is the proposer. where the local process is the proposer.
The Application may also choose not to include any vote extension.
Tendermint calls it when is about to send a non-`nil` precommit message.
If the Application does not have vote extension information to provide, it returns a 0-length byte array as its vote extension.
Tendermint calls `ExtendVote` when is about to send a non-`nil` precommit message.
* [**VerifyVoteExtension:**](./abci++_methods_002_draft.md#verifyvoteextension) It allows validators to validate the vote extension data attached to a precommit message. If the validation fails, the precommit message will be deemed invalid and ignored * [**VerifyVoteExtension:**](./abci++_methods_002_draft.md#verifyvoteextension) It allows validators to validate the vote extension data attached to a precommit message. If the validation fails, the precommit message will be deemed invalid and ignored
by Tendermint. This has a negative impact on Tendermint's liveness, i.e., if vote extensions repeatedly cannot be verified by correct validators, Tendermint may not be able to finalize a block even if sufficiently many (+2/3) of the validators send precommit votes for that block. Thus, `VerifyVoteExtension` should be used with special care. by Tendermint. This has a negative impact on Tendermint's liveness, i.e., if vote extensions repeatedly cannot be verified by correct validators, Tendermint may not be able to finalize a block even if sufficiently many (+2/3) of the validators send precommit votes for that block. Thus, `VerifyVoteExtension` should be used with special care.


+ 13
- 5
spec/abci++/abci++_methods_002_draft.md View File

@ -318,8 +318,11 @@ title: Methods
* If the Application wants to add a new transaction, then the Application should include it in `tx_records` and _mark_ it as `ADD`. In this case, Tendermint will add it to the mempool. * If the Application wants to add a new transaction, then the Application should include it in `tx_records` and _mark_ it as `ADD`. In this case, Tendermint will add it to the mempool.
* The Application should be aware that removing and adding transactions may compromise _traceability_. * The Application should be aware that removing and adding transactions may compromise _traceability_.
> Consider the following example: the Application transforms a client-submitted transaction `t1` into a second transaction `t2`, i.e., the Application asks Tendermint to remove `t1` and add `t2` to the mempool. If a client wants to eventually check what happened to `t1`, it will discover that `t_1` is not in the mempool or in a committed block, getting the wrong idea that `t_1` did not make it into a block. Note that `t_2` _will be_ in a committed block, but unless the Application tracks this information, no component will be aware of it. Thus, if the Application wants traceability, it is its responsability to support it. For instance, the Application could attach to a transformed transaction a list with the hashes of the transactions it derives from. > Consider the following example: the Application transforms a client-submitted transaction `t1` into a second transaction `t2`, i.e., the Application asks Tendermint to remove `t1` and add `t2` to the mempool. If a client wants to eventually check what happened to `t1`, it will discover that `t_1` is not in the mempool or in a committed block, getting the wrong idea that `t_1` did not make it into a block. Note that `t_2` _will be_ in a committed block, but unless the Application tracks this information, no component will be aware of it. Thus, if the Application wants traceability, it is its responsability to support it. For instance, the Application could attach to a transformed transaction a list with the hashes of the transactions it derives from.
* If the Application modifies the set of transactions, the modified transactions MUST NOT exceed the configured maximum size `RequestPrepareProposal.max_tx_bytes`.
* If the Application does not modify the preliminary set of transactions `txs`, then it sets `ResponsePrepareProposal.modified_tx_status` to `UNMODIFIED`. In this case, Tendermint will ignore the contents of `ResponsePrepareProposal.tx_records`. * If the Application does not modify the preliminary set of transactions `txs`, then it sets `ResponsePrepareProposal.modified_tx_status` to `UNMODIFIED`. In this case, Tendermint will ignore the contents of `ResponsePrepareProposal.tx_records`.
* In order to optimize the block size in busy blockchains, Tendermint might include a list of transactions in `RequestPrepareProposal.txs` whose size
in bytes exceeds `RequestPrepareProposal.max_tx_bytes`. In this case:
* The Application MUST still abide by the `RequestPrepareProposal.max_tx_bytes` limit in the list of transaction records it returns (excluding those marked as `REMOVE`).
* It is _invalid_ to return `ResponsePrepareProposal.modified_tx_status` set to `UNMODIFIED`: Tendermint will panic.
* In same-block execution mode, the Application must provide values for `ResponsePrepareProposal.app_hash`, * In same-block execution mode, the Application must provide values for `ResponsePrepareProposal.app_hash`,
`ResponsePrepareProposal.tx_results`, `ResponsePrepareProposal.validator_updates`, and `ResponsePrepareProposal.tx_results`, `ResponsePrepareProposal.validator_updates`, and
`ResponsePrepareProposal.consensus_param_updates`, as a result of fully executing the block. `ResponsePrepareProposal.consensus_param_updates`, as a result of fully executing the block.
@ -529,7 +532,7 @@ a [CanonicalVoteExtension](#canonicalvoteextension) field in the `precommit nil`
| hash | bytes | The header hash of the propsed block that the vote extension refers to. | 1 | | hash | bytes | The header hash of the propsed block that the vote extension refers to. | 1 |
| validator_address | bytes | [Address](../core/data_structures.md#address) of the validator that signed the extension | 2 | | validator_address | bytes | [Address](../core/data_structures.md#address) of the validator that signed the extension | 2 |
| height | int64 | Height of the block (for sanity check). | 3 | | height | int64 | Height of the block (for sanity check). | 3 |
| vote_extension | bytes | Optional information signed by Tendermint. | 4 |
| vote_extension | bytes | Application-specific information signed by Tendermint. Can have 0 length | 4 |
* **Response**: * **Response**:
@ -538,6 +541,9 @@ a [CanonicalVoteExtension](#canonicalvoteextension) field in the `precommit nil`
| status | [VerifyStatus](#VerifyStatus) | `enum` signaling if the application accepts the vote extension | 1 | | status | [VerifyStatus](#VerifyStatus) | `enum` signaling if the application accepts the vote extension | 1 |
* **Usage**: * **Usage**:
* `RequestVerifyVoteExtension.vote_extension` can be an empty byte array. The Application's interpretation of it should be
that the Application running at the process that sent the vote chose not to extend it.
Tendermint will always call `RequestVerifyVoteExtension`, even for 0 length vote extensions.
* If `ResponseVerifyVoteExtension.status` is `REJECT`, Tendermint will reject the whole received vote. * If `ResponseVerifyVoteExtension.status` is `REJECT`, Tendermint will reject the whole received vote.
See the [Requirements](abci++_app_requirements_002_draft.md) section to understand the potential See the [Requirements](abci++_app_requirements_002_draft.md) section to understand the potential
liveness implications of this. liveness implications of this.
@ -553,9 +559,11 @@ a [CanonicalVoteExtension](#canonicalvoteextension) field in the `precommit nil`
When a validator _p_ is in Tendermint consensus round _r_, height _h_, state _prevote_ (**TODO** discuss: I think I must remove the state When a validator _p_ is in Tendermint consensus round _r_, height _h_, state _prevote_ (**TODO** discuss: I think I must remove the state
from this condition, but not sure), and _p_ receives a Precommit message for round _r_, height _h_ from _q_: from this condition, but not sure), and _p_ receives a Precommit message for round _r_, height _h_ from _q_:
1. _p_'s Tendermint calls `RequestVerifyVoteExtension`.
2. The Application returns _accept_ or _reject_ via `ResponseVerifyVoteExtension.status`.
3. If the Application returns
1. If the Precommit message does not contain a vote extension with a valid signature, Tendermint discards the message as invalid.
* a 0-length vote extension is valid as long as its accompanying signature is also valid.
2. Else, _p_'s Tendermint calls `RequestVerifyVoteExtension`.
3. The Application returns _accept_ or _reject_ via `ResponseVerifyVoteExtension.status`.
4. If the Application returns
* _accept_, _p_'s Tendermint will keep the received vote, together with its corresponding * _accept_, _p_'s Tendermint will keep the received vote, together with its corresponding
vote extension in its internal data structures. It will be used to populate the [ExtendedCommitInfo](#extendedcommitinfo) vote extension in its internal data structures. It will be used to populate the [ExtendedCommitInfo](#extendedcommitinfo)
structure in calls to `RequestPrepareProposal`, in rounds of height _h + 1_ where _p_ is the proposer. structure in calls to `RequestPrepareProposal`, in rounds of height _h + 1_ where _p_ is the proposer.


+ 4
- 1
spec/abci++/abci++_tmint_expected_behavior_002_draft.md View File

@ -202,7 +202,10 @@ to undergo any changes in their implementation.
As for the new methods: As for the new methods:
* `PrepareProposal` should set `ResponsePrepareProposal.modified_tx` to _false_ and return.
* `PrepareProposal` should check whether the size of transactions exceeds the byte limit.
* If it does: remove transactions at the end of the list until the total byte size conforms to the limit,
then set `ResponsePrepareProposal.modified_tx_status` to `MODIFIED` and return.
* Else, set `ResponsePrepareProposal.modified_tx_status` to `UNMODIFIED` and return.
* `ProcessProposal` should set `ResponseProcessProposal.accept` to _true_ and return. * `ProcessProposal` should set `ResponseProcessProposal.accept` to _true_ and return.
* `ExtendVote` should set `ResponseExtendVote.extension` to an empty byte array and return. * `ExtendVote` should set `ResponseExtendVote.extension` to an empty byte array and return.
* `VerifyVoteExtension` should set `ResponseVerifyVoteExtension.accept` to _true_ if the extension is an empty byte array * `VerifyVoteExtension` should set `ResponseVerifyVoteExtension.accept` to _true_ if the extension is an empty byte array


Write Preview
Loading…
Cancel
Save