zolfa
/
tendermint
1
0
Fork 0
Code Issues 0 Pull Requests 0 Projects 0 Releases 221 Wiki Activity
Browse Source

ABCI++: Major refactor of spec's structure. Addressed Josef's comments. Merged ABCI's methods and data structs that didn't change. Added introductory paragraphs

pull/7804/head
Sergio Mena 2 years ago
parent
57cc810744
commit
523974167a
8 changed files with 1558 additions and 721 deletions
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. +5
    -0
      README.md
  2. +2
    -0
      spec/README.md
  3. +44
    -0
      spec/abci++/README.md
  4. +136
    -0
      spec/abci++/abci++_app_requirements_002_draft.md
  5. +372
    -0
      spec/abci++/abci++_basic_concepts_002_draft.md
  6. +815
    -0
      spec/abci++/abci++_methods_002_draft.md
  7. +0
    -721
      spec/abci++/abci++_properties_001_draft.md
  8. +184
    -0
      spec/abci++/abci++_tmint_expected_behavior_002_draft.md

+ 5
- 0
README.md View File

@ -39,6 +39,11 @@ maintained by Informal Systems.
- [ABCI](./spec/abci/README.md): Details about interactions between the
application and consensus engine over ABCI
### ABCI++
- [ABCI++](./spec/abci++/README.md): Specification of interactions between the
application and consensus engine over ABCI++
### RFC
- [RFC](./rfc/README.md): RFCs describe proposals to change the spec.


+ 2
- 0
spec/README.md View File

@ -50,6 +50,8 @@ please submit them to our [bug bounty](https://tendermint.com/security)!
- [ABCI](./abci/README.md): Details about interactions between the
application and consensus engine over ABCI
- [ABCI++](./abci++/README.md): Specification of interactions between the
application and consensus engine over ABCI++
- [Write-Ahead Log](./consensus/wal.md): Details about how the consensus
engine preserves data and recovers from crash failures


+ 44
- 0
spec/abci++/README.md View File

@ -0,0 +1,44 @@
---
order: 1
parent:
title: ABCI++
order: 3
---
# ABCI++
## Introduction
ABCI++ is a major evolution of ABCI (**A**pplication **B**lock**c**hain **I**nterface).
Like its predecessor, ABCI++ is the interface between Tendermint (a state-machine
replication engine) and the actual state machine being replicated (i.e., the Application).
The API consists of a set of _methods_, each with a corresponding `Request` and `Response`
message type.
The methods are always initiated by Tendermint. The Application implements its logic
for handling all ABCI++ methods.
Thus, Tendermint always sends the `Request*` messages and receives the `Response*` messages
in return.
All ABCI++ messages and methods are defined in
[protocol buffers](https://github.com/tendermint/spec/blob/master/proto/tendermint/abci/types.proto).
This allows Tendermint to run with applications written in many programming languages.
This specification is split as follows:
- [Basic concepts and definitions](./abci++_basic_concepts_002_draft.md) - definitions and descriptions
of concepts that are needed to understand other parts of this sepcification.
- [Methods](./abci++_methods_002_draft.md) - complete details on all ABCI++ methods
and message types.
- [Requirements for the Application](./abci++_app_requirements_002_draft.md) - formal requirements
on the Application's logic to ensure liveness of Tendermint. These requirements define what
Tendermint expects from the Application.
- [Tendermint's expected behavior](./abci++_tmint_expected_behavior_002_draft.md) - specification of
how the different ABCI++ methods may be called by Tendermint. This explains what the Application
is to expect from Tendermint.
>**TODO** Re-read these and remove redundant in fo
- [Applications](../abci/apps.md) - how to manage ABCI application state and other
details about building ABCI applications
- [Client and Server](../abci/client-server.md) - for those looking to implement their
own ABCI application servers

+ 136
- 0
spec/abci++/abci++_app_requirements_002_draft.md View File

@ -0,0 +1,136 @@
---
order: 3
title: Application Requirements
---
# Application Requirements
This section specifies what Tendermint expects from the Application. It is structured as a set
of formal requirement that can be used for testing and verification of the Application's logic.
Let $p$ and $q$ be two different correct proposers in rounds $r_p$ and $r_q$ respectively, in height $h$.
Let $s_{p,h-1}$ be $p$'s Application's state committed for height $h-1$.
Let $v_p$ (resp. $v_q$) be the block that $p$'s (resp. $q$'s) Tendermint passes on to the Application
via `RequestPrepareProposal` as proposer of round $r_p$ (resp $r_q$), height $h$, also known as the
raw proposal.
Let $v'_p$ (resp. $v'_q$) the possibly modified block $p$'s (resp. $q$'s) Application returns via
`ResponsePrepareProposal` to Tendermint, also known as the prepared proposal.
Process $p$'s prepared proposal can differ in two different rounds where $p$ is the proposer.
* Requirement 1 [`PrepareProposal`, header-changes] When the blockchain is in same-block execution mode,
$p$'s Application provides values for the following parameters in `ResponsePrepareProposal`:
_AppHash_, _TxResults_, _ConsensusParams_, _ValidatorUpdates_.
Parameters _AppHash_, _TxResults_, _ConsensusParams_, and _ValidatorUpdates_ are used by Tendermint to
compute various hashes in the block header that will finally be part of the proposal.
* Requirement 2 [`PrepareProposal`, no-header-changes] When the blockchain is in next-block execution
mode, $p$'s Application does not provides values for the following parameters in `ResponsePrepareProposal`:
_AppHash_, _TxResults_, _ConsensusParams_, _ValidatorUpdates_.
In practical terms, Requirement 2 implies that Tendermint will ignore those parameters in `ResponsePrepareProposal`.
* Requirement 3 [`PrepareProposal`, `ProcessProposal`, coherence]: For any two correct processes $p$ and $q$,
if $q$'s Tendermint calls `RequestProcessProposal` on $v'_p$,
$q$'s Application returns Accept in `ResponseProcessProposal`.
Requirement 3 makes sure that blocks proposed by correct processes _always_ pass the correct receiving process's
`ProcessProposal` check.
On the other hand, if there is a deterministic bug in `PrepareProposal` or `ProcessProposal` (or in both),
strictly speaking, this makes all processes that hit the bug byzantine. This is a problem in practice,
as very often validators are running the Application from the same codebase, so potentially _all_ would
likely hit the bug at the same time. This would result in most (or all) processes prevoting `nil`, with the
serious consequences on Tendermint's liveness that this entails.
* Requirement 4 [`ProcessProposal`, determinism-1]: For any correct process $p$, and any arbitrary block $v'$,
if $p$'s Tendermint calls `RequestProcessProposal` on $v'$ at height $h$,
then $p$'s Application's acceptance or rejection exclusively depends on $v'$ and $s_{p,h-1}$.
* Requirement 5 [`ProcessProposal`, determinism-2]: For any two correct processes $p$ and $q$, and any arbitrary block $v'$,
if $p$'s (resp. $q$'s) Tendermint calls `RequestProcessProposal` on $v'$ at height $h$,
then $p$'s Application accepts $v'$ if and only if $q$'s Application accepts $v'$.
Note that this requirement follows from Requirement 4 and the Agreement property of consensus.
Requirements 4 and 5 ensure that all correct processes will react in the same way to a proposed block, even
if the proposer is Byzantine. However, `ProcessProposal` may contain a bug that renders the
acceptance or rejection of the block non-deterministic, and therefore prevents processes hitting
the bug from fulfilling Requirements 4 or 5 (effectively making those processes Byzantine).
In such a scenario, Tendermint's liveness cannot be guaranteed.
Again, this is a problem in practice if most validators are running the same software, as they are likely
to hit the bug at the same point. There is currently no clear solution to help with this situation, so
the Application designers/implementors must proceed very carefully with the logic/implementation
of `ProcessProposal`. As a general rule `ProcessProposal` _should_ always accept the block.
According to the Tendermint algorithm, a correct process can only broadcast one precommit message in round $r$, height $h$.
Since, as stated in the [Description](#description) section, `ResponseExtendVote` is only called when Tendermint
is about to broadcast a non-`nil` precommit message, a correct process can only produce one vote extension in round $r$, height $h$.
Let $e^r_p$ the vote extension that the Application of a correct process $p$ returns via `ResponseExtendVote` in round $r$, height $h$.
Let $w^r_p$ the proposed block that $p$'s Tendermint passes to the Application via `RequestExtendVote` in round $r$, height $h$.
* Requirement 6 [`ExtendVote`, `VerifyVoteExtension`, coherence]: For any two correct processes $p$ and $q$, if $q$ receives $e^r_p$
from $p$ in height $h$, $q$'s Application returns Accept in `ResponseVerifyVoteExtension`.
Requirement 6 constrains the creation and handling of vote extensions in a similar way as Requirement 3
contrains the creation and handling of proposed blocks.
Requirement 6 ensures that extensions created by correct processes _always_ pass the `VerifyVoteExtension`
checks performed by correct processes receiving those extensions.
However, if there is a (deterministic) bug in `ExtendVote` or `VerifyVoteExtension` (or in both),
we will face the same liveness issues as described for Requirement 3, as Precommit messages with invalid vote
extensions will be discarded.
* Requirement 7 [`VerifyVoteExtension`, determinism-1]: For any correct process $p$,
and any arbitrary vote extension $e$, and any arbitrary block $w$,
if $p$'s (resp. $q$'s) Tendermint calls `RequestVerifyVoteExtension` on $e$ and $w$ at height $h$,
then $p$'s Application's acceptance or rejection exclusively depends on $e$, $w$ and $s_{p,h-1}$.
* Requirement 8 [`VerifyVoteExtension`, determinism-2]: For any two correct processes $p$ and $q$,
and any arbitrary vote extension $e$, and any arbitrary block $w$,
if $p$'s (resp. $q$'s) Tendermint calls `RequestVerifyVoteExtension` on $e$ and $w$ at height $h$,
then $p$'s Application accepts $e$ if and only if $q$'s Application accepts $e$.
Note that this requirement follows from Requirement 7 and the Agreement property of consensus.
Requirements 7 and 8 ensure that the validation of vote extensions will be deterministic at all
correct processes.
Requirements 7 and 8 protect against arbitrary vote extension data from Byzantine processes
similarly to Requirements 4 and 5 and proposed blocks.
Requirements 7 and 8 can be violated by a bug inducing non-determinism in `ExtendVote` or
`VerifyVoteExtension`. In this case liveness can be compromised.
Extra care should be put in the implementation of `ExtendVote` and `VerifyVoteExtension` and,
as a general rule, `VerifyVoteExtension` _should_ always accept the vote extension.
* Requirement 9 [_all_, no-side-effects]: $p$'s calls to `RequestPrepareProposal`,
`RequestProcessProposal`, `RequestExtendVote`, and `RequestVerifyVoteExtension` at height $h$ do
not modify $s_{p,h-1}$.
* Requirement 10 [`ExtendVote`, `FinalizeBlock`, non-dependency]: for any correct process $p$,
and any vote extension $e$ that $q$ received at height $h$, the computation of
$s{p,h}$ does not depend on $e$.
The call to correct process $p$'s `RequestFinalizeBlock` at height $h$, with block $v_{p,h}$
passed as parameter, creates state $s_{p,h}$.
Additionally, $p$'s `FinalizeBlock` creates a set of transaction results $T_{p,h}$.
>**TODO** I have left out all the "events" as they don't have any impact in safety or liveness
>(same for consensus params, and validator set)
* Requirement 11 [`FinalizeBlock`, determinism-1]: For any correct process $p$,
$s_{p,h}$ exclusively depends on $s_{p,h-1}$ and $v_{p,h}$.
* Requirement 12 [`FinalizeBlock`, determinism-2]: For any correct process $p$,
the contents of $T_{p,h}$ exclusively depend on $s_{p,h-1}$ and $v_{p,h}$.
Note that Requirements 11 and 12, combined with Agreement property of consensus ensure
the Application state evolves consistently at all correct processes.
Finally, notice that neither `PrepareProposal` nor `ExtendVote` have determinism-related
requirements associated.
Indeed, `PrepareProposal` is not required to be deterministic:
* $v'_p$ may depend on $v_p$ and $s_{p,h-1}$, but may also depend on other values or operations.
* $v_p = v_q \nRightarrow v'_p = v'_q$.
Likewise, `ExtendVote` can also be non-deterministic:
* $e^r_p$ may depend on $w^r_p$ and $s_{p,h-1}$, but may also depend on other values or operations.
* $w^r_p = w^r_q \nRightarrow e^r_p = e^r_q$

+ 372
- 0
spec/abci++/abci++_basic_concepts_002_draft.md View File

@ -0,0 +1,372 @@
---
order: 1
title: Basic concepts and definitions
---
# Basic concepts and definitions
## Connections
ABCI++ applications can run either within the _same_ process as the Tendermint
state-machine replication engine, or as a _separate_ process from the state-machine
replication engine. When run within the same process, Tendermint will call the ABCI++
application methods directly as Go method calls.
When Tendermint and the ABCI++ application are run as separate processes, Tendermint
opens four connections to the application for ABCI++ methods. The connections each
handle a subset of the ABCI++ method calls. These subsets are defined as follows:
#### **Consensus** connection
* Driven by a consensus protocol and is responsible for block execution.
* Handles the `InitChain`, `PrepareProposal`, `ProcessProposal`, `ExtendVote`,
`VerifyVoteExtension`, and `FinalizeBlock` method calls.
#### **Mempool** connection
* For validating new transactions, before they're shared or included in a block.
* Handles the `CheckTx` calls.
#### **Info** connection
* For initialization and for queries from the user.
* Handles the `Info` and `Query` calls.
#### **Snapshot** connection
* For serving and restoring [state sync snapshots](../abci/apps.md#state-sync).
* Handles the `ListSnapshots`, `LoadSnapshotChunk`, `OfferSnapshot`, and `ApplySnapshotChunk` calls.
Additionally, there is a `Flush` method that is called on every connection,
and an `Echo` method that is just for debugging.
>**TODO** Figure out what to do with this.
More details on managing state across connections can be found in the section on
[ABCI Applications](../abci/apps.md).
## Errors
The `Query`, and `CheckTx` methods include a `Code` field in their `Response*`.
The `Code` field is also included in type `DeliverTxResult`, used by
method `FinalizeBlock`'s `Response*`.
Field `Code` is meant to contain an application-specific response code.
A response code of `0` indicates no error. Any other response code
indicates to Tendermint that an error occurred.
These methods also return a `Codespace` string to Tendermint. This field is
used to disambiguate `Code` values returned by different domains of the
application. The `Codespace` is a namespace for the `Code`.
Methods `Echo`, `Info`, and `InitChain` do not return errors.
An error in any of these methods represents a critical issue that Tendermint
has no reasonable way to handle. If there is an error in one
of these methods, the application must crash to ensure that the error is safely
handled by an operator.
Method `FinalizeBlock` is a special case. It contains a number of
`Code` and `Codespace` fields as part of type `DeliverTxResult`. Each of
these codes reports errors related to the transaction it is attached to.
However, `FinalizeBlock` does not return errors at the top level, so the
same considerations on critical issues made for `Echo`, `Info`, and
`InitChain` also apply here.
The handling of non-zero response codes by Tendermint is described below
### `CheckTx`
The `CheckTx` ABCI++ method controls what transactions are considered for inclusion
in a block.
When Tendermint receives a `ResponseCheckTx` with a non-zero `Code`, the associated
transaction will not be added to Tendermint's mempool or it will be removed if
it is already included.
### `DeliverTxResult` (as part of `FinalizeBlock`)
The `DeliverTxResult` type delivers transactions from Tendermint to the application.
When Tendermint receives a `ReponseFinalizeBlock` containing a `DeliverTxResult`
with a non-zero `Code`, the response code is logged.
The transaction was already included in a block, so the `Code` does not influence
Tendermint consensus.
### `Query`
The `Query` ABCI++ method queries the application for information about application state.
When Tendermint receives a `ResponseQuery` with a non-zero `Code`, this code is
returned directly to the client that initiated the query.
## Events
Method `CheckTx` includes an `Events` field in its `Response*`.
Method `FinalizeBlock` includes an `Events` field at the top level in its
`Response*`, and one `tx_events` field per transaction included in the block.
Applications may respond to these ABCI++ methods with a set of events.
Events allow applications to associate metadata about ABCI++ method execution with the
transactions and blocks this metadata relates to.
Events returned via these ABCI++ methods do not impact Tendermint consensus in any way
and instead exist to power subscriptions and queries of Tendermint state.
An `Event` contains a `type` and a list of `EventAttributes`, which are key-value
string pairs denoting metadata about what happened during the method's (or transaction's)
execution. `Event` values can be used to index transactions and blocks according to what
happened during their execution.
Each event has a `type` which is meant to categorize the event for a particular
`Response*` or `Tx`. A `Response*` or `Tx` may contain multiple events with duplicate
`type` values, where each distinct entry is meant to categorize attributes for a
particular event. Every key and value in an event's attributes must be UTF-8
encoded strings along with the event type itself.
```protobuf
message Event {
string type = 1;
repeated EventAttribute attributes = 2;
}
```
The attributes of an `Event` consist of a `key`, a `value`, and an `index` flag. The
index flag notifies the Tendermint indexer to index the attribute. The value of
the `index` flag is non-deterministic and may vary across different nodes in the network.
```protobuf
message EventAttribute {
bytes key = 1;
bytes value = 2;
bool index = 3; // nondeterministic
}
```
Example:
```go
abci.ResponseCheckTx{
// ...
Events: []abci.Event{
{
Type: "validator.provisions",
Attributes: []abci.EventAttribute{
abci.EventAttribute{Key: []byte("address"), Value: []byte("..."), Index: true},
abci.EventAttribute{Key: []byte("amount"), Value: []byte("..."), Index: true},
abci.EventAttribute{Key: []byte("balance"), Value: []byte("..."), Index: true},
},
},
{
Type: "validator.provisions",
Attributes: []abci.EventAttribute{
abci.EventAttribute{Key: []byte("address"), Value: []byte("..."), Index: true},
abci.EventAttribute{Key: []byte("amount"), Value: []byte("..."), Index: false},
abci.EventAttribute{Key: []byte("balance"), Value: []byte("..."), Index: false},
},
},
{
Type: "validator.slashed",
Attributes: []abci.EventAttribute{
abci.EventAttribute{Key: []byte("address"), Value: []byte("..."), Index: false},
abci.EventAttribute{Key: []byte("amount"), Value: []byte("..."), Index: true},
abci.EventAttribute{Key: []byte("reason"), Value: []byte("..."), Index: true},
},
},
// ...
},
}
```
## EvidenceType
Tendermint's security model relies on the use of "evidence". Evidence is proof of
malicious behaviour by a network participant. It is the responsibility of Tendermint
to detect such malicious behaviour. When malicious behavior is detected, Tendermint
will gossip evidence of the behavior to other nodes and commit the evidence to
the chain once it is verified by all validators. This evidence will then be
passed on to the application through ABCI++. It is the responsibility of the
application to handle the evidence and exercise punishment.
EvidenceType has the following protobuf format:
```protobuf
enum EvidenceType {
UNKNOWN = 0;
DUPLICATE_VOTE = 1;
LIGHT_CLIENT_ATTACK = 2;
}
```
There are two forms of evidence: Duplicate Vote and Light Client Attack. More
information can be found in either [data structures](https://github.com/tendermint/spec/blob/master/spec/core/data_structures.md)
or [accountability](https://github.com/tendermint/spec/blob/master/spec/light-client/accountability/)
## Vote Extensions
According to the Tendermint algorithm, a proposed block needs at least a predefined
number of precommit votes in order to be decided. Tendermint gathers all the valid
precommit votes for the decided block that it receives before the block is decided,
and then includes these votes in the proposed block for the next height whenever
the local process is the proposer of the round.
When Tendermint's consensus is about to send a non-`nil` precommit message, it calls
method `ExtendVote`, which gives the Application the opportunity to include
non-deterministic data, opaque to Tendermint, that will be attached to the precommit
message. The data, called _vote extension_, will also be part of the proposed block
in the next height, along with the vote it is extending.
The vote extension data is split into two parts, one signed by Tendermint as part
of the vote data structure, and the other (optionally) signed by the application.
The Application may also choose not to include any vote extension.
When another process receives a precommit message with and vote extension, it calls
method `VerifyVoteExtension` so that the application can validate the data received.
If the validation fails, the precommit message will be deemed invalid and ignored
by Tendermint. This has strong implications on Tendermint's liveness.
As a general rule, an Application that detects an invalid vote extension SHOULD
accept it in `ResponseVerifyVoteExtension` and ignore it in its own logic.
## Determinism
ABCI++ applications must implement deterministic finite-state machines to be
securely replicated by the Tendermint consensus engine. This means block execution
over the Consensus Connection must be strictly deterministic: given the same
ordered set of requests, all nodes will compute identical responses, for all
successive `FinalizeBlock` calls. This is critical, because the
responses are included in the header of the next block, either via a Merkle root
or directly, so all nodes must agree on exactly what they are.
For this reason, it is recommended that applications not be exposed to any
external user or process except via the ABCI connections to a consensus engine
like Tendermint Core. The application must only change its state based on input
from block execution (`FinalizeBlock` calls), and not through
any other kind of request. This is the only way to ensure all nodes see the same
transactions and compute the same results.
Some Applications may choose to execute the blocks that are about to be proposed
(via `PrepareProposal`), or those that the application is asked to validate
(via `Processproposal`). However the state changes caused by processing those
proposed blocks must never replace the previous state until `FinalizeBlock` confirms
the block decided.
Additionally, vote extensions or the validation thereof (via `ExtendVote` or
`VerifyVoteExtension`) must _never_ have side effects on the current state.
They can only be used when their data is included in a block.
If there is some non-determinism in the state machine, consensus will eventually
fail as nodes disagree over the correct values for the block header. The
non-determinism must be fixed and the nodes restarted.
Sources of non-determinism in applications may include:
* Hardware failures
* Cosmic rays, overheating, etc.
* Node-dependent state
* Random numbers
* Time
* Underspecification
* Library version changes
* Race conditions
* Floating point numbers
* JSON serialization
* Iterating through hash-tables/maps/dictionaries
* External Sources
* Filesystem
* Network calls (eg. some external REST API service)
See [#56](https://github.com/tendermint/abci/issues/56) for original discussion.
Note that some methods (`Query, CheckTx, FinalizeBlock`) return
explicitly non-deterministic data in the form of `Info` and `Log` fields. The `Log` is
intended for the literal output from the application's logger, while the
`Info` is any additional info that should be returned. These are the only fields
that are not included in block header computations, so we don't need agreement
on them. All other fields in the `Response*` must be strictly deterministic.
## Block Execution
The first time a new blockchain is started, Tendermint calls
`InitChain`. From then on, method `FinalizeBlock` is executed at the end of each
block, resulting in an updated application state.
During consensus execution of a block height, before method `FinalizeBlock` is
called, methods `PrepareProposal`, `ProcessProposal`, `ExtendVote`, and
`VerifyVoteExtension` may be called a number of times.
See [Application Requirements](abci++_app_requirements_002_draft.md) for
details on the possible call sequences of these methods.
Method `PrepareProposal` is called every time Tendermint is about to send
a proposal message. Tendermint gathers outstanding transactions from the mempool
(see [PrepareProposal]()#PrepareProposal), generates a block header and uses
them to create a block to propose. Then, it calls `RequestPrepareProposal`
with the newly created proposal, called _raw proposal_. The Application can
make changes to the raw proposal, such as modifying transactions, and returns
the (potentially) modified proposal, called _prepared proposal_ in the
`Response*` call. The logic modifying the raw proposal can be non-deterministic.
When Tendermint receives a prepared proposal it uses method `ProcessProposal`
to inform the Application of the proposal just received. The Application cannot
modify the proposal at this point but can reject it if it realises it is invalid.
If that is the case, Tendermint will prevote `nil` on the proposal, which has
strong liveness implications for Tendermint. As a general rule, the Application
SHOULD accept a prepared proposal passed via `ProcessProposal`, even if a part of
the proposal is invalid (e.g., an invalid transaction); the Application can later
ignore the invalid part of the prepared proposal at block execution time.
Cryptographic commitments to the block and transaction results, via the corresponding
parameters in `FinalizeBlockResponse` are included in the header of the next block.
## Next-block execution and same-block execution
With ABCI++ predecessor, ABCI, the only moment when the Application had access to a
block was when it was decided. This led to a block execution model, called _next-block
execution_, where some fields in a block refer to the execution of the previous block,
namely:
* the merkle root of the Application's state
* the transaction results
* the consensus parameter updates
* the validator updates
With ABCI++, an Application may decide to keep using the next-block execution model,
however the new methods introduced, `PrepareProposal` and `ProcessProposal` allow
for a new execution model, called same-block execution. An Application implementing
this execution model, upon receiving a raw proposal via `PrepareProposal` and
potentially modifying the list of transactions, the Application fully executes the
resulting prepared proposal as though it was the decided block. The results of the
block execution are used as follows:
* the Application keeps the events generated and provides them if `FinalizeBlock`
is finally called on this prepared proposal.
* the merkle root resulting from executing the prepared proposal is provided in
`ResponsePrepareProposal` and thus refers to the **current block**. Tendermint
will use it in the prepared proposal's header.
* likewise, the transaction results from executing the prepared proposal are
provided in `ResponsePrepareProposal` and refer to the transactions in the
**current block**. Tendermint will use them to calculate the results hash
in the prepared proposal's header.
* the consensus parameter updates and validator updates are also provided in
`ResponsePrepareProposal` and reflect the result of the prepared proposal's
execution. They come into force in the next height (as opposed to the H+2 rule
in next-block execution model).
If the Application decides to keep the next-block execution model, it will not
provide any data in `ResponsePrepareProposal`, other than an optionally modified
transaction list.
## State Sync
State sync allows new nodes to rapidly bootstrap by discovering, fetching, and applying
state machine snapshots instead of replaying historical blocks. For more details, see the
[state sync section](../spec/p2p/messages/state-sync.md).
New nodes will discover and request snapshots from other nodes in the P2P network.
A Tendermint node that receives a request for snapshots from a peer will call
`ListSnapshots` on its application to retrieve any local state snapshots. After receiving
snapshots from peers, the new node will offer each snapshot received from a peer
to its local application via the `OfferSnapshot` method.
Snapshots may be quite large and are thus broken into smaller "chunks" that can be
assembled into the whole snapshot. Once the application accepts a snapshot and
begins restoring it, Tendermint will fetch snapshot "chunks" from existing nodes.
The node providing "chunks" will fetch them from its local application using
the `LoadSnapshotChunk` method.
As the new node receives "chunks" it will apply them sequentially to the local
application with `ApplySnapshotChunk`. When all chunks have been applied, the application
`AppHash` is retrieved via an `Info` query. The `AppHash` is then compared to
the blockchain's `AppHash` which is verified via
[light client verification](../spec/light-client/verification/README.md).

+ 815
- 0
spec/abci++/abci++_methods_002_draft.md View File

@ -0,0 +1,815 @@
---
order: 2
title: Methods
---
# Methods
## Methods existing in ABCI
### Echo
* **Request**:
* `Message (string)`: A string to echo back
* **Response**:
* `Message (string)`: The input string
* **Usage**:
* Echo a string to test an abci client/server implementation
### Flush
* **Usage**:
* Signals that messages queued on the client should be flushed to
the server. It is called periodically by the client
implementation to ensure asynchronous requests are actually
sent, and is called immediately to make a synchronous request,
which returns when the Flush response comes back.
### Info
* **Request**:
| Name | Type | Description | Field Number |
|---------------|--------|------------------------------------------|--------------|
| version | string | The Tendermint software semantic version | 1 |
| block_version | uint64 | The Tendermint Block Protocol version | 2 |
| p2p_version | uint64 | The Tendermint P2P Protocol version | 3 |
| abci_version | string | The Tendermint ABCI semantic version | 4 |
* **Response**:
| Name | Type | Description | Field Number |
|---------------------|--------|--------------------------------------------------|--------------|
| data | string | Some arbitrary information | 1 |
| version | string | The application software semantic version | 2 |
| app_version | uint64 | The application protocol version | 3 |
| last_block_height | int64 | Latest block for which the app has called Commit | 4 |
| last_block_app_hash | bytes | Latest result of Commit | 5 |
* **Usage**:
* Return information about the application state.
* Used to sync Tendermint with the application during a handshake
that happens on startup.
* The returned `app_version` will be included in the Header of every block.
* Tendermint expects `last_block_app_hash` and `last_block_height` to
be updated during `Commit`, ensuring that `Commit` is never
called twice for the same block height.
> Note: Semantic version is a reference to [semantic versioning](https://semver.org/). Semantic versions in info will be displayed as X.X.x.
### InitChain
* **Request**:
| Name | Type | Description | Field Number |
|------------------|--------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------|--------------|
| time | [google.protobuf.Timestamp](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Timestamp) | Genesis time | 1 |
| chain_id | string | ID of the blockchain. | 2 |
| consensus_params | [ConsensusParams](#consensusparams) | Initial consensus-critical parameters. | 3 |
| validators | repeated [ValidatorUpdate](#validatorupdate) | Initial genesis validators, sorted by voting power. | 4 |
| app_state_bytes | bytes | Serialized initial application state. JSON bytes. | 5 |
| initial_height | int64 | Height of the initial block (typically `1`). | 6 |
* **Response**:
| Name | Type | Description | Field Number |
|------------------|----------------------------------------------|-------------------------------------------------|--------------|
| consensus_params | [ConsensusParams](#consensusparams) | Initial consensus-critical parameters (optional | 1 |
| validators | repeated [ValidatorUpdate](#validatorupdate) | Initial validator set (optional). | 2 |
| app_hash | bytes | Initial application hash. | 3 |
* **Usage**:
* Called once upon genesis.
* If ResponseInitChain.Validators is empty, the initial validator set will be the RequestInitChain.Validators
* If ResponseInitChain.Validators is not empty, it will be the initial
validator set (regardless of what is in RequestInitChain.Validators).
* This allows the app to decide if it wants to accept the initial validator
set proposed by tendermint (ie. in the genesis file), or if it wants to use
a different one (perhaps computed based on some application specific
information in the genesis file).
### Query
* **Request**:
| Name | Type | Description | Field Number |
|--------|--------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| data | bytes | Raw query bytes. Can be used with or in lieu of Path. | 1 |
| path | string | Path field of the request URI. Can be used with or in lieu of `data`. Apps MUST interpret `/store` as a query by key on the underlying store. The key SHOULD be specified in the `data` field. Apps SHOULD allow queries over specific types like `/accounts/...` or `/votes/...` | 2 |
| height | int64 | The block height for which you want the query (default=0 returns data for the latest committed block). Note that this is the height of the block containing the application's Merkle root hash, which represents the state as it was after committing the block at Height-1 | 3 |
| prove | bool | Return Merkle proof with response if possible | 4 |
* **Response**:
| Name | Type | Description | Field Number |
|-----------|-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| code | uint32 | Response code. | 1 |
| log | string | The output of the application's logger. **May be non-deterministic.** | 3 |
| info | string | Additional information. **May be non-deterministic.** | 4 |
| index | int64 | The index of the key in the tree. | 5 |
| key | bytes | The key of the matching data. | 6 |
| value | bytes | The value of the matching data. | 7 |
| proof_ops | [ProofOps](#proofops) | Serialized proof for the value data, if requested, to be verified against the `app_hash` for the given Height. | 8 |
| height | int64 | The block height from which data was derived. Note that this is the height of the block containing the application's Merkle root hash, which represents the state as it was after committing the block at Height-1 | 9 |
| codespace | string | Namespace for the `code`. | 10 |
* **Usage**:
* Query for data from the application at current or past height.
* Optionally return Merkle proof.
* Merkle proof includes self-describing `type` field to support many types
of Merkle trees and encoding formats.
### CheckTx
* **Request**:
| Name | Type | Description | Field Number |
|------|-------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| tx | bytes | The request transaction bytes | 1 |
| type | CheckTxType | One of `CheckTx_New` or `CheckTx_Recheck`. `CheckTx_New` is the default and means that a full check of the tranasaction is required. `CheckTx_Recheck` types are used when the mempool is initiating a normal recheck of a transaction. | 2 |
* **Response**:
| Name | Type | Description | Field Number |
|------------|-------------------------------------------------------------|-----------------------------------------------------------------------|--------------|
| code | uint32 | Response code. | 1 |
| data | bytes | Result bytes, if any. | 2 |
| log | string | The output of the application's logger. **May be non-deterministic.** | 3 |
| info | string | Additional information. **May be non-deterministic.** | 4 |
| gas_wanted | int64 | Amount of gas requested for transaction. | 5 |
| gas_used | int64 | Amount of gas consumed by transaction. | 6 |
| events | repeated [Event](abci++_basic_concepts_002_draft.md#events) | Type & Key-Value events for indexing transactions (eg. by account). | 7 |
| codespace | string | Namespace for the `code`. | 8 |
| sender | string | The transaction's sender (e.g. the signer) | 9 |
| priority | int64 | The transaction's priority (for mempool ordering) | 10 |
* **Usage**:
* Technically optional - not involved in processing blocks.
* Guardian of the mempool: every node runs `CheckTx` before letting a
transaction into its local mempool.
* The transaction may come from an external user or another node
* `CheckTx` validates the transaction against the current state of the application,
for example, checking signatures and account balances, but does not apply any
of the state changes described in the transaction.
not running code in a virtual machine.
* Transactions where `ResponseCheckTx.Code != 0` will be rejected - they will not be broadcast to
other nodes or included in a proposal block.
* Tendermint attributes no other value to the response code
### ListSnapshots
* **Request**:
| Name | Type | Description | Field Number |
|--------|-------|------------------------------------|--------------|
Empty request asking the application for a list of snapshots.
* **Response**:
| Name | Type | Description | Field Number |
|-----------|--------------------------------|--------------------------------|--------------|
| snapshots | repeated [Snapshot](#snapshot) | List of local state snapshots. | 1 |
* **Usage**:
* Used during state sync to discover available snapshots on peers.
* See `Snapshot` data type for details.
### LoadSnapshotChunk
* **Request**:
| Name | Type | Description | Field Number |
|--------|--------|-----------------------------------------------------------------------|--------------|
| height | uint64 | The height of the snapshot the chunks belongs to. | 1 |
| format | uint32 | The application-specific format of the snapshot the chunk belongs to. | 2 |
| chunk | uint32 | The chunk index, starting from `0` for the initial chunk. | 3 |
* **Response**:
| Name | Type | Description | Field Number |
|-------|-------|-------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| chunk | bytes | The binary chunk contents, in an arbitray format. Chunk messages cannot be larger than 16 MB _including metadata_, so 10 MB is a good starting point. | 1 |
* **Usage**:
* Used during state sync to retrieve snapshot chunks from peers.
### OfferSnapshot
* **Request**:
| Name | Type | Description | Field Number |
|----------|-----------------------|--------------------------------------------------------------------------|--------------|
| snapshot | [Snapshot](#snapshot) | The snapshot offered for restoration. | 1 |
| app_hash | bytes | The light client-verified app hash for this height, from the blockchain. | 2 |
* **Response**:
| Name | Type | Description | Field Number |
|--------|-------------------|-----------------------------------|--------------|
| result | [Result](#result) | The result of the snapshot offer. | 1 |
#### Result
```protobuf
enum Result {
UNKNOWN = 0; // Unknown result, abort all snapshot restoration
ACCEPT = 1; // Snapshot is accepted, start applying chunks.
ABORT = 2; // Abort snapshot restoration, and don't try any other snapshots.
REJECT = 3; // Reject this specific snapshot, try others.
REJECT_FORMAT = 4; // Reject all snapshots with this `format`, try others.
REJECT_SENDER = 5; // Reject all snapshots from all senders of this snapshot, try others.
}
```
* **Usage**:
* `OfferSnapshot` is called when bootstrapping a node using state sync. The application may
accept or reject snapshots as appropriate. Upon accepting, Tendermint will retrieve and
apply snapshot chunks via `ApplySnapshotChunk`. The application may also choose to reject a
snapshot in the chunk response, in which case it should be prepared to accept further
`OfferSnapshot` calls.
* Only `AppHash` can be trusted, as it has been verified by the light client. Any other data
can be spoofed by adversaries, so applications should employ additional verification schemes
to avoid denial-of-service attacks. The verified `AppHash` is automatically checked against
the restored application at the end of snapshot restoration.
* For more information, see the `Snapshot` data type or the [state sync section](../spec/p2p/messages/state-sync.md).
### ApplySnapshotChunk
* **Request**:
| Name | Type | Description | Field Number |
|--------|--------|-----------------------------------------------------------------------------|--------------|
| index | uint32 | The chunk index, starting from `0`. Tendermint applies chunks sequentially. | 1 |
| chunk | bytes | The binary chunk contents, as returned by `LoadSnapshotChunk`. | 2 |
| sender | string | The P2P ID of the node who sent this chunk. | 3 |
* **Response**:
| Name | Type | Description | Field Number |
|----------------|---------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| result | Result (see below) | The result of applying this chunk. | 1 |
| refetch_chunks | repeated uint32 | Refetch and reapply the given chunks, regardless of `result`. Only the listed chunks will be refetched, and reapplied in sequential order. | 2 |
| reject_senders | repeated string | Reject the given P2P senders, regardless of `Result`. Any chunks already applied will not be refetched unless explicitly requested, but queued chunks from these senders will be discarded, and new chunks or other snapshots rejected. | 3 |
```proto
enum Result {
UNKNOWN = 0; // Unknown result, abort all snapshot restoration
ACCEPT = 1; // The chunk was accepted.
ABORT = 2; // Abort snapshot restoration, and don't try any other snapshots.
RETRY = 3; // Reapply this chunk, combine with `RefetchChunks` and `RejectSenders` as appropriate.
RETRY_SNAPSHOT = 4; // Restart this snapshot from `OfferSnapshot`, reusing chunks unless instructed otherwise.
REJECT_SNAPSHOT = 5; // Reject this snapshot, try a different one.
}
```
* **Usage**:
* The application can choose to refetch chunks and/or ban P2P peers as appropriate. Tendermint
will not do this unless instructed by the application.
* The application may want to verify each chunk, e.g. by attaching chunk hashes in
`Snapshot.Metadata` and/or incrementally verifying contents against `AppHash`.
* When all chunks have been accepted, Tendermint will make an ABCI `Info` call to verify that
`LastBlockAppHash` and `LastBlockHeight` matches the expected values, and record the
`AppVersion` in the node state. It then switches to fast sync or consensus and joins the
network.
* If Tendermint is unable to retrieve the next chunk after some time (e.g. because no suitable
peers are available), it will reject the snapshot and try a different one via `OfferSnapshot`.
The application should be prepared to reset and accept it or abort as appropriate.
## New methods introduced in ABCI++
### PrepareProposal
#### Parameters and Types
* **Request**:
| Name | Type | Description | Field Number |
|-------------------------|---------------------------------------------|------------------------------------------------------------------------------------------------------------------|--------------|
| hash | bytes | The block header's hash of the block to propose. Present for convenience (can be derived from the block header). | 1 |
| header | [Header](../core/data_structures.md#header) | The header of the block to propose. | 2 |
| tx | repeated bytes | Preliminary list of transactions that have been picked as part of the block to propose. | 3 |
| byzantine_validators | repeated [Evidence](#evidence) | List of evidence of validators that acted maliciously. | 4 |
| last_commit_info | [LastCommitInfo](#lastcommitinfo) | Info about the last commit, including the round, the validator list, and which ones signed the last block. | 5 |
>**TODO**: Add the changes needed in LastCommitInfo for vote extensions
>**TODO**: DISCUSS: We need to make clear whether a proposer is also running the logic of a non-proposer node (in particular "ProcessProposal")
From the App's perspective, they'll probably skip ProcessProposal
* **Response**:
| Name | Type | Description | Field Number |
|-------------------------|--------------------------------------------------|---------------------------------------------------------------------------------------------|--------------|
| modified | bool | The Application sets it to true to denote it did not make changes | 1 |
| tx | repeated [TransactionRecord](#transactionrecord) | Possibly modified list of transactions that have been picked as part of the proposed block. | 2 |
| data | bytes | The Merkle root hash of the application state. | 3 |
| validator_updates | repeated [ValidatorUpdate](#validatorupdate) | Changes to validator set (set voting power to 0 to remove). | 4 |
| consensus_param_updates | [ConsensusParams](#consensusparams) | Changes to consensus-critical gas, size, and other parameters. | 5 |
* **Usage**:
* Contains a preliminary block to be proposed, which the Application can modify.
* The parameters and types of `RequestPrepareProposal` are the same as `RequestProcessProposal`
and `RequestFinalizeBlock`.
* The header contains the height, timestamp, and more - it exactly matches the
Tendermint block header.
* The Application can modify the parameters received in `RequestPrepareProposal` before sending
them in `ResponsePrepareProposal`. In that case, `ResponsePrepareProposal.modified` is set to true.
* In same-block execution mode, the Application can (and should) modify `ResponsePrepareProposal.data`,
`ResponsePrepareProposal.validator_updates`, and `ResponsePrepareProposal.consensus_param_updates`.
* In next-block execution mode, the Application can only modify `ResponsePrepareProposal.tx`, Tendermint
will ignore any modification to the other fields.
* If `ResponsePrepareProposal.modified` is false, then Tendermint will ignore the rest of
parameters in `ResponsePrepareProposal`.
* As a result of executing the block to propose, the Application may produce header events or transaction events.
The Application must keep those events until a block is decided and then pass them on to Tendermint via
`ResponseFinalizeBlock`.
* Likewise, the Application must keep all responses to executing transactions until it can call `ResponseFinalizeBlock`.
* The Application can change the transaction list via `ResponsePrepareProposal.tx`.
See [TransactionRecord](#transactionrecord) for further information on how to use it. Some notes:
* To remove a transaction from the proposed block the Application _marks_ the transaction as
"REMOVE". It does not remove it from the list.
* Removing a transaction from the list means it is too early to propose that transaction,
so it will be excluded from the proposal but will stay in the mempool for later proposals.
The Application should be extra-careful, as abusing this feature may cause transactions to
stay forever in the mempool.
* The `new_hashes` field, besides helping with mempool maintenance, helps Tendermint handle
queries such as "what happened with this Tx?", by answering "it was modified into these ones".
* The Application _can_ reorder the transactions in the list.
* As a sanity check, Tendermint will check the returned parameters for validity if the Application modified them.
In particular, `ResponsePrepareProposal.tx` will be deemed invalid if
* There is a duplicate transaction in the list.
* The `new_hashes` field contains a dangling reference to a non-existing transaction.
* A new or modified transaction is marked as "UNMODIFIED" or "REMOVED".
* An unmodified transaction is marked as "ADDED".
* A transaction is marked as "UNKNOWN".
* If Tendermint's sanity checks on the parameters of `ResponsePrepareProposal` fails, then it will drop the proposal
and proceed to the next round (thus simulating a network loss/delay of the proposal).
* **TODO**: [From discussion with William] Another possibility here is to panic. What do folks think we should do here?
* The implementation of `PrepareProposal` can be non-deterministic.
#### When does Tendermint call it?
When a validator _p_ enters Tendermint consensus round _r_, height _h_, in which _p_ is the proposer,
and _p_'s _validValue_ is `nil`:
1. _p_'s Tendermint collects outstanding transactions from the mempool
* The transactions will be collected in order of priority
* Let $C$ the list of currently collected transactions
* The collection stops when any of the following conditions are met
* the mempool is empty
* the total size of transactions $\in C$ is greater than or equal to `consensusParams.block.max_bytes`
* the sum of `GasWanted` field of transactions $\in C$ is greater than or equal to
`consensusParams.block.max_gas`
* _p_'s Tendermint creates a block header.
2. _p_'s Tendermint calls `RequestPrepareProposal` with the newly generated block.
The call is synchronous: Tendermint's execution will block until the Application returns from the call.
3. The Application checks the block (header, transactions, commit info, evidences). Besides,
* in same-block execution mode, the Application can (and should) provide `ResponsePrepareProposal.data`,
`ResponsePrepareProposal.validator_updates`, or
`ResponsePrepareProposal.consensus_param_updates`.
* in "next-block execution" mode, _p_'s Tendermint will ignore the values for `ResponsePrepareProposal.data`,
`ResponsePrepareProposal.validator_updates`, and `ResponsePrepareProposal.consensus_param_updates`.
* in both modes, the Application can manipulate transactions
* leave transactions untouched - `TxAction = UNMODIFIED`
* add new transactions (not previously in the mempool) - `TxAction = ADDED`
* removed transactions (invalid) from the proposal and from the mempool - `TxAction = REMOVED`
* remove transactions from the proposal but not from the mempool (effectively _delaying_ them) - the
Application removes the transaction from the list
* modify transactions (e.g. aggregate them) - `TxAction = ADDED` followed by `TxAction = REMOVED`
* reorder transactions - the Application reorders transactions in the list
4. If the block is modified, the Application sets `ResponsePrepareProposal.modified` to true,
and includes the modified block in the return parameters (see the rules in section _Usage_).
The Application returns from the call.
5. _p_'s Tendermint uses the (possibly) modified block as _p_'s proposal in round _r_, height _h_.
Note that, if _p_ has a non-`nil` _validValue_, Tendermint will use it as proposal and will not call `RequestPrepareProposal`.
### ProcessProposal
#### Parameters and Types
* **Request**:
| Name | Type | Description | Field Number |
|----------------------|---------------------------------------------|------------------------------------------------------------------------------------------------------------------|--------------|
| hash | bytes | The block header's hash of the proposed block. Present for convenience (can be derived from the block header). | 1 |
| header | [Header](../core/data_structures.md#header) | The proposed block's header. | 2 |
| tx | repeated bytes | List of transactions that have been picked as part of the proposed block. | 3 |
| byzantine_validators | repeated [Evidence](#evidence) | List of evidence of validators that acted maliciously. | 4 |
| last_commit_info | [LastCommitInfo](#lastcommitinfo) | Info about the last commit, including the round , the validator list, and which ones signed the last block. | 5 |
* **Response**:
| Name | Type | Description | Field Number |
|--------|------|--------------------------------------------------|--------------|
| accept | bool | If false, the received block failed verification | 1 |
* **Usage**:
* Contains a full proposed block.
* The parameters and types of `RequestProcessProposal` are the same as `RequestPrepareProposal`
and `RequestFinalizeBlock`.
* The Application may fully execute the block as though it was handling `RequestFinalizeBlock`.
However, any resulting state changes must be kept as _canditade state_,
and the Application should be ready to backtrack/discard it in case the decided block is different.
* The header contains the height, timestamp, and more - it exactly matches the
Tendermint block header.
* If `ResponseProcessProposal.accept` is _false_, Tendermint assumes the proposal received
is not valid.
* The implementation of `ProcessProposal` MUST be deterministic. Moreover, the value of
`ResponseProcessProposal.accept` MUST *exclusively* depend on the parameters passed in
the call to `RequestProcessProposal`, and the last committed Application state
(see [Properties](#properties) section below).
* Moreover, application implementors SHOULD always set `ResponseProcessProposal.accept` to _true_,
unless they _really_ know what the potential liveness implications of returning _false_ are.
>**TODO**: should `ResponseProcessProposal.accept` be of type `Result` rather than `bool`? (so we are able to extend the possible values in the future?)
#### When does Tendermint call it?
When a validator _p_ enters Tendermint consensus round _r_, height _h_, in which _q_ is the proposer (possibly _p_ = _q_):
1. _p_ sets up timer `ProposeTimeout`.
2. If _p_ is the proposer, _p_ executes steps 1-6 in [PrepareProposal](#prepareproposal).
3. Upon reception of Proposal message (which contains the header) for round _r_, height _h_ from _q_, _p_'s Tendermint verifies the block header.
4. Upon reception of Proposal message, along with all the block parts, for round _r_, height _h_ from _q_, _p_'s Tendermint follows its algorithm
to check whether it should prevote for the block just received, or `nil`
5. If Tendermint should prevote for the block just received
1. Tendermint calls `RequestProcessProposal` with the block. The call is synchronous.
2. The Application checks/processes the proposed block, which is read-only, and returns true (_accept_) or false (_reject_) in `ResponseProcessProposal.accept`.
* The Application, depending on its needs, may call `ResponseProcessProposal`
* either after it has completely processed the block (the simpler case),
* or immediately (after doing some basic checks), and process the block asynchronously. In this case the Application will
not be able to reject the block, or force prevote/precommit `nil` afterwards.
3. If the returned value is
* _accept_, Tendermint prevotes on this proposal for round _r_, height _h_.
* _reject_, Tendermint prevotes `nil`.
### ExtendVote
#### Parameters and Types
* **Request**:
| Name | Type | Description | Field Number |
|--------|-------|-----------------------------------------------------------------------|--------------|
| hash | bytes | The hash of the proposed block that the vote extension is to refer to | 1 |
| height | int64 | Height of the proposed block (for sanity check). | 2 |
* **Response**:
| Name | Type | Description | Field Number |
|-----------|-------|---------------------------------------------------------------------|--------------|
| extension | bytes | Optional information that will be attached to the Precommit message | 1 |
* **Usage**:
* `RequestExtendVote.hash` corresponds to the hash of a proposed block that was made available to the application
in a previous call to `ProcessProposal` for the current height.
* `ResponseExtendVote.extension` will always be attached to a non-`nil` Precommit message. If Tendermint is to
precommit `nil`, it will not call `RequestExtendVote`.
* The Application logic that creates the extension can be non-deterministic.
#### When does Tendermint call it?
When a validator _p_ is in Tendermint consensus state _prevote_ of round _r_, height _h_, in which _q_ is the proposer; and _p_ has received
* the Proposal message _v_ for round _r_, height _h_, along with all the block parts, from _q_,
* `Prevote` messages from _2f + 1_ validators' voting power for round _r_, height _h_, prevoting for the same block _id(v)_,
then _p_'s Tendermint locks _v_ and sends a Precommit message in the following way
1. _p_'s Tendermint sets _lockedValue_ and _validValue_ to _v_, and sets _lockedRound_ and _validRound_ to _r_
2. _p_'s Tendermint calls `RequestExtendVote` with _id(v)_ (`RequestExtendVote.hash`). The call is synchronous.
3. The Application returns an array of bytes, `ResponseExtendVote.extension`, which is not interpreted by Tendermint.
4. _p_'s Tendermint includes `ResponseExtendVote.extension` as a new field in the Precommit message.
5. _p_'s Tendermint signs and broadcasts the Precommit message.
In the cases when _p_'s Tendermint is to broadcast `precommit nil` messages (either _2f+1_ `prevote nil` messages received, or _timeoutPrevote_ triggered), _p_'s Tendermint does **not** call `RequestExtendVote` and will include an empty byte array as vote extension in the `precommit nil` message.
### VerifyVoteExtension
#### Parameters and Types
* **Request**:
| Name | Type | Description | Field Number |
|-----------|-------|------------------------------------------------------------------------------------------|--------------|
| extension | bytes | Sender Application's vote extension to be validated | 1 |
| hash | bytes | The hash of the propsed block that the vote extension refers to | 2 |
| address | bytes | [Address](../core/data_structures.md#address) of the validator that signed the extension | 3 |
| height | int64 | Height of the block (for sanity check). | 4 |
* **Response**:
| Name | Type | Description | Field Number |
|--------|------|-------------------------------------------------------|--------------|
| accept | bool | If false, Application is rejecting the vote extension | 1 |
* **Usage**:
* If `ResponseVerifyVoteExtension.accept` is _false_, Tendermint will reject the whole received vote.
See the [Properties](#properties) section to understand the potential liveness implications of this.
* The implementation of `VerifyVoteExtension` MUST be deterministic. Moreover, the value of
`ResponseVerifyVoteExtension.accept` MUST *exclusively* depend on the parameters passed in
the call to `RequestVerifyVoteExtension`, and the last committed Application state
(see [Properties](#properties) section below).
* Moreover, application implementors SHOULD always set `ResponseVerifyVoteExtension.accept` to _true_,
unless they _really_ know what the potential liveness implications of returning _false_ are.
#### When does Tendermint call it?
When a validator _p_ is in Tendermint consensus round _r_, height _h_, state _prevote_ (**TODO** discuss: I think I must remove the state
from this condition, but not sure), and _p_ receives a Precommit message for round _r_, height _h_ from _q_:
1. _p_'s Tendermint calls `RequestVerifyVoteExtension`.
2. The Application returns _accept_ or _reject_ via `ResponseVerifyVoteExtension.accept`.
3. If the Application returns
* _accept_, _p_'s Tendermint will keep the received vote, together with its corresponding
vote extension in its internal data structures. It will be used to:
* calculate field _LastCommitHash_ in the header of the block proposed for height _h + 1_
(in the rounds where _p_ will be proposer).
* populate _LastCommitInfo_ in calls to `RequestPrepareProposal`, `RequestProcessProposal`,
and `RequestFinalizeBlock` in height _h + 1_.
* _reject_, _p_'s Tendermint will deem the Precommit message invalid and discard it.
### FinalizeBlock
#### Parameters and Types
* **Request**:
| Name | Type | Description | Field Number |
|----------------------|---------------------------------------------|-------------------------------------------------------------------------------------------------------------------|--------------|
| hash | bytes | The block header's hash. Present for convenience (can be derived from the block header). | 1 |
| header | [Header](../core/data_structures.md#header) | The block header. | 2 |
| tx | repeated bytes | List of transactions committed as part of the block. | 3 |
| byzantine_validators | repeated [Evidence](#evidence) | List of evidence of validators that acted maliciously. | 4 |
| last_commit_info | [LastCommitInfo](#lastcommitinfo) | Info about the last commit, including the round, and the list of validators and which ones signed the last block. | 5 |
* **Response**:
| Name | Type | Description | Field Number |
|-------------------------|-------------------------------------------------------------|---------------------------------------------------------------------------------|--------------|
| block_events | repeated [Event](abci++_basic_concepts_002_draft.md#events) | Type & Key-Value events for indexing | 1 |
| tx_result | repeated [DeliverTxResult](#delivertxresult) | List of structures containing the data resulting from executing the transaction | 2 |
| validator_updates | repeated [ValidatorUpdate](#validatorupdate) | Changes to validator set (set voting power to 0 to remove). | 3 |
| consensus_param_updates | [ConsensusParams](#consensusparams) | Changes to consensus-critical gas, size, and other parameters. | 4 |
| app_data | bytes | The Merkle root hash of the application state. | 6 |
| retain_height | int64 | Blocks below this height may be removed. Defaults to `0` (retain all). | 7 |
* **Usage**:
* Contains a newly decided block.
* This method is equivalent to the call sequence `BeginBlock`, [`DeliverTx`],
`EndBlock`, `Commit` in the previous version of ABCI.
* The header contains the height, timestamp, and more - it exactly matches the
Tendermint block header.
* The Application can use `RequestFinalizeBlock.last_commit_info` and `RequestFinalizeBlock.byzantine_validators`
to determine rewards and punishments for the validators.
* The application must execute the transactions in full, in the order they appear in `RequestFinalizeBlock.tx`,
before returning control to Tendermint. Alternatively, it can commit the candidate state corresponding to the same block
previously executed via `PrepareProposal` or `ProcessProposal`.
* `ResponseFinalizeBlock.tx_result[i].Code == 0` only if the _i_-th transaction is fully valid.
* Optional `ResponseFinalizeBlock.validator_updates` triggered by block `H`. These updates affect validation
for blocks `H+1`, `H+2`, and `H+3`. Heights following a validator update are affected in the following way:
* `H+1`: `NextValidatorsHash` includes the new `validator_updates` value.
* `H+2`: The validator set change takes effect and `ValidatorsHash` is updated.
* `H+3`: `last_commit_info` is changed to include the altered validator set.
* `ResponseFinalizeBlock.consensus_param_updates` returned for block `H` apply to the consensus
params for block `H+1`. For more information on the consensus parameters,
see the [application spec entry on consensus parameters](../abci/apps.md#consensus-parameters).
* Application is expected to persist its state at the end of this call, before calling `ResponseFinalizeBlock`.
* `ResponseFinalizeBlock.app_data` contains an (optional) Merkle root hash of the application state.
* `ResponseFinalizeBlock.app_data` is included
* [in next-block execution mode] as the `Header.AppHash` in the next block.
* [in same-block execution mode] as the `Header.AppHash` in the current block. In this case,
`PrepareProposal` is required to fully execute the block and set the App hash before
returning the proposed block to Tendermint.
* `ResponseFinalizeBlock.app_data` may also be empty or hard-coded, but MUST be
**deterministic** - it must not be a function of anything that did not come from the parameters
of `RequestFinalizeBlock` and the previous committed state.
* Later calls to `Query` can return proofs about the application state anchored
in this Merkle root hash.
* Use `retain_height` with caution! If all nodes in the network remove historical
blocks then this data is permanently lost, and no new nodes will be able to
join the network and bootstrap. Historical blocks may also be required for
other purposes, e.g. auditing, replay of non-persisted heights, light client
verification, and so on.
* Just as `ProcessProposal`, the implementation of `FinalizeBlock` MUST be deterministic, since it is
making the Application's state evolve in the context of state machine replication.
* Currently, Tendermint will fill up all fields in `RequestFinalizeBlock`, even if they were
already passed on to the Application via `RequestPrepareProposal` or `RequestProcessProposal`.
If the Application is in same-block execution mode, it applies the right candidate state here
(rather than executing the whole block). In this case the Application disregards all parameters in
`RequestFinalizeBlock` except `RequestFinalizeBlock.hash`.
#### When does Tendermint call it?
When a validator _p_ is in Tendermint consensus height _h_, and _p_ receives
* the Proposal message with block _v_ for a round _r_, along with all its block parts, from _q_,
which is the proposer of round _r_, height _h_,
* `Precommit` messages from _2f + 1_ validators' voting power for round _r_, height _h_,
precommitting the same block _id(v)_,
then _p_'s Tendermint decides block _v_ and finalizes consensus for height _h_ in the following way
1. _p_'s Tendermint persists _v_ as decision for height _h_.
2. _p_'s Tendermint locks the mempool -- no calls to checkTx on new transactions.
3. _p_'s Tendermint calls `RequestFinalizeBlock` with _id(v)_. The call is synchronous.
4. _p_'s Application processes block _v_, received in a previous call to `RequestProcessProposal`.
5. _p_'s Application commits and persists the state resulting from processing the block.
6. _p_'s Application calculates and returns the _AppHash_, along with an array of arrays of bytes representing the output of each of the transactions
7. _p_'s Tendermint hashes the array of transaction outputs and stores it in _ResultHash_
8. _p_'s Tendermint persists _AppHash_ and _ResultHash_
9. _p_'s Tendermint unlocks the mempool -- newly received transactions can now be checked.
10. _p_'s starts consensus for a new height _h+1_, round 0
## Data Types existing in ABCI
Most of the data structures used in ABCI are shared [common data structures](../spec/core/data_structures.md). In certain cases, ABCI uses different data structures which are documented here:
### Validator
* **Fields**:
| Name | Type | Description | Field Number |
|---------|-------|---------------------------------------------------------------------|--------------|
| address | bytes | [Address](../core/data_structures.md#address) of validator | 1 |
| power | int64 | Voting power of the validator | 3 |
* **Usage**:
* Validator identified by address
* Used in RequestBeginBlock as part of VoteInfo
* Does not include PubKey to avoid sending potentially large quantum pubkeys
over the ABCI
### ValidatorUpdate
* **Fields**:
| Name | Type | Description | Field Number |
|---------|--------------------------------------------------|-------------------------------|--------------|
| pub_key | [Public Key](../core/data_structures.md#pub_key) | Public key of the validator | 1 |
| power | int64 | Voting power of the validator | 2 |
* **Usage**:
* Validator identified by PubKey
* Used to tell Tendermint to update the validator set
### VoteInfo
* **Fields**:
| Name | Type | Description | Field Number |
|-------------------|-------------------------|--------------------------------------------------------------|--------------|
| validator | [Validator](#validator) | A validator | 1 |
| signed_last_block | bool | Indicates whether or not the validator signed the last block | 2 |
* **Usage**:
* Indicates whether a validator signed the last block, allowing for rewards
based on validator availability
### Evidence
* **Fields**:
| Name | Type | Description | Field Number |
|--------------------|--------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------|--------------|
| type | [EvidenceType](#evidencetype) | Type of the evidence. An enum of possible evidence's. | 1 |
| validator | [Validator](#validator) | The offending validator | 2 |
| height | int64 | Height when the offense occurred | 3 |
| time | [google.protobuf.Timestamp](https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Timestamp) | Time of the block that was committed at the height that the offense occurred | 4 |
| total_voting_power | int64 | Total voting power of the validator set at height `Height` | 5 |
#### EvidenceType
* **Fields**
EvidenceType is an enum with the listed fields:
| Name | Field Number |
|---------------------|--------------|
| UNKNOWN | 0 |
| DUPLICATE_VOTE | 1 |
| LIGHT_CLIENT_ATTACK | 2 |
### LastCommitInfo
* **Fields**:
| Name | Type | Description | Field Number |
|-------|--------------------------------|-----------------------------------------------------------------------------------------------------------------------|--------------|
| round | int32 | Commit round. Reflects the total amount of rounds it took to come to consensus for the current block. | 1 |
| votes | repeated [VoteInfo](#voteinfo) | List of validators addresses in the last validator set with their voting power and whether or not they signed a vote. | 2 |
### ConsensusParams
* **Fields**:
| Name | Type | Description | Field Number |
|-----------|---------------------------------------------------------------|------------------------------------------------------------------------------|--------------|
| block | [BlockParams](../core/data_structures.md#blockparams) | Parameters limiting the size of a block and time between consecutive blocks. | 1 |
| evidence | [EvidenceParams](../core/data_structures.md#evidenceparams) | Parameters limiting the validity of evidence of byzantine behaviour. | 2 |
| validator | [ValidatorParams](../core/data_structures.md#validatorparams) | Parameters limiting the types of public keys validators can use. | 3 |
| version | [VersionsParams](../core/data_structures.md#versionparams) | The ABCI application version. | 4 |
### ProofOps
* **Fields**:
| Name | Type | Description | Field Number |
|------|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| ops | repeated [ProofOp](#proofop) | List of chained Merkle proofs, of possibly different types. The Merkle root of one op is the value being proven in the next op. The Merkle root of the final op should equal the ultimate root hash being verified against.. | 1 |
### ProofOp
* **Fields**:
| Name | Type | Description | Field Number |
|------|--------|------------------------------------------------|--------------|
| type | string | Type of Merkle proof and how it's encoded. | 1 |
| key | bytes | Key in the Merkle tree that this proof is for. | 2 |
| data | bytes | Encoded Merkle proof for the key. | 3 |
### Snapshot
* **Fields**:
| Name | Type | Description | Field Number |
|----------|--------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| height | uint64 | The height at which the snapshot was taken (after commit). | 1 |
| format | uint32 | An application-specific snapshot format, allowing applications to version their snapshot data format and make backwards-incompatible changes. Tendermint does not interpret this. | 2 |
| chunks | uint32 | The number of chunks in the snapshot. Must be at least 1 (even if empty). | 3 |
| hash | bytes | TAn arbitrary snapshot hash. Must be equal only for identical snapshots across nodes. Tendermint does not interpret the hash, it only compares them. | 3 |
| metadata | bytes | Arbitrary application metadata, for example chunk hashes or other verification data. | 3 |
* **Usage**:
* Used for state sync snapshots, see the [state sync section](../spec/p2p/messages/state-sync.md) for details.
* A snapshot is considered identical across nodes only if _all_ fields are equal (including
`Metadata`). Chunks may be retrieved from all nodes that have the same snapshot.
* When sent across the network, a snapshot message can be at most 4 MB.
## Data types introduced in ABCI++
### DeliverTxResult
* **Fields**:
| Name | Type | Description | Field Number |
|------------|-------------------------------------------------------------|-----------------------------------------------------------------------|--------------|
| code | uint32 | Response code. | 1 |
| data | bytes | Result bytes, if any. | 2 |
| log | string | The output of the application's logger. **May be non-deterministic.** | 3 |
| info | string | Additional information. **May be non-deterministic.** | 4 |
| gas_wanted | int64 | Amount of gas requested for transaction. | 5 |
| gas_used | int64 | Amount of gas consumed by transaction. | 6 |
| tx_events | repeated [Event](abci++_basic_concepts_002_draft.md#events) | Type & Key-Value events for indexing transactions (e.g. by account). | 7 |
| codespace | string | Namespace for the `code`. | 8 |
### TxAction
```protobuf
enum TxAction {
UNKNOWN = 0; // Unknown action
UNMODIFIED = 1; // The Application did not modify this transaction. Ignore new_hashes field
ADDED = 2; // The Application added this transaction. Ignore new_hashes field
REMOVED = 3; // The Application wants this transaction removed from the proposal and the mempool. Use new_hashes field if the transaction was modified
}
```
* **Usage**:
* If `Action` is UNKNOWN, a problem happened in the Application. Tendermint will ignore this transaction. **TODO** should we panic?
* If `Action` is UNMODIFIED, Tendermint includes the transaction in the proposal. Nothing to do on the mempool. Field `new_hashes` is ignored.
* If `Action` is ADDED, Tendermint includes the transaction in the proposal. The transaction is also added to the mempool and gossipped. Field `new_hashes` is ignored.
* If `Action` is REMOVED, Tendermint excludes the transaction from the proposal. The transaction is also removed from the mempool if it exists,
similar to `CheckTx` returning _false_. Tendermint can use field `new_hashes` to help client trace transactions that have been modified into other transactions.
### TransactionRecord
* **Fields**:
| Name | Type | Description | Field Number |
|------------|-----------------------|------------------------------------------------------------------|--------------|
| action | [TxAction](#txaction) | What should Tendermint do with this transaction? | 1 |
| tx | bytes | Transaction contents | 2 |
| new_hashes | repeated bytes | List of hashes of successor transactions | 3 |
* **Usage**:
* As `new_hashes` is a list, `TransactionRecord` allows to trace many-to-many modifications. Some examples:
* Transaction $t1$ modified into $t2$ is represented with these records
* $t2$ "ADDED"
* $t1$ "REMOVED"; `new_hashes` contains [$id(t2)$]
* Transaction $t1$ modified into $t2$ and $t3$ is represented with these `TransactionRecord` records
* $t2$ "ADDED"
* $t3$ "ADDED"
* $t1$ "REMOVED"; `new_hashes` contains [$id(t2)$, $id(t3)$]
* Transactions $t1$ and $t2$ aggregated into $t3$ is represented with these `TransactionRecord` records
* $t3$ "ADDED"
* $t1$ "REMOVED"; `new_hashes` contains [$id(t3)$]
* $t2$ "REMOVED"; `new_hashes` contains [$id(t3)$]
* Transactions $t1$ and $t2$ combined into $t3$ and $t4$ is represented with these `TransactionRecord` records
* $t3$ "ADDED"
* $t4$ "ADDED"
* $t1$ "REMOVED" and `new_hashes` containing [$id(t3)$, $id(t4)$]
* $t2$ "REMOVED" and `new_hashes` containing [$id(t3)$, $id(t4)$]

+ 0
- 721
spec/abci++/abci++_properties_001_draft.md View File

@ -1,721 +0,0 @@
---
order: 1
title: New Methods
---
# New Methods
## Description
### PrepareProposal
#### Parameters and Types
>**TODO**: Hyperlinks for ConsensusParams, LastCommitInfo, Evidence, Event, and ValidatorUpdate are
broken because they are defined in abci.md (and not copied over for the moment).
* **Request**:
| Name | Type | Description | Field Number |
|-------------------------|---------------------------------------------|------------------------------------------------------------------------------------------------------------------|--------------|
| hash | bytes | The block header's hash of the block to propose. Present for convenience (can be derived from the block header). | 1 |
| header | [Header](../core/data_structures.md#header) | The header of the block to propose. | 2 |
| tx | repeated bytes | Preliminary list of transactions that have been picked as part of the block to propose. | 3 |
| byzantine_validators | repeated [Evidence](#evidence) | List of evidence of validators that acted maliciously. | 4 |
| last_commit_info | [LastCommitInfo](#lastcommitinfo) | Info about the last commit, including the round, the validator list, and which ones signed the last block. | 5 |
>**TODO**: DISCUSS: We need to make clear whether a proposer is also running the logic of a non-proposer node (in particular "ProcessProposal")
From the App's perspective, they'll probably skip ProcessProposal
* **Response**:
| Name | Type | Description | Field Number |
|-------------------------|--------------------------------------------------|---------------------------------------------------------------------------------------------|--------------|
| modified | bool | The Application sets it to true to denote that it made changes | 1 |
| tx | repeated [TransactionRecord](#transactionrecord) | Possibly modified list of transactions that have been picked as part of the proposed block. | 2 |
| data | bytes | The Merkle root hash of the application state. | 3 |
| validator_updates | repeated [ValidatorUpdate](#validatorupdate) | Changes to validator set (set voting power to 0 to remove). | 4 |
| consensus_param_updates | [ConsensusParams](#consensusparams) | Changes to consensus-critical gas, size, and other parameters. | 5 |
* **Usage**:
* Contains a preliminary block to be proposed, which the Application can modify.
* The parameters and types of `RequestPrepareProposal` are the same as `RequestProcessProposal`
and `RequestFinalizeBlock`.
* The header contains the height, timestamp, and more - it exactly matches the
Tendermint block header.
* The Application can modify the parameters received in `RequestPrepareProposal` before sending
them in `ResponsePrepareProposal`. In that case, `ResponsePrepareProposal.modified` is set to true.
* In same-block execution mode, the Application can (and should) modify `ResponsePrepareProposal.data`,
`ResponsePrepareProposal.validator_updates`, and `ResponsePrepareProposal.consensus_param_updates`.
* In next-block execution mode, the Application can only modify `ResponsePrepareProposal.tx`, Tendermint
will ignore any modification to the other fields.
* If `ResponsePrepareProposal.modified` is false, then Tendermint will ignore the rest of
parameters in `ResponsePrepareProposal`.
* As a result of executing the block to propose, the Application may produce header events or transaction events.
The Application must keep those events until a block is decided and then pass them on to Tendermint via
`ResponseFinalizeBlock`.
* Likewise, the Application must keep all responses to executing transactions until it can call `ResponseFinalizeBlock`.
* The Application can change the transaction list via `ResponsePrepareProposal.tx`.
See [TransactionRecord](#transactionrecord) for further information on how to use it. Some notes:
* To remove a transaction from the proposed block the Application _marks_ the transaction as
"REMOVE". It does not remove it from the list.
* Removing a transaction from the list means it is too early to propose that transaction,
so it will be excluded from the proposal but will stay in the mempool for later proposals.
The Application should be extra-careful with this feature to avoid leaks in the mempool.
* The `new_hashes` field, besides helping with mempool maintenance, helps Tendermint handle
queries such as "what happened with this Tx?", by answering "it was modified into these ones".
* The Application _can_ reorder the transactions in the list.
* As a sanity check, Tendermint will check the returned parameters for validity if the Application modified them.
In particular, `ResponsePrepareProposal.tx` will be deemed invalid if
* There is a duplicate transaction in the list.
* The `new_hashes` field contains a dangling reference to a non-existing transaction.
* A new or modified transaction is marked as "UNMODIFIED" or "REMOVED".
* An unmodified transaction is marked as "ADDED".
* A transaction is marked as "UNKNOWN".
* If Tendermint's sanity checks on the parameters of `ResponsePrepareProposal` fails, then it will drop the proposal
and proceed to the next round (thus simulating a network loss/delay of the proposal).
* **TODO**: [From discussion with William] Another possibility here is to panic. What do folks think we should do here?
* The implementation of `PrepareProposal` can be non-deterministic.
#### When does Tendermint call it?
When a validator _p_ enters Tendermint consensus round _r_, height _h_, in which _p_ is the proposer,
and _p_'s _validValue_ is `nil`:
1. _p_'s Tendermint collects outstanding transactions from the mempool
* The transactions will be collected in order of priority
* Let $C$ the list of currently collected transactions
* The collection stops when any of the following conditions are met
* the mempool is empty
* the total size of transactions $\in C$ is greater than or equal to `consensusParams.block.max_bytes`
* the sum of `GasWanted` field of transactions $\in C$ is greater than or equal to
`consensusParams.block.max_gas`
* _p_'s Tendermint creates a block header.
2. _p_'s Tendermint calls `RequestPrepareProposal` with the newly generated block.
The call is synchronous: Tendermint's execution will block until the Application returns from the call.
3. The Application checks the block (header, transactions, commit info, evidences). Besides,
* in "same-block execution" mode, the Application can (and should) provide `ResponsePrepareProposal.data`,
`ResponsePrepareProposal.validator_updates`, or
`ResponsePrepareProposal.consensus_param_updates`.
* in "next-block execution" mode, _p_'s Tendermint will ignore the values for `ResponsePrepareProposal.data`,
`ResponsePrepareProposal.validator_updates`, and `ResponsePrepareProposal.consensus_param_updates`.
* in both modes, the Application can manipulate transactions
* leave transactions untouched - `TxAction = UNMODIFIED`
* add new transactions (not previously in the mempool) - `TxAction = ADDED`
* remove transactions (invalid) from the proposal and from the mempool - `TxAction = REMOVED`
* remove transactions from the proposal but not from the mempool (effectively _delaying_ them) - the
Application removes the transaction from the list
* modify transactions (e.g. aggregate them) - `TxAction = ADDED` followed by `TxAction = REMOVED`
* reorder transactions - the Application reorders transactions in the list
4. If the block is modified, the Application sets `ResponsePrepareProposal.modified` to true,
and includes the modified block in the return parameters (see the rules in section _Usage_).
The Application returns from the call.
5. _p_'s Tendermint uses the (possibly) modified block as _p_'s proposal in round _r_, height _h_.
Note that, if _p_ has a non-`nil` _validValue_, Tendermint will use it as proposal and will not call `RequestPrepareProposal`.
### ProcessProposal
#### Parameters and Types
* **Request**:
| Name | Type | Description | Field Number |
|----------------------|---------------------------------------------|------------------------------------------------------------------------------------------------------------------|--------------|
| hash | bytes | The block header's hash of the proposed block. Present for convenience (can be derived from the block header). | 1 |
| header | [Header](../core/data_structures.md#header) | The proposed block's header. | 2 |
| tx | repeated bytes | List of transactions that have been picked as part of the proposed block. | 3 |
| byzantine_validators | repeated [Evidence](#evidence) | List of evidence of validators that acted maliciously. | 4 |
| last_commit_info | [LastCommitInfo](#lastcommitinfo) | Info about the last commit, including the round , the validator list, and which ones signed the last block. | 5 |
* **Response**:
| Name | Type | Description | Field Number |
|--------|------|--------------------------------------------------|--------------|
| accept | bool | If false, the received block failed verification | 1 |
* **Usage**:
* Contains a full proposed block.
* The parameters and types of `RequestProcessProposal` are the same as `RequestPrepareProposal`
and `RequestFinalizeBlock`.
* In "same-block execution" mode, the Application will fully execute the block
as though it was handling `RequestFinalizeBlock`.
However, any resulting state changes must be kept as _canditade state_,
and the Application should be ready to
backtrack/discard it in case the decided block is different.
* The header contains the height, timestamp, and more - it exactly matches the
Tendermint block header.
* If `ResponseProcessProposal.accept` is _false_, Tendermint assumes the proposal received
is not valid.
* The implementation of `ProcessProposal` MUST be deterministic. Moreover, the value of
`ResponseProcessProposal.accept` MUST *exclusively* depend on the parameters passed in
the call to `RequestProcessProposal`, and the last committed Application state
(see [Properties](#properties) section below).
* Moreover, application implementors SHOULD always set `ResponseProcessProposal.accept` to _true_,
unless they _really_ know what the potential liveness implications of returning _false_ are.
>**TODO**: should `ResponseProcessProposal.accept` be of type `Result` rather than `bool`? (so we are able to extend the possible values in the future?)
#### When does Tendermint call it?
When a validator _p_ enters Tendermint consensus round _r_, height _h_, in which _q_ is the proposer (possibly _p_ = _q_):
1. _p_ sets up timer `ProposeTimeout`.
2. If _p_ is the proposer, _p_ executes steps 1-6 in _PrepareProposal_ (see above).
3. Upon reception of Proposal message (which contains the header) for round _r_, height _h_ from _q_, _p_'s Tendermint verifies the block header.
4. Upon reception of Proposal message, along with all the block parts, for round _r_, height _h_ from _q_, _p_'s Tendermint follows its algorithm
to check whether it should prevote for the block just received, or `nil`
5. If Tendermint should prevote for the block just received
1. Tendermint calls `RequestProcessProposal` with the block. The call is synchronous.
2. The Application checks/processes the proposed block, which is read-only, and returns true (_accept_) or false (_reject_) in `ResponseProcessProposal.accept`.
* The Application, depending on its needs, may call `ResponseProcessProposal`
* either after it has completely processed the block (the simpler case),
* or immediately (after doing some basic checks), and process the block asynchronously. In this case the Application will
not be able to reject the block, or force prevote/precommit `nil` afterwards.
3. If the returned value is
* _accept_, Tendermint prevotes on this proposal for round _r_, height _h_.
* _reject_, Tendermint prevotes `nil`.
### ExtendVote
#### Parameters and Types
* **Request**:
| Name | Type | Description | Field Number |
|--------|-------|-----------------------------------------------------------------------|--------------|
| hash | bytes | The hash of the proposed block that the vote extension is to refer to | 1 |
| height | int64 | Height of the proposed block (for sanity check). | 2 |
* **Response**:
| Name | Type | Description | Field Number |
|-----------|-------|---------------------------------------------------------------------|--------------|
| extension | bytes | Optional information that will be attached to the Precommit message | 1 |
* **Usage**:
* `RequestExtendVote.hash` corresponds to the hash of a proposed block that was made available to the application
in a previous call to `ProcessProposal` for the current height.
* `ResponseExtendVote.extension` will always be attached to a non-`nil` Precommit message. If Tendermint is to
precommit `nil`, it will not call `RequestExtendVote`.
* The Application logic that creates the extension can be non-deterministic.
#### When does Tendermint call it?
When a validator _p_ is in Tendermint consensus state _prevote_ of round _r_, height _h_, in which _q_ is the proposer; and _p_ has received
* the Proposal message _v_ for round _r_, height _h_, along with all the block parts, from _q_,
* `Prevote` messages from _2f + 1_ validators' voting power for round _r_, height _h_, prevoting for the same block _id(v)_,
then _p_'s Tendermint locks _v_ and sends a Precommit message in the following way
1. _p_'s Tendermint sets _lockedValue_ and _validValue_ to _v_, and sets _lockedRound_ and _validRound_ to _r_
2. _p_'s Tendermint calls `RequestExtendVote` with _id(v)_ (`RequestExtendVote.hash`). The call is synchronous.
3. The Application returns an array of bytes, `ResponseExtendVote.extension`, which is not interpreted by Tendermint.
4. _p_'s Tendermint includes `ResponseExtendVote.extension` as a new field in the Precommit message.
5. _p_'s Tendermint signs and broadcasts the Precommit message.
In the cases when _p_'s Tendermint is to broadcast `precommit nil` messages (either _2f+1_ `prevote nil` messages received, or _timeoutPrevote_ triggered), _p_'s Tendermint does **not** call `RequestExtendVote` and will include an empty byte array as vote extension in the `precommit nil` message.
### VerifyVoteExtension
#### Parameters and Types
* **Request**:
| Name | Type | Description | Field Number |
|-----------|-------|------------------------------------------------------------------------------------------|--------------|
| extension | bytes | Sender Application's vote extension to be validated | 1 |
| hash | bytes | The hash of the propsed block that the vote extension refers to | 2 |
| address | bytes | [Address](../core/data_structures.md#address) of the validator that signed the extension | 3 |
| height | int64 | Height of the block (for sanity check). | 4 |
* **Response**:
| Name | Type | Description | Field Number |
|--------|------|-------------------------------------------------------|--------------|
| accept | bool | If false, Application is rejecting the vote extension | 1 |
* **Usage**:
* If `ResponseVerifyVoteExtension.accept` is _false_, Tendermint will reject the whole received vote.
See the [Properties](#properties) section to understand the potential liveness implications of this.
* The implementation of `VerifyVoteExtension` MUST be deterministic. Moreover, the value of
`ResponseVerifyVoteExtension.accept` MUST *exclusively* depend on the parameters passed in
the call to `RequestVerifyVoteExtension`, and the last committed Application state
(see [Properties](#properties) section below).
* Moreover, application implementors SHOULD always set `ResponseVerifyVoteExtension.accept` to _true_,
unless they _really_ know what the potential liveness implications of returning _false_ are.
#### When does Tendermint call it?
When a validator _p_ is in Tendermint consensus round _r_, height _h_, state _prevote_ (**TODO** discuss: I think I must remove the state
from this condition, but not sure), and _p_ receives a Precommit message for round _r_, height _h_ from _q_:
1. _p_'s Tendermint calls `RequestVerifyVoteExtension`.
2. The Application returns _accept_ or _reject_ via `ResponseVerifyVoteExtension.accept`.
3. If the Application returns
* _accept_, _p_'s Tendermint will keep the received vote, together with its corresponding
vote extension in its internal data structures. It will be used to:
* calculate field _LastCommitHash_ in the header of the block proposed for height _h + 1_
(in the rounds where _p_ will be proposer).
* populate _LastCommitInfo_ in calls to `RequestPrepareProposal`, `RequestProcessProposal`,
and `RequestFinalizeBlock` in height _h + 1_.
* _reject_, _p_'s Tendermint will deem the Precommit message invalid and discard it.
### FinalizeBlock
#### Parameters and Types
* **Request**:
| Name | Type | Description | Field Number |
|----------------------|---------------------------------------------|-------------------------------------------------------------------------------------------------------------------|--------------|
| hash | bytes | The block header's hash. Present for convenience (can be derived from the block header). | 1 |
| header | [Header](../core/data_structures.md#header) | The block header. | 2 |
| tx | repeated bytes | List of transactions committed as part of the block. | 3 |
| byzantine_validators | repeated [Evidence](#evidence) | List of evidence of validators that acted maliciously. | 4 |
| last_commit_info | [LastCommitInfo](#lastcommitinfo) | Info about the last commit, including the round, and the list of validators and which ones signed the last block. | 5 |
* **Response**:
| Name | Type | Description | Field Number |
|-------------------------|----------------------------------------------|---------------------------------------------------------------------------------|--------------|
| header_events | repeated [Event](#events) | Type & Key-Value events for indexing | 1 |
| tx_result | repeated [DeliverTxResult](#delivertxresult) | List of structures containing the data resulting from executing the transaction | 2 |
| validator_updates | repeated [ValidatorUpdate](#validatorupdate) | Changes to validator set (set voting power to 0 to remove). | 3 |
| consensus_param_updates | [ConsensusParams](#consensusparams) | Changes to consensus-critical gas, size, and other parameters. | 4 |
| block_events | repeated [Event](#events) | Type & Key-Value events for indexing | 5 |
| app_data | bytes | The Merkle root hash of the application state. | 6 |
| retain_height | int64 | Blocks below this height may be removed. Defaults to `0` (retain all). | 7 |
* **Usage**:
* Contains a newly decided block.
* This method is equivalent to the call sequence `BeginBlock`, [`DeliverTx`],
`EndBlock`, `Commit` in the previous version of ABCI.
* The header contains the height, timestamp, and more - it exactly matches the
Tendermint block header.
* The Application can use `RequestFinalizeBlock.last_commit_info` and `RequestFinalizeBlock.byzantine_validators`
to determine rewards and punishments for the validators.
* The application must execute the transactions in full, in the order they appear in `RequestFinalizeBlock.tx`,
before returning control to Tendermint. Alternatively, it can commit the candidate state corresponding to the same block
previously executed via `PrepareProposal` or `ProcessProposal`.
* `ResponseFinalizeBlock.tx_result[i].Code == 0` only if the _i_-th transaction is fully valid.
* Optional `ResponseFinalizeBlock.validator_updates` triggered by block `H`. These updates affect validation
for blocks `H+1`, `H+2`, and `H+3`. Heights following a validator update are affected in the following way:
* `H+1`: `NextValidatorsHash` includes the new `validator_updates` value.
* `H+2`: The validator set change takes effect and `ValidatorsHash` is updated.
* `H+3`: `last_commit_info` is changed to include the altered validator set.
* `ResponseFinalizeBlock.consensus_param_updates` returned for block `H` apply to the consensus
params for block `H+1`. For more information on the consensus parameters,
see the [application spec entry on consensus parameters](../abci/apps.md#consensus-parameters).
* Application is expected to persist its state at the end of this call, before calling `ResponseFinalizeBlock`.
* `ResponseFinalizeBlock.app_data` contains an (optional) Merkle root hash of the application state.
* `ResponseFinalizeBlock.app_data` is included
* [in next-block execution mode] as the `Header.AppHash` in the next block.
* [in same-block execution mode] as the `Header.AppHash` in the current block. In this case,
`PrepareProposal` is required to fully execute the block and set the App hash before
returning the proposed block to Tendermint.
* `ResponseFinalizeBlock.app_data` may also be empty or hard-coded, but MUST be
**deterministic** - it must not be a function of anything that did not come from the parameters
of `RequestFinalizeBlock` and the previous committed state.
* Later calls to `Query` can return proofs about the application state anchored
in this Merkle root hash.
* Use `retain_height` with caution! If all nodes in the network remove historical
blocks then this data is permanently lost, and no new nodes will be able to
join the network and bootstrap. Historical blocks may also be required for
other purposes, e.g. auditing, replay of non-persisted heights, light client
verification, and so on.
* Just as `ProcessProposal`, the implementation of `FinalizeBlock` MUST be deterministic, since it is
making the Application's state evolve in the context of state machine replication.
* Currently, Tendermint will fill up all fields in `RequestFinalizeBlock`, even if they were
already passed on to the Application via `RequestPrepareProposal` or `RequestProcessProposal`.
If the Application is in same-block execution mode, it applies the right candidate state here
(rather than executing the whole block). In this case the Application disregards all parameters in
`RequestFinalizeBlock` except `RequestFinalizeBlock.hash`.
#### When is it called?
When a validator _p_ is in Tendermint consensus height _h_, and _p_ receives
* the Proposal message with block _v_ for a round _r_, along with all its block parts, from _q_,
which is the proposer of round _r_, height _h_,
* `Precommit` messages from _2f + 1_ validators' voting power for round _r_, height _h_,
precommitting the same block _id(v)_,
then _p_'s Tendermint decides block _v_ and finalizes consensus for height _h_ in the following way
1. _p_'s Tendermint persists _v_ as decision for height _h_.
2. _p_'s Tendermint locks the mempool -- no calls to checkTx on new transactions.
3. _p_'s Tendermint calls `RequestFinalizeBlock` with _id(v)_. The call is synchronous.
4. _p_'s Application processes block _v_, received in a previous call to `RequestProcessProposal`.
5. _p_'s Application commits and persists the state resulting from processing the block.
6. _p_'s Application calculates and returns the _AppHash_, along with an array of arrays of bytes representing the output of each of the transactions
7. _p_'s Tendermint hashes the array of transaction outputs and stores it in _ResultHash_
8. _p_'s Tendermint persists _AppHash_ and _ResultHash_
9. _p_'s Tendermint unlocks the mempool -- newly received transactions can now be checked.
10. _p_'s starts consensus for a new height _h+1_, round 0
### [Points to discuss further]
#### Byzantine proposer
>**TODO** [From Josef] We should understand the influence of equivocation on proposals in ABCI++
>
>N.B.#1: If Byzantine proposer proposes both A and B in the same round, today we might only receive A (proposal) and not B (due to p2p implementation)
>
>Sergio: some thoughts:
>
>[Thought 1] If Tendermint delivers all proposals from a Byzantine proposer _p_ in _h_ and _r_ to the App, we're vulnerable to DoS attacks,
>as _p_ can send as many as it wishes:
>
>* Assuming N.B.#1 above gets "fixed" (A and B get delivered by p2p)
>* Assuming the App fully executes the proposed block at _ProcessProposal_ time
>
>So, whenever N.B.#1 is "fixed" at p2p level, we need to ensure that only the the first proposal in round _r_, height _h_
>gets delivered to the Application. Any subsequent proposal for that round should just be used to submit evidence, but not given to the App.
>
>[Thought 2] In terms of the consensus's safety properties, as far as pure equivocation on proposals is concerned,
>I think we are OK as long as _f<n/3_.
#### `ProcessProposal`'s timeout (a.k.a. Zarko's Github comment)
>**TODO** (to discuss): `PrepareProposal` is called synchronously. `ProcessProposal` may also want to fully process the block synchronously.
>However, they stand on Tendermint's critical path, so the Tendermint's Propose timeout needs to accomodate that.
>
>Idea: Make propose timestamp (currently hardcoded to 3 secs in the Tendermint Go implementation) part of ConsensusParams,
>so the App can adjust it with its knowledge of the time may take to prepare/process the proposal.
## Data types
### DeliverTxResult
* **Fields**:
| Name | Type | Description | Field Number |
|------------|---------------------------|-----------------------------------------------------------------------|--------------|
| code | uint32 | Response code. | 1 |
| data | bytes | Result bytes, if any. | 2 |
| log | string | The output of the application's logger. **May be non-deterministic.** | 3 |
| info | string | Additional information. **May be non-deterministic.** | 4 |
| gas_wanted | int64 | Amount of gas requested for transaction. | 5 |
| gas_used | int64 | Amount of gas consumed by transaction. | 6 |
| tx_events | repeated [Event](#events) | Type & Key-Value events for indexing transactions (e.g. by account). | 7 |
| codespace | string | Namespace for the `code`. | 8 |
### TxAction
```proto
enum TxAction {
UNKNOWN = 0; // Unknown action
UNMODIFIED = 1; // The Application did not modify this transaction. Ignore new_hashes field
ADDED = 2; // The Application added this transaction. Ignore new_hashes field
REMOVED = 3; // The Application wants this transaction removed from the proposal and the mempool. Use new_hashes field if the transaction was modified
}
```
* **Usage**:
* If `Action` is UNKNOWN, a problem happened in the Application. Tendermint will ignore this transaction. **TODO** should we panic?
* If `Action` is UNMODIFIED, Tendermint includes the transaction in the proposal. Nothing to do on the mempool. Field `new_hashes` is ignored.
* If `Action` is ADDED, Tendermint includes the transaction in the proposal. The transaction is also added to the mempool and gossipped. Field `new_hashes` is ignored.
* If `Action` is REMOVED, Tendermint excludes the transaction from the proposal. The transaction is also removed from the mempool if it exists,
similar to `CheckTx` returning _false_. Tendermint can use field `new_hashes` to help client trace transactions that have been modified into other transactions.
### TransactionRecord
* **Fields**:
| Name | Type | Description | Field Number |
|------------|-----------------------|------------------------------------------------------------------|--------------|
| action | [TxAction](#txaction) | What should Tendermint do with this transaction? | 1 |
| tx | bytes | Transaction contents | 2 |
| new_hashes | repeated bytes | List of hashes of successor transactions | 3 |
* **Usage**:
* As `new_hashes` is a list, `TransactionRecord` allows to trace many-to-many modifications. Some examples:
* Transaction $t1$ modified into $t2$ is represented with these records
* $t2$ "ADDED"
* $t1$ "REMOVED"; `new_hashes` contains [$id(t2)$]
* Transaction $t1$ modified into $t2$ and $t3$ is represented with these `TransactionRecord` records
* $t2$ "ADDED"
* $t3$ "ADDED"
* $t1$ "REMOVED"; `new_hashes` contains [$id(t2)$, $id(t3)$]
* Transactions $t1$ and $t2$ aggregated into $t3$ is represented with these `TransactionRecord` records
* $t3$ "ADDED"
* $t1$ "REMOVED"; `new_hashes` contains [$id(t3)$]
* $t2$ "REMOVED"; `new_hashes` contains [$id(t3)$]
* Transactions $t1$ and $t2$ combined into $t3$ and $t4$ is represented with these `TransactionRecord` records
* $t3$ "ADDED"
* $t4$ "ADDED"
* $t1$ "REMOVED" and `new_hashes` containing [$id(t3)$, $id(t4)$]
* $t2$ "REMOVED" and `new_hashes` containing [$id(t3)$, $id(t4)$]
## Properties
### What Tendermint expects from the Application
Let $p$ and $q$ be two different correct proposers in rounds $r_p$ and $r_q$ respectively, in height $h$.
Let $s_{p,h-1}$ be $p$'s Application's state committed for height $h-1$.
Let $v_p$ (resp. $v_q$) be the block that $p$'s (resp. $q$'s) Tendermint passes on to the Application via `RequestPrepareProposal`
as proposer of round $r_p$ (resp $r_q$), height $h$.
Let $v'_p$ (resp. $v'_q$) the possibly modified block $p$'s (resp. $q$'s) Application returns via `ResponsePrepareProposal` to Tendermint.
The value proposed by $p$ can differ in two different rounds where $p$ is the proposer.
* Property 1 [`PrepareProposal`, header-changes] When the blockchain is in same-block execution mode,
$p$'s Application can change the following parameters in `ResponsePrepareProposal`:
_AppHash_, _ConsensusParams_, _ValidatorUpdates_.
Parameters _AppHash_, _ConsensusParams_, and _ValidatorUpdates_ are used by Tendermint to compute various hashes in
the block header that will finally be part of the proposal.
* Property 2 [`PrepareProposal`, no-header-changes] When the blockchain is in next-block execution
mode, $p$'s Application cannot make any changes to the following parameters in `ResponsePrepareProposal`:
_AppHash_, _ConsensusParams_, _ValidatorUpdates_.
In practical terms, Property 2 implies that Tendermint will ignore those parameters in `ResponsePrepareProposal`.
* Property 3 [`PrepareProposal`, `ProcessProposal`, coherence]: For any two correct processes $p$ and $q$,
if $q$'s Tendermint calls `RequestProcessProposal` on $v'_p$,
$q$'s Application returns Accept in `ResponseProcessProposal`.
Property 3 makes sure that blocks proposed by correct processes _always_ pass the correct receiving process's
`ProcessProposal` check.
On the other hand, if there is a deterministic bug in `PrepareProposal` or `ProcessProposal` (or in both),
strictly speaking, this makes all processes that hit the bug byzantine. This is a problem in practice,
as very often validators are running the Application from the same codebase, so potentially _all_ would
likely hit the bug at the same time. This would result in most (or all) processes prevoting `nil`, with the
serious consequences on Tendermint's liveness that this entails. Due to its criticality, Property 3 is a target for extensive testing and automated verification.
* Property 4 [`ProcessProposal`, determinism-1]: `ProcessProposal` is a (deterministic) function of the current state and the block that is about to be applied. In other words, for any correct process $p$, and any arbitrary block $v'$,
if $p$'s Tendermint calls `RequestProcessProposal` on $v'$ at height $h$,
then $p$'s Application's acceptance or rejection **exclusively** depends on $v'$ and $s_{p,h-1}$.
* Property 5 [`ProcessProposal`, determinism-2]: For any two correct processes $p$ and $q$, and any arbitrary block $v'$,
if $p$'s (resp. $q$'s) Tendermint calls `RequestProcessProposal` on $v'$ at height $h$,
then $p$'s Application accepts $v'$ if and only if $q$'s Application accepts $v'$.
Note that this Property follows from Property 4 and the Agreement property of consensus.
Properties 4 and 5 ensure that all correct processes will react in the same way to a proposed block, even
if the proposer is Byzantine. However, `ProcessProposal` may contain a bug that renders the
acceptance or rejection of the block non-deterministic, and therefore prevents processes hitting
the bug from fulfilling Properties 4 or 5 (effectively making those processes Byzantine).
In such a scenario, Tendermint's liveness cannot be guaranteed.
Again, this is a problem in practice if most validators are running the same software, as they are likely
to hit the bug at the same point. There is currently no clear solution to help with this situation, so
the Application designers/implementors must proceed very carefully with the logic/implementation
of `ProcessProposal`. As a general rule `ProcessProposal` _should_ always accept the block.
According to the Tendermint algorithm, a correct process can broadcast at most one precommit message in round $r$, height $h$.
Since, as stated in the [Description](#description) section, `ResponseExtendVote` is only called when Tendermint
is about to broadcast a non-`nil` precommit message, a correct process can only produce one vote extension in round $r$, height $h$.
Let $e^r_p$ the vote extension that the Application of a correct process $p$ returns via `ResponseExtendVote` in round $r$, height $h$.
Let $w^r_p$ the proposed block that $p$'s Tendermint passes to the Application via `RequestExtendVote` in round $r$, height $h$.
* Property 6 [`ExtendVote`, `VerifyVoteExtension`, coherence]: For any two correct processes $p$ and $q$, if $q$ receives $e^r_p$
from $p$ in height $h$, $q$'s Application returns Accept in `ResponseVerifyVoteExtension`.
Property 6 constrains the creation and handling of vote extensions in a similar way as Property 3
contrains the creation and handling of proposed blocks.
Property 6 ensures that extensions created by correct processes _always_ pass the `VerifyVoteExtension`
checks performed by correct processes receiving those extensions.
However, if there is a (deterministic) bug in `ExtendVote` or `VerifyVoteExtension` (or in both),
we will face the same liveness issues as described for Property 3, as Precommit messages with invalid vote
extensions will be discarded.
* Property 7 [`VerifyVoteExtension`, determinism-1]: For any correct process $p$,
and any arbitrary vote extension $e$, and any arbitrary block $w$,
if $p$'s (resp. $q$'s) Tendermint calls `RequestVerifyVoteExtension` on $e$ and $w$ at height $h$,
then $p$'s Application's acceptance or rejection exclusively depends on $e$, $w$ and $s_{p,h-1}$.
* Property 8 [`VerifyVoteExtension`, determinism-2]: For any two correct processes $p$ and $q$,
and any arbitrary vote extension $e$, and any arbitrary block $w$,
if $p$'s (resp. $q$'s) Tendermint calls `RequestVerifyVoteExtension` on $e$ and $w$ at height $h$,
then $p$'s Application accepts $e$ if and only if $q$'s Application accepts $e$.
Note that this property follows from Property 7 and the Agreement property of consensus.
Properties 7 and 8 ensure that the validation of vote extensions will be deterministic at all
correct processes.
Properties 7 and 8 protect against arbitrary vote extension data from Byzantine processes
similarly to Properties 4 and 5 and proposed blocks.
Properties 7 and 8 can be violated by a bug inducing non-determinism in `ExtendVote` or
`VerifyVoteExtension`. In this case liveness can be compromised.
Extra care should be put in the implementation of `ExtendVote` and `VerifyVoteExtension` and,
as a general rule, `VerifyVoteExtension` _should_ always accept the vote extension.
* Property 9 [_all_, read-only]: $p$'s calls to `RequestPrepareProposal`, `RequestProcessProposal`,
`RequestExtendVote`, and `RequestVerifyVoteExtension` at height $h$ do not modify $s_{p,h-1}$.
* Property 10 [`ExtendVote`, `FinalizeBlock`, non-dependency]: for any correct process $p$,
and any vote extension $e$ that $q$ received at height $h$, the computation of
$s{p,h}$ does not depend on $e$.
The call to correct process $p$'s `RequestFinalizeBlock` at height $h$, with block $v_{p,h}$
passed as parameter, creates state $s_{p,h}$.
Additionally, $p$'s `FinalizeBlock` creates a set of transaction results $T_{p,h}$.
>**TODO** I have left out all the "events" as they don't have any impact in safety or liveness
>(same for consensus params, and validator set)
* Property 11 [`FinalizeBlock`, determinism-1]: For any correct process $p$,
the contents of $s_{p,h}$ exclusively depend on $s_{p,h-1}$ and $v_{p,h}$.
* Property 12 [`FinalizeBlock`, determinism-2]: For any correct process $p$,
the contents of $T_{p,h}$ exclusively depend on $s_{p,h-1}$ and $v_{p,h}$.
Note that Properties 11 and 12, combined with Agreement property of consensus ensure
the Application state evolves consistently at all correct processes.
Finally, notice that neither `PrepareProposal` nor `ExtendVote` have determinism-related properties associated.
Indeed, `PrepareProposal` is not required to be deterministic:
* $v'_p$ may depend on $v_p$ and $s_{p,h-1}$, but may also depend on other values or operations.
* $v_p = v_q \nRightarrow v'_p = v'_q$.
Likewise, `ExtendVote` can also be non-deterministic:
* $e^r_p$ may depend on $w^r_p$ and $s_{p,h-1}$, but may also depend on other values or operations.
* $w^r_p = w^r_q \nRightarrow e^r_p = e^r_q$
### What the Application can expect from Tendermint
The following section uses these definitions:
* We define the _optimal case_ as a run in which (a) the system behaves synchronously, and (b) there are no Byzantine processes.
The optimal case captures the conditions that hold most of the time, but not always.
* We define the _suboptimal case_ as a run in which (a) the system behaves asynchronously in round 0, height h -- messages may be delayed,
timeouts may be triggered --, (b) it behaves synchronously in all subsequent rounds (_r>0_), and (c) there are no Byzantine processes.
The _suboptimal case_ captures a possible glitch in the network, or some sudden, sporadic performance issue in some validator
(network card glitch, thrashing peak, etc.).
* We define the _general case_ as a run in which (a) the system behaves asynchronously for a number of rounds,
(b) it behaves synchronously after some time, denoted _GST_, which is unknown to the processes,
and (c) there may be up to _f_ Byzantine processes.
#### `PrepareProposal`: Application's expectations
Given a block height _h_, process _p_'s Tendermint calls `RequestPrepareProposal` under
the following conditions:
* _p_'s Tendermint may decide at height _h_, without calling `RequestPrepareProposal`.
In the optimal case, this will happen often, as there will only be one proposer for height _h_
(the one for round 0).
* _p_'s Tendermint may call `RequestPrepareProposal` with a block with no transactions, if
_ConsensusParams_ is configured to produce empty blocks when there are outstanding transactions.
If _ConsensusParams_ is configured to avoid empty blocks, any block passed with a call to
`RequestPrepareProposal` will contain at lesat one transaction.
* In the optimal case, _p_'s Tendermint will call `RequestPrepareProposal` at most once,
as there is only one round.
* In the suboptimal case, _p_'s Tendermint
* will not call `RequestPrepareProposal` for height _h_, if _p_ is neither the proposer
of round 0, nor round 1
* will call `RequestPrepareProposal` once for height _h_, if _p_ is either the proposer
of round 0, or round 1 (but not both).
* will call `RequestPrepareProposal` twice for height _h_, in the unlikely case that
_p_ is the proposer of round 0, and round 1.
* In the general case, _p_'s Tendermint will potentially span many rounds. So, it will call
`RequestPrepareProposal` a number of times which is impossible to predict. Thus, the
Application will to return numerous _amended_ blocks via `ResponsePrepareProposal`.
If the application is fully executing the blocks it returns via `ResponsePrepareProposal`,
it should be careful about its usage of resources (memory, disk, etc.) as the number
of `PrepareProposal` for a particular height is not bounded.
If `PrepareProposal` is called more than once for a height _h_, the Application _is_ allowed
to return different blocks each time.
#### `ProcessProposal`: Application's expectations
Given a block height _h_, process _p_'s Tendermint calls `RequestProcessProposal` depending on the case:
* In the optimal case, all Tendermint processes decide in round _r=0_. Let's call _v_ the block proposed by the proposer of round _r=0_, height _h_.
Process _p_'s Application will receive
* exactly one call to `RequestPrepareProposal` if _p_ is the proposer of round 0. If that is the case, _p_ will return _v_ in its call to
`ResponsePrepareProposal`
* exactly one call to `RequestProcessProposal` for block height _h_ containing _v_.
In addition, the Application may execute _v_ as part of handling the `RequestProcessProposal` call, keep the resulting state as a candidate state,
and apply it when the call to `RequestFinalizeBlock` for _h_ confirms that _v_ is indeed the block decided in height _h_.
* In the suboptimal case, Tendermint processes may decide in round _r=0_ or in round _r=1_.
Therefore, the Application of a process _q_ may receive zero, one or two calls to `RequestProcessProposal`, depending on
whether _q_ is the proposer of round 0 or 1 of height _h_.
Likewise, the Application of a process _q_ may receive one or two calls to `RequestProcessProposal`
with two different values _v0_ and _v1_ while in height _h_.
* There is no way for the Application to predict whether proposed block _v0_ or _v1_ will be decided before `RequestFinalizeBlock` is called.
* The Application may choose to execute either (or both) proposed block as part of handling the `RequestProcessProposal`
call and keep the resulting state as a _candidate state_.
At decision time (i.e. when Tendermint calls `RequestFinalizeBlock`), if the block decided corresponds to one of the candidate states,
it can be committed, otherwise the Application will have to execute the block at this point.
* In the general case, the round in which a Tendermint processes _p_ will decide cannot be forecast.
The number of time _p_'s Tendermint may call `RequestProcessProposal` with different proposed blocks for a given height is *unbounded*.
As a result, the Application may need to deal with an unbounded number of different proposals for a given height,
and also an unbounded number of candidate states if it is fully executing the proposed blocks upon `PrepareProposal` or `ProcessProposal`.
In order protects the processes' stability (memory, CPU), the Application has to:
* Be ready to discard candidate states if they become too many. In other words, the set of candidate states should be managed like a cache.
* Be able to execute the blocks upon `FinalizeBlock` if the block decided was one whose candidate state was discarded.
#### `ExtendVote`: expectations from the Application
>**TODO** (in different rounds). [Finish first discussion above]
--
If `ProcessProposal`'s outcome is _Reject_ for some proposed block. Tendermint guarantees that the block will not be the decision.
--
The validity of every transaction in a block (from the App's point of view), as well as the hashes in its header can be guaranteed if:
* `ProcessProposal` *synchronously* handles every proposed block as though Tendermint had already decided on it.
* All the properties of `ProcessProposal` and `FinalizeBlock` mentioned above hold.
## Failure modes
>**TODO** Is it worth explaining the failure modes? Since we're going for halt, and can't configure them.
## Application modes
[This is a sketch ATM]
Mode 1: Simple mode (equivalent to ABCI).
**TODO**: Define _candidate block_
Definition: "candidate state" is the App state resulting from the optimistic exectution of a block that is not decided yet by consensus. An application managing candidate states needs to be able to discard them and recover the previous state
* PrepareProposal: Set `ResponsePrepareProposal.modified` to false and return
* ProcessProposal: keep the block data in memory (indexed by app hash) as _candidate block_ and return Accept
* ExtendVote: return empty byte array
* VerifyVoteExtension: if the byte array is empty, return Accept; else, return Reject
* Finalize block: look up the block by the app hash and fully execute it. Discard all other candidate blocks
Mode 2: Basic checks on proposals.
* PrepareProposal: Go through the transactions, apply basic app-dependent checks based on the block and last committed state. Remove/reorder invalid transactions in the block
* ProcessProposal: Same as PrepareProposal; return Reject if invalid transactions are detected. If Accept, keep the block data in memory (indexed by app hash) as _candidate block_.
* ExtendVote: return empty byte array
* VerifyVoteExtension: if the byte array is empty, return Accept; else, return Reject
* Finalize block: look up the block by the app hash and fully execute it. Discard all other candidate blocks
Mode 3: Full check of proposals. Optimistic (or immediate) execution. No candidate state management.
* PrepareProposal: fully execute the block, but discard the resulting state. Remove/reorder invalid transactions in the block.
* ProcessProposal: Same as PrepareProposal. Return Reject if invalid transactions are detected. If Accept, keep the block data in memory (indexed by app hash) as _candidate block_.
* ExtendVote: return empty byte array
* VerifyVoteExtension: if the byte array is empty, return Accept; else, return Reject
* Finalize block: look up the block by the app hash and fully execute it. Discard all other candidate blocks
This mode guarantees that no invalid transactions will ever make it into a committed block
Mode 4: Mode 3 + candidate state management.
* PrepareProposal: First Remove/reorder invalid transactions in the block. If _v_ is not in the set of candidate states, fully execute the block, add the resulting state as candidate state for value _v_, height _h_.
* ProcessProposal: Same as PrepareProposal. Return Reject if invalid transactions are detected
* ExtendVote: return empty byte array
* VerifyVoteExtension: if the byte array is empty, return Accept; else, return Reject
* Finalize block: commit candidate state corresponding to _v_, discard all other candidate states.
Mode 5: Mode 4 + AppHash for heigh _h_ is in _h_'s header
>**TODO** Mode 6: With vote extensions (?)
>**TODO**: Explain the two workflows discussed with Callum: App hash on N+1 vs App hash on N. How to capture it in ABCI++ ?

+ 184
- 0
spec/abci++/abci++_tmint_expected_behavior_002_draft.md View File

@ -0,0 +1,184 @@
---
order: 4
title: Tendermint's expected behavior
---
# Tendermint's expected behavior
>**TODO**: Try the regex-based explanation for this section.
>
> First summarize the current text into: all good vs all bad
> Then set out the rules (regex? grammar?)
This section describes what the Application can expect from Tendermint
The following section uses these definitions:
* We define the _optimal case_ as a run in which (a) the system behaves synchronously, and (b) there are no Byzantine processes.
The optimal case captures the conditions that hold most of the time, but not always.
* We define the _suboptimal case_ as a run in which (a) the system behaves asynchronously in round 0, height h -- messages may be delayed,
timeouts may be triggered --, (b) it behaves synchronously in all subsequent rounds (_r>0_), and (c) there are no Byzantine processes.
The _suboptimal case_ captures a possible glitch in the network, or some sudden, sporadic performance issue in some validator
(network card glitch, thrashing peak, etc.).
* We define the _general case_ as a run in which (a) the system behaves asynchronously for a number of rounds,
(b) it behaves synchronously after some time, denoted _GST_, which is unknown to the processes,
and (c) there may be up to _f_ Byzantine processes.
### `PrepareProposal`: Application's expectations
Given a block height _h_, process _p_'s Tendermint calls `RequestPrepareProposal` under
the following conditions:
* _p_'s Tendermint may decide at height _h_, without calling `RequestPrepareProposal`.
In the optimal case, this will happen often, as there will only be one proposer for height _h_
(the one for round 0).
* _p_'s Tendermint may call `RequestPrepareProposal` with a block with no transactions, if
_ConsensusParams_ is configured to produce empty blocks when there are outstanding transactions.
If _ConsensusParams_ is configured to avoid empty blocks, any block passed with a call to
`RequestPrepareProposal` will contain at lesat one transaction.
* In the optimal case, _p_'s Tendermint will call `RequestPrepareProposal` at most once,
as there is only one round.
* In the suboptimal case, _p_'s Tendermint
* will not call `RequestPrepareProposal` for height _h_, if _p_ is neither the proposer
of round 0, nor round 1
* will call `RequestPrepareProposal` once for height _h_, if _p_ is either the proposer
of round 0, or round 1 (but not both).
* will call `RequestPrepareProposal` twice for height _h_, in the unlikely case that
_p_ is the proposer of round 0, and round 1.
* In the general case, _p_'s Tendermint will potentially span many rounds. So, it will call
`RequestPrepareProposal` a number of times which is impossible to predict. Thus, the
Application will to return numerous _amended_ blocks via `ResponsePrepareProposal`.
If the application is fully executing the blocks it returns via `ResponsePrepareProposal`,
it should be careful about its usage of resources (memory, disk, etc.) as the number
of `PrepareProposal` for a particular height is not bounded.
If `PrepareProposal` is called more than once for a height _h_, the Application _is_ allowed
to return different blocks each time.
### `ProcessProposal`: Application's expectations
Given a block height _h_, process _p_'s Tendermint calls `RequestProcessProposal` depending on the case:
* In the optimal case, all Tendermint processes decide in round _r=0_. Let's call _v_ the block proposed by the proposer of round _r=0_, height _h_.
Process _p_'s Application will receive
* exactly one call to `RequestPrepareProposal` if _p_ is the proposer of round 0. If that is the case, _p_ will return _v_ in its call to
`ResponsePrepareProposal`
* exactly one call to `RequestProcessProposal` for block height _h_ containing _v_.
In addition, the Application may execute _v_ as part of handling the `RequestProcessProposal` call, keep the resulting state as a candidate state,
and apply it when the call to `RequestFinalizeBlock` for _h_ confirms that _v_ is indeed the block decided in height _h_.
* In the suboptimal case, Tendermint processes may decide in round _r=0_ or in round _r=1_.
Therefore, the Application of a process _q_ may receive zero, one or two calls to `RequestProcessProposal`, depending on
whether _q_ is the proposer of round 0 or 1 of height _h_.
Likewise, the Application of a process _q_ may receive one or two calls to `RequestProcessProposal`
with two different values _v0_ and _v1_ while in height _h_.
* There is no way for the Application to predict whether proposed block _v0_ or _v1_ will be decided before `RequestFinalizeBlock` is called.
* The Application may choose to execute either (or both) proposed block as part of handling the `RequestProcessProposal`
call and keep the resulting state as a _candidate state_.
At decision time (i.e. when Tendermint calls `RequestFinalizeBlock`), if the block decided corresponds to one of the candidate states,
it can be committed, otherwise the Application will have to execute the block at this point.
* In the general case, the round in which a Tendermint processes _p_ will decide cannot be forecast.
The number of time _p_'s Tendermint may call `RequestProcessProposal` with different proposed blocks for a given height is *unbounded*.
As a result, the Application may need to deal with an unbounded number of different proposals for a given height,
and also an unbounded number of candidate states if it is fully executing the proposed blocks upon `PrepareProposal` or `ProcessProposal`.
In order protects the processes' stability (memory, CPU), the Application has to:
* Be ready to discard candidate states if they become too many. In other words, the set of candidate states should be managed like a cache.
* Be able to execute the blocks upon `FinalizeBlock` if the block decided was one whose candidate state was discarded.
### `ExtendVote`: expectations from the Application
>**TODO** (in different rounds). [Finish first discussion above]
--
If `ProcessProposal`'s outcome is _Reject_ for some proposed block. Tendermint guarantees that the block will not be the decision.
--
The validity of every transaction in a block (from the App's point of view), as well as the hashes in its header can be guaranteed if:
* `ProcessProposal` *synchronously* handles every proposed block as though Tendermint had already decided on it.
* All the properties of `ProcessProposal` and `FinalizeBlock` mentioned above hold.
## [Points to discuss further]
### Byzantine proposer
>**TODO** [From Josef] We should understand the influence of equivocation on proposals in ABCI++
>
>N.B.#1: If Byzantine proposer proposes both A and B in the same round, today we might only receive A (proposal) and not B (due to p2p implementation)
>
>Sergio: some thoughts:
>
>[Thought 1] If Tendermint delivers all proposals from a Byzantine proposer _p_ in _h_ and _r_ to the App, we're vulnerable to DoS attacks,
>as _p_ can send as many as it wishes:
>
>* Assuming N.B.#1 above gets "fixed" (A and B get delivered by p2p)
>* Assuming the App fully executes the proposed block at _ProcessProposal_ time
>
>So, whenever N.B.#1 is "fixed" at p2p level, we need to ensure that only the the first proposal in round _r_, height _h_
>gets delivered to the Application. Any subsequent proposal for that round should just be used to submit evidence, but not given to the App.
>
>[Thought 2] In terms of the consensus's safety properties, as far as pure equivocation on proposals is concerned,
>I think we are OK as long as _f<n/3_.
### `ProcessProposal`'s timeout (a.k.a. Zarko's Github comment)
>**TODO** (to discuss): `PrepareProposal` is called synchronously. `ProcessProposal` may also want to fully process the block synchronously.
>However, they stand on Tendermint's critical path, so the Tendermint's Propose timeout needs to accomodate that.
>
>Idea: Make propose timestamp (currently hardcoded to 3 secs in the Tendermint Go implementation) part of ConsensusParams,
>so the App can adjust it with its knowledge of the time may take to prepare/process the proposal.
# Failure modes
>**TODO** Is it worth explaining the failure modes? Since we're going for halt, and can't configure them.
# Application modes
[This is a sketch ATM]
Mode 1: Simple mode (equivalent to ABCI).
**TODO**: Define _candidate block_
Definition: "candidate state" is the App state resulting from the optimistic exectution of a block that is not decided yet by consensus. An application managing candidate states needs to be able to discard them and recover the previous state
* PrepareProposal: Set `ResponsePrepareProposal.modified` to false and return
* ProcessProposal: keep the block data in memory (indexed by app hash) as _candidate block_ and return Accept
* ExtendVote: return empty byte array
* VerifyVoteExtension: if the byte array is empty, return Accept; else, return Reject
* Finalize block: look up the block by the app hash and fully execute it. Discard all other candidate blocks
Mode 2: Basic checks on proposals.
* PrepareProposal: Go through the transactions, apply basic app-dependent checks based on the block and last committed state. Remove/reorder invalid transactions in the block
* ProcessProposal: Same as PrepareProposal; return Reject if invalid transactions are detected. If Accept, keep the block data in memory (indexed by app hash) as _candidate block_.
* ExtendVote: return empty byte array
* VerifyVoteExtension: if the byte array is empty, return Accept; else, return Reject
* Finalize block: look up the block by the app hash and fully execute it. Discard all other candidate blocks
Mode 3: Full check of proposals. Optimistic (or immediate) execution. No candidate state management.
* PrepareProposal: fully execute the block, but discard the resulting state. Remove/reorder invalid transactions in the block.
* ProcessProposal: Same as PrepareProposal. Return Reject if invalid transactions are detected. If Accept, keep the block data in memory (indexed by app hash) as _candidate block_.
* ExtendVote: return empty byte array
* VerifyVoteExtension: if the byte array is empty, return Accept; else, return Reject
* Finalize block: look up the block by the app hash and fully execute it. Discard all other candidate blocks
This mode guarantees that no invalid transactions will ever make it into a committed block
Mode 4: Mode 3 + candidate state management.
* PrepareProposal: First Remove/reorder invalid transactions in the block. If _v_ is not in the set of candidate states, fully execute the block, add the resulting state as candidate state for value _v_, height _h_.
* ProcessProposal: Same as PrepareProposal. Return Reject if invalid transactions are detected
* ExtendVote: return empty byte array
* VerifyVoteExtension: if the byte array is empty, return Accept; else, return Reject
* Finalize block: commit candidate state corresponding to _v_, discard all other candidate states.
Mode 5: Mode 4 + AppHash for heigh _h_ is in _h_'s header
>**TODO** Mode 6: With vote extensions (?)
>**TODO**: Explain the two workflows discussed with Callum: App hash on N+1 vs App hash on N. How to capture it in ABCI++ ?

Write Preview
Loading…
Cancel
Save