diff --git a/mempool/docs/README.md b/mempool/docs/README.md new file mode 100644 index 000000000..138b287a5 --- /dev/null +++ b/mempool/docs/README.md @@ -0,0 +1,11 @@ +# Mempool Specification + +This package contains documents specifying the functionality +of the mempool module. + +Components: + +* [Config](./config.md) - how to configure it +* [External Messages](./messages.md) - The messages we accept over p2p and rpc interfaces +* [Functionality](./functionality.md) - high-level description of the functionality it provides +* [Concurrency Model](./concurrency.md) - What guarantees we provide, what locks we require. diff --git a/mempool/docs/concurrency.md b/mempool/docs/concurrency.md new file mode 100644 index 000000000..991113e6d --- /dev/null +++ b/mempool/docs/concurrency.md @@ -0,0 +1,8 @@ +# Mempool Concurrency + +Look at the concurrency model this uses... + +* Receiving CheckTx +* Broadcasting new tx +* Interfaces with consensus engine, reap/update while checking +* Calling the ABCI app (ordering. callbacks. how proxy works alongside the blockchain proxy which actually writes blocks) diff --git a/mempool/docs/config.md b/mempool/docs/config.md new file mode 100644 index 000000000..776149ba0 --- /dev/null +++ b/mempool/docs/config.md @@ -0,0 +1,59 @@ +# Mempool Configuration + +Here we describe configuration options around mempool. +For the purposes of this document, they are described +as command-line flags, but they can also be passed in as +environmental variables or in the config.toml file. The +following are all equivalent: + +Flag: `--mempool.recheck_empty=false` + +Environment: `TM_MEMPOOL_RECHECK_EMPTY=false` + +Config: +``` +[mempool] +recheck_empty = false +``` + + +## Recheck + +`--mempool.recheck=false` (default: true) + +`--mempool.recheck_empty=false` (default: true) + +Recheck determines if the mempool rechecks all pending +transactions after a block was committed. Once a block +is committed, the mempool removes all valid transactions +that were successfully included in the block. + +If `recheck` is true, then it will rerun CheckTx on +all remaining transactions with the new block state. + +If the block contained no transactions, it will skip the +recheck unless `recheck_empty` is true. + +## Broadcast + +`--mempool.broadcast=false` (default: true) + +Determines whether this node gossips any valid transactions +that arrive in mempool. Default is to gossip anything that +passes checktx. If this is disabled, transactions are not +gossiped, but instead stored locally and added to the next +block this node is the proposer. + +## WalDir + +`--mempool.wal_dir=/tmp/gaia/mempool.wal` (default: $TM_HOME/data/mempool.wal) + +This defines the directory where mempool writes the write-ahead +logs. These files can be used to reload unbroadcasted +transactions if the node crashes. + +If the directory passed in is an absolute path, the wal file is +created there. If the directory is a relative path, the path is +appended to home directory of the tendermint process to +generate an absolute path to the wal directory +(default `$HOME/.tendermint` or set via `TM_HOME` or `--home``) diff --git a/mempool/docs/functionality.md b/mempool/docs/functionality.md new file mode 100644 index 000000000..85c3dc58d --- /dev/null +++ b/mempool/docs/functionality.md @@ -0,0 +1,37 @@ +# Mempool Functionality + +The mempool maintains a list of potentially valid transactions, +both to broadcast to other nodes, as well as to provide to the +consensus reactor when it is selected as the block proposer. + +There are two sides to the mempool state: + +* External: get, check, and broadcast new transactions +* Internal: return valid transaction, update list after block commit + + +## External functionality + +External functionality is exposed via network interfaces +to potentially untrusted actors. + +* CheckTx - triggered via RPC or P2P +* Broadcast - gossip messages after a successful check + +## Internal functionality + +Internal functionality is exposed via method calls to other +code compiled into the tendermint binary. + +* Reap - get tx to propose in next block +* Update - remove tx that were included in last block +* ABCI.CheckTx - call ABCI app to validate the tx + +What does it provide the consensus reactor? +What guarantees does it need from the ABCI app? +(talk about interleaving processes in concurrency) + +## Optimizations + +Talk about the LRU cache to make sure we don't process any +tx that we have seen before diff --git a/mempool/docs/messages.md b/mempool/docs/messages.md new file mode 100644 index 000000000..5bd1d1e55 --- /dev/null +++ b/mempool/docs/messages.md @@ -0,0 +1,60 @@ +# Mempool Messages + +## P2P Messages + +There is currently only one message that Mempool broadcasts +and receives over the p2p gossip network (via the reactor): +`TxMessage` + +```go +// TxMessage is a MempoolMessage containing a transaction. +type TxMessage struct { + Tx types.Tx +} +``` + +TxMessage is go-wire encoded and prepended with `0x1` as a +"type byte". This is followed by a go-wire encoded byte-slice. +Prefix of 40=0x28 byte tx is: `0x010128...` followed by +the actual 40-byte tx. Prefix of 350=0x015e byte tx is: +`0x0102015e...` followed by the actual 350 byte tx. + +(Please see the [go-wire repo](https://github.com/tendermint/go-wire#an-interface-example) for more information) + +## RPC Messages + +Mempool exposes `CheckTx([]byte)` over the RPC interface. + +It can be posted via `broadcast_commit`, `broadcast_sync` or +`broadcast_async`. They all parse a message with one argument, +`"tx": "HEX_ENCODED_BINARY"` and differ in only how long they +wait before returning (sync makes sure CheckTx passes, commit +makes sure it was included in a signed block). + +Request (`POST http://gaia.zone:46657/`): +```json +{ + "id": "", + "jsonrpc": "2.0", + "method": "broadcast_sync", + "params": { + "tx": "F012A4BC68..." + } +} +``` + + +Response: +```json +{ + "error": "", + "result": { + "hash": "E39AAB7A537ABAA237831742DCE1117F187C3C52", + "log": "", + "data": "", + "code": 0 + }, + "id": "", + "jsonrpc": "2.0" +} +```