This is a follow-up from https://github.com/tendermint/tendermint/pull/2724

Closes #2761

* only invoke InitWAL/CloseWAL if WalPath is not empty

Closes #2717

* panic if WAL is not initialized when calling CloseWAL

* add a changelog entry

* mempool: ErrPreCheck and more log info

* change Pre/PostCheckFunc to return errors

also, continue execution when checking txs in mempool_test if
err=PreCheckErr

* Add additional metrics
Continues addressing https://github.com/cosmos/cosmos-sdk/issues/2169.
* Add nop metrics to fix NPE
* Tweak buckets, code review
* Update buckets
* Update docs with new metrics
* Code review updates

* remove some XXX

* config: remove Mempool.RecheckEmpty

* docs: remove recheck_empty

* mempool: update some comments

* make build_c

* docs: notes about databases and WAL files

* docs: determinism. closes #1279

* docs: small note about query paths. closes #2090

* docs: gas

* docs: abci consensus params

This also refactors the prior mempool to filter to be known as
"precheck filter" and this new filter is called "postcheck filter"

This PR also fixes a bug where the precheck filter previously didn't
account for the amino overhead, which could a maliciously sized tx to
halt blocks from getting any txs in them.

* Move maxGas outside of function definition to avoid race condition
* Type filter funcs and make public
* Use helper method for post check
* Remove superfluous Filter suffix
* Move default pre/post checks into package
* Fix broken references
* Fix typos
* Expand on examples for checks

* follow up to removing some consensus params Refs #2382
* change args type to int64 in state#makeParams
* make valsCount and evidenceCount ints again
* MaxEvidenceBytesPerBlock: include magic number in godoc
* [spec] creating a proposal
* test state#TxFilter
* panic if MaxDataBytes is less than 0
* fixes after review
* use amino#UvarintSize to calculate overhead
0c74291f3b/encoder.go (L85-L90)
* avoid cyclic imports
* you can do better Go, come on
* remove testdouble package

Closes #2399

* Make mempool aware of MaxGas requirement

* update spec

* Add tests

* Switch GasWanted from kv store to persistent kv store

* Fix typo in test name

* switch back to using kvstore, not persistent kv store

* ReapMaxTxs: return all txs if max is negative

this mirrors ReapMaxBytes behavior

See https://github.com/tendermint/tendermint/pull/2184#discussion_r214439950

* increase MaxAminoOverheadForBlock

tested with:

```
func TestMaxAminoOverheadForBlock(t *testing.T) {

        maxChainID := ""
        for i := 0; i < MaxChainIDLen; i++ {
                maxChainID += "𠜎"
        }

        h := Header{
                ChainID:            maxChainID,
                Height:             10,
                Time:               time.Now().UTC(),
                NumTxs:             100,
                TotalTxs:           200,
                LastBlockID:        makeBlockID(make([]byte, 20), 300, make([]byte, 20)),
                LastCommitHash:     tmhash.Sum([]byte("last_commit_hash")),
                DataHash:           tmhash.Sum([]byte("data_hash")),
                ValidatorsHash:     tmhash.Sum([]byte("validators_hash")),
                NextValidatorsHash: tmhash.Sum([]byte("next_validators_hash")),
                ConsensusHash:      tmhash.Sum([]byte("consensus_hash")),
                AppHash:            tmhash.Sum([]byte("app_hash")),
                LastResultsHash:    tmhash.Sum([]byte("last_results_hash")),
                EvidenceHash:       tmhash.Sum([]byte("evidence_hash")),
                ProposerAddress:    tmhash.Sum([]byte("proposer_address")),
        }
        b := Block{
                Header:     h,
                Data:       Data{Txs: makeTxs(10000, 100)},
                Evidence:   EvidenceData{},
                LastCommit: &Commit{},
        }

        bz, err := cdc.MarshalBinary(b)
        require.NoError(t, err)

        assert.Equal(t, MaxHeaderBytes+MaxAminoOverheadForBlock-2, len(bz)-1000000-20000-1)
}
```

* fix MaxYYY constants calculation

by using math.MaxInt64

See https://github.com/tendermint/tendermint/pull/2184#discussion_r214444244

* pass mempool filter as an option

See https://github.com/tendermint/tendermint/pull/2184#discussion_r214445869

* fixes after Dev's comments

See ADR 020: Limiting txs size inside a block docs/architecture/adr-020-block-size.md

Refs #2035

* mempool: Store txs by hash inside of cache

This allows for large cachesizes, without fear of the memory
consumption growing rapidly.

* (squash this) rename hashedTx -> txHash

* mempool: Keep cache hashmap and linked list in sync

This removes bugs with the linked list being full, but hashmap empty

* address PR comments

* switch clist back to list

* libs: Remove usage of custom Fmt, in favor of fmt.Sprintf

Closes #2193

* Fix bug that was masked by custom Fmt!

* #1920 try to fix race condition on proposal height for published txs

- related to create_empty_blocks=false
- published height for accepted tx can be wrong (too low)
- use the actual mempool height + 1 for the proposal
- expose Height() on mempool

* #1920 add initial test for mempool.Height()

- not sure how to test the lock
- can the mutex reference be of type Locker?
-- this way, we can use a "mock" of the mutex to test triggering

* #1920 use the ConsensusState height in favor of mempool

- gets rid of indirections
- doesn't need any "+1" magic

* #1920 cosmetic

- if we use cs.Height, it's enough to evaluate right before propose

* #1920 cleanup TODO and non-needed code

* #1920 add changelog entry

This reverts commit d02c5d1e30.

This reverts commit 96a3502

- move prometheus metrics into internal packages
- *Option structs
- misc. format changes

to better handle zero cache size

* updateToState exits early if the state isn't new, which happens after
* fast syncing. This results in not sending a NewRoundStep message. The mempool
* reactor depends on PeerState, which is updated by NewRoundStep
* messages. If the peer never sends a NewRoundStep, the mempool reactor
* will think they're behind, and never forward transactions. Note this
* only happens when `create_empty_blocks = false`, because otherwise
* peers will move through the consensus state and send a NewRoundStep
* for a new step soon anyways. Simple fix is just to send the
* NewRoundStep message during updateToState even if exit early

no longer relevant, I guess. since ABCI only defines 0 (success) code.

Refs #345

Refs #345

https://github.com/tendermint/tendermint/issues/1539#issuecomment-387240606

BEFORE:

```
./tm-bench -c 5 -r 1000 127.0.0.1:46657
Stats          Avg       StdDev     Max
Txs/sec        1826      843        2744
Blocks/sec     1.100     0.300      2
```

AFTER:

```
./tm-bench -T 30 -c 5 -r 1000 127.0.0.1:46657
Stats          Avg       StdDev     Max
Txs/sec        6120      1970       9776
Blocks/sec     1.000     0.000      1
```

https://play.golang.org/p/gN21yO9IRs3

```
func waitWithCancel(f func() *clist.CElement, ctx context.Context) *clist.CElement {
	el := make(chan *clist.CElement, 1)
	select {
	case el <- f():
```
will just run f() blockingly, so this doesn't change much in terms of behavior.

if we call it after, we might receive a "fresh" transaction from
  `broadcast_tx_sync` before old transactions (which were not
  committed).

Refs #1091

```
Commit is called with a lock on the mempool, meaning no calls to CheckTx
can start. However, since CheckTx is called async in the mempool
connection, some CheckTx might have already "sailed", when the lock is
released in the mempool and Commit proceeds.

Then, that spurious CheckTx has not yet "begun" in the ABCI app (stuck
in transport?). Instead, ABCI app manages to start to process the
Commit. Next, the spurious, "sailed" CheckTx happens in the wrong place.
```

uint64 is considered dangerous. the details will follow in a blog post.

Fixes https://github.com/tendermint/tendermint/issues/890

Add a CloseWAL method to Mempool to close the underlying WAL file
and then discard it so that further writes to it will have no effect.