* state sync: tune request timeout and chunkers (#6566)

(cherry picked from commit 7d961b55b2)

# Conflicts:
#	CHANGELOG_PENDING.md
#	config/config.go
#	internal/statesync/reactor.go
#	internal/statesync/reactor_test.go
#	node/node.go
#	statesync/syncer.go

* fix build

* fix config

* fix config

Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Aleksandr Bezobchuk <aleks.bezobchuk@gmail.com>

* version: revert version through ldflag only (#6494)

Add version back to versions, but allow it to be overridden via a ldflag.

Reason:

Many users are not setting the ldflag causing issues with tooling that relies on it (cosmjs)

closes #6488

cc @webmaster128

* revert variable rename

* Update CHANGELOG_PENDING.md

(cherry picked from commit d36a5905a6)

Co-authored-by: Sam Kleinman <garen@tychoish.com>

* rpc/jsonrpc/server: return an error in WriteRPCResponseHTTP(Error) (#6204)

instead of panicking
Closes #5529

(cherry picked from commit 00b9524168)

# Conflicts:
#	CHANGELOG_PENDING.md
#	rpc/jsonrpc/server/http_json_handler.go
#	rpc/jsonrpc/server/http_server.go
#	rpc/jsonrpc/server/http_server_test.go
#	rpc/jsonrpc/server/http_uri_handler.go

* resolve conflicts

* fix linting

* fix conflict

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
Co-authored-by: Marko Baricevic <marbar3778@yahoo.com>

* rpc/jsonrpc: Unmarshal RPCRequest correctly (#6191)

i.e. without double pointer. With double pointer, it was possible to
submit `null` value, which will crash the server.

```
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x189ddc0]

goroutine 1 [running]:
github.com/tendermint/tendermint/rpc/jsonrpc/types.(*RPCRequest).UnmarshalJSON(0xc0000147e0, 0xc00029f201, 0x4, 0x1ff, 0x883baa0, 0xc0000147e0)
        /Users/anton/go/src/github.com/tendermint/tendermint/rpc/jsonrpc/types/types.go:70 +0x100
encoding/json.(*decodeState).literalStore(0xc000216bb0, 0xc00029f201, 0x4, 0x1ff, 0x1998800, 0xc0000147e0, 0x199, 0xc000231700, 0x10e0a5e, 0x197)
        /usr/local/Cellar/go/1.16/libexec/src/encoding/json/decode.go:860 +0x30ce
encoding/json.(*decodeState).value(0xc000216bb0, 0x1998800, 0xc0000147e0, 0x199, 0x1998800, 0xc0000147e0)
        /usr/local/Cellar/go/1.16/libexec/src/encoding/json/decode.go:384 +0x40c
encoding/json.(*decodeState).array(0xc000216bb0, 0x18df040, 0xc0001be540, 0x16, 0xc000216bd8, 0x10e405b)
        /usr/local/Cellar/go/1.16/libexec/src/encoding/json/decode.go:558 +0x365
encoding/json.(*decodeState).value(0xc000216bb0, 0x18df040, 0xc0001be540, 0x16, 0x16, 0x6e)
        /usr/local/Cellar/go/1.16/libexec/src/encoding/json/decode.go:360 +0x22f
encoding/json.(*decodeState).unmarshal(0xc000216bb0, 0x18df040, 0xc0001be540, 0xc000216bd8, 0x0)
        /usr/local/Cellar/go/1.16/libexec/src/encoding/json/decode.go:180 +0x2c9
encoding/json.Unmarshal(0xc00029f200, 0x6, 0x200, 0x18df040, 0xc0001be540, 0x0, 0x0)
        /usr/local/Cellar/go/1.16/libexec/src/encoding/json/decode.go:107 +0x15d
```

(cherry picked from commit fe4e97afe0)

# Conflicts:
#	CHANGELOG_PENDING.md

* fix conflict

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

Closes #6146

Closes #6010

Fixes #5839.

Fixes #5866. Inbound traffic monitoring (and by extension inbound rate limiting) was inadvertently removed in 660e72a.

@p4u from vocdoni.io reported that the mempool might behave incorrectly under a
high load. The consequences can range from pauses between blocks to the peers
disconnecting from this node.

My current theory is that the flowrate lib we're using to control flow
(multiplex over a single TCP connection) was not designed w/ large blobs
(1MB batch of txs) in mind.

I've tried decreasing the Mempool reactor priority, but that did not
have any visible effect. What actually worked is adding a time.Sleep
into mempool.Reactor#broadcastTxRoutine after an each successful send ==
manual control flow of sort.

As a temporary remedy (until the mempool package
is refactored), the max-batch-bytes was disabled. Transactions will be sent
one by one without batching

Closes #5796

When set to true, an invalid transaction will be kept in the cache (this may help some applications to protect against spam).

NOTE: this is a temporary config option. The more correct solution would be to add a TTL to each transaction (i.e. CheckTx may return a TTL in ResponseCheckTx).

Closes: #5751

This caused stack overflow panics in E2E tests, e.g.:

```
2020-11-24T02:37:17.6085640Z validator04    | runtime: goroutine stack exceeds 1000000000-byte limit
2020-11-24T02:37:17.6087818Z validator04    | runtime: sp=0xc0234b23c0 stack=[0xc0234b2000, 0xc0434b2000]
2020-11-24T02:37:17.6088920Z validator04    | fatal error: stack overflow
2020-11-24T02:37:17.6089776Z validator04    |
2020-11-24T02:37:17.6090569Z validator04    | runtime stack:
2020-11-24T02:37:17.6091677Z validator04    | runtime.throw(0x12dc476, 0xe)
2020-11-24T02:37:17.6093123Z validator04    | 	/usr/local/go/src/runtime/panic.go:1116 +0x72
2020-11-24T02:37:17.6094320Z validator04    | runtime.newstack()
2020-11-24T02:37:17.6095374Z validator04    | 	/usr/local/go/src/runtime/stack.go:1067 +0x78d
2020-11-24T02:37:17.6096381Z validator04    | runtime.morestack()
2020-11-24T02:37:17.6097657Z validator04    | 	/usr/local/go/src/runtime/asm_amd64.s:449 +0x8f
2020-11-24T02:37:17.6098505Z validator04    |
2020-11-24T02:37:17.6099328Z validator04    | goroutine 88 [running]:
2020-11-24T02:37:17.6100470Z validator04    | runtime.heapBitsSetType(0xc009565380, 0x20, 0x18, 0x1137e00)
2020-11-24T02:37:17.6101961Z validator04    | 	/usr/local/go/src/runtime/mbitmap.go:911 +0xaa5 fp=0xc0234b23d0 sp=0xc0234b23c8 pc=0x432625
2020-11-24T02:37:17.6103906Z validator04    | runtime.mallocgc(0x20, 0x1137e00, 0x117b601, 0x11e9240)
2020-11-24T02:37:17.6105179Z validator04    | 	/usr/local/go/src/runtime/malloc.go:1090 +0x5a5 fp=0xc0234b2470 sp=0xc0234b23d0 pc=0x428b25
2020-11-24T02:37:17.6106540Z validator04    | runtime.convTslice(0xc002743710, 0x21, 0x21, 0xc0234b24e8)
2020-11-24T02:37:17.6107861Z validator04    | 	/usr/local/go/src/runtime/iface.go:385 +0x59 fp=0xc0234b24a0 sp=0xc0234b2470 pc=0x426379
2020-11-24T02:37:17.6109315Z validator04    | github.com/tendermint/tendermint/crypto/secp256k1.PubKey.String(...)
2020-11-24T02:37:17.6151692Z validator04    | 	/src/tendermint/crypto/secp256k1/secp256k1.go:161
2020-11-24T02:37:17.6153872Z validator04    | github.com/tendermint/tendermint/crypto/secp256k1.(*PubKey).String(0xc009565360, 0x11e9240, 0xc009565360)
2020-11-24T02:37:17.6157421Z validator04    | 	<autogenerated>:1 +0x65 fp=0xc0234b24f8 sp=0xc0234b24a0 pc=0x656965
2020-11-24T02:37:17.6159134Z validator04    | fmt.(*pp).handleMethods(0xc00956c680, 0x58, 0xc0234b2801)
2020-11-24T02:37:17.6161462Z validator04    | 	/usr/local/go/src/fmt/print.go:630 +0x30a fp=0xc0234b2768 sp=0xc0234b24f8 pc=0x518b8a
[...]
2020-11-24T02:37:17.6649685Z validator04    | 	/usr/local/go/src/fmt/print.go:630 +0x30a fp=0xc0234b7f48 sp=0xc0234b7cd8 pc=0x518b8a
2020-11-24T02:37:17.6651177Z validator04    | created by github.com/tendermint/tendermint/node.startStateSync
2020-11-24T02:37:17.6652521Z validator04    | 	/src/tendermint/node/node.go:587 +0x150

```

"Squashes" the changelog from RCs 2-6 into one changelog message for 0.34.0, and adds the changelog pending.

…l based on it (#5638)

Partially closes #5550

also, since multiple StopPeerForError calls may be executed in parallel,
only execute StopPeerForError once

Closes #5541

make response values non nullable in privval

Does this need a changelog for master?

Closes: #5581

cc @tarcieri

Fixes #5422. That turned out to be a whole lot easier than expected.

Backport of #5536.

Before: scheduler receives psBlockProcessed event, but does not mark block as processed because peer timed out (or was removed for other reasons) and all associated blocks were rescheduled.

After: scheduler receives psBlockProcessed event and marks block as processed in any case (even if peer who provided this block errors).

Closes #5387

When a peer is stopped due to some network issue, the Reactor calls scheduler#handleRemovePeer, which removes the peer from the scheduler. BUT the peer stays in the processor, which sometimes could lead to "duplicate block enqueued by processor" panic WHEN the same block is requested by the scheduler again from a different peer. The solution is to return scPeerError, which will be propagated to the processor. The processor will clean up the blocks associated with the peer in purgePeer.

Closes #5513, #5517

Fixes #5439. This is really a workaround for #5519 (unless we require async implementations to return ordered responses, but that kind of defeats the purpose of having an async API).

Fixes #5511.