Addresses one of the concerns with #7041.

Provides a mechanism (via the RPC interface) to delete a single transaction, described by its hash, from the mempool. The method returns an error if the transaction cannot be found. Once the transaction is removed it remains in the cache and cannot be resubmitted until the cache is cleared or it expires from the cache.

(cherry picked from commit 851d2e3bde)

Co-authored-by: Sam Kleinman <garen@tychoish.com>

The code in the Tendermint repository makes heavy use of import aliasing.
This is made necessary by our extensive reuse of common base package names, and
by repetition of similar names across different subdirectories.

Unfortunately we have not been very consistent about which packages we alias in
various circumstances, and the aliases we use vary. In the spirit of the advice
in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports,
his change makes an effort to clean up and normalize import aliasing.

This change makes no API or behavioral changes. It is a pure cleanup intended
o help make the code more readable to developers (including myself) trying to
understand what is being imported where.

Only unexported names have been modified, and the changes were generated and
applied mechanically with gofmt -r and comby, respecting the lexical and
syntactic rules of Go.  Even so, I did not fix every inconsistency. Where the
changes would be too disruptive, I left it alone.

The principles I followed in this cleanup are:

- Remove aliases that restate the package name.
- Remove aliases where the base package name is unambiguous.
- Move overly-terse abbreviations from the import to the usage site.
- Fix lexical issues (remove underscores, remove capitalization).
- Fix import groupings to more closely match the style guide.
- Group blank (side-effecting) imports and ensure they are commented.
- Add aliases to multiple imports with the same base package name.

The responses from node RPCs encode hash values as hexadecimal strings. This
behaviour is stipulated in our OpenAPI documentation. In some cases, however,
hashes received as JSON parameters were being decoded as byte buffers, as is
the convention for JSON.

This resulted in the confusing situation that a hash reported by one request
(e.g., broadcast_tx_commit) could not be passed as a parameter to another
(e.g., tx) via JSON, without translating the hex-encoded output hash into the
base64 encoding used by JSON for opaque bytes.

Fixes #6802.

* node: minimize hardcoded service initialization

* hacking

* nil safety

* reduce space

* remove genesis state store

* fix lint

* fix pex

* unwind some odering effects

* fix tests

* remove unused experiment

Addresses  #3931

Closes #5145

also applies to light/client

## Description

Enable errcheck linter throughout the codebase

Closes: #5059

## Description

Allow dialing of private and unconditional peers through the RPC

Closes: #1705

Closes #4549

* rpc/core: return an error if `page=0`

Closes #4942

affected endpoints:

- /validators
- /tx_search

* swagger: update doc for /unconfirmed_txs

Ethermint currently has to maintain a map height-> block hash on the store (see here) as it needs to expose the eth_getBlockByHash JSON-RPC query for Web3 compatibility. This query is currently not supported by the tendermint RPC client.

Closes https://github.com/tendermint/tendermint/issues/3857

Moves `lib/` folder to `jsonrpc/`.

Renames:

**packages**

`rpc` package -> `jsonrpc` package
`rpcclient` package -> `client` package
`rpcserver` package -> `server` package

**structs and interfaces**

```
JSONRPCClient to Client
JSONRPCRequestBatch to RequestBatch
JSONRPCCaller to Caller
```

**functions**

```
StartHTTPServer to Serve
StartHTTPAndTLSServer to ServeTLS

rpc/jsonrpc/client: rename NewURIClient to NewURI

NewJSONRPCClient to New
NewJSONRPCClientWithHTTPClient to NewWithHTTPClient
NewWSClient to NewWS
```

**misc**

- unexpose `ResponseWriterWrapper`
- remove unused http_params.go

Closes #4603

Commands used (VIM):

```
:args `rg -l errors.Wrap`
:argdo normal @q | update
```

where q is a macros rewriting the `errors.Wrap` to `fmt.Errorf`.

* rpc/client: initial split into directories

* lite2: split out test package

* rpc/client: simplify client constructurs

* updated docs

* updated changelog

I have added order_by which can be "asc" or "desc" (should be in string format) in the tx_search RPC method.

Fixes: #3333

Author: @princesinha19

* pass `outCapacity` to `eventBus#Subscribe` when subscribing using a local client

Fixes #4256

* use outCap directly

* libs/common: Refactor libs/common 4

- move byte function out of cmn to its own pkg
- move tempfile out of cmn to its own pkg
- move throttletimer to its own pkg

ref #4147

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry

* fix linting issues

Refs #1771

ADR: https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-044-lite-client-with-weak-subjectivity.md

## Commits:

* add Verifier and VerifyCommitTrusting

* add two more checks

make trustLevel an option

* float32 for trustLevel

* check newHeader time

* started writing lite Client

* unify Verify methods

* ensure h2.Header.bfttime < h1.Header.bfttime + tp

* move trust checks into Verify function

* add more comments

* more docs

* started writing tests

* unbonding period failures

* tests are green

* export ErrNewHeaderTooFarIntoFuture

* make golangci happy

* test for non-adjusted headers

* more precision

* providers and stores

* VerifyHeader and VerifyHeaderAtHeight funcs

* fix compile errors

* remove lastVerifiedHeight, persist new trusted header

* sequential verification

* remove TrustedStore option

* started writing tests for light client

* cover basic cases for linear verification

* bisection tests PASS

* rename BisectingVerification to SkippingVerification

* refactor the code

* add TrustedHeader method

* consolidate sequential verification tests

* consolidate skipping verification tests

* rename trustedVals to trustedNextVals

* start writing docs

* ValidateTrustLevel func and ErrOldHeaderExpired error

* AutoClient and example tests

* fix errors

* update doc

* remove ErrNewHeaderTooFarIntoFuture

This check is unnecessary given existing a) ErrOldHeaderExpired b)
h2.Time > now checks.

* return an error if we're at more recent height

* add comments

* add LastSignedHeaderHeight method to Store

I think it's fine if Store tracks last height

* copy over proxy from old lite package

* make TrustedHeader return latest if height=0

* modify LastSignedHeaderHeight to return an error if no headers exist

* copy over proxy impl

* refactor proxy and start http lite client

* Tx and BlockchainInfo methods

* Block method

* commit method

* code compiles again

* lite client compiles

* extract updateLiteClientIfNeededTo func

* move final parts

* add placeholder for tests

* force usage of lite http client in proxy

* comment out query tests for now

* explicitly mention tp: trusting period

* verify nextVals in VerifyHeader

* refactor bisection

* move the NextValidatorsHash check into updateTrustedHeaderAndVals

+ update the comment

* add ConsensusParams method to RPC client

* add ConsensusParams to rpc/mock/client

* change trustLevel type to a new cmn.Fraction type

+ update SkippingVerification comment

* stress out trustLevel is only used for non-adjusted headers

* fixes after Fede's review

Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>

* compare newHeader with a header from an alternative provider

* save pivot header

Refs https://github.com/tendermint/tendermint/pull/3989#discussion_r349122824

* check header can still be trusted in TrustedHeader

Refs https://github.com/tendermint/tendermint/pull/3989#discussion_r349101424

* lite: update Validators and Block endpoints

- Block no longer contains BlockMeta
- Validators now accept two additional params: page and perPage

* make linter happy

* Add pagination to /validators

- closes #3472

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add swagger params, default returns all

* address pr comments

* golint fix

* swagger default change, change to default in comment

* swagger.yaml: replace x-example with example

https://swagger.io/docs/specification/adding-examples/

* Revert "swagger.yaml: replace x-example with example"

This reverts commit 9df1b006de.

* update changelog and remove extra body close

* Fix long line errors in abci, crypto, and libs packages

* Fix long lines in p2p and rpc packages

* Fix long lines in abci, state, and tools packages

* Fix long lines in behaviour and blockchain packages

* Fix long lines in cmd and config packages

* Begin fixing long lines in consensus package

* Finish fixing long lines in consensus package

* Add lll exclusion for lines containing URLs

* Fix long lines in crypto package

* Fix long lines in evidence package

* Fix long lines in mempool and node packages

* Fix long lines in libs package

* Fix long lines in lite package

* Fix new long line in node package

* Fix long lines in p2p package

* Ignore gocritic warning

* Fix long lines in privval package

* Fix long lines in rpc package

* Fix long lines in scripts package

* Fix long lines in state package

* Fix long lines in tools package

* Fix long lines in types package

* Enable lll linter

* implement broadcast_duplicate_vote endpoint

* fix test_cover

* address comments

* address comments

* Update abci/example/kvstore/persistent_kvstore.go

Co-Authored-By: mossid <torecursedivine@gmail.com>

* Update rpc/client/main_test.go

Co-Authored-By: mossid <torecursedivine@gmail.com>

* address comments in progress

* reformat the code

* make linter happy

* make tests pass

* replace BroadcastDuplicateVote with BroadcastEvidence

* fix test

* fix endpoint name

* improve doc

* fix TestBroadcastEvidenceDuplicateVote

* Update rpc/core/evidence.go

Co-Authored-By: Thane Thomson <connect@thanethomson.com>

* add changelog entry

* fix TestBroadcastEvidenceDuplicateVote

## PR

This PR introduces a fundamental breaking change to the structure of ABCI response and tx tags and the way they're processed. Namely, the SDK can support more complex and aggregated events for distribution and slashing. In addition, block responses can include duplicate keys in events.

    Implement new Event type. An event has a type and a list of KV pairs (ie. list-of-lists). Typical events may look like:

"rewards": [{"amount": "5000uatom", "validator": "...", "recipient": "..."}]
"sender": [{"address": "...", "balance": "100uatom"}]

The events are indexed by {even.type}.{even.attribute[i].key}/.... In this case a client would subscribe or query for rewards.recipient='...'

    ABCI response types and related types now include Events []Event instead of Tags []cmn.KVPair.
    PubSub logic now publishes/matches against map[string][]string instead of map[string]string to support duplicate keys in response events (from #1385). A match is successful if the value is found in the slice of strings.

closes: #1859
closes: #2905

## Commits:

* Implement Event ABCI type and updates responses to use events

* Update messages_test.go

* Update kvstore.go

* Update event_bus.go

* Update subscription.go

* Update pubsub.go

* Update kvstore.go

* Update query logic to handle slice of strings in events

* Update Empty#Matches and unit tests

* Update pubsub logic

* Update EventBus#Publish

* Update kv tx indexer

* Update godocs

* Update ResultEvent to use slice of strings; update RPC

* Update more tests

* Update abci.md

* Check for key in validateAndStringifyEvents

* Fix KV indexer to skip empty keys

* Fix linting errors

* Update CHANGELOG_PENDING.md

* Update docs/spec/abci/abci.md

Co-Authored-By: Federico Kunze <31522760+fedekunze@users.noreply.github.com>

* Update abci/types/types.proto

Co-Authored-By: Ethan Buchman <ethan@coinculture.info>

* Update docs/spec/abci/abci.md

Co-Authored-By: Ethan Buchman <ethan@coinculture.info>

* Update libs/pubsub/query/query.go

Co-Authored-By: Ethan Buchman <ethan@coinculture.info>

* Update match function to match if ANY value matches

* Implement TestSubscribeDuplicateKeys

* Update TestMatches to include multi-key test cases

* Update events.go

* Update Query interface godoc

* Update match godoc

* Add godoc for matchValue

* DRY-up tx indexing

* Return error from PublishWithEvents in EventBus#Publish

* Update PublishEventNewBlockHeader to return an error

* Fix build

* Update events doc in ABCI

* Update ABCI events godoc

* Implement TestEventBusPublishEventTxDuplicateKeys

* Update TestSubscribeDuplicateKeys to be table-driven

* Remove mod file

* Remove markdown from events godoc

* Implement TestTxSearchDeprecatedIndexing test

I think it's nice when the Client interface has all the methods. If someone does not need a particular method/set of methods, she can use individual interfaces (e.g. NetworkClient, MempoolClient) or write her own interface.

technically breaking

Fixes #3458

* limit number of /subscribe clients and queries per client

Add the following config variables (under [rpc] section):
  * max_subscription_clients
  * max_subscriptions_per_client
  * timeout_broadcast_tx_commit

Fixes #2826

new HTTPClient interface for subscriptions

finalize HTTPClient events interface

remove EventSubscriber

fix data race

```
WARNING: DATA RACE
Read at 0x00c000a36060 by goroutine 129:
  github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1()
      /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0

Previous write at 0x00c000a36060 by goroutine 132:
  github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe()
      /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0
  github.com/tendermint/tendermint/rpc/client.WaitForOneEvent()
      /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178
  github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1()
      /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 129 (running) created at:
  github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe()
      /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7
  github.com/tendermint/tendermint/rpc/client.WaitForOneEvent()
      /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178
  github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1()
      /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Goroutine 132 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:878 +0x659
  github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync()
      /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
==================
```

lite client works (tested manually)

godoc comments

httpclient: do not close the out channel

use TimeoutBroadcastTxCommit

no timeout for unsubscribe

but 1s Local (5s HTTP) timeout for resubscribe

format code

change Subscribe#out cap to 1

and replace config vars with RPCConfig

TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout

rpc: Context as first parameter to all functions

reformat code

fixes after my own review

fixes after Ethan's review

add test stubs

fix config.toml

* fixes after manual testing

- rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes
Tendermint resources (pubsub)
- rpc: better error in Subscribe and BroadcastTxCommit
- HTTPClient: do not resubscribe if err = ErrAlreadySubscribed

* fixes after Ismail's review

* Update rpc/grpc/grpc_test.go

Co-Authored-By: melekes <anton.kalyaev@gmail.com>

* green pubsub tests :OK:

* get rid of clientToQueryMap

* Subscribe and SubscribeUnbuffered

* start adapting other pkgs to new pubsub

* nope

* rename MsgAndTags to Message

* remove TagMap

it does not bring any additional benefits

* bring back EventSubscriber

* fix test

* fix data race in TestStartNextHeightCorrectly

```
Write at 0x00c0001c7418 by goroutine 796:
  github.com/tendermint/tendermint/consensus.TestStartNextHeightCorrectly()
      /go/src/github.com/tendermint/tendermint/consensus/state_test.go:1296 +0xad
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162

Previous read at 0x00c0001c7418 by goroutine 858:
  github.com/tendermint/tendermint/consensus.(*ConsensusState).addVote()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1631 +0x1366
  github.com/tendermint/tendermint/consensus.(*ConsensusState).tryAddVote()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1476 +0x8f
  github.com/tendermint/tendermint/consensus.(*ConsensusState).handleMsg()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:667 +0xa1e
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:628 +0x794

Goroutine 796 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:878 +0x659
  testing.runTests.func1()
      /usr/local/go/src/testing/testing.go:1119 +0xa8
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
  testing.runTests()
      /usr/local/go/src/testing/testing.go:1117 +0x4ee
  testing.(*M).Run()
      /usr/local/go/src/testing/testing.go:1034 +0x2ee
  main.main()
      _testmain.go:214 +0x332

Goroutine 858 (running) created at:
  github.com/tendermint/tendermint/consensus.(*ConsensusState).startRoutines()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:334 +0x221
  github.com/tendermint/tendermint/consensus.startTestRound()
      /go/src/github.com/tendermint/tendermint/consensus/common_test.go:122 +0x63
  github.com/tendermint/tendermint/consensus.TestStateFullRound1()
      /go/src/github.com/tendermint/tendermint/consensus/state_test.go:255 +0x397
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:827 +0x162
```

* fixes after my own review

* fix formatting

* wait 100ms before kicking a subscriber out

+ a test for indexer_service

* fixes after my second review

* no timeout

* add changelog entries

* fix merge conflicts

* fix typos after Thane's review

Co-Authored-By: melekes <anton.kalyaev@gmail.com>

* reformat code

* rewrite indexer service in the attempt to fix failing test

https://github.com/tendermint/tendermint/pull/3227/#issuecomment-462316527

* Revert "rewrite indexer service in the attempt to fix failing test"

This reverts commit 0d9107a098.

* another attempt to fix indexer

* fixes after Ethan's review

* use unbuffered channel when indexing transactions

Refs https://github.com/tendermint/tendermint/pull/3227#discussion_r258786716

* add a comment for EventBus#SubscribeUnbuffered

* format code

* first commit

finalize rebase

add protoc_merkle to Makefile

* in progress

* fix kvstore

* fix tests

* remove iavl dependency

* fix tx_test

* fix test_abci_cli

fix test_apps

* fix test_apps

* fix test_cover

* rm rebase residue

* address comment in progress

* finalize rebase

Currently the top level directory contains basically all of the code
for the crypto package. This PR moves the crypto code into submodules
in a similar manner to what `golang/x/crypto` does. This improves code
organization.

Ref discussion: https://github.com/tendermint/tendermint/pull/1966

Closes #1956

This reverts commit d02c5d1e30.

This reverts commit 96a3502

* all: gofmt

Run 'gofmt -w .' from project root.

* Update changelog to say that I ran gofmt

* Revert "Update changelog to say that I ran gofmt"

This reverts commit 956f133ff0.

Refs #847

Refs #909

Refs #1296

Refs #1126