prevent proposer from proposing duplicate pieces of evidence

The number of evidence that can be committed in a single block is capped by a new evidence parameter called MaxNum

Predominantly following the discussions regarding the conventions of using mocks, I have decided to revert back to the previous state where mocks were specialized and stored in the separate packages that used them rather then have a generalized mock in the evidence package.

This also was a problem as the state package were running tests too slow and occasionally timing out unnecessarily.

For the replay file I renamed mockEvidencePool to emptyEvidencePool to illustrate that it was intentionally like this and not give the impression that testing software was being used in production

Closes: #4786

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* format: add format cmd & goimport repo

- replaced format command
- added goimports to format command
- ran goimports

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix outliers & undo proto file changes

closes #3602

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

* test functions take time.Now and other minor changes

* updated remaining test files

* Update validation_test.go

* fix typo

* go fmt

* import time

Co-authored-by: Marko <marbar3778@yahoo.com>

* evidence: introduce time.Duration to evidence params

- add time.duration to evidence
- this pr is taking pr #2606 and updating it to use both time and height

- closes #2565

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix testing and genesis cfg in signer harness

* remove debugging fmt

* change maxageheight to maxagenumblocks, rename other things to block instead of height

* further check of duration

* check duration to not send peers outdated evidence

* change some lines, onward and upward

* refactor evidence package

* add a changelog pending entry

* make mockbadevidence have time and use it

* add what could possibly be called a test case

* remove mockbadevidence and mockgoodevidence in favor of mockevidence

* add a comment for err that is returned

* add a changelog for removal of good & bad evidence

* add a test for adding evidence

* fix test

* add ev to types in testcase

* Update evidence/pool_test.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* Update evidence/pool_test.go

Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com>

* fix tests

* fix linting

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

* types: change `Commit` to consist of just signatures

These are final changes towards removing votes from commit and leaving
only signatures (see ADR-25)

Fixes #1648

* bring back TestCommitToVoteSetWithVotesForAnotherBlockOrNilBlock

+ add absent flag to Vote to indicate that it's for another block

* encode nil votes as CommitSig with BlockIDFlagAbsent

+ make Commit#Precommits array of non-pointers
because precommit will never be nil

* add NewCommitSigAbsent and Absent() funcs

* uncomment validation in CommitSig#ValidateBasic

* add comments to ValidatorSet funcs

* add a changelog entry

* break instead of continue

continue does not make sense in these cases

* types: rename Commit#Precommits to Signatures

* swagger: fix /commit response

* swagger: change block_id_flag type

* fix merge conflicts

* Removal of TotalTx & NumTx

- Removed totalTx and numTx

closes #2521

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* abci proto changes

* proto number fix

* txfilter_test fix

* comments on PR

* further changes

* bring back metrics

* fix indexer

* fix TestBlockMaxDataBytes and TestBlockMaxDataBytesUnknownEvidence

* indexer service back to header

* statistics.go fix

* fix ci

* listen for blocks, not headers

to be able to record txs throughput

* fix TestNetworkNewBlock

* fix tests

* fix tests in types package

* fixes after Anton's review

* fix tests

* bring back `consensus_total_txs` metric

I mistakenly thought it was removed.

* improve changelog

* remove LastBlockTotalTx from state

* docs: remove getNumTxs from BeginBlock Java example

* Fix long line errors in abci, crypto, and libs packages

* Fix long lines in p2p and rpc packages

* Fix long lines in abci, state, and tools packages

* Fix long lines in behaviour and blockchain packages

* Fix long lines in cmd and config packages

* Begin fixing long lines in consensus package

* Finish fixing long lines in consensus package

* Add lll exclusion for lines containing URLs

* Fix long lines in crypto package

* Fix long lines in evidence package

* Fix long lines in mempool and node packages

* Fix long lines in libs package

* Fix long lines in lite package

* Fix new long line in node package

* Fix long lines in p2p package

* Ignore gocritic warning

* Fix long lines in privval package

* Fix long lines in rpc package

* Fix long lines in scripts package

* Fix long lines in state package

* Fix long lines in tools package

* Fix long lines in types package

* Enable lll linter

to the types package

Paritally Fixes #3584

* Expose priv validators for use in testing

* Generalize block header validation test past height 1

* Remove ineffectual assignment

* Remove redundant SaveState call

* Reorder comment for clarity

* Use the block executor ApplyBlock function instead of implementing a stripped-down version of it

* Remove commented-out code

* Remove unnecessary test

The required tests already appear to be implemented (implicitly) through
the TestValidateBlockHeader test.

* Allow for catching of specific error types during TestValidateBlockCommit

* Make return error testable

* Clean up and add TestValidateBlockCommit code

* Fix formatting

* Extract function to create a new mock test app

* Update comment for clarity

* Fix comment

* Add skeleton code for evidence-related test

* Allow for addressing priv val by address

* Generalize test beyond a single validator

* Generalize TestValidateBlockEvidence past first height

* Reorder code to clearly separate tests and utility code

* Use a common constant for stop height for testing in state/validation_test.go

* Refactor errors to resemble existing conventions

* Fix formatting

* Extract common helper functions

Having the tests littered with helper functions makes them less easily
readable imho, so I've pulled them out into a separate file. This also
makes it easier to see what helper functions are available during
testing, so we minimize the chance of duplication when writing new
tests.

* Remove unused parameter

* Remove unused parameters

* Add field keys

* Remove unused height constant

* Fix typo

* Fix incorrect return error

* Add field keys

* Use separate package for tests

This refactors all of the state package's tests into a state_test
package, so as to keep any usage of the state package's internal methods
explicit.

Any internal methods/constants used by tests are now explicitly exported
in state/export_test.go

* Refactor: extract helper function to make, validate, execute and commit a block

* Rename state function to makeState

* Remove redundant constant for number of validators

* Refactor mock evidence registration into TestMain

* Remove extraneous nVals variable

* Replace function-level TODOs with file-level TODO and explanation

* Remove extraneous comment

* Fix linting issues brought up by GolangCI (pulled in from latest merge from develop)

* make BlockTimeIota a consensus parameter, not a locally configurable option

Refs #2920

* make TimeIota int64 ms

Refs #2920

* update Gopkg.toml

* fixes after Ethan's review

* fix TestRemoteSignerProposalSigningFailed

* update changelog

* evidence: NewEvidencePool takes evidenceDB

* evidence: failing TestStoreCommitDuplicate

tendermint/security#35

* GetEvidence -> GetEvidenceInfo

* fix TestStoreCommitDuplicate

* comment in VerifyEvidence

* add check if evidence was already seen
 - modify EventPool interface (EventStore is not known in ApplyBlock):
   - add IsCommitted method to iface
 - add test

* update changelog

* fix TestStoreMark:
 - priority in evidence info gets reset to zero after evidence gets committed

* review comments: simplify EvidencePool.IsCommitted
 - delete obsolete EvidenceStore.IsCommitted

* add simple test for IsCommitted

* update changelog: this is actually breaking (PR number still missing)

* fix TestStoreMark:
 - priority in evidence info gets reset to zero after evidence gets
 committed

* review suggestion: simplify return

* consensus: createProposalBlock function

* blockExecutor.CreateProposalBlock

- factored out of consensus pkg into a method on blockExec
- new private interfaces for mempool ("txNotifier") and evpool with one function each
- consensus tests still require more mempool methods

* failing test for CreateProposalBlock

* Fix bug in include evidece into block

* evidence: change maxBytes to maxSize

* MaxEvidencePerBlock

- changed to return both the max number and the max bytes
- preparation for #2590

* changelog

* fix linter

* Fix from review

Co-Authored-By: ebuchman <ethan@coinculture.info>

* types: add Version to Header

* abci: add Version to Header

* state: add Version to State

* node: check software and state protocol versions match

* update changelog

* docs/spec: update for versions

* state: more tests

* remove TODOs

* remove empty test

* require block.Time of the fist block to be genesis time

Refs #2587:

```
We only start validating block.Time when Height > 1, because there is no
commit to compute the median timestamp from for the first block. This
means a faulty proposer could make the first block with whatever time
they want.

Instead, we should require the timestamp of block 1 to match the genesis
time.

I discovered this while refactoring the ValidateBlock tests to be
table-driven while working on tests for #2560.
```

* do not accept blocks with negative height

* update changelog and spec

* nanos precision for test genesis time

* Fix failing test (#2607)

* cap evidence in block validation

* state: use table-driven test for ValidateBlockHeader

* state: test evidence cap

* fixes from review

- state.MakeBlock takes a proposerAddr
- validateBlock only checks that the ProposerAddress is in the validator
  set
- fix raceyness from bad proposer test:
  - use privValidator to get the proposer address (instead of racy
    state)
  - note we had to remove the test that checked the correct proposer was
    included for higher rounds because we don't have a good way to test
    this with multiple consensus states and not using the
    privValidator.Address while calling createProposalBlock was a hack!

Refs #1134

Validation:

- ignored in block.ValidateBasic since it's stateful information
- checked in blockExec.ValidateBlock

This reverts commit d02c5d1e30.

This reverts commit 96a3502