Closes #5120

Closes #4934

* light: do not compare trusted header w/ witnesses

we don't have trusted state to bisect from

* check header before checking height

otherwise you can get nil panic

This PR removes lite & renames lite2 to light throughout the repo

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

Closes: #4944

Closes #4801

Closes: #4530

This PR contains logic for both submitting an evidence by the light client (lite2 package) and receiving it on the Tendermint side (/broadcast_evidence RPC and/or EvidenceReactor#Receive). Upon receiving the ConflictingHeadersEvidence (introduced by this PR), the Tendermint validates it, then breaks it down into smaller pieces (DuplicateVoteEvidence, LunaticValidatorEvidence, PhantomValidatorEvidence, PotentialAmnesiaEvidence). Afterwards, each piece of evidence is verified against the state of the full node and added to the pool, from which it's reaped upon block creation.

* rpc/client: do not pass height param if height ptr is nil

* rpc/core: validate incoming evidence!

* only accept ConflictingHeadersEvidence if one

of the headers is committed from this full node's perspective

This simplifies the code. Plus, if there are multiple forks, we'll
likely to receive multiple ConflictingHeadersEvidence anyway.

* swap CommitSig with Vote in LunaticValidatorEvidence

Vote is needed to validate signature

* no need to embed client

http is a provider and should not be used as a client

Closes: #4537

Uses SignedHeaderBefore to find header before unverified header and then bisection to verify the header. Only when header is between first and last trusted header height else if before the first trusted header height then regular backwards verification is used.

Closes: #4420 

Created a new error ErrInvalidHeaderwhich can be formed during the verification process verifier.go and will result in the replacement of the primary provider with a witness by executing: replacePrimaryProvider()

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

* rename adjusted to adjacent

Refs https://github.com/tendermint/tendermint/pull/3989#discussion_r352140829

* rename ErrTooMuchChange to ErrNotEnoughVotingPowerSigned

Refs https://github.com/tendermint/tendermint/pull/3989#discussion_r352142785

* verify commit is properly signed

* remove no longer trusted headers

* restore trustedHeader and trustedNextVals

* check trustedHeader using options

Refs https://github.com/tendermint/tendermint/pull/4209#issuecomment-562462165

* use correct var when checking if headers are adjacent

in bisection func
+ replace TODO with a comment

https://github.com/tendermint/tendermint/pull/3989#discussion_r352125455

* return header in VerifyHeaderAtHeight

because that way we avoid DB call

+ add godoc comments
+ check if there are no headers yet in AutoClient

https://github.com/tendermint/tendermint/pull/3989#pullrequestreview-315454506

* TestVerifyAdjacentHeaders: add 2 more test-cases

+ add TestVerifyReturnsErrorIfTrustLevelIsInvalid

* lite: avoid overflow when parsing key in db store!

* lite: rename AutoClient#Err to Errs

* lite: add a test for AutoClient

* lite: fix keyPattern and call itr.Next in db store

* lite: add two tests for db store

* lite: add TestClientRemovesNoLongerTrustedHeaders

* lite: test Client#Cleanup

* lite: test restoring trustedHeader

https://github.com/tendermint/tendermint/pull/4209#issuecomment-562462165

* lite: comment out unused code in test_helpers

* fix TestVerifyReturnsErrorIfTrustLevelIsInvalid after merge

* change defaultRemoveNoLongerTrustedHeadersPeriod

and add docs

* write more doc

* lite: uncomment testable examples

* use stdlog.Fatal to stop AutoClient tests

* make lll linter happy

* separate errors for 2 cases

- the validator set of a skipped header cannot be trusted, i.e. <1/3rd
  of h1 validator set has signed (new error, something like
  ErrNewValSetCantBeTrusted)
- the validator set is trusted but < 2/3rds has signed
  (ErrNewHeaderCantBeTrusted)

https://github.com/tendermint/tendermint/pull/4209#discussion_r360331253

* remove all headers (even the last one) that are outside

of the trusting period. By doing this, we avoid checking the
trustedHeader's hash in checkTrustedHeaderUsingOptions (case #1).

https://github.com/tendermint/tendermint/pull/4209#discussion_r360332460

* explain restoreTrustedHeaderAndNextVals better

https://github.com/tendermint/tendermint/pull/4209#discussion_r360602328

* add ConfirmationFunction option

for optionally prompting for user input Y/n before removing headers

Refs https://github.com/tendermint/tendermint/pull/4209#discussion_r360602945

* make cleaning optional

https://github.com/tendermint/tendermint/pull/4209#discussion_r364838189

* return error when user refused to remove headers

* check for double votes in VerifyCommitTrusting

* leave only ErrNewValSetCantBeTrusted error

to differenciate between h2Vals.VerifyCommit and
h1NextVals.VerifyCommitTrusting

* fix example tests

* remove unnecessary if condition

https://github.com/tendermint/tendermint/pull/4209#discussion_r365171847

It will be handled by the above switch.

* verifyCommitBasic does not depend on vals

Co-authored-by: Marko <marbar3778@yahoo.com>

Refs #1771

ADR: https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-044-lite-client-with-weak-subjectivity.md

## Commits:

* add Verifier and VerifyCommitTrusting

* add two more checks

make trustLevel an option

* float32 for trustLevel

* check newHeader time

* started writing lite Client

* unify Verify methods

* ensure h2.Header.bfttime < h1.Header.bfttime + tp

* move trust checks into Verify function

* add more comments

* more docs

* started writing tests

* unbonding period failures

* tests are green

* export ErrNewHeaderTooFarIntoFuture

* make golangci happy

* test for non-adjusted headers

* more precision

* providers and stores

* VerifyHeader and VerifyHeaderAtHeight funcs

* fix compile errors

* remove lastVerifiedHeight, persist new trusted header

* sequential verification

* remove TrustedStore option

* started writing tests for light client

* cover basic cases for linear verification

* bisection tests PASS

* rename BisectingVerification to SkippingVerification

* refactor the code

* add TrustedHeader method

* consolidate sequential verification tests

* consolidate skipping verification tests

* rename trustedVals to trustedNextVals

* start writing docs

* ValidateTrustLevel func and ErrOldHeaderExpired error

* AutoClient and example tests

* fix errors

* update doc

* remove ErrNewHeaderTooFarIntoFuture

This check is unnecessary given existing a) ErrOldHeaderExpired b)
h2.Time > now checks.

* return an error if we're at more recent height

* add comments

* add LastSignedHeaderHeight method to Store

I think it's fine if Store tracks last height

* copy over proxy from old lite package

* make TrustedHeader return latest if height=0

* modify LastSignedHeaderHeight to return an error if no headers exist

* copy over proxy impl

* refactor proxy and start http lite client

* Tx and BlockchainInfo methods

* Block method

* commit method

* code compiles again

* lite client compiles

* extract updateLiteClientIfNeededTo func

* move final parts

* add placeholder for tests

* force usage of lite http client in proxy

* comment out query tests for now

* explicitly mention tp: trusting period

* verify nextVals in VerifyHeader

* refactor bisection

* move the NextValidatorsHash check into updateTrustedHeaderAndVals

+ update the comment

* add ConsensusParams method to RPC client

* add ConsensusParams to rpc/mock/client

* change trustLevel type to a new cmn.Fraction type

+ update SkippingVerification comment

* stress out trustLevel is only used for non-adjusted headers

* fixes after Fede's review

Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>

* compare newHeader with a header from an alternative provider

* save pivot header

Refs https://github.com/tendermint/tendermint/pull/3989#discussion_r349122824

* check header can still be trusted in TrustedHeader

Refs https://github.com/tendermint/tendermint/pull/3989#discussion_r349101424

* lite: update Validators and Block endpoints

- Block no longer contains BlockMeta
- Validators now accept two additional params: page and perPage

* make linter happy