* witnesses are dropped after no response

* test witness dropout

* corrected import structure

* moved non responsiveness check to compare function

* removed dropout test as witnesses are never dropped

* created test to compare witnesses

* validate trust options

* add NewClientFromTrustedStore func

* make maxRetryAttempts an option

Closes #4370

* hash size should be equal to tmhash.Size

* make maxRetryAttempts uint

* make maxRetryAttempts uint16

maxRetryAttempts possible - 68 years

* we do not store trustingPeriod

* added test to create client from trusted store

* remove header and vals from primary

to make sure we're restoring them from the DB

Closes #4328

When TrustedHeader(height) is called, if the height is less than the trusted height but the header is not in the trusted store then a function finds the previous lowest height with a trusted header and performs a forwards sequential verification to the header of the height that was given. If no error is found it updates the trusted store with the header and validator set for that height and can then return them to the user.

Commits:

* drafted trusted header

* created function to find previous trusted height

* updates missing headers less than the trusted height

* minor cosmetic tweaks

* incorporated suggestions

* lite2: implement Backwards verification

and add SignedHeaderAfter func to Store interface

Refs https://github.com/tendermint/tendermint/issues/4328#issuecomment-581878549

* remove unused method

* write tests

* start with next height in SignedHeaderAfter func

* fix linter errors

* address Callum's comments

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

Closes issue #4338

Uses a wrapper function around both the signedHeader and validatorSet calls to the primary provider which attempts to retrieve the information 5 times before deeming the provider unavailable and replacing the primary provider with the first alternative before trying recursively again (until all alternatives are depleted)

Employs a mutex lock for any operations involving the providers of the light client to ensure no operations occurs whilst the new primary is chosen.

Commits:

* created swapProvider function

* eliminates old primary provider after replacement. Uses a mutex when changing providers

* renamed to replaceProvider

* created wrapped functions for signed header and val set

* created test for primary provider replacement

* implemented suggested revisions

* created Witnesses() and Primary()

* modified backoffAndJitterTime

* modified backoffAndJitterTime

* changed backoff base and jitter to functional arguments

* implemented suggested changes

* removed backoff function

* changed exp function to match go version

* halved the backoff time

* removed seeding and added comments

* fixed incorrect test

* extract backoff timeout calc into a function

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

* lite2: make witnesses mandatory

at least one witness is required

* lite2: return an error if there are no witnesses

https://github.com/tendermint/tendermint/pull/4358#pullrequestreview-350635444

* cmd/lite: add witnesses flag

* fix linter errors

* lite2: add Start method

There are few reasons to do that:

1) separation of state and dynamics (some users will want to delay
   starting the light client; does not matter we should not allow them
   to create a light client object)
2) less important, but some users might not need autoUpdateRoutine and
   removeNoLongerTrustedHeadersRoutine routines

* lite2: wait till routines are finished in Stop

because they are started in Start, it feels more natural to wait for
them to finish in Stop.

* lite2: add TrustedValidatorSet func

* lite2: advance to latest header

without any exponential steps

rename autoUpdate to autoUpdateRoutine

* lite2: wait in Cleanup until goroutines finished running

* lite2: move AutoClient into Client

Most of the users will want auto update feature, so it makes sense to
move it into the Client itself, rather than having a separate
abstraction (it makes the code cleaner, but introduces an extra thing
the user will need to learn).

Also, add `FirstTrustedHeight` func to Client to get first trusted height.

* fix db store tests

* separate examples for auto and manual clients

* AutoUpdate tries to update to latest state

NOT 1 header at a time

* fix errors

* lite2: make Logger an option

remove SetLogger func

* fix lite cmd

* lite2: make concurrency assumptions explicit

* fixes after my own review

* no need for nextHeightFn

sequence func will download intermediate headers

* correct comment

* rename adjusted to adjacent

Refs https://github.com/tendermint/tendermint/pull/3989#discussion_r352140829

* rename ErrTooMuchChange to ErrNotEnoughVotingPowerSigned

Refs https://github.com/tendermint/tendermint/pull/3989#discussion_r352142785

* verify commit is properly signed

* remove no longer trusted headers

* restore trustedHeader and trustedNextVals

* check trustedHeader using options

Refs https://github.com/tendermint/tendermint/pull/4209#issuecomment-562462165

* use correct var when checking if headers are adjacent

in bisection func
+ replace TODO with a comment

https://github.com/tendermint/tendermint/pull/3989#discussion_r352125455

* return header in VerifyHeaderAtHeight

because that way we avoid DB call

+ add godoc comments
+ check if there are no headers yet in AutoClient

https://github.com/tendermint/tendermint/pull/3989#pullrequestreview-315454506

* TestVerifyAdjacentHeaders: add 2 more test-cases

+ add TestVerifyReturnsErrorIfTrustLevelIsInvalid

* lite: avoid overflow when parsing key in db store!

* lite: rename AutoClient#Err to Errs

* lite: add a test for AutoClient

* lite: fix keyPattern and call itr.Next in db store

* lite: add two tests for db store

* lite: add TestClientRemovesNoLongerTrustedHeaders

* lite: test Client#Cleanup

* lite: test restoring trustedHeader

https://github.com/tendermint/tendermint/pull/4209#issuecomment-562462165

* lite: comment out unused code in test_helpers

* fix TestVerifyReturnsErrorIfTrustLevelIsInvalid after merge

* change defaultRemoveNoLongerTrustedHeadersPeriod

and add docs

* write more doc

* lite: uncomment testable examples

* use stdlog.Fatal to stop AutoClient tests

* make lll linter happy

* separate errors for 2 cases

- the validator set of a skipped header cannot be trusted, i.e. <1/3rd
  of h1 validator set has signed (new error, something like
  ErrNewValSetCantBeTrusted)
- the validator set is trusted but < 2/3rds has signed
  (ErrNewHeaderCantBeTrusted)

https://github.com/tendermint/tendermint/pull/4209#discussion_r360331253

* remove all headers (even the last one) that are outside

of the trusting period. By doing this, we avoid checking the
trustedHeader's hash in checkTrustedHeaderUsingOptions (case #1).

https://github.com/tendermint/tendermint/pull/4209#discussion_r360332460

* explain restoreTrustedHeaderAndNextVals better

https://github.com/tendermint/tendermint/pull/4209#discussion_r360602328

* add ConfirmationFunction option

for optionally prompting for user input Y/n before removing headers

Refs https://github.com/tendermint/tendermint/pull/4209#discussion_r360602945

* make cleaning optional

https://github.com/tendermint/tendermint/pull/4209#discussion_r364838189

* return error when user refused to remove headers

* check for double votes in VerifyCommitTrusting

* leave only ErrNewValSetCantBeTrusted error

to differenciate between h2Vals.VerifyCommit and
h1NextVals.VerifyCommitTrusting

* fix example tests

* remove unnecessary if condition

https://github.com/tendermint/tendermint/pull/4209#discussion_r365171847

It will be handled by the above switch.

* verifyCommitBasic does not depend on vals

Co-authored-by: Marko <marbar3778@yahoo.com>

Refs #1771

ADR: https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-044-lite-client-with-weak-subjectivity.md

## Commits:

* add Verifier and VerifyCommitTrusting

* add two more checks

make trustLevel an option

* float32 for trustLevel

* check newHeader time

* started writing lite Client

* unify Verify methods

* ensure h2.Header.bfttime < h1.Header.bfttime + tp

* move trust checks into Verify function

* add more comments

* more docs

* started writing tests

* unbonding period failures

* tests are green

* export ErrNewHeaderTooFarIntoFuture

* make golangci happy

* test for non-adjusted headers

* more precision

* providers and stores

* VerifyHeader and VerifyHeaderAtHeight funcs

* fix compile errors

* remove lastVerifiedHeight, persist new trusted header

* sequential verification

* remove TrustedStore option

* started writing tests for light client

* cover basic cases for linear verification

* bisection tests PASS

* rename BisectingVerification to SkippingVerification

* refactor the code

* add TrustedHeader method

* consolidate sequential verification tests

* consolidate skipping verification tests

* rename trustedVals to trustedNextVals

* start writing docs

* ValidateTrustLevel func and ErrOldHeaderExpired error

* AutoClient and example tests

* fix errors

* update doc

* remove ErrNewHeaderTooFarIntoFuture

This check is unnecessary given existing a) ErrOldHeaderExpired b)
h2.Time > now checks.

* return an error if we're at more recent height

* add comments

* add LastSignedHeaderHeight method to Store

I think it's fine if Store tracks last height

* copy over proxy from old lite package

* make TrustedHeader return latest if height=0

* modify LastSignedHeaderHeight to return an error if no headers exist

* copy over proxy impl

* refactor proxy and start http lite client

* Tx and BlockchainInfo methods

* Block method

* commit method

* code compiles again

* lite client compiles

* extract updateLiteClientIfNeededTo func

* move final parts

* add placeholder for tests

* force usage of lite http client in proxy

* comment out query tests for now

* explicitly mention tp: trusting period

* verify nextVals in VerifyHeader

* refactor bisection

* move the NextValidatorsHash check into updateTrustedHeaderAndVals

+ update the comment

* add ConsensusParams method to RPC client

* add ConsensusParams to rpc/mock/client

* change trustLevel type to a new cmn.Fraction type

+ update SkippingVerification comment

* stress out trustLevel is only used for non-adjusted headers

* fixes after Fede's review

Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>

* compare newHeader with a header from an alternative provider

* save pivot header

Refs https://github.com/tendermint/tendermint/pull/3989#discussion_r349122824

* check header can still be trusted in TrustedHeader

Refs https://github.com/tendermint/tendermint/pull/3989#discussion_r349101424

* lite: update Validators and Block endpoints

- Block no longer contains BlockMeta
- Validators now accept two additional params: page and perPage

* make linter happy