* libs/common: refactor libs common 3

- move nil.go into types folder and make private
- move service & baseservice out of common into service pkg

ref #4147

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry

* libs/common: refactor libs/common 2

- move random function to there own pkg

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* change imports and usage throughout repo

* fix goimports

* add changelog entry

* libs/common: Refactor libs/common 01

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* regenerate proto files, move intslice to where its used

* update kv.KVPair(s) to kv.Pair(s)

* add changelog entry

* make intInSlice private

Refs #1771

ADR: https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-044-lite-client-with-weak-subjectivity.md

## Commits:

* add Verifier and VerifyCommitTrusting

* add two more checks

make trustLevel an option

* float32 for trustLevel

* check newHeader time

* started writing lite Client

* unify Verify methods

* ensure h2.Header.bfttime < h1.Header.bfttime + tp

* move trust checks into Verify function

* add more comments

* more docs

* started writing tests

* unbonding period failures

* tests are green

* export ErrNewHeaderTooFarIntoFuture

* make golangci happy

* test for non-adjusted headers

* more precision

* providers and stores

* VerifyHeader and VerifyHeaderAtHeight funcs

* fix compile errors

* remove lastVerifiedHeight, persist new trusted header

* sequential verification

* remove TrustedStore option

* started writing tests for light client

* cover basic cases for linear verification

* bisection tests PASS

* rename BisectingVerification to SkippingVerification

* refactor the code

* add TrustedHeader method

* consolidate sequential verification tests

* consolidate skipping verification tests

* rename trustedVals to trustedNextVals

* start writing docs

* ValidateTrustLevel func and ErrOldHeaderExpired error

* AutoClient and example tests

* fix errors

* update doc

* remove ErrNewHeaderTooFarIntoFuture

This check is unnecessary given existing a) ErrOldHeaderExpired b)
h2.Time > now checks.

* return an error if we're at more recent height

* add comments

* add LastSignedHeaderHeight method to Store

I think it's fine if Store tracks last height

* copy over proxy from old lite package

* make TrustedHeader return latest if height=0

* modify LastSignedHeaderHeight to return an error if no headers exist

* copy over proxy impl

* refactor proxy and start http lite client

* Tx and BlockchainInfo methods

* Block method

* commit method

* code compiles again

* lite client compiles

* extract updateLiteClientIfNeededTo func

* move final parts

* add placeholder for tests

* force usage of lite http client in proxy

* comment out query tests for now

* explicitly mention tp: trusting period

* verify nextVals in VerifyHeader

* refactor bisection

* move the NextValidatorsHash check into updateTrustedHeaderAndVals

+ update the comment

* add ConsensusParams method to RPC client

* add ConsensusParams to rpc/mock/client

* change trustLevel type to a new cmn.Fraction type

+ update SkippingVerification comment

* stress out trustLevel is only used for non-adjusted headers

* fixes after Fede's review

Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>

* compare newHeader with a header from an alternative provider

* save pivot header

Refs https://github.com/tendermint/tendermint/pull/3989#discussion_r349122824

* check header can still be trusted in TrustedHeader

Refs https://github.com/tendermint/tendermint/pull/3989#discussion_r349101424

* lite: update Validators and Block endpoints

- Block no longer contains BlockMeta
- Validators now accept two additional params: page and perPage

* make linter happy

Fixes #3986

This pull request is prefixing all the types in proto to avoid conflict.

When a go application is using Tendermint as a library and also define similar types in gogo proto some conflicts might occur (as types is a common package in go).

By prefixing the types with tendermint, this highly reduces the risk of conflicts.

BREAKING CHANGE.

This modification breaks the ABCI Application endpoint.

What was accessible before with `/types.ABCIApplication/Flush` is now accessible with `/tendermint.abci.types.ABCIApplication/Flush`.

*libs/common/errors: remove package

- remove errors file from cmn pkg

- use errorf instead of wrap in async function

- add changelog entry

- closes #3862

Refs #3262

* Fix long line errors in abci, crypto, and libs packages

* Fix long lines in p2p and rpc packages

* Fix long lines in abci, state, and tools packages

* Fix long lines in behaviour and blockchain packages

* Fix long lines in cmd and config packages

* Begin fixing long lines in consensus package

* Finish fixing long lines in consensus package

* Add lll exclusion for lines containing URLs

* Fix long lines in crypto package

* Fix long lines in evidence package

* Fix long lines in mempool and node packages

* Fix long lines in libs package

* Fix long lines in lite package

* Fix new long line in node package

* Fix long lines in p2p package

* Ignore gocritic warning

* Fix long lines in privval package

* Fix long lines in rpc package

* Fix long lines in scripts package

* Fix long lines in state package

* Fix long lines in tools package

* Fix long lines in types package

* Enable lll linter

protoc 3.7.0
gogo v1.3.0

* Remove omitempty from *pb.go

- remove omitempty from *pb.go files
- added command to makefile for everytime `make protoc_all` is run

- open question:
  - Do we want to further remove omitempty from other places
	- https://github.com/tendermint/tendermint/blob/master/rpc/lib/types/types.go#L151
	- and other places
ref #3882

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* bring back omitempty to *pb.go

* Update types/tx.go

* custom marshlers

* undo benchmark `omitepmty`

* golangci lint fix

* cleanup comments

* changelog_pending entry

* Pin range scope vars

* Don't disable scopelint

This PR repairs linter errors seen when running the following commands:
golangci-lint run --no-config --disable-all=true --enable=scopelint

Contributes to #3262

Refs #3262

Add gocritic as a linter

    The linting is not complete, but should i complete in this PR or in a following.

    23 files have been touched so it may be better to do in a following PR


Commits:

* Add gocritic to linting

- Added gocritic to linting

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* gocritic

* pr comments

* remove switch in cmdBatch

cleanup to add linter

    grpc change:
        https://godoc.org/google.golang.org/grpc#WithContextDialer
        https://godoc.org/google.golang.org/grpc#WithDialer
        grpc/grpc-go#2627
    prometheous change:
        due to UninstrumentedHandler, being deprecated in the future
    empty branch = empty if or else statement
        didn't delete them entirely but commented
        couldn't find a reason to have them
    could not replicate the issue #3406
        but if want to keep it commented then we should comment out the if statement as well

- The removed functions are not used in Iavl, Cosmos-sdk and tendermint repos
- Code-hygenie `whoop whoop`

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* Remove file heap.go

- cmn.Heap is not being used in cosmos-sdk, iavl nor tendermint repo.

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* changelog entry

* Update CHANGELOG_PENDING.md

closes #2432

- libs/errors: commented out errors.go
- libs/common: unused colors.go
- libs/db: unused debugDB

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* Remove deprecated PanicXXX functions from codebase

As per discussion over
[here](https://github.com/tendermint/tendermint/pull/3456#discussion_r278423492),
we need to remove these `PanicXXX` functions and eliminate our
dependence on them. In this PR, each and every `PanicXXX` function call
is replaced with a simple `panic` call.

* add a changelog entry

Fixes #3457

The topic of the issue is that : write a BlockRequest int requestsCh channel will create an timer at the same time that stop the peer 15s later if no block have been received . But pop a BlockRequest from requestsCh and send it out may delay more than 15s later. So that the peer will be stopped for error("send nothing to us").
Extracting requestsCh into its own goroutine can make sure that every BlockRequest been handled timely.

Instead of the requestsCh handling, we should probably pull the didProcessCh handling in a separate go routine since this is the one "starving" the other channel handlers. I believe the way it is right now, we still have issues with high delays in errorsCh handling that might cause sending requests to invalid/ disconnected peers.

* p2p: refactor Switch#Broadcast func

- call wg.Add only once
- do not call peers.List twice!
  * bad for perfomance
  * peers list can change in between calls!

Refs #3306

* p2p: use time.Ticker instead of RepeatTimer

no need in RepeatTimer since we don't Reset them

Refs #3306

* libs/common: remove RepeatTimer (also TimerMaker and Ticker interface)

"ancient code that’s caused no end of trouble" Ethan

I believe there's much simplier way to write a ticker than can be reset
https://medium.com/@arpith/resetting-a-ticker-in-go-63858a2c17ec

* deps: update gogo/proto from 1.1.1 to 1.2.1

- verified changes manually
  git diff 636bf030~ ba06b47c --stat -- ':!*.pb.go' ':!test'

* deps: update golang/protobuf from 1.1.0 to 1.3.0

- verified changes manually
  git diff b4deda0~ c823c79 -- ':!*.pb.go' ':!test'

* libs/common: TrapSignal accepts logger as a first parameter

 and does not block anymore
* previously it was dumping "captured ..." msg to os.Stdout
* TrapSignal should not be responsible for blocking thread of execution

Refs #3238

* exit with zero (0) code upon receiving SIGTERM/SIGINT

Refs #3238

* fix formatting in docs/app-dev/abci-cli.md

Co-Authored-By: melekes <anton.kalyaev@gmail.com>

* fix formatting in docs/app-dev/abci-cli.md

Co-Authored-By: melekes <anton.kalyaev@gmail.com>

* make govet linter pass

Refs #3262

* close PipeReader and check for err

* Enforce validators can only use the correct pubkey type

* adapt to variable renames

* Address comments from #2636

* separate updating and validation logic

* update spec

* Add test case for TestStringSliceEqual, clarify slice copying code

* Address @ebuchman's comments

* Split up testing validator update execution, and its validation

* catch up with amino changes in
https://github.com/tendermint/go-amino/pull/222

* WIP: update to amino v0.13.0

* update to fixed amino release

* Change error output format

* Update tests

* 🤦

* apply suggestion

* Fix random distribution in bitArray.PickRandom

Previously it was very biased. 63 "_" followed by a single "x" had
much greater odds of being chosen. Additionally, the last element was
skewed. This fixes that by first preproccessing the set of all true
indices, and then randomly selecting a single element from there.

This commit also makes the code here significantly simpler, and
improves test cases.

* unlock mtx right after we select true indices

also, fix potential bug in Or function

* tmlibs -> libs

* update changelog

* address some comments from review of #2298

These functions were not used anywhere within tendermint, or the
cosmos-sdk. (The functionality is already duplicated in the cosmos-sdk
types package)

* common: Delete unused functions within byteslice
* remove more unused code from strings.go and int.go
* Remove more unused code from int.go
* Fix testcase

We didn't use this code anywhere in the codebase. As such, we probably
should reduce the surface area we support. In the event that we do
in fact require 256 bit words inside of tendermint, we should adapt
the stdlibs' internal word representations, which also handles SIMD.

Inside of the SDK, a separate solution for big ints / larger words
is employed, which uses big ints. This in turn does utilize the stdlibs
SIMD support.

also, return ErrNotStarted when trying to stop a not-running service

Fixes #2320

All functions in cmap have just one code path. Thus there is not a reason
to use defer statements.

* libs: Remove usage of custom Fmt, in favor of fmt.Sprintf

Closes #2193

* Fix bug that was masked by custom Fmt!

* remove gogoproto from tools

because it's not a binary

* update protobuf version to 3.6.1 in `make get_protoc`

* update libs/common/types.pb.go and rpc/grpc/types.pb.go

* fix app tests

These test vectors are needed for comparison with the Rust implementation.
To implement this effectively, a "RandBool" method was added to cmn.Rand.

Our Tempfile was just a wrapper on ioutil that panicked instead of error.

Our Tempdir was a less safe variant of ioutil's Tempdir.

This uses the stdlib's method of creating a tempfile in our write
file atomimc method, with a few modifications. We use a 64 bit number
rather than 32 bit, and therefore a corresponding LCG. This is to
reduce collision probability. (Note we currently used 32 bytes previously,
so this is likely a concern)

We handle reseeding the LCG in such a way that multiple threads are
even less likely to reuse the same seed.

This now makes bit array functions which take in a second bit array, thread
safe. Previously there was a warning on bitarray.Update to be lock the
second parameter externally if thread safety wasrequired.
This was not done within the codebase, so it was fine to change here.

Closes #2080

Currently the top level directory contains basically all of the code
for the crypto package. This PR moves the crypto code into submodules
in a similar manner to what `golang/x/crypto` does. This improves code
organization.

Ref discussion: https://github.com/tendermint/tendermint/pull/1966

Closes #1956

Signed-off-by: Silas Davis <silas@monax.io>