* Fix random distribution in bitArray.PickRandom

Previously it was very biased. 63 "_" followed by a single "x" had
much greater odds of being chosen. Additionally, the last element was
skewed. This fixes that by first preproccessing the set of all true
indices, and then randomly selecting a single element from there.

This commit also makes the code here significantly simpler, and
improves test cases.

* unlock mtx right after we select true indices

There's a time window after we call RotateFile() where autofile#index+1
does not exist. It will be created during the next call to Write(). BUT
if somebody calls NewReader() before Write(), it will fail with "open
  /tmp/wal#index+1/wal: no such file or directory"

We must create file (either by calling gr.Head.openFile() or directly)
during NewReader() to ensure read calls succeed.

Closes #2538

also, fix potential bug in Or function

This change doubles the speed of the mempool's reaping.
Before:

BenchmarkReap-8   	    5000	    365390 ns/op	  122887 B/op

After:

BenchmarkReap-8   	   10000	    158274 ns/op	  122882 B/op

* tmlibs -> libs

* update changelog

* address some comments from review of #2298

* fix Group.RotateFile need call Flush() before rename. #2428
* fix some review issue. #2428
 refactor Group's config: replace  setting member with initial option
* fix a handwriting mistake
* fix a time window error between rename and write.
* fix a syntax mistake.
* change option name Get_ to With_
* fix review issue
* fix review issue

* handle SIGHUP explicitly inside autofile
Refs #2260
* libs: Use consistent channel suffix

Follow-up on https://github.com/tendermint/tendermint/pull/2411

These functions were not used anywhere within tendermint, or the
cosmos-sdk. (The functionality is already duplicated in the cosmos-sdk
types package)

* common: Delete unused functions within byteslice
* remove more unused code from strings.go and int.go
* Remove more unused code from int.go
* Fix testcase

Closes #2371

We didn't use this code anywhere in the codebase. As such, we probably
should reduce the surface area we support. In the event that we do
in fact require 256 bit words inside of tendermint, we should adapt
the stdlibs' internal word representations, which also handles SIMD.

Inside of the SDK, a separate solution for big ints / larger words
is employed, which uses big ints. This in turn does utilize the stdlibs
SIMD support.

Why:
original fork is abandoned and not supported anymore.

Changes:
- LevelDB 1.19 (LevelDB and Snappy are both compiled and linked statically, so while you will not need them installed on your target machine, you should have a roughly compatible version of libstdc++.)
- snappy and lz4 libs included by default

* Enforce file permissions in case they've changed

* test behaviour for autofile

* use testify in tests and rename `fInf` to `fileInfo`

* return an error if file permissions have changed

- if we can't read the file, we'll still panic

* get rid of "github.com/pkg/errors" dependency

* address review comments:

- prefix instead of suffix
- add state to err and construct formatting in Error() method

* address review comments:

- move error to libs/errors

also, return ErrNotStarted when trying to stop a not-running service

Fixes #2320

* explicitly panic if max capacity is reached

* address review comments

* comments and a test

* Implement BFT time

* set LastValidators when creating state in state helper

for heights >= 2

* libs/autofile: bring back loops

* changelog, version

Closes: #2292

* clist: Speedup detachNext() and detachPrev()

We used unnecessary function calls, defers, and extra mutexes.
These are not our friends for writing fast code in our libs.

* Remove more defers from clist functions

* Add more benchmarks

All functions in cmap have just one code path. Thus there is not a reason
to use defer statements.

* libs: Remove usage of custom Fmt, in favor of fmt.Sprintf

Closes #2193

* Fix bug that was masked by custom Fmt!

* remove gogoproto from tools

because it's not a binary

* update protobuf version to 3.6.1 in `make get_protoc`

* update libs/common/types.pb.go and rpc/grpc/types.pb.go

* fix app tests

1) no need to stop the ticker in createTestGroup() method
2) now there is a symmetry - we start the ticker in OnStart(), we stop it
in OnStop()

Refs #2072

also, remove excessive log message

Refs #2072

NOTE: from the ticker#Stop documentation:

```
Stop does not close the channel, to prevent a read from the channel
succeeding incorrectly.
https://golang.org/src/time/tick.go?s=1318:1341#L35
```

Refs #2072

These test vectors are needed for comparison with the Rust implementation.
To implement this effectively, a "RandBool" method was added to cmn.Rand.

Our Tempfile was just a wrapper on ioutil that panicked instead of error.

Our Tempdir was a less safe variant of ioutil's Tempdir.

This uses the stdlib's method of creating a tempfile in our write
file atomimc method, with a few modifications. We use a 64 bit number
rather than 32 bit, and therefore a corresponding LCG. This is to
reduce collision probability. (Note we currently used 32 bytes previously,
so this is likely a concern)

We handle reseeding the LCG in such a way that multiple threads are
even less likely to reuse the same seed.

Previously we logged `Testing for i <i>` for all i in [0,100).
This was unnecessary. This changes it to just log the value for i on
error messages, to reduce the unnecessary verbosity in log files.

This now makes bit array functions which take in a second bit array, thread
safe. Previously there was a warning on bitarray.Update to be lock the
second parameter externally if thread safety wasrequired.
This was not done within the codebase, so it was fine to change here.

Closes #2080

Currently the top level directory contains basically all of the code
for the crypto package. This PR moves the crypto code into submodules
in a similar manner to what `golang/x/crypto` does. This improves code
organization.

Ref discussion: https://github.com/tendermint/tendermint/pull/1966

Closes #1956

Signed-off-by: Silas Davis <silas@monax.io>

Signed-off-by: Silas Davis <silas@monax.io>

gogoproto.nullable compatible with GRPC downstream of ABCI and libs
protbuf types

We were computing these functions incorrectly.
I'm not sure what distribution these numbers are, but it isn't the
normal exponential distribution. (We're making the probability of
getting a number of a particular bitlength equal, but the number in
that bitlength thats gets chosen is uniformly chosen)

We weren't using these functions anywhere in our codebase, and they
had a nomenclature error. (There aren't exponentially distributed
integers, instead they would be geometrically distributed)

The godocs fell out of sync with the code here. Additionally we had
warning that these randomness functions weren't for cryptographic
use on every function. However these warnings are confusing, since
there was no implication that they would be secure there, and a
single warning on the actual Rand type would suffice. (This is what
is done in golang's math/rand godoc)

Additionally we indicated that rand.Bytes() was reading OS randomness
but in fact that had been changed.

This commit switches all usage of math/rand to cmn's rand. The only
exceptions are within the random file itself, the tools package, and the
crypto package. In tools you don't want it to lock between the go-routines.
The crypto package doesn't use it so the crypto package have no other
dependencies within tendermint/tendermint for easier portability.

Crypto/rand usage is unadjusted.

Closes #1343

Refs #693

This reverts commit d02c5d1e30.