This change fixes a potential exploitable vulnerability
that can cause the WAL to be consistently truncated by falsely
supplying the WAL path which would be any arbitrary dirrectory.

Fixes #6427

#5852 fixed an issue with error propagation in `os.EnsureDir()`. However, this function is basically identical to `os.MkdirAll()`, and can be replaced entirely with a call to it. We keep the function for backwards compatibility.

Fixes #5839.

Remove unused ReadFile (unused) and
WriteFile (almost unused, alias of ioutil.WriteFile).

Add testcases for Must{Read,Write}File.

Closes: #4578

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>

allow ReapX and CheckTx functions to run in parallel, making it not possible to block certain proposers from creating a new block.

Closes: #2972

* libs/common: Refactor libs/common 5

- move mathematical functions and types out of `libs/common` to math pkg
- move net functions out of `libs/common` to net pkg
- move string functions out of `libs/common` to strings pkg
- move async functions out of `libs/common` to async pkg
- move bit functions out of `libs/common` to bits pkg
- move cmap functions out of `libs/common` to cmap pkg
- move os functions out of `libs/common` to os pkg

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* fix testing issues

* fix tests

closes #41417

woooooooooooooooooo kill the cmn pkg

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add changelog entry

* fix goimport issues

* run gofmt

Refs #3262

- The removed functions are not used in Iavl, Cosmos-sdk and tendermint repos
- Code-hygenie `whoop whoop`

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* libs/common: TrapSignal accepts logger as a first parameter

 and does not block anymore
* previously it was dumping "captured ..." msg to os.Stdout
* TrapSignal should not be responsible for blocking thread of execution

Refs #3238

* exit with zero (0) code upon receiving SIGTERM/SIGINT

Refs #3238

* fix formatting in docs/app-dev/abci-cli.md

Co-Authored-By: melekes <anton.kalyaev@gmail.com>

* fix formatting in docs/app-dev/abci-cli.md

Co-Authored-By: melekes <anton.kalyaev@gmail.com>

* libs: Remove usage of custom Fmt, in favor of fmt.Sprintf

Closes #2193

* Fix bug that was masked by custom Fmt!

This reverts commit d02c5d1e30.

* use increment and decrement operators.

* remove unnecessary else branches.

* fix receiver names.

* remove omittable code.

* fix dot imports.

Must close file before rename it.

We can make the implementation more robust by adjusting our assumptions
and leverage explicit file modes for syncing. Additionally we going to
assume that we want to clean up and can't really recover if thos
operations (file close and removal) fail.

* utilise file mode for majority of concerns
* improve test coverage by covering more assumptions
* signature parity with ioutil.WriteFile
* always clean up

Replaces #160

We can make the implementation more robust by adjusting our assumptions
and leverage explicit file modes for syncing. Additionally we going to
assume that we want to clean up and can't really recover if thos
operations (file close and removal) fail.

* utilise file mode for majority of concerns
* improve test coverage by covering more assumptions
* signature parity with ioutil.WriteFile
* always clean up

Replaces #160

otherwise it could try to execute go binary and panic if no go binary
found. See https://github.com/tendermint/tendermint/issues/782

Currently IsDirEmpty returns true, err if it encounters
any error after trying to os.Open the directory.
I noticed this while studying the code and recalled a bug
from an earlier project in which doing the exact same thing
on code without permissions would trip out and falsely report
that the directory was empty.
Given demo.go in https://play.golang.org/p/vhTPU2RiCJ

* Demo:
```shell
$ mkdir -p sample-demo/1 && touch sample-demo/2
$ echo "1st round" && go run demo.go sample-demo
$ sudo chown root sample-demo && sudo chmod 0700 sample-demo
$ echo "2nd round" && go run demo.go sample-demo
```

That then prints out
```shell
1st round
original:: empty: false err: <nil>
updated::  empty: false err: <nil>
2nd round
original:: empty: true err: open data/: permission denied
updated::  empty: false err: open data/: permission denied
```

where in "2nd round", the original code falsely reports that
the directory is empty but that's a permission error.

I could write a code test for it, but that test requires me to change
users and switch to root as a Go user so no point in complicating our
tests, but otherwise it is a 1-to-1 translation between shell and Go.

We use WriteFileAtomic in two places:

```
p2p/addrbook.go
338:    err = cmn.WriteFileAtomic(filePath, jsonBytes, 0644)

types/priv_validator.go
162:    err = WriteFileAtomic(privVal.filePath, jsonBytes, 0600)
```

and we don't need .bak in any of the above. We save priv_validator every
10ms and addrbook every 2 min.

The problem with {filePath}.new is that it is not safe for concurrent
use! Calling this function with the same params results in the following
error:

```
panic: Panicked on a Crisis: rename /root/.tendermint_test/consensus_replay_test/priv_validator.json.new /root/.tendermint_test/consensus_replay_test/priv_validator.json: no such file or directory

goroutine 47860 [running]:
github.com/tendermint/tendermint/vendor/github.com/tendermint/tmlibs/common.PanicCrisis(0xcba800, 0xc42152d640)
	/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/tmlibs/common/errors.go:33 +0x10f
github.com/tendermint/tendermint/types.(*PrivValidator).save(0xc42235f2c0)
	/go/src/github.com/tendermint/tendermint/types/priv_validator.go:165 +0x159
github.com/tendermint/tendermint/types.(*PrivValidator).signBytesHRS(0xc42235f2c0, 0x6, 0x0, 0xc424e88f03, 0xc429908580, 0xca, 0x155, 0x80, 0xc424e88f00, 0x7f4ecafc88d0, ...)
	/go/src/github.com/tendermint/tendermint/types/priv_validator.go:249 +0x2bb
github.com/tendermint/tendermint/types.(*PrivValidator).SignVote(0xc42235f2c0, 0xc4228c7460, 0xf, 0xc424e88f00, 0x0, 0x0)
	/go/src/github.com/tendermint/tendermint/types/priv_validator.go:186 +0x1a2
github.com/tendermint/tendermint/consensus.(*ConsensusState).signVote(0xc424efd520, 0xc400000002, 0xc422d5e3c0, 0x14, 0x20, 0x1, 0xc4247b6560, 0x14, 0x20, 0x0, ...)
	github.com/tendermint/tendermint/consensus/_test/_obj_test/state.go:1556 +0x35e
github.com/tendermint/tendermint/consensus.(*ConsensusState).signAddVote(0xc424efd520, 0x2, 0xc422d5e3c0, 0x14, 0x20, 0x1, 0xc4247b6560, 0x14, 0x20, 0xc42001b300)
	github.com/tendermint/tendermint/consensus/_test/_obj_test/state.go:1568 +0x200
github.com/tendermint/tendermint/consensus.(*ConsensusState).enterPrecommit(0xc424efd520, 0x6, 0x0)
	github.com/tendermint/tendermint/consensus/_test/_obj_test/state.go:1082 +0x13a4
github.com/tendermint/tendermint/consensus.(*ConsensusState).addVote(0xc424efd520, 0xc424e88780, 0x0, 0x0, 0x39, 0x1dc, 0x28c)
	github.com/tendermint/tendermint/consensus/_test/_obj_test/state.go:1477 +0x1be5
github.com/tendermint/tendermint/consensus.(*ConsensusState).tryAddVote(0xc424efd520, 0xc424e88780, 0x0, 0x0, 0xd7fb00, 0xc42152ce00)
	github.com/tendermint/tendermint/consensus/_test/_obj_test/state.go:1382 +0x93
github.com/tendermint/tendermint/consensus.(*ConsensusState).handleMsg(0xc424efd520, 0xcb58e0, 0xc42547feb8, 0x0, 0x0, 0x6, 0x0, 0x4, 0xed10ca07e, 0x3077bfea, ...)
	github.com/tendermint/tendermint/consensus/_test/_obj_test/state.go:660 +0x9fc
github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine(0xc424efd520, 0x0)
	github.com/tendermint/tendermint/consensus/_test/_obj_test/state.go:615 +0x5f5
created by github.com/tendermint/tendermint/consensus.(*ConsensusState).OnStart
	github.com/tendermint/tendermint/consensus/_test/_obj_test/state.go:332 +0x4a7
exit status 2
FAIL	github.com/tendermint/tendermint/consensus	76.644s
make: *** [test_integrations] Error 1
```

See https://github.com/tendermint/tendermint/pull/568