This averts a log-after-close issue. We should probably also chase the shutdown
issues, but since ABCI clients should generally only shut down once per process
I don't think this is a real priority, and the trace is hairy.

Decrease the likelihood that two flaky tests will hit
the same batch.

Account for the increase in test load from #7608.

The test filter was looking for "TestGoFiles", which does not include tests in
a separate package (e.g., "package foo_test" for "package foo").
This caused several packages not to be tested in CI, including:

  github.com/tendermint/tendermint/abci/client
  github.com/tendermint/tendermint/crypto
  github.com/tendermint/tendermint/crypto/tmhash
  github.com/tendermint/tendermint/internal/eventbus
  github.com/tendermint/tendermint/internal/evidence
  github.com/tendermint/tendermint/internal/inspect
  github.com/tendermint/tendermint/internal/jsontypes
  github.com/tendermint/tendermint/internal/libs/protoio
  github.com/tendermint/tendermint/internal/libs/sync
  github.com/tendermint/tendermint/internal/p2p/pex
  github.com/tendermint/tendermint/internal/pubsub
  github.com/tendermint/tendermint/internal/pubsub/query
  github.com/tendermint/tendermint/internal/pubsub/query/syntax
  github.com/tendermint/tendermint/internal/state/indexer
  github.com/tendermint/tendermint/internal/state/indexer/block/kv
  github.com/tendermint/tendermint/libs/json
  github.com/tendermint/tendermint/libs/log
  github.com/tendermint/tendermint/libs/os
  github.com/tendermint/tendermint/light
  github.com/tendermint/tendermint/light/provider/http
  github.com/tendermint/tendermint/privval/grpc
  github.com/tendermint/tendermint/proto/tendermint/blocksync
  github.com/tendermint/tendermint/proto/tendermint/consensus
  github.com/tendermint/tendermint/proto/tendermint/statesync
  github.com/tendermint/tendermint/rpc/client
  github.com/tendermint/tendermint/rpc/client/mock
  github.com/tendermint/tendermint/test/e2e/tests
  github.com/tendermint/tendermint/test/fuzz/mempool
  github.com/tendermint/tendermint/test/fuzz/p2p/secretconnection
  github.com/tendermint/tendermint/test/fuzz/rpc/jsonrpc/server

Updates #7626 and #7634.

The interaction between defers and t.Cleanup can be delicate.
For this case, which regularly flakes in CI, be explicit:
Defer the closes and waits before making any attempt to leaktest.

During file rotation and WAL shutdown, there was a race condition between users
of an autofile and its termination. To fix this, ensure operations on an
autofile are properly synchronized, and report errors when attempting to use an
autofile after it was closed.

Notably:

- Simplify the cancellation protocol between signal and Close.
- Exclude writers to an autofile during rotation.
- Add documentation about what is going on.

There is a lot more that could be improved here, but this addresses the more
obvious races that have been panicking unit tests.

Reverts and improves on #7622. The problem turns out not to be on the PEX
channel side, but on the pass-through (Go) channel.

There was a path on which computing the next delivery time did not hold the
lock, defying the admonition on its comment.

## What does this pull request do?
This pull requests adds two metrics intended for use in calculating an experimental value for `MessageDelay`.

The metrics are as follows:
```
# HELP tendermint_consensus_complete_prevote_message_delay Difference in seconds between the proposal timestamp and the timestamp of the prevote that achieved 100% of the voting power in the prevote step.
# TYPE tendermint_consensus_complete_prevote_message_delay gauge
tendermint_consensus_complete_prevote_message_delay{chain_id="test-chain-aZbwF1"} 0.013025505

# HELP tendermint_consensus_quorum_prevote_message_delay Difference in seconds between the proposal timestamp and the timestamp of the prevote that achieved a quorum in the prevote step.
# TYPE tendermint_consensus_quorum_prevote_message_delay gauge
tendermint_consensus_quorum_prevote_message_delay{chain_id="test-chain-aZbwF1"} 0.013025505
```

## Why this change?

 For more information on what these metrics are calculating, see #7202. The aim is to merge to backport these metrics to v0.34 and run nodes on a few popular chains with these metrics to determine the experimental values for `MessageDelay` on these popular chains and use these to select our default `SynchronyParams.MessageDelay` value.

## Why Gauges for the metrics?
Gauges allow us to overwrite the metric on each successive observation. We can then capture these metrics over time to track the highest and lowest observed value.

This commit changes the behaviour of the /unconfirmed_txs endpoint by replacing limit with a page and perPage parameter for pagination.
The test case for unconfirmed_txs have been accommodated to properly test this change and the documentation for the API as well.

This reverts commit 22e8aafa03.

Co-authored-by: Callum Waters <cmwaters19@gmail.com>

Co-authored-by: Josef Widder <44643235+josef-widder@users.noreply.github.com>

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

Co-authored-by: Josef Widder <44643235+josef-widder@users.noreply.github.com>

Co-authored-by: Josef Widder <44643235+josef-widder@users.noreply.github.com>

The custom error types in the provider package did not propagate their wrapped
underlying reasons, making it difficult for the test to check that the correct
error was observed.

- Fix the custom errors to have a true underlying error (not just a string).
- Add Unwrap methods to support inspection by errors.Is.
- Update usage in a few places.
- Fix the test to check for acceptable variation.

Fixes #7609.

After writing and then reading a bunch of random messages, the test was
checking that it did not read the same number of messages that it wrote.
The sense of this check was inverted; they should match.

Introduced by accident in #7522. I'm not sure why this did not show up in CI.

Edit: I now know why it didn't show up in ci: #7608.

Fixed typo in exchange

Co-authored-by: M. J. Fromberger <fromberger@interchain.io>