Early but functional prototype of the new `p2p.Router`, see its GoDoc comment for details on how it works. Expect much of this logic to change and improve as we evolve the new P2P stack.
There is a simple test that sets up an in-memory network of four routers with reactors and passes messages between them, but otherwise no exhaustive tests since this is very much a work-in-progress.
E2E tests often fail due to fast sync stalls causing the validator to miss signing blocks. This increases the tolerance for missed signatures to 2/3 to allow validators to spend more time starting up.
#5852 fixed an issue with error propagation in `os.EnsureDir()`. However, this function is basically identical to `os.MkdirAll()`, and can be replaced entirely with a call to it. We keep the function for backwards compatibility.
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Marko <marbar3778@yahoo.com>
blockchain/vX reactor priority was decreased because during the normal operation
(i.e. when the node is not fast syncing) blockchain priority can't be
the same as consensus reactor priority. Otherwise, it's theoretically possible to
slow down consensus by constantly requesting blocks from the node.
NOTE: ideally blockchain/vX reactor priority would be dynamic. e.g. when
the node is fast syncing, the priority is 10 (max), but when it's done
fast syncing - the priority gets decreased to 5 (only to serve blocks
for other nodes). But it's not possible now, therefore I decided to
focus on the normal operation (priority = 5).
evidence and consensus critical messages are more important than
the mempool ones, hence priorities are bumped by 1 (from 5 to 6).
statesync reactor priority was changed from 1 to 5 to be the same as
blockchain/vX priority.
Refs https://github.com/tendermint/tendermint/issues/5816
Bumps [github.com/cosmos/iavl](https://github.com/cosmos/iavl) from 0.15.2 to 0.15.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/cosmos/iavl/releases">github.com/cosmos/iavl's releases</a>.</em></p>
<blockquote>
<h2>v0.15.3</h2>
<p><a href="257e8b9292/CHANGELOG.md (0153-december-21-2020</a></p>)
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/cosmos/iavl/blob/master/CHANGELOG.md">github.com/cosmos/iavl's changelog</a>.</em></p>
<blockquote>
<h2>0.15.3 (December 21, 2020)</h2>
<p>Special thanks to external contributors on this release: <a href="https://github.com/odeke-em">@odeke-em</a></p>
<h3>Improvements</h3>
<ul>
<li><a href="https://github-redirect.dependabot.com/cosmos/iavl/pull/352">#352</a> Reuse buffer to improve performance of <code>GetMembershipProof()</code> and <code>GetNonMembershipProof()</code>.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="257e8b9292"><code>257e8b9</code></a> changelog: release 0.15.3 (<a href="https://github-redirect.dependabot.com/cosmos/iavl/issues/353">#353</a>)</li>
<li><a href="b2dffed4b2"><code>b2dffed</code></a> convertVarIntToBytes: use reusable bytes array (<a href="https://github-redirect.dependabot.com/cosmos/iavl/issues/352">#352</a>)</li>
<li><a href="9e510e5a64"><code>9e510e5</code></a> github: run tests with 32-bit arch as well (<a href="https://github-redirect.dependabot.com/cosmos/iavl/issues/350">#350</a>)</li>
<li>See full diff in <a href="https://github.com/cosmos/iavl/compare/v0.15.2...v0.15.3">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
@p4u from vocdoni.io reported that the mempool might behave incorrectly under a
high load. The consequences can range from pauses between blocks to the peers
disconnecting from this node.
My current theory is that the flowrate lib we're using to control flow
(multiplex over a single TCP connection) was not designed w/ large blobs
(1MB batch of txs) in mind.
I've tried decreasing the Mempool reactor priority, but that did not
have any visible effect. What actually worked is adding a time.Sleep
into mempool.Reactor#broadcastTxRoutine after an each successful send ==
manual control flow of sort.
As a temporary remedy (until the mempool package
is refactored), the max-batch-bytes was disabled. Transactions will be sent
one by one without batching
Closes#5796
When set to true, an invalid transaction will be kept in the cache (this may help some applications to protect against spam).
NOTE: this is a temporary config option. The more correct solution would be to add a TTL to each transaction (i.e. CheckTx may return a TTL in ResponseCheckTx).
Closes: #5751
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.8.0 to 1.9.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/prometheus/client_golang/releases">github.com/prometheus/client_golang's releases</a>.</em></p>
<blockquote>
<h2>1.9.0 / 2020-12-17</h2>
<ul>
<li>[FEATURE] <code>NewPidFileFn</code> helper to create process collectors for processes whose PID is read from a file. <a href="https://github-redirect.dependabot.com/prometheus/client_golang/issues/804">#804</a></li>
<li>[BUGFIX] promhttp: Prevent endless loop in <code>InstrumentHandler...</code> middlewares with invalid metric or label names. <a href="https://github-redirect.dependabot.com/prometheus/client_golang/issues/823">#823</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/prometheus/client_golang/blob/master/CHANGELOG.md">github.com/prometheus/client_golang's changelog</a>.</em></p>
<blockquote>
<h2>1.9.0 / 2020-12-17</h2>
<ul>
<li>[FEATURE] <code>NewPidFileFn</code> helper to create process collectors for processes whose PID is read from a file. <a href="https://github-redirect.dependabot.com/prometheus/client_golang/issues/804">#804</a></li>
<li>[BUGFIX] promhttp: Prevent endless loop in <code>InstrumentHandler...</code> middlewares with invalid metric or label names. <a href="https://github-redirect.dependabot.com/prometheus/client_golang/issues/823">#823</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="d89cf5af88"><code>d89cf5a</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/prometheus/client_golang/issues/826">#826</a> from prometheus/beorn7/release</li>
<li><a href="80ca9cdc4e"><code>80ca9cd</code></a> Cut release 1.9.0</li>
<li><a href="8d16199dea"><code>8d16199</code></a> Update dependencies</li>
<li><a href="8b73bd904c"><code>8b73bd9</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/prometheus/client_golang/issues/823">#823</a> from prometheus/beorn7/promhttp</li>
<li><a href="98eb6cbf7c"><code>98eb6cb</code></a> promhttp: Correctly detect invalid metric and label names</li>
<li><a href="37c26edd5b"><code>37c26ed</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/prometheus/client_golang/issues/821">#821</a> from prometheus/beorn7/multierror</li>
<li><a href="34ca120377"><code>34ca120</code></a> Be more explicit about the multi-line properties of MultiError</li>
<li><a href="fd6d368676"><code>fd6d368</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/prometheus/client_golang/issues/819">#819</a> from jubalh/sp</li>
<li><a href="cf6dc82780"><code>cf6dc82</code></a> Correct spelling: possibilites -> possibilities</li>
<li><a href="39b478e90c"><code>39b478e</code></a> Added example api code showing how to add auth tokens and user agents to prom...</li>
<li>Additional commits viewable in <a href="https://github.com/prometheus/client_golang/compare/v1.8.0...v1.9.0">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
time_iota_ms is intended to ensure that an honest validator always generates timestamps
with time increasing monotonically. For this purpose, it always suffices to have this parameter
set to `1ms`. Allowing users to choose different numbers increases bug surface area.
Thus the code now ignores the user provided time_iota_ms parameter (marking it as unused),
and uses 1ms internally.
The MTU (Maximum Transmission Unit) for Ethernet is 1500 bytes.
The IP header and the TCP header take up 20 bytes each at least (unless
optional header fields are used) and thus the max for (non-Jumbo frame)
Ethernet is 1500 - 20 -20 = 1460
Source: https://stackoverflow.com/a/3074427/820520
While debugging the mempool issue (#5796), I've noticed we're spending
quite a bit of time encoding blobs of data, which never get printed! The
reason is filtering occurs on the level below, so Go runtime rightfully
evaluates function arguments.
I think it's okay to not format raw bytes.
The `NodeInfo` interface does not appear to serve any purpose at all, so I removed it and renamed the `DefaultNodeInfo` struct to `NodeInfo` (including the Protobuf representations). Let me know if this is actually needed for anything.
Only the Protobuf rename is listed in the changelog, since we do not officially support API stability of the `p2p` package (according to `README.md`). The on-wire protocol remains compatible.
This implements a new `Transport` interface and related types for the P2P refactor in #5670. Previously, `conn.MConnection` was very tightly coupled to the `Peer` implementation -- in order to allow alternative non-multiplexed transports (e.g. QUIC), MConnection has now been moved below the `Transport` interface, as `MConnTransport`, and decoupled from the peer. Since the `p2p` package is not covered by our Go API stability, this is not considered a breaking change, and not listed in the changelog.
The initial approach was to implement the new interface in its final form (which also involved possible protocol changes, see https://github.com/tendermint/spec/pull/227). However, it turned out that this would require a large amount of changes to existing P2P code because of the previous tight coupling between `Peer` and `MConnection` and the reliance on subtleties in the MConnection behavior. Instead, I have broadened the `Transport` interface to expose much of the existing MConnection interface, preserved much of the existing MConnection logic and behavior in the transport implementation, and tried to make as few changes to the rest of the P2P stack as possible. We will instead reduce this interface gradually as we refactor other parts of the P2P stack.
The low-level transport code and protocol (e.g. MConnection, SecretConnection and so on) has not been significantly changed, and refactoring this is not a priority until we come up with a plan for QUIC adoption, as we may end up discarding the MConnection code entirely.
There are no tests of the new `MConnTransport`, as this code is likely to evolve as we proceed with the P2P refactor, but tests should be added before a final release. The E2E tests are sufficient for basic validation in the meanwhile.
Erik Grinaker
Callum Waters
Marko
Aleksandr Bezobchuk
Tess Rinearson
dependabot[bot]
John Adler
Anton Kaliaev
Dev Ojha
PaddyMc