tendermint/docs/architecture/adr-081-protobuf-mgmt.md

ADR 081: Protocol Buffers Management

Changelog

• 2022-02-28: First draft

Status

Accepted

Tracking issue

Context

At present, we manage the Protocol Buffers schema files ("protos") that define our wire-level data formats within the Tendermint repository itself (see the proto directory). Recently, we have been making use of Buf, both locally and in CI, in order to generate Go stubs, and lint and check .proto files for breaking changes.

The version of Buf used at the time of this decision was v1beta1, and it was discussed in #7975 and in weekly calls as to whether we should upgrade to v1 and harmonize our approach with that used by the Cosmos SDK. The team managing the Cosmos SDK was primarily interested in having our protos versioned and easily accessible from the Buf registry.

The three main sets of stakeholders for the .proto files and their needs, as currently understood, are as follows.

1. Tendermint needs Go code generated from .proto files.
2. Consumers of Tendermint's .proto files, specifically projects that want to interoperate with Tendermint and need to generate code for their own programming language, want to be able to access these files in a reliable and efficient way.
3. The Tendermint Core team wants to provide stable interfaces that are as easy as possible to maintain, on which consumers can depend, and to be able to notify those consumers promptly when those interfaces change. To this end, we want to:
   1. Prevent any breaking changes from being introduced in minor/patch releases of Tendermint. Only major version updates should be able to contain breaking interface changes.
   2. Prevent generated code from diverging from the Protobuf schema files.

There was also discussion surrounding the notion of automated documentation generation and hosting, but it is not clear at this time whether this would be that valuable to any of our stakeholders. What will, of course, be valuable at minimum would be better documentation (in comments) of the .proto files themselves.

Alternative Approaches

Meeting stakeholders' needs

1. Go stub generation from protos. We could use:
   1. Buf. This approach has been rather cumbersome up to this point, and it is not clear what Buf really provides beyond that which protoc provides to justify the additional complexity in configuring Buf for stub generation.
   2. protoc - the Protocol Buffers compiler.
2. Notification of breaking changes:
   1. Buf in CI for all pull requests to release branches only (and not on master).
   2. Buf in CI on every pull request to every branch (this was the case at the time of this decision, and the team decided that the signal-to-noise ratio for this approach was too low to be of value).
3. .proto linting:
   1. Buf in CI on every pull request
4. .proto formatting:
   1. clang-format locally and a clang-format GitHub Action in CI to check that files are formatted properly on every pull request.
5. Sharing of .proto files in a versioned, reliable manner:
   1. Consumers could simply clone the Tendermint repository, check out a specific commit, tag or branch and manually copy out all of the .proto files they need. This requires no effort from the Tendermint Core team and will continue to be an option for consumers. The drawback of this approach is that it requires manual coding/scripting to implement and is brittle in the face of bigger changes.
   2. Uploading our .proto files to Buf's registry on every release. This is by far the most seamless for consumers of our .proto files, but requires the dependency on Buf. This has the additional benefit that the Buf registry will automatically generate and host documentation for these protos.
   3. We could create a process that, upon release, creates a .zip file containing our .proto files.

Popular alternatives to Buf

Prototool was not considered as it appears deprecated, and the ecosystem seems to be converging on Buf at this time.

Tooling complexity

The more tools we have in our build/CI processes, the more complex and fragile repository/CI management becomes, and the longer it takes to onboard new team members. Maintainability is a core concern here.

Buf sustainability and costs

One of the primary considerations regarding the usage of Buf is whether, for example, access to its registry will eventually become a paid-for/subscription-based service and whether this is valuable enough for us and the ecosystem to pay for such a service. At this time, it appears as though Buf will never charge for hosting open source projects' protos.

Another consideration was Buf's sustainability as a project - what happens when their resources run out? Will there be a strong and broad enough open source community to continue maintaining it?

Local Buf usage options

Local usage of Buf (i.e. not in CI) can be accomplished in two ways:

1. Installing the relevant tools individually.
2. By way of its Docker image.

Local installation of Buf requires developers to manually keep their toolchains up-to-date. The Docker option comes with a number of complexities, including how the file system permissions of code generated by a Docker container differ between platforms (e.g. on Linux, Buf-generated code ends up being owned by root).

The trouble with the Docker-based approach is that we make use of the gogoprotobuf plugin for protoc. Continuing to use the Docker-based approach to using Buf will mean that we will have to continue building our own custom Docker image with embedded gogoprotobuf.

Along these lines, we could eventually consider coming up with a Nix- or redo-based approach to developer tooling to ensure tooling consistency across the team and for anyone who wants to be able to contribute to Tendermint.

Decision

1. We will adopt Buf for now for proto generation, linting, breakage checking and its registry (mainly in CI, with optional usage locally).
2. Failing CI when checking for breaking changes in .proto files will only happen when performing minor/patch releases.
3. Local tooling will be favored over Docker-based tooling.

Detailed Design

We currently aim to:

1. Update to Buf v1 to facilitate linting, breakage checking and uploading to the Buf registry.
2. Configure CI appropriately for proto management:
   1. Uploading protos to the Buf registry on every release (e.g. the approach used by the Cosmos SDK).
   2. Linting on every pull request (e.g. the approach used by the Cosmos SDK). The linter passing should be considered a requirement for accepting PRs.
   3. Checking for breaking changes in minor/patch version releases and failing CI accordingly - see #8003.
   4. Add clang-format GitHub Action to check .proto file formatting. Format checking should be considered a requirement for accepting PRs.
3. Update the Tendermint Makefile to primarily facilitate local Protobuf stub generation, linting, formatting and breaking change checking. More specifically:
   1. This includes removing the dependency on Docker and introducing the dependency on local toolchain installation. CI-based equivalents, where relevant, will rely on specific GitHub Actions instead of the Makefile.
   2. Go code generation will rely on protoc directly.

Consequences

Positive

• We will still offer Go stub generation, proto linting and breakage checking.
• Breakage checking will only happen on minor/patch releases to increase the signal-to-noise ratio in CI.
• Versioned protos will be made available via Buf's registry upon every release.

Negative

• Developers/contributors will need to install the relevant Protocol Buffers-related tooling (Buf, gogoprotobuf, clang-format) locally in order to build, lint, format and check .proto files for breaking changes.

Neutral

References

• Protocol Buffers
• Buf
• #7975
• protoc - The Protocol Buffers compiler