diff --git a/abci/README.md b/abci/README.md index 63b43e54f..110ad40e9 100644 --- a/abci/README.md +++ b/abci/README.md @@ -1,7 +1,5 @@ # Application BlockChain Interface (ABCI) -[![CircleCI](https://circleci.com/gh/tendermint/abci.svg?style=svg)](https://circleci.com/gh/tendermint/abci) - Blockchains are systems for multi-master state machine replication. **ABCI** is an interface that defines the boundary between the replication engine (the blockchain), and the state machine (the application). @@ -12,160 +10,28 @@ Previously, the ABCI was referred to as TMSP. The community has provided a number of addtional implementations, see the [Tendermint Ecosystem](https://tendermint.com/ecosystem) -## Specification -A detailed description of the ABCI methods and message types is contained in: +## Installation & Usage -- [A prose specification](specification.md) -- [A protobuf file](https://github.com/tendermint/tendermint/blob/master/abci/types/types.proto) -- [A Go interface](https://github.com/tendermint/tendermint/blob/master/abci/types/application.go). +To get up and running quickly, see the [getting started guide](../docs/app-dev/getting-started.md) along with the [abci-cli documentation](../docs/app-dev/abci-cli.md) which will go through the examples found in the [examples](./example/) directory. -For more background information on ABCI, motivations, and tendermint, please visit [the documentation](https://tendermint.com/docs/). -The two guides to focus on are the `Application Development Guide` and `Using ABCI-CLI`. +## Specification + +A detailed description of the ABCI methods and message types is contained in: +- [The main spec](../docs/spec/abci/abci.md) +- [A protobuf file](./types/types.proto) +- [A Go interface](./types/application.go) ## Protocol Buffers -To compile the protobuf file, run: +To compile the protobuf file, run (from the root of the repo): ``` -cd $GOPATH/src/github.com/tendermint/tendermint/; make protoc_abci +make protoc_abci ``` See `protoc --help` and [the Protocol Buffers site](https://developers.google.com/protocol-buffers) -for details on compiling for other languages. Note we also include a [GRPC](http://www.grpc.io/docs) +for details on compiling for other languages. Note we also include a [GRPC](https://www.grpc.io/docs) service definition. -## Install ABCI-CLI - -The `abci-cli` is a simple tool for debugging ABCI servers and running some -example apps. To install it: - -``` -mkdir -p $GOPATH/src/github.com/tendermint -cd $GOPATH/src/github.com/tendermint -git clone https://github.com/tendermint/tendermint.git -cd tendermint -make get_tools -make get_vendor_deps -make install_abci -``` - -## Implementation - -We provide three implementations of the ABCI in Go: - -- Golang in-process -- ABCI-socket -- GRPC - -Note the GRPC version is maintained primarily to simplify onboarding and prototyping and is not receiving the same -attention to security and performance as the others - -### In Process - -The simplest implementation just uses function calls within Go. -This means ABCI applications written in Golang can be compiled with TendermintCore and run as a single binary. - -See the [examples](#examples) below for more information. - -### Socket (TSP) - -ABCI is best implemented as a streaming protocol. -The socket implementation provides for asynchronous, ordered message passing over unix or tcp. -Messages are serialized using Protobuf3 and length-prefixed with a [signed Varint](https://developers.google.com/protocol-buffers/docs/encoding?csw=1#signed-integers) - -For example, if the Protobuf3 encoded ABCI message is `0xDEADBEEF` (4 bytes), the length-prefixed message is `0x08DEADBEEF`, since `0x08` is the signed varint -encoding of `4`. If the Protobuf3 encoded ABCI message is 65535 bytes long, the length-prefixed message would be like `0xFEFF07...`. - -Note the benefit of using this `varint` encoding over the old version (where integers were encoded as `` is that -it is the standard way to encode integers in Protobuf. It is also generally shorter. - -### GRPC - -GRPC is an rpc framework native to Protocol Buffers with support in many languages. -Implementing the ABCI using GRPC can allow for faster prototyping, but is expected to be much slower than -the ordered, asynchronous socket protocol. The implementation has also not received as much testing or review. - -Note the length-prefixing used in the socket implementation does not apply for GRPC. - -## Usage - -The `abci-cli` tool wraps an ABCI client and can be used for probing/testing an ABCI server. -For instance, `abci-cli test` will run a test sequence against a listening server running the Counter application (see below). -It can also be used to run some example applications. -See [the documentation](https://tendermint.com/docs/) for more details. - -### Examples - -Check out the variety of example applications in the [example directory](example/). -It also contains the code refered to by the `counter` and `kvstore` apps; these apps come -built into the `abci-cli` binary. - -#### Counter - -The `abci-cli counter` application illustrates nonce checking in transactions. It's code looks like: - -```golang -func cmdCounter(cmd *cobra.Command, args []string) error { - - app := counter.NewCounterApplication(flagSerial) - - logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout)) - - // Start the listener - srv, err := server.NewServer(flagAddrC, flagAbci, app) - if err != nil { - return err - } - srv.SetLogger(logger.With("module", "abci-server")) - if err := srv.Start(); err != nil { - return err - } - - // Wait forever - cmn.TrapSignal(func() { - // Cleanup - srv.Stop() - }) - return nil -} -``` - -and can be found in [this file](cmd/abci-cli/abci-cli.go). - -#### kvstore - -The `abci-cli kvstore` application, which illustrates a simple key-value Merkle tree - -```golang -func cmdKVStore(cmd *cobra.Command, args []string) error { - logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout)) - - // Create the application - in memory or persisted to disk - var app types.Application - if flagPersist == "" { - app = kvstore.NewKVStoreApplication() - } else { - app = kvstore.NewPersistentKVStoreApplication(flagPersist) - app.(*kvstore.PersistentKVStoreApplication).SetLogger(logger.With("module", "kvstore")) - } - - // Start the listener - srv, err := server.NewServer(flagAddrD, flagAbci, app) - if err != nil { - return err - } - srv.SetLogger(logger.With("module", "abci-server")) - if err := srv.Start(); err != nil { - return err - } - - // Wait forever - cmn.TrapSignal(func() { - // Cleanup - srv.Stop() - }) - return nil -} -``` diff --git a/docs/app-dev/app-development.md b/docs/app-dev/app-development.md index 2618bb1e9..d157ce378 100644 --- a/docs/app-dev/app-development.md +++ b/docs/app-dev/app-development.md @@ -47,90 +47,6 @@ The mempool and consensus logic act as clients, and each maintains an open ABCI connection with the application, which hosts an ABCI server. Shown are the request and response types sent on each connection. -## Message Protocol - -The message protocol consists of pairs of requests and responses. Some -messages have no fields, while others may include byte-arrays, strings, -or integers. See the `message Request` and `message Response` -definitions in [the protobuf definition -file](https://github.com/tendermint/tendermint/blob/develop/abci/types/types.proto), -and the [protobuf -documentation](https://developers.google.com/protocol-buffers/docs/overview) -for more details. - -For each request, a server should respond with the corresponding -response, where order of requests is preserved in the order of -responses. - -## Server - -To use ABCI in your programming language of choice, there must be a ABCI -server in that language. Tendermint supports two kinds of implementation -of the server: - -- Asynchronous, raw socket server (Tendermint Socket Protocol, also - known as TSP or Teaspoon) -- GRPC - -Both can be tested using the `abci-cli` by setting the `--abci` flag -appropriately (ie. to `socket` or `grpc`). - -See examples, in various stages of maintenance, in -[Go](https://github.com/tendermint/tendermint/tree/develop/abci/server), -[JavaScript](https://github.com/tendermint/js-abci), -[Python](https://github.com/tendermint/tendermint/tree/develop/abci/example/python3/abci), -[C++](https://github.com/mdyring/cpp-tmsp), and -[Java](https://github.com/jTendermint/jabci). - -### GRPC - -If GRPC is available in your language, this is the easiest approach, -though it will have significant performance overhead. - -To get started with GRPC, copy in the [protobuf -file](https://github.com/tendermint/tendermint/blob/develop/abci/types/types.proto) -and compile it using the GRPC plugin for your language. For instance, -for golang, the command is `protoc --go_out=plugins=grpc:. types.proto`. -See the [grpc documentation for more details](http://www.grpc.io/docs/). -`protoc` will autogenerate all the necessary code for ABCI client and -server in your language, including whatever interface your application -must satisfy to be used by the ABCI server for handling requests. - -### TSP - -If GRPC is not available in your language, or you require higher -performance, or otherwise enjoy programming, you may implement your own -ABCI server using the Tendermint Socket Protocol, known affectionately -as Teaspoon. The first step is still to auto-generate the relevant data -types and codec in your language using `protoc`. Messages coming over -the socket are proto3 encoded, but additionally length-prefixed to -facilitate use as a streaming protocol. proto3 doesn't have an -official length-prefix standard, so we use our own. The first byte in -the prefix represents the length of the Big Endian encoded length. The -remaining bytes in the prefix are the Big Endian encoded length. - -For example, if the proto3 encoded ABCI message is 0xDEADBEEF (4 -bytes), the length-prefixed message is 0x0104DEADBEEF. If the proto3 -encoded ABCI message is 65535 bytes long, the length-prefixed message -would be like 0x02FFFF.... - -Note this prefixing does not apply for grpc. - -An ABCI server must also be able to support multiple connections, as -Tendermint uses three connections. - -## Client - -There are currently two use-cases for an ABCI client. One is a testing -tool, as in the `abci-cli`, which allows ABCI requests to be sent via -command line. The other is a consensus engine, such as Tendermint Core, -which makes requests to the application every time a new transaction is -received or a block is committed. - -It is unlikely that you will need to implement a client. For details of -our client, see -[here](https://github.com/tendermint/tendermint/tree/develop/abci/client). - Most of the examples below are from [kvstore application](https://github.com/tendermint/tendermint/blob/develop/abci/example/kvstore/kvstore.go), which is a part of the abci repo. [persistent_kvstore diff --git a/docs/spec/abci/client-server.md b/docs/spec/abci/client-server.md index 822bfd1fc..5ac7b3eb4 100644 --- a/docs/spec/abci/client-server.md +++ b/docs/spec/abci/client-server.md @@ -3,12 +3,8 @@ This section is for those looking to implement their own ABCI Server, perhaps in a new programming language. -You are expected to have read [ABCI Methods and Types](abci.md) and [ABCI -Applications](apps.md). - -See additional details in the [ABCI -readme](https://github.com/tendermint/tendermint/blob/develop/abci/README.md)(TODO: deduplicate -those details). +You are expected to have read [ABCI Methods and Types](./abci.md) and [ABCI +Applications](./apps.md). ## Message Protocol @@ -24,17 +20,16 @@ For each request, a server should respond with the corresponding response, where the order of requests is preserved in the order of responses. -## Server +## Server Implementations To use ABCI in your programming language of choice, there must be a ABCI -server in that language. Tendermint supports two kinds of implementation -of the server: +server in that language. Tendermint supports three implementations of the ABCI, written in Go: -- Asynchronous, raw socket server (Tendermint Socket Protocol, also - known as TSP or Teaspoon) +- In-process (Golang only) +- ABCI-socket - GRPC -Both can be tested using the `abci-cli` by setting the `--abci` flag +The latter two can be tested using the `abci-cli` by setting the `--abci` flag appropriately (ie. to `socket` or `grpc`). See examples, in various stages of maintenance, in @@ -44,6 +39,12 @@ See examples, in various stages of maintenance, in [C++](https://github.com/mdyring/cpp-tmsp), and [Java](https://github.com/jTendermint/jabci). +### In Process + +The simplest implementation uses function calls within Golang. +This means ABCI applications written in Golang can be compiled with TendermintCore and run as a single binary. + + ### GRPC If GRPC is available in your language, this is the easiest approach, @@ -58,15 +59,18 @@ See the [grpc documentation for more details](http://www.grpc.io/docs/). server in your language, including whatever interface your application must satisfy to be used by the ABCI server for handling requests. +Note the length-prefixing used in the socket implementation (TSP) does not apply for GRPC. + ### TSP +Tendermint Socket Protocol is an asynchronous, raw socket server which provides ordered message passing over unix or tcp. +Messages are serialized using Protobuf3 and length-prefixed with a [signed Varint](https://developers.google.com/protocol-buffers/docs/encoding?csw=1#signed-integers) + If GRPC is not available in your language, or you require higher performance, or otherwise enjoy programming, you may implement your own -ABCI server using the Tendermint Socket Protocol, known affectionately -as Teaspoon. The first step is still to auto-generate the relevant data -types and codec in your language using `protoc`. Messages coming over -the socket are proto3 encoded, but additionally length-prefixed to -facilitate use as a streaming protocol. proto3 doesn't have an +ABCI server using the Tendermint Socket Protocol. The first step is still to auto-generate the relevant data +types and codec in your language using `protoc`. In addition to being proto3 encoded, messages coming over +the socket are length-prefixed to facilitate use as a streaming protocol. proto3 doesn't have an official length-prefix standard, so we use our own. The first byte in the prefix represents the length of the Big Endian encoded length. The remaining bytes in the prefix are the Big Endian encoded length. @@ -76,12 +80,14 @@ bytes), the length-prefixed message is 0x0104DEADBEEF. If the proto3 encoded ABCI message is 65535 bytes long, the length-prefixed message would be like 0x02FFFF.... -Note this prefixing does not apply for grpc. +The benefit of using this `varint` encoding over the old version (where integers were encoded as `` is that +it is the standard way to encode integers in Protobuf. It is also generally shorter. + +As noted above, this prefixing does not apply for GRPC. An ABCI server must also be able to support multiple connections, as Tendermint uses three connections. - ### Async vs Sync The main ABCI server (ie. non-GRPC) provides ordered asynchronous messages.