From c8be091d4a41976252e931f2a570c121be235d41 Mon Sep 17 00:00:00 2001 From: Ethan Buchman Date: Mon, 28 May 2018 16:44:14 -0400 Subject: [PATCH 1/3] gut docs/app-arch --- docs/app-architecture.rst | 151 ++++++++------------------------------ 1 file changed, 30 insertions(+), 121 deletions(-) diff --git a/docs/app-architecture.rst b/docs/app-architecture.rst index 4a7c414ec..cbcac4d12 100644 --- a/docs/app-architecture.rst +++ b/docs/app-architecture.rst @@ -1,133 +1,42 @@ Application Architecture Guide ============================== -Overview --------- +Here we provide a brief guide on the recommended architecture of a Tendermint blockchain +application. -A blockchain application is more than the consensus engine and the -transaction logic (eg. smart contracts, business logic) as implemented -in the ABCI app. There are also (mobile, web, desktop) clients that will -need to connect and make use of the app. We will assume for now that you -have a well designed transactions and database model, but maybe this -will be the topic of another article. This article is more interested in -various ways of setting up the "plumbing" and connecting these pieces, -and demonstrating some evolving best practices. +The following diagram provides a superb example: -Security --------- +https://lh3.googleusercontent.com/eOT9KuaBjv7xKBg2xN8u2O1nU2Iw1-6PtFXrBSFWW8LYhfAMu84EwXW4RwyOnuiNqCoOoqXxE8Pkhr4Fyq9f=w2559-h1303-rw -A very important aspect when constructing a blockchain is security. The -consensus model can be DoSed (no consensus possible) by corrupting 1/3 -of the validators and exploited (writing arbitrary blocks) by corrupting -2/3 of the validators. So, while the security is not that of the -"weakest link", you should take care that the "average link" is -sufficiently hardened. +The end-user application here is the Cosmos Voyager, at the bottom left. +Voyager communicates with a REST API exposed by a local Light-Client Daemon. +The Light-Client Daemon is an application specific program that communciates with +Tendermint nodes and verifies Tendermint light-client proofs through the Tendermint Core RPC. +The Tendermint Core process communicates with a local ABCI application, where the +user query or transaction is actually processed. -One big attack surface on the validators is the communication between -the ABCI app and the tendermint core. This should be highly protected. -Ideally, the app and the core are running on the same machine, so no -external agent can target the communication channel. You can use unix -sockets (with permissions preventing access from other users), or even -compile the two apps into one binary if the ABCI app is also writen in -go. If you are unable to do that due to language support, then the ABCI -app should bind a TCP connection to localhost (127.0.0.1), which is less -efficient and secure, but still not reachable from outside. If you must -run the ABCI app and tendermint core on separate machines, make sure you -have a secure communication channel (ssh tunnel?) +The ABCI application must be a deterministic result of the Tendermint consensus - any external influence +on the application state that didn't come through Tendermint could cause a +consensus failure. Thus *nothing* should communicate with the application except Tendermint via ABCI. -Now assuming, you have linked together your app and the core securely, -you must also make sure no one can get on the machine it is hosted on. -At this point it is basic network security. Run on a secure operating -system (SELinux?). Limit who has access to the machine (user accounts, -but also where the physical machine is hosted). Turn off all services -except for ssh, which should only be accessible by some well-guarded -public/private key pairs (no password). And maybe even firewall off -access to the ports used by the validators, so only known validators can -connect. +If the application is written in Go, it can be compiled into the Tendermint binary. +Otherwise, it should use a unix socket to communicate with Tendermint. +If it's necessary to use TCP, extra care must be taken to encrypt and authenticate the connection. -There was also a suggestion on slack from @jhon about compiling -everything together with a unikernel for more security, such as -`Mirage `__ or -`UNIK `__. +All reads from the app happen through the Tendermint `/abci_query` endpoint. +All writes to the app happen through the Tendermint `/broadcast_tx_*` endpoints. -Connecting your client to the blockchain ----------------------------------------- +The Light-Client Daemon is what provides light clients (end users) with nearly all the security of a full node. +It formats and broadcasts transactions, and verifies proofs of queries and transaction results. +Note that it need not be a daemon - the Light-Client logic could instead be implemented in the same process as the end-user application. -Tendermint Core RPC -~~~~~~~~~~~~~~~~~~~ +Note for those ABCI applications with weaker security requirements, the functionality of the Light-Client Daemon can be moved +into the ABCI application process itself. That said, exposing the application process to anything besides Tendermint over ABCI +requires extreme caution, as all transactions, and possibly all queries, should still pass through Tendermint. -The concept is that the ABCI app is completely hidden from the outside -world and only communicated through a tested and secured `interface -exposed by the tendermint core <./specification/rpc.html>`__. This interface -exposes a lot of data on the block header and consensus process, which -is quite useful for externally verifying the system. It also includes -3(!) methods to broadcast a transaction (propose it for the blockchain, -and possibly await a response). And one method to query app-specific -data from the ABCI application. - -Pros: - -- Server code already written -- Access to block headers to validate merkle proofs (nice for light clients) -- Basic read/write functionality is supported - -Cons: - -- Limited interface to app. All queries must be serialized into []byte (less expressive than JSON over HTTP) and there is no way to push data from ABCI app to the client (eg. notify me if account X receives a transaction) - -Custom ABCI server -~~~~~~~~~~~~~~~~~~ - -This was proposed by @wolfposd on slack and demonstrated by -`TMChat `__, a sample app. The -concept is to write a custom server for your app (with typical REST -API/websockets/etc for easy use by a mobile app). This custom server is -in the same binary as the ABCI app and data store, so can easily react -to complex events there that involve understanding the data format (send -a message if my balance drops below 500). All "writes" sent to this -server are proxied via websocket/JSON-RPC to tendermint core. When they -come back as deliver\_tx over ABCI, they will be written to the data -store. For "reads", we can do any queries we wish that are supported by -our architecture, using any web technology that is useful. The general -architecture is shown in the following diagram: - -.. figure:: assets/tm-application-example.png - -Pros: - -- Separates application logic from blockchain logic -- Allows much richer, more flexible client-facing API -- Allows pub-sub, watching certain fields, etc. - -Cons: - -- Access to ABCI app can be dangerous (be VERY careful not to write unless it comes from the validator node) -- No direct access to the blockchain headers to verify tx -- You must write your own API (but maybe that's a pro...) - -Hybrid solutions -~~~~~~~~~~~~~~~~ - -Likely the least secure but most versatile. The client can access both -the tendermint node for all blockchain info, as well as a custom app -server, for complex queries and pub-sub on the abci app. - -Pros: - -- All from both above solutions - -Cons: - -- Even more complexity; even more attack vectors (less -security) - -Scalability ------------ - -Read replica using non-validating nodes? They could forward transactions -to the validators (fewer connections, more security), and locally allow -all queries in any of the above configurations. Thus, while -transaction-processing speed is limited by the speed of the abci app and -the number of validators, one should be able to scale our read -performance to quite an extent (until the replication process drains too -many resources from the validator nodes). +See the following for more extensive documentation: +- [Interchain Standard for the Light-Client REST API](https://github.com/cosmos/cosmos-sdk/pull/1028) +- [Tendermint RPC Docs](https://tendermint.github.io/slate/) +- [Tendermint in Production](https://github.com/tendermint/tendermint/pull/1618) +- [Tendermint Basics](https://tendermint.readthedocs.io/en/master/using-tendermint.html) +- [ABCI spec](https://github.com/tendermint/abci/blob/master/specification.rst) From 1e87ef7f75eb46a989ed33be18d293935a4c889b Mon Sep 17 00:00:00 2001 From: Ethan Buchman Date: Tue, 29 May 2018 09:28:29 -0400 Subject: [PATCH 2/3] circle: add GOCACHE=off and -v to tests --- .circleci/config.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index 4209d531e..30b70f77e 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -139,7 +139,7 @@ jobs: for pkg in $(go list github.com/tendermint/tendermint/... | grep -v /vendor/ | circleci tests split --split-by=timings); do id=$(basename "$pkg") - go test -timeout 5m -race -coverprofile=/tmp/workspace/profiles/$id.out -covermode=atomic "$pkg" + GOCACHE=off go test -v -timeout 5m -race -coverprofile=/tmp/workspace/profiles/$id.out -covermode=atomic "$pkg" done - persist_to_workspace: root: /tmp/workspace From d584e034272dff8c088304c8f00a2f970e4e30fd Mon Sep 17 00:00:00 2001 From: Ethan Buchman Date: Wed, 30 May 2018 23:23:39 -0400 Subject: [PATCH 3/3] fix link and typo --- docs/app-architecture.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/app-architecture.rst b/docs/app-architecture.rst index cbcac4d12..c303ba4a5 100644 --- a/docs/app-architecture.rst +++ b/docs/app-architecture.rst @@ -6,11 +6,11 @@ application. The following diagram provides a superb example: -https://lh3.googleusercontent.com/eOT9KuaBjv7xKBg2xN8u2O1nU2Iw1-6PtFXrBSFWW8LYhfAMu84EwXW4RwyOnuiNqCoOoqXxE8Pkhr4Fyq9f=w2559-h1303-rw +https://drive.google.com/open?id=1yR2XpRi9YCY9H9uMfcw8-RMJpvDyvjz9 The end-user application here is the Cosmos Voyager, at the bottom left. Voyager communicates with a REST API exposed by a local Light-Client Daemon. -The Light-Client Daemon is an application specific program that communciates with +The Light-Client Daemon is an application specific program that communicates with Tendermint nodes and verifies Tendermint light-client proofs through the Tendermint Core RPC. The Tendermint Core process communicates with a local ABCI application, where the user query or transaction is actually processed.