moar

6 years ago · df07f8b36a
--- a/docs/subscribing-to-events-via-websocket.md
+++ b/docs/subscribing-to-events-via-websocket.md
@ -0,0 +1,26 @@
 # Subscribing to events via Websocket

 Tendermint emits different events, to which you can subscribe via
 [Websocket](https://en.wikipedia.org/wiki/WebSocket). This can be useful
 for third-party applications (for analysys) or inspecting state.

 [List of events](https://godoc.org/github.com/tendermint/tendermint/types#pkg-constants)

 You can subscribe to any of the events above by calling `subscribe` RPC
 method via Websocket.

    {
        "jsonrpc": "2.0",
        "method": "subscribe",
        "id": "0",
        "params": {
            "query": "tm.event='NewBlock'"
        }
    }

 Check out [API docs](https://tendermint.github.io/slate/#subscribe) for
 more information on query syntax and other options.

 You can also use tags, given you had included them into DeliverTx
 response, to query transaction results. See [Indexing
 transactions](./indexing-transactions.md) for details.
--- a/docs/subscribing-to-events-via-websocket.rst
+++ b/docs/subscribing-to-events-via-websocket.rst
@ -1,28 +0,0 @@
 Subscribing to events via Websocket
 ===================================

 Tendermint emits different events, to which you can subscribe via `Websocket
 <https://en.wikipedia.org/wiki/WebSocket>`__. This can be useful for
 third-party applications (for analysys) or inspecting state.

 `List of events <https://godoc.org/github.com/tendermint/tendermint/types#pkg-constants>`__

 You can subscribe to any of the events above by calling ``subscribe`` RPC method via Websocket.

 ::

    {
        "jsonrpc": "2.0",
        "method": "subscribe",
        "id": "0",
        "params": {
            "query": "tm.event='NewBlock'"
        }
    }

 Check out `API docs <https://tendermint.github.io/slate/#subscribe>`__ for more
 information on query syntax and other options.

 You can also use tags, given you had included them into DeliverTx response, to
 query transaction results. See `Indexing transactions
 <./indexing-transactions.html>`__ for details.
--- a/docs/terraform-and-ansible.md
+++ b/docs/terraform-and-ansible.md
@ -0,0 +1,145 @@
 # Terraform & Ansible

 Automated deployments are done using
 [Terraform](https://www.terraform.io/) to create servers on Digital
 Ocean then [Ansible](http://www.ansible.com/) to create and manage
 testnets on those servers.

 ## Install

 NOTE: see the [integration bash
 script](https://github.com/tendermint/tendermint/blob/develop/networks/remote/integration.sh)
 that can be run on a fresh DO droplet and will automatically spin up a 4
 node testnet. The script more or less does everything described below.

 -   Install [Terraform](https://www.terraform.io/downloads.html) and
    [Ansible](http://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html)
    on a Linux machine.
 -   Create a [DigitalOcean API
    token](https://cloud.digitalocean.com/settings/api/tokens) with read
    and write capability.
 -   Install the python dopy package (`pip install dopy`)
 -   Create SSH keys (`ssh-keygen`)
 -   Set environment variables:

    export DO_API_TOKEN="abcdef01234567890abcdef01234567890"
    export SSH_KEY_FILE="$HOME/.ssh/id_rsa.pub"

 These will be used by both `terraform` and `ansible`.

 ### Terraform

 This step will create four Digital Ocean droplets. First, go to the
 correct directory:

    cd $GOPATH/src/github.com/tendermint/tendermint/networks/remote/terraform

 then:

    terraform init
    terraform apply -var DO_API_TOKEN="$DO_API_TOKEN" -var SSH_KEY_FILE="$SSH_KEY_FILE"

 and you will get a list of IP addresses that belong to your droplets.

 With the droplets created and running, let's setup Ansible.

 ### Ansible

 The playbooks in [the ansible
 directory](https://github.com/tendermint/tendermint/tree/master/networks/remote/ansible)
 run ansible roles to configure the sentry node architecture. You must
 switch to this directory to run ansible
 (`cd $GOPATH/src/github.com/tendermint/tendermint/networks/remote/ansible`).

 There are several roles that are self-explanatory:

 First, we configure our droplets by specifying the paths for tendermint
 (`BINARY`) and the node files (`CONFIGDIR`). The latter expects any
 number of directories named `node0, node1, ...` and so on (equal to the
 number of droplets created). For this example, we use pre-created files
 from [this
 directory](https://github.com/tendermint/tendermint/tree/master/docs/examples).
 To create your own files, use either the `tendermint testnet` command or
 review [manual deployments](./deploy-testnets.md).

 Here's the command to run:

    ansible-playbook -i inventory/digital_ocean.py -l sentrynet config.yml -e BINARY=$GOPATH/src/github.com/tendermint/tendermint/build/tendermint -e CONFIGDIR=$GOPATH/src/github.com/tendermint/tendermint/docs/examples

 Voila! All your droplets now have the `tendermint` binary and required
 configuration files to run a testnet.

 Next, we run the install role:

    ansible-playbook -i inventory/digital_ocean.py -l sentrynet install.yml

 which as you'll see below, executes
 `tendermint node --proxy_app=kvstore` on all droplets. Although we'll
 soon be modifying this role and running it again, this first execution
 allows us to get each `node_info.id` that corresponds to each
 `node_info.listen_addr`. (This part will be automated in the future). In
 your browser (or using `curl`), for every droplet, go to IP:46657/status
 and note the two just mentioned `node_info` fields. Notice that blocks
 aren't being created (`latest_block_height` should be zero and not
 increasing).

 Next, open `roles/install/templates/systemd.service.j2` and look for the
 line `ExecStart` which should look something like:

    ExecStart=/usr/bin/tendermint node --proxy_app=kvstore

 and add the `--p2p.persistent_peers` flag with the relevant information
 for each node. The resulting file should look something like:

    [Unit]
    Description={{service}}
    Requires=network-online.target
    After=network-online.target

    [Service]
    Restart=on-failure
    User={{service}}
    Group={{service}}
    PermissionsStartOnly=true
    ExecStart=/usr/bin/tendermint node --proxy_app=kvstore --p2p.persistent_peers=167b80242c300bf0ccfb3ced3dec60dc2a81776e@165.227.41.206:46656,3c7a5920811550c04bf7a0b2f1e02ab52317b5e6@165.227.43.146:46656,303a1a4312c30525c99ba66522dd81cca56a361a@159.89.115.32:46656,b686c2a7f4b1b46dca96af3a0f31a6a7beae0be4@159.89.119.125:46656
    ExecReload=/bin/kill -HUP $MAINPID
    KillSignal=SIGTERM

    [Install]
    WantedBy=multi-user.target

 Then, stop the nodes:

    ansible-playbook -i inventory/digital_ocean.py -l sentrynet stop.yml

 Finally, we run the install role again:

    ansible-playbook -i inventory/digital_ocean.py -l sentrynet install.yml

 to re-run `tendermint node` with the new flag, on all droplets. The
 `latest_block_hash` should now be changing and `latest_block_height`
 increasing. Your testnet is now up and running :)

 Peek at the logs with the status role:

    ansible-playbook -i inventory/digital_ocean.py -l sentrynet status.yml

 ### Logging

 The crudest way is the status role described above. You can also ship
 logs to Logz.io, an Elastic stack (Elastic search, Logstash and Kibana)
 service provider. You can set up your nodes to log there automatically.
 Create an account and get your API key from the notes on [this
 page](https://app.logz.io/#/dashboard/data-sources/Filebeat), then:

    yum install systemd-devel || echo "This will only work on RHEL-based systems."
    apt-get install libsystemd-dev || echo "This will only work on Debian-based systems."

    go get github.com/mheese/journalbeat
    ansible-playbook -i inventory/digital_ocean.py -l sentrynet logzio.yml -e LOGZIO_TOKEN=ABCDEFGHIJKLMNOPQRSTUVWXYZ012345

 ### Cleanup

 To remove your droplets, run:

    terraform destroy -var DO_API_TOKEN="$DO_API_TOKEN" -var SSH_KEY_FILE="$SSH_KEY_FILE"
--- a/docs/terraform-and-ansible.rst
+++ b/docs/terraform-and-ansible.rst
@ -1,138 +0,0 @@
 Terraform & Ansible
 ===================

 Automated deployments are done using `Terraform <https://www.terraform.io/>`__ to create servers on Digital Ocean then
 `Ansible <http://www.ansible.com/>`__ to create and manage testnets on those servers.

 Install
 -------

 NOTE: see the `integration bash script <https://github.com/tendermint/tendermint/blob/develop/networks/remote/integration.sh>`__ that can be run on a fresh DO droplet and will automatically spin up a 4 node testnet. The script more or less does everything described below.

 - Install `Terraform <https://www.terraform.io/downloads.html>`__ and `Ansible <http://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html>`__ on a Linux machine.
 - Create a `DigitalOcean API token <https://cloud.digitalocean.com/settings/api/tokens>`__ with read and write capability.
 - Install the python dopy package (``pip install dopy``)
 - Create SSH keys (``ssh-keygen``)
 - Set environment variables:

 ::

    export DO_API_TOKEN="abcdef01234567890abcdef01234567890"
    export SSH_KEY_FILE="$HOME/.ssh/id_rsa.pub"

 These will be used by both ``terraform`` and ``ansible``.

 Terraform
 ---------

 This step will create four Digital Ocean droplets. First, go to the correct directory:

 ::

    cd $GOPATH/src/github.com/tendermint/tendermint/networks/remote/terraform

 then:

 ::

    terraform init
    terraform apply -var DO_API_TOKEN="$DO_API_TOKEN" -var SSH_KEY_FILE="$SSH_KEY_FILE"

 and you will get a list of IP addresses that belong to your droplets.

 With the droplets created and running, let's setup Ansible.

 Using Ansible
 -------------

 The playbooks in `the ansible directory <https://github.com/tendermint/tendermint/tree/master/networks/remote/ansible>`__
 run ansible roles to configure the sentry node architecture. You must switch to this directory to run ansible (``cd $GOPATH/src/github.com/tendermint/tendermint/networks/remote/ansible``).

 There are several roles that are self-explanatory:

 First, we configure our droplets by specifying the paths for tendermint (``BINARY``) and the node files (``CONFIGDIR``). The latter expects any number of directories named ``node0, node1, ...`` and so on (equal to the number of droplets created). For this example, we use pre-created files from `this directory <https://github.com/tendermint/tendermint/tree/master/docs/examples>`__. To create your own files, use either the ``tendermint testnet`` command or review `manual deployments <./deploy-testnets.html>`__.

 Here's the command to run:

 ::

    ansible-playbook -i inventory/digital_ocean.py -l sentrynet config.yml -e BINARY=$GOPATH/src/github.com/tendermint/tendermint/build/tendermint -e CONFIGDIR=$GOPATH/src/github.com/tendermint/tendermint/docs/examples

 Voila! All your droplets now have the ``tendermint`` binary and required configuration files to run a testnet.

 Next, we run the install role:

 ::

    ansible-playbook -i inventory/digital_ocean.py -l sentrynet install.yml

 which as you'll see below, executes ``tendermint node --proxy_app=kvstore`` on all droplets. Although we'll soon be modifying this role and running it again, this first execution allows us to get each ``node_info.id`` that corresponds to each ``node_info.listen_addr``. (This part will be automated in the future). In your browser (or using ``curl``), for every droplet, go to IP:46657/status and note the two just mentioned ``node_info`` fields. Notice that blocks aren't being created (``latest_block_height`` should be zero and not increasing).

 Next, open ``roles/install/templates/systemd.service.j2`` and look for the line ``ExecStart`` which should look something like:

 ::

    ExecStart=/usr/bin/tendermint node --proxy_app=kvstore

 and add the ``--p2p.persistent_peers`` flag with the relevant information for each node. The resulting file should look something like:

 ::

    [Unit]
    Description={{service}}
    Requires=network-online.target
    After=network-online.target
    
    [Service]
    Restart=on-failure
    User={{service}}
    Group={{service}}
    PermissionsStartOnly=true
    ExecStart=/usr/bin/tendermint node --proxy_app=kvstore --p2p.persistent_peers=167b80242c300bf0ccfb3ced3dec60dc2a81776e@165.227.41.206:46656,3c7a5920811550c04bf7a0b2f1e02ab52317b5e6@165.227.43.146:46656,303a1a4312c30525c99ba66522dd81cca56a361a@159.89.115.32:46656,b686c2a7f4b1b46dca96af3a0f31a6a7beae0be4@159.89.119.125:46656
    ExecReload=/bin/kill -HUP $MAINPID
    KillSignal=SIGTERM

    [Install]
    WantedBy=multi-user.target

 Then, stop the nodes:

 ::

    ansible-playbook -i inventory/digital_ocean.py -l sentrynet stop.yml

 Finally, we run the install role again:

 ::

    ansible-playbook -i inventory/digital_ocean.py -l sentrynet install.yml

 to re-run ``tendermint node`` with the new flag, on all droplets. The ``latest_block_hash`` should now be changing and ``latest_block_height`` increasing. Your testnet is now up and running :)

 Peek at the logs with the status role:

 ::

    ansible-playbook -i inventory/digital_ocean.py -l sentrynet status.yml

 Logging
 -------

 The crudest way is the status role described above. You can also ship logs to Logz.io, an Elastic stack (Elastic search, Logstash and Kibana) service provider. You can set up your nodes to log there automatically. Create an account and get your API key from the notes on `this page <https://app.logz.io/#/dashboard/data-sources/Filebeat>`__, then:

 ::

   yum install systemd-devel || echo "This will only work on RHEL-based systems."
   apt-get install libsystemd-dev || echo "This will only work on Debian-based systems."

   go get github.com/mheese/journalbeat
   ansible-playbook -i inventory/digital_ocean.py -l sentrynet logzio.yml -e LOGZIO_TOKEN=ABCDEFGHIJKLMNOPQRSTUVWXYZ012345

 Cleanup
 -------

 To remove your droplets, run:

 ::

    terraform destroy -var DO_API_TOKEN="$DO_API_TOKEN" -var SSH_KEY_FILE="$SSH_KEY_FILE"
--- a/docs/transactional-semantics.md
+++ b/docs/transactional-semantics.md
@ -0,0 +1,25 @@
 # Transactional Semantics

 In [Using Tendermint](./using-tendermint.md#broadcast-api) we
 discussed different API endpoints for sending transactions and
 differences between them.

 What we have not yet covered is transactional semantics.

 When you send a transaction using one of the available methods, it first
 goes to the mempool. Currently, it does not provide strong guarantees
 like "if the transaction were accepted, it would be eventually included
 in a block (given CheckTx passes)."

 For instance a tx could enter the mempool, but before it can be sent to
 peers the node crashes.

 We are planning to provide such guarantees by using a WAL and replaying
 transactions (See
 [this issue](https://github.com/tendermint/tendermint/issues/248)), but
 it's non-trivial to do this all efficiently.

 The temporary solution is for clients to monitor the node and resubmit
 transaction(s) and/or send them to more nodes at once, so the
 probability of all of them crashing at the same time and losing the msg
 decreases substantially.
--- a/docs/transactional-semantics.rst
+++ b/docs/transactional-semantics.rst
@ -1,27 +0,0 @@
 Transactional Semantics
 =======================

 In `Using
 Tendermint <./using-tendermint.html#broadcast-api>`__ we
 discussed different API endpoints for sending transactions and
 differences between them.

 What we have not yet covered is transactional semantics.

 When you send a transaction using one of the available methods, it
 first goes to the mempool. Currently, it does not provide strong
 guarantees like "if the transaction were accepted, it would be
 eventually included in a block (given CheckTx passes)."

 For instance a tx could enter the mempool, but before it can be sent
 to peers the node crashes.

 We are planning to provide such guarantees by using a WAL and
 replaying transactions (See
 `GH#248 <https://github.com/tendermint/tendermint/issues/248>`__), but
 it's non-trivial to do this all efficiently.

 The temporary solution is for clients to monitor the node and resubmit
 transaction(s) or/and send them to more nodes at once, so the
 probability of all of them crashing at the same time and losing the
 msg decreases substantially.