You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
138 lines
5.8 KiB
138 lines
5.8 KiB
Terraform & Ansible
|
|
===================
|
|
|
|
Automated deployments are done using `Terraform <https://www.terraform.io/>`__ to create servers on Digital Ocean then
|
|
`Ansible <http://www.ansible.com/>`__ to create and manage testnets on those servers.
|
|
|
|
Install
|
|
-------
|
|
|
|
NOTE: see the `integration bash script <https://github.com/tendermint/tendermint/blob/zach/ansible/networks/remote/integration.sh>`__ that can be run on a fresh DO droplet and will automatically spin up a 4 node testnet. The script more or less does everything described below.
|
|
|
|
- Install `Terraform <https://www.terraform.io/downloads.html>`__ and `Ansible <http://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html>`__ on a Linux machine.
|
|
- Create a `DigitalOcean API token <https://cloud.digitalocean.com/settings/api/tokens>`__ with read and write capability.
|
|
- Install the python dopy package (``pip install dopy``)
|
|
- Create SSH keys (``ssh-keygen``)
|
|
- Set environment variables:
|
|
|
|
::
|
|
|
|
export DO_API_TOKEN="abcdef01234567890abcdef01234567890"
|
|
export SSH_KEY_FILE="$HOME/.ssh/id_rsa.pub"
|
|
|
|
These will be used by both ``terraform`` and ``ansible``.
|
|
|
|
Terraform
|
|
---------
|
|
|
|
This step will create four Digital Ocean droplets. First, go to the correct directory:
|
|
|
|
::
|
|
|
|
cd $GOPATH/src/github.com/tendermint/tendermint/networks/remote/terraform
|
|
|
|
then:
|
|
|
|
::
|
|
|
|
terraform init
|
|
terraform apply -var DO_API_TOKEN="$DO_API_TOKEN" -var SSH_KEY_FILE="$SSH_KEY_FILE"
|
|
|
|
and you will get a list of IP addresses that belong to your droplets.
|
|
|
|
With the droplets created and running, let's setup Ansible.
|
|
|
|
Using Ansible
|
|
-------------
|
|
|
|
The playbooks in `the ansible directory <https://github.com/tendermint/tendermint/tree/master/networks/remote/ansible>`__
|
|
run ansible roles to configure the sentry node architecture. You must switch to this directory to run ansible (``cd $GOPATH/src/github.com/tendermint/tendermint/networks/remote/ansible``).
|
|
|
|
There are several roles that are self-explanatory:
|
|
|
|
First, we configure our droplets by specifying the paths for tendermint (``BINARY``) and the node files (``CONFIGDIR``). The latter expects any number of directories named ``node0, node1, ...`` and so on (equal to the number of droplets created). For this example, we use pre-created files from `this directory <https://github.com/tendermint/tendermint/tree/master/docs/examples>`__. To create your own files, use either the ``tendermint testnet`` command or review `manual deployments <./deploy-testnets.html>`__.
|
|
|
|
Here's the command to run:
|
|
|
|
::
|
|
|
|
ansible-playbook -i inventory/digital_ocean.py -l sentrynet config.yml -e BINARY=$GOPATH/src/github.com/tendermint/tendermint/build/tendermint -e CONFIGDIR=$GOPATH/src/github.com/tendermint/tendermint/docs/examples
|
|
|
|
Voila! All your droplets now have the ``tendermint`` binary and required configuration files to run a testnet.
|
|
|
|
Next, we run the install role:
|
|
|
|
::
|
|
|
|
ansible-playbook -i inventory/digital_ocean.py -l sentrynet install.yml
|
|
|
|
which as you'll see below, executes ``tendermint node --proxy_app=kvstore`` on all droplets. Although we'll soon be modifying this role and running it again, this first execution allows us to get each ``node_info.id`` that corresponds to each ``node_info.listen_addr``. (This part will be automated in the future). In your browser (or using ``curl``), for every droplet, go to IP:46657/status and note the two just mentioned ``node_info`` fields. Notice that blocks aren't being created (``latest_block_height`` should be zero and not increasing).
|
|
|
|
Next, open ``roles/install/templates/systemd.service.j2`` and look for the line ``ExecStart`` which should look something like:
|
|
|
|
::
|
|
|
|
ExecStart=/usr/bin/tendermint node --proxy_app=kvstore
|
|
|
|
and add the ``--p2p.persistent_peers`` flag with the relevant information for each node. The resulting file should look something like:
|
|
|
|
::
|
|
|
|
[Unit]
|
|
Description={{service}}
|
|
Requires=network-online.target
|
|
After=network-online.target
|
|
|
|
[Service]
|
|
Restart=on-failure
|
|
User={{service}}
|
|
Group={{service}}
|
|
PermissionsStartOnly=true
|
|
ExecStart=/usr/bin/tendermint node --proxy_app=kvstore --p2p.persistent_peers=167b80242c300bf0ccfb3ced3dec60dc2a81776e@165.227.41.206:46656,3c7a5920811550c04bf7a0b2f1e02ab52317b5e6@165.227.43.146:46656,303a1a4312c30525c99ba66522dd81cca56a361a@159.89.115.32:46656,b686c2a7f4b1b46dca96af3a0f31a6a7beae0be4@159.89.119.125:46656
|
|
ExecReload=/bin/kill -HUP $MAINPID
|
|
KillSignal=SIGTERM
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|
|
|
|
Then, stop the nodes:
|
|
|
|
::
|
|
|
|
ansible-playbook -i inventory/digital_ocean.py -l sentrynet stop.yml
|
|
|
|
Finally, we run the install role again:
|
|
|
|
::
|
|
|
|
ansible-playbook -i inventory/digital_ocean.py -l sentrynet install.yml
|
|
|
|
to re-run ``tendermint node`` with the new flag, on all droplets. The ``latest_block_hash`` should now be changing and ``latest_block_height`` increasing. Your testnet is now up and running :)
|
|
|
|
Peek at the logs with the status role:
|
|
|
|
::
|
|
|
|
ansible-playbook -i inventory/digital_ocean.py -l sentrynet status.yml
|
|
|
|
Logging
|
|
-------
|
|
|
|
The crudest way is the status role described above. You can also ship logs to Logz.io, an Elastic stack (Elastic search, Logstash and Kibana) service provider. You can set up your nodes to log there automatically. Create an account and get your API key from the notes on `this page <https://app.logz.io/#/dashboard/data-sources/Filebeat>`__, then:
|
|
|
|
::
|
|
|
|
yum install systemd-devel || echo "This will only work on RHEL-based systems."
|
|
apt-get install libsystemd-dev || echo "This will only work on Debian-based systems."
|
|
|
|
go get github.com/mheese/journalbeat
|
|
ansible-playbook -i inventory/digital_ocean.py -l sentrynet logzio.yml -e LOGZIO_TOKEN=ABCDEFGHIJKLMNOPQRSTUVWXYZ012345
|
|
|
|
Cleanup
|
|
-------
|
|
|
|
To remove your droplets, run:
|
|
|
|
::
|
|
|
|
terraform destroy -var DO_API_TOKEN="$DO_API_TOKEN" -var SSH_KEY_FILE="$SSH_KEY_FILE"
|