diff --git a/DOCKER/Dockerfile b/DOCKER/Dockerfile index ecd863d88..de7f5a1d0 100644 --- a/DOCKER/Dockerfile +++ b/DOCKER/Dockerfile @@ -1,18 +1,35 @@ FROM alpine:3.7 MAINTAINER Greg Szabo -#Default home for tendermint. The node command will look for $TMHOME/config/genesis.json at initialization. +# Tendermint will be looking for the genesis file in /tendermint/config/genesis.json +# (unless you change `genesis_file` in config.toml). You can put your config.toml and +# private validator file into /tendermint/config. +# +# The /tendermint/data dir is used by tendermint to store state. ENV TMHOME /tendermint + +# OS environment setup +# Set user right away for determinism, create directory for persistence and give our user ownership +# jq and curl used for extracting `pub_key` from private validator while +# deploying tendermint with Kubernetes. It is nice to have bash so the users +# could execute bash commands. RUN apk update && \ apk upgrade && \ - apk --no-cache add curl jq && \ + apk --no-cache add curl jq bash && \ addgroup tmuser && \ adduser -S -G tmuser tmuser -h "$TMHOME" +# Run the container with tmuser by default. (UID=100, GID=1000) USER tmuser + +# Expose the data directory as a volume since there's mutable state in there VOLUME [ $TMHOME ] + WORKDIR $TMHOME + +# p2p and rpc port EXPOSE 46656 46657 + ENTRYPOINT ["/usr/bin/tendermint"] CMD ["node", "--moniker=`hostname`"] STOPSIGNAL SIGTERM diff --git a/Makefile b/Makefile index 809352509..3ce4f291b 100755 --- a/Makefile +++ b/Makefile @@ -194,12 +194,12 @@ build-linux: GOOS=linux GOARCH=amd64 $(MAKE) build # Run a 4-node testnet locally -docker-start: +localnet-start: @if ! [ -f build/node0/config/genesis.json ]; then docker run --rm -v $(CURDIR)/build:/tendermint:Z tendermint/localnode testnet --v 4 --o . --populate-persistent-peers --starting-ip-address 192.167.10.2 ; fi docker-compose up # Stop testnet -docker-stop: +localnet-stop: docker-compose down ########################################################### @@ -225,5 +225,5 @@ server-destroy: # To avoid unintended conflicts with file names, always add to .PHONY # unless there is a reason not to. # https://www.gnu.org/software/make/manual/html_node/Phony-Targets.html -.PHONY: check build build_race dist install check_tools get_tools update_tools get_vendor_deps draw_deps test_cover test_apps test_persistence test_p2p test test_race test_integrations test_release test100 vagrant_test fmt build-linux docker-start docker-stop build-docker server-setup server-config server-destroy +.PHONY: check build build_race dist install check_tools get_tools update_tools get_vendor_deps draw_deps test_cover test_apps test_persistence test_p2p test test_race test_integrations test_release test100 vagrant_test fmt build-linux localnet-start localnet-stop build-docker server-setup server-config server-destroy diff --git a/networks/remote/ansible/README.rst b/networks/remote/ansible/README.rst index 5c416c018..0f95de73f 100644 --- a/networks/remote/ansible/README.rst +++ b/networks/remote/ansible/README.rst @@ -6,286 +6,34 @@ Using Ansible Ansible plus Tendermint -The playbooks in `our ansible directory `__ -run ansible `roles `__ which: - -- install and configure basecoind or ethermint -- start/stop basecoind or ethermint and reset their configuration +The playbooks in `the ansible directory `__ +run ansible `roles `__ to configure the sentry node architecture. Prerequisites ------------- -- Ansible 2.0 or higher -- SSH key to the servers - -Optional for DigitalOcean droplets: - -- DigitalOcean API Token -- python dopy package - -For a description on how to get a DigitalOcean API Token, see the explanation -in the `using terraform tutorial <./terraform-digitalocean.html>`__. - -Optional for Amazon AWS instances: - -- Amazon AWS API access key ID and secret access key. +- Install `Ansible 2.0 or higher `__ on a linux machine. +- Create a `DigitalOcean API token `__ with read and write capability. +- Create SSH keys +- Install the python dopy package (for the digital_ocean.py script) -The cloud inventory scripts come from the ansible team at their -`GitHub `__ page. You can get the -latest version from the ``contrib/inventory`` folder. -Setup +Build ----- -Ansible requires a "command machine" or "local machine" or "orchestrator -machine" to run on. This can be your laptop or any machine that can run -ansible. (It does not have to be part of the cloud network that hosts -your servers.) - -Use the official `Ansible installation -guide `__ to -install Ansible. Here are a few examples on basic installation commands: - -Ubuntu/Debian: - -:: - - sudo apt-get install ansible - -CentOS/RedHat: - -:: - - sudo yum install epel-release - sudo yum install ansible - -Mac OSX: If you have `Homebrew `__ installed, then it's: - -:: - - brew install ansible - -If not, you can install it using ``pip``: - -:: - - sudo easy_install pip - sudo pip install ansible - -To make life easier, you can start an SSH Agent and load your SSH -key(s). This way ansible will have an uninterrupted way of connecting to -your servers. - -:: - - ssh-agent > ~/.ssh/ssh.env - source ~/.ssh/ssh.env - - ssh-add private.key - -Subsequently, as long as the agent is running, you can use -``source ~/.ssh/ssh.env`` to load the keys to the current session. Note: -On Mac OSX, you can add the ``-K`` option to ssh-add to store the -passphrase in your keychain. The security of this feature is debated but -it is convenient. - -Optional cloud dependencies -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If you are using a cloud provider to host your servers, you need the -below dependencies installed on your local machine. - -DigitalOcean inventory dependencies: -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Ubuntu/Debian: - -:: - - sudo apt-get install python-pip - sudo pip install dopy - -CentOS/RedHat: - -:: - - sudo yum install python-pip - sudo pip install dopy - -Mac OSX: - -:: - - sudo pip install dopy - -Amazon AWS inventory dependencies: -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Ubuntu/Debian: - -:: - - sudo apt-get install python-boto - -CentOS/RedHat: - :: - sudo yum install python-boto - -Mac OSX: - -:: - - sudo pip install boto - -Refreshing the DigitalOcean inventory -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If you just finished creating droplets, the local DigitalOcean inventory -cache is not up-to-date. To refresh it, run: - -:: - - DO_API_TOKEN="" - python -u inventory/digital_ocean.py --refresh-cache 1> /dev/null - -Refreshing the Amazon AWS inventory -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If you just finished creating Amazon AWS EC2 instances, the local AWS -inventory cache is not up-to-date. To refresh it, run: - -:: - - AWS_ACCESS_KEY_ID='' - AWS_SECRET_ACCESS_KEY='' - python -u inventory/ec2.py --refresh-cache 1> /dev/null - -Note: you don't need the access key and secret key set, if you are -running ansible on an Amazon AMI instance with the proper IAM -permissions set. - -Running the playbooks ---------------------- - -The playbooks are locked down to only run if the environment variable -``TF_VAR_TESTNET_NAME`` is populated. This is a precaution so you don't -accidentally run the playbook on all your servers. - -The variable ``TF_VAR_TESTNET_NAME`` contains the testnet name which -ansible translates into an ansible group. If you used Terraform to -create the servers, it was the testnet name used there. - -If the playbook cannot connect to the servers because of public key -denial, your SSH Agent is not set up properly. Alternatively you can add -the SSH key to ansible using the ``--private-key`` option. - -If you need to connect to the nodes as root but your local username is -different, use the ansible option ``-u root`` to tell ansible to connect -to the servers and authenticate as the root user. - -If you secured your server and you need to ``sudo`` for root access, use -the the ``-b`` or ``--become`` option to tell ansible to sudo to root -after connecting to the server. In the Terraform-DigitalOcean example, -if you created the ec2-user by adding the ``noroot=true`` option (or if -you are simply on Amazon AWS), you need to add the options -``-u ec2-user -b`` to ansible to tell it to connect as the ec2-user and -then sudo to root to run the playbook. - -DigitalOcean -~~~~~~~~~~~~ - -:: - - DO_API_TOKEN="" - TF_VAR_TESTNET_NAME="testnet-servers" - ansible-playbook -i inventory/digital_ocean.py install.yml -e service=basecoind - -Amazon AWS -~~~~~~~~~~ - -:: - - AWS_ACCESS_KEY_ID='' - AWS_SECRET_ACCESS_KEY='' - TF_VAR_TESTNET_NAME="testnet-servers" - ansible-playbook -i inventory/ec2.py install.yml -e service=basecoind - -Installing custom versions -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -By default ansible installs the tendermint, basecoind or ethermint binary -versions from the latest release in the repository. If you build your -own version of the binaries, you can tell ansible to install that -instead. - -:: - - GOPATH="" - go get -u github.com/tendermint/basecoin/cmd/basecoind - - DO_API_TOKEN="" - TF_VAR_TESTNET_NAME="testnet-servers" - ansible-playbook -i inventory/digital_ocean.py install.yml -e service=basecoind -e release_install=false - -Alternatively you can change the variable settings in -``group_vars/all``. - -Other commands and roles ------------------------- - -There are few extra playbooks to make life easier managing your servers. - -- install.yml - Install basecoind or ethermint applications. (Tendermint - gets installed automatically.) Use the ``service`` parameter to - define which application to install. Defaults to ``basecoind``. -- reset.yml - Stop the application, reset the configuration and data, - then start the application again. You need to pass - ``-e service=``, like ``-e service=basecoind``. It will - restart the underlying tendermint application too. -- restart.yml - Restart a service on all nodes. You need to pass - ``-e service=``, like ``-e service=basecoind``. It will - restart the underlying tendermint application too. -- stop.yml - Stop the application. You need to pass - ``-e service=``. -- status.yml - Check the service status and print it. You need to pass - ``-e service=``. -- start.yml - Start the application. You need to pass - ``-e service=``. -- ubuntu16-patch.yml - Ubuntu 16.04 does not have the minimum required - python package installed to be able to run ansible. If you are using - ubuntu, run this playbook first on the target machines. This will - install the python pacakge that is required for ansible to work - correctly on the remote nodes. -- upgrade.yml - Upgrade the ``service`` on your testnet. It will stop - the service and restart it at the end. It will only work if the - upgraded version is backward compatible with the installed version. -- upgrade-reset.yml - Upgrade the ``service`` on your testnet and reset - the database. It will stop the service and restart it at the end. It - will work for upgrades where the new version is not - backward-compatible with the installed version - however it will - reset the testnet to its default. + export DO_API_TOKEN="abcdef01234567890abcdef01234567890" + export SSH_KEY_FILE="$HOME/.ssh/id_rsa.pub" -The roles are self-sufficient under the ``roles/`` folder. + + ansible-playbook -i inventory/digital_ocean.py -l remotenet install.yml -- install - install the application defined in the ``service`` - parameter. It can install release packages and update them with - custom-compiled binaries. -- unsafe\_reset - delete the database for a service, including the - tendermint database. -- config - configure the application defined in ``service``. It also - configures the underlying tendermint service. Check - ``group_vars/all`` for options. -- stop - stop an application. Requires the ``service`` parameter set. -- status - check the status of an application. Requires the ``service`` - parameter set. -- start - start an application. Requires the ``service`` parameter set. + # The scripts assume that you have your validator set up already. + # You can create the folder structure for the sentry nodes using `tendermint testnet`. + # For example: tendermint testnet --v 0 --n 4 --o build/ + # Then copy your genesis.json and modify the config.toml as you see fit. -Default variables ------------------ + # Reconfig the sentry nodes with a new BINARY and the configuration files from the build folder: + ansible-playbook -i inventory/digital_ocean.py -l remotenet config.yml -e BINARY=`pwd`/build/tendermint -e CONFIGDIR=`pwd`/build -Default variables are documented under ``group_vars/all``. You can the -parameters there to deploy a previously created genesis.json file -(instead of dynamically creating it) or if you want to deploy custom -built binaries instead of deploying a released version.