Added Terraform-DigitalOcean security option, added ansible featrues. (ethermint, separated tendermint and basecoin)

ansible/README.md

@@ -125,19 +125,23 @@ python -u inventory/ec2.py --refresh-cache 1> /dev/null
 
 Note: you don't need the access key and secret key set, if you are running ansible on an Amazon AMI instance with the proper IAM permissions set.
 
-## Running the playbook
+## Running the playbooks
 
-The playbook is locked down to only run if the environment variable `TF_VAR_TESTNET_NAME` is populated. This is a precaution so you don't accidentally run the playbook on all your servers.
+The playbooks are locked down to only run if the environment variable `TF_VAR_TESTNET_NAME` is populated. This is a precaution so you don't accidentally run the playbook on all your servers.
 
 The variable `TF_VAR_TESTNET_NAME` contains the testnet name which ansible translates into an ansible group. If you used Terraform to create the servers, it was the testnet name used there.
 
 If the playbook cannot connect to the servers because of public key denial, your SSH Agent is not set up properly. Alternatively you can add the SSH key to ansible using the `--private-key` option.
 
+If you need to connect to the nodes as root but your local username is different, use the ansible option `-u root` to tell ansible to connect to the servers and authenticate as the root user.
+
+If you secured your server and you need to `sudo` for root access, use the the `-b` or `--become` option to tell ansible to sudo to root after connecting to the server. In the Terraform-DigitalOcean example, if you created the ec2-user (or if you are simply on Amazon AWS), you need to add the options `-u ec2-user -b` to ansible to tell it to connect as the ec2-user and then sudo to root to run the playbook.
+
 ### DigitalOcean
 ```
 DO_API_TOKEN="<The API token received from DigitalOcean>"
 TF_VAR_TESTNET_NAME="testnet-servers"
-ansible-playbook -i inventory/digital_ocean.py install.yml
+ansible-playbook -i inventory/digital_ocean.py install-basecoin.yml
 ```
 
 ### Amazon AWS
@@ -145,21 +149,22 @@ ansible-playbook -i inventory/digital_ocean.py install.yml
 AWS_ACCESS_KEY_ID='<The API access key ID received from Amazon>'
 AWS_SECRET_ACCESS_KEY='<The API secret access key received from Amazon>'
 TF_VAR_TESTNET_NAME="testnet-servers"
-ansible-playbook -i inventory/ec2.py install.yml
+ansible-playbook -i inventory/ec2.py install-basecoin.yml
 ```
 
 ### Installing custom versions
 
-By default ansible installs the tendermint and basecoin binary versions defined in its [default variables](#Default variables). If you build your own version of the binaries, you can tell ansible to install that instead.
+By default ansible installs the tendermint, basecoin or ethermint binary versions defined in its [default variables](#Default variables). If you build your own version of the binaries, you can tell ansible to install that instead.
 
 ```
 GOPATH="<your go path>"
 go get -u github.com/tendermint/tendermint/cmd/tendermint
 go get -u github.com/tendermint/basecoin/cmd/basecoin
+go get -u github.com/tendermint/ethermint/cmd/basecoin
 
 DO_API_TOKEN="<The API token received from DigitalOcean>"
 TF_VAR_TESTNET_NAME="testnet-servers"
-ansible-playbook -i inventory/digital_ocean.py install.yml -e tendermint_release_install=false -e basecoin_release_install=false
+ansible-playbook -i inventory/digital_ocean.py install-basecoin.yml -e tendermint_release_install=false -e basecoin_release_install=false
 ```
 
 Alternatively you can change the variable settings in `group_vars/all`.
@@ -168,22 +173,30 @@ Alternatively you can change the variable settings in `group_vars/all`.
 
 There are few extra playbooks to make life easier managing your servers.
 
-* install.yml - the all-in-one playbook to install and configure tendermint + basecoin
-* reset.yml - stop the application, reset the configuration (blockchain), then start the application again
-* stop.yml - stop the application
-* start.yml - start the application
-* restart.yml - restart the application
+* install-tendermint-core.yml - Only install the tendermint application. This is only useful if you are developing your own ABCI.
+* install-basecoin.yml - Install tendermint and basecoin applications.
+* install-ethermint.yml - Install tendermint and ethermint applications.
+* reset.yml - Stop the application, reset the configuration and data, then start the application again. You need to pass `-e service=<servicename>`, like `-e service=basecoin`. It will restart the underlying tendermint application too.
+* restart.yml - Restart a service on all nodes. You need to pass `-e service=<servicename>`, like `-e service=basecoin`. It will restart the underlying tendermint application too.
+* stop.yml - Stop the application. You need to pass `-e service=<servicename>`.
+* start.yml - Start the application. You need to pass `-e service=<servicename>`.
+* stop-basecoin.yml - Stop the basecoin and tendermint applications.
+* start-basecoin.yml - Start the basecoin and tendermint applications.
+* stop-ethermint.yml - Stop the ethermint and tendermint applications.
+* start-ethermint.yml - Start the ethermint and tendermint applications.
 
 The roles are self-sufficient under the `roles/` folder.
 
 * install-tendermint - install the tendermint application. It can install release packages or custom-compiled binaries.
 * install-basecoin - install the basecoin application. It can install release packages or custom-compiled binaries.
-* cleanupconfig - delete all tendermint and basecoin configuration.
-* config - configure tendermint and basecoin
-* stop - stop the application.
-* start - start the application.
+* install-ethermint - install the ethermint application. It can install release packages or custom-compiled binaries.
+* cleanupconfig - delete all tendermint configuration and data.
+* config - configure the tendermint application
+* stop - stop an application. Requires the `service` parameter set.
+* start - start an application. Requires the `service` parameter set.
 
 ## Default variables
 
-Default variables are documented under `group_vars/all`.
+Default variables are documented under `group_vars/all`. You can the parameters there to deploy a previously created genesis.json file (instead of dynamically creating it) or if you want to deploy custom built binaries instead of deploying a released version.
+
 

ansible/group_vars/all

@@ -21,12 +21,16 @@
 ##
 #tendermint_user: tendermint
 #tendermint_group: tendermint
-#tendermint_home: /var/lib/tendermint
+#tendermint_home: /etc/tendermint
 
 ## tendermint_log_file stores the path to the tendermint application log
 ##
 #tendermint_log_file: /var/log/tendermint.log
 
+## tendermint_genesis_file contains the path and filename to a previously generated genesis.json. If undefined, the json file is dynamically generated.
+##
+#tendermint_genesis_file: "<undefined>"
+
 ###
 ### Basecoin installation
 ###
@@ -45,9 +49,9 @@
 
 ## Details of the linux user the basecoin app uses
 ##
-#basecoin_user: basecoin
-#basecoin_group: basecoin
-#basecoin_home: /var/lib/basecoin
+#basecoin_user: tendermint
+#basecoin_group: tendermint
+#basecoin_home: /etc/tendermint
 
 ## basecoin_log_file stores the path to the basecoin application log
 ##

ansible/install-basecoin.yml

@@ -0,0 +1,11 @@
+---
+
+- hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
+  roles:
+    - install-tendermint
+    - install-basecoin
+    - cleanupconfig
+    - { role: config, testnet_name: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}" }
+    - { role: start, service: basecoin }
+    - { role: start, service: tendermint }
+

ansible/install-ethermint.yml

@@ -0,0 +1,11 @@
+---
+
+- hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
+  roles:
+    - install-tendermint
+    - install-ethermint
+    - cleanupconfig
+    - { role: config, testnet_name: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}" }
+    - { role: start, service: ethermint }
+    - { role: start, service: tendermint }
+

ansible/reset.yml

@@ -1,9 +1,13 @@
 ---
 
+#variable "service" is required
+
 - hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
-  user: root
   roles:
+    - { role: stop, service: tendermint }
     - stop
     - { role: cleanupconfig, when: not reconfig|default(false)|bool }
     - { role: config, testnet_name: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}" }
     - start
+    - { role: start, service: tendermint }
+

ansible/restart.yml

@@ -1,7 +1,8 @@
 ---
 
+#variable "service" is required
+
 - hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
-  user: root
   roles:
     - stop
     - start

ansible/roles/cleanupconfig/defaults/main.yml

@@ -1,6 +1,3 @@
 ---
-tendermint_home: /var/lib/tendermint
-
-basecoin_home: /var/lib/basecoin
-basecoin_inprocess: false
+tendermint_home: /etc/tendermint
 

ansible/roles/cleanupconfig/tasks/basecoin.yml

@@ -1,5 +0,0 @@
----
-
-- name: clear basecoin configuration folder
-  file: "path={{basecoin_home}}/.basecoin state=absent"
-

ansible/roles/cleanupconfig/tasks/main.yml

@@ -1,9 +1,10 @@
 ---
 
-- include: tendermint.yml
-  when: not basecoin_inprocess|bool
-  tags: config-tendermint
-
-- include: basecoin.yml
-  tags: config-basecoin
+- name: clear tendermint configuration folder
+  file: "path={{tendermint_home}}/{{item}} state=absent"
+  with_items:
+  - config.toml
+  - genesis.json
+  - priv_validator.json
+  - data
 

ansible/roles/cleanupconfig/tasks/tendermint.yml

@@ -1,5 +0,0 @@
----
-
-- name: clear tendermint configuration folder
-  file: "path={{tendermint_home}}/.tendermint state=absent"
-

ansible/roles/config/defaults/main.yml

@@ -1,12 +1,8 @@
 ---
 tendermint_user: tendermint
 tendermint_group: tendermint
-tendermint_home: /var/lib/tendermint
-
-basecoin_user: basecoin
-basecoin_group: basecoin
-basecoin_home: /var/lib/basecoin
-basecoin_inprocess: false
+tendermint_home: /etc/tendermint
 
+#tendermint_genesis_file="<undefined>"
 testnet_name: testnet1
 dev_money: false

ansible/roles/config/tasks/basecoin.yml

@@ -1,42 +0,0 @@
----
-
-- name: set basecoin configuration folder
-  file: "path={{basecoin_home}}/.basecoin state=directory mode=0700 owner={{basecoin_user}} group={{basecoin_group}}"
-
-- name: generate basecoin keys
-  when: basecoin_inprocess|bool
-  shell: "tendermint gen_validator > {{basecoin_home}}/.basecoin/priv_validator.json && chmod 0400 {{basecoin_home}}/.basecoin/priv_validator.json"
-  args:
-    warn: no
-    creates: "{{basecoin_home}}/.basecoin/priv_validator.json"
-  become: yes
-  become_user: "{{basecoin_user}}"
-
-- name: gather basecoin public keys
-  when: basecoin_inprocess|bool
-  command: "jq '.pub_key | .data' {{basecoin_home}}/.basecoin/priv_validator.json"
-  become: yes
-  become_user: "{{basecoin_user}}"
-  register: basecoinpubkeys
-  changed_when: false
-
-- name: register basecoin public keys as host facts
-  when: basecoin_inprocess|bool
-  set_fact: "pubkey={{basecoinpubkeys.stdout_lines[0]}}"
-  connection: local
-
-- name: copy basecoin genesis.json - genesis_time will be updated
-  template:
-    src: genesis-basecoin.json.j2
-    dest: "{{basecoin_home}}/.basecoin/genesis.json"
-  become: yes
-  become_user: "{{basecoin_user}}"
-
-- name: copy basecoin config.toml
-  when: basecoin_inprocess|bool
-  template:
-    src: config.toml.j2
-    dest: "{{basecoin_home}}/.basecoin/config.toml"
-  become: yes
-  become_user: "{{basecoin_user}}"
-

ansible/roles/config/tasks/main.yml

@@ -1,9 +1,41 @@
 ---
 
-- include: tendermint.yml
-  when: not basecoin_inprocess|bool
-  tags: config-tendermint
+- name: generate tendermint keys
+  shell: "tendermint gen_validator > {{tendermint_home}}/priv_validator.json"
+  args:
+    warn: no
+    creates: "{{tendermint_home}}/priv_validator.json"
 
-- include: basecoin.yml
-  tags: config-basecoin
+- name: secure priv_validator.json
+  file: "path={{tendermint_home}}/priv_validator.json mode=0400 owner={{tendermint_user}} group={{tendermint_group}}"
+
+- name: gather tendermint public keys
+  when: tendermint_genesis_file is not defined
+  command: "jq '.pub_key | .data' {{tendermint_home}}/priv_validator.json"
+  register: pubkeys
+  changed_when: false
+
+- name: register tendermint public keys as host facts
+  when: tendermint_genesis_file is not defined
+  set_fact: "pubkey={{pubkeys.stdout_lines[0]}}"
+  connection: local
+
+- name: copy generated tendermint genesis.json - genesis_time will be updated
+  when: tendermint_genesis_file is not defined
+  template:
+    src: genesis.json.j2
+    dest: "{{tendermint_home}}/genesis.json"
+    owner: "{{tendermint_user}}"
+    group: "{{tendermint_group}}"
+
+- name: copy pre-created genesis.json
+  when: tendermint_genesis_file is defined
+  copy: "src={{tendermint_genesis_file}} dest={{tendermint_home}}/genesis.json owner={{tendermint_user}} group={{tendermint_group}}"
+
+- name: copy tendermint config.toml
+  template:
+    src: config.toml.j2
+    dest: "{{tendermint_home}}/config.toml"
+    owner: "{{tendermint_user}}"
+    group: "{{tendermint_group}}"
 

ansible/roles/config/tasks/tendermint.yml

@@ -1,38 +0,0 @@
----
-
-- name: set tendermint configuration folder
-  file: "path={{tendermint_home}}/.tendermint state=directory mode=0700 owner={{tendermint_user}} group={{tendermint_group}}"
-
-- name: generate tendermint keys
-  shell: "tendermint gen_validator > {{tendermint_home}}/.tendermint/priv_validator.json && chmod 0400 {{tendermint_home}}/.tendermint/priv_validator.json"
-  args:
-    warn: no
-    creates: "{{tendermint_home}}/.tendermint/priv_validator.json"
-  become: yes
-  become_user: "{{tendermint_user}}"
-
-- name: gather tendermint public keys
-  command: "jq '.pub_key | .data' {{tendermint_home}}/.tendermint/priv_validator.json"
-  become: yes
-  become_user: "{{tendermint_user}}"
-  register: tendermintpubkeys
-  changed_when: false
-
-- name: register tendermint public keys as host facts
-  set_fact: "pubkey={{tendermintpubkeys.stdout_lines[0]}}"
-  connection: local
-
-- name: copy tendermint genesis.json - genesis_time will be updated
-  template:
-    src: genesis-tendermint.json.j2
-    dest: "{{tendermint_home}}/.tendermint/genesis.json"
-  become: yes
-  become_user: "{{tendermint_user}}"
-
-- name: copy tendermint config.toml
-  template:
-    src: config.toml.j2
-    dest: "{{tendermint_home}}/.tendermint/config.toml"
-  become: yes
-  become_user: "{{tendermint_user}}"
-

ansible/roles/config/templates/config.toml.j2

@@ -8,9 +8,9 @@ db_backend = "leveldb"
 log_level = "debug"
 
 [rpc]
-laddr = "tcp://127.0.0.1:46657"
+laddr = "tcp://0.0.0.0:46657"
 
 [p2p]
-laddr = "tcp://{{inventory_hostname}}:46656"
+laddr = "tcp://0.0.0.0:46656"
 {% set comma = joiner(",") %}seeds = "{% for host in groups[testnet_name]|difference(inventory_hostname) %}{{ comma() }}{{hostvars[host]["inventory_hostname"]}}:46656{% endfor %}"
 

ansible/roles/config/templates/genesis-tendermint.json.j2

@@ -1,18 +0,0 @@
-{
-  "genesis_time":"{{ansible_date_time.iso8601}}",
-  "chain_id":"{{testnet_name}}",
-  "validators":
-  [
-{% set comma = joiner(",") %}
-{% for host in groups[testnet_name] %}
-    {{ comma() }}
-    {
-      "pub_key":{"type":"ed25519","data":{{hostvars[host]["pubkey"]}}},
-      "amount":10,
-      "name":"{{hostvars[host]["ansible_hostname"]}}"
-    }
-{% endfor %}
-  ],
-  "app_hash":""
-}
-

ansible/roles/install-basecoin/defaults/main.yml

@@ -4,10 +4,10 @@ basecoin_download: "https://s3-us-west-2.amazonaws.com/tendermint/binaries/basec
 
 basecoin_binary: "{{ lookup('env','GOPATH') | default('') }}/bin/basecoin"
 
-basecoin_user: basecoin
-basecoin_group: basecoin
+basecoin_user: tendermint
+basecoin_group: tendermint
 
-basecoin_home: /var/lib/basecoin
+basecoin_home: /etc/tendermint
 basecoin_log_file: /var/log/basecoin.log
 
 basecoin_inprocess: false

ansible/roles/install-basecoin/handlers/main.yml

@@ -1,6 +1,5 @@
 ---
 
 - name: reload systemd
-  become: true
   command: systemctl daemon-reload
 

ansible/roles/install-basecoin/templates/basecoin.conf.j2

@@ -6,7 +6,7 @@ stop on runlevel [!2345]
 
 script
   # Use su to become tendermint user non-interactively on old Upstart versions (see http://superuser.com/a/234541/76168)
-  exec su -s /bin/sh -c 'BCHOME={{basecoin_home}}/.basecoin exec "$0" "$@" >> {{ basecoin_log_file }} 2>&1' {{basecoin_user}} -- /usr/local/bin/basecoin start{{basecoin_inprocess|bool | ternary('',' --without-tendermint')}}
+  exec su -s /bin/sh -c 'BCHOME={{basecoin_home}} exec "$0" "$@" >> {{ basecoin_log_file }} 2>&1' {{basecoin_user}} -- /usr/local/bin/basecoin start{{basecoin_inprocess|bool | ternary('',' --without-tendermint')}}
 end script
 
 respawn

ansible/roles/install-basecoin/templates/basecoin.systemd.j2

@@ -4,7 +4,7 @@ Requires=network-online.target
 After=network-online.target
 
 [Service]
-Environment="BCHOME={{basecoin_home}}/.basecoin"
+Environment="BCHOME={{basecoin_home}}"
 Restart=on-failure
 User={{ basecoin_user }}
 Group={{ basecoin_group }}

ansible/roles/install-ethermint/defaults/main.yml

@@ -0,0 +1,11 @@
+---
+ethermint_release_install: true
+ethermint_download: "https://s3.eu-central-1.amazonaws.com/ethermint/0.2.0/ethermint_0.2.0_ethermint-linux-amd64.zip"
+ethermint_binary: "{{ lookup('env','GOPATH') | default('') }}/bin/ethermint"
+
+ethermint_user: tendermint
+ethermint_group: tendermint
+
+ethermint_home: /etc/tendermint
+ethermint_log_file: /var/log/ethermint.log
+

ansible/roles/install-ethermint/handlers/main.yml

@@ -0,0 +1,5 @@
+---
+
+- name: reload systemd
+  command: systemctl daemon-reload
+

ansible/roles/install-ethermint/tasks/install.yml

@@ -0,0 +1,104 @@
+---
+- name: update apt
+  apt: >
+    update_cache=yes
+    cache_valid_time=3600
+  when: ansible_os_family == "Debian"
+
+- name: install deps (Ubuntu)
+  apt: >
+    pkg={{item}}
+    state=installed
+  with_items:
+    - unzip
+    - jq
+  when: ansible_os_family == "Debian"
+
+- name: install deps (RHEL)
+  yum: >
+    pkg={{item}}
+    state=installed
+  with_items:
+    - unzip
+    - jq
+  when: ansible_os_family == "RedHat"
+
+- name: create group
+  group: >
+    name={{ethermint_group}}
+    state=present
+    system=yes
+  register: ethermint_group_created
+
+- name: create user
+  user: >
+    home={{ethermint_home}}
+    name={{ethermint_user}}
+    group={{ethermint_group}}
+    system=yes
+  when: ethermint_group_created|changed
+
+- name: create directory
+  file: >
+    path={{ethermint_home}}
+    state=directory
+    owner={{ethermint_user}}
+    group={{ethermint_group}}
+    mode=0755
+
+# Check before creating log dir to prevent aggressively overwriting permissions
+- name: check for log directory
+  stat: >
+    path={{ ethermint_log_file|dirname }}
+  register: logdir
+
+- name: create log directory if it does not exist
+  file: >
+    state=directory
+    path={{ ethermint_log_file|dirname }}
+    owner={{ ethermint_user }}
+    group={{ ethermint_group }}
+  when: not logdir.stat.exists
+
+- name: touch the log file
+  file: >
+    state=touch
+    path={{ ethermint_log_file }}
+    owner={{ ethermint_user }}
+    group={{ ethermint_group }}
+  changed_when: false
+
+- name: copy and unpack release binary
+  when: ethermint_release_install|bool
+  unarchive: >
+    src={{ethermint_download}}
+    dest=/usr/local/bin
+    remote_src=true
+    mode=0755
+
+- name: copy compiled binary
+  when: not ethermint_release_install|bool
+  copy: >
+    src={{ethermint_binary}}
+    dest=/usr/local/bin
+    mode=0755
+
+- name: copy upstart script
+  template: >
+    src=ethermint.conf.j2
+    dest=/etc/init/ethermint.conf
+    owner=root
+    group=root
+    mode=0644
+  when: "ansible_service_mgr == 'upstart'"
+
+- name: copy systemd script
+  template: >
+    src=ethermint.systemd.j2
+    dest=/etc/systemd/system/ethermint.service
+    owner=root
+    group=root
+    mode=0644
+  when: "ansible_service_mgr == 'systemd'"
+  notify:
+    - reload systemd

ansible/roles/install-ethermint/tasks/main.yml

@@ -0,0 +1,8 @@
+---
+- name: gather OS specific variables
+  include_vars: "{{ item }}"
+  with_first_found:
+    - "{{ ansible_os_family }}-{{ ansible_distribution_major_version}}.yml"
+    - "{{ ansible_os_family }}.yml"
+- include: install.yml
+

ansible/roles/install-ethermint/templates/ethermint.conf.j2

@@ -0,0 +1,15 @@
+# Ethermint (Upstart unit)
+description "Ethermint"
+
+start on runlevel [2345]
+stop on runlevel [!2345]
+
+script
+  # Use su to become ethermint user non-interactively on old Upstart versions (see http://superuser.com/a/234541/76168)
+  exec su -s /bin/sh -c 'EMHOME={{ethermint_home}} exec "$0" "$@" >> {{ ethermint_log_file }} 2>&1' {{ ethermint_user }} -- /usr/local/bin/ethermint start
+end script
+
+respawn
+respawn limit 10 10
+
+kill timeout 10

ansible/roles/install-ethermint/templates/ethermint.systemd.j2

@@ -0,0 +1,17 @@
+[Unit]
+Description=Ethermint
+Requires=network-online.target
+After=network-online.target
+
+[Service]
+Environment="EMHOME={{ethermint_home}}"
+Restart=on-failure
+User={{ ethermint_user }}
+Group={{ ethermint_group }}
+PermissionsStartOnly=true
+ExecStart=/usr/local/bin/ethermint start
+ExecReload=/bin/kill -HUP $MAINPID
+KillSignal=SIGINT
+
+[Install]
+WantedBy=multi-user.target

ansible/roles/install-ethermint/vars/Debian.yml

@@ -0,0 +1,2 @@
+---
+#Placeholder

ansible/roles/install-ethermint/vars/RedHat.yml

@@ -0,0 +1,2 @@
+---
+#Placeholder

ansible/roles/install-tendermint/defaults/main.yml

@@ -7,7 +7,7 @@ tendermint_binary: "{{ lookup('env','GOPATH') | default('') }}/bin/tendermint"
 tendermint_user: tendermint
 tendermint_group: tendermint
 
-tendermint_home: /var/lib/tendermint
+tendermint_home: /etc/tendermint
 tendermint_log_file: /var/log/tendermint.log
 
 basecoin_inprocess: false

ansible/roles/install-tendermint/handlers/main.yml

@@ -1,6 +1,5 @@
 ---
 
 - name: reload systemd
-  become: true
   command: systemctl daemon-reload
 

ansible/roles/install-tendermint/templates/tendermint.conf.j2

@@ -6,7 +6,7 @@ description "Tendermint"
 
 script
   # Use su to become tendermint user non-interactively on old Upstart versions (see http://superuser.com/a/234541/76168)
-  exec su -s /bin/sh -c 'TMHOME={{tendermint_home}}/.tendermint exec "$0" "$@" >> {{ tendermint_log_file }} 2>&1' {{ tendermint_user }} -- /usr/local/bin/tendermint node
+  exec su -s /bin/sh -c 'TMHOME={{tendermint_home}} exec "$0" "$@" >> {{ tendermint_log_file }} 2>&1' {{ tendermint_user }} -- /usr/local/bin/tendermint node
 end script
 
 respawn

ansible/roles/install-tendermint/templates/tendermint.systemd.j2

@@ -4,7 +4,7 @@ Requires=network-online.target{{basecoin_inprocess|bool | ternary('',' basecoin.
 After=network-online.target
 
 [Service]
-Environment="TMHOME={{tendermint_home}}/.tendermint"
+Environment="TMHOME={{tendermint_home}}"
 Restart=on-failure
 User={{ tendermint_user }}
 Group={{ tendermint_group }}

ansible/roles/start/tasks/main.yml

@@ -1,9 +1,5 @@
 ---
 
-- name: start basecoin
-  service: name=basecoin state=started
-
-- name: start tendermint
-  when: not basecoin_inprocess|bool
-  service: name=tendermint state=started
+- name: start service
+  service: "name={{service}} state=started"
 

ansible/roles/stop/tasks/main.yml

@@ -1,9 +1,5 @@
 ---
 
-- name: stop tendermint
-  #when: not basecoin_inprocess|bool
-  service: name=tendermint state=stopped
-
-- name: stop basecoin
-  service: name=basecoin state=stopped
+- name: stop service
+  service: "name={{service}} state=stopped"
 

ansible/start-basecoin.yml

@@ -0,0 +1,7 @@
+---
+
+- hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
+  roles:
+    - { role: start, service: "basecoin" }
+    - { role: start, service: "tendermint" }
+

ansible/start-ethermint.yml

@@ -0,0 +1,7 @@
+---
+
+- hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
+  roles:
+    - { role: start, service: "ethermint" }
+    - { role: start, service: "tendermint" }
+

ansible/start.yml

@@ -1,7 +1,8 @@
 ---
 
+#variable "service" is required
+
 - hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
-  user: root
   roles:
     - start
 

ansible/stop-basecoin.yml

@@ -0,0 +1,7 @@
+---
+
+- hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
+  roles:
+    - { role: stop, service: "tendermint"}
+    - { role: stop, service: "basecoin"}
+

ansible/stop-ethermint.yml

@@ -0,0 +1,7 @@
+---
+
+- hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
+  roles:
+    - { role: stop, service: "tendermint"}
+    - { role: stop, service: "ethermint"}
+

ansible/stop.yml

@@ -1,7 +1,8 @@
 ---
 
+#variable "service" is required
+
 - hosts: "{{ lookup('env','TF_VAR_TESTNET_NAME') }}"
-  user: root
   roles:
     - stop
 

terraform-digitalocean/README.md

@@ -50,8 +50,16 @@ TF_VAR_TESTNET_NAME="testnet-servers"
 terraform-apply
 ```
 
+## Security
+
+DigitalOcean uses the root user by default on its droplets. This is fine as long as SSH keys are used. However some people still would like to disable root and use an alternative user to connect to the droplets - then `sudo` from there.
+There is a way to do this with Terraform but it requires SSH agent running on the machine where terraform is run, with one of the SSH keys of the droplets added to the agent. (This will be neede for ansible too, so it's worth setting it up here. Check out the [ansible](https://github.com/tendermint/tools/tree/master/ansible) page for more information.)
+After setting up the SSH key, uncomment the `provider` block under `cluster/main.tf` and run `terraform apply` to create your droplets. Terraform will create a user called `ec2-user` and move the SSH keys over, this way disabling SSH login for root. It also adds the `ec2-user` to the sudoers file, so after logging in as ec2-user you can `sudo` to `root`.
+
+DigitalOcean announced firewalls but the current version of Terraform (0.9.6 as of this writing) does not support it yet. Fortunately it is quite easy to set it up through the web interface (and not that bad through the [RESTful API](https://developers.digitalocean.com/documentation/v2/#firewalls) either). When adding droplets to a firewall rule, you can add tags, so it's enough to define the testnet name. It is not necessary to add the nodes one-by-one. Also, the firewall rule "remembers" the testnet name tag so if you change the servers but keep the name, the firewall rules will still apply.
+
 # What's next
 
-After setting up the nodes, head over to the [ansible folder](https://github.com/tendermint/tools) to set up tendermint and basecoin.
+After setting up the nodes, head over to the [ansible folder](https://github.com/tendermint/tools/tree/master/ansible) to set up tendermint and basecoin.
 
 

terraform-digitalocean/cluster/main.tf

@@ -21,6 +21,18 @@ resource "digitalocean_droplet" "cluster" {
     timeout = "30s"
   }
 
+#  #Additional SSH security: add ec2-user and remove root user credentials. You need to have ssh-agent running with your key for this to work.
+#  provisioner "remote-exec" {
+#    inline = [
+#      "useradd -m -s /bin/bash ec2-user",
+#      "echo 'ec2-user ALL=(ALL) NOPASSWD:ALL' > /etc/sudoers.d/ec2-user",
+#      "cp -r /root/.ssh /home/ec2-user/.ssh",
+#      "chown -R ec2-user.ec2-user /home/ec2-user/.ssh",
+#      "chmod -R 700 /home/ec2-user/.ssh",
+#      "rm -rf /root/.ssh"
+#    ]
+#  }
+
 }
 
 #resource "digitalocean_floating_ip" "cluster" {