diff --git a/tools/mintnet-kubernetes/README.rst b/tools/mintnet-kubernetes/README.rst deleted file mode 100644 index edf5e81e5..000000000 --- a/tools/mintnet-kubernetes/README.rst +++ /dev/null @@ -1,290 +0,0 @@ -Using Kubernetes -================ - -.. figure:: assets/t_plus_k.png - :alt: Tendermint plus Kubernetes - - Tendermint plus Kubernetes - -This should primarily be used for testing purposes or for -tightly-defined chains operated by a single stakeholder (see `the -security precautions <#security>`__). If your desire is to launch an -application with many stakeholders, consider using our set of Ansible -scripts. - -Quick Start ------------ - -For either platform, see the `requirements `__ - -MacOS -^^^^^ - -:: - - curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/kubectl - curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.18.0/minikube-darwin-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/ - minikube start - - git clone https://github.com/tendermint/tools.git && cd tools/mintnet-kubernetes/examples/basecoin && make create - -Linux -^^^^^ - -:: - - curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/kubectl - curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.18.0/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/ - minikube start - - git clone https://github.com/tendermint/tools.git && cd tools/mintnet-kubernetes/examples/basecoin && make create - -Verify it worked -~~~~~~~~~~~~~~~~ - -**Using a shell:** - -First wait until all the pods are ``Running``: - -``kubectl get pods -w -o wide -L tm`` - -then query the Tendermint app logs from the first pod: - -``kubectl logs -c tm -f tm-0`` - -finally, use our `Rest API `__ to fetch the status of the second pod's Tendermint app. - -Note we are using ``kubectl exec`` because pods are not exposed (and should not be) to the -outer network: - -``kubectl exec -c tm tm-0 -- curl -s http://tm-1.basecoin:26657/status | json_pp`` - -**Using the dashboard:** - -:: - - minikube dashboard - -Clean up -~~~~~~~~ - -:: - - make destroy - -Usage ------ - -Setup a Kubernetes cluster -^^^^^^^^^^^^^^^^^^^^^^^^^^ - -- locally using `Minikube `__ -- on GCE with a single click in the web UI -- on AWS using `Kubernetes - Operations `__ -- on Linux machines (Digital Ocean) using - `kubeadm `__ -- on AWS, Azure, GCE or bare metal using `Kargo - (Ansible) `__ - -Please refer to `the official -documentation `__ -for overview and comparison of different options. - -Kubernetes on Digital Ocean -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Available options: - -- `kubeadm (alpha) `__ -- `kargo `__ -- `rancher `__ -- `terraform `__ - -As you can see, there is no single tool for creating a cluster on DO. -Therefore, choose the one you know and comfortable working with. If you know -and used `terraform `__ before, then choose it. If you -know Ansible, then pick kargo. If none of these seem familiar to you, go with -``kubeadm``. Rancher is a beautiful UI for deploying and managing containers in -production. - -Kubernetes on Google Cloud Engine -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Review the `Official Documentation `__ for Kubernetes on Google Compute -Engine. - -**Create a cluster** - -The recommended way is to use `Google Container -Engine `__. You should be able -to create a fully fledged cluster with just a few clicks. - -**Connect to it** - -Install ``gcloud`` as a part of `Google Cloud SDK `__. - -Make sure you have credentials for GCloud by running ``gcloud auth login``. - -In order to make API calls against GCE, you must also run ``gcloud auth -application-default login``. - -Press ``Connect``: - -.. figure:: assets/gce1.png - -and execute the first command in your shell. Then start a proxy by -executing ``kubectl` proxy``. - -.. figure:: assets/gce2.png - -Now you should be able to run ``kubectl`` command to create resources, get -resource info, logs, etc. - -**Make sure you have Kubernetes >= 1.5, because you will be using -StatefulSets, which is a beta feature in 1.5.** - -Create a configuration file -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Download a template: - -:: - - curl -Lo app.yaml https://github.com/tendermint/tools/raw/master/mintnet-kubernetes/app.template.yaml - -Open ``app.yaml`` in your favorite editor and configure your app -container (navigate to ``- name: app``). Kubernetes DSL (Domain Specific -Language) is very simple, so it should be easy. You will need to set -Docker image, command and/or run arguments. Replace variables prefixed -with ``YOUR_APP`` with corresponding values. Set genesis time to now and -preferable chain ID in ConfigMap. - -Please note if you are changing ``replicas`` number, do not forget to -update ``validators`` set in ConfigMap. You will be able to scale the -cluster up or down later, but new pods (nodes) won't become validators -automatically. - -Deploy your application -^^^^^^^^^^^^^^^^^^^^^^^ - -:: - - kubectl create -f ./app.yaml - -Observe your cluster -^^^^^^^^^^^^^^^^^^^^ - -`web UI `__ - -The easiest way to access Dashboard is to use ``kubectl``. Run the following -command in your desktop environment: - -:: - - kubectl proxy - -``kubectl`` will handle authentication with apiserver and make Dashboard -available at http://localhost:8001/ui - -**shell** - -List all the pods: - -:: - - kubectl get pods -o wide -L tm - -StatefulSet details: - -:: - - kubectl describe statefulsets tm - -First pod details: - -:: - - kubectl describe pod tm-0 - -Tendermint app logs from the first pod: - -:: - - kubectl logs tm-0 -c tm -f - -App logs from the first pod: - -:: - - kubectl logs tm-0 -c app -f - -Status of the second pod's Tendermint app: - -:: - - kubectl exec -c tm tm-0 -- curl -s http://tm-1.:26657/status | json_pp - -Security --------- - -Due to the nature of Kubernetes, where you typically have a single -master, the master could be a SPOF (Single Point Of Failure). Therefore, -you need to make sure only authorized people can access it. And these -people themselves had taken basic measures in order not to get hacked. - -These are the best practices: - -- all access to the master is over TLS -- access to the API Server is X.509 certificate or token based -- etcd is not exposed directly to the cluster -- ensure that images are free of vulnerabilities - (`1 `__) -- ensure that only authorized images are used in your environment -- disable direct access to Kubernetes nodes (no SSH) -- define resource quota - -Resources: - -- https://kubernetes.io/docs/admin/accessing-the-api/ -- http://blog.kubernetes.io/2016/08/security-best-practices-kubernetes-deployment.html -- https://blog.openshift.com/securing-kubernetes/ - -Fault tolerance ---------------- - -Having a single master (API server) is a bad thing also because if -something happens to it, you risk being left without an access to the -application. - -To avoid that you can `run Kubernetes in multiple -zones `__, each zone -running an `API -server `__ and load -balance requests between them. Do not forget to make sure only one -instance of scheduler and controller-manager are running at once. - -Running in multiple zones is a lightweight version of a broader `Cluster -Federation feature `__. -Federated deployments could span across multiple regions (not zones). We -haven't tried this feature yet, so any feedback is highly appreciated! -Especially, related to additional latency and cost of exchanging data -between the regions. - -Resources: - -- https://kubernetes.io/docs/admin/high-availability/ - -Starting process ----------------- - -.. figure:: assets/statefulset.png - :alt: StatefulSet - - StatefulSet - -Init containers (``tm-gen-validator``) are run before all other -containers, creating public-private key pair for each pod. Every ``tm`` -container then asks other pods for their public keys, which are served -with nginx (``pub-key`` container). When ``tm`` container have all the -keys, it forms a genesis file and starts the Tendermint process. diff --git a/tools/mintnet-kubernetes/app.template.yaml b/tools/mintnet-kubernetes/app.template.yaml deleted file mode 100644 index ca7452921..000000000 --- a/tools/mintnet-kubernetes/app.template.yaml +++ /dev/null @@ -1,265 +0,0 @@ ---- -apiVersion: v1 -kind: Service -metadata: - annotations: - service.alpha.kubernetes.io/tolerate-unready-endpoints: "true" - name: YOUR_APP_NAME - labels: - app: YOUR_APP_NAME -spec: - ports: - - port: 26656 - name: p2p - - port: 26657 - name: rpc - clusterIP: None - selector: - app: tm ---- -apiVersion: v1 -kind: ConfigMap -metadata: - name: tm-config -data: - seeds: "tm-0,tm-1,tm-2,tm-3" - validators: "tm-0,tm-1,tm-2,tm-3" - validator.power: "10" - genesis.json: |- - { - "genesis_time": "2017-01-02T10:10:10.164Z", - "chain_id": "chain-B5XXm5", - "validators": [], - "app_hash": "" - } - pub_key_nginx.conf: |- - server { - listen 80 default_server; - listen [::]:80 default_server ipv6only=on; - location /pub_key.json { root /usr/share/nginx/; } - } ---- -apiVersion: policy/v1beta1 -kind: PodDisruptionBudget -metadata: - name: tm-budget -spec: - selector: - matchLabels: - app: tm - minAvailable: 2 ---- -apiVersion: apps/v1beta1 -kind: StatefulSet -metadata: - name: tm -spec: - serviceName: YOUR_APP_NAME - replicas: 4 - template: - metadata: - labels: - app: tm - version: v1 - annotations: - pod.beta.kubernetes.io/init-containers: '[{ - "name": "tm-gen-validator", - "image": "tendermint/tendermint:0.10.0", - "imagePullPolicy": "IfNotPresent", - "command": ["bash", "-c", " - set -ex\n - if [ ! -f /tendermint/priv_validator.json ]; then\n - tendermint gen_validator > /tendermint/priv_validator.json\n - # pub_key.json will be served by pub-key container\n - cat /tendermint/priv_validator.json | jq \".pub_key\" > /tendermint/pub_key.json\n - fi\n - "], - "volumeMounts": [ - {"name": "tmdir", "mountPath": "/tendermint"} - ] - }]' - spec: - containers: - - name: tm - imagePullPolicy: IfNotPresent - image: tendermint/tendermint:0.10.0 - resources: - requests: - cpu: 50m - memory: 128Mi - limits: - cpu: 100m - memory: 256Mi - ports: - - containerPort: 26656 - name: p2p - - containerPort: 26657 - name: rpc - env: - - name: SEEDS - valueFrom: - configMapKeyRef: - name: tm-config - key: seeds - - name: VALIDATOR_POWER - valueFrom: - configMapKeyRef: - name: tm-config - key: validator.power - - name: VALIDATORS - valueFrom: - configMapKeyRef: - name: tm-config - key: validators - - name: TMHOME - value: /tendermint - command: - - bash - - "-c" - - | - set -ex - - # copy template - cp /etc/tendermint/genesis.json /tendermint/genesis.json - - # fill genesis file with validators - IFS=',' read -ra VALS_ARR <<< "$VALIDATORS" - fqdn_suffix=$(hostname -f | sed 's#[^.]*\.\(\)#\1#') - for v in "${VALS_ARR[@]}"; do - # wait until validator generates priv/pub key pair - set +e - - curl -s --fail "http://$v.$fqdn_suffix/pub_key.json" > /dev/null - ERR=$? - while [ "$ERR" != 0 ]; do - sleep 5 - curl -s --fail "http://$v.$fqdn_suffix/pub_key.json" > /dev/null - ERR=$? - done - set -e - - # add validator to genesis file along with its pub_key - curl -s "http://$v.$fqdn_suffix/pub_key.json" | jq ". as \$k | {pub_key: \$k, amount: $VALIDATOR_POWER, name: \"$v\"}" > pub_validator.json - cat /tendermint/genesis.json | jq ".validators |= .+ [$(cat pub_validator.json)]" > tmpgenesis && mv tmpgenesis /tendermint/genesis.json - rm pub_validator.json - done - - # construct seeds - IFS=',' read -ra SEEDS_ARR <<< "$SEEDS" - seeds=() - for s in "${SEEDS_ARR[@]}"; do - seeds+=("$s.$fqdn_suffix:26656") - done - seeds=$(IFS=','; echo "${seeds[*]}") - - tendermint node --p2p.seeds="$seeds" --moniker="`hostname`" --proxy-app="unix:///socks/app.sock" - volumeMounts: - - name: tmdir - mountPath: /tendermint - - mountPath: /etc/tendermint/genesis.json - name: configdir - subPath: genesis.json - - name: socksdir - mountPath: /socks - - - name: app - imagePullPolicy: IfNotPresent - image: YOUR_APP_IMAGE - args: ["--addr=\"unix:///socks/app.sock\""] - volumeMounts: - - name: socksdir - mountPath: /socks - - ######## OR ######## - # - # - name: app - # imagePullPolicy: IfNotPresent - # image: golang:1.7.5 - # resources: - # requests: - # cpu: YOUR_APP_CPU_REQ - # memory: YOUR_APP_MEM_REQ - # limits: - # cpu: YOUR_APP_CPU_LIMIT - # memory: YOUR_APP_MEM_LIMIT - # command: - # - bash - # - "-c" - # - | - # set -ex - - # go get -d YOUR_APP_PACKAGE - # cd $GOPATH/YOUR_APP_PACKAGE - # make install - # - # rm -f /socks/app.sock # remove old socket - - # YOUR_APP_EXEC --addr="unix:///socks/app.sock" - # volumeMounts: - # - name: socksdir - # mountPath: /socks - - ######## OPTIONALLY ######## - # - # - name: data - # imagePullPolicy: IfNotPresent - # image: golang:1.7.5 - # command: - # - bash - # - "-c" - # - | - # set -ex - # go get github.com/tendermint/merkleeyes/cmd/merkleeyes - # rm -f /socks/data.sock # remove old socket - # merkleeyes server --address="unix:///socks/data.sock" - # volumeMounts: - # - name: socksdir - # mountPath: /socks - - - name: pub-key - imagePullPolicy: IfNotPresent - image: nginx:1.11.9 - resources: - requests: - cpu: 10m - memory: 12Mi - limits: - cpu: 20m - memory: 24Mi - ports: - - containerPort: 80 - name: pub-key - command: - - bash - - "-c" - - | - set -ex - # fixes 403 Permission Denied (open() "/tendermint/pub_key.json" failed (13: Permission denied)) - # => we cannot serve from /tendermint, so we copy the file - mkdir -p /usr/share/nginx - cp /tendermint/pub_key.json /usr/share/nginx/pub_key.json - nginx -g "daemon off;" - volumeMounts: - - name: tmdir - mountPath: /tendermint - - mountPath: /etc/nginx/conf.d/pub_key.conf - name: configdir - subPath: pub_key_nginx.conf - - volumes: - - name: configdir - configMap: - name: tm-config - - name: socksdir - emptyDir: {} - - volumeClaimTemplates: - - metadata: - name: tmdir - annotations: - volume.alpha.kubernetes.io/storage-class: anything - spec: - accessModes: ["ReadWriteOnce"] - resources: - requests: - storage: 2Gi diff --git a/tools/mintnet-kubernetes/assets/gce1.png b/tools/mintnet-kubernetes/assets/gce1.png deleted file mode 100644 index 3bf3ad005..000000000 Binary files a/tools/mintnet-kubernetes/assets/gce1.png and /dev/null differ diff --git a/tools/mintnet-kubernetes/assets/gce2.png b/tools/mintnet-kubernetes/assets/gce2.png deleted file mode 100644 index 358dcc04b..000000000 Binary files a/tools/mintnet-kubernetes/assets/gce2.png and /dev/null differ diff --git a/tools/mintnet-kubernetes/assets/statefulset.png b/tools/mintnet-kubernetes/assets/statefulset.png deleted file mode 100644 index ac68d22b7..000000000 Binary files a/tools/mintnet-kubernetes/assets/statefulset.png and /dev/null differ diff --git a/tools/mintnet-kubernetes/assets/t_plus_k.png b/tools/mintnet-kubernetes/assets/t_plus_k.png deleted file mode 100644 index bee9fe56e..000000000 Binary files a/tools/mintnet-kubernetes/assets/t_plus_k.png and /dev/null differ diff --git a/tools/mintnet-kubernetes/examples/counter/Makefile b/tools/mintnet-kubernetes/examples/counter/Makefile deleted file mode 100644 index 6d54d57d6..000000000 --- a/tools/mintnet-kubernetes/examples/counter/Makefile +++ /dev/null @@ -1,10 +0,0 @@ -create: - @echo "==> Creating deployment" - @kubectl create -f app.yaml - -destroy: - @echo "==> Destroying deployment" - @kubectl delete -f app.yaml - @kubectl delete pvc -l app=tm - -.PHONY: create destroy diff --git a/tools/mintnet-kubernetes/examples/counter/app.yaml b/tools/mintnet-kubernetes/examples/counter/app.yaml deleted file mode 100644 index 649a1e5b3..000000000 --- a/tools/mintnet-kubernetes/examples/counter/app.yaml +++ /dev/null @@ -1,214 +0,0 @@ ---- -apiVersion: v1 -kind: Service -metadata: - annotations: - service.alpha.kubernetes.io/tolerate-unready-endpoints: "true" - name: counter - labels: - app: counter -spec: - ports: - - port: 26656 - name: p2p - - port: 26657 - name: rpc - clusterIP: None - selector: - app: tm ---- -apiVersion: v1 -kind: ConfigMap -metadata: - name: tm-config -data: - seeds: "tm-0,tm-1,tm-2,tm-3" - validators: "tm-0,tm-1,tm-2,tm-3" - validator.power: "10" - genesis.json: |- - { - "genesis_time": "2016-02-05T23:17:31.164Z", - "chain_id": "chain-B5XXm5", - "validators": [], - "app_hash": "" - } - pub_key_nginx.conf: |- - server { - listen 80 default_server; - listen [::]:80 default_server ipv6only=on; - location /pub_key.json { root /usr/share/nginx/; } - } ---- -apiVersion: policy/v1beta1 -kind: PodDisruptionBudget -metadata: - name: tm-budget -spec: - selector: - matchLabels: - app: tm - minAvailable: 2 ---- -apiVersion: apps/v1beta1 -kind: StatefulSet -metadata: - name: tm -spec: - serviceName: counter - replicas: 4 - template: - metadata: - labels: - app: tm - annotations: - pod.beta.kubernetes.io/init-containers: '[{ - "name": "tm-gen-validator", - "image": "tendermint/tendermint:0.10.0", - "imagePullPolicy": "IfNotPresent", - "command": ["bash", "-c", " - set -ex\n - if [ ! -f /tendermint/priv_validator.json ]; then\n - tendermint gen_validator > /tendermint/priv_validator.json\n - # pub_key.json will be served by pub-key container\n - cat /tendermint/priv_validator.json | jq \".pub_key\" > /tendermint/pub_key.json\n - fi\n - "], - "volumeMounts": [ - {"name": "tmdir", "mountPath": "/tendermint"} - ] - }]' - spec: - containers: - - name: tm - imagePullPolicy: IfNotPresent - image: tendermint/tendermint:0.10.0 - ports: - - containerPort: 26656 - name: p2p - - containerPort: 26657 - name: rpc - env: - - name: SEEDS - valueFrom: - configMapKeyRef: - name: tm-config - key: seeds - - name: VALIDATOR_POWER - valueFrom: - configMapKeyRef: - name: tm-config - key: validator.power - - name: VALIDATORS - valueFrom: - configMapKeyRef: - name: tm-config - key: validators - - name: TMHOME - value: /tendermint - command: - - bash - - "-c" - - | - set -ex - - # copy template - cp /etc/tendermint/genesis.json /tendermint/genesis.json - - # fill genesis file with validators - IFS=',' read -ra VALS_ARR <<< "$VALIDATORS" - fqdn_suffix=$(hostname -f | sed 's#[^.]*\.\(\)#\1#') - for v in "${VALS_ARR[@]}"; do - # wait until validator generates priv/pub key pair - set +e - - curl -s --fail "http://$v.$fqdn_suffix/pub_key.json" > /dev/null - ERR=$? - while [ "$ERR" != 0 ]; do - sleep 5 - curl -s --fail "http://$v.$fqdn_suffix/pub_key.json" > /dev/null - ERR=$? - done - set -e - - # add validator to genesis file along with its pub_key - curl -s "http://$v.$fqdn_suffix/pub_key.json" | jq ". as \$k | {pub_key: \$k, amount: $VALIDATOR_POWER, name: \"$v\"}" > pub_validator.json - cat /tendermint/genesis.json | jq ".validators |= .+ [$(cat pub_validator.json)]" > tmpgenesis && mv tmpgenesis /tendermint/genesis.json - rm pub_validator.json - done - - # construct seeds - IFS=',' read -ra SEEDS_ARR <<< "$SEEDS" - seeds=() - for s in "${SEEDS_ARR[@]}"; do - seeds+=("$s.$fqdn_suffix:26656") - done - seeds=$(IFS=','; echo "${seeds[*]}") - - tendermint node --p2p.seeds="$seeds" --moniker="`hostname`" --proxy-app="unix:///socks/app.sock" - volumeMounts: - - name: tmdir - mountPath: /tendermint - - mountPath: /etc/tendermint/genesis.json - name: tmconfigdir - subPath: genesis.json - - name: socksdir - mountPath: /socks - - - name: app - imagePullPolicy: IfNotPresent - image: golang:latest - command: - - bash - - "-c" - - | - set -ex - - go get github.com/tendermint/tendermint/abci/cmd/abci-cli - - rm -f /socks/app.sock # remove old socket - - abci-cli counter --serial=true --address="unix:///socks/app.sock" - volumeMounts: - - name: socksdir - mountPath: /socks - - - name: pub-key - imagePullPolicy: IfNotPresent - image: nginx:latest - ports: - - containerPort: 80 - name: pub-key - command: - - bash - - "-c" - - | - set -ex - # fixes 403 Permission Denied (open() "/tendermint/pub_key.json" failed (13: Permission denied)) - # => we cannot serve from /tendermint, so we copy the file - mkdir -p /usr/share/nginx - cp /tendermint/pub_key.json /usr/share/nginx/pub_key.json - nginx -g "daemon off;" - volumeMounts: - - name: tmdir - mountPath: /tendermint - - mountPath: /etc/nginx/conf.d/pub_key.conf - name: tmconfigdir - subPath: pub_key_nginx.conf - - volumes: - - name: tmconfigdir - configMap: - name: tm-config - - name: socksdir - emptyDir: {} - - volumeClaimTemplates: - - metadata: - name: tmdir - annotations: - volume.alpha.kubernetes.io/storage-class: anything - spec: - accessModes: ["ReadWriteOnce"] - resources: - requests: - storage: 2Gi diff --git a/tools/mintnet-kubernetes/examples/dummy/Makefile b/tools/mintnet-kubernetes/examples/dummy/Makefile deleted file mode 100644 index 825487fcd..000000000 --- a/tools/mintnet-kubernetes/examples/dummy/Makefile +++ /dev/null @@ -1,17 +0,0 @@ -create: - @echo "==> Creating deployment" - @kubectl create -f app.yaml - @echo "==> Waiting 10s until it is probably ready" - @sleep 10 - @echo "==> Creating monitor and transacter pods" - @kubectl create -f tm-monitor-pod.yaml - @kubectl create -f transacter-pod.yaml - -destroy: - @echo "==> Destroying deployment" - @kubectl delete -f transacter-pod.yaml - @kubectl delete -f tm-monitor-pod.yaml - @kubectl delete -f app.yaml - @kubectl delete pvc -l app=tm - -.PHONY: create destroy diff --git a/tools/mintnet-kubernetes/examples/dummy/app.yaml b/tools/mintnet-kubernetes/examples/dummy/app.yaml deleted file mode 100644 index 266bd0083..000000000 --- a/tools/mintnet-kubernetes/examples/dummy/app.yaml +++ /dev/null @@ -1,196 +0,0 @@ ---- -apiVersion: v1 -kind: Service -metadata: - annotations: - service.alpha.kubernetes.io/tolerate-unready-endpoints: "true" - name: dummy - labels: - app: dummy -spec: - ports: - - port: 26656 - name: p2p - - port: 26657 - name: rpc - clusterIP: None - selector: - app: tm ---- -apiVersion: v1 -kind: ConfigMap -metadata: - name: tm-config -data: - seeds: "tm-0,tm-1,tm-2,tm-3" - validators: "tm-0,tm-1,tm-2,tm-3" - validator.power: "10" - genesis.json: |- - { - "genesis_time": "2016-02-05T23:17:31.164Z", - "chain_id": "chain-B5XXm5", - "validators": [], - "app_hash": "" - } - pub_key_nginx.conf: |- - server { - listen 80 default_server; - listen [::]:80 default_server ipv6only=on; - location /pub_key.json { root /usr/share/nginx/; } - } ---- -apiVersion: policy/v1beta1 -kind: PodDisruptionBudget -metadata: - name: tm-budget -spec: - selector: - matchLabels: - app: tm - minAvailable: 2 ---- -apiVersion: apps/v1beta1 -kind: StatefulSet -metadata: - name: tm -spec: - serviceName: dummy - replicas: 4 - template: - metadata: - labels: - app: tm - annotations: - pod.beta.kubernetes.io/init-containers: '[{ - "name": "tm-gen-validator", - "image": "tendermint/tendermint:0.10.0", - "imagePullPolicy": "IfNotPresent", - "command": ["bash", "-c", " - set -ex\n - if [ ! -f /tendermint/priv_validator.json ]; then\n - tendermint gen_validator > /tendermint/priv_validator.json\n - # pub_key.json will be served by pub-key container\n - cat /tendermint/priv_validator.json | jq \".pub_key\" > /tendermint/pub_key.json\n - fi\n - "], - "volumeMounts": [ - {"name": "tmdir", "mountPath": "/tendermint"} - ] - }]' - spec: - containers: - - name: tm - imagePullPolicy: IfNotPresent - image: tendermint/tendermint:0.10.0 - ports: - - containerPort: 26656 - name: p2p - - containerPort: 26657 - name: rpc - env: - - name: SEEDS - valueFrom: - configMapKeyRef: - name: tm-config - key: seeds - - name: VALIDATOR_POWER - valueFrom: - configMapKeyRef: - name: tm-config - key: validator.power - - name: VALIDATORS - valueFrom: - configMapKeyRef: - name: tm-config - key: validators - - name: TMHOME - value: /tendermint - command: - - bash - - "-c" - - | - set -ex - - # copy template - cp /etc/tendermint/genesis.json /tendermint/genesis.json - - # fill genesis file with validators - IFS=',' read -ra VALS_ARR <<< "$VALIDATORS" - fqdn_suffix=$(hostname -f | sed 's#[^.]*\.\(\)#\1#') - for v in "${VALS_ARR[@]}"; do - # wait until validator generates priv/pub key pair - set +e - - curl -s --fail "http://$v.$fqdn_suffix/pub_key.json" > /dev/null - ERR=$? - while [ "$ERR" != 0 ]; do - sleep 5 - curl -s --fail "http://$v.$fqdn_suffix/pub_key.json" > /dev/null - ERR=$? - done - set -e - - # add validator to genesis file along with its pub_key - curl -s "http://$v.$fqdn_suffix/pub_key.json" | jq ". as \$k | {pub_key: \$k, amount: $VALIDATOR_POWER, name: \"$v\"}" > pub_validator.json - cat /tendermint/genesis.json | jq ".validators |= .+ [$(cat pub_validator.json)]" > tmpgenesis && mv tmpgenesis /tendermint/genesis.json - rm pub_validator.json - done - - # construct seeds - IFS=',' read -ra SEEDS_ARR <<< "$SEEDS" - seeds=() - for s in "${SEEDS_ARR[@]}"; do - seeds+=("$s.$fqdn_suffix:26656") - done - seeds=$(IFS=','; echo "${seeds[*]}") - - tendermint node --p2p.seeds="$seeds" --moniker="`hostname`" --proxy-app="dummy" - volumeMounts: - - name: tmdir - mountPath: /tendermint - - mountPath: /etc/tendermint/genesis.json - name: tmconfigdir - subPath: genesis.json - - name: socksdir - mountPath: /socks - - - name: pub-key - imagePullPolicy: IfNotPresent - image: nginx:latest - ports: - - containerPort: 80 - name: pub-key - command: - - bash - - "-c" - - | - set -ex - # fixes 403 Permission Denied (open() "/tendermint/pub_key.json" failed (13: Permission denied)) - # => we cannot serve from /tendermint, so we copy the file - mkdir -p /usr/share/nginx - cp /tendermint/pub_key.json /usr/share/nginx/pub_key.json - nginx -g "daemon off;" - volumeMounts: - - name: tmdir - mountPath: /tendermint - - mountPath: /etc/nginx/conf.d/pub_key.conf - name: tmconfigdir - subPath: pub_key_nginx.conf - - volumes: - - name: tmconfigdir - configMap: - name: tm-config - - name: socksdir - emptyDir: {} - - volumeClaimTemplates: - - metadata: - name: tmdir - annotations: - volume.alpha.kubernetes.io/storage-class: anything - spec: - accessModes: ["ReadWriteOnce"] - resources: - requests: - storage: 2Gi diff --git a/tools/mintnet-kubernetes/examples/dummy/tm-monitor-pod.yaml b/tools/mintnet-kubernetes/examples/dummy/tm-monitor-pod.yaml deleted file mode 100644 index fb0bf7236..000000000 --- a/tools/mintnet-kubernetes/examples/dummy/tm-monitor-pod.yaml +++ /dev/null @@ -1,13 +0,0 @@ ---- -apiVersion: v1 -kind: Pod -metadata: - name: monitor -spec: - containers: - - name: monitor - image: tendermint/monitor - args: ["-listen-addr=tcp://0.0.0.0:26670", "tm-0.dummy:26657,tm-1.dummy:26657,tm-2.dummy:26657,tm-3.dummy:26657"] - ports: - - containerPort: 26670 - name: rpc diff --git a/tools/mintnet-kubernetes/examples/dummy/transacter-pod.yaml b/tools/mintnet-kubernetes/examples/dummy/transacter-pod.yaml deleted file mode 100644 index 6598e2a8b..000000000 --- a/tools/mintnet-kubernetes/examples/dummy/transacter-pod.yaml +++ /dev/null @@ -1,19 +0,0 @@ ---- -apiVersion: v1 -kind: Pod -metadata: - name: transacter -spec: - containers: - - name: transacter - image: tendermint/transacter - command: - - bash - - "-c" - - | - set -ex - while true - do - ./transact 100 "tm-0.dummy:26657" - sleep 1 - done