generate RPC docs using Slate (#1612)

* generate RPC docs using Slate (#691)

* update changelog

* skip if branch not develop

* slate: only build if rpc/core has changes

* fetch develop to compare against

* slate: build on master only

* [rpc/core] use original repo, not fork in README

.circleci/config.yml

@@ -77,6 +77,22 @@ jobs:
           paths:
             - "bin/abci*"
 
+  build_slate:
+    <<: *defaults
+    steps:
+      - attach_workspace:
+          at: /tmp/workspace
+      - restore_cache:
+          key: v1-pkg-cache
+      - restore_cache:
+          key: v1-tree-{{ .Environment.CIRCLE_SHA1 }}
+      - run:
+          name: slate docs
+          command: |
+            set -ex
+            export PATH="$GOBIN:$PATH"
+            make build-slate
+
   lint:
     <<: *defaults
     steps:
@@ -180,6 +196,9 @@ workflows:
   test-suite:
     jobs:
       - setup_dependencies
+      - build_slate:
+          requires:
+            - setup_dependencies
       - setup_abci:
           requires:
             - setup_dependencies

CHANGELOG.md

@@ -2,7 +2,9 @@
 
 ## 0.19.6
 
-*TBD*
+FEATURES
+
+- [rpc] the RPC documentation is now published to https://tendermint.github.io/slate
 
 IMPROVEMENTS:
 

Makefile

@@ -226,8 +226,11 @@ sentry-stop:
 	@if [ -z "$(DO_API_TOKEN)" ]; then echo "DO_API_TOKEN environment variable not set." ; false ; fi
 	cd networks/remote/terraform && terraform destroy -var DO_API_TOKEN="$(DO_API_TOKEN)" -var SSH_KEY_FILE="$(HOME)/.ssh/id_rsa.pub"
 
+# meant for the CI, inspect script & adapt accordingly
+build-slate:
+	bash scripts/slate.sh
+
 # To avoid unintended conflicts with file names, always add to .PHONY
 # unless there is a reason not to.
 # https://www.gnu.org/software/make/manual/html_node/Phony-Targets.html
-.PHONY: check build build_race dist install check_tools get_tools update_tools get_vendor_deps draw_deps test_cover test_apps test_persistence test_p2p test test_race test_integrations test_release test100 vagrant_test fmt build-linux localnet-start localnet-stop build-docker build-docker-localnode sentry-start sentry-config sentry-stop
-
+.PHONY: check build build_race dist install check_tools get_tools update_tools get_vendor_deps draw_deps test_cover test_apps test_persistence test_p2p test test_race test_integrations test_release test100 vagrant_test fmt build-linux localnet-start localnet-stop build-docker build-docker-localnode sentry-start sentry-config sentry-stop build-slate

docs/specification/rpc.rst

@@ -1,190 +1,4 @@
 RPC
 ===
 
-Coming soon: RPC docs powered by `slate <https://github.com/lord/slate>`__. Until then, read on.
-
-Tendermint supports the following RPC protocols:
-
--  URI over HTTP
--  JSONRPC over HTTP
--  JSONRPC over websockets
-
-Tendermint RPC is build using `our own RPC
-library <https://github.com/tendermint/tendermint/tree/master/rpc/lib>`__.
-Documentation and tests for that library could be found at
-``tendermint/rpc/lib`` directory.
-
-Configuration
-~~~~~~~~~~~~~
-
-Set the ``laddr`` config parameter under ``[rpc]`` table in the
-$TMHOME/config/config.toml file or the ``--rpc.laddr`` command-line flag to the
-desired protocol://host:port setting. Default: ``tcp://0.0.0.0:46657``.
-
-Arguments
-~~~~~~~~~
-
-Arguments which expect strings or byte arrays may be passed as quoted
-strings, like ``"abc"`` or as ``0x``-prefixed strings, like
-``0x616263``.
-
-URI/HTTP
-~~~~~~~~
-
-Example request:
-
-.. code:: bash
-
-    curl -s 'http://localhost:46657/broadcast_tx_sync?tx="abc"' | jq .
-
-Response:
-
-.. code:: json
-
-    {
-      "error": "",
-      "result": {
-        "hash": "2B8EC32BA2579B3B8606E42C06DE2F7AFA2556EF",
-        "log": "",
-        "data": "",
-        "code": 0
-      },
-      "id": "",
-      "jsonrpc": "2.0"
-    }
-
-The first entry in the result-array (``96``) is the method this response
-correlates with. ``96`` refers to "ResultTypeBroadcastTx", see
-`responses.go <https://github.com/tendermint/tendermint/blob/master/rpc/core/types/responses.go>`__
-for a complete overview.
-
-JSONRPC/HTTP
-~~~~~~~~~~~~
-
-JSONRPC requests can be POST'd to the root RPC endpoint via HTTP (e.g.
-``http://localhost:46657/``).
-
-Example request:
-
-.. code:: json
-
-    {
-      "method": "broadcast_tx_sync",
-      "jsonrpc": "2.0",
-      "params": [ "abc" ],
-      "id": "dontcare"
-    }
-
-JSONRPC/websockets
-~~~~~~~~~~~~~~~~~~
-
-JSONRPC requests can be made via websocket. The websocket endpoint is at
-``/websocket``, e.g. ``http://localhost:46657/websocket``. Asynchronous
-RPC functions like event ``subscribe`` and ``unsubscribe`` are only
-available via websockets.
-
-Endpoints
-~~~~~~~~~
-
-An HTTP Get request to the root RPC endpoint (e.g.
-``http://localhost:46657``) shows a list of available endpoints.
-
-::
-
-    Available endpoints:
-    http://localhost:46657/abci_info
-    http://localhost:46657/dump_consensus_state
-    http://localhost:46657/genesis
-    http://localhost:46657/net_info
-    http://localhost:46657/num_unconfirmed_txs
-    http://localhost:46657/health
-    http://localhost:46657/status
-    http://localhost:46657/unconfirmed_txs
-    http://localhost:46657/unsafe_flush_mempool
-    http://localhost:46657/unsafe_stop_cpu_profiler
-    http://localhost:46657/validators
-
-    Endpoints that require arguments:
-    http://localhost:46657/abci_query?path=_&data=_&prove=_
-    http://localhost:46657/block?height=_
-    http://localhost:46657/blockchain?minHeight=_&maxHeight=_
-    http://localhost:46657/broadcast_tx_async?tx=_
-    http://localhost:46657/broadcast_tx_commit?tx=_
-    http://localhost:46657/broadcast_tx_sync?tx=_
-    http://localhost:46657/commit?height=_
-    http://localhost:46657/dial_seeds?seeds=_
-    http://localhost:46657/dial_peers?peers=_&persistent=_
-    http://localhost:46657/subscribe?event=_
-    http://localhost:46657/tx?hash=_&prove=_
-    http://localhost:46657/unsafe_start_cpu_profiler?filename=_
-    http://localhost:46657/unsafe_write_heap_profile?filename=_
-    http://localhost:46657/unsubscribe?event=_
-
-tx
-~~
-
-Returns a transaction matching the given transaction hash.
-
-**Parameters**
-
-1. hash - the transaction hash
-2. prove - include a proof of the transaction inclusion in the block in
-   the result (optional, default: false)
-
-**Returns**
-
--  ``proof``: the ``types.TxProof`` object
--  ``tx``: ``[]byte`` - the transaction
--  ``tx_result``: the ``abci.Result`` object
--  ``index``: ``int`` - index of the transaction
--  ``height``: ``int`` - height of the block where this transaction was
-   in
-
-**Example**
-
-.. code:: bash
-
-    curl -s 'http://localhost:46657/broadcast_tx_commit?tx="abc"' | jq .
-    # {
-    #   "error": "",
-    #   "result": {
-    #     "hash": "2B8EC32BA2579B3B8606E42C06DE2F7AFA2556EF",
-    #     "log": "",
-    #     "data": "",
-    #     "code": 0
-    #   },
-    #   "id": "",
-    #   "jsonrpc": "2.0"
-    # }
-
-    curl -s 'http://localhost:46657/tx?hash=0x2B8EC32BA2579B3B8606E42C06DE2F7AFA2556EF' | jq .
-    # {
-    #   "error": "",
-    #   "result": {
-    #     "proof": {
-    #       "Proof": {
-    #         "aunts": []
-    #       },
-    #       "Data": "YWJjZA==",
-    #       "RootHash": "2B8EC32BA2579B3B8606E42C06DE2F7AFA2556EF",
-    #       "Total": 1,
-    #       "Index": 0
-    #     },
-    #     "tx": "YWJjZA==",
-    #     "tx_result": {
-    #       "log": "",
-    #       "data": "",
-    #       "code": 0
-    #     },
-    #     "index": 0,
-    #     "height": 52
-    #   },
-    #   "id": "",
-    #   "jsonrpc": "2.0"
-    # }
-
-More Examples
-~~~~~~~~~~~~~
-
-See the various bash tests using curl in ``test/``, and examples using
-the ``Go`` API in ``rpc/client/``.
+The RPC documentation is hosted `here <https://tendermint.github.io/slate>`__ and is generated by the CI from our `Slate repo <https://github.com/tendermint/slate>`__. To update the documentation, edit the relevant ``godoc`` comments in the `rpc/core directory <https://github.com/tendermint/tendermint/tree/develop/rpc/core>`__.

rpc/core/README.md

@@ -3,17 +3,16 @@
 ## Generate markdown for [Slate](https://github.com/tendermint/slate)
 
 We are using [Slate](https://github.com/tendermint/slate) to power our RPC
-documentation. If you are changing a comment, make sure to copy the resulting
-changes to the slate repo and make a PR
-[there](https://github.com/tendermint/slate) as well. For generating markdown
-use:
+documentation. For generating markdown use:
 
 ```shell
-go get github.com/melekes/godoc2md
+go get github.com/davecheney/godoc2md
 
 godoc2md -template rpc/core/doc_template.txt github.com/tendermint/tendermint/rpc/core | grep -v -e "pipe.go" -e "routes.go" -e "dev.go" | sed 's$/src/target$https://github.com/tendermint/tendermint/tree/master/rpc/core$'
 ```
 
+For more information see the [CI script for building the Slate docs](/scripts/slate.sh)
+
 ## Pagination
 
 Requests that return multiple items will be paginated to 30 items by default.

rpc/core/doc.go

@@ -7,7 +7,7 @@ Tendermint supports the following RPC protocols:
 * JSONRPC over HTTP
 * JSONRPC over websockets
 
-Tendermint RPC is built using [our own RPC library](https://github.com/tendermint/tendermint/tree/master/rpc/lib). Documentation and tests for that library could be found at `tendermint/rpc/lib` directory.
+Tendermint RPC is built using [our own RPC library](https://github.com/tendermint/tendermint/tree/master/rpc/lib) which contains its own set of documentation and tests.
 
 ## Configuration
 

scripts/slate.sh

@@ -0,0 +1,77 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+if [ "$CIRCLE_BRANCH" == "" ]; then
+	echo "this script is meant to be run on CircleCI, exiting"
+	echo 1
+fi
+
+# check for changes in the `rpc/core` directory
+did_rpc_change=$(git diff --name-status $CIRCLE_BRANCH origin/master | grep rpc/core)
+
+if [ "$did_rpc_change" == "" ]; then
+	echo "no changes detected in rpc/core, exiting"
+	exit 0
+else
+	echo "changes detected in rpc/core, continuing"
+fi
+
+# only run this script on changes to rpc/core committed to develop
+if [ "$CIRCLE_BRANCH" != "master" ]; then
+	echo "the branch being built isn't master, exiting"
+	exit 0
+else
+	echo "on master, building the RPC docs"
+fi
+
+# godoc2md used to convert the go documentation from
+# `rpc/core` into a markdown file consumed by Slate
+go get github.com/davecheney/godoc2md
+
+# slate works via forks, and we'll be committing to
+# master branch, which will trigger our fork to run
+# the `./deploy.sh` and publish via the `gh-pages` branch
+slate_repo=github.com/tendermint/slate
+slate_path="$GOPATH"/src/"$slate_repo"
+
+if [ ! -d "$slate_path" ]; then
+	git clone https://"$slate_repo".git $slate_path
+fi
+
+# the main file we need to update if rpc/core changed
+destination="$slate_path"/source/index.html.md
+
+# we remove it then re-create it with the latest changes
+rm $destination
+
+header="---
+title: RPC Reference
+
+language_tabs:
+  - shell
+  - go
+
+toc_footers:
+  - <a href='https://tendermint.com/'>Tendermint</a>
+  - <a href='https://github.com/lord/slate'>Documentation Powered by Slate</a>
+
+search: true
+---"
+
+# write header to the main slate file
+echo "$header" > "$destination"
+
+# generate a markdown from the godoc comments, using a template
+rpc_docs=$(godoc2md -template rpc/core/doc_template.txt github.com/tendermint/tendermint/rpc/core | grep -v -e "pipe.go" -e "routes.go" -e "dev.go" | sed 's$/src/target$https://github.com/tendermint/tendermint/tree/master/rpc/core$')
+
+# append core RPC docs
+echo "$rpc_docs" >> "$destination"
+
+# commit the changes
+cd $slate_path
+
+git config --global user.email "github@tendermint.com"
+git config --global user.name "tenderbot"
+
+git commit -a -m "Update tendermint RPC docs via CircleCI"
+git push -q https://${GITHUB_ACCESS_TOKEN}@github.com/tendermint/slate.git master