You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

330 lines
11 KiB

6 years ago
9 years ago
[RPC] Static swagger (#3880) * manually swagging Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * three definitions with polymorphism Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * added blockchain and block Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * low quality generation, commit, block_response and validators Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * genesis and consensus states endpoints Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * fix indentation Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * consensus parameters Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * fix indentation Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * add height to consensus parameters endpoint Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * unconfirmed_txs and num_unconfirmed_txs Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * add missing query parameter Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * add ABCI queries Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * added index document for swagger documentation Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * add missing routes Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * contract tests added on CCI Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * contract tests job should be in the test suite Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * simplify requirements to test Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * typo Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * build is a prerequisite to start localnet Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * reduce nodejs size, move goodman to get_tools, add docs, fix comments Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * Update scripts/get_tools.sh That's cleaner, thanks! Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * xz not supported by cci image, let's keep it simple Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * REMOVE-indirect debug of CCI paths Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * dirty experiment, volume is empty but binary has been produced Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * dirty experiment, volume is empty but binary has been produced Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * dirty experiment going on Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * locally works, CCI have difficulties with second layaer containers volumes Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * restore experiment, use machine instead of docker for contract tests Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * simplify a bit Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * rollback on machine golang Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * Document the changes Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * Changelog Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * comments Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com>
5 years ago
[RPC] Static swagger (#3880) * manually swagging Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * three definitions with polymorphism Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * added blockchain and block Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * low quality generation, commit, block_response and validators Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * genesis and consensus states endpoints Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * fix indentation Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * consensus parameters Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * fix indentation Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * add height to consensus parameters endpoint Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * unconfirmed_txs and num_unconfirmed_txs Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * add missing query parameter Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * add ABCI queries Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * added index document for swagger documentation Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * add missing routes Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * contract tests added on CCI Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * contract tests job should be in the test suite Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * simplify requirements to test Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * typo Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * build is a prerequisite to start localnet Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * reduce nodejs size, move goodman to get_tools, add docs, fix comments Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * Update scripts/get_tools.sh That's cleaner, thanks! Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * xz not supported by cci image, let's keep it simple Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * REMOVE-indirect debug of CCI paths Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * dirty experiment, volume is empty but binary has been produced Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * dirty experiment, volume is empty but binary has been produced Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * dirty experiment going on Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * locally works, CCI have difficulties with second layaer containers volumes Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * restore experiment, use machine instead of docker for contract tests Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * simplify a bit Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * rollback on machine golang Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * Document the changes Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * Changelog Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com> * comments Signed-off-by: Karoly Albert Szabo <szabo.karoly.a@gmail.com>
5 years ago
  1. GOTOOLS = \
  2. github.com/mitchellh/gox \
  3. github.com/golangci/golangci-lint/cmd/golangci-lint \
  4. github.com/gogo/protobuf/protoc-gen-gogo \
  5. github.com/square/certstrap
  6. GOBIN?=${GOPATH}/bin
  7. PACKAGES=$(shell go list ./...)
  8. OUTPUT?=build/tendermint
  9. export GO111MODULE = on
  10. INCLUDE = -I=. -I=${GOPATH}/src -I=${GOPATH}/src/github.com/gogo/protobuf/protobuf
  11. BUILD_TAGS?='tendermint'
  12. LD_FLAGS = -X github.com/tendermint/tendermint/version.GitCommit=`git rev-parse --short=8 HEAD` -s -w
  13. BUILD_FLAGS = -mod=readonly -ldflags "$(LD_FLAGS)"
  14. all: check build test install
  15. check: check_tools
  16. ########################################
  17. ### Build Tendermint
  18. build:
  19. CGO_ENABLED=0 go build $(BUILD_FLAGS) -tags $(BUILD_TAGS) -o $(OUTPUT) ./cmd/tendermint/
  20. build_c:
  21. CGO_ENABLED=1 go build $(BUILD_FLAGS) -tags "$(BUILD_TAGS) cleveldb" -o $(OUTPUT) ./cmd/tendermint/
  22. build_race:
  23. CGO_ENABLED=1 go build -race $(BUILD_FLAGS) -tags $(BUILD_TAGS) -o $(OUTPUT) ./cmd/tendermint
  24. install:
  25. CGO_ENABLED=0 go install $(BUILD_FLAGS) -tags $(BUILD_TAGS) ./cmd/tendermint
  26. install_c:
  27. CGO_ENABLED=1 go install $(BUILD_FLAGS) -tags "$(BUILD_TAGS) cleveldb" ./cmd/tendermint
  28. ########################################
  29. ### Protobuf
  30. protoc_all: protoc_libs protoc_merkle protoc_abci protoc_grpc protoc_proto3types
  31. %.pb.go: %.proto
  32. ## If you get the following error,
  33. ## "error while loading shared libraries: libprotobuf.so.14: cannot open shared object file: No such file or directory"
  34. ## See https://stackoverflow.com/a/25518702
  35. ## Note the $< here is substituted for the %.proto
  36. ## Note the $@ here is substituted for the %.pb.go
  37. protoc $(INCLUDE) $< --gogo_out=Mgoogle/protobuf/timestamp.proto=github.com/golang/protobuf/ptypes/timestamp,plugins=grpc:.
  38. ########################################
  39. ### Build ABCI
  40. # see protobuf section above
  41. protoc_abci: abci/types/types.pb.go
  42. protoc_proto3types: types/proto3/block.pb.go
  43. build_abci:
  44. @go build -mod=readonly -i ./abci/cmd/...
  45. install_abci:
  46. @go install -mod=readonly ./abci/cmd/...
  47. ########################################
  48. ### Distribution
  49. # dist builds binaries for all platforms and packages them for distribution
  50. # TODO add abci to these scripts
  51. dist:
  52. @BUILD_TAGS=$(BUILD_TAGS) sh -c "'$(CURDIR)/scripts/dist.sh'"
  53. ########################################
  54. ### Tools & dependencies
  55. check_tools:
  56. @# https://stackoverflow.com/a/25668869
  57. @echo "Found tools: $(foreach tool,$(notdir $(GOTOOLS)),\
  58. $(if $(shell which $(tool)),$(tool),$(error "No $(tool) in PATH")))"
  59. get_tools:
  60. @echo "--> Installing tools"
  61. ./scripts/get_tools.sh
  62. update_tools:
  63. @echo "--> Updating tools"
  64. ./scripts/get_tools.sh
  65. #For ABCI and libs
  66. get_protoc:
  67. @# https://github.com/google/protobuf/releases
  68. curl -L https://github.com/google/protobuf/releases/download/v3.6.1/protobuf-cpp-3.6.1.tar.gz | tar xvz && \
  69. cd protobuf-3.6.1 && \
  70. DIST_LANG=cpp ./configure && \
  71. make && \
  72. make check && \
  73. sudo make install && \
  74. sudo ldconfig && \
  75. cd .. && \
  76. rm -rf protobuf-3.6.1
  77. draw_deps:
  78. @# requires brew install graphviz or apt-get install graphviz
  79. go get github.com/RobotsAndPencils/goviz
  80. @goviz -i github.com/tendermint/tendermint/cmd/tendermint -d 3 | dot -Tpng -o dependency-graph.png
  81. get_deps_bin_size:
  82. @# Copy of build recipe with additional flags to perform binary size analysis
  83. $(eval $(shell go build -work -a $(BUILD_FLAGS) -tags $(BUILD_TAGS) -o $(OUTPUT) ./cmd/tendermint/ 2>&1))
  84. @find $(WORK) -type f -name "*.a" | xargs -I{} du -hxs "{}" | sort -rh | sed -e s:${WORK}/::g > deps_bin_size.log
  85. @echo "Results can be found here: $(CURDIR)/deps_bin_size.log"
  86. ########################################
  87. ### Libs
  88. protoc_libs: libs/common/types.pb.go
  89. # generates certificates for TLS testing in remotedb and RPC server
  90. gen_certs: clean_certs
  91. certstrap init --common-name "tendermint.com" --passphrase ""
  92. certstrap request-cert --common-name "remotedb" -ip "127.0.0.1" --passphrase ""
  93. certstrap sign "remotedb" --CA "tendermint.com" --passphrase ""
  94. mv out/remotedb.crt libs/db/remotedb/test.crt
  95. mv out/remotedb.key libs/db/remotedb/test.key
  96. certstrap request-cert --common-name "server" -ip "127.0.0.1" --passphrase ""
  97. certstrap sign "server" --CA "tendermint.com" --passphrase ""
  98. mv out/server.crt rpc/lib/server/test.crt
  99. mv out/server.key rpc/lib/server/test.key
  100. rm -rf out
  101. # deletes generated certificates
  102. clean_certs:
  103. rm -f libs/db/remotedb/test.crt
  104. rm -f libs/db/remotedb/test.key
  105. rm -f rpc/lib/server/test.crt
  106. rm -f rpc/lib/server/test.key
  107. test_libs:
  108. go test -tags clevedb boltdb $(PACKAGES)
  109. grpc_dbserver:
  110. protoc -I libs/db/remotedb/proto/ libs/db/remotedb/proto/defs.proto --go_out=plugins=grpc:libs/db/remotedb/proto
  111. protoc_grpc: rpc/grpc/types.pb.go
  112. protoc_merkle: crypto/merkle/merkle.pb.go
  113. ########################################
  114. ### Testing
  115. ## required to be run first by most tests
  116. build_docker_test_image:
  117. docker build -t tester -f ./test/docker/Dockerfile .
  118. ### coverage, app, persistence, and libs tests
  119. test_cover:
  120. # run the go unit tests with coverage
  121. bash test/test_cover.sh
  122. test_apps:
  123. # run the app tests using bash
  124. # requires `abci-cli` and `tendermint` binaries installed
  125. bash test/app/test.sh
  126. test_abci_apps:
  127. bash abci/tests/test_app/test.sh
  128. test_abci_cli:
  129. # test the cli against the examples in the tutorial at:
  130. # ./docs/abci-cli.md
  131. # if test fails, update the docs ^
  132. @ bash abci/tests/test_cli/test.sh
  133. test_persistence:
  134. # run the persistence tests using bash
  135. # requires `abci-cli` installed
  136. docker run --name run_persistence -t tester bash test/persist/test_failure_indices.sh
  137. # TODO undockerize
  138. # bash test/persist/test_failure_indices.sh
  139. test_p2p:
  140. docker rm -f rsyslog || true
  141. rm -rf test/logs || true
  142. mkdir test/logs
  143. cd test/
  144. docker run -d -v "logs:/var/log/" -p 127.0.0.1:5514:514/udp --name rsyslog voxxit/rsyslog
  145. cd ..
  146. # requires 'tester' the image from above
  147. bash test/p2p/test.sh tester
  148. # the `docker cp` takes a really long time; uncomment for debugging
  149. #
  150. # mkdir -p test/p2p/logs && docker cp rsyslog:/var/log test/p2p/logs
  151. test_integrations:
  152. make build_docker_test_image
  153. make get_tools
  154. make install
  155. make test_cover
  156. make test_apps
  157. make test_abci_apps
  158. make test_abci_cli
  159. make test_libs
  160. make test_persistence
  161. make test_p2p
  162. test_release:
  163. @go test -tags release $(PACKAGES)
  164. test100:
  165. @for i in {1..100}; do make test; done
  166. vagrant_test:
  167. vagrant up
  168. vagrant ssh -c 'make test_integrations'
  169. ### go tests
  170. test:
  171. @echo "--> Running go test"
  172. @go test -p 1 $(PACKAGES)
  173. test_race:
  174. @echo "--> Running go test --race"
  175. @go test -p 1 -v -race $(PACKAGES)
  176. # uses https://github.com/sasha-s/go-deadlock/ to detect potential deadlocks
  177. test_with_deadlock:
  178. make set_with_deadlock
  179. make test
  180. make cleanup_after_test_with_deadlock
  181. set_with_deadlock:
  182. find . -name "*.go" | grep -v "vendor/" | xargs -n 1 sed -i.bak 's/sync.RWMutex/deadlock.RWMutex/'
  183. find . -name "*.go" | grep -v "vendor/" | xargs -n 1 sed -i.bak 's/sync.Mutex/deadlock.Mutex/'
  184. find . -name "*.go" | grep -v "vendor/" | xargs -n 1 goimports -w
  185. # cleanes up after you ran test_with_deadlock
  186. cleanup_after_test_with_deadlock:
  187. find . -name "*.go" | grep -v "vendor/" | xargs -n 1 sed -i.bak 's/deadlock.RWMutex/sync.RWMutex/'
  188. find . -name "*.go" | grep -v "vendor/" | xargs -n 1 sed -i.bak 's/deadlock.Mutex/sync.Mutex/'
  189. find . -name "*.go" | grep -v "vendor/" | xargs -n 1 goimports -w
  190. ########################################
  191. ### Formatting, linting, and vetting
  192. fmt:
  193. @go fmt ./...
  194. lint:
  195. @echo "--> Running linter"
  196. @golangci-lint run
  197. DESTINATION = ./index.html.md
  198. rpc-docs:
  199. cat rpc/core/slate_header.txt > $(DESTINATION)
  200. godoc2md -template rpc/core/doc_template.txt github.com/tendermint/tendermint/rpc/core | grep -v -e "pipe.go" -e "routes.go" -e "dev.go" | sed 's,/src/target,https://github.com/tendermint/tendermint/tree/master/rpc/core,' >> $(DESTINATION)
  201. ###########################################################
  202. ### Docker image
  203. build-docker:
  204. cp $(OUTPUT) DOCKER/tendermint
  205. docker build --label=tendermint --tag="tendermint/tendermint" DOCKER
  206. rm -rf DOCKER/tendermint
  207. ###########################################################
  208. ### Local testnet using docker
  209. # Build linux binary on other platforms
  210. build-linux: get_tools
  211. GOOS=linux GOARCH=amd64 $(MAKE) build
  212. build-docker-localnode:
  213. @cd networks/local && make
  214. # Run a 4-node testnet locally
  215. localnet-start: localnet-stop build-docker-localnode
  216. @if ! [ -f build/node0/config/genesis.json ]; then docker run --rm -v $(CURDIR)/build:/tendermint:Z tendermint/localnode testnet --config /etc/tendermint/config-template.toml --v 4 --o . --populate-persistent-peers --starting-ip-address 192.167.10.2; fi
  217. docker-compose up
  218. # Stop testnet
  219. localnet-stop:
  220. docker-compose down
  221. ###########################################################
  222. ### Remote full-nodes (sentry) using terraform and ansible
  223. # Server management
  224. sentry-start:
  225. @if [ -z "$(DO_API_TOKEN)" ]; then echo "DO_API_TOKEN environment variable not set." ; false ; fi
  226. @if ! [ -f $(HOME)/.ssh/id_rsa.pub ]; then ssh-keygen ; fi
  227. cd networks/remote/terraform && terraform init && terraform apply -var DO_API_TOKEN="$(DO_API_TOKEN)" -var SSH_KEY_FILE="$(HOME)/.ssh/id_rsa.pub"
  228. @if ! [ -f $(CURDIR)/build/node0/config/genesis.json ]; then docker run --rm -v $(CURDIR)/build:/tendermint:Z tendermint/localnode testnet --v 0 --n 4 --o . ; fi
  229. cd networks/remote/ansible && ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -i inventory/digital_ocean.py -l sentrynet install.yml
  230. @echo "Next step: Add your validator setup in the genesis.json and config.tml files and run \"make sentry-config\". (Public key of validator, chain ID, peer IP and node ID.)"
  231. # Configuration management
  232. sentry-config:
  233. cd networks/remote/ansible && ansible-playbook -i inventory/digital_ocean.py -l sentrynet config.yml -e BINARY=$(CURDIR)/build/tendermint -e CONFIGDIR=$(CURDIR)/build
  234. sentry-stop:
  235. @if [ -z "$(DO_API_TOKEN)" ]; then echo "DO_API_TOKEN environment variable not set." ; false ; fi
  236. cd networks/remote/terraform && terraform destroy -var DO_API_TOKEN="$(DO_API_TOKEN)" -var SSH_KEY_FILE="$(HOME)/.ssh/id_rsa.pub"
  237. # meant for the CI, inspect script & adapt accordingly
  238. build-slate:
  239. bash scripts/slate.sh
  240. # Build hooks for dredd, to skip or add information on some steps
  241. build-contract-tests-hooks:
  242. ifeq ($(OS),Windows_NT)
  243. go build -mod=readonly $(BUILD_FLAGS) -o build/contract_tests.exe ./cmd/contract_tests
  244. else
  245. go build -mod=readonly $(BUILD_FLAGS) -o build/contract_tests ./cmd/contract_tests
  246. endif
  247. # Run a nodejs tool to test endpoints against a localnet
  248. # The command takes care of starting and stopping the network
  249. # prerequisits: build-contract-tests-hooks build-linux
  250. # the two build commands were not added to let this command run from generic containers or machines.
  251. # The binaries should be built beforehand
  252. contract-tests:
  253. dredd
  254. # To avoid unintended conflicts with file names, always add to .PHONY
  255. # unless there is a reason not to.
  256. # https://www.gnu.org/software/make/manual/html_node/Phony-Targets.html
  257. .PHONY: check build build_race build_abci dist install install_abci check_tools get_tools update_tools draw_deps get_protoc protoc_abci protoc_libs gen_certs clean_certs grpc_dbserver test_cover test_apps test_persistence test_p2p test test_race test_integrations test_release test100 vagrant_test fmt rpc-docs build-linux localnet-start localnet-stop build-docker build-docker-localnode sentry-start sentry-config sentry-stop build-slate protoc_grpc protoc_all build_c install_c test_with_deadlock cleanup_after_test_with_deadlock lint build-contract-tests-hooks contract-tests