You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

294 lines
8.2 KiB

cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
rpc: add support for batched requests/responses (#3534) Continues from #3280 in building support for batched requests/responses in the JSON RPC (as per issue #3213). * Add JSON RPC batching for client and server As per #3213, this adds support for [JSON RPC batch requests and responses](https://www.jsonrpc.org/specification#batch). * Add additional checks to ensure client responses are the same as results * Fix case where a notification is sent and no response is expected * Add test to check that JSON RPC notifications in a batch are left out in responses * Update CHANGELOG_PENDING.md * Update PR number now that PR has been created * Make errors start with lowercase letter * Refactor batch functionality to be standalone This refactors the batching functionality to rather act in a standalone way. In light of supporting concurrent goroutines making use of the same client, it would make sense to have batching functionality where one could create a batch of requests per goroutine and send that batch without interfering with a batch from another goroutine. * Add examples for simple and batch HTTP client usage * Check errors from writer and remove nolinter directives * Make error strings start with lowercase letter * Refactor examples to make them testable * Use safer deferred shutdown for example Tendermint test node * Recompose rpcClient interface from pre-existing interface components * Rename WaitGroup for brevity * Replace empty ID string with request ID * Remove extraneous test case * Convert first letter of errors.Wrap() messages to lowercase * Remove extraneous function parameter * Make variable declaration terse * Reorder WaitGroup.Done call to help prevent race conditions in the face of failure * Swap mutex to value representation and remove initialization * Restore empty JSONRPC string ID in response to prevent nil * Make JSONRPCBufferedRequest private * Revert PR hard link in CHANGELOG_PENDING * Add client ID for JSONRPCClient This adds code to automatically generate a randomized client ID for the JSONRPCClient, and adds a check of the IDs in the responses (if one was set in the requests). * Extract response ID validation into separate function * Remove extraneous comments * Reorder fields to indicate clearly which are protected by the mutex * Refactor for loop to remove indexing * Restructure and combine loop * Flatten conditional block for better readability * Make multi-variable declaration slightly more readable * Change for loop style * Compress error check statements * Make function description more generic to show that we support different protocols * Preallocate memory for request and result objects
6 years ago
rpc: add support for batched requests/responses (#3534) Continues from #3280 in building support for batched requests/responses in the JSON RPC (as per issue #3213). * Add JSON RPC batching for client and server As per #3213, this adds support for [JSON RPC batch requests and responses](https://www.jsonrpc.org/specification#batch). * Add additional checks to ensure client responses are the same as results * Fix case where a notification is sent and no response is expected * Add test to check that JSON RPC notifications in a batch are left out in responses * Update CHANGELOG_PENDING.md * Update PR number now that PR has been created * Make errors start with lowercase letter * Refactor batch functionality to be standalone This refactors the batching functionality to rather act in a standalone way. In light of supporting concurrent goroutines making use of the same client, it would make sense to have batching functionality where one could create a batch of requests per goroutine and send that batch without interfering with a batch from another goroutine. * Add examples for simple and batch HTTP client usage * Check errors from writer and remove nolinter directives * Make error strings start with lowercase letter * Refactor examples to make them testable * Use safer deferred shutdown for example Tendermint test node * Recompose rpcClient interface from pre-existing interface components * Rename WaitGroup for brevity * Replace empty ID string with request ID * Remove extraneous test case * Convert first letter of errors.Wrap() messages to lowercase * Remove extraneous function parameter * Make variable declaration terse * Reorder WaitGroup.Done call to help prevent race conditions in the face of failure * Swap mutex to value representation and remove initialization * Restore empty JSONRPC string ID in response to prevent nil * Make JSONRPCBufferedRequest private * Revert PR hard link in CHANGELOG_PENDING * Add client ID for JSONRPCClient This adds code to automatically generate a randomized client ID for the JSONRPCClient, and adds a check of the IDs in the responses (if one was set in the requests). * Extract response ID validation into separate function * Remove extraneous comments * Reorder fields to indicate clearly which are protected by the mutex * Refactor for loop to remove indexing * Restructure and combine loop * Flatten conditional block for better readability * Make multi-variable declaration slightly more readable * Change for loop style * Compress error check statements * Make function description more generic to show that we support different protocols * Preallocate memory for request and result objects
6 years ago
  1. // Commons for HTTP handling
  2. package server
  3. import (
  4. "context"
  5. "encoding/json"
  6. "errors"
  7. "fmt"
  8. "net"
  9. "net/http"
  10. "runtime/debug"
  11. "strings"
  12. "time"
  13. "golang.org/x/net/netutil"
  14. "github.com/tendermint/tendermint/libs/log"
  15. rpctypes "github.com/tendermint/tendermint/rpc/jsonrpc/types"
  16. )
  17. // Config is a RPC server configuration.
  18. type Config struct {
  19. // see netutil.LimitListener
  20. MaxOpenConnections int
  21. // mirrors http.Server#ReadTimeout
  22. ReadTimeout time.Duration
  23. // mirrors http.Server#WriteTimeout
  24. WriteTimeout time.Duration
  25. // MaxBodyBytes controls the maximum number of bytes the
  26. // server will read parsing the request body.
  27. MaxBodyBytes int64
  28. // mirrors http.Server#MaxHeaderBytes
  29. MaxHeaderBytes int
  30. }
  31. // DefaultConfig returns a default configuration.
  32. func DefaultConfig() *Config {
  33. return &Config{
  34. MaxOpenConnections: 0, // unlimited
  35. ReadTimeout: 10 * time.Second,
  36. WriteTimeout: 10 * time.Second,
  37. MaxBodyBytes: int64(1000000), // 1MB
  38. MaxHeaderBytes: 1 << 20, // same as the net/http default
  39. }
  40. }
  41. // Serve creates a http.Server and calls Serve with the given listener. It
  42. // wraps handler to recover panics and limit the request body size.
  43. func Serve(
  44. ctx context.Context,
  45. listener net.Listener,
  46. handler http.Handler,
  47. logger log.Logger,
  48. config *Config,
  49. ) error {
  50. logger.Info(fmt.Sprintf("Starting RPC HTTP server on %s", listener.Addr()))
  51. h := recoverAndLogHandler(MaxBytesHandler(handler, config.MaxBodyBytes), logger)
  52. s := &http.Server{
  53. Handler: h,
  54. ReadTimeout: config.ReadTimeout,
  55. WriteTimeout: config.WriteTimeout,
  56. MaxHeaderBytes: config.MaxHeaderBytes,
  57. }
  58. sig := make(chan struct{})
  59. go func() {
  60. select {
  61. case <-ctx.Done():
  62. sctx, cancel := context.WithTimeout(context.Background(), time.Second)
  63. defer cancel()
  64. _ = s.Shutdown(sctx)
  65. case <-sig:
  66. }
  67. }()
  68. if err := s.Serve(listener); err != nil {
  69. logger.Info("RPC HTTP server stopped", "err", err)
  70. close(sig)
  71. return err
  72. }
  73. return nil
  74. }
  75. // Serve creates a http.Server and calls ServeTLS with the given listener,
  76. // certFile and keyFile. It wraps handler to recover panics and limit the
  77. // request body size.
  78. func ServeTLS(
  79. ctx context.Context,
  80. listener net.Listener,
  81. handler http.Handler,
  82. certFile, keyFile string,
  83. logger log.Logger,
  84. config *Config,
  85. ) error {
  86. logger.Info(fmt.Sprintf("Starting RPC HTTPS server on %s (cert: %q, key: %q)",
  87. listener.Addr(), certFile, keyFile))
  88. h := recoverAndLogHandler(MaxBytesHandler(handler, config.MaxBodyBytes), logger)
  89. s := &http.Server{
  90. Handler: h,
  91. ReadTimeout: config.ReadTimeout,
  92. WriteTimeout: config.WriteTimeout,
  93. MaxHeaderBytes: config.MaxHeaderBytes,
  94. }
  95. sig := make(chan struct{})
  96. go func() {
  97. select {
  98. case <-ctx.Done():
  99. sctx, cancel := context.WithTimeout(context.Background(), time.Second)
  100. defer cancel()
  101. _ = s.Shutdown(sctx)
  102. case <-sig:
  103. }
  104. }()
  105. if err := s.ServeTLS(listener, certFile, keyFile); err != nil {
  106. logger.Error("RPC HTTPS server stopped", "err", err)
  107. close(sig)
  108. return err
  109. }
  110. return nil
  111. }
  112. // writeInternalError writes an internal server error (500) to w with the text
  113. // of err in the body. This is a fallback used when a handler is unable to
  114. // write the expected response.
  115. func writeInternalError(w http.ResponseWriter, err error) {
  116. w.Header().Set("Content-Type", "text/plain")
  117. w.WriteHeader(http.StatusInternalServerError)
  118. fmt.Fprintln(w, err.Error())
  119. }
  120. // writeHTTPResponse writes a JSON-RPC response to w. If rsp encodes an error,
  121. // the response body is its error object; otherwise its responses is the result.
  122. //
  123. // Unless there is an error encoding the response, the status is 200 OK.
  124. func writeHTTPResponse(w http.ResponseWriter, log log.Logger, rsp rpctypes.RPCResponse) {
  125. var body []byte
  126. var err error
  127. if rsp.Error != nil {
  128. body, err = json.Marshal(rsp.Error)
  129. } else {
  130. body = rsp.Result
  131. }
  132. if err != nil {
  133. log.Error("Error encoding RPC response: %w", err)
  134. writeInternalError(w, err)
  135. return
  136. }
  137. w.Header().Set("Content-Type", "application/json")
  138. w.WriteHeader(http.StatusOK)
  139. _, _ = w.Write(body)
  140. }
  141. // writeRPCResponse writes one or more JSON-RPC responses to w. A single
  142. // response is encoded as an object, otherwise the response is sent as a batch
  143. // (array) of response objects.
  144. //
  145. // Unless there is an error encoding the responses, the status is 200 OK.
  146. func writeRPCResponse(w http.ResponseWriter, log log.Logger, rsps ...rpctypes.RPCResponse) {
  147. var body []byte
  148. var err error
  149. if len(rsps) == 1 {
  150. body, err = json.Marshal(rsps[0])
  151. } else {
  152. body, err = json.Marshal(rsps)
  153. }
  154. if err != nil {
  155. log.Error("Error encoding RPC response: %w", err)
  156. writeInternalError(w, err)
  157. return
  158. }
  159. w.Header().Set("Content-Type", "application/json")
  160. w.WriteHeader(http.StatusOK)
  161. _, _ = w.Write(body)
  162. }
  163. //-----------------------------------------------------------------------------
  164. // recoverAndLogHandler wraps an HTTP handler, adding error logging. If the
  165. // inner handler panics, the wrapper recovers, logs, sends an HTTP 500 error
  166. // response to the client.
  167. func recoverAndLogHandler(handler http.Handler, logger log.Logger) http.Handler {
  168. return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
  169. // Capture the HTTP status written by the handler.
  170. var httpStatus int
  171. rww := newStatusWriter(w, &httpStatus)
  172. // Recover panics from inside handler and try to send the client
  173. // 500 Internal server error. If the handler panicked after already
  174. // sending a (partial) response, this is a no-op.
  175. defer func() {
  176. if v := recover(); v != nil {
  177. var err error
  178. switch e := v.(type) {
  179. case error:
  180. err = e
  181. case string:
  182. err = errors.New(e)
  183. case fmt.Stringer:
  184. err = errors.New(e.String())
  185. default:
  186. err = fmt.Errorf("panic with value %v", v)
  187. }
  188. logger.Error("Panic in RPC HTTP handler",
  189. "err", err, "stack", string(debug.Stack()))
  190. writeInternalError(rww, err)
  191. }
  192. }()
  193. // Log timing and response information from the handler.
  194. begin := time.Now()
  195. defer func() {
  196. elapsed := time.Since(begin)
  197. logger.Debug("served RPC HTTP response",
  198. "method", r.Method,
  199. "url", r.URL,
  200. "status", httpStatus,
  201. "duration-sec", elapsed.Seconds(),
  202. "remoteAddr", r.RemoteAddr,
  203. )
  204. }()
  205. rww.Header().Set("X-Server-Time", fmt.Sprintf("%v", begin.Unix()))
  206. handler.ServeHTTP(rww, r)
  207. })
  208. }
  209. // MaxBytesHandler wraps h in a handler that limits the size of the request
  210. // body to at most maxBytes. If maxBytes <= 0, the request body is not limited.
  211. func MaxBytesHandler(h http.Handler, maxBytes int64) http.Handler {
  212. if maxBytes <= 0 {
  213. return h
  214. }
  215. return maxBytesHandler{handler: h, maxBytes: maxBytes}
  216. }
  217. type maxBytesHandler struct {
  218. handler http.Handler
  219. maxBytes int64
  220. }
  221. func (h maxBytesHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
  222. req.Body = http.MaxBytesReader(w, req.Body, h.maxBytes)
  223. h.handler.ServeHTTP(w, req)
  224. }
  225. // newStatusWriter wraps an http.ResponseWriter to capture the HTTP status code
  226. // in *code.
  227. func newStatusWriter(w http.ResponseWriter, code *int) statusWriter {
  228. return statusWriter{
  229. ResponseWriter: w,
  230. Hijacker: w.(http.Hijacker),
  231. code: code,
  232. }
  233. }
  234. type statusWriter struct {
  235. http.ResponseWriter
  236. http.Hijacker // to support websocket upgrade
  237. code *int
  238. }
  239. // WriteHeader implements part of http.ResponseWriter. It delegates to the
  240. // wrapped writer, and as a side effect captures the written code.
  241. //
  242. // Note that if a request does not explicitly call WriteHeader, the code will
  243. // not be updated.
  244. func (w statusWriter) WriteHeader(code int) {
  245. *w.code = code
  246. w.ResponseWriter.WriteHeader(code)
  247. }
  248. // Listen starts a new net.Listener on the given address.
  249. // It returns an error if the address is invalid or the call to Listen() fails.
  250. func Listen(addr string, maxOpenConnections int) (listener net.Listener, err error) {
  251. parts := strings.SplitN(addr, "://", 2)
  252. if len(parts) != 2 {
  253. return nil, fmt.Errorf(
  254. "invalid listening address %s (use fully formed addresses, including the tcp:// or unix:// prefix)",
  255. addr,
  256. )
  257. }
  258. proto, addr := parts[0], parts[1]
  259. listener, err = net.Listen(proto, addr)
  260. if err != nil {
  261. return nil, fmt.Errorf("failed to listen on %v: %v", addr, err)
  262. }
  263. if maxOpenConnections > 0 {
  264. listener = netutil.LimitListener(listener, maxOpenConnections)
  265. }
  266. return listener, nil
  267. }