You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

156 lines
3.4 KiB

privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
  1. package privval
  2. import (
  3. "fmt"
  4. "net"
  5. "sync"
  6. "time"
  7. "github.com/pkg/errors"
  8. cmn "github.com/tendermint/tendermint/libs/common"
  9. )
  10. const (
  11. defaultTimeoutReadWriteSeconds = 3
  12. )
  13. type signerEndpoint struct {
  14. cmn.BaseService
  15. connMtx sync.Mutex
  16. conn net.Conn
  17. timeoutReadWrite time.Duration
  18. }
  19. // Close closes the underlying net.Conn.
  20. func (se *signerEndpoint) Close() error {
  21. se.DropConnection()
  22. return nil
  23. }
  24. // IsConnected indicates if there is an active connection
  25. func (se *signerEndpoint) IsConnected() bool {
  26. se.connMtx.Lock()
  27. defer se.connMtx.Unlock()
  28. return se.isConnected()
  29. }
  30. // TryGetConnection retrieves a connection if it is already available
  31. func (se *signerEndpoint) GetAvailableConnection(connectionAvailableCh chan net.Conn) bool {
  32. se.connMtx.Lock()
  33. defer se.connMtx.Unlock()
  34. // Is there a connection ready?
  35. select {
  36. case se.conn = <-connectionAvailableCh:
  37. return true
  38. default:
  39. }
  40. return false
  41. }
  42. // TryGetConnection retrieves a connection if it is already available
  43. func (se *signerEndpoint) WaitConnection(connectionAvailableCh chan net.Conn, maxWait time.Duration) error {
  44. se.connMtx.Lock()
  45. defer se.connMtx.Unlock()
  46. select {
  47. case se.conn = <-connectionAvailableCh:
  48. case <-time.After(maxWait):
  49. return ErrConnectionTimeout
  50. }
  51. return nil
  52. }
  53. // SetConnection replaces the current connection object
  54. func (se *signerEndpoint) SetConnection(newConnection net.Conn) {
  55. se.connMtx.Lock()
  56. defer se.connMtx.Unlock()
  57. se.conn = newConnection
  58. }
  59. // IsConnected indicates if there is an active connection
  60. func (se *signerEndpoint) DropConnection() {
  61. se.connMtx.Lock()
  62. defer se.connMtx.Unlock()
  63. se.dropConnection()
  64. }
  65. // ReadMessage reads a message from the endpoint
  66. func (se *signerEndpoint) ReadMessage() (msg SignerMessage, err error) {
  67. se.connMtx.Lock()
  68. defer se.connMtx.Unlock()
  69. if !se.isConnected() {
  70. return nil, fmt.Errorf("endpoint is not connected")
  71. }
  72. // Reset read deadline
  73. deadline := time.Now().Add(se.timeoutReadWrite)
  74. err = se.conn.SetReadDeadline(deadline)
  75. if err != nil {
  76. return
  77. }
  78. const maxRemoteSignerMsgSize = 1024 * 10
  79. _, err = cdc.UnmarshalBinaryLengthPrefixedReader(se.conn, &msg, maxRemoteSignerMsgSize)
  80. if _, ok := err.(timeoutError); ok {
  81. if err != nil {
  82. err = errors.Wrap(ErrReadTimeout, err.Error())
  83. } else {
  84. err = errors.Wrap(ErrReadTimeout, "Empty error")
  85. }
  86. se.Logger.Debug("Dropping [read]", "obj", se)
  87. se.dropConnection()
  88. }
  89. return
  90. }
  91. // WriteMessage writes a message from the endpoint
  92. func (se *signerEndpoint) WriteMessage(msg SignerMessage) (err error) {
  93. se.connMtx.Lock()
  94. defer se.connMtx.Unlock()
  95. if !se.isConnected() {
  96. return errors.Wrap(ErrNoConnection, "endpoint is not connected")
  97. }
  98. // Reset read deadline
  99. deadline := time.Now().Add(se.timeoutReadWrite)
  100. se.Logger.Debug("Write::Error Resetting deadline", "obj", se)
  101. err = se.conn.SetWriteDeadline(deadline)
  102. if err != nil {
  103. return
  104. }
  105. _, err = cdc.MarshalBinaryLengthPrefixedWriter(se.conn, msg)
  106. if _, ok := err.(timeoutError); ok {
  107. if err != nil {
  108. err = errors.Wrap(ErrWriteTimeout, err.Error())
  109. } else {
  110. err = errors.Wrap(ErrWriteTimeout, "Empty error")
  111. }
  112. se.dropConnection()
  113. }
  114. return
  115. }
  116. func (se *signerEndpoint) isConnected() bool {
  117. return se.conn != nil
  118. }
  119. func (se *signerEndpoint) dropConnection() {
  120. if se.conn != nil {
  121. if err := se.conn.Close(); err != nil {
  122. se.Logger.Error("signerEndpoint::dropConnection", "err", err)
  123. }
  124. se.conn = nil
  125. }
  126. }