You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

275 lines
7.0 KiB

Close and retry a RemoteSigner on err (#2923) * Close and recreate a RemoteSigner on err * Update changelog * Address Anton's comments / suggestions: - update changelog - restart TCPVal - shut down on `ErrUnexpectedResponse` * re-init remote signer client with fresh connection if Ping fails - add/update TODOs in secret connection - rename tcp.go -> tcp_client.go, same with ipc to clarify their purpose * account for `conn returned by waitConnection can be `nil` - also add TODO about RemoteSigner conn field * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn - add rwmutex for conn field in IPC * comments and doc.go * fix ipc tests. fixes #2677 * use constants for tests * cleanup some error statements * fixes #2784, race in tests * remove print statement * minor fixes from review * update comment on sts spec * cosmetics * p2p/conn: add failing tests * p2p/conn: make SecretConnection thread safe * changelog * IPCVal signer refactor - use a .reset() method - don't use embedded RemoteSignerClient - guard RemoteSignerClient with mutex - drop the .conn - expose Close() on RemoteSignerClient * apply IPCVal refactor to TCPVal * remove mtx from RemoteSignerClient * consolidate IPCVal and TCPVal, fixes #3104 - done in tcp_client.go - now called SocketVal - takes a listener in the constructor - make tcpListener and unixListener contain all the differences * delete ipc files * introduce unix and tcp dialer for RemoteSigner * rename files - drop tcp_ prefix - rename priv_validator.go to file.go * bring back listener options * fix node * fix priv_val_server * fix node test * minor cleanup and comments
6 years ago
Close and retry a RemoteSigner on err (#2923) * Close and recreate a RemoteSigner on err * Update changelog * Address Anton's comments / suggestions: - update changelog - restart TCPVal - shut down on `ErrUnexpectedResponse` * re-init remote signer client with fresh connection if Ping fails - add/update TODOs in secret connection - rename tcp.go -> tcp_client.go, same with ipc to clarify their purpose * account for `conn returned by waitConnection can be `nil` - also add TODO about RemoteSigner conn field * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn - add rwmutex for conn field in IPC * comments and doc.go * fix ipc tests. fixes #2677 * use constants for tests * cleanup some error statements * fixes #2784, race in tests * remove print statement * minor fixes from review * update comment on sts spec * cosmetics * p2p/conn: add failing tests * p2p/conn: make SecretConnection thread safe * changelog * IPCVal signer refactor - use a .reset() method - don't use embedded RemoteSignerClient - guard RemoteSignerClient with mutex - drop the .conn - expose Close() on RemoteSignerClient * apply IPCVal refactor to TCPVal * remove mtx from RemoteSignerClient * consolidate IPCVal and TCPVal, fixes #3104 - done in tcp_client.go - now called SocketVal - takes a listener in the constructor - make tcpListener and unixListener contain all the differences * delete ipc files * introduce unix and tcp dialer for RemoteSigner * rename files - drop tcp_ prefix - rename priv_validator.go to file.go * bring back listener options * fix node * fix priv_val_server * fix node test * minor cleanup and comments
6 years ago
Close and retry a RemoteSigner on err (#2923) * Close and recreate a RemoteSigner on err * Update changelog * Address Anton's comments / suggestions: - update changelog - restart TCPVal - shut down on `ErrUnexpectedResponse` * re-init remote signer client with fresh connection if Ping fails - add/update TODOs in secret connection - rename tcp.go -> tcp_client.go, same with ipc to clarify their purpose * account for `conn returned by waitConnection can be `nil` - also add TODO about RemoteSigner conn field * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn - add rwmutex for conn field in IPC * comments and doc.go * fix ipc tests. fixes #2677 * use constants for tests * cleanup some error statements * fixes #2784, race in tests * remove print statement * minor fixes from review * update comment on sts spec * cosmetics * p2p/conn: add failing tests * p2p/conn: make SecretConnection thread safe * changelog * IPCVal signer refactor - use a .reset() method - don't use embedded RemoteSignerClient - guard RemoteSignerClient with mutex - drop the .conn - expose Close() on RemoteSignerClient * apply IPCVal refactor to TCPVal * remove mtx from RemoteSignerClient * consolidate IPCVal and TCPVal, fixes #3104 - done in tcp_client.go - now called SocketVal - takes a listener in the constructor - make tcpListener and unixListener contain all the differences * delete ipc files * introduce unix and tcp dialer for RemoteSigner * rename files - drop tcp_ prefix - rename priv_validator.go to file.go * bring back listener options * fix node * fix priv_val_server * fix node test * minor cleanup and comments
6 years ago
Close and retry a RemoteSigner on err (#2923) * Close and recreate a RemoteSigner on err * Update changelog * Address Anton's comments / suggestions: - update changelog - restart TCPVal - shut down on `ErrUnexpectedResponse` * re-init remote signer client with fresh connection if Ping fails - add/update TODOs in secret connection - rename tcp.go -> tcp_client.go, same with ipc to clarify their purpose * account for `conn returned by waitConnection can be `nil` - also add TODO about RemoteSigner conn field * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn - add rwmutex for conn field in IPC * comments and doc.go * fix ipc tests. fixes #2677 * use constants for tests * cleanup some error statements * fixes #2784, race in tests * remove print statement * minor fixes from review * update comment on sts spec * cosmetics * p2p/conn: add failing tests * p2p/conn: make SecretConnection thread safe * changelog * IPCVal signer refactor - use a .reset() method - don't use embedded RemoteSignerClient - guard RemoteSignerClient with mutex - drop the .conn - expose Close() on RemoteSignerClient * apply IPCVal refactor to TCPVal * remove mtx from RemoteSignerClient * consolidate IPCVal and TCPVal, fixes #3104 - done in tcp_client.go - now called SocketVal - takes a listener in the constructor - make tcpListener and unixListener contain all the differences * delete ipc files * introduce unix and tcp dialer for RemoteSigner * rename files - drop tcp_ prefix - rename priv_validator.go to file.go * bring back listener options * fix node * fix priv_val_server * fix node test * minor cleanup and comments
6 years ago
Close and retry a RemoteSigner on err (#2923) * Close and recreate a RemoteSigner on err * Update changelog * Address Anton's comments / suggestions: - update changelog - restart TCPVal - shut down on `ErrUnexpectedResponse` * re-init remote signer client with fresh connection if Ping fails - add/update TODOs in secret connection - rename tcp.go -> tcp_client.go, same with ipc to clarify their purpose * account for `conn returned by waitConnection can be `nil` - also add TODO about RemoteSigner conn field * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn - add rwmutex for conn field in IPC * comments and doc.go * fix ipc tests. fixes #2677 * use constants for tests * cleanup some error statements * fixes #2784, race in tests * remove print statement * minor fixes from review * update comment on sts spec * cosmetics * p2p/conn: add failing tests * p2p/conn: make SecretConnection thread safe * changelog * IPCVal signer refactor - use a .reset() method - don't use embedded RemoteSignerClient - guard RemoteSignerClient with mutex - drop the .conn - expose Close() on RemoteSignerClient * apply IPCVal refactor to TCPVal * remove mtx from RemoteSignerClient * consolidate IPCVal and TCPVal, fixes #3104 - done in tcp_client.go - now called SocketVal - takes a listener in the constructor - make tcpListener and unixListener contain all the differences * delete ipc files * introduce unix and tcp dialer for RemoteSigner * rename files - drop tcp_ prefix - rename priv_validator.go to file.go * bring back listener options * fix node * fix priv_val_server * fix node test * minor cleanup and comments
6 years ago
Close and retry a RemoteSigner on err (#2923) * Close and recreate a RemoteSigner on err * Update changelog * Address Anton's comments / suggestions: - update changelog - restart TCPVal - shut down on `ErrUnexpectedResponse` * re-init remote signer client with fresh connection if Ping fails - add/update TODOs in secret connection - rename tcp.go -> tcp_client.go, same with ipc to clarify their purpose * account for `conn returned by waitConnection can be `nil` - also add TODO about RemoteSigner conn field * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn - add rwmutex for conn field in IPC * comments and doc.go * fix ipc tests. fixes #2677 * use constants for tests * cleanup some error statements * fixes #2784, race in tests * remove print statement * minor fixes from review * update comment on sts spec * cosmetics * p2p/conn: add failing tests * p2p/conn: make SecretConnection thread safe * changelog * IPCVal signer refactor - use a .reset() method - don't use embedded RemoteSignerClient - guard RemoteSignerClient with mutex - drop the .conn - expose Close() on RemoteSignerClient * apply IPCVal refactor to TCPVal * remove mtx from RemoteSignerClient * consolidate IPCVal and TCPVal, fixes #3104 - done in tcp_client.go - now called SocketVal - takes a listener in the constructor - make tcpListener and unixListener contain all the differences * delete ipc files * introduce unix and tcp dialer for RemoteSigner * rename files - drop tcp_ prefix - rename priv_validator.go to file.go * bring back listener options * fix node * fix priv_val_server * fix node test * minor cleanup and comments
6 years ago
  1. package privval
  2. import (
  3. "fmt"
  4. "io"
  5. "net"
  6. "github.com/pkg/errors"
  7. amino "github.com/tendermint/go-amino"
  8. "github.com/tendermint/tendermint/crypto"
  9. cmn "github.com/tendermint/tendermint/libs/common"
  10. "github.com/tendermint/tendermint/types"
  11. )
  12. // Socket errors.
  13. var (
  14. ErrConnTimeout = errors.New("remote signer timed out")
  15. )
  16. // RemoteSignerClient implements PrivValidator.
  17. // It uses a net.Conn to request signatures
  18. // from an external process.
  19. type RemoteSignerClient struct {
  20. conn net.Conn
  21. // memoized
  22. consensusPubKey crypto.PubKey
  23. }
  24. // Check that RemoteSignerClient implements PrivValidator.
  25. var _ types.PrivValidator = (*RemoteSignerClient)(nil)
  26. // NewRemoteSignerClient returns an instance of RemoteSignerClient.
  27. func NewRemoteSignerClient(conn net.Conn) (*RemoteSignerClient, error) {
  28. // retrieve and memoize the consensus public key once.
  29. pubKey, err := getPubKey(conn)
  30. if err != nil {
  31. return nil, cmn.ErrorWrap(err, "error while retrieving public key for remote signer")
  32. }
  33. return &RemoteSignerClient{
  34. conn: conn,
  35. consensusPubKey: pubKey,
  36. }, nil
  37. }
  38. // Close calls Close on the underlying net.Conn.
  39. func (sc *RemoteSignerClient) Close() error {
  40. return sc.conn.Close()
  41. }
  42. // GetPubKey implements PrivValidator.
  43. func (sc *RemoteSignerClient) GetPubKey() crypto.PubKey {
  44. return sc.consensusPubKey
  45. }
  46. // not thread-safe (only called on startup).
  47. func getPubKey(conn net.Conn) (crypto.PubKey, error) {
  48. err := writeMsg(conn, &PubKeyRequest{})
  49. if err != nil {
  50. return nil, err
  51. }
  52. res, err := readMsg(conn)
  53. if err != nil {
  54. return nil, err
  55. }
  56. pubKeyResp, ok := res.(*PubKeyResponse)
  57. if !ok {
  58. return nil, errors.Wrap(ErrUnexpectedResponse, "response is not PubKeyResponse")
  59. }
  60. if pubKeyResp.Error != nil {
  61. return nil, errors.Wrap(pubKeyResp.Error, "failed to get private validator's public key")
  62. }
  63. return pubKeyResp.PubKey, nil
  64. }
  65. // SignVote implements PrivValidator.
  66. func (sc *RemoteSignerClient) SignVote(chainID string, vote *types.Vote) error {
  67. err := writeMsg(sc.conn, &SignVoteRequest{Vote: vote})
  68. if err != nil {
  69. return err
  70. }
  71. res, err := readMsg(sc.conn)
  72. if err != nil {
  73. return err
  74. }
  75. resp, ok := res.(*SignedVoteResponse)
  76. if !ok {
  77. return ErrUnexpectedResponse
  78. }
  79. if resp.Error != nil {
  80. return resp.Error
  81. }
  82. *vote = *resp.Vote
  83. return nil
  84. }
  85. // SignProposal implements PrivValidator.
  86. func (sc *RemoteSignerClient) SignProposal(
  87. chainID string,
  88. proposal *types.Proposal,
  89. ) error {
  90. err := writeMsg(sc.conn, &SignProposalRequest{Proposal: proposal})
  91. if err != nil {
  92. return err
  93. }
  94. res, err := readMsg(sc.conn)
  95. if err != nil {
  96. return err
  97. }
  98. resp, ok := res.(*SignedProposalResponse)
  99. if !ok {
  100. return ErrUnexpectedResponse
  101. }
  102. if resp.Error != nil {
  103. return resp.Error
  104. }
  105. *proposal = *resp.Proposal
  106. return nil
  107. }
  108. // Ping is used to check connection health.
  109. func (sc *RemoteSignerClient) Ping() error {
  110. err := writeMsg(sc.conn, &PingRequest{})
  111. if err != nil {
  112. return err
  113. }
  114. res, err := readMsg(sc.conn)
  115. if err != nil {
  116. return err
  117. }
  118. _, ok := res.(*PingResponse)
  119. if !ok {
  120. return ErrUnexpectedResponse
  121. }
  122. return nil
  123. }
  124. // RemoteSignerMsg is sent between RemoteSigner and the RemoteSigner client.
  125. type RemoteSignerMsg interface{}
  126. func RegisterRemoteSignerMsg(cdc *amino.Codec) {
  127. cdc.RegisterInterface((*RemoteSignerMsg)(nil), nil)
  128. cdc.RegisterConcrete(&PubKeyRequest{}, "tendermint/remotesigner/PubKeyRequest", nil)
  129. cdc.RegisterConcrete(&PubKeyResponse{}, "tendermint/remotesigner/PubKeyResponse", nil)
  130. cdc.RegisterConcrete(&SignVoteRequest{}, "tendermint/remotesigner/SignVoteRequest", nil)
  131. cdc.RegisterConcrete(&SignedVoteResponse{}, "tendermint/remotesigner/SignedVoteResponse", nil)
  132. cdc.RegisterConcrete(&SignProposalRequest{}, "tendermint/remotesigner/SignProposalRequest", nil)
  133. cdc.RegisterConcrete(&SignedProposalResponse{}, "tendermint/remotesigner/SignedProposalResponse", nil)
  134. cdc.RegisterConcrete(&PingRequest{}, "tendermint/remotesigner/PingRequest", nil)
  135. cdc.RegisterConcrete(&PingResponse{}, "tendermint/remotesigner/PingResponse", nil)
  136. }
  137. // PubKeyRequest requests the consensus public key from the remote signer.
  138. type PubKeyRequest struct{}
  139. // PubKeyResponse is a PrivValidatorSocket message containing the public key.
  140. type PubKeyResponse struct {
  141. PubKey crypto.PubKey
  142. Error *RemoteSignerError
  143. }
  144. // SignVoteRequest is a PrivValidatorSocket message containing a vote.
  145. type SignVoteRequest struct {
  146. Vote *types.Vote
  147. }
  148. // SignedVoteResponse is a PrivValidatorSocket message containing a signed vote along with a potenial error message.
  149. type SignedVoteResponse struct {
  150. Vote *types.Vote
  151. Error *RemoteSignerError
  152. }
  153. // SignProposalRequest is a PrivValidatorSocket message containing a Proposal.
  154. type SignProposalRequest struct {
  155. Proposal *types.Proposal
  156. }
  157. type SignedProposalResponse struct {
  158. Proposal *types.Proposal
  159. Error *RemoteSignerError
  160. }
  161. // PingRequest is a PrivValidatorSocket message to keep the connection alive.
  162. type PingRequest struct {
  163. }
  164. type PingResponse struct {
  165. }
  166. // RemoteSignerError allows (remote) validators to include meaningful error descriptions in their reply.
  167. type RemoteSignerError struct {
  168. // TODO(ismail): create an enum of known errors
  169. Code int
  170. Description string
  171. }
  172. func (e *RemoteSignerError) Error() string {
  173. return fmt.Sprintf("RemoteSigner returned error #%d: %s", e.Code, e.Description)
  174. }
  175. func readMsg(r io.Reader) (msg RemoteSignerMsg, err error) {
  176. const maxRemoteSignerMsgSize = 1024 * 10
  177. _, err = cdc.UnmarshalBinaryLengthPrefixedReader(r, &msg, maxRemoteSignerMsgSize)
  178. if _, ok := err.(timeoutError); ok {
  179. err = cmn.ErrorWrap(ErrConnTimeout, err.Error())
  180. }
  181. return
  182. }
  183. func writeMsg(w io.Writer, msg interface{}) (err error) {
  184. _, err = cdc.MarshalBinaryLengthPrefixedWriter(w, msg)
  185. if _, ok := err.(timeoutError); ok {
  186. err = cmn.ErrorWrap(ErrConnTimeout, err.Error())
  187. }
  188. return
  189. }
  190. func handleRequest(req RemoteSignerMsg, chainID string, privVal types.PrivValidator) (RemoteSignerMsg, error) {
  191. var res RemoteSignerMsg
  192. var err error
  193. switch r := req.(type) {
  194. case *PubKeyRequest:
  195. var p crypto.PubKey
  196. p = privVal.GetPubKey()
  197. res = &PubKeyResponse{p, nil}
  198. case *SignVoteRequest:
  199. err = privVal.SignVote(chainID, r.Vote)
  200. if err != nil {
  201. res = &SignedVoteResponse{nil, &RemoteSignerError{0, err.Error()}}
  202. } else {
  203. res = &SignedVoteResponse{r.Vote, nil}
  204. }
  205. case *SignProposalRequest:
  206. err = privVal.SignProposal(chainID, r.Proposal)
  207. if err != nil {
  208. res = &SignedProposalResponse{nil, &RemoteSignerError{0, err.Error()}}
  209. } else {
  210. res = &SignedProposalResponse{r.Proposal, nil}
  211. }
  212. case *PingRequest:
  213. res = &PingResponse{}
  214. default:
  215. err = fmt.Errorf("unknown msg: %v", r)
  216. }
  217. return res, err
  218. }
  219. // IsConnTimeout returns a boolean indicating whether the error is known to
  220. // report that a connection timeout occurred. This detects both fundamental
  221. // network timeouts, as well as ErrConnTimeout errors.
  222. func IsConnTimeout(err error) bool {
  223. if cmnErr, ok := err.(cmn.Error); ok {
  224. if cmnErr.Data() == ErrConnTimeout {
  225. return true
  226. }
  227. }
  228. if _, ok := err.(timeoutError); ok {
  229. return true
  230. }
  231. return false
  232. }