You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

353 lines
11 KiB

privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
  1. package privval
  2. import (
  3. "context"
  4. "encoding/base64"
  5. "encoding/json"
  6. "fmt"
  7. "os"
  8. "testing"
  9. "time"
  10. "github.com/stretchr/testify/assert"
  11. "github.com/stretchr/testify/require"
  12. "github.com/tendermint/tendermint/crypto/ed25519"
  13. "github.com/tendermint/tendermint/crypto/tmhash"
  14. tmjson "github.com/tendermint/tendermint/libs/json"
  15. tmrand "github.com/tendermint/tendermint/libs/rand"
  16. tmtime "github.com/tendermint/tendermint/libs/time"
  17. tmproto "github.com/tendermint/tendermint/proto/tendermint/types"
  18. "github.com/tendermint/tendermint/types"
  19. )
  20. func TestGenLoadValidator(t *testing.T) {
  21. tempKeyFile, err := os.CreateTemp("", "priv_validator_key_")
  22. require.NoError(t, err)
  23. tempStateFile, err := os.CreateTemp("", "priv_validator_state_")
  24. require.NoError(t, err)
  25. privVal, err := GenFilePV(tempKeyFile.Name(), tempStateFile.Name(), "")
  26. require.NoError(t, err)
  27. height := int64(100)
  28. privVal.LastSignState.Height = height
  29. require.NoError(t, privVal.Save())
  30. addr := privVal.GetAddress()
  31. privVal, err = LoadFilePV(tempKeyFile.Name(), tempStateFile.Name())
  32. assert.NoError(t, err)
  33. assert.Equal(t, addr, privVal.GetAddress(), "expected privval addr to be the same")
  34. assert.Equal(t, height, privVal.LastSignState.Height, "expected privval.LastHeight to have been saved")
  35. }
  36. func TestResetValidator(t *testing.T) {
  37. ctx, cancel := context.WithCancel(context.Background())
  38. defer cancel()
  39. tempKeyFile, err := os.CreateTemp("", "priv_validator_key_")
  40. require.NoError(t, err)
  41. tempStateFile, err := os.CreateTemp("", "priv_validator_state_")
  42. require.NoError(t, err)
  43. privVal, err := GenFilePV(tempKeyFile.Name(), tempStateFile.Name(), "")
  44. require.NoError(t, err)
  45. emptyState := FilePVLastSignState{filePath: tempStateFile.Name()}
  46. // new priv val has empty state
  47. assert.Equal(t, privVal.LastSignState, emptyState)
  48. // test vote
  49. height, round := int64(10), int32(1)
  50. voteType := tmproto.PrevoteType
  51. randBytes := tmrand.Bytes(tmhash.Size)
  52. blockID := types.BlockID{Hash: randBytes, PartSetHeader: types.PartSetHeader{}}
  53. vote := newVote(privVal.Key.Address, 0, height, round, voteType, blockID)
  54. err = privVal.SignVote(ctx, "mychainid", vote.ToProto())
  55. assert.NoError(t, err, "expected no error signing vote")
  56. // priv val after signing is not same as empty
  57. assert.NotEqual(t, privVal.LastSignState, emptyState)
  58. // priv val after AcceptNewConnection is same as empty
  59. require.NoError(t, privVal.Reset())
  60. assert.Equal(t, privVal.LastSignState, emptyState)
  61. }
  62. func TestLoadOrGenValidator(t *testing.T) {
  63. tempKeyFile, err := os.CreateTemp("", "priv_validator_key_")
  64. require.NoError(t, err)
  65. tempStateFile, err := os.CreateTemp("", "priv_validator_state_")
  66. require.NoError(t, err)
  67. tempKeyFilePath := tempKeyFile.Name()
  68. if err := os.Remove(tempKeyFilePath); err != nil {
  69. t.Error(err)
  70. }
  71. tempStateFilePath := tempStateFile.Name()
  72. if err := os.Remove(tempStateFilePath); err != nil {
  73. t.Error(err)
  74. }
  75. privVal, err := LoadOrGenFilePV(tempKeyFilePath, tempStateFilePath)
  76. require.NoError(t, err)
  77. addr := privVal.GetAddress()
  78. privVal, err = LoadOrGenFilePV(tempKeyFilePath, tempStateFilePath)
  79. require.NoError(t, err)
  80. assert.Equal(t, addr, privVal.GetAddress(), "expected privval addr to be the same")
  81. }
  82. func TestUnmarshalValidatorState(t *testing.T) {
  83. // create some fixed values
  84. serialized := `{
  85. "height": "1",
  86. "round": 1,
  87. "step": 1
  88. }`
  89. val := FilePVLastSignState{}
  90. err := json.Unmarshal([]byte(serialized), &val)
  91. require.NoError(t, err)
  92. // make sure the values match
  93. assert.EqualValues(t, val.Height, 1)
  94. assert.EqualValues(t, val.Round, 1)
  95. assert.EqualValues(t, val.Step, 1)
  96. // export it and make sure it is the same
  97. out, err := json.Marshal(val)
  98. require.NoError(t, err)
  99. assert.JSONEq(t, serialized, string(out))
  100. }
  101. func TestUnmarshalValidatorKey(t *testing.T) {
  102. // create some fixed values
  103. privKey := ed25519.GenPrivKey()
  104. pubKey := privKey.PubKey()
  105. addr := pubKey.Address()
  106. pubBytes := pubKey.Bytes()
  107. privBytes := privKey.Bytes()
  108. pubB64 := base64.StdEncoding.EncodeToString(pubBytes)
  109. privB64 := base64.StdEncoding.EncodeToString(privBytes)
  110. serialized := fmt.Sprintf(`{
  111. "address": "%s",
  112. "pub_key": {
  113. "type": "tendermint/PubKeyEd25519",
  114. "value": "%s"
  115. },
  116. "priv_key": {
  117. "type": "tendermint/PrivKeyEd25519",
  118. "value": "%s"
  119. }
  120. }`, addr, pubB64, privB64)
  121. val := FilePVKey{}
  122. err := tmjson.Unmarshal([]byte(serialized), &val)
  123. require.NoError(t, err)
  124. // make sure the values match
  125. assert.EqualValues(t, addr, val.Address)
  126. assert.EqualValues(t, pubKey, val.PubKey)
  127. assert.EqualValues(t, privKey, val.PrivKey)
  128. // export it and make sure it is the same
  129. out, err := tmjson.Marshal(val)
  130. require.NoError(t, err)
  131. assert.JSONEq(t, serialized, string(out))
  132. }
  133. func TestSignVote(t *testing.T) {
  134. ctx, cancel := context.WithCancel(context.Background())
  135. defer cancel()
  136. tempKeyFile, err := os.CreateTemp("", "priv_validator_key_")
  137. require.NoError(t, err)
  138. tempStateFile, err := os.CreateTemp("", "priv_validator_state_")
  139. require.NoError(t, err)
  140. privVal, err := GenFilePV(tempKeyFile.Name(), tempStateFile.Name(), "")
  141. require.NoError(t, err)
  142. randbytes := tmrand.Bytes(tmhash.Size)
  143. randbytes2 := tmrand.Bytes(tmhash.Size)
  144. block1 := types.BlockID{Hash: randbytes,
  145. PartSetHeader: types.PartSetHeader{Total: 5, Hash: randbytes}}
  146. block2 := types.BlockID{Hash: randbytes2,
  147. PartSetHeader: types.PartSetHeader{Total: 10, Hash: randbytes2}}
  148. height, round := int64(10), int32(1)
  149. voteType := tmproto.PrevoteType
  150. // sign a vote for first time
  151. vote := newVote(privVal.Key.Address, 0, height, round, voteType, block1)
  152. v := vote.ToProto()
  153. err = privVal.SignVote(ctx, "mychainid", v)
  154. assert.NoError(t, err, "expected no error signing vote")
  155. // try to sign the same vote again; should be fine
  156. err = privVal.SignVote(ctx, "mychainid", v)
  157. assert.NoError(t, err, "expected no error on signing same vote")
  158. // now try some bad votes
  159. cases := []*types.Vote{
  160. newVote(privVal.Key.Address, 0, height, round-1, voteType, block1), // round regression
  161. newVote(privVal.Key.Address, 0, height-1, round, voteType, block1), // height regression
  162. newVote(privVal.Key.Address, 0, height-2, round+4, voteType, block1), // height regression and different round
  163. newVote(privVal.Key.Address, 0, height, round, voteType, block2), // different block
  164. }
  165. for _, c := range cases {
  166. assert.Error(t, privVal.SignVote(ctx, "mychainid", c.ToProto()),
  167. "expected error on signing conflicting vote")
  168. }
  169. // try signing a vote with a different time stamp
  170. sig := vote.Signature
  171. vote.Timestamp = vote.Timestamp.Add(time.Duration(1000))
  172. err = privVal.SignVote(ctx, "mychainid", v)
  173. assert.NoError(t, err)
  174. assert.Equal(t, sig, vote.Signature)
  175. }
  176. func TestSignProposal(t *testing.T) {
  177. ctx, cancel := context.WithCancel(context.Background())
  178. defer cancel()
  179. tempKeyFile, err := os.CreateTemp("", "priv_validator_key_")
  180. require.NoError(t, err)
  181. tempStateFile, err := os.CreateTemp("", "priv_validator_state_")
  182. require.NoError(t, err)
  183. privVal, err := GenFilePV(tempKeyFile.Name(), tempStateFile.Name(), "")
  184. require.NoError(t, err)
  185. randbytes := tmrand.Bytes(tmhash.Size)
  186. randbytes2 := tmrand.Bytes(tmhash.Size)
  187. block1 := types.BlockID{Hash: randbytes,
  188. PartSetHeader: types.PartSetHeader{Total: 5, Hash: randbytes}}
  189. block2 := types.BlockID{Hash: randbytes2,
  190. PartSetHeader: types.PartSetHeader{Total: 10, Hash: randbytes2}}
  191. height, round := int64(10), int32(1)
  192. // sign a proposal for first time
  193. proposal := newProposal(height, round, block1)
  194. pbp := proposal.ToProto()
  195. err = privVal.SignProposal(ctx, "mychainid", pbp)
  196. assert.NoError(t, err, "expected no error signing proposal")
  197. // try to sign the same proposal again; should be fine
  198. err = privVal.SignProposal(ctx, "mychainid", pbp)
  199. assert.NoError(t, err, "expected no error on signing same proposal")
  200. // now try some bad Proposals
  201. cases := []*types.Proposal{
  202. newProposal(height, round-1, block1), // round regression
  203. newProposal(height-1, round, block1), // height regression
  204. newProposal(height-2, round+4, block1), // height regression and different round
  205. newProposal(height, round, block2), // different block
  206. }
  207. for _, c := range cases {
  208. assert.Error(t, privVal.SignProposal(ctx, "mychainid", c.ToProto()),
  209. "expected error on signing conflicting proposal")
  210. }
  211. // try signing a proposal with a different time stamp
  212. sig := proposal.Signature
  213. proposal.Timestamp = proposal.Timestamp.Add(time.Duration(1000))
  214. err = privVal.SignProposal(ctx, "mychainid", pbp)
  215. assert.NoError(t, err)
  216. assert.Equal(t, sig, proposal.Signature)
  217. }
  218. func TestDifferByTimestamp(t *testing.T) {
  219. ctx, cancel := context.WithCancel(context.Background())
  220. defer cancel()
  221. tempKeyFile, err := os.CreateTemp("", "priv_validator_key_")
  222. require.NoError(t, err)
  223. tempStateFile, err := os.CreateTemp("", "priv_validator_state_")
  224. require.NoError(t, err)
  225. privVal, err := GenFilePV(tempKeyFile.Name(), tempStateFile.Name(), "")
  226. require.NoError(t, err)
  227. randbytes := tmrand.Bytes(tmhash.Size)
  228. block1 := types.BlockID{Hash: randbytes, PartSetHeader: types.PartSetHeader{Total: 5, Hash: randbytes}}
  229. height, round := int64(10), int32(1)
  230. chainID := "mychainid"
  231. // test proposal
  232. {
  233. proposal := newProposal(height, round, block1)
  234. pb := proposal.ToProto()
  235. err := privVal.SignProposal(ctx, chainID, pb)
  236. require.NoError(t, err, "expected no error signing proposal")
  237. signBytes := types.ProposalSignBytes(chainID, pb)
  238. sig := proposal.Signature
  239. timeStamp := proposal.Timestamp
  240. // manipulate the timestamp. should get changed back
  241. pb.Timestamp = pb.Timestamp.Add(time.Millisecond)
  242. var emptySig []byte
  243. proposal.Signature = emptySig
  244. err = privVal.SignProposal(ctx, "mychainid", pb)
  245. require.NoError(t, err, "expected no error on signing same proposal")
  246. assert.Equal(t, timeStamp, pb.Timestamp)
  247. assert.Equal(t, signBytes, types.ProposalSignBytes(chainID, pb))
  248. assert.Equal(t, sig, proposal.Signature)
  249. }
  250. // test vote
  251. {
  252. voteType := tmproto.PrevoteType
  253. blockID := types.BlockID{Hash: randbytes, PartSetHeader: types.PartSetHeader{}}
  254. vote := newVote(privVal.Key.Address, 0, height, round, voteType, blockID)
  255. v := vote.ToProto()
  256. err := privVal.SignVote(ctx, "mychainid", v)
  257. require.NoError(t, err, "expected no error signing vote")
  258. signBytes := types.VoteSignBytes(chainID, v)
  259. sig := v.Signature
  260. timeStamp := vote.Timestamp
  261. // manipulate the timestamp. should get changed back
  262. v.Timestamp = v.Timestamp.Add(time.Millisecond)
  263. var emptySig []byte
  264. v.Signature = emptySig
  265. err = privVal.SignVote(ctx, "mychainid", v)
  266. require.NoError(t, err, "expected no error on signing same vote")
  267. assert.Equal(t, timeStamp, v.Timestamp)
  268. assert.Equal(t, signBytes, types.VoteSignBytes(chainID, v))
  269. assert.Equal(t, sig, v.Signature)
  270. }
  271. }
  272. func newVote(addr types.Address, idx int32, height int64, round int32,
  273. typ tmproto.SignedMsgType, blockID types.BlockID) *types.Vote {
  274. return &types.Vote{
  275. ValidatorAddress: addr,
  276. ValidatorIndex: idx,
  277. Height: height,
  278. Round: round,
  279. Type: typ,
  280. Timestamp: tmtime.Now(),
  281. BlockID: blockID,
  282. }
  283. }
  284. func newProposal(height int64, round int32, blockID types.BlockID) *types.Proposal {
  285. return &types.Proposal{
  286. Height: height,
  287. Round: round,
  288. BlockID: blockID,
  289. Timestamp: tmtime.Now(),
  290. }
  291. }