You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

322 lines
10 KiB

privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
  1. package privval
  2. import (
  3. "encoding/base64"
  4. "fmt"
  5. "io/ioutil"
  6. "os"
  7. "testing"
  8. "time"
  9. "github.com/stretchr/testify/assert"
  10. "github.com/stretchr/testify/require"
  11. "github.com/tendermint/tendermint/crypto/ed25519"
  12. tmproto "github.com/tendermint/tendermint/proto/types"
  13. "github.com/tendermint/tendermint/types"
  14. tmtime "github.com/tendermint/tendermint/types/time"
  15. )
  16. func TestGenLoadValidator(t *testing.T) {
  17. assert := assert.New(t)
  18. tempKeyFile, err := ioutil.TempFile("", "priv_validator_key_")
  19. require.Nil(t, err)
  20. tempStateFile, err := ioutil.TempFile("", "priv_validator_state_")
  21. require.Nil(t, err)
  22. privVal := GenFilePV(tempKeyFile.Name(), tempStateFile.Name())
  23. height := int64(100)
  24. privVal.LastSignState.Height = height
  25. privVal.Save()
  26. addr := privVal.GetAddress()
  27. privVal = LoadFilePV(tempKeyFile.Name(), tempStateFile.Name())
  28. assert.Equal(addr, privVal.GetAddress(), "expected privval addr to be the same")
  29. assert.Equal(height, privVal.LastSignState.Height, "expected privval.LastHeight to have been saved")
  30. }
  31. func TestResetValidator(t *testing.T) {
  32. tempKeyFile, err := ioutil.TempFile("", "priv_validator_key_")
  33. require.Nil(t, err)
  34. tempStateFile, err := ioutil.TempFile("", "priv_validator_state_")
  35. require.Nil(t, err)
  36. privVal := GenFilePV(tempKeyFile.Name(), tempStateFile.Name())
  37. emptyState := FilePVLastSignState{filePath: tempStateFile.Name()}
  38. // new priv val has empty state
  39. assert.Equal(t, privVal.LastSignState, emptyState)
  40. // test vote
  41. height, round := int64(10), int32(1)
  42. voteType := byte(tmproto.PrevoteType)
  43. blockID := types.BlockID{Hash: []byte{1, 2, 3}, PartsHeader: types.PartSetHeader{}}
  44. vote := newVote(privVal.Key.Address, 0, height, round, voteType, blockID)
  45. err = privVal.SignVote("mychainid", vote)
  46. assert.NoError(t, err, "expected no error signing vote")
  47. // priv val after signing is not same as empty
  48. assert.NotEqual(t, privVal.LastSignState, emptyState)
  49. // priv val after AcceptNewConnection is same as empty
  50. privVal.Reset()
  51. assert.Equal(t, privVal.LastSignState, emptyState)
  52. }
  53. func TestLoadOrGenValidator(t *testing.T) {
  54. assert := assert.New(t)
  55. tempKeyFile, err := ioutil.TempFile("", "priv_validator_key_")
  56. require.Nil(t, err)
  57. tempStateFile, err := ioutil.TempFile("", "priv_validator_state_")
  58. require.Nil(t, err)
  59. tempKeyFilePath := tempKeyFile.Name()
  60. if err := os.Remove(tempKeyFilePath); err != nil {
  61. t.Error(err)
  62. }
  63. tempStateFilePath := tempStateFile.Name()
  64. if err := os.Remove(tempStateFilePath); err != nil {
  65. t.Error(err)
  66. }
  67. privVal := LoadOrGenFilePV(tempKeyFilePath, tempStateFilePath)
  68. addr := privVal.GetAddress()
  69. privVal = LoadOrGenFilePV(tempKeyFilePath, tempStateFilePath)
  70. assert.Equal(addr, privVal.GetAddress(), "expected privval addr to be the same")
  71. }
  72. func TestUnmarshalValidatorState(t *testing.T) {
  73. assert, require := assert.New(t), require.New(t)
  74. // create some fixed values
  75. serialized := `{
  76. "height": "1",
  77. "round": 1,
  78. "step": 1
  79. }`
  80. val := FilePVLastSignState{}
  81. err := cdc.UnmarshalJSON([]byte(serialized), &val)
  82. require.Nil(err, "%+v", err)
  83. // make sure the values match
  84. assert.EqualValues(val.Height, 1)
  85. assert.EqualValues(val.Round, 1)
  86. assert.EqualValues(val.Step, 1)
  87. // export it and make sure it is the same
  88. out, err := cdc.MarshalJSON(val)
  89. require.Nil(err, "%+v", err)
  90. assert.JSONEq(serialized, string(out))
  91. }
  92. func TestUnmarshalValidatorKey(t *testing.T) {
  93. assert, require := assert.New(t), require.New(t)
  94. // create some fixed values
  95. privKey := ed25519.GenPrivKey()
  96. pubKey := privKey.PubKey()
  97. addr := pubKey.Address()
  98. pubBytes := []byte(pubKey.(ed25519.PubKey))
  99. privBytes := []byte(privKey)
  100. pubB64 := base64.StdEncoding.EncodeToString(pubBytes)
  101. privB64 := base64.StdEncoding.EncodeToString(privBytes)
  102. serialized := fmt.Sprintf(`{
  103. "address": "%s",
  104. "pub_key": {
  105. "type": "tendermint/PubKeyEd25519",
  106. "value": "%s"
  107. },
  108. "priv_key": {
  109. "type": "tendermint/PrivKeyEd25519",
  110. "value": "%s"
  111. }
  112. }`, addr, pubB64, privB64)
  113. val := FilePVKey{}
  114. err := cdc.UnmarshalJSON([]byte(serialized), &val)
  115. require.Nil(err, "%+v", err)
  116. // make sure the values match
  117. assert.EqualValues(addr, val.Address)
  118. assert.EqualValues(pubKey, val.PubKey)
  119. assert.EqualValues(privKey, val.PrivKey)
  120. // export it and make sure it is the same
  121. out, err := cdc.MarshalJSON(val)
  122. require.Nil(err, "%+v", err)
  123. assert.JSONEq(serialized, string(out))
  124. }
  125. func TestSignVote(t *testing.T) {
  126. assert := assert.New(t)
  127. tempKeyFile, err := ioutil.TempFile("", "priv_validator_key_")
  128. require.Nil(t, err)
  129. tempStateFile, err := ioutil.TempFile("", "priv_validator_state_")
  130. require.Nil(t, err)
  131. privVal := GenFilePV(tempKeyFile.Name(), tempStateFile.Name())
  132. block1 := types.BlockID{Hash: []byte{1, 2, 3}, PartsHeader: types.PartSetHeader{}}
  133. block2 := types.BlockID{Hash: []byte{3, 2, 1}, PartsHeader: types.PartSetHeader{}}
  134. height, round := int64(10), int32(1)
  135. voteType := byte(tmproto.PrevoteType)
  136. // sign a vote for first time
  137. vote := newVote(privVal.Key.Address, 0, height, round, voteType, block1)
  138. err = privVal.SignVote("mychainid", vote)
  139. assert.NoError(err, "expected no error signing vote")
  140. // try to sign the same vote again; should be fine
  141. err = privVal.SignVote("mychainid", vote)
  142. assert.NoError(err, "expected no error on signing same vote")
  143. // now try some bad votes
  144. cases := []*types.Vote{
  145. newVote(privVal.Key.Address, 0, height, round-1, voteType, block1), // round regression
  146. newVote(privVal.Key.Address, 0, height-1, round, voteType, block1), // height regression
  147. newVote(privVal.Key.Address, 0, height-2, round+4, voteType, block1), // height regression and different round
  148. newVote(privVal.Key.Address, 0, height, round, voteType, block2), // different block
  149. }
  150. for _, c := range cases {
  151. err = privVal.SignVote("mychainid", c)
  152. assert.Error(err, "expected error on signing conflicting vote")
  153. }
  154. // try signing a vote with a different time stamp
  155. sig := vote.Signature
  156. vote.Timestamp = vote.Timestamp.Add(time.Duration(1000))
  157. err = privVal.SignVote("mychainid", vote)
  158. assert.NoError(err)
  159. assert.Equal(sig, vote.Signature)
  160. }
  161. func TestSignProposal(t *testing.T) {
  162. assert := assert.New(t)
  163. tempKeyFile, err := ioutil.TempFile("", "priv_validator_key_")
  164. require.Nil(t, err)
  165. tempStateFile, err := ioutil.TempFile("", "priv_validator_state_")
  166. require.Nil(t, err)
  167. privVal := GenFilePV(tempKeyFile.Name(), tempStateFile.Name())
  168. block1 := types.BlockID{Hash: []byte{1, 2, 3}, PartsHeader: types.PartSetHeader{Total: 5, Hash: []byte{1, 2, 3}}}
  169. block2 := types.BlockID{Hash: []byte{3, 2, 1}, PartsHeader: types.PartSetHeader{Total: 10, Hash: []byte{3, 2, 1}}}
  170. height, round := int64(10), int32(1)
  171. // sign a proposal for first time
  172. proposal := newProposal(height, round, block1)
  173. err = privVal.SignProposal("mychainid", proposal)
  174. assert.NoError(err, "expected no error signing proposal")
  175. // try to sign the same proposal again; should be fine
  176. err = privVal.SignProposal("mychainid", proposal)
  177. assert.NoError(err, "expected no error on signing same proposal")
  178. // now try some bad Proposals
  179. cases := []*types.Proposal{
  180. newProposal(height, round-1, block1), // round regression
  181. newProposal(height-1, round, block1), // height regression
  182. newProposal(height-2, round+4, block1), // height regression and different round
  183. newProposal(height, round, block2), // different block
  184. }
  185. for _, c := range cases {
  186. err = privVal.SignProposal("mychainid", c)
  187. assert.Error(err, "expected error on signing conflicting proposal")
  188. }
  189. // try signing a proposal with a different time stamp
  190. sig := proposal.Signature
  191. proposal.Timestamp = proposal.Timestamp.Add(time.Duration(1000))
  192. err = privVal.SignProposal("mychainid", proposal)
  193. assert.NoError(err)
  194. assert.Equal(sig, proposal.Signature)
  195. }
  196. func TestDifferByTimestamp(t *testing.T) {
  197. tempKeyFile, err := ioutil.TempFile("", "priv_validator_key_")
  198. require.Nil(t, err)
  199. tempStateFile, err := ioutil.TempFile("", "priv_validator_state_")
  200. require.Nil(t, err)
  201. privVal := GenFilePV(tempKeyFile.Name(), tempStateFile.Name())
  202. block1 := types.BlockID{Hash: []byte{1, 2, 3}, PartsHeader: types.PartSetHeader{Total: 5, Hash: []byte{1, 2, 3}}}
  203. height, round := int64(10), int32(1)
  204. chainID := "mychainid"
  205. // test proposal
  206. {
  207. proposal := newProposal(height, round, block1)
  208. err := privVal.SignProposal(chainID, proposal)
  209. assert.NoError(t, err, "expected no error signing proposal")
  210. signBytes := proposal.SignBytes(chainID)
  211. sig := proposal.Signature
  212. timeStamp := proposal.Timestamp
  213. // manipulate the timestamp. should get changed back
  214. proposal.Timestamp = proposal.Timestamp.Add(time.Millisecond)
  215. var emptySig []byte
  216. proposal.Signature = emptySig
  217. err = privVal.SignProposal("mychainid", proposal)
  218. assert.NoError(t, err, "expected no error on signing same proposal")
  219. assert.Equal(t, timeStamp, proposal.Timestamp)
  220. assert.Equal(t, signBytes, proposal.SignBytes(chainID))
  221. assert.Equal(t, sig, proposal.Signature)
  222. }
  223. // test vote
  224. {
  225. voteType := byte(tmproto.PrevoteType)
  226. blockID := types.BlockID{Hash: []byte{1, 2, 3}, PartsHeader: types.PartSetHeader{}}
  227. vote := newVote(privVal.Key.Address, 0, height, round, voteType, blockID)
  228. err := privVal.SignVote("mychainid", vote)
  229. assert.NoError(t, err, "expected no error signing vote")
  230. signBytes := vote.SignBytes(chainID)
  231. sig := vote.Signature
  232. timeStamp := vote.Timestamp
  233. // manipulate the timestamp. should get changed back
  234. vote.Timestamp = vote.Timestamp.Add(time.Millisecond)
  235. var emptySig []byte
  236. vote.Signature = emptySig
  237. err = privVal.SignVote("mychainid", vote)
  238. assert.NoError(t, err, "expected no error on signing same vote")
  239. assert.Equal(t, timeStamp, vote.Timestamp)
  240. assert.Equal(t, signBytes, vote.SignBytes(chainID))
  241. assert.Equal(t, sig, vote.Signature)
  242. }
  243. }
  244. func newVote(addr types.Address, idx int32, height int64, round int32, typ byte, blockID types.BlockID) *types.Vote {
  245. return &types.Vote{
  246. ValidatorAddress: addr,
  247. ValidatorIndex: idx,
  248. Height: height,
  249. Round: round,
  250. Type: tmproto.SignedMsgType(typ),
  251. Timestamp: tmtime.Now(),
  252. BlockID: blockID,
  253. }
  254. }
  255. func newProposal(height int64, round int32, blockID types.BlockID) *types.Proposal {
  256. return &types.Proposal{
  257. Height: height,
  258. Round: round,
  259. BlockID: blockID,
  260. Timestamp: tmtime.Now(),
  261. }
  262. }