You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

343 lines
11 KiB

privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
  1. package privval
  2. import (
  3. "encoding/base64"
  4. "fmt"
  5. "io/ioutil"
  6. "os"
  7. "testing"
  8. "time"
  9. "github.com/stretchr/testify/assert"
  10. "github.com/stretchr/testify/require"
  11. "github.com/tendermint/tendermint/crypto/ed25519"
  12. "github.com/tendermint/tendermint/crypto/tmhash"
  13. tmjson "github.com/tendermint/tendermint/libs/json"
  14. tmrand "github.com/tendermint/tendermint/libs/rand"
  15. tmproto "github.com/tendermint/tendermint/proto/types"
  16. "github.com/tendermint/tendermint/types"
  17. tmtime "github.com/tendermint/tendermint/types/time"
  18. )
  19. func TestGenLoadValidator(t *testing.T) {
  20. assert := assert.New(t)
  21. tempKeyFile, err := ioutil.TempFile("", "priv_validator_key_")
  22. require.Nil(t, err)
  23. tempStateFile, err := ioutil.TempFile("", "priv_validator_state_")
  24. require.Nil(t, err)
  25. privVal := GenFilePV(tempKeyFile.Name(), tempStateFile.Name())
  26. height := int64(100)
  27. privVal.LastSignState.Height = height
  28. privVal.Save()
  29. addr := privVal.GetAddress()
  30. privVal = LoadFilePV(tempKeyFile.Name(), tempStateFile.Name())
  31. assert.Equal(addr, privVal.GetAddress(), "expected privval addr to be the same")
  32. assert.Equal(height, privVal.LastSignState.Height, "expected privval.LastHeight to have been saved")
  33. }
  34. func TestResetValidator(t *testing.T) {
  35. tempKeyFile, err := ioutil.TempFile("", "priv_validator_key_")
  36. require.Nil(t, err)
  37. tempStateFile, err := ioutil.TempFile("", "priv_validator_state_")
  38. require.Nil(t, err)
  39. privVal := GenFilePV(tempKeyFile.Name(), tempStateFile.Name())
  40. emptyState := FilePVLastSignState{filePath: tempStateFile.Name()}
  41. // new priv val has empty state
  42. assert.Equal(t, privVal.LastSignState, emptyState)
  43. // test vote
  44. height, round := int64(10), int32(1)
  45. voteType := tmproto.PrevoteType
  46. randBytes := tmrand.Bytes(tmhash.Size)
  47. blockID := types.BlockID{Hash: randBytes, PartSetHeader: types.PartSetHeader{}}
  48. vote := newVote(privVal.Key.Address, 0, height, round, voteType, blockID)
  49. err = privVal.SignVote("mychainid", vote.ToProto())
  50. assert.NoError(t, err, "expected no error signing vote")
  51. // priv val after signing is not same as empty
  52. assert.NotEqual(t, privVal.LastSignState, emptyState)
  53. // priv val after AcceptNewConnection is same as empty
  54. privVal.Reset()
  55. assert.Equal(t, privVal.LastSignState, emptyState)
  56. }
  57. func TestLoadOrGenValidator(t *testing.T) {
  58. assert := assert.New(t)
  59. tempKeyFile, err := ioutil.TempFile("", "priv_validator_key_")
  60. require.Nil(t, err)
  61. tempStateFile, err := ioutil.TempFile("", "priv_validator_state_")
  62. require.Nil(t, err)
  63. tempKeyFilePath := tempKeyFile.Name()
  64. if err := os.Remove(tempKeyFilePath); err != nil {
  65. t.Error(err)
  66. }
  67. tempStateFilePath := tempStateFile.Name()
  68. if err := os.Remove(tempStateFilePath); err != nil {
  69. t.Error(err)
  70. }
  71. privVal := LoadOrGenFilePV(tempKeyFilePath, tempStateFilePath)
  72. addr := privVal.GetAddress()
  73. privVal = LoadOrGenFilePV(tempKeyFilePath, tempStateFilePath)
  74. assert.Equal(addr, privVal.GetAddress(), "expected privval addr to be the same")
  75. }
  76. func TestUnmarshalValidatorState(t *testing.T) {
  77. assert, require := assert.New(t), require.New(t)
  78. // create some fixed values
  79. serialized := `{
  80. "height": "1",
  81. "round": 1,
  82. "step": 1
  83. }`
  84. val := FilePVLastSignState{}
  85. err := tmjson.Unmarshal([]byte(serialized), &val)
  86. require.Nil(err, "%+v", err)
  87. // make sure the values match
  88. assert.EqualValues(val.Height, 1)
  89. assert.EqualValues(val.Round, 1)
  90. assert.EqualValues(val.Step, 1)
  91. // export it and make sure it is the same
  92. out, err := tmjson.Marshal(val)
  93. require.Nil(err, "%+v", err)
  94. assert.JSONEq(serialized, string(out))
  95. }
  96. func TestUnmarshalValidatorKey(t *testing.T) {
  97. assert, require := assert.New(t), require.New(t)
  98. // create some fixed values
  99. privKey := ed25519.GenPrivKey()
  100. pubKey := privKey.PubKey()
  101. addr := pubKey.Address()
  102. pubBytes := pubKey.Bytes()
  103. privBytes := privKey.Bytes()
  104. pubB64 := base64.StdEncoding.EncodeToString(pubBytes)
  105. privB64 := base64.StdEncoding.EncodeToString(privBytes)
  106. serialized := fmt.Sprintf(`{
  107. "address": "%s",
  108. "pub_key": {
  109. "type": "tendermint/PubKeyEd25519",
  110. "value": "%s"
  111. },
  112. "priv_key": {
  113. "type": "tendermint/PrivKeyEd25519",
  114. "value": "%s"
  115. }
  116. }`, addr, pubB64, privB64)
  117. val := FilePVKey{}
  118. err := tmjson.Unmarshal([]byte(serialized), &val)
  119. require.Nil(err, "%+v", err)
  120. // make sure the values match
  121. assert.EqualValues(addr, val.Address)
  122. assert.EqualValues(pubKey, val.PubKey)
  123. assert.EqualValues(privKey, val.PrivKey)
  124. // export it and make sure it is the same
  125. out, err := tmjson.Marshal(val)
  126. require.Nil(err, "%+v", err)
  127. assert.JSONEq(serialized, string(out))
  128. }
  129. func TestSignVote(t *testing.T) {
  130. assert := assert.New(t)
  131. tempKeyFile, err := ioutil.TempFile("", "priv_validator_key_")
  132. require.Nil(t, err)
  133. tempStateFile, err := ioutil.TempFile("", "priv_validator_state_")
  134. require.Nil(t, err)
  135. privVal := GenFilePV(tempKeyFile.Name(), tempStateFile.Name())
  136. randbytes := tmrand.Bytes(tmhash.Size)
  137. randbytes2 := tmrand.Bytes(tmhash.Size)
  138. block1 := types.BlockID{Hash: randbytes,
  139. PartSetHeader: types.PartSetHeader{Total: 5, Hash: randbytes}}
  140. block2 := types.BlockID{Hash: randbytes2,
  141. PartSetHeader: types.PartSetHeader{Total: 10, Hash: randbytes2}}
  142. height, round := int64(10), int32(1)
  143. voteType := tmproto.PrevoteType
  144. // sign a vote for first time
  145. vote := newVote(privVal.Key.Address, 0, height, round, voteType, block1)
  146. v := vote.ToProto()
  147. err = privVal.SignVote("mychainid", v)
  148. assert.NoError(err, "expected no error signing vote")
  149. // try to sign the same vote again; should be fine
  150. err = privVal.SignVote("mychainid", v)
  151. assert.NoError(err, "expected no error on signing same vote")
  152. // now try some bad votes
  153. cases := []*types.Vote{
  154. newVote(privVal.Key.Address, 0, height, round-1, voteType, block1), // round regression
  155. newVote(privVal.Key.Address, 0, height-1, round, voteType, block1), // height regression
  156. newVote(privVal.Key.Address, 0, height-2, round+4, voteType, block1), // height regression and different round
  157. newVote(privVal.Key.Address, 0, height, round, voteType, block2), // different block
  158. }
  159. for _, c := range cases {
  160. cpb := c.ToProto()
  161. err = privVal.SignVote("mychainid", cpb)
  162. assert.Error(err, "expected error on signing conflicting vote")
  163. }
  164. // try signing a vote with a different time stamp
  165. sig := vote.Signature
  166. vote.Timestamp = vote.Timestamp.Add(time.Duration(1000))
  167. err = privVal.SignVote("mychainid", v)
  168. assert.NoError(err)
  169. assert.Equal(sig, vote.Signature)
  170. }
  171. func TestSignProposal(t *testing.T) {
  172. assert := assert.New(t)
  173. tempKeyFile, err := ioutil.TempFile("", "priv_validator_key_")
  174. require.Nil(t, err)
  175. tempStateFile, err := ioutil.TempFile("", "priv_validator_state_")
  176. require.Nil(t, err)
  177. privVal := GenFilePV(tempKeyFile.Name(), tempStateFile.Name())
  178. randbytes := tmrand.Bytes(tmhash.Size)
  179. randbytes2 := tmrand.Bytes(tmhash.Size)
  180. block1 := types.BlockID{Hash: randbytes,
  181. PartSetHeader: types.PartSetHeader{Total: 5, Hash: randbytes}}
  182. block2 := types.BlockID{Hash: randbytes2,
  183. PartSetHeader: types.PartSetHeader{Total: 10, Hash: randbytes2}}
  184. height, round := int64(10), int32(1)
  185. // sign a proposal for first time
  186. proposal := newProposal(height, round, block1)
  187. pbp := proposal.ToProto()
  188. err = privVal.SignProposal("mychainid", pbp)
  189. assert.NoError(err, "expected no error signing proposal")
  190. // try to sign the same proposal again; should be fine
  191. err = privVal.SignProposal("mychainid", pbp)
  192. assert.NoError(err, "expected no error on signing same proposal")
  193. // now try some bad Proposals
  194. cases := []*types.Proposal{
  195. newProposal(height, round-1, block1), // round regression
  196. newProposal(height-1, round, block1), // height regression
  197. newProposal(height-2, round+4, block1), // height regression and different round
  198. newProposal(height, round, block2), // different block
  199. }
  200. for _, c := range cases {
  201. err = privVal.SignProposal("mychainid", c.ToProto())
  202. assert.Error(err, "expected error on signing conflicting proposal")
  203. }
  204. // try signing a proposal with a different time stamp
  205. sig := proposal.Signature
  206. proposal.Timestamp = proposal.Timestamp.Add(time.Duration(1000))
  207. err = privVal.SignProposal("mychainid", pbp)
  208. assert.NoError(err)
  209. assert.Equal(sig, proposal.Signature)
  210. }
  211. func TestDifferByTimestamp(t *testing.T) {
  212. tempKeyFile, err := ioutil.TempFile("", "priv_validator_key_")
  213. require.Nil(t, err)
  214. tempStateFile, err := ioutil.TempFile("", "priv_validator_state_")
  215. require.Nil(t, err)
  216. privVal := GenFilePV(tempKeyFile.Name(), tempStateFile.Name())
  217. randbytes := tmrand.Bytes(tmhash.Size)
  218. block1 := types.BlockID{Hash: randbytes, PartSetHeader: types.PartSetHeader{Total: 5, Hash: randbytes}}
  219. height, round := int64(10), int32(1)
  220. chainID := "mychainid"
  221. // test proposal
  222. {
  223. proposal := newProposal(height, round, block1)
  224. pb := proposal.ToProto()
  225. err := privVal.SignProposal(chainID, pb)
  226. assert.NoError(t, err, "expected no error signing proposal")
  227. signBytes := types.ProposalSignBytes(chainID, pb)
  228. sig := proposal.Signature
  229. timeStamp := proposal.Timestamp
  230. // manipulate the timestamp. should get changed back
  231. pb.Timestamp = pb.Timestamp.Add(time.Millisecond)
  232. var emptySig []byte
  233. proposal.Signature = emptySig
  234. err = privVal.SignProposal("mychainid", pb)
  235. assert.NoError(t, err, "expected no error on signing same proposal")
  236. assert.Equal(t, timeStamp, pb.Timestamp)
  237. assert.Equal(t, signBytes, types.ProposalSignBytes(chainID, pb))
  238. assert.Equal(t, sig, proposal.Signature)
  239. }
  240. // test vote
  241. {
  242. voteType := tmproto.PrevoteType
  243. blockID := types.BlockID{Hash: randbytes, PartSetHeader: types.PartSetHeader{}}
  244. vote := newVote(privVal.Key.Address, 0, height, round, voteType, blockID)
  245. v := vote.ToProto()
  246. err := privVal.SignVote("mychainid", v)
  247. assert.NoError(t, err, "expected no error signing vote")
  248. signBytes := types.VoteSignBytes(chainID, v)
  249. sig := v.Signature
  250. timeStamp := vote.Timestamp
  251. // manipulate the timestamp. should get changed back
  252. v.Timestamp = v.Timestamp.Add(time.Millisecond)
  253. var emptySig []byte
  254. v.Signature = emptySig
  255. err = privVal.SignVote("mychainid", v)
  256. assert.NoError(t, err, "expected no error on signing same vote")
  257. assert.Equal(t, timeStamp, v.Timestamp)
  258. assert.Equal(t, signBytes, types.VoteSignBytes(chainID, v))
  259. assert.Equal(t, sig, v.Signature)
  260. }
  261. }
  262. func newVote(addr types.Address, idx int32, height int64, round int32,
  263. typ tmproto.SignedMsgType, blockID types.BlockID) *types.Vote {
  264. return &types.Vote{
  265. ValidatorAddress: addr,
  266. ValidatorIndex: idx,
  267. Height: height,
  268. Round: round,
  269. Type: typ,
  270. Timestamp: tmtime.Now(),
  271. BlockID: blockID,
  272. }
  273. }
  274. func newProposal(height int64, round int32, blockID types.BlockID) *types.Proposal {
  275. return &types.Proposal{
  276. Height: height,
  277. Round: round,
  278. BlockID: blockID,
  279. Timestamp: tmtime.Now(),
  280. }
  281. }