You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

398 lines
12 KiB

Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: improve Remote Signer implementation (#3351) This issue is related to #3107 This is a first renaming/refactoring step before reworking and removing heartbeats. As discussed with @Liamsi , we preferred to go for a couple of independent and separate PRs to simplify review work. The changes: Help to clarify the relation between the validator and remote signer endpoints Differentiate between timeouts and deadlines Prepare to encapsulate networking related code behind RemoteSigner in the next PR My intention is to separate and encapsulate the "network related" code from the actual signer. SignerRemote ---(uses/contains)--> SignerValidatorEndpoint <--(connects to)--> SignerServiceEndpoint ---> SignerService (future.. not here yet but would like to decouple too) All reconnection/heartbeat/whatever code goes in the endpoints. Signer[Remote/Service] do not need to know about that. I agree Endpoint may not be the perfect name. I tried to find something "Go-ish" enough. It is a common name in go-kit, kubernetes, etc. Right now: SignerValidatorEndpoint: handles the listener contains SignerRemote Implements the PrivValidator interface connects and sets a connection object in a contained SignerRemote delegates PrivValidator some calls to SignerRemote which in turn uses the conn object that was set externally SignerRemote: Implements the PrivValidator interface read/writes from a connection object directly handles heartbeats SignerServiceEndpoint: Does most things in a single place delegates to a PrivValidator IIRC. * cleanup * Refactoring step 1 * Refactoring step 2 * move messages to another file * mark for future work / next steps * mark deprecated classes in docs * Fix linter problems * additional linter fixes
6 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: improve Remote Signer implementation (#3351) This issue is related to #3107 This is a first renaming/refactoring step before reworking and removing heartbeats. As discussed with @Liamsi , we preferred to go for a couple of independent and separate PRs to simplify review work. The changes: Help to clarify the relation between the validator and remote signer endpoints Differentiate between timeouts and deadlines Prepare to encapsulate networking related code behind RemoteSigner in the next PR My intention is to separate and encapsulate the "network related" code from the actual signer. SignerRemote ---(uses/contains)--> SignerValidatorEndpoint <--(connects to)--> SignerServiceEndpoint ---> SignerService (future.. not here yet but would like to decouple too) All reconnection/heartbeat/whatever code goes in the endpoints. Signer[Remote/Service] do not need to know about that. I agree Endpoint may not be the perfect name. I tried to find something "Go-ish" enough. It is a common name in go-kit, kubernetes, etc. Right now: SignerValidatorEndpoint: handles the listener contains SignerRemote Implements the PrivValidator interface connects and sets a connection object in a contained SignerRemote delegates PrivValidator some calls to SignerRemote which in turn uses the conn object that was set externally SignerRemote: Implements the PrivValidator interface read/writes from a connection object directly handles heartbeats SignerServiceEndpoint: Does most things in a single place delegates to a PrivValidator IIRC. * cleanup * Refactoring step 1 * Refactoring step 2 * move messages to another file * mark for future work / next steps * mark deprecated classes in docs * Fix linter problems * additional linter fixes
6 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
Add remote signer test harness (KMS) (#3149) * WIP: Starts adding remote signer test harness This commit adds a new command to Tendermint to allow for us to build a standalone binary to test remote signers such as KMS (https://github.com/tendermint/kms). Right now, all it does is test that the local public key matches the public key reported by the client, and fails at the point where it attempts to get the client to sign a proposal. * Fixes typo * Fixes proposal validation test This commit fixes the proposal validation test as per #3149. It also moves the test harness into its own internal package to isolate its exports from the `privval` package. * Adds vote signing validation * Applying recommendations from #3149 * Adds function descriptions for test harness * Adds ability to ask remote signer to shut down Prior to this commit, the remote signer needs to manually be shut down, which is not ideal for automated testing. This commit allows us to send a poison pill message to the KMS to let it shut down gracefully once testing is done (whether the tests pass or fail). * Adds tests for remote signer test harness This commit makes some minor modifications to a few files to allow for testing of the remote signer test harness. Two tests are added here: checking for a fully successful (the ideal) case, and for the case where the maximum number of retries has been reached when attempting to accept incoming connections from the remote signer. * Condenses serialization of proposals and votes using existing Tendermint functions * Removes now-unnecessary amino import and codec * Adds error message for vote signing failure * Adds key extraction command for integration test Took the code from here: https://gist.github.com/Liamsi/a80993f24bff574bbfdbbfa9efa84bc7 to create a simple utility command to extract a key from a local Tendermint validator for use in KMS integration testing. * Makes path expansion success non-compulsory * Fixes segfault on SIGTERM We need an additional variable to keep track of whether we're successfully connected, otherwise hitting Ctrl+Break during execution causes a segmentation fault. This now allows for a clean shutdown. * Consolidates shutdown checks * Adds comments indicating codes for easy lookup * Adds Docker build for remote signer harness Updates the `DOCKER/build.sh` and `DOCKER/push.sh` files to allow one to override the image name and Dockerfile using environment variables. Updates the primary `Makefile` as well as the `DOCKER/Makefile` to allow for building the `remote_val_harness` Docker image. * Adds build_remote_val_harness_docker_image to .PHONY * Removes remote signer poison pill messaging functionality * Reduces fluff code in command line parsing As per https://github.com/tendermint/tendermint/pull/3149#pullrequestreview-196171788, this reduces the amount of fluff code in the PR down to the bare minimum. * Fixes ordering of error check and info log * Moves remove_val_harness cmd into tools folder It seems to make sense to rather keep the remote signer test harness in its own tool folder (now rather named `tm-signer-harness` to keep with the tool naming convention). It is actually a separate tool, not meant to be one of the core binaries, but supplementary and supportive. * Updates documentation for tm-signer-harness * Refactors flag parsing to be more compact and less redundant * Adds version sub-command help * Removes extraneous flags parsing * Adds CHANGELOG_PENDING entry for tm-signer-harness * Improves test coverage Adds a few extra parameters to the `MockPV` type to fake broken vote and proposal signing. Also adds some more tests for the test harness so as to increase coverage for failed cases. * Fixes formatting for CHANGELOG_PENDING.md * Fix formatting for documentation config * Point users towards official Tendermint docs for tools documentation * Point users towards official Tendermint docs for tm-signer-harness * Remove extraneous constant * Rename TestHarness.sc to TestHarness.spv for naming consistency * Refactor to remove redundant goroutine * Refactor conditional to cleaner switch statement and better error handling for listener protocol * Remove extraneous goroutine * Add note about installing tmkms via Cargo * Fix typo in naming of output signing key * Add note about where to find chain ID * Replace /home/user with ~/ for brevity * Fixes "signer.key" typo * Minor edits for clarification for tm-signer-harness bulid/setup process
6 years ago
  1. package internal
  2. import (
  3. "fmt"
  4. "net"
  5. "os"
  6. "os/signal"
  7. "time"
  8. "github.com/tendermint/tendermint/crypto/tmhash"
  9. "github.com/tendermint/tendermint/crypto/ed25519"
  10. "github.com/tendermint/tendermint/privval"
  11. "github.com/tendermint/tendermint/state"
  12. cmn "github.com/tendermint/tendermint/libs/common"
  13. "github.com/tendermint/tendermint/libs/log"
  14. "github.com/tendermint/tendermint/types"
  15. )
  16. // Test harness error codes (which act as exit codes when the test harness fails).
  17. const (
  18. NoError int = iota // 0
  19. ErrInvalidParameters // 1
  20. ErrMaxAcceptRetriesReached // 2
  21. ErrFailedToLoadGenesisFile // 3
  22. ErrFailedToCreateListener // 4
  23. ErrFailedToStartListener // 5
  24. ErrInterrupted // 6
  25. ErrOther // 7
  26. ErrTestPublicKeyFailed // 8
  27. ErrTestSignProposalFailed // 9
  28. ErrTestSignVoteFailed // 10
  29. )
  30. var voteTypes = []types.SignedMsgType{types.PrevoteType, types.PrecommitType}
  31. // TestHarnessError allows us to keep track of which exit code should be used
  32. // when exiting the main program.
  33. type TestHarnessError struct {
  34. Code int // The exit code to return
  35. Err error // The original error
  36. Info string // Any additional information
  37. }
  38. var _ error = (*TestHarnessError)(nil)
  39. // TestHarness allows for testing of a remote signer to ensure compatibility
  40. // with this version of Tendermint.
  41. type TestHarness struct {
  42. addr string
  43. signerClient *privval.SignerClient
  44. fpv *privval.FilePV
  45. chainID string
  46. acceptRetries int
  47. logger log.Logger
  48. exitWhenComplete bool
  49. exitCode int
  50. }
  51. // TestHarnessConfig provides configuration to set up a remote signer test
  52. // harness.
  53. type TestHarnessConfig struct {
  54. BindAddr string
  55. KeyFile string
  56. StateFile string
  57. GenesisFile string
  58. AcceptDeadline time.Duration
  59. ConnDeadline time.Duration
  60. AcceptRetries int
  61. SecretConnKey ed25519.PrivKeyEd25519
  62. ExitWhenComplete bool // Whether or not to call os.Exit when the harness has completed.
  63. }
  64. // timeoutError can be used to check if an error returned from the netp package
  65. // was due to a timeout.
  66. type timeoutError interface {
  67. Timeout() bool
  68. }
  69. // NewTestHarness will load Tendermint data from the given files (including
  70. // validator public/private keypairs and chain details) and create a new
  71. // harness.
  72. func NewTestHarness(logger log.Logger, cfg TestHarnessConfig) (*TestHarness, error) {
  73. keyFile := ExpandPath(cfg.KeyFile)
  74. stateFile := ExpandPath(cfg.StateFile)
  75. logger.Info("Loading private validator configuration", "keyFile", keyFile, "stateFile", stateFile)
  76. // NOTE: LoadFilePV ultimately calls os.Exit on failure. No error will be
  77. // returned if this call fails.
  78. fpv := privval.LoadFilePV(keyFile, stateFile)
  79. genesisFile := ExpandPath(cfg.GenesisFile)
  80. logger.Info("Loading chain ID from genesis file", "genesisFile", genesisFile)
  81. st, err := state.MakeGenesisDocFromFile(genesisFile)
  82. if err != nil {
  83. return nil, newTestHarnessError(ErrFailedToLoadGenesisFile, err, genesisFile)
  84. }
  85. logger.Info("Loaded genesis file", "chainID", st.ChainID)
  86. spv, err := newTestHarnessListener(logger, cfg)
  87. if err != nil {
  88. return nil, newTestHarnessError(ErrFailedToCreateListener, err, "")
  89. }
  90. signerClient, err := privval.NewSignerClient(spv)
  91. if err != nil {
  92. return nil, newTestHarnessError(ErrFailedToCreateListener, err, "")
  93. }
  94. return &TestHarness{
  95. addr: cfg.BindAddr,
  96. signerClient: signerClient,
  97. fpv: fpv,
  98. chainID: st.ChainID,
  99. acceptRetries: cfg.AcceptRetries,
  100. logger: logger,
  101. exitWhenComplete: cfg.ExitWhenComplete,
  102. exitCode: 0,
  103. }, nil
  104. }
  105. // Run will execute the tests associated with this test harness. The intention
  106. // here is to call this from one's `main` function, as the way it succeeds or
  107. // fails at present is to call os.Exit() with an exit code related to the error
  108. // that caused the tests to fail, or exit code 0 on success.
  109. func (th *TestHarness) Run() {
  110. c := make(chan os.Signal, 1)
  111. signal.Notify(c, os.Interrupt)
  112. go func() {
  113. for sig := range c {
  114. th.logger.Info("Caught interrupt, terminating...", "sig", sig)
  115. th.Shutdown(newTestHarnessError(ErrInterrupted, nil, ""))
  116. }
  117. }()
  118. th.logger.Info("Starting test harness")
  119. accepted := false
  120. var startErr error
  121. for acceptRetries := th.acceptRetries; acceptRetries > 0; acceptRetries-- {
  122. th.logger.Info("Attempting to accept incoming connection", "acceptRetries", acceptRetries)
  123. if err := th.signerClient.WaitForConnection(10 * time.Millisecond); err != nil {
  124. // if it wasn't a timeout error
  125. if _, ok := err.(timeoutError); !ok {
  126. th.logger.Error("Failed to start listener", "err", err)
  127. th.Shutdown(newTestHarnessError(ErrFailedToStartListener, err, ""))
  128. // we need the return statements in case this is being run
  129. // from a unit test - otherwise this function will just die
  130. // when os.Exit is called
  131. return
  132. }
  133. startErr = err
  134. } else {
  135. th.logger.Info("Accepted external connection")
  136. accepted = true
  137. break
  138. }
  139. }
  140. if !accepted {
  141. th.logger.Error("Maximum accept retries reached", "acceptRetries", th.acceptRetries)
  142. th.Shutdown(newTestHarnessError(ErrMaxAcceptRetriesReached, startErr, ""))
  143. return
  144. }
  145. // Run the tests
  146. if err := th.TestPublicKey(); err != nil {
  147. th.Shutdown(err)
  148. return
  149. }
  150. if err := th.TestSignProposal(); err != nil {
  151. th.Shutdown(err)
  152. return
  153. }
  154. if err := th.TestSignVote(); err != nil {
  155. th.Shutdown(err)
  156. return
  157. }
  158. th.logger.Info("SUCCESS! All tests passed.")
  159. th.Shutdown(nil)
  160. }
  161. // TestPublicKey just validates that we can (1) fetch the public key from the
  162. // remote signer, and (2) it matches the public key we've configured for our
  163. // local Tendermint version.
  164. func (th *TestHarness) TestPublicKey() error {
  165. th.logger.Info("TEST: Public key of remote signer")
  166. th.logger.Info("Local", "pubKey", th.fpv.GetPubKey())
  167. th.logger.Info("Remote", "pubKey", th.signerClient.GetPubKey())
  168. if th.fpv.GetPubKey() != th.signerClient.GetPubKey() {
  169. th.logger.Error("FAILED: Local and remote public keys do not match")
  170. return newTestHarnessError(ErrTestPublicKeyFailed, nil, "")
  171. }
  172. return nil
  173. }
  174. // TestSignProposal makes sure the remote signer can successfully sign
  175. // proposals.
  176. func (th *TestHarness) TestSignProposal() error {
  177. th.logger.Info("TEST: Signing of proposals")
  178. // sha256 hash of "hash"
  179. hash := tmhash.Sum([]byte("hash"))
  180. prop := &types.Proposal{
  181. Type: types.ProposalType,
  182. Height: 100,
  183. Round: 0,
  184. POLRound: -1,
  185. BlockID: types.BlockID{
  186. Hash: hash,
  187. PartsHeader: types.PartSetHeader{
  188. Hash: hash,
  189. Total: 1000000,
  190. },
  191. },
  192. Timestamp: time.Now(),
  193. }
  194. propBytes := prop.SignBytes(th.chainID)
  195. if err := th.signerClient.SignProposal(th.chainID, prop); err != nil {
  196. th.logger.Error("FAILED: Signing of proposal", "err", err)
  197. return newTestHarnessError(ErrTestSignProposalFailed, err, "")
  198. }
  199. th.logger.Debug("Signed proposal", "prop", prop)
  200. // first check that it's a basically valid proposal
  201. if err := prop.ValidateBasic(); err != nil {
  202. th.logger.Error("FAILED: Signed proposal is invalid", "err", err)
  203. return newTestHarnessError(ErrTestSignProposalFailed, err, "")
  204. }
  205. // now validate the signature on the proposal
  206. if th.signerClient.GetPubKey().VerifyBytes(propBytes, prop.Signature) {
  207. th.logger.Info("Successfully validated proposal signature")
  208. } else {
  209. th.logger.Error("FAILED: Proposal signature validation failed")
  210. return newTestHarnessError(ErrTestSignProposalFailed, nil, "signature validation failed")
  211. }
  212. return nil
  213. }
  214. // TestSignVote makes sure the remote signer can successfully sign all kinds of
  215. // votes.
  216. func (th *TestHarness) TestSignVote() error {
  217. th.logger.Info("TEST: Signing of votes")
  218. for _, voteType := range voteTypes {
  219. th.logger.Info("Testing vote type", "type", voteType)
  220. hash := tmhash.Sum([]byte("hash"))
  221. vote := &types.Vote{
  222. Type: voteType,
  223. Height: 101,
  224. Round: 0,
  225. BlockID: types.BlockID{
  226. Hash: hash,
  227. PartsHeader: types.PartSetHeader{
  228. Hash: hash,
  229. Total: 1000000,
  230. },
  231. },
  232. ValidatorIndex: 0,
  233. ValidatorAddress: tmhash.SumTruncated([]byte("addr")),
  234. Timestamp: time.Now(),
  235. }
  236. voteBytes := vote.SignBytes(th.chainID)
  237. // sign the vote
  238. if err := th.signerClient.SignVote(th.chainID, vote); err != nil {
  239. th.logger.Error("FAILED: Signing of vote", "err", err)
  240. return newTestHarnessError(ErrTestSignVoteFailed, err, fmt.Sprintf("voteType=%d", voteType))
  241. }
  242. th.logger.Debug("Signed vote", "vote", vote)
  243. // validate the contents of the vote
  244. if err := vote.ValidateBasic(); err != nil {
  245. th.logger.Error("FAILED: Signed vote is invalid", "err", err)
  246. return newTestHarnessError(ErrTestSignVoteFailed, err, fmt.Sprintf("voteType=%d", voteType))
  247. }
  248. // now validate the signature on the proposal
  249. if th.signerClient.GetPubKey().VerifyBytes(voteBytes, vote.Signature) {
  250. th.logger.Info("Successfully validated vote signature", "type", voteType)
  251. } else {
  252. th.logger.Error("FAILED: Vote signature validation failed", "type", voteType)
  253. return newTestHarnessError(ErrTestSignVoteFailed, nil, "signature validation failed")
  254. }
  255. }
  256. return nil
  257. }
  258. // Shutdown will kill the test harness and attempt to close all open sockets
  259. // gracefully. If the supplied error is nil, it is assumed that the exit code
  260. // should be 0. If err is not nil, it will exit with an exit code related to the
  261. // error.
  262. func (th *TestHarness) Shutdown(err error) {
  263. var exitCode int
  264. if err == nil {
  265. exitCode = NoError
  266. } else if therr, ok := err.(*TestHarnessError); ok {
  267. exitCode = therr.Code
  268. } else {
  269. exitCode = ErrOther
  270. }
  271. th.exitCode = exitCode
  272. // in case sc.Stop() takes too long
  273. if th.exitWhenComplete {
  274. go func() {
  275. time.Sleep(time.Duration(5) * time.Second)
  276. th.logger.Error("Forcibly exiting program after timeout")
  277. os.Exit(exitCode)
  278. }()
  279. }
  280. err = th.signerClient.Close()
  281. if err != nil {
  282. th.logger.Error("Failed to cleanly stop listener: %s", err.Error())
  283. }
  284. if th.exitWhenComplete {
  285. os.Exit(exitCode)
  286. }
  287. }
  288. // newTestHarnessListener creates our client instance which we will use for testing.
  289. func newTestHarnessListener(logger log.Logger, cfg TestHarnessConfig) (*privval.SignerListenerEndpoint, error) {
  290. proto, addr := cmn.ProtocolAndAddress(cfg.BindAddr)
  291. if proto == "unix" {
  292. // make sure the socket doesn't exist - if so, try to delete it
  293. if cmn.FileExists(addr) {
  294. if err := os.Remove(addr); err != nil {
  295. logger.Error("Failed to remove existing Unix domain socket", "addr", addr)
  296. return nil, err
  297. }
  298. }
  299. }
  300. ln, err := net.Listen(proto, addr)
  301. if err != nil {
  302. return nil, err
  303. }
  304. logger.Info("Listening", "proto", proto, "addr", addr)
  305. var svln net.Listener
  306. switch proto {
  307. case "unix":
  308. unixLn := privval.NewUnixListener(ln)
  309. privval.UnixListenerTimeoutAccept(cfg.AcceptDeadline)(unixLn)
  310. privval.UnixListenerTimeoutReadWrite(cfg.ConnDeadline)(unixLn)
  311. svln = unixLn
  312. case "tcp":
  313. tcpLn := privval.NewTCPListener(ln, cfg.SecretConnKey)
  314. privval.TCPListenerTimeoutAccept(cfg.AcceptDeadline)(tcpLn)
  315. privval.TCPListenerTimeoutReadWrite(cfg.ConnDeadline)(tcpLn)
  316. logger.Info("Resolved TCP address for listener", "addr", tcpLn.Addr())
  317. svln = tcpLn
  318. default:
  319. logger.Error("Unsupported protocol (must be unix:// or tcp://)", "proto", proto)
  320. return nil, newTestHarnessError(ErrInvalidParameters, nil, fmt.Sprintf("Unsupported protocol: %s", proto))
  321. }
  322. return privval.NewSignerListenerEndpoint(logger, svln), nil
  323. }
  324. func newTestHarnessError(code int, err error, info string) *TestHarnessError {
  325. return &TestHarnessError{
  326. Code: code,
  327. Err: err,
  328. Info: info,
  329. }
  330. }
  331. func (e *TestHarnessError) Error() string {
  332. var msg string
  333. switch e.Code {
  334. case ErrInvalidParameters:
  335. msg = "Invalid parameters supplied to application"
  336. case ErrMaxAcceptRetriesReached:
  337. msg = "Maximum accept retries reached"
  338. case ErrFailedToLoadGenesisFile:
  339. msg = "Failed to load genesis file"
  340. case ErrFailedToCreateListener:
  341. msg = "Failed to create listener"
  342. case ErrFailedToStartListener:
  343. msg = "Failed to start listener"
  344. case ErrInterrupted:
  345. msg = "Interrupted"
  346. case ErrTestPublicKeyFailed:
  347. msg = "Public key validation test failed"
  348. case ErrTestSignProposalFailed:
  349. msg = "Proposal signing validation test failed"
  350. case ErrTestSignVoteFailed:
  351. msg = "Vote signing validation test failed"
  352. default:
  353. msg = "Unknown error"
  354. }
  355. if len(e.Info) > 0 {
  356. msg = fmt.Sprintf("%s: %s", msg, e.Info)
  357. }
  358. if e.Err != nil {
  359. msg = fmt.Sprintf("%s (original error: %s)", msg, e.Err.Error())
  360. }
  361. return msg
  362. }