You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

709 lines
19 KiB

cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
fix non deterministic test failures and race in privval socket (#3258) * node: decrease retry conn timeout in test Should fix #3256 The retry timeout was set to the default, which is the same as the accept timeout, so it's no wonder this would fail. Here we decrease the retry timeout so we can try many times before the accept timeout. * p2p: increase handshake timeout in test This fails sometimes, presumably because the handshake timeout is so low (only 50ms). So increase it to 1s. Should fix #3187 * privval: fix race with ping. closes #3237 Pings happen in a go-routine and can happen concurrently with other messages. Since we use a request/response protocol, we expect to send a request and get back the corresponding response. But with pings happening concurrently, this assumption could be violated. We were using a mutex, but only a RWMutex, where the RLock was being held for sending messages - this was to allow the underlying connection to be replaced if it fails. Turns out we actually need to use a full lock (not just a read lock) to prevent multiple requests from happening concurrently. * node: fix test name. DelayedStop -> DelayedStart * autofile: Wait() method In the TestWALTruncate in consensus/wal_test.go we remove the WAL directory at the end of the test. However the wal.Stop() does not properly wait for the autofile group to finish shutting down. Hence it was possible that the group's go-routine is still running when the cleanup happens, which causes a panic since the directory disappeared. Here we add a Wait() method to properly wait until the go-routine exits so we can safely clean up. This fixes #2852.
6 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
fix non deterministic test failures and race in privval socket (#3258) * node: decrease retry conn timeout in test Should fix #3256 The retry timeout was set to the default, which is the same as the accept timeout, so it's no wonder this would fail. Here we decrease the retry timeout so we can try many times before the accept timeout. * p2p: increase handshake timeout in test This fails sometimes, presumably because the handshake timeout is so low (only 50ms). So increase it to 1s. Should fix #3187 * privval: fix race with ping. closes #3237 Pings happen in a go-routine and can happen concurrently with other messages. Since we use a request/response protocol, we expect to send a request and get back the corresponding response. But with pings happening concurrently, this assumption could be violated. We were using a mutex, but only a RWMutex, where the RLock was being held for sending messages - this was to allow the underlying connection to be replaced if it fails. Turns out we actually need to use a full lock (not just a read lock) to prevent multiple requests from happening concurrently. * node: fix test name. DelayedStop -> DelayedStart * autofile: Wait() method In the TestWALTruncate in consensus/wal_test.go we remove the WAL directory at the end of the test. However the wal.Stop() does not properly wait for the autofile group to finish shutting down. Hence it was possible that the group's go-routine is still running when the cleanup happens, which causes a panic since the directory disappeared. Here we add a Wait() method to properly wait until the go-routine exits so we can safely clean up. This fixes #2852.
6 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
Close and retry a RemoteSigner on err (#2923) * Close and recreate a RemoteSigner on err * Update changelog * Address Anton's comments / suggestions: - update changelog - restart TCPVal - shut down on `ErrUnexpectedResponse` * re-init remote signer client with fresh connection if Ping fails - add/update TODOs in secret connection - rename tcp.go -> tcp_client.go, same with ipc to clarify their purpose * account for `conn returned by waitConnection can be `nil` - also add TODO about RemoteSigner conn field * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn - add rwmutex for conn field in IPC * comments and doc.go * fix ipc tests. fixes #2677 * use constants for tests * cleanup some error statements * fixes #2784, race in tests * remove print statement * minor fixes from review * update comment on sts spec * cosmetics * p2p/conn: add failing tests * p2p/conn: make SecretConnection thread safe * changelog * IPCVal signer refactor - use a .reset() method - don't use embedded RemoteSignerClient - guard RemoteSignerClient with mutex - drop the .conn - expose Close() on RemoteSignerClient * apply IPCVal refactor to TCPVal * remove mtx from RemoteSignerClient * consolidate IPCVal and TCPVal, fixes #3104 - done in tcp_client.go - now called SocketVal - takes a listener in the constructor - make tcpListener and unixListener contain all the differences * delete ipc files * introduce unix and tcp dialer for RemoteSigner * rename files - drop tcp_ prefix - rename priv_validator.go to file.go * bring back listener options * fix node * fix priv_val_server * fix node test * minor cleanup and comments
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
Close and retry a RemoteSigner on err (#2923) * Close and recreate a RemoteSigner on err * Update changelog * Address Anton's comments / suggestions: - update changelog - restart TCPVal - shut down on `ErrUnexpectedResponse` * re-init remote signer client with fresh connection if Ping fails - add/update TODOs in secret connection - rename tcp.go -> tcp_client.go, same with ipc to clarify their purpose * account for `conn returned by waitConnection can be `nil` - also add TODO about RemoteSigner conn field * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn - add rwmutex for conn field in IPC * comments and doc.go * fix ipc tests. fixes #2677 * use constants for tests * cleanup some error statements * fixes #2784, race in tests * remove print statement * minor fixes from review * update comment on sts spec * cosmetics * p2p/conn: add failing tests * p2p/conn: make SecretConnection thread safe * changelog * IPCVal signer refactor - use a .reset() method - don't use embedded RemoteSignerClient - guard RemoteSignerClient with mutex - drop the .conn - expose Close() on RemoteSignerClient * apply IPCVal refactor to TCPVal * remove mtx from RemoteSignerClient * consolidate IPCVal and TCPVal, fixes #3104 - done in tcp_client.go - now called SocketVal - takes a listener in the constructor - make tcpListener and unixListener contain all the differences * delete ipc files * introduce unix and tcp dialer for RemoteSigner * rename files - drop tcp_ prefix - rename priv_validator.go to file.go * bring back listener options * fix node * fix priv_val_server * fix node test * minor cleanup and comments
6 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
Close and retry a RemoteSigner on err (#2923) * Close and recreate a RemoteSigner on err * Update changelog * Address Anton's comments / suggestions: - update changelog - restart TCPVal - shut down on `ErrUnexpectedResponse` * re-init remote signer client with fresh connection if Ping fails - add/update TODOs in secret connection - rename tcp.go -> tcp_client.go, same with ipc to clarify their purpose * account for `conn returned by waitConnection can be `nil` - also add TODO about RemoteSigner conn field * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn * Tests for retrying: IPC / TCP - shorter info log on success - set conn and use it in tests to close conn - add rwmutex for conn field in IPC * comments and doc.go * fix ipc tests. fixes #2677 * use constants for tests * cleanup some error statements * fixes #2784, race in tests * remove print statement * minor fixes from review * update comment on sts spec * cosmetics * p2p/conn: add failing tests * p2p/conn: make SecretConnection thread safe * changelog * IPCVal signer refactor - use a .reset() method - don't use embedded RemoteSignerClient - guard RemoteSignerClient with mutex - drop the .conn - expose Close() on RemoteSignerClient * apply IPCVal refactor to TCPVal * remove mtx from RemoteSignerClient * consolidate IPCVal and TCPVal, fixes #3104 - done in tcp_client.go - now called SocketVal - takes a listener in the constructor - make tcpListener and unixListener contain all the differences * delete ipc files * introduce unix and tcp dialer for RemoteSigner * rename files - drop tcp_ prefix - rename priv_validator.go to file.go * bring back listener options * fix node * fix priv_val_server * fix node test * minor cleanup and comments
6 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
privval: refactor Remote signers (#3370) This PR is related to #3107 and a continuation of #3351 It is important to emphasise that in the privval original design, client/server and listening/dialing roles are inverted and do not follow a conventional interaction. Given two hosts A and B: Host A is listener/client Host B is dialer/server (contains the secret key) When A requires a signature, it needs to wait for B to dial in before it can issue a request. A only accepts a single connection and any failure leads to dropping the connection and waiting for B to reconnect. The original rationale behind this design was based on security. Host B only allows outbound connections to a list of whitelisted hosts. It is not possible to reach B unless B dials in. There are no listening/open ports in B. This PR results in the following changes: Refactors ping/heartbeat to avoid previously existing race conditions. Separates transport (dialer/listener) from signing (client/server) concerns to simplify workflow. Unifies and abstracts away the differences between unix and tcp sockets. A single signer endpoint implementation unifies connection handling code (read/write/close/connection obj) The signer request handler (server side) is customizable to increase testability. Updates and extends unit tests A high level overview of the classes is as follows: Transport (endpoints): The following classes take care of establishing a connection SignerDialerEndpoint SignerListeningEndpoint SignerEndpoint groups common functionality (read/write/timeouts/etc.) Signing (client/server): The following classes take care of exchanging request/responses SignerClient SignerServer This PR also closes #3601 Commits: * refactoring - work in progress * reworking unit tests * Encapsulating and fixing unit tests * Improve tests * Clean up * Fix/improve unit tests * clean up tests * Improving service endpoint * fixing unit test * fix linter issues * avoid invalid cache values (improve later?) * complete implementation * wip * improved connection loop * Improve reconnections + fixing unit tests * addressing comments * small formatting changes * clean up * Update node/node.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * check during initialization * dropping connecting when writing fails * removing break * use t.log instead * unifying and using cmn.GetFreePort() * review fixes * reordering and unifying drop connection * closing instead of signalling * refactored service loop * removed superfluous brackets * GetPubKey can return errors * Revert "GetPubKey can return errors" This reverts commit 68c06f19b4650389d7e5ab1659b318889028202c. * adding entry to changelog * Update CHANGELOG_PENDING.md Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_client.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_dialer_endpoint.go Co-Authored-By: jleni <juan.leni@zondax.ch> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: jleni <juan.leni@zondax.ch> * updating node.go * review fixes * fixes linter * fixing unit test * small fixes in comments * addressing review comments * addressing review comments 2 * reverting suggestion * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_client_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update privval/signer_listener_endpoint_test.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * do not expose brokenSignerDialerEndpoint * clean up logging * unifying methods shorten test time signer also drops * reenabling pings * improving testability + unit test * fixing go fmt + unit test * remove unused code * Addressing review comments * simplifying connection workflow * fix linter/go import issue * using base service quit * updating comment * Simplifying design + adjusting names * fixing linter issues * refactoring test harness + fixes * Addressing review comments * cleaning up * adding additional error check
5 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
cleanup: Reduce and normalize import path aliasing. (#6975) The code in the Tendermint repository makes heavy use of import aliasing. This is made necessary by our extensive reuse of common base package names, and by repetition of similar names across different subdirectories. Unfortunately we have not been very consistent about which packages we alias in various circumstances, and the aliases we use vary. In the spirit of the advice in the style guide and https://github.com/golang/go/wiki/CodeReviewComments#imports, his change makes an effort to clean up and normalize import aliasing. This change makes no API or behavioral changes. It is a pure cleanup intended o help make the code more readable to developers (including myself) trying to understand what is being imported where. Only unexported names have been modified, and the changes were generated and applied mechanically with gofmt -r and comby, respecting the lexical and syntactic rules of Go. Even so, I did not fix every inconsistency. Where the changes would be too disruptive, I left it alone. The principles I followed in this cleanup are: - Remove aliases that restate the package name. - Remove aliases where the base package name is unambiguous. - Move overly-terse abbreviations from the import to the usage site. - Fix lexical issues (remove underscores, remove capitalization). - Fix import groupings to more closely match the style guide. - Group blank (side-effecting) imports and ensure they are commented. - Add aliases to multiple imports with the same base package name.
3 years ago
  1. package node
  2. import (
  3. "context"
  4. "errors"
  5. "fmt"
  6. "math"
  7. "net"
  8. "os"
  9. "testing"
  10. "time"
  11. "github.com/stretchr/testify/assert"
  12. "github.com/stretchr/testify/require"
  13. dbm "github.com/tendermint/tm-db"
  14. abciclient "github.com/tendermint/tendermint/abci/client"
  15. "github.com/tendermint/tendermint/abci/example/kvstore"
  16. "github.com/tendermint/tendermint/config"
  17. "github.com/tendermint/tendermint/crypto"
  18. "github.com/tendermint/tendermint/crypto/ed25519"
  19. "github.com/tendermint/tendermint/crypto/tmhash"
  20. "github.com/tendermint/tendermint/internal/eventbus"
  21. "github.com/tendermint/tendermint/internal/evidence"
  22. "github.com/tendermint/tendermint/internal/mempool"
  23. "github.com/tendermint/tendermint/internal/proxy"
  24. "github.com/tendermint/tendermint/internal/pubsub"
  25. sm "github.com/tendermint/tendermint/internal/state"
  26. "github.com/tendermint/tendermint/internal/state/indexer"
  27. "github.com/tendermint/tendermint/internal/store"
  28. "github.com/tendermint/tendermint/internal/test/factory"
  29. "github.com/tendermint/tendermint/libs/log"
  30. tmrand "github.com/tendermint/tendermint/libs/rand"
  31. "github.com/tendermint/tendermint/libs/service"
  32. tmtime "github.com/tendermint/tendermint/libs/time"
  33. "github.com/tendermint/tendermint/privval"
  34. "github.com/tendermint/tendermint/types"
  35. )
  36. func TestNodeStartStop(t *testing.T) {
  37. cfg, err := config.ResetTestRoot("node_node_test")
  38. require.NoError(t, err)
  39. defer os.RemoveAll(cfg.RootDir)
  40. ctx, bcancel := context.WithCancel(context.Background())
  41. defer bcancel()
  42. // create & start node
  43. ns, err := newDefaultNode(ctx, cfg, log.TestingLogger())
  44. require.NoError(t, err)
  45. n, ok := ns.(*nodeImpl)
  46. require.True(t, ok)
  47. t.Cleanup(func() {
  48. if n.IsRunning() {
  49. bcancel()
  50. n.Wait()
  51. }
  52. })
  53. require.NoError(t, n.Start(ctx))
  54. // wait for the node to produce a block
  55. tctx, cancel := context.WithTimeout(ctx, time.Second)
  56. defer cancel()
  57. blocksSub, err := n.EventBus().SubscribeWithArgs(tctx, pubsub.SubscribeArgs{
  58. ClientID: "node_test",
  59. Query: types.EventQueryNewBlock,
  60. })
  61. require.NoError(t, err)
  62. _, err = blocksSub.Next(tctx)
  63. require.NoError(t, err, "waiting for event")
  64. cancel() // stop the subscription context
  65. bcancel() // stop the base context
  66. n.Wait()
  67. require.False(t, n.IsRunning(), "node must shut down")
  68. }
  69. func getTestNode(ctx context.Context, t *testing.T, conf *config.Config, logger log.Logger) *nodeImpl {
  70. t.Helper()
  71. ctx, cancel := context.WithCancel(ctx)
  72. defer cancel()
  73. ns, err := newDefaultNode(ctx, conf, logger)
  74. require.NoError(t, err)
  75. n, ok := ns.(*nodeImpl)
  76. require.True(t, ok)
  77. t.Cleanup(func() {
  78. cancel()
  79. if n.IsRunning() {
  80. ns.Wait()
  81. }
  82. })
  83. return n
  84. }
  85. func TestNodeDelayedStart(t *testing.T) {
  86. cfg, err := config.ResetTestRoot("node_delayed_start_test")
  87. require.NoError(t, err)
  88. defer os.RemoveAll(cfg.RootDir)
  89. now := tmtime.Now()
  90. ctx, cancel := context.WithCancel(context.Background())
  91. defer cancel()
  92. // create & start node
  93. n := getTestNode(ctx, t, cfg, log.TestingLogger())
  94. n.GenesisDoc().GenesisTime = now.Add(2 * time.Second)
  95. require.NoError(t, n.Start(ctx))
  96. startTime := tmtime.Now()
  97. assert.Equal(t, true, startTime.After(n.GenesisDoc().GenesisTime))
  98. }
  99. func TestNodeSetAppVersion(t *testing.T) {
  100. cfg, err := config.ResetTestRoot("node_app_version_test")
  101. require.NoError(t, err)
  102. defer os.RemoveAll(cfg.RootDir)
  103. ctx, cancel := context.WithCancel(context.Background())
  104. defer cancel()
  105. // create node
  106. n := getTestNode(ctx, t, cfg, log.TestingLogger())
  107. // default config uses the kvstore app
  108. appVersion := kvstore.ProtocolVersion
  109. // check version is set in state
  110. state, err := n.stateStore.Load()
  111. require.NoError(t, err)
  112. assert.Equal(t, state.Version.Consensus.App, appVersion)
  113. // check version is set in node info
  114. assert.Equal(t, n.nodeInfo.ProtocolVersion.App, appVersion)
  115. }
  116. func TestNodeSetPrivValTCP(t *testing.T) {
  117. addr := "tcp://" + testFreeAddr(t)
  118. ctx, cancel := context.WithCancel(context.Background())
  119. defer cancel()
  120. cfg, err := config.ResetTestRoot("node_priv_val_tcp_test")
  121. require.NoError(t, err)
  122. defer os.RemoveAll(cfg.RootDir)
  123. cfg.PrivValidator.ListenAddr = addr
  124. dialer := privval.DialTCPFn(addr, 100*time.Millisecond, ed25519.GenPrivKey())
  125. dialerEndpoint := privval.NewSignerDialerEndpoint(
  126. log.TestingLogger(),
  127. dialer,
  128. )
  129. privval.SignerDialerEndpointTimeoutReadWrite(100 * time.Millisecond)(dialerEndpoint)
  130. signerServer := privval.NewSignerServer(
  131. dialerEndpoint,
  132. cfg.ChainID(),
  133. types.NewMockPV(),
  134. )
  135. go func() {
  136. err := signerServer.Start(ctx)
  137. if err != nil {
  138. panic(err)
  139. }
  140. }()
  141. defer signerServer.Stop() //nolint:errcheck // ignore for tests
  142. n := getTestNode(ctx, t, cfg, log.TestingLogger())
  143. assert.IsType(t, &privval.RetrySignerClient{}, n.PrivValidator())
  144. }
  145. // address without a protocol must result in error
  146. func TestPrivValidatorListenAddrNoProtocol(t *testing.T) {
  147. ctx, cancel := context.WithCancel(context.Background())
  148. defer cancel()
  149. addrNoPrefix := testFreeAddr(t)
  150. cfg, err := config.ResetTestRoot("node_priv_val_tcp_test")
  151. require.NoError(t, err)
  152. defer os.RemoveAll(cfg.RootDir)
  153. cfg.PrivValidator.ListenAddr = addrNoPrefix
  154. n, err := newDefaultNode(ctx, cfg, log.TestingLogger())
  155. assert.Error(t, err)
  156. if n != nil && n.IsRunning() {
  157. cancel()
  158. n.Wait()
  159. }
  160. }
  161. func TestNodeSetPrivValIPC(t *testing.T) {
  162. tmpfile := "/tmp/kms." + tmrand.Str(6) + ".sock"
  163. defer os.Remove(tmpfile) // clean up
  164. ctx, cancel := context.WithCancel(context.Background())
  165. defer cancel()
  166. cfg, err := config.ResetTestRoot("node_priv_val_tcp_test")
  167. require.NoError(t, err)
  168. defer os.RemoveAll(cfg.RootDir)
  169. cfg.PrivValidator.ListenAddr = "unix://" + tmpfile
  170. dialer := privval.DialUnixFn(tmpfile)
  171. dialerEndpoint := privval.NewSignerDialerEndpoint(
  172. log.TestingLogger(),
  173. dialer,
  174. )
  175. privval.SignerDialerEndpointTimeoutReadWrite(100 * time.Millisecond)(dialerEndpoint)
  176. pvsc := privval.NewSignerServer(
  177. dialerEndpoint,
  178. cfg.ChainID(),
  179. types.NewMockPV(),
  180. )
  181. go func() {
  182. err := pvsc.Start(ctx)
  183. require.NoError(t, err)
  184. }()
  185. defer pvsc.Stop() //nolint:errcheck // ignore for tests
  186. n := getTestNode(ctx, t, cfg, log.TestingLogger())
  187. assert.IsType(t, &privval.RetrySignerClient{}, n.PrivValidator())
  188. }
  189. // testFreeAddr claims a free port so we don't block on listener being ready.
  190. func testFreeAddr(t *testing.T) string {
  191. ln, err := net.Listen("tcp", "127.0.0.1:0")
  192. require.NoError(t, err)
  193. defer ln.Close()
  194. return fmt.Sprintf("127.0.0.1:%d", ln.Addr().(*net.TCPAddr).Port)
  195. }
  196. // create a proposal block using real and full
  197. // mempool and evidence pool and validate it.
  198. func TestCreateProposalBlock(t *testing.T) {
  199. ctx, cancel := context.WithCancel(context.Background())
  200. defer cancel()
  201. cfg, err := config.ResetTestRoot("node_create_proposal")
  202. require.NoError(t, err)
  203. defer os.RemoveAll(cfg.RootDir)
  204. cc := abciclient.NewLocalCreator(kvstore.NewApplication())
  205. proxyApp := proxy.NewAppConns(cc, log.TestingLogger(), proxy.NopMetrics())
  206. err = proxyApp.Start(ctx)
  207. require.Nil(t, err)
  208. logger := log.TestingLogger()
  209. const height int64 = 1
  210. state, stateDB, privVals := state(t, 1, height)
  211. stateStore := sm.NewStore(stateDB)
  212. maxBytes := 16384
  213. const partSize uint32 = 256
  214. maxEvidenceBytes := int64(maxBytes / 2)
  215. state.ConsensusParams.Block.MaxBytes = int64(maxBytes)
  216. state.ConsensusParams.Evidence.MaxBytes = maxEvidenceBytes
  217. proposerAddr, _ := state.Validators.GetByIndex(0)
  218. mp := mempool.NewTxMempool(
  219. logger.With("module", "mempool"),
  220. cfg.Mempool,
  221. proxyApp.Mempool(),
  222. state.LastBlockHeight,
  223. )
  224. // Make EvidencePool
  225. evidenceDB := dbm.NewMemDB()
  226. blockStore := store.NewBlockStore(dbm.NewMemDB())
  227. evidencePool, err := evidence.NewPool(logger, evidenceDB, stateStore, blockStore)
  228. require.NoError(t, err)
  229. // fill the evidence pool with more evidence
  230. // than can fit in a block
  231. var currentBytes int64
  232. for currentBytes <= maxEvidenceBytes {
  233. ev := types.NewMockDuplicateVoteEvidenceWithValidator(height, time.Now(), privVals[0], "test-chain")
  234. currentBytes += int64(len(ev.Bytes()))
  235. evidencePool.ReportConflictingVotes(ev.VoteA, ev.VoteB)
  236. }
  237. evList, size := evidencePool.PendingEvidence(state.ConsensusParams.Evidence.MaxBytes)
  238. require.Less(t, size, state.ConsensusParams.Evidence.MaxBytes+1)
  239. evData := &types.EvidenceData{Evidence: evList}
  240. require.EqualValues(t, size, evData.ByteSize())
  241. // fill the mempool with more txs
  242. // than can fit in a block
  243. txLength := 100
  244. for i := 0; i <= maxBytes/txLength; i++ {
  245. tx := tmrand.Bytes(txLength)
  246. err := mp.CheckTx(ctx, tx, nil, mempool.TxInfo{})
  247. assert.NoError(t, err)
  248. }
  249. blockExec := sm.NewBlockExecutor(
  250. stateStore,
  251. logger,
  252. proxyApp.Consensus(),
  253. mp,
  254. evidencePool,
  255. blockStore,
  256. )
  257. commit := types.NewCommit(height-1, 0, types.BlockID{}, nil)
  258. block, _ := blockExec.CreateProposalBlock(
  259. height,
  260. state, commit,
  261. proposerAddr,
  262. )
  263. // check that the part set does not exceed the maximum block size
  264. partSet := block.MakePartSet(partSize)
  265. assert.Less(t, partSet.ByteSize(), int64(maxBytes))
  266. partSetFromHeader := types.NewPartSetFromHeader(partSet.Header())
  267. for partSetFromHeader.Count() < partSetFromHeader.Total() {
  268. added, err := partSetFromHeader.AddPart(partSet.GetPart(int(partSetFromHeader.Count())))
  269. require.NoError(t, err)
  270. require.True(t, added)
  271. }
  272. assert.EqualValues(t, partSetFromHeader.ByteSize(), partSet.ByteSize())
  273. err = blockExec.ValidateBlock(state, block)
  274. assert.NoError(t, err)
  275. }
  276. func TestMaxTxsProposalBlockSize(t *testing.T) {
  277. ctx, cancel := context.WithCancel(context.Background())
  278. defer cancel()
  279. cfg, err := config.ResetTestRoot("node_create_proposal")
  280. require.NoError(t, err)
  281. defer os.RemoveAll(cfg.RootDir)
  282. cc := abciclient.NewLocalCreator(kvstore.NewApplication())
  283. proxyApp := proxy.NewAppConns(cc, log.TestingLogger(), proxy.NopMetrics())
  284. err = proxyApp.Start(ctx)
  285. require.Nil(t, err)
  286. logger := log.TestingLogger()
  287. const height int64 = 1
  288. state, stateDB, _ := state(t, 1, height)
  289. stateStore := sm.NewStore(stateDB)
  290. blockStore := store.NewBlockStore(dbm.NewMemDB())
  291. const maxBytes int64 = 16384
  292. const partSize uint32 = 256
  293. state.ConsensusParams.Block.MaxBytes = maxBytes
  294. proposerAddr, _ := state.Validators.GetByIndex(0)
  295. // Make Mempool
  296. mp := mempool.NewTxMempool(
  297. logger.With("module", "mempool"),
  298. cfg.Mempool,
  299. proxyApp.Mempool(),
  300. state.LastBlockHeight,
  301. )
  302. // fill the mempool with one txs just below the maximum size
  303. txLength := int(types.MaxDataBytesNoEvidence(maxBytes, 1))
  304. tx := tmrand.Bytes(txLength - 4) // to account for the varint
  305. err = mp.CheckTx(ctx, tx, nil, mempool.TxInfo{})
  306. assert.NoError(t, err)
  307. blockExec := sm.NewBlockExecutor(
  308. stateStore,
  309. logger,
  310. proxyApp.Consensus(),
  311. mp,
  312. sm.EmptyEvidencePool{},
  313. blockStore,
  314. )
  315. commit := types.NewCommit(height-1, 0, types.BlockID{}, nil)
  316. block, _ := blockExec.CreateProposalBlock(
  317. height,
  318. state, commit,
  319. proposerAddr,
  320. )
  321. pb, err := block.ToProto()
  322. require.NoError(t, err)
  323. assert.Less(t, int64(pb.Size()), maxBytes)
  324. // check that the part set does not exceed the maximum block size
  325. partSet := block.MakePartSet(partSize)
  326. assert.EqualValues(t, partSet.ByteSize(), int64(pb.Size()))
  327. }
  328. func TestMaxProposalBlockSize(t *testing.T) {
  329. ctx, cancel := context.WithCancel(context.Background())
  330. defer cancel()
  331. cfg, err := config.ResetTestRoot("node_create_proposal")
  332. require.NoError(t, err)
  333. defer os.RemoveAll(cfg.RootDir)
  334. cc := abciclient.NewLocalCreator(kvstore.NewApplication())
  335. proxyApp := proxy.NewAppConns(cc, log.TestingLogger(), proxy.NopMetrics())
  336. err = proxyApp.Start(ctx)
  337. require.Nil(t, err)
  338. logger := log.TestingLogger()
  339. state, stateDB, _ := state(t, types.MaxVotesCount, int64(1))
  340. stateStore := sm.NewStore(stateDB)
  341. blockStore := store.NewBlockStore(dbm.NewMemDB())
  342. const maxBytes int64 = 1024 * 1024 * 2
  343. state.ConsensusParams.Block.MaxBytes = maxBytes
  344. proposerAddr, _ := state.Validators.GetByIndex(0)
  345. // Make Mempool
  346. mp := mempool.NewTxMempool(
  347. logger.With("module", "mempool"),
  348. cfg.Mempool,
  349. proxyApp.Mempool(),
  350. state.LastBlockHeight,
  351. )
  352. // fill the mempool with one txs just below the maximum size
  353. txLength := int(types.MaxDataBytesNoEvidence(maxBytes, types.MaxVotesCount))
  354. tx := tmrand.Bytes(txLength - 6) // to account for the varint
  355. err = mp.CheckTx(ctx, tx, nil, mempool.TxInfo{})
  356. assert.NoError(t, err)
  357. // now produce more txs than what a normal block can hold with 10 smaller txs
  358. // At the end of the test, only the single big tx should be added
  359. for i := 0; i < 10; i++ {
  360. tx := tmrand.Bytes(10)
  361. err = mp.CheckTx(ctx, tx, nil, mempool.TxInfo{})
  362. assert.NoError(t, err)
  363. }
  364. blockExec := sm.NewBlockExecutor(
  365. stateStore,
  366. logger,
  367. proxyApp.Consensus(),
  368. mp,
  369. sm.EmptyEvidencePool{},
  370. blockStore,
  371. )
  372. blockID := types.BlockID{
  373. Hash: tmhash.Sum([]byte("blockID_hash")),
  374. PartSetHeader: types.PartSetHeader{
  375. Total: math.MaxInt32,
  376. Hash: tmhash.Sum([]byte("blockID_part_set_header_hash")),
  377. },
  378. }
  379. timestamp := time.Date(math.MaxInt64, 0, 0, 0, 0, 0, math.MaxInt64, time.UTC)
  380. // change state in order to produce the largest accepted header
  381. state.LastBlockID = blockID
  382. state.LastBlockHeight = math.MaxInt64 - 1
  383. state.LastBlockTime = timestamp
  384. state.LastResultsHash = tmhash.Sum([]byte("last_results_hash"))
  385. state.AppHash = tmhash.Sum([]byte("app_hash"))
  386. state.Version.Consensus.Block = math.MaxInt64
  387. state.Version.Consensus.App = math.MaxInt64
  388. maxChainID := ""
  389. for i := 0; i < types.MaxChainIDLen; i++ {
  390. maxChainID += "𠜎"
  391. }
  392. state.ChainID = maxChainID
  393. cs := types.CommitSig{
  394. BlockIDFlag: types.BlockIDFlagNil,
  395. ValidatorAddress: crypto.AddressHash([]byte("validator_address")),
  396. Timestamp: timestamp,
  397. Signature: crypto.CRandBytes(types.MaxSignatureSize),
  398. }
  399. commit := &types.Commit{
  400. Height: math.MaxInt64,
  401. Round: math.MaxInt32,
  402. BlockID: blockID,
  403. }
  404. // add maximum amount of signatures to a single commit
  405. for i := 0; i < types.MaxVotesCount; i++ {
  406. commit.Signatures = append(commit.Signatures, cs)
  407. }
  408. block, partSet := blockExec.CreateProposalBlock(
  409. math.MaxInt64,
  410. state, commit,
  411. proposerAddr,
  412. )
  413. // this ensures that the header is at max size
  414. block.Header.Time = timestamp
  415. pb, err := block.ToProto()
  416. require.NoError(t, err)
  417. // require that the header and commit be the max possible size
  418. require.Equal(t, int64(pb.Header.Size()), types.MaxHeaderBytes)
  419. require.Equal(t, int64(pb.LastCommit.Size()), types.MaxCommitBytes(types.MaxVotesCount))
  420. // make sure that the block is less than the max possible size
  421. assert.Equal(t, int64(pb.Size()), maxBytes)
  422. // because of the proto overhead we expect the part set bytes to be equal or
  423. // less than the pb block size
  424. assert.LessOrEqual(t, partSet.ByteSize(), int64(pb.Size()))
  425. }
  426. func TestNodeNewSeedNode(t *testing.T) {
  427. cfg, err := config.ResetTestRoot("node_new_node_custom_reactors_test")
  428. require.NoError(t, err)
  429. cfg.Mode = config.ModeSeed
  430. defer os.RemoveAll(cfg.RootDir)
  431. ctx, cancel := context.WithCancel(context.Background())
  432. defer cancel()
  433. nodeKey, err := types.LoadOrGenNodeKey(cfg.NodeKeyFile())
  434. require.NoError(t, err)
  435. ns, err := makeSeedNode(ctx,
  436. cfg,
  437. config.DefaultDBProvider,
  438. nodeKey,
  439. defaultGenesisDocProviderFunc(cfg),
  440. log.TestingLogger(),
  441. )
  442. t.Cleanup(ns.Wait)
  443. require.NoError(t, err)
  444. n, ok := ns.(*nodeImpl)
  445. require.True(t, ok)
  446. err = n.Start(ctx)
  447. require.NoError(t, err)
  448. assert.True(t, n.pexReactor.IsRunning())
  449. cancel()
  450. n.Wait()
  451. assert.False(t, n.pexReactor.IsRunning())
  452. }
  453. func TestNodeSetEventSink(t *testing.T) {
  454. cfg, err := config.ResetTestRoot("node_app_version_test")
  455. require.NoError(t, err)
  456. defer os.RemoveAll(cfg.RootDir)
  457. ctx, cancel := context.WithCancel(context.Background())
  458. defer cancel()
  459. logger := log.TestingLogger()
  460. setupTest := func(t *testing.T, conf *config.Config) []indexer.EventSink {
  461. eventBus := eventbus.NewDefault(logger.With("module", "events"))
  462. require.NoError(t, eventBus.Start(ctx))
  463. t.Cleanup(eventBus.Wait)
  464. genDoc, err := types.GenesisDocFromFile(cfg.GenesisFile())
  465. require.NoError(t, err)
  466. indexService, eventSinks, err := createAndStartIndexerService(ctx, cfg,
  467. config.DefaultDBProvider, eventBus, logger, genDoc.ChainID,
  468. indexer.NopMetrics())
  469. require.NoError(t, err)
  470. t.Cleanup(indexService.Wait)
  471. return eventSinks
  472. }
  473. cleanup := func(ns service.Service) func() {
  474. return func() {
  475. n, ok := ns.(*nodeImpl)
  476. if !ok {
  477. return
  478. }
  479. if n == nil {
  480. return
  481. }
  482. if !n.IsRunning() {
  483. return
  484. }
  485. cancel()
  486. n.Wait()
  487. }
  488. }
  489. eventSinks := setupTest(t, cfg)
  490. assert.Equal(t, 1, len(eventSinks))
  491. assert.Equal(t, indexer.KV, eventSinks[0].Type())
  492. cfg.TxIndex.Indexer = []string{"null"}
  493. eventSinks = setupTest(t, cfg)
  494. assert.Equal(t, 1, len(eventSinks))
  495. assert.Equal(t, indexer.NULL, eventSinks[0].Type())
  496. cfg.TxIndex.Indexer = []string{"null", "kv"}
  497. eventSinks = setupTest(t, cfg)
  498. assert.Equal(t, 1, len(eventSinks))
  499. assert.Equal(t, indexer.NULL, eventSinks[0].Type())
  500. cfg.TxIndex.Indexer = []string{"kvv"}
  501. ns, err := newDefaultNode(ctx, cfg, logger)
  502. assert.Nil(t, ns)
  503. assert.Contains(t, err.Error(), "unsupported event sink type")
  504. t.Cleanup(cleanup(ns))
  505. cfg.TxIndex.Indexer = []string{}
  506. eventSinks = setupTest(t, cfg)
  507. assert.Equal(t, 1, len(eventSinks))
  508. assert.Equal(t, indexer.NULL, eventSinks[0].Type())
  509. cfg.TxIndex.Indexer = []string{"psql"}
  510. ns, err = newDefaultNode(ctx, cfg, logger)
  511. assert.Nil(t, ns)
  512. assert.Contains(t, err.Error(), "the psql connection settings cannot be empty")
  513. t.Cleanup(cleanup(ns))
  514. // N.B. We can't create a PSQL event sink without starting a postgres
  515. // instance for it to talk to. The indexer service tests exercise that case.
  516. var e = errors.New("found duplicated sinks, please check the tx-index section in the config.toml")
  517. cfg.TxIndex.Indexer = []string{"null", "kv", "Kv"}
  518. ns, err = newDefaultNode(ctx, cfg, logger)
  519. require.Error(t, err)
  520. assert.Contains(t, err.Error(), e.Error())
  521. t.Cleanup(cleanup(ns))
  522. cfg.TxIndex.Indexer = []string{"Null", "kV", "kv", "nUlL"}
  523. ns, err = newDefaultNode(ctx, cfg, logger)
  524. require.Error(t, err)
  525. assert.Contains(t, err.Error(), e.Error())
  526. t.Cleanup(cleanup(ns))
  527. }
  528. func state(t *testing.T, nVals int, height int64) (sm.State, dbm.DB, []types.PrivValidator) {
  529. t.Helper()
  530. privVals := make([]types.PrivValidator, nVals)
  531. vals := make([]types.GenesisValidator, nVals)
  532. for i := 0; i < nVals; i++ {
  533. privVal := types.NewMockPV()
  534. privVals[i] = privVal
  535. vals[i] = types.GenesisValidator{
  536. Address: privVal.PrivKey.PubKey().Address(),
  537. PubKey: privVal.PrivKey.PubKey(),
  538. Power: 1000,
  539. Name: fmt.Sprintf("test%d", i),
  540. }
  541. }
  542. s, _ := sm.MakeGenesisState(&types.GenesisDoc{
  543. ChainID: "test-chain",
  544. Validators: vals,
  545. AppHash: nil,
  546. })
  547. // save validators to db for 2 heights
  548. stateDB := dbm.NewMemDB()
  549. t.Cleanup(func() { require.NoError(t, stateDB.Close()) })
  550. stateStore := sm.NewStore(stateDB)
  551. require.NoError(t, stateStore.Save(s))
  552. for i := 1; i < int(height); i++ {
  553. s.LastBlockHeight++
  554. s.LastValidators = s.Validators.Copy()
  555. require.NoError(t, stateStore.Save(s))
  556. }
  557. return s, stateDB, privVals
  558. }
  559. func TestLoadStateFromGenesis(t *testing.T) {
  560. _ = loadStatefromGenesis(t)
  561. }
  562. func loadStatefromGenesis(t *testing.T) sm.State {
  563. t.Helper()
  564. stateDB := dbm.NewMemDB()
  565. stateStore := sm.NewStore(stateDB)
  566. cfg, err := config.ResetTestRoot("load_state_from_genesis")
  567. require.NoError(t, err)
  568. loadedState, err := stateStore.Load()
  569. require.NoError(t, err)
  570. require.True(t, loadedState.IsEmpty())
  571. genDoc, _ := factory.RandGenesisDoc(cfg, 0, false, 10)
  572. state, err := loadStateFromDBOrGenesisDocProvider(
  573. stateStore,
  574. genDoc,
  575. )
  576. require.NoError(t, err)
  577. require.NotNil(t, state)
  578. return state
  579. }