You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

1397 lines
42 KiB

types: prevent spurious validator power overflow warnings when changing the validator set (#4183) Fix for #4164 The general problem is that in certain conditions an overflow warning is issued when attempting to update a validator set even if the final set's total voting power is not over the maximum allowed. Root cause is that in verifyUpdates(), updates are verified wrt to total voting power in the order of validator address. It is then possible that a low address validator may increase its power such that the temporary total voting power count goes over MaxTotalVotingPower. Scenarios where removing and adding/ updating validators with high voting power, in the same update operation, cause the same false warning and the updates are not applied. Main changes to fix this are in verifyUpdate() that now does the verification starting with the decreases in power. It also takes into account the removals that are part of the update. ## Commits: * tests for overflow detection and prevention * test fix * more tests * fix the false overflow warnings and golint * scopelint warning fix * review comments * variant with using sort by amount of change in power * compute separately number new validators in update * types: use a switch in processChanges * more review comments * types: use HasAddress in numNewValidators * types: refactor verifyUpdates copy updates, sort them by delta and use resulting slice to calculate tvpAfterUpdatesBeforeRemovals. * remove unused structs * review comments * update changelog
5 years ago
types: prevent spurious validator power overflow warnings when changing the validator set (#4183) Fix for #4164 The general problem is that in certain conditions an overflow warning is issued when attempting to update a validator set even if the final set's total voting power is not over the maximum allowed. Root cause is that in verifyUpdates(), updates are verified wrt to total voting power in the order of validator address. It is then possible that a low address validator may increase its power such that the temporary total voting power count goes over MaxTotalVotingPower. Scenarios where removing and adding/ updating validators with high voting power, in the same update operation, cause the same false warning and the updates are not applied. Main changes to fix this are in verifyUpdate() that now does the verification starting with the decreases in power. It also takes into account the removals that are part of the update. ## Commits: * tests for overflow detection and prevention * test fix * more tests * fix the false overflow warnings and golint * scopelint warning fix * review comments * variant with using sort by amount of change in power * compute separately number new validators in update * types: use a switch in processChanges * more review comments * types: use HasAddress in numNewValidators * types: refactor verifyUpdates copy updates, sort them by delta and use resulting slice to calculate tvpAfterUpdatesBeforeRemovals. * remove unused structs * review comments * update changelog
5 years ago
types: prevent spurious validator power overflow warnings when changing the validator set (#4183) Fix for #4164 The general problem is that in certain conditions an overflow warning is issued when attempting to update a validator set even if the final set's total voting power is not over the maximum allowed. Root cause is that in verifyUpdates(), updates are verified wrt to total voting power in the order of validator address. It is then possible that a low address validator may increase its power such that the temporary total voting power count goes over MaxTotalVotingPower. Scenarios where removing and adding/ updating validators with high voting power, in the same update operation, cause the same false warning and the updates are not applied. Main changes to fix this are in verifyUpdate() that now does the verification starting with the decreases in power. It also takes into account the removals that are part of the update. ## Commits: * tests for overflow detection and prevention * test fix * more tests * fix the false overflow warnings and golint * scopelint warning fix * review comments * variant with using sort by amount of change in power * compute separately number new validators in update * types: use a switch in processChanges * more review comments * types: use HasAddress in numNewValidators * types: refactor verifyUpdates copy updates, sort them by delta and use resulting slice to calculate tvpAfterUpdatesBeforeRemovals. * remove unused structs * review comments * update changelog
5 years ago
types: prevent spurious validator power overflow warnings when changing the validator set (#4183) Fix for #4164 The general problem is that in certain conditions an overflow warning is issued when attempting to update a validator set even if the final set's total voting power is not over the maximum allowed. Root cause is that in verifyUpdates(), updates are verified wrt to total voting power in the order of validator address. It is then possible that a low address validator may increase its power such that the temporary total voting power count goes over MaxTotalVotingPower. Scenarios where removing and adding/ updating validators with high voting power, in the same update operation, cause the same false warning and the updates are not applied. Main changes to fix this are in verifyUpdate() that now does the verification starting with the decreases in power. It also takes into account the removals that are part of the update. ## Commits: * tests for overflow detection and prevention * test fix * more tests * fix the false overflow warnings and golint * scopelint warning fix * review comments * variant with using sort by amount of change in power * compute separately number new validators in update * types: use a switch in processChanges * more review comments * types: use HasAddress in numNewValidators * types: refactor verifyUpdates copy updates, sort them by delta and use resulting slice to calculate tvpAfterUpdatesBeforeRemovals. * remove unused structs * review comments * update changelog
5 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
types: prevent spurious validator power overflow warnings when changing the validator set (#4183) Fix for #4164 The general problem is that in certain conditions an overflow warning is issued when attempting to update a validator set even if the final set's total voting power is not over the maximum allowed. Root cause is that in verifyUpdates(), updates are verified wrt to total voting power in the order of validator address. It is then possible that a low address validator may increase its power such that the temporary total voting power count goes over MaxTotalVotingPower. Scenarios where removing and adding/ updating validators with high voting power, in the same update operation, cause the same false warning and the updates are not applied. Main changes to fix this are in verifyUpdate() that now does the verification starting with the decreases in power. It also takes into account the removals that are part of the update. ## Commits: * tests for overflow detection and prevention * test fix * more tests * fix the false overflow warnings and golint * scopelint warning fix * review comments * variant with using sort by amount of change in power * compute separately number new validators in update * types: use a switch in processChanges * more review comments * types: use HasAddress in numNewValidators * types: refactor verifyUpdates copy updates, sort them by delta and use resulting slice to calculate tvpAfterUpdatesBeforeRemovals. * remove unused structs * review comments * update changelog
5 years ago
types: prevent spurious validator power overflow warnings when changing the validator set (#4183) Fix for #4164 The general problem is that in certain conditions an overflow warning is issued when attempting to update a validator set even if the final set's total voting power is not over the maximum allowed. Root cause is that in verifyUpdates(), updates are verified wrt to total voting power in the order of validator address. It is then possible that a low address validator may increase its power such that the temporary total voting power count goes over MaxTotalVotingPower. Scenarios where removing and adding/ updating validators with high voting power, in the same update operation, cause the same false warning and the updates are not applied. Main changes to fix this are in verifyUpdate() that now does the verification starting with the decreases in power. It also takes into account the removals that are part of the update. ## Commits: * tests for overflow detection and prevention * test fix * more tests * fix the false overflow warnings and golint * scopelint warning fix * review comments * variant with using sort by amount of change in power * compute separately number new validators in update * types: use a switch in processChanges * more review comments * types: use HasAddress in numNewValidators * types: refactor verifyUpdates copy updates, sort them by delta and use resulting slice to calculate tvpAfterUpdatesBeforeRemovals. * remove unused structs * review comments * update changelog
5 years ago
types: prevent spurious validator power overflow warnings when changing the validator set (#4183) Fix for #4164 The general problem is that in certain conditions an overflow warning is issued when attempting to update a validator set even if the final set's total voting power is not over the maximum allowed. Root cause is that in verifyUpdates(), updates are verified wrt to total voting power in the order of validator address. It is then possible that a low address validator may increase its power such that the temporary total voting power count goes over MaxTotalVotingPower. Scenarios where removing and adding/ updating validators with high voting power, in the same update operation, cause the same false warning and the updates are not applied. Main changes to fix this are in verifyUpdate() that now does the verification starting with the decreases in power. It also takes into account the removals that are part of the update. ## Commits: * tests for overflow detection and prevention * test fix * more tests * fix the false overflow warnings and golint * scopelint warning fix * review comments * variant with using sort by amount of change in power * compute separately number new validators in update * types: use a switch in processChanges * more review comments * types: use HasAddress in numNewValidators * types: refactor verifyUpdates copy updates, sort them by delta and use resulting slice to calculate tvpAfterUpdatesBeforeRemovals. * remove unused structs * review comments * update changelog
5 years ago
types: prevent spurious validator power overflow warnings when changing the validator set (#4183) Fix for #4164 The general problem is that in certain conditions an overflow warning is issued when attempting to update a validator set even if the final set's total voting power is not over the maximum allowed. Root cause is that in verifyUpdates(), updates are verified wrt to total voting power in the order of validator address. It is then possible that a low address validator may increase its power such that the temporary total voting power count goes over MaxTotalVotingPower. Scenarios where removing and adding/ updating validators with high voting power, in the same update operation, cause the same false warning and the updates are not applied. Main changes to fix this are in verifyUpdate() that now does the verification starting with the decreases in power. It also takes into account the removals that are part of the update. ## Commits: * tests for overflow detection and prevention * test fix * more tests * fix the false overflow warnings and golint * scopelint warning fix * review comments * variant with using sort by amount of change in power * compute separately number new validators in update * types: use a switch in processChanges * more review comments * types: use HasAddress in numNewValidators * types: refactor verifyUpdates copy updates, sort them by delta and use resulting slice to calculate tvpAfterUpdatesBeforeRemovals. * remove unused structs * review comments * update changelog
5 years ago
types: prevent spurious validator power overflow warnings when changing the validator set (#4183) Fix for #4164 The general problem is that in certain conditions an overflow warning is issued when attempting to update a validator set even if the final set's total voting power is not over the maximum allowed. Root cause is that in verifyUpdates(), updates are verified wrt to total voting power in the order of validator address. It is then possible that a low address validator may increase its power such that the temporary total voting power count goes over MaxTotalVotingPower. Scenarios where removing and adding/ updating validators with high voting power, in the same update operation, cause the same false warning and the updates are not applied. Main changes to fix this are in verifyUpdate() that now does the verification starting with the decreases in power. It also takes into account the removals that are part of the update. ## Commits: * tests for overflow detection and prevention * test fix * more tests * fix the false overflow warnings and golint * scopelint warning fix * review comments * variant with using sort by amount of change in power * compute separately number new validators in update * types: use a switch in processChanges * more review comments * types: use HasAddress in numNewValidators * types: refactor verifyUpdates copy updates, sort them by delta and use resulting slice to calculate tvpAfterUpdatesBeforeRemovals. * remove unused structs * review comments * update changelog
5 years ago
types: prevent spurious validator power overflow warnings when changing the validator set (#4183) Fix for #4164 The general problem is that in certain conditions an overflow warning is issued when attempting to update a validator set even if the final set's total voting power is not over the maximum allowed. Root cause is that in verifyUpdates(), updates are verified wrt to total voting power in the order of validator address. It is then possible that a low address validator may increase its power such that the temporary total voting power count goes over MaxTotalVotingPower. Scenarios where removing and adding/ updating validators with high voting power, in the same update operation, cause the same false warning and the updates are not applied. Main changes to fix this are in verifyUpdate() that now does the verification starting with the decreases in power. It also takes into account the removals that are part of the update. ## Commits: * tests for overflow detection and prevention * test fix * more tests * fix the false overflow warnings and golint * scopelint warning fix * review comments * variant with using sort by amount of change in power * compute separately number new validators in update * types: use a switch in processChanges * more review comments * types: use HasAddress in numNewValidators * types: refactor verifyUpdates copy updates, sort them by delta and use resulting slice to calculate tvpAfterUpdatesBeforeRemovals. * remove unused structs * review comments * update changelog
5 years ago
types: prevent spurious validator power overflow warnings when changing the validator set (#4183) Fix for #4164 The general problem is that in certain conditions an overflow warning is issued when attempting to update a validator set even if the final set's total voting power is not over the maximum allowed. Root cause is that in verifyUpdates(), updates are verified wrt to total voting power in the order of validator address. It is then possible that a low address validator may increase its power such that the temporary total voting power count goes over MaxTotalVotingPower. Scenarios where removing and adding/ updating validators with high voting power, in the same update operation, cause the same false warning and the updates are not applied. Main changes to fix this are in verifyUpdate() that now does the verification starting with the decreases in power. It also takes into account the removals that are part of the update. ## Commits: * tests for overflow detection and prevention * test fix * more tests * fix the false overflow warnings and golint * scopelint warning fix * review comments * variant with using sort by amount of change in power * compute separately number new validators in update * types: use a switch in processChanges * more review comments * types: use HasAddress in numNewValidators * types: refactor verifyUpdates copy updates, sort them by delta and use resulting slice to calculate tvpAfterUpdatesBeforeRemovals. * remove unused structs * review comments * update changelog
5 years ago
  1. package types
  2. import (
  3. "bytes"
  4. "fmt"
  5. "math"
  6. "sort"
  7. "strings"
  8. "testing"
  9. "testing/quick"
  10. "github.com/stretchr/testify/assert"
  11. "github.com/tendermint/tendermint/crypto"
  12. "github.com/tendermint/tendermint/crypto/ed25519"
  13. tmmath "github.com/tendermint/tendermint/libs/math"
  14. tmrand "github.com/tendermint/tendermint/libs/rand"
  15. tmtime "github.com/tendermint/tendermint/types/time"
  16. )
  17. func TestValidatorSetBasic(t *testing.T) {
  18. // empty or nil validator lists are allowed,
  19. // but attempting to IncrementProposerPriority on them will panic.
  20. vset := NewValidatorSet([]*Validator{})
  21. assert.Panics(t, func() { vset.IncrementProposerPriority(1) })
  22. vset = NewValidatorSet(nil)
  23. assert.Panics(t, func() { vset.IncrementProposerPriority(1) })
  24. assert.EqualValues(t, vset, vset.Copy())
  25. assert.False(t, vset.HasAddress([]byte("some val")))
  26. idx, val := vset.GetByAddress([]byte("some val"))
  27. assert.Equal(t, -1, idx)
  28. assert.Nil(t, val)
  29. addr, val := vset.GetByIndex(-100)
  30. assert.Nil(t, addr)
  31. assert.Nil(t, val)
  32. addr, val = vset.GetByIndex(0)
  33. assert.Nil(t, addr)
  34. assert.Nil(t, val)
  35. addr, val = vset.GetByIndex(100)
  36. assert.Nil(t, addr)
  37. assert.Nil(t, val)
  38. assert.Zero(t, vset.Size())
  39. assert.Equal(t, int64(0), vset.TotalVotingPower())
  40. assert.Nil(t, vset.GetProposer())
  41. assert.Nil(t, vset.Hash())
  42. // add
  43. val = randValidator(vset.TotalVotingPower())
  44. assert.NoError(t, vset.UpdateWithChangeSet([]*Validator{val}))
  45. assert.True(t, vset.HasAddress(val.Address))
  46. idx, _ = vset.GetByAddress(val.Address)
  47. assert.Equal(t, 0, idx)
  48. addr, _ = vset.GetByIndex(0)
  49. assert.Equal(t, []byte(val.Address), addr)
  50. assert.Equal(t, 1, vset.Size())
  51. assert.Equal(t, val.VotingPower, vset.TotalVotingPower())
  52. assert.NotNil(t, vset.Hash())
  53. assert.NotPanics(t, func() { vset.IncrementProposerPriority(1) })
  54. assert.Equal(t, val.Address, vset.GetProposer().Address)
  55. // update
  56. val = randValidator(vset.TotalVotingPower())
  57. assert.NoError(t, vset.UpdateWithChangeSet([]*Validator{val}))
  58. _, val = vset.GetByAddress(val.Address)
  59. val.VotingPower += 100
  60. proposerPriority := val.ProposerPriority
  61. val.ProposerPriority = 0
  62. assert.NoError(t, vset.UpdateWithChangeSet([]*Validator{val}))
  63. _, val = vset.GetByAddress(val.Address)
  64. assert.Equal(t, proposerPriority, val.ProposerPriority)
  65. }
  66. func TestCopy(t *testing.T) {
  67. vset := randValidatorSet(10)
  68. vsetHash := vset.Hash()
  69. if len(vsetHash) == 0 {
  70. t.Fatalf("ValidatorSet had unexpected zero hash")
  71. }
  72. vsetCopy := vset.Copy()
  73. vsetCopyHash := vsetCopy.Hash()
  74. if !bytes.Equal(vsetHash, vsetCopyHash) {
  75. t.Fatalf("ValidatorSet copy had wrong hash. Orig: %X, Copy: %X", vsetHash, vsetCopyHash)
  76. }
  77. }
  78. // Test that IncrementProposerPriority requires positive times.
  79. func TestIncrementProposerPriorityPositiveTimes(t *testing.T) {
  80. vset := NewValidatorSet([]*Validator{
  81. newValidator([]byte("foo"), 1000),
  82. newValidator([]byte("bar"), 300),
  83. newValidator([]byte("baz"), 330),
  84. })
  85. assert.Panics(t, func() { vset.IncrementProposerPriority(-1) })
  86. assert.Panics(t, func() { vset.IncrementProposerPriority(0) })
  87. vset.IncrementProposerPriority(1)
  88. }
  89. func BenchmarkValidatorSetCopy(b *testing.B) {
  90. b.StopTimer()
  91. vset := NewValidatorSet([]*Validator{})
  92. for i := 0; i < 1000; i++ {
  93. privKey := ed25519.GenPrivKey()
  94. pubKey := privKey.PubKey()
  95. val := NewValidator(pubKey, 10)
  96. err := vset.UpdateWithChangeSet([]*Validator{val})
  97. if err != nil {
  98. panic("Failed to add validator")
  99. }
  100. }
  101. b.StartTimer()
  102. for i := 0; i < b.N; i++ {
  103. vset.Copy()
  104. }
  105. }
  106. //-------------------------------------------------------------------
  107. func TestProposerSelection1(t *testing.T) {
  108. vset := NewValidatorSet([]*Validator{
  109. newValidator([]byte("foo"), 1000),
  110. newValidator([]byte("bar"), 300),
  111. newValidator([]byte("baz"), 330),
  112. })
  113. var proposers []string
  114. for i := 0; i < 99; i++ {
  115. val := vset.GetProposer()
  116. proposers = append(proposers, string(val.Address))
  117. vset.IncrementProposerPriority(1)
  118. }
  119. expected := `foo baz foo bar foo foo baz foo bar foo foo baz foo foo bar foo baz foo foo bar` +
  120. ` foo foo baz foo bar foo foo baz foo bar foo foo baz foo foo bar foo baz foo foo bar` +
  121. ` foo baz foo foo bar foo baz foo foo bar foo baz foo foo foo baz bar foo foo foo baz` +
  122. ` foo bar foo foo baz foo bar foo foo baz foo bar foo foo baz foo bar foo foo baz foo` +
  123. ` foo bar foo baz foo foo bar foo baz foo foo bar foo baz foo foo`
  124. if expected != strings.Join(proposers, " ") {
  125. t.Errorf("expected sequence of proposers was\n%v\nbut got \n%v", expected, strings.Join(proposers, " "))
  126. }
  127. }
  128. func TestProposerSelection2(t *testing.T) {
  129. addr0 := []byte{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}
  130. addr1 := []byte{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1}
  131. addr2 := []byte{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2}
  132. // when all voting power is same, we go in order of addresses
  133. val0, val1, val2 := newValidator(addr0, 100), newValidator(addr1, 100), newValidator(addr2, 100)
  134. valList := []*Validator{val0, val1, val2}
  135. vals := NewValidatorSet(valList)
  136. for i := 0; i < len(valList)*5; i++ {
  137. ii := (i) % len(valList)
  138. prop := vals.GetProposer()
  139. if !bytes.Equal(prop.Address, valList[ii].Address) {
  140. t.Fatalf("(%d): Expected %X. Got %X", i, valList[ii].Address, prop.Address)
  141. }
  142. vals.IncrementProposerPriority(1)
  143. }
  144. // One validator has more than the others, but not enough to propose twice in a row
  145. *val2 = *newValidator(addr2, 400)
  146. vals = NewValidatorSet(valList)
  147. // vals.IncrementProposerPriority(1)
  148. prop := vals.GetProposer()
  149. if !bytes.Equal(prop.Address, addr2) {
  150. t.Fatalf("Expected address with highest voting power to be first proposer. Got %X", prop.Address)
  151. }
  152. vals.IncrementProposerPriority(1)
  153. prop = vals.GetProposer()
  154. if !bytes.Equal(prop.Address, addr0) {
  155. t.Fatalf("Expected smallest address to be validator. Got %X", prop.Address)
  156. }
  157. // One validator has more than the others, and enough to be proposer twice in a row
  158. *val2 = *newValidator(addr2, 401)
  159. vals = NewValidatorSet(valList)
  160. prop = vals.GetProposer()
  161. if !bytes.Equal(prop.Address, addr2) {
  162. t.Fatalf("Expected address with highest voting power to be first proposer. Got %X", prop.Address)
  163. }
  164. vals.IncrementProposerPriority(1)
  165. prop = vals.GetProposer()
  166. if !bytes.Equal(prop.Address, addr2) {
  167. t.Fatalf("Expected address with highest voting power to be second proposer. Got %X", prop.Address)
  168. }
  169. vals.IncrementProposerPriority(1)
  170. prop = vals.GetProposer()
  171. if !bytes.Equal(prop.Address, addr0) {
  172. t.Fatalf("Expected smallest address to be validator. Got %X", prop.Address)
  173. }
  174. // each validator should be the proposer a proportional number of times
  175. val0, val1, val2 = newValidator(addr0, 4), newValidator(addr1, 5), newValidator(addr2, 3)
  176. valList = []*Validator{val0, val1, val2}
  177. propCount := make([]int, 3)
  178. vals = NewValidatorSet(valList)
  179. N := 1
  180. for i := 0; i < 120*N; i++ {
  181. prop := vals.GetProposer()
  182. ii := prop.Address[19]
  183. propCount[ii]++
  184. vals.IncrementProposerPriority(1)
  185. }
  186. if propCount[0] != 40*N {
  187. t.Fatalf(
  188. "Expected prop count for validator with 4/12 of voting power to be %d/%d. Got %d/%d",
  189. 40*N,
  190. 120*N,
  191. propCount[0],
  192. 120*N,
  193. )
  194. }
  195. if propCount[1] != 50*N {
  196. t.Fatalf(
  197. "Expected prop count for validator with 5/12 of voting power to be %d/%d. Got %d/%d",
  198. 50*N,
  199. 120*N,
  200. propCount[1],
  201. 120*N,
  202. )
  203. }
  204. if propCount[2] != 30*N {
  205. t.Fatalf(
  206. "Expected prop count for validator with 3/12 of voting power to be %d/%d. Got %d/%d",
  207. 30*N,
  208. 120*N,
  209. propCount[2],
  210. 120*N,
  211. )
  212. }
  213. }
  214. func TestProposerSelection3(t *testing.T) {
  215. vset := NewValidatorSet([]*Validator{
  216. newValidator([]byte("a"), 1),
  217. newValidator([]byte("b"), 1),
  218. newValidator([]byte("c"), 1),
  219. newValidator([]byte("d"), 1),
  220. })
  221. proposerOrder := make([]*Validator, 4)
  222. for i := 0; i < 4; i++ {
  223. proposerOrder[i] = vset.GetProposer()
  224. vset.IncrementProposerPriority(1)
  225. }
  226. // i for the loop
  227. // j for the times
  228. // we should go in order for ever, despite some IncrementProposerPriority with times > 1
  229. var i, j int
  230. for ; i < 10000; i++ {
  231. got := vset.GetProposer().Address
  232. expected := proposerOrder[j%4].Address
  233. if !bytes.Equal(got, expected) {
  234. t.Fatalf(fmt.Sprintf("vset.Proposer (%X) does not match expected proposer (%X) for (%d, %d)", got, expected, i, j))
  235. }
  236. // serialize, deserialize, check proposer
  237. b := vset.toBytes()
  238. vset.fromBytes(b)
  239. computed := vset.GetProposer() // findGetProposer()
  240. if i != 0 {
  241. if !bytes.Equal(got, computed.Address) {
  242. t.Fatalf(
  243. fmt.Sprintf(
  244. "vset.Proposer (%X) does not match computed proposer (%X) for (%d, %d)",
  245. got,
  246. computed.Address,
  247. i,
  248. j,
  249. ),
  250. )
  251. }
  252. }
  253. // times is usually 1
  254. times := 1
  255. mod := (tmrand.Int() % 5) + 1
  256. if tmrand.Int()%mod > 0 {
  257. // sometimes its up to 5
  258. times = (tmrand.Int() % 4) + 1
  259. }
  260. vset.IncrementProposerPriority(times)
  261. j += times
  262. }
  263. }
  264. func newValidator(address []byte, power int64) *Validator {
  265. return &Validator{Address: address, VotingPower: power}
  266. }
  267. func randPubKey() crypto.PubKey {
  268. var pubKey [32]byte
  269. copy(pubKey[:], tmrand.Bytes(32))
  270. return ed25519.PubKeyEd25519(pubKey)
  271. }
  272. func randValidator(totalVotingPower int64) *Validator {
  273. // this modulo limits the ProposerPriority/VotingPower to stay in the
  274. // bounds of MaxTotalVotingPower minus the already existing voting power:
  275. val := NewValidator(randPubKey(), int64(tmrand.Uint64()%uint64((MaxTotalVotingPower-totalVotingPower))))
  276. val.ProposerPriority = tmrand.Int64() % (MaxTotalVotingPower - totalVotingPower)
  277. return val
  278. }
  279. func randValidatorSet(numValidators int) *ValidatorSet {
  280. validators := make([]*Validator, numValidators)
  281. totalVotingPower := int64(0)
  282. for i := 0; i < numValidators; i++ {
  283. validators[i] = randValidator(totalVotingPower)
  284. totalVotingPower += validators[i].VotingPower
  285. }
  286. return NewValidatorSet(validators)
  287. }
  288. func (vals *ValidatorSet) toBytes() []byte {
  289. bz, err := cdc.MarshalBinaryLengthPrefixed(vals)
  290. if err != nil {
  291. panic(err)
  292. }
  293. return bz
  294. }
  295. func (vals *ValidatorSet) fromBytes(b []byte) {
  296. err := cdc.UnmarshalBinaryLengthPrefixed(b, &vals)
  297. if err != nil {
  298. // DATA HAS BEEN CORRUPTED OR THE SPEC HAS CHANGED
  299. panic(err)
  300. }
  301. }
  302. //-------------------------------------------------------------------
  303. func TestValidatorSetTotalVotingPowerPanicsOnOverflow(t *testing.T) {
  304. // NewValidatorSet calls IncrementProposerPriority which calls TotalVotingPower()
  305. // which should panic on overflows:
  306. shouldPanic := func() {
  307. NewValidatorSet([]*Validator{
  308. {Address: []byte("a"), VotingPower: math.MaxInt64, ProposerPriority: 0},
  309. {Address: []byte("b"), VotingPower: math.MaxInt64, ProposerPriority: 0},
  310. {Address: []byte("c"), VotingPower: math.MaxInt64, ProposerPriority: 0},
  311. })
  312. }
  313. assert.Panics(t, shouldPanic)
  314. }
  315. func TestAvgProposerPriority(t *testing.T) {
  316. // Create Validator set without calling IncrementProposerPriority:
  317. tcs := []struct {
  318. vs ValidatorSet
  319. want int64
  320. }{
  321. 0: {ValidatorSet{Validators: []*Validator{{ProposerPriority: 0}, {ProposerPriority: 0}, {ProposerPriority: 0}}}, 0},
  322. 1: {
  323. ValidatorSet{
  324. Validators: []*Validator{{ProposerPriority: math.MaxInt64}, {ProposerPriority: 0}, {ProposerPriority: 0}},
  325. }, math.MaxInt64 / 3,
  326. },
  327. 2: {
  328. ValidatorSet{
  329. Validators: []*Validator{{ProposerPriority: math.MaxInt64}, {ProposerPriority: 0}},
  330. }, math.MaxInt64 / 2,
  331. },
  332. 3: {
  333. ValidatorSet{
  334. Validators: []*Validator{{ProposerPriority: math.MaxInt64}, {ProposerPriority: math.MaxInt64}},
  335. }, math.MaxInt64,
  336. },
  337. 4: {
  338. ValidatorSet{
  339. Validators: []*Validator{{ProposerPriority: math.MinInt64}, {ProposerPriority: math.MinInt64}},
  340. }, math.MinInt64,
  341. },
  342. }
  343. for i, tc := range tcs {
  344. got := tc.vs.computeAvgProposerPriority()
  345. assert.Equal(t, tc.want, got, "test case: %v", i)
  346. }
  347. }
  348. func TestAveragingInIncrementProposerPriority(t *testing.T) {
  349. // Test that the averaging works as expected inside of IncrementProposerPriority.
  350. // Each validator comes with zero voting power which simplifies reasoning about
  351. // the expected ProposerPriority.
  352. tcs := []struct {
  353. vs ValidatorSet
  354. times int
  355. avg int64
  356. }{
  357. 0: {ValidatorSet{
  358. Validators: []*Validator{
  359. {Address: []byte("a"), ProposerPriority: 1},
  360. {Address: []byte("b"), ProposerPriority: 2},
  361. {Address: []byte("c"), ProposerPriority: 3}}},
  362. 1, 2},
  363. 1: {ValidatorSet{
  364. Validators: []*Validator{
  365. {Address: []byte("a"), ProposerPriority: 10},
  366. {Address: []byte("b"), ProposerPriority: -10},
  367. {Address: []byte("c"), ProposerPriority: 1}}},
  368. // this should average twice but the average should be 0 after the first iteration
  369. // (voting power is 0 -> no changes)
  370. 11, 1 / 3},
  371. 2: {ValidatorSet{
  372. Validators: []*Validator{
  373. {Address: []byte("a"), ProposerPriority: 100},
  374. {Address: []byte("b"), ProposerPriority: -10},
  375. {Address: []byte("c"), ProposerPriority: 1}}},
  376. 1, 91 / 3},
  377. }
  378. for i, tc := range tcs {
  379. // work on copy to have the old ProposerPriorities:
  380. newVset := tc.vs.CopyIncrementProposerPriority(tc.times)
  381. for _, val := range tc.vs.Validators {
  382. _, updatedVal := newVset.GetByAddress(val.Address)
  383. assert.Equal(t, updatedVal.ProposerPriority, val.ProposerPriority-tc.avg, "test case: %v", i)
  384. }
  385. }
  386. }
  387. func TestAveragingInIncrementProposerPriorityWithVotingPower(t *testing.T) {
  388. // Other than TestAveragingInIncrementProposerPriority this is a more complete test showing
  389. // how each ProposerPriority changes in relation to the validator's voting power respectively.
  390. // average is zero in each round:
  391. vp0 := int64(10)
  392. vp1 := int64(1)
  393. vp2 := int64(1)
  394. total := vp0 + vp1 + vp2
  395. avg := (vp0 + vp1 + vp2 - total) / 3
  396. vals := ValidatorSet{Validators: []*Validator{
  397. {Address: []byte{0}, ProposerPriority: 0, VotingPower: vp0},
  398. {Address: []byte{1}, ProposerPriority: 0, VotingPower: vp1},
  399. {Address: []byte{2}, ProposerPriority: 0, VotingPower: vp2}}}
  400. tcs := []struct {
  401. vals *ValidatorSet
  402. wantProposerPrioritys []int64
  403. times int
  404. wantProposer *Validator
  405. }{
  406. 0: {
  407. vals.Copy(),
  408. []int64{
  409. // Acumm+VotingPower-Avg:
  410. 0 + vp0 - total - avg, // mostest will be subtracted by total voting power (12)
  411. 0 + vp1,
  412. 0 + vp2},
  413. 1,
  414. vals.Validators[0]},
  415. 1: {
  416. vals.Copy(),
  417. []int64{
  418. (0 + vp0 - total) + vp0 - total - avg, // this will be mostest on 2nd iter, too
  419. (0 + vp1) + vp1,
  420. (0 + vp2) + vp2},
  421. 2,
  422. vals.Validators[0]}, // increment twice -> expect average to be subtracted twice
  423. 2: {
  424. vals.Copy(),
  425. []int64{
  426. 0 + 3*(vp0-total) - avg, // still mostest
  427. 0 + 3*vp1,
  428. 0 + 3*vp2},
  429. 3,
  430. vals.Validators[0]},
  431. 3: {
  432. vals.Copy(),
  433. []int64{
  434. 0 + 4*(vp0-total), // still mostest
  435. 0 + 4*vp1,
  436. 0 + 4*vp2},
  437. 4,
  438. vals.Validators[0]},
  439. 4: {
  440. vals.Copy(),
  441. []int64{
  442. 0 + 4*(vp0-total) + vp0, // 4 iters was mostest
  443. 0 + 5*vp1 - total, // now this val is mostest for the 1st time (hence -12==totalVotingPower)
  444. 0 + 5*vp2},
  445. 5,
  446. vals.Validators[1]},
  447. 5: {
  448. vals.Copy(),
  449. []int64{
  450. 0 + 6*vp0 - 5*total, // mostest again
  451. 0 + 6*vp1 - total, // mostest once up to here
  452. 0 + 6*vp2},
  453. 6,
  454. vals.Validators[0]},
  455. 6: {
  456. vals.Copy(),
  457. []int64{
  458. 0 + 7*vp0 - 6*total, // in 7 iters this val is mostest 6 times
  459. 0 + 7*vp1 - total, // in 7 iters this val is mostest 1 time
  460. 0 + 7*vp2},
  461. 7,
  462. vals.Validators[0]},
  463. 7: {
  464. vals.Copy(),
  465. []int64{
  466. 0 + 8*vp0 - 7*total, // mostest again
  467. 0 + 8*vp1 - total,
  468. 0 + 8*vp2},
  469. 8,
  470. vals.Validators[0]},
  471. 8: {
  472. vals.Copy(),
  473. []int64{
  474. 0 + 9*vp0 - 7*total,
  475. 0 + 9*vp1 - total,
  476. 0 + 9*vp2 - total}, // mostest
  477. 9,
  478. vals.Validators[2]},
  479. 9: {
  480. vals.Copy(),
  481. []int64{
  482. 0 + 10*vp0 - 8*total, // after 10 iters this is mostest again
  483. 0 + 10*vp1 - total, // after 6 iters this val is "mostest" once and not in between
  484. 0 + 10*vp2 - total}, // in between 10 iters this val is "mostest" once
  485. 10,
  486. vals.Validators[0]},
  487. 10: {
  488. vals.Copy(),
  489. []int64{
  490. 0 + 11*vp0 - 9*total,
  491. 0 + 11*vp1 - total, // after 6 iters this val is "mostest" once and not in between
  492. 0 + 11*vp2 - total}, // after 10 iters this val is "mostest" once
  493. 11,
  494. vals.Validators[0]},
  495. }
  496. for i, tc := range tcs {
  497. tc.vals.IncrementProposerPriority(tc.times)
  498. assert.Equal(t, tc.wantProposer.Address, tc.vals.GetProposer().Address,
  499. "test case: %v",
  500. i)
  501. for valIdx, val := range tc.vals.Validators {
  502. assert.Equal(t,
  503. tc.wantProposerPrioritys[valIdx],
  504. val.ProposerPriority,
  505. "test case: %v, validator: %v",
  506. i,
  507. valIdx)
  508. }
  509. }
  510. }
  511. func TestSafeAdd(t *testing.T) {
  512. f := func(a, b int64) bool {
  513. c, overflow := safeAdd(a, b)
  514. return overflow || (!overflow && c == a+b)
  515. }
  516. if err := quick.Check(f, nil); err != nil {
  517. t.Error(err)
  518. }
  519. }
  520. func TestSafeAddClip(t *testing.T) {
  521. assert.EqualValues(t, math.MaxInt64, safeAddClip(math.MaxInt64, 10))
  522. assert.EqualValues(t, math.MaxInt64, safeAddClip(math.MaxInt64, math.MaxInt64))
  523. assert.EqualValues(t, math.MinInt64, safeAddClip(math.MinInt64, -10))
  524. }
  525. func TestSafeSubClip(t *testing.T) {
  526. assert.EqualValues(t, math.MinInt64, safeSubClip(math.MinInt64, 10))
  527. assert.EqualValues(t, 0, safeSubClip(math.MinInt64, math.MinInt64))
  528. assert.EqualValues(t, math.MinInt64, safeSubClip(math.MinInt64, math.MaxInt64))
  529. assert.EqualValues(t, math.MaxInt64, safeSubClip(math.MaxInt64, -10))
  530. }
  531. //-------------------------------------------------------------------
  532. func TestValidatorSetVerifyCommit(t *testing.T) {
  533. privKey := ed25519.GenPrivKey()
  534. pubKey := privKey.PubKey()
  535. v1 := NewValidator(pubKey, 1000)
  536. vset := NewValidatorSet([]*Validator{v1})
  537. // good
  538. var (
  539. chainID = "mychainID"
  540. blockID = makeBlockIDRandom()
  541. height = int64(5)
  542. )
  543. vote := &Vote{
  544. ValidatorAddress: v1.Address,
  545. ValidatorIndex: 0,
  546. Height: height,
  547. Round: 0,
  548. Timestamp: tmtime.Now(),
  549. Type: PrecommitType,
  550. BlockID: blockID,
  551. }
  552. sig, err := privKey.Sign(vote.SignBytes(chainID))
  553. assert.NoError(t, err)
  554. vote.Signature = sig
  555. commit := NewCommit(vote.Height, vote.Round, blockID, []CommitSig{vote.CommitSig()})
  556. // bad
  557. var (
  558. badChainID = "notmychainID"
  559. badBlockID = BlockID{Hash: []byte("goodbye")}
  560. badHeight = height + 1
  561. badCommit = NewCommit(badHeight, 0, blockID, []CommitSig{{BlockIDFlag: BlockIDFlagAbsent}})
  562. )
  563. // test some error cases
  564. // TODO: test more cases!
  565. cases := []struct {
  566. chainID string
  567. blockID BlockID
  568. height int64
  569. commit *Commit
  570. }{
  571. {badChainID, blockID, height, commit},
  572. {chainID, badBlockID, height, commit},
  573. {chainID, blockID, badHeight, commit},
  574. {chainID, blockID, height, badCommit},
  575. }
  576. for i, c := range cases {
  577. err := vset.VerifyCommit(c.chainID, c.blockID, c.height, c.commit)
  578. assert.NotNil(t, err, i)
  579. }
  580. // test a good one
  581. err = vset.VerifyCommit(chainID, blockID, height, commit)
  582. assert.Nil(t, err)
  583. }
  584. func TestEmptySet(t *testing.T) {
  585. var valList []*Validator
  586. valSet := NewValidatorSet(valList)
  587. assert.Panics(t, func() { valSet.IncrementProposerPriority(1) })
  588. assert.Panics(t, func() { valSet.RescalePriorities(100) })
  589. assert.Panics(t, func() { valSet.shiftByAvgProposerPriority() })
  590. assert.Panics(t, func() { assert.Zero(t, computeMaxMinPriorityDiff(valSet)) })
  591. valSet.GetProposer()
  592. // Add to empty set
  593. v1 := newValidator([]byte("v1"), 100)
  594. v2 := newValidator([]byte("v2"), 100)
  595. valList = []*Validator{v1, v2}
  596. assert.NoError(t, valSet.UpdateWithChangeSet(valList))
  597. verifyValidatorSet(t, valSet)
  598. // Delete all validators from set
  599. v1 = newValidator([]byte("v1"), 0)
  600. v2 = newValidator([]byte("v2"), 0)
  601. delList := []*Validator{v1, v2}
  602. assert.Error(t, valSet.UpdateWithChangeSet(delList))
  603. // Attempt delete from empty set
  604. assert.Error(t, valSet.UpdateWithChangeSet(delList))
  605. }
  606. func TestUpdatesForNewValidatorSet(t *testing.T) {
  607. v1 := newValidator([]byte("v1"), 100)
  608. v2 := newValidator([]byte("v2"), 100)
  609. valList := []*Validator{v1, v2}
  610. valSet := NewValidatorSet(valList)
  611. verifyValidatorSet(t, valSet)
  612. // Verify duplicates are caught in NewValidatorSet() and it panics
  613. v111 := newValidator([]byte("v1"), 100)
  614. v112 := newValidator([]byte("v1"), 123)
  615. v113 := newValidator([]byte("v1"), 234)
  616. valList = []*Validator{v111, v112, v113}
  617. assert.Panics(t, func() { NewValidatorSet(valList) })
  618. // Verify set including validator with voting power 0 cannot be created
  619. v1 = newValidator([]byte("v1"), 0)
  620. v2 = newValidator([]byte("v2"), 22)
  621. v3 := newValidator([]byte("v3"), 33)
  622. valList = []*Validator{v1, v2, v3}
  623. assert.Panics(t, func() { NewValidatorSet(valList) })
  624. // Verify set including validator with negative voting power cannot be created
  625. v1 = newValidator([]byte("v1"), 10)
  626. v2 = newValidator([]byte("v2"), -20)
  627. v3 = newValidator([]byte("v3"), 30)
  628. valList = []*Validator{v1, v2, v3}
  629. assert.Panics(t, func() { NewValidatorSet(valList) })
  630. }
  631. type testVal struct {
  632. name string
  633. power int64
  634. }
  635. func permutation(valList []testVal) []testVal {
  636. if len(valList) == 0 {
  637. return nil
  638. }
  639. permList := make([]testVal, len(valList))
  640. perm := tmrand.Perm(len(valList))
  641. for i, v := range perm {
  642. permList[v] = valList[i]
  643. }
  644. return permList
  645. }
  646. func createNewValidatorList(testValList []testVal) []*Validator {
  647. valList := make([]*Validator, 0, len(testValList))
  648. for _, val := range testValList {
  649. valList = append(valList, newValidator([]byte(val.name), val.power))
  650. }
  651. return valList
  652. }
  653. func createNewValidatorSet(testValList []testVal) *ValidatorSet {
  654. return NewValidatorSet(createNewValidatorList(testValList))
  655. }
  656. func valSetTotalProposerPriority(valSet *ValidatorSet) int64 {
  657. sum := int64(0)
  658. for _, val := range valSet.Validators {
  659. // mind overflow
  660. sum = safeAddClip(sum, val.ProposerPriority)
  661. }
  662. return sum
  663. }
  664. func verifyValidatorSet(t *testing.T, valSet *ValidatorSet) {
  665. // verify that the capacity and length of validators is the same
  666. assert.Equal(t, len(valSet.Validators), cap(valSet.Validators))
  667. // verify that the set's total voting power has been updated
  668. tvp := valSet.totalVotingPower
  669. valSet.updateTotalVotingPower()
  670. expectedTvp := valSet.TotalVotingPower()
  671. assert.Equal(t, expectedTvp, tvp,
  672. "expected TVP %d. Got %d, valSet=%s", expectedTvp, tvp, valSet)
  673. // verify that validator priorities are centered
  674. valsCount := int64(len(valSet.Validators))
  675. tpp := valSetTotalProposerPriority(valSet)
  676. assert.True(t, tpp < valsCount && tpp > -valsCount,
  677. "expected total priority in (-%d, %d). Got %d", valsCount, valsCount, tpp)
  678. // verify that priorities are scaled
  679. dist := computeMaxMinPriorityDiff(valSet)
  680. assert.True(t, dist <= PriorityWindowSizeFactor*tvp,
  681. "expected priority distance < %d. Got %d", PriorityWindowSizeFactor*tvp, dist)
  682. }
  683. func toTestValList(valList []*Validator) []testVal {
  684. testList := make([]testVal, len(valList))
  685. for i, val := range valList {
  686. testList[i].name = string(val.Address)
  687. testList[i].power = val.VotingPower
  688. }
  689. return testList
  690. }
  691. func testValSet(nVals int, power int64) []testVal {
  692. vals := make([]testVal, nVals)
  693. for i := 0; i < nVals; i++ {
  694. vals[i] = testVal{fmt.Sprintf("v%d", i+1), power}
  695. }
  696. return vals
  697. }
  698. type valSetErrTestCase struct {
  699. startVals []testVal
  700. updateVals []testVal
  701. }
  702. func executeValSetErrTestCase(t *testing.T, idx int, tt valSetErrTestCase) {
  703. // create a new set and apply updates, keeping copies for the checks
  704. valSet := createNewValidatorSet(tt.startVals)
  705. valSetCopy := valSet.Copy()
  706. valList := createNewValidatorList(tt.updateVals)
  707. valListCopy := validatorListCopy(valList)
  708. err := valSet.UpdateWithChangeSet(valList)
  709. // for errors check the validator set has not been changed
  710. assert.Error(t, err, "test %d", idx)
  711. assert.Equal(t, valSet, valSetCopy, "test %v", idx)
  712. // check the parameter list has not changed
  713. assert.Equal(t, valList, valListCopy, "test %v", idx)
  714. }
  715. func TestValSetUpdatesDuplicateEntries(t *testing.T) {
  716. testCases := []valSetErrTestCase{
  717. // Duplicate entries in changes
  718. { // first entry is duplicated change
  719. testValSet(2, 10),
  720. []testVal{{"v1", 11}, {"v1", 22}},
  721. },
  722. { // second entry is duplicated change
  723. testValSet(2, 10),
  724. []testVal{{"v2", 11}, {"v2", 22}},
  725. },
  726. { // change duplicates are separated by a valid change
  727. testValSet(2, 10),
  728. []testVal{{"v1", 11}, {"v2", 22}, {"v1", 12}},
  729. },
  730. { // change duplicates are separated by a valid change
  731. testValSet(3, 10),
  732. []testVal{{"v1", 11}, {"v3", 22}, {"v1", 12}},
  733. },
  734. // Duplicate entries in remove
  735. { // first entry is duplicated remove
  736. testValSet(2, 10),
  737. []testVal{{"v1", 0}, {"v1", 0}},
  738. },
  739. { // second entry is duplicated remove
  740. testValSet(2, 10),
  741. []testVal{{"v2", 0}, {"v2", 0}},
  742. },
  743. { // remove duplicates are separated by a valid remove
  744. testValSet(2, 10),
  745. []testVal{{"v1", 0}, {"v2", 0}, {"v1", 0}},
  746. },
  747. { // remove duplicates are separated by a valid remove
  748. testValSet(3, 10),
  749. []testVal{{"v1", 0}, {"v3", 0}, {"v1", 0}},
  750. },
  751. { // remove and update same val
  752. testValSet(2, 10),
  753. []testVal{{"v1", 0}, {"v2", 20}, {"v1", 30}},
  754. },
  755. { // duplicate entries in removes + changes
  756. testValSet(2, 10),
  757. []testVal{{"v1", 0}, {"v2", 20}, {"v2", 30}, {"v1", 0}},
  758. },
  759. { // duplicate entries in removes + changes
  760. testValSet(3, 10),
  761. []testVal{{"v1", 0}, {"v3", 5}, {"v2", 20}, {"v2", 30}, {"v1", 0}},
  762. },
  763. }
  764. for i, tt := range testCases {
  765. executeValSetErrTestCase(t, i, tt)
  766. }
  767. }
  768. func TestValSetUpdatesOverflows(t *testing.T) {
  769. maxVP := MaxTotalVotingPower
  770. testCases := []valSetErrTestCase{
  771. { // single update leading to overflow
  772. testValSet(2, 10),
  773. []testVal{{"v1", math.MaxInt64}},
  774. },
  775. { // single update leading to overflow
  776. testValSet(2, 10),
  777. []testVal{{"v2", math.MaxInt64}},
  778. },
  779. { // add validator leading to overflow
  780. testValSet(1, maxVP),
  781. []testVal{{"v2", math.MaxInt64}},
  782. },
  783. { // add validator leading to exceed Max
  784. testValSet(1, maxVP-1),
  785. []testVal{{"v2", 5}},
  786. },
  787. { // add validator leading to exceed Max
  788. testValSet(2, maxVP/3),
  789. []testVal{{"v3", maxVP / 2}},
  790. },
  791. { // add validator leading to exceed Max
  792. testValSet(1, maxVP),
  793. []testVal{{"v2", maxVP}},
  794. },
  795. }
  796. for i, tt := range testCases {
  797. executeValSetErrTestCase(t, i, tt)
  798. }
  799. }
  800. func TestValSetUpdatesOtherErrors(t *testing.T) {
  801. testCases := []valSetErrTestCase{
  802. { // update with negative voting power
  803. testValSet(2, 10),
  804. []testVal{{"v1", -123}},
  805. },
  806. { // update with negative voting power
  807. testValSet(2, 10),
  808. []testVal{{"v2", -123}},
  809. },
  810. { // remove non-existing validator
  811. testValSet(2, 10),
  812. []testVal{{"v3", 0}},
  813. },
  814. { // delete all validators
  815. []testVal{{"v1", 10}, {"v2", 20}, {"v3", 30}},
  816. []testVal{{"v1", 0}, {"v2", 0}, {"v3", 0}},
  817. },
  818. }
  819. for i, tt := range testCases {
  820. executeValSetErrTestCase(t, i, tt)
  821. }
  822. }
  823. func TestValSetUpdatesBasicTestsExecute(t *testing.T) {
  824. valSetUpdatesBasicTests := []struct {
  825. startVals []testVal
  826. updateVals []testVal
  827. expectedVals []testVal
  828. }{
  829. { // no changes
  830. testValSet(2, 10),
  831. []testVal{},
  832. testValSet(2, 10),
  833. },
  834. { // voting power changes
  835. testValSet(2, 10),
  836. []testVal{{"v1", 11}, {"v2", 22}},
  837. []testVal{{"v1", 11}, {"v2", 22}},
  838. },
  839. { // add new validators
  840. []testVal{{"v1", 10}, {"v2", 20}},
  841. []testVal{{"v3", 30}, {"v4", 40}},
  842. []testVal{{"v1", 10}, {"v2", 20}, {"v3", 30}, {"v4", 40}},
  843. },
  844. { // add new validator to middle
  845. []testVal{{"v1", 10}, {"v3", 20}},
  846. []testVal{{"v2", 30}},
  847. []testVal{{"v1", 10}, {"v2", 30}, {"v3", 20}},
  848. },
  849. { // add new validator to beginning
  850. []testVal{{"v2", 10}, {"v3", 20}},
  851. []testVal{{"v1", 30}},
  852. []testVal{{"v1", 30}, {"v2", 10}, {"v3", 20}},
  853. },
  854. { // delete validators
  855. []testVal{{"v1", 10}, {"v2", 20}, {"v3", 30}},
  856. []testVal{{"v2", 0}},
  857. []testVal{{"v1", 10}, {"v3", 30}},
  858. },
  859. }
  860. for i, tt := range valSetUpdatesBasicTests {
  861. // create a new set and apply updates, keeping copies for the checks
  862. valSet := createNewValidatorSet(tt.startVals)
  863. valList := createNewValidatorList(tt.updateVals)
  864. err := valSet.UpdateWithChangeSet(valList)
  865. assert.NoError(t, err, "test %d", i)
  866. valListCopy := validatorListCopy(valSet.Validators)
  867. // check that the voting power in the set's validators is not changing if the voting power
  868. // is changed in the list of validators previously passed as parameter to UpdateWithChangeSet.
  869. // this is to make sure copies of the validators are made by UpdateWithChangeSet.
  870. if len(valList) > 0 {
  871. valList[0].VotingPower++
  872. assert.Equal(t, toTestValList(valListCopy), toTestValList(valSet.Validators), "test %v", i)
  873. }
  874. // check the final validator list is as expected and the set is properly scaled and centered.
  875. assert.Equal(t, tt.expectedVals, toTestValList(valSet.Validators), "test %v", i)
  876. verifyValidatorSet(t, valSet)
  877. }
  878. }
  879. // Test that different permutations of an update give the same result.
  880. func TestValSetUpdatesOrderIndependenceTestsExecute(t *testing.T) {
  881. // startVals - initial validators to create the set with
  882. // updateVals - a sequence of updates to be applied to the set.
  883. // updateVals is shuffled a number of times during testing to check for same resulting validator set.
  884. valSetUpdatesOrderTests := []struct {
  885. startVals []testVal
  886. updateVals []testVal
  887. }{
  888. 0: { // order of changes should not matter, the final validator sets should be the same
  889. []testVal{{"v1", 10}, {"v2", 10}, {"v3", 30}, {"v4", 40}},
  890. []testVal{{"v1", 11}, {"v2", 22}, {"v3", 33}, {"v4", 44}}},
  891. 1: { // order of additions should not matter
  892. []testVal{{"v1", 10}, {"v2", 20}},
  893. []testVal{{"v3", 30}, {"v4", 40}, {"v5", 50}, {"v6", 60}}},
  894. 2: { // order of removals should not matter
  895. []testVal{{"v1", 10}, {"v2", 20}, {"v3", 30}, {"v4", 40}},
  896. []testVal{{"v1", 0}, {"v3", 0}, {"v4", 0}}},
  897. 3: { // order of mixed operations should not matter
  898. []testVal{{"v1", 10}, {"v2", 20}, {"v3", 30}, {"v4", 40}},
  899. []testVal{{"v1", 0}, {"v3", 0}, {"v2", 22}, {"v5", 50}, {"v4", 44}}},
  900. }
  901. for i, tt := range valSetUpdatesOrderTests {
  902. // create a new set and apply updates
  903. valSet := createNewValidatorSet(tt.startVals)
  904. valSetCopy := valSet.Copy()
  905. valList := createNewValidatorList(tt.updateVals)
  906. assert.NoError(t, valSetCopy.UpdateWithChangeSet(valList))
  907. // save the result as expected for next updates
  908. valSetExp := valSetCopy.Copy()
  909. // perform at most 20 permutations on the updates and call UpdateWithChangeSet()
  910. n := len(tt.updateVals)
  911. maxNumPerms := tmmath.MinInt(20, n*n)
  912. for j := 0; j < maxNumPerms; j++ {
  913. // create a copy of original set and apply a random permutation of updates
  914. valSetCopy := valSet.Copy()
  915. valList := createNewValidatorList(permutation(tt.updateVals))
  916. // check there was no error and the set is properly scaled and centered.
  917. assert.NoError(t, valSetCopy.UpdateWithChangeSet(valList),
  918. "test %v failed for permutation %v", i, valList)
  919. verifyValidatorSet(t, valSetCopy)
  920. // verify the resulting test is same as the expected
  921. assert.Equal(t, valSetCopy, valSetExp,
  922. "test %v failed for permutation %v", i, valList)
  923. }
  924. }
  925. }
  926. // This tests the private function validator_set.go:applyUpdates() function, used only for additions and changes.
  927. // Should perform a proper merge of updatedVals and startVals
  928. func TestValSetApplyUpdatesTestsExecute(t *testing.T) {
  929. valSetUpdatesBasicTests := []struct {
  930. startVals []testVal
  931. updateVals []testVal
  932. expectedVals []testVal
  933. }{
  934. // additions
  935. 0: { // prepend
  936. []testVal{{"v4", 44}, {"v5", 55}},
  937. []testVal{{"v1", 11}},
  938. []testVal{{"v1", 11}, {"v4", 44}, {"v5", 55}}},
  939. 1: { // append
  940. []testVal{{"v4", 44}, {"v5", 55}},
  941. []testVal{{"v6", 66}},
  942. []testVal{{"v4", 44}, {"v5", 55}, {"v6", 66}}},
  943. 2: { // insert
  944. []testVal{{"v4", 44}, {"v6", 66}},
  945. []testVal{{"v5", 55}},
  946. []testVal{{"v4", 44}, {"v5", 55}, {"v6", 66}}},
  947. 3: { // insert multi
  948. []testVal{{"v4", 44}, {"v6", 66}, {"v9", 99}},
  949. []testVal{{"v5", 55}, {"v7", 77}, {"v8", 88}},
  950. []testVal{{"v4", 44}, {"v5", 55}, {"v6", 66}, {"v7", 77}, {"v8", 88}, {"v9", 99}}},
  951. // changes
  952. 4: { // head
  953. []testVal{{"v1", 111}, {"v2", 22}},
  954. []testVal{{"v1", 11}},
  955. []testVal{{"v1", 11}, {"v2", 22}}},
  956. 5: { // tail
  957. []testVal{{"v1", 11}, {"v2", 222}},
  958. []testVal{{"v2", 22}},
  959. []testVal{{"v1", 11}, {"v2", 22}}},
  960. 6: { // middle
  961. []testVal{{"v1", 11}, {"v2", 222}, {"v3", 33}},
  962. []testVal{{"v2", 22}},
  963. []testVal{{"v1", 11}, {"v2", 22}, {"v3", 33}}},
  964. 7: { // multi
  965. []testVal{{"v1", 111}, {"v2", 222}, {"v3", 333}},
  966. []testVal{{"v1", 11}, {"v2", 22}, {"v3", 33}},
  967. []testVal{{"v1", 11}, {"v2", 22}, {"v3", 33}}},
  968. // additions and changes
  969. 8: {
  970. []testVal{{"v1", 111}, {"v2", 22}},
  971. []testVal{{"v1", 11}, {"v3", 33}, {"v4", 44}},
  972. []testVal{{"v1", 11}, {"v2", 22}, {"v3", 33}, {"v4", 44}}},
  973. }
  974. for i, tt := range valSetUpdatesBasicTests {
  975. // create a new validator set with the start values
  976. valSet := createNewValidatorSet(tt.startVals)
  977. // applyUpdates() with the update values
  978. valList := createNewValidatorList(tt.updateVals)
  979. valSet.applyUpdates(valList)
  980. // check the new list of validators for proper merge
  981. assert.Equal(t, toTestValList(valSet.Validators), tt.expectedVals, "test %v", i)
  982. }
  983. }
  984. type testVSetCfg struct {
  985. name string
  986. startVals []testVal
  987. deletedVals []testVal
  988. updatedVals []testVal
  989. addedVals []testVal
  990. expectedVals []testVal
  991. wantErr bool
  992. }
  993. func randTestVSetCfg(t *testing.T, nBase, nAddMax int) testVSetCfg {
  994. if nBase <= 0 || nAddMax < 0 {
  995. panic(fmt.Sprintf("bad parameters %v %v", nBase, nAddMax))
  996. }
  997. const maxPower = 1000
  998. var nOld, nDel, nChanged, nAdd int
  999. nOld = int(tmrand.Uint()%uint(nBase)) + 1
  1000. if nBase-nOld > 0 {
  1001. nDel = int(tmrand.Uint() % uint(nBase-nOld))
  1002. }
  1003. nChanged = nBase - nOld - nDel
  1004. if nAddMax > 0 {
  1005. nAdd = tmrand.Int()%nAddMax + 1
  1006. }
  1007. cfg := testVSetCfg{}
  1008. cfg.startVals = make([]testVal, nBase)
  1009. cfg.deletedVals = make([]testVal, nDel)
  1010. cfg.addedVals = make([]testVal, nAdd)
  1011. cfg.updatedVals = make([]testVal, nChanged)
  1012. cfg.expectedVals = make([]testVal, nBase-nDel+nAdd)
  1013. for i := 0; i < nBase; i++ {
  1014. cfg.startVals[i] = testVal{fmt.Sprintf("v%d", i), int64(tmrand.Uint()%maxPower + 1)}
  1015. if i < nOld {
  1016. cfg.expectedVals[i] = cfg.startVals[i]
  1017. }
  1018. if i >= nOld && i < nOld+nChanged {
  1019. cfg.updatedVals[i-nOld] = testVal{fmt.Sprintf("v%d", i), int64(tmrand.Uint()%maxPower + 1)}
  1020. cfg.expectedVals[i] = cfg.updatedVals[i-nOld]
  1021. }
  1022. if i >= nOld+nChanged {
  1023. cfg.deletedVals[i-nOld-nChanged] = testVal{fmt.Sprintf("v%d", i), 0}
  1024. }
  1025. }
  1026. for i := nBase; i < nBase+nAdd; i++ {
  1027. cfg.addedVals[i-nBase] = testVal{fmt.Sprintf("v%d", i), int64(tmrand.Uint()%maxPower + 1)}
  1028. cfg.expectedVals[i-nDel] = cfg.addedVals[i-nBase]
  1029. }
  1030. sort.Sort(testValsByAddress(cfg.startVals))
  1031. sort.Sort(testValsByAddress(cfg.deletedVals))
  1032. sort.Sort(testValsByAddress(cfg.updatedVals))
  1033. sort.Sort(testValsByAddress(cfg.addedVals))
  1034. sort.Sort(testValsByAddress(cfg.expectedVals))
  1035. return cfg
  1036. }
  1037. func applyChangesToValSet(t *testing.T, wantErr bool, valSet *ValidatorSet, valsLists ...[]testVal) {
  1038. changes := make([]testVal, 0)
  1039. for _, valsList := range valsLists {
  1040. changes = append(changes, valsList...)
  1041. }
  1042. valList := createNewValidatorList(changes)
  1043. err := valSet.UpdateWithChangeSet(valList)
  1044. assert.Equal(t, wantErr, err != nil, "got error %v", err)
  1045. }
  1046. func TestValSetUpdatePriorityOrderTests(t *testing.T) {
  1047. const nMaxElections = 5000
  1048. testCases := []testVSetCfg{
  1049. 0: { // remove high power validator, keep old equal lower power validators
  1050. startVals: []testVal{{"v1", 1}, {"v2", 1}, {"v3", 1000}},
  1051. deletedVals: []testVal{{"v3", 0}},
  1052. updatedVals: []testVal{},
  1053. addedVals: []testVal{},
  1054. expectedVals: []testVal{{"v1", 1}, {"v2", 1}},
  1055. },
  1056. 1: { // remove high power validator, keep old different power validators
  1057. startVals: []testVal{{"v1", 1}, {"v2", 10}, {"v3", 1000}},
  1058. deletedVals: []testVal{{"v3", 0}},
  1059. updatedVals: []testVal{},
  1060. addedVals: []testVal{},
  1061. expectedVals: []testVal{{"v1", 1}, {"v2", 10}},
  1062. },
  1063. 2: { // remove high power validator, add new low power validators, keep old lower power
  1064. startVals: []testVal{{"v1", 1}, {"v2", 2}, {"v3", 1000}},
  1065. deletedVals: []testVal{{"v3", 0}},
  1066. updatedVals: []testVal{{"v2", 1}},
  1067. addedVals: []testVal{{"v4", 40}, {"v5", 50}},
  1068. expectedVals: []testVal{{"v1", 1}, {"v2", 1}, {"v4", 40}, {"v5", 50}},
  1069. },
  1070. // generate a configuration with 100 validators,
  1071. // randomly select validators for updates and deletes, and
  1072. // generate 10 new validators to be added
  1073. 3: randTestVSetCfg(t, 100, 10),
  1074. 4: randTestVSetCfg(t, 1000, 100),
  1075. 5: randTestVSetCfg(t, 10, 100),
  1076. 6: randTestVSetCfg(t, 100, 1000),
  1077. 7: randTestVSetCfg(t, 1000, 1000),
  1078. 8: randTestVSetCfg(t, 10000, 1000),
  1079. 9: randTestVSetCfg(t, 1000, 10000),
  1080. }
  1081. for _, cfg := range testCases {
  1082. // create a new validator set
  1083. valSet := createNewValidatorSet(cfg.startVals)
  1084. verifyValidatorSet(t, valSet)
  1085. // run election up to nMaxElections times, apply changes and verify that the priority order is correct
  1086. verifyValSetUpdatePriorityOrder(t, valSet, cfg, nMaxElections)
  1087. }
  1088. }
  1089. func verifyValSetUpdatePriorityOrder(t *testing.T, valSet *ValidatorSet, cfg testVSetCfg, nMaxElections int) {
  1090. // Run election up to nMaxElections times, sort validators by priorities
  1091. valSet.IncrementProposerPriority(tmrand.Int()%nMaxElections + 1)
  1092. origValsPriSorted := validatorListCopy(valSet.Validators)
  1093. sort.Sort(validatorsByPriority(origValsPriSorted))
  1094. // apply the changes, get the updated validators, sort by priorities
  1095. applyChangesToValSet(t, false, valSet, cfg.addedVals, cfg.updatedVals, cfg.deletedVals)
  1096. updatedValsPriSorted := validatorListCopy(valSet.Validators)
  1097. sort.Sort(validatorsByPriority(updatedValsPriSorted))
  1098. // basic checks
  1099. assert.Equal(t, cfg.expectedVals, toTestValList(valSet.Validators))
  1100. verifyValidatorSet(t, valSet)
  1101. // verify that the added validators have the smallest priority:
  1102. // - they should be at the beginning of valListNewPriority since it is sorted by priority
  1103. if len(cfg.addedVals) > 0 {
  1104. addedValsPriSlice := updatedValsPriSorted[:len(cfg.addedVals)]
  1105. sort.Sort(ValidatorsByAddress(addedValsPriSlice))
  1106. assert.Equal(t, cfg.addedVals, toTestValList(addedValsPriSlice))
  1107. // - and should all have the same priority
  1108. expectedPri := addedValsPriSlice[0].ProposerPriority
  1109. for _, val := range addedValsPriSlice[1:] {
  1110. assert.Equal(t, expectedPri, val.ProposerPriority)
  1111. }
  1112. }
  1113. }
  1114. func TestValSetUpdateOverflowRelated(t *testing.T) {
  1115. testCases := []testVSetCfg{
  1116. {
  1117. name: "1 no false overflow error messages for updates",
  1118. startVals: []testVal{{"v1", 1}, {"v2", MaxTotalVotingPower - 1}},
  1119. updatedVals: []testVal{{"v1", MaxTotalVotingPower - 1}, {"v2", 1}},
  1120. expectedVals: []testVal{{"v1", MaxTotalVotingPower - 1}, {"v2", 1}},
  1121. wantErr: false,
  1122. },
  1123. {
  1124. // this test shows that it is important to apply the updates in the order of the change in power
  1125. // i.e. apply first updates with decreases in power, v2 change in this case.
  1126. name: "2 no false overflow error messages for updates",
  1127. startVals: []testVal{{"v1", 1}, {"v2", MaxTotalVotingPower - 1}},
  1128. updatedVals: []testVal{{"v1", MaxTotalVotingPower/2 - 1}, {"v2", MaxTotalVotingPower / 2}},
  1129. expectedVals: []testVal{{"v1", MaxTotalVotingPower/2 - 1}, {"v2", MaxTotalVotingPower / 2}},
  1130. wantErr: false,
  1131. },
  1132. {
  1133. name: "3 no false overflow error messages for deletes",
  1134. startVals: []testVal{{"v1", MaxTotalVotingPower - 2}, {"v2", 1}, {"v3", 1}},
  1135. deletedVals: []testVal{{"v1", 0}},
  1136. addedVals: []testVal{{"v4", MaxTotalVotingPower - 2}},
  1137. expectedVals: []testVal{{"v2", 1}, {"v3", 1}, {"v4", MaxTotalVotingPower - 2}},
  1138. wantErr: false,
  1139. },
  1140. {
  1141. name: "4 no false overflow error messages for adds, updates and deletes",
  1142. startVals: []testVal{
  1143. {"v1", MaxTotalVotingPower / 4}, {"v2", MaxTotalVotingPower / 4},
  1144. {"v3", MaxTotalVotingPower / 4}, {"v4", MaxTotalVotingPower / 4}},
  1145. deletedVals: []testVal{{"v2", 0}},
  1146. updatedVals: []testVal{
  1147. {"v1", MaxTotalVotingPower/2 - 2}, {"v3", MaxTotalVotingPower/2 - 3}, {"v4", 2}},
  1148. addedVals: []testVal{{"v5", 3}},
  1149. expectedVals: []testVal{
  1150. {"v1", MaxTotalVotingPower/2 - 2}, {"v3", MaxTotalVotingPower/2 - 3}, {"v4", 2}, {"v5", 3}},
  1151. wantErr: false,
  1152. },
  1153. {
  1154. name: "5 check panic on overflow is prevented: update 8 validators with power int64(math.MaxInt64)/8",
  1155. startVals: []testVal{
  1156. {"v1", 1}, {"v2", 1}, {"v3", 1}, {"v4", 1}, {"v5", 1},
  1157. {"v6", 1}, {"v7", 1}, {"v8", 1}, {"v9", 1}},
  1158. updatedVals: []testVal{
  1159. {"v1", MaxTotalVotingPower}, {"v2", MaxTotalVotingPower}, {"v3", MaxTotalVotingPower},
  1160. {"v4", MaxTotalVotingPower}, {"v5", MaxTotalVotingPower}, {"v6", MaxTotalVotingPower},
  1161. {"v7", MaxTotalVotingPower}, {"v8", MaxTotalVotingPower}, {"v9", 8}},
  1162. expectedVals: []testVal{
  1163. {"v1", 1}, {"v2", 1}, {"v3", 1}, {"v4", 1}, {"v5", 1},
  1164. {"v6", 1}, {"v7", 1}, {"v8", 1}, {"v9", 1}},
  1165. wantErr: true,
  1166. },
  1167. }
  1168. for _, tt := range testCases {
  1169. tt := tt
  1170. t.Run(tt.name, func(t *testing.T) {
  1171. valSet := createNewValidatorSet(tt.startVals)
  1172. verifyValidatorSet(t, valSet)
  1173. // execute update and verify returned error is as expected
  1174. applyChangesToValSet(t, tt.wantErr, valSet, tt.addedVals, tt.updatedVals, tt.deletedVals)
  1175. // verify updated validator set is as expected
  1176. assert.Equal(t, tt.expectedVals, toTestValList(valSet.Validators))
  1177. verifyValidatorSet(t, valSet)
  1178. })
  1179. }
  1180. }
  1181. //---------------------
  1182. // Sort validators by priority and address
  1183. type validatorsByPriority []*Validator
  1184. func (valz validatorsByPriority) Len() int {
  1185. return len(valz)
  1186. }
  1187. func (valz validatorsByPriority) Less(i, j int) bool {
  1188. if valz[i].ProposerPriority < valz[j].ProposerPriority {
  1189. return true
  1190. }
  1191. if valz[i].ProposerPriority > valz[j].ProposerPriority {
  1192. return false
  1193. }
  1194. return bytes.Compare(valz[i].Address, valz[j].Address) < 0
  1195. }
  1196. func (valz validatorsByPriority) Swap(i, j int) {
  1197. it := valz[i]
  1198. valz[i] = valz[j]
  1199. valz[j] = it
  1200. }
  1201. //-------------------------------------
  1202. // Sort testVal-s by address.
  1203. type testValsByAddress []testVal
  1204. func (tvals testValsByAddress) Len() int {
  1205. return len(tvals)
  1206. }
  1207. func (tvals testValsByAddress) Less(i, j int) bool {
  1208. return bytes.Compare([]byte(tvals[i].name), []byte(tvals[j].name)) == -1
  1209. }
  1210. func (tvals testValsByAddress) Swap(i, j int) {
  1211. it := tvals[i]
  1212. tvals[i] = tvals[j]
  1213. tvals[j] = it
  1214. }
  1215. //-------------------------------------
  1216. // Benchmark tests
  1217. //
  1218. func BenchmarkUpdates(b *testing.B) {
  1219. const (
  1220. n = 100
  1221. m = 2000
  1222. )
  1223. // Init with n validators
  1224. vs := make([]*Validator, n)
  1225. for j := 0; j < n; j++ {
  1226. vs[j] = newValidator([]byte(fmt.Sprintf("v%d", j)), 100)
  1227. }
  1228. valSet := NewValidatorSet(vs)
  1229. l := len(valSet.Validators)
  1230. // Make m new validators
  1231. newValList := make([]*Validator, m)
  1232. for j := 0; j < m; j++ {
  1233. newValList[j] = newValidator([]byte(fmt.Sprintf("v%d", j+l)), 1000)
  1234. }
  1235. b.ResetTimer()
  1236. for i := 0; i < b.N; i++ {
  1237. // Add m validators to valSetCopy
  1238. valSetCopy := valSet.Copy()
  1239. assert.NoError(b, valSetCopy.UpdateWithChangeSet(newValList))
  1240. }
  1241. }