You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

850 lines
26 KiB

10 years ago
10 years ago
10 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
state: add more tests for block validation (#3674) * Expose priv validators for use in testing * Generalize block header validation test past height 1 * Remove ineffectual assignment * Remove redundant SaveState call * Reorder comment for clarity * Use the block executor ApplyBlock function instead of implementing a stripped-down version of it * Remove commented-out code * Remove unnecessary test The required tests already appear to be implemented (implicitly) through the TestValidateBlockHeader test. * Allow for catching of specific error types during TestValidateBlockCommit * Make return error testable * Clean up and add TestValidateBlockCommit code * Fix formatting * Extract function to create a new mock test app * Update comment for clarity * Fix comment * Add skeleton code for evidence-related test * Allow for addressing priv val by address * Generalize test beyond a single validator * Generalize TestValidateBlockEvidence past first height * Reorder code to clearly separate tests and utility code * Use a common constant for stop height for testing in state/validation_test.go * Refactor errors to resemble existing conventions * Fix formatting * Extract common helper functions Having the tests littered with helper functions makes them less easily readable imho, so I've pulled them out into a separate file. This also makes it easier to see what helper functions are available during testing, so we minimize the chance of duplication when writing new tests. * Remove unused parameter * Remove unused parameters * Add field keys * Remove unused height constant * Fix typo * Fix incorrect return error * Add field keys * Use separate package for tests This refactors all of the state package's tests into a state_test package, so as to keep any usage of the state package's internal methods explicit. Any internal methods/constants used by tests are now explicitly exported in state/export_test.go * Refactor: extract helper function to make, validate, execute and commit a block * Rename state function to makeState * Remove redundant constant for number of validators * Refactor mock evidence registration into TestMain * Remove extraneous nVals variable * Replace function-level TODOs with file-level TODO and explanation * Remove extraneous comment * Fix linting issues brought up by GolangCI (pulled in from latest merge from develop)
5 years ago
state: add more tests for block validation (#3674) * Expose priv validators for use in testing * Generalize block header validation test past height 1 * Remove ineffectual assignment * Remove redundant SaveState call * Reorder comment for clarity * Use the block executor ApplyBlock function instead of implementing a stripped-down version of it * Remove commented-out code * Remove unnecessary test The required tests already appear to be implemented (implicitly) through the TestValidateBlockHeader test. * Allow for catching of specific error types during TestValidateBlockCommit * Make return error testable * Clean up and add TestValidateBlockCommit code * Fix formatting * Extract function to create a new mock test app * Update comment for clarity * Fix comment * Add skeleton code for evidence-related test * Allow for addressing priv val by address * Generalize test beyond a single validator * Generalize TestValidateBlockEvidence past first height * Reorder code to clearly separate tests and utility code * Use a common constant for stop height for testing in state/validation_test.go * Refactor errors to resemble existing conventions * Fix formatting * Extract common helper functions Having the tests littered with helper functions makes them less easily readable imho, so I've pulled them out into a separate file. This also makes it easier to see what helper functions are available during testing, so we minimize the chance of duplication when writing new tests. * Remove unused parameter * Remove unused parameters * Add field keys * Remove unused height constant * Fix typo * Fix incorrect return error * Add field keys * Use separate package for tests This refactors all of the state package's tests into a state_test package, so as to keep any usage of the state package's internal methods explicit. Any internal methods/constants used by tests are now explicitly exported in state/export_test.go * Refactor: extract helper function to make, validate, execute and commit a block * Rename state function to makeState * Remove redundant constant for number of validators * Refactor mock evidence registration into TestMain * Remove extraneous nVals variable * Replace function-level TODOs with file-level TODO and explanation * Remove extraneous comment * Fix linting issues brought up by GolangCI (pulled in from latest merge from develop)
5 years ago
7 years ago
7 years ago
  1. package types
  2. import (
  3. "bytes"
  4. "fmt"
  5. "math"
  6. "math/big"
  7. "sort"
  8. "strings"
  9. "github.com/pkg/errors"
  10. "github.com/tendermint/tendermint/crypto/merkle"
  11. )
  12. // MaxTotalVotingPower - the maximum allowed total voting power.
  13. // It needs to be sufficiently small to, in all cases:
  14. // 1. prevent clipping in incrementProposerPriority()
  15. // 2. let (diff+diffMax-1) not overflow in IncrementProposerPriority()
  16. // (Proof of 1 is tricky, left to the reader).
  17. // It could be higher, but this is sufficiently large for our purposes,
  18. // and leaves room for defensive purposes.
  19. // PriorityWindowSizeFactor - is a constant that when multiplied with the total voting power gives
  20. // the maximum allowed distance between validator priorities.
  21. const (
  22. MaxTotalVotingPower = int64(math.MaxInt64) / 8
  23. PriorityWindowSizeFactor = 2
  24. )
  25. // ValidatorSet represent a set of *Validator at a given height.
  26. // The validators can be fetched by address or index.
  27. // The index is in order of .Address, so the indices are fixed
  28. // for all rounds of a given blockchain height - ie. the validators
  29. // are sorted by their address.
  30. // On the other hand, the .ProposerPriority of each validator and
  31. // the designated .GetProposer() of a set changes every round,
  32. // upon calling .IncrementProposerPriority().
  33. // NOTE: Not goroutine-safe.
  34. // NOTE: All get/set to validators should copy the value for safety.
  35. type ValidatorSet struct {
  36. // NOTE: persisted via reflect, must be exported.
  37. Validators []*Validator `json:"validators"`
  38. Proposer *Validator `json:"proposer"`
  39. // cached (unexported)
  40. totalVotingPower int64
  41. }
  42. // NewValidatorSet initializes a ValidatorSet by copying over the
  43. // values from `valz`, a list of Validators. If valz is nil or empty,
  44. // the new ValidatorSet will have an empty list of Validators.
  45. // The addresses of validators in `valz` must be unique otherwise the
  46. // function panics.
  47. // Note the validator set size has an implied limit equal to that of the MaxVotesCount -
  48. // commits by a validator set larger than this will fail validation.
  49. func NewValidatorSet(valz []*Validator) *ValidatorSet {
  50. vals := &ValidatorSet{}
  51. err := vals.updateWithChangeSet(valz, false)
  52. if err != nil {
  53. panic(fmt.Sprintf("cannot create validator set: %s", err))
  54. }
  55. if len(valz) > 0 {
  56. vals.IncrementProposerPriority(1)
  57. }
  58. return vals
  59. }
  60. // Nil or empty validator sets are invalid.
  61. func (vals *ValidatorSet) IsNilOrEmpty() bool {
  62. return vals == nil || len(vals.Validators) == 0
  63. }
  64. // Increment ProposerPriority and update the proposer on a copy, and return it.
  65. func (vals *ValidatorSet) CopyIncrementProposerPriority(times int) *ValidatorSet {
  66. copy := vals.Copy()
  67. copy.IncrementProposerPriority(times)
  68. return copy
  69. }
  70. // IncrementProposerPriority increments ProposerPriority of each validator and updates the
  71. // proposer. Panics if validator set is empty.
  72. // `times` must be positive.
  73. func (vals *ValidatorSet) IncrementProposerPriority(times int) {
  74. if vals.IsNilOrEmpty() {
  75. panic("empty validator set")
  76. }
  77. if times <= 0 {
  78. panic("Cannot call IncrementProposerPriority with non-positive times")
  79. }
  80. // Cap the difference between priorities to be proportional to 2*totalPower by
  81. // re-normalizing priorities, i.e., rescale all priorities by multiplying with:
  82. // 2*totalVotingPower/(maxPriority - minPriority)
  83. diffMax := PriorityWindowSizeFactor * vals.TotalVotingPower()
  84. vals.RescalePriorities(diffMax)
  85. vals.shiftByAvgProposerPriority()
  86. var proposer *Validator
  87. // Call IncrementProposerPriority(1) times times.
  88. for i := 0; i < times; i++ {
  89. proposer = vals.incrementProposerPriority()
  90. }
  91. vals.Proposer = proposer
  92. }
  93. func (vals *ValidatorSet) RescalePriorities(diffMax int64) {
  94. if vals.IsNilOrEmpty() {
  95. panic("empty validator set")
  96. }
  97. // NOTE: This check is merely a sanity check which could be
  98. // removed if all tests would init. voting power appropriately;
  99. // i.e. diffMax should always be > 0
  100. if diffMax <= 0 {
  101. return
  102. }
  103. // Calculating ceil(diff/diffMax):
  104. // Re-normalization is performed by dividing by an integer for simplicity.
  105. // NOTE: This may make debugging priority issues easier as well.
  106. diff := computeMaxMinPriorityDiff(vals)
  107. ratio := (diff + diffMax - 1) / diffMax
  108. if diff > diffMax {
  109. for _, val := range vals.Validators {
  110. val.ProposerPriority /= ratio
  111. }
  112. }
  113. }
  114. func (vals *ValidatorSet) incrementProposerPriority() *Validator {
  115. for _, val := range vals.Validators {
  116. // Check for overflow for sum.
  117. newPrio := safeAddClip(val.ProposerPriority, val.VotingPower)
  118. val.ProposerPriority = newPrio
  119. }
  120. // Decrement the validator with most ProposerPriority.
  121. mostest := vals.getValWithMostPriority()
  122. // Mind the underflow.
  123. mostest.ProposerPriority = safeSubClip(mostest.ProposerPriority, vals.TotalVotingPower())
  124. return mostest
  125. }
  126. // Should not be called on an empty validator set.
  127. func (vals *ValidatorSet) computeAvgProposerPriority() int64 {
  128. n := int64(len(vals.Validators))
  129. sum := big.NewInt(0)
  130. for _, val := range vals.Validators {
  131. sum.Add(sum, big.NewInt(val.ProposerPriority))
  132. }
  133. avg := sum.Div(sum, big.NewInt(n))
  134. if avg.IsInt64() {
  135. return avg.Int64()
  136. }
  137. // This should never happen: each val.ProposerPriority is in bounds of int64.
  138. panic(fmt.Sprintf("Cannot represent avg ProposerPriority as an int64 %v", avg))
  139. }
  140. // Compute the difference between the max and min ProposerPriority of that set.
  141. func computeMaxMinPriorityDiff(vals *ValidatorSet) int64 {
  142. if vals.IsNilOrEmpty() {
  143. panic("empty validator set")
  144. }
  145. max := int64(math.MinInt64)
  146. min := int64(math.MaxInt64)
  147. for _, v := range vals.Validators {
  148. if v.ProposerPriority < min {
  149. min = v.ProposerPriority
  150. }
  151. if v.ProposerPriority > max {
  152. max = v.ProposerPriority
  153. }
  154. }
  155. diff := max - min
  156. if diff < 0 {
  157. return -1 * diff
  158. } else {
  159. return diff
  160. }
  161. }
  162. func (vals *ValidatorSet) getValWithMostPriority() *Validator {
  163. var res *Validator
  164. for _, val := range vals.Validators {
  165. res = res.CompareProposerPriority(val)
  166. }
  167. return res
  168. }
  169. func (vals *ValidatorSet) shiftByAvgProposerPriority() {
  170. if vals.IsNilOrEmpty() {
  171. panic("empty validator set")
  172. }
  173. avgProposerPriority := vals.computeAvgProposerPriority()
  174. for _, val := range vals.Validators {
  175. val.ProposerPriority = safeSubClip(val.ProposerPriority, avgProposerPriority)
  176. }
  177. }
  178. // Makes a copy of the validator list.
  179. func validatorListCopy(valsList []*Validator) []*Validator {
  180. if valsList == nil {
  181. return nil
  182. }
  183. valsCopy := make([]*Validator, len(valsList))
  184. for i, val := range valsList {
  185. valsCopy[i] = val.Copy()
  186. }
  187. return valsCopy
  188. }
  189. // Copy each validator into a new ValidatorSet.
  190. func (vals *ValidatorSet) Copy() *ValidatorSet {
  191. return &ValidatorSet{
  192. Validators: validatorListCopy(vals.Validators),
  193. Proposer: vals.Proposer,
  194. totalVotingPower: vals.totalVotingPower,
  195. }
  196. }
  197. // HasAddress returns true if address given is in the validator set, false -
  198. // otherwise.
  199. func (vals *ValidatorSet) HasAddress(address []byte) bool {
  200. idx := sort.Search(len(vals.Validators), func(i int) bool {
  201. return bytes.Compare(address, vals.Validators[i].Address) <= 0
  202. })
  203. return idx < len(vals.Validators) && bytes.Equal(vals.Validators[idx].Address, address)
  204. }
  205. // GetByAddress returns an index of the validator with address and validator
  206. // itself if found. Otherwise, -1 and nil are returned.
  207. func (vals *ValidatorSet) GetByAddress(address []byte) (index int, val *Validator) {
  208. idx := sort.Search(len(vals.Validators), func(i int) bool {
  209. return bytes.Compare(address, vals.Validators[i].Address) <= 0
  210. })
  211. if idx < len(vals.Validators) && bytes.Equal(vals.Validators[idx].Address, address) {
  212. return idx, vals.Validators[idx].Copy()
  213. }
  214. return -1, nil
  215. }
  216. // GetByIndex returns the validator's address and validator itself by index.
  217. // It returns nil values if index is less than 0 or greater or equal to
  218. // len(ValidatorSet.Validators).
  219. func (vals *ValidatorSet) GetByIndex(index int) (address []byte, val *Validator) {
  220. if index < 0 || index >= len(vals.Validators) {
  221. return nil, nil
  222. }
  223. val = vals.Validators[index]
  224. return val.Address, val.Copy()
  225. }
  226. // Size returns the length of the validator set.
  227. func (vals *ValidatorSet) Size() int {
  228. return len(vals.Validators)
  229. }
  230. // Force recalculation of the set's total voting power.
  231. func (vals *ValidatorSet) updateTotalVotingPower() {
  232. sum := int64(0)
  233. for _, val := range vals.Validators {
  234. // mind overflow
  235. sum = safeAddClip(sum, val.VotingPower)
  236. if sum > MaxTotalVotingPower {
  237. panic(fmt.Sprintf(
  238. "Total voting power should be guarded to not exceed %v; got: %v",
  239. MaxTotalVotingPower,
  240. sum))
  241. }
  242. }
  243. vals.totalVotingPower = sum
  244. }
  245. // TotalVotingPower returns the sum of the voting powers of all validators.
  246. // It recomputes the total voting power if required.
  247. func (vals *ValidatorSet) TotalVotingPower() int64 {
  248. if vals.totalVotingPower == 0 {
  249. vals.updateTotalVotingPower()
  250. }
  251. return vals.totalVotingPower
  252. }
  253. // GetProposer returns the current proposer. If the validator set is empty, nil
  254. // is returned.
  255. func (vals *ValidatorSet) GetProposer() (proposer *Validator) {
  256. if len(vals.Validators) == 0 {
  257. return nil
  258. }
  259. if vals.Proposer == nil {
  260. vals.Proposer = vals.findProposer()
  261. }
  262. return vals.Proposer.Copy()
  263. }
  264. func (vals *ValidatorSet) findProposer() *Validator {
  265. var proposer *Validator
  266. for _, val := range vals.Validators {
  267. if proposer == nil || !bytes.Equal(val.Address, proposer.Address) {
  268. proposer = proposer.CompareProposerPriority(val)
  269. }
  270. }
  271. return proposer
  272. }
  273. // Hash returns the Merkle root hash build using validators (as leaves) in the
  274. // set.
  275. func (vals *ValidatorSet) Hash() []byte {
  276. if len(vals.Validators) == 0 {
  277. return nil
  278. }
  279. bzs := make([][]byte, len(vals.Validators))
  280. for i, val := range vals.Validators {
  281. bzs[i] = val.Bytes()
  282. }
  283. return merkle.SimpleHashFromByteSlices(bzs)
  284. }
  285. // Iterate will run the given function over the set.
  286. func (vals *ValidatorSet) Iterate(fn func(index int, val *Validator) bool) {
  287. for i, val := range vals.Validators {
  288. stop := fn(i, val.Copy())
  289. if stop {
  290. break
  291. }
  292. }
  293. }
  294. // Checks changes against duplicates, splits the changes in updates and removals, sorts them by address.
  295. //
  296. // Returns:
  297. // updates, removals - the sorted lists of updates and removals
  298. // err - non-nil if duplicate entries or entries with negative voting power are seen
  299. //
  300. // No changes are made to 'origChanges'.
  301. func processChanges(origChanges []*Validator) (updates, removals []*Validator, err error) {
  302. // Make a deep copy of the changes and sort by address.
  303. changes := validatorListCopy(origChanges)
  304. sort.Sort(ValidatorsByAddress(changes))
  305. removals = make([]*Validator, 0, len(changes))
  306. updates = make([]*Validator, 0, len(changes))
  307. var prevAddr Address
  308. // Scan changes by address and append valid validators to updates or removals lists.
  309. for _, valUpdate := range changes {
  310. if bytes.Equal(valUpdate.Address, prevAddr) {
  311. err = fmt.Errorf("duplicate entry %v in %v", valUpdate, changes)
  312. return nil, nil, err
  313. }
  314. if valUpdate.VotingPower < 0 {
  315. err = fmt.Errorf("voting power can't be negative: %v", valUpdate)
  316. return nil, nil, err
  317. }
  318. if valUpdate.VotingPower > MaxTotalVotingPower {
  319. err = fmt.Errorf("to prevent clipping/ overflow, voting power can't be higher than %v: %v ",
  320. MaxTotalVotingPower, valUpdate)
  321. return nil, nil, err
  322. }
  323. if valUpdate.VotingPower == 0 {
  324. removals = append(removals, valUpdate)
  325. } else {
  326. updates = append(updates, valUpdate)
  327. }
  328. prevAddr = valUpdate.Address
  329. }
  330. return updates, removals, err
  331. }
  332. // Verifies a list of updates against a validator set, making sure the allowed
  333. // total voting power would not be exceeded if these updates would be applied to the set.
  334. //
  335. // Returns:
  336. // updatedTotalVotingPower - the new total voting power if these updates would be applied
  337. // numNewValidators - number of new validators
  338. // err - non-nil if the maximum allowed total voting power would be exceeded
  339. //
  340. // 'updates' should be a list of proper validator changes, i.e. they have been verified
  341. // by processChanges for duplicates and invalid values.
  342. // No changes are made to the validator set 'vals'.
  343. func verifyUpdates(
  344. updates []*Validator,
  345. vals *ValidatorSet,
  346. ) (updatedTotalVotingPower int64, numNewValidators int, err error) {
  347. updatedTotalVotingPower = vals.TotalVotingPower()
  348. for _, valUpdate := range updates {
  349. address := valUpdate.Address
  350. _, val := vals.GetByAddress(address)
  351. if val == nil {
  352. // New validator, add its voting power the the total.
  353. updatedTotalVotingPower += valUpdate.VotingPower
  354. numNewValidators++
  355. } else {
  356. // Updated validator, add the difference in power to the total.
  357. updatedTotalVotingPower += valUpdate.VotingPower - val.VotingPower
  358. }
  359. overflow := updatedTotalVotingPower > MaxTotalVotingPower
  360. if overflow {
  361. err = fmt.Errorf(
  362. "failed to add/update validator %v, total voting power would exceed the max allowed %v",
  363. valUpdate, MaxTotalVotingPower)
  364. return 0, 0, err
  365. }
  366. }
  367. return updatedTotalVotingPower, numNewValidators, nil
  368. }
  369. // Computes the proposer priority for the validators not present in the set based on 'updatedTotalVotingPower'.
  370. // Leaves unchanged the priorities of validators that are changed.
  371. //
  372. // 'updates' parameter must be a list of unique validators to be added or updated.
  373. // No changes are made to the validator set 'vals'.
  374. func computeNewPriorities(updates []*Validator, vals *ValidatorSet, updatedTotalVotingPower int64) {
  375. for _, valUpdate := range updates {
  376. address := valUpdate.Address
  377. _, val := vals.GetByAddress(address)
  378. if val == nil {
  379. // add val
  380. // Set ProposerPriority to -C*totalVotingPower (with C ~= 1.125) to make sure validators can't
  381. // un-bond and then re-bond to reset their (potentially previously negative) ProposerPriority to zero.
  382. //
  383. // Contract: updatedVotingPower < MaxTotalVotingPower to ensure ProposerPriority does
  384. // not exceed the bounds of int64.
  385. //
  386. // Compute ProposerPriority = -1.125*totalVotingPower == -(updatedVotingPower + (updatedVotingPower >> 3)).
  387. valUpdate.ProposerPriority = -(updatedTotalVotingPower + (updatedTotalVotingPower >> 3))
  388. } else {
  389. valUpdate.ProposerPriority = val.ProposerPriority
  390. }
  391. }
  392. }
  393. // Merges the vals' validator list with the updates list.
  394. // When two elements with same address are seen, the one from updates is selected.
  395. // Expects updates to be a list of updates sorted by address with no duplicates or errors,
  396. // must have been validated with verifyUpdates() and priorities computed with computeNewPriorities().
  397. func (vals *ValidatorSet) applyUpdates(updates []*Validator) {
  398. existing := vals.Validators
  399. merged := make([]*Validator, len(existing)+len(updates))
  400. i := 0
  401. for len(existing) > 0 && len(updates) > 0 {
  402. if bytes.Compare(existing[0].Address, updates[0].Address) < 0 { // unchanged validator
  403. merged[i] = existing[0]
  404. existing = existing[1:]
  405. } else {
  406. // Apply add or update.
  407. merged[i] = updates[0]
  408. if bytes.Equal(existing[0].Address, updates[0].Address) {
  409. // Validator is present in both, advance existing.
  410. existing = existing[1:]
  411. }
  412. updates = updates[1:]
  413. }
  414. i++
  415. }
  416. // Add the elements which are left.
  417. for j := 0; j < len(existing); j++ {
  418. merged[i] = existing[j]
  419. i++
  420. }
  421. // OR add updates which are left.
  422. for j := 0; j < len(updates); j++ {
  423. merged[i] = updates[j]
  424. i++
  425. }
  426. vals.Validators = merged[:i]
  427. }
  428. // Checks that the validators to be removed are part of the validator set.
  429. // No changes are made to the validator set 'vals'.
  430. func verifyRemovals(deletes []*Validator, vals *ValidatorSet) error {
  431. for _, valUpdate := range deletes {
  432. address := valUpdate.Address
  433. _, val := vals.GetByAddress(address)
  434. if val == nil {
  435. return fmt.Errorf("failed to find validator %X to remove", address)
  436. }
  437. }
  438. if len(deletes) > len(vals.Validators) {
  439. panic("more deletes than validators")
  440. }
  441. return nil
  442. }
  443. // Removes the validators specified in 'deletes' from validator set 'vals'.
  444. // Should not fail as verification has been done before.
  445. func (vals *ValidatorSet) applyRemovals(deletes []*Validator) {
  446. existing := vals.Validators
  447. merged := make([]*Validator, len(existing)-len(deletes))
  448. i := 0
  449. // Loop over deletes until we removed all of them.
  450. for len(deletes) > 0 {
  451. if bytes.Equal(existing[0].Address, deletes[0].Address) {
  452. deletes = deletes[1:]
  453. } else { // Leave it in the resulting slice.
  454. merged[i] = existing[0]
  455. i++
  456. }
  457. existing = existing[1:]
  458. }
  459. // Add the elements which are left.
  460. for j := 0; j < len(existing); j++ {
  461. merged[i] = existing[j]
  462. i++
  463. }
  464. vals.Validators = merged[:i]
  465. }
  466. // Main function used by UpdateWithChangeSet() and NewValidatorSet().
  467. // If 'allowDeletes' is false then delete operations (identified by validators with voting power 0)
  468. // are not allowed and will trigger an error if present in 'changes'.
  469. // The 'allowDeletes' flag is set to false by NewValidatorSet() and to true by UpdateWithChangeSet().
  470. func (vals *ValidatorSet) updateWithChangeSet(changes []*Validator, allowDeletes bool) error {
  471. if len(changes) == 0 {
  472. return nil
  473. }
  474. // Check for duplicates within changes, split in 'updates' and 'deletes' lists (sorted).
  475. updates, deletes, err := processChanges(changes)
  476. if err != nil {
  477. return err
  478. }
  479. if !allowDeletes && len(deletes) != 0 {
  480. return fmt.Errorf("cannot process validators with voting power 0: %v", deletes)
  481. }
  482. // Verify that applying the 'deletes' against 'vals' will not result in error.
  483. if err := verifyRemovals(deletes, vals); err != nil {
  484. return err
  485. }
  486. // Verify that applying the 'updates' against 'vals' will not result in error.
  487. updatedTotalVotingPower, numNewValidators, err := verifyUpdates(updates, vals)
  488. if err != nil {
  489. return err
  490. }
  491. // Check that the resulting set will not be empty.
  492. if numNewValidators == 0 && len(vals.Validators) == len(deletes) {
  493. return errors.New("applying the validator changes would result in empty set")
  494. }
  495. // Compute the priorities for updates.
  496. computeNewPriorities(updates, vals, updatedTotalVotingPower)
  497. // Apply updates and removals.
  498. vals.applyUpdates(updates)
  499. vals.applyRemovals(deletes)
  500. vals.updateTotalVotingPower()
  501. // Scale and center.
  502. vals.RescalePriorities(PriorityWindowSizeFactor * vals.TotalVotingPower())
  503. vals.shiftByAvgProposerPriority()
  504. return nil
  505. }
  506. // UpdateWithChangeSet attempts to update the validator set with 'changes'.
  507. // It performs the following steps:
  508. // - validates the changes making sure there are no duplicates and splits them in updates and deletes
  509. // - verifies that applying the changes will not result in errors
  510. // - computes the total voting power BEFORE removals to ensure that in the next steps the priorities
  511. // across old and newly added validators are fair
  512. // - computes the priorities of new validators against the final set
  513. // - applies the updates against the validator set
  514. // - applies the removals against the validator set
  515. // - performs scaling and centering of priority values
  516. // If an error is detected during verification steps, it is returned and the validator set
  517. // is not changed.
  518. func (vals *ValidatorSet) UpdateWithChangeSet(changes []*Validator) error {
  519. return vals.updateWithChangeSet(changes, true)
  520. }
  521. // Verify that +2/3 of the set had signed the given signBytes.
  522. func (vals *ValidatorSet) VerifyCommit(chainID string, blockID BlockID, height int64, commit *Commit) error {
  523. if err := commit.ValidateBasic(); err != nil {
  524. return err
  525. }
  526. if vals.Size() != len(commit.Precommits) {
  527. return NewErrInvalidCommitPrecommits(vals.Size(), len(commit.Precommits))
  528. }
  529. if height != commit.Height() {
  530. return NewErrInvalidCommitHeight(height, commit.Height())
  531. }
  532. if !blockID.Equals(commit.BlockID) {
  533. return fmt.Errorf("Invalid commit -- wrong block id: want %v got %v",
  534. blockID, commit.BlockID)
  535. }
  536. talliedVotingPower := int64(0)
  537. for idx, precommit := range commit.Precommits {
  538. if precommit == nil {
  539. continue // OK, some precommits can be missing.
  540. }
  541. _, val := vals.GetByIndex(idx)
  542. // Validate signature.
  543. precommitSignBytes := commit.VoteSignBytes(chainID, idx)
  544. if !val.PubKey.VerifyBytes(precommitSignBytes, precommit.Signature) {
  545. return fmt.Errorf("Invalid commit -- invalid signature: %v", precommit)
  546. }
  547. // Good precommit!
  548. if blockID.Equals(precommit.BlockID) {
  549. talliedVotingPower += val.VotingPower
  550. }
  551. // else {
  552. // It's OK that the BlockID doesn't match. We include stray
  553. // precommits to measure validator availability.
  554. // }
  555. }
  556. if talliedVotingPower > vals.TotalVotingPower()*2/3 {
  557. return nil
  558. }
  559. return errTooMuchChange{talliedVotingPower, vals.TotalVotingPower()*2/3 + 1}
  560. }
  561. // VerifyFutureCommit will check to see if the set would be valid with a different
  562. // validator set.
  563. //
  564. // vals is the old validator set that we know. Over 2/3 of the power in old
  565. // signed this block.
  566. //
  567. // In Tendermint, 1/3 of the voting power can halt or fork the chain, but 1/3
  568. // can't make arbitrary state transitions. You still need > 2/3 Byzantine to
  569. // make arbitrary state transitions.
  570. //
  571. // To preserve this property in the light client, we also require > 2/3 of the
  572. // old vals to sign the future commit at H, that way we preserve the property
  573. // that if they weren't being truthful about the validator set at H (block hash
  574. // -> vals hash) or about the app state (block hash -> app hash) we can slash
  575. // > 2/3. Otherwise, the lite client isn't providing the same security
  576. // guarantees.
  577. //
  578. // Even if we added a slashing condition that if you sign a block header with
  579. // the wrong validator set, then we would only need > 1/3 of signatures from
  580. // the old vals on the new commit, it wouldn't be sufficient because the new
  581. // vals can be arbitrary and commit some arbitrary app hash.
  582. //
  583. // newSet is the validator set that signed this block. Only votes from new are
  584. // sufficient for 2/3 majority in the new set as well, for it to be a valid
  585. // commit.
  586. //
  587. // NOTE: This doesn't check whether the commit is a future commit, because the
  588. // current height isn't part of the ValidatorSet. Caller must check that the
  589. // commit height is greater than the height for this validator set.
  590. func (vals *ValidatorSet) VerifyFutureCommit(newSet *ValidatorSet, chainID string,
  591. blockID BlockID, height int64, commit *Commit) error {
  592. oldVals := vals
  593. // Commit must be a valid commit for newSet.
  594. err := newSet.VerifyCommit(chainID, blockID, height, commit)
  595. if err != nil {
  596. return err
  597. }
  598. // Check old voting power.
  599. oldVotingPower := int64(0)
  600. seen := map[int]bool{}
  601. round := commit.Round()
  602. for idx, precommit := range commit.Precommits {
  603. if precommit == nil {
  604. continue
  605. }
  606. if precommit.Height != height {
  607. return errors.Errorf("Blocks don't match - %d vs %d", round, precommit.Round)
  608. }
  609. if precommit.Round != round {
  610. return errors.Errorf("Invalid commit -- wrong round: %v vs %v", round, precommit.Round)
  611. }
  612. if precommit.Type != PrecommitType {
  613. return errors.Errorf("Invalid commit -- not precommit @ index %v", idx)
  614. }
  615. // See if this validator is in oldVals.
  616. oldIdx, val := oldVals.GetByAddress(precommit.ValidatorAddress)
  617. if val == nil || seen[oldIdx] {
  618. continue // missing or double vote...
  619. }
  620. seen[oldIdx] = true
  621. // Validate signature.
  622. precommitSignBytes := commit.VoteSignBytes(chainID, idx)
  623. if !val.PubKey.VerifyBytes(precommitSignBytes, precommit.Signature) {
  624. return errors.Errorf("Invalid commit -- invalid signature: %v", precommit)
  625. }
  626. // Good precommit!
  627. if blockID.Equals(precommit.BlockID) {
  628. oldVotingPower += val.VotingPower
  629. }
  630. // else {
  631. // It's OK that the BlockID doesn't match. We include stray
  632. // precommits to measure validator availability.
  633. // }
  634. }
  635. if oldVotingPower <= oldVals.TotalVotingPower()*2/3 {
  636. return errTooMuchChange{oldVotingPower, oldVals.TotalVotingPower()*2/3 + 1}
  637. }
  638. return nil
  639. }
  640. //-----------------
  641. // ErrTooMuchChange
  642. func IsErrTooMuchChange(err error) bool {
  643. _, ok := errors.Cause(err).(errTooMuchChange)
  644. return ok
  645. }
  646. type errTooMuchChange struct {
  647. got int64
  648. needed int64
  649. }
  650. func (e errTooMuchChange) Error() string {
  651. return fmt.Sprintf("Invalid commit -- insufficient old voting power: got %v, needed %v", e.got, e.needed)
  652. }
  653. //----------------
  654. func (vals *ValidatorSet) String() string {
  655. return vals.StringIndented("")
  656. }
  657. // String
  658. func (vals *ValidatorSet) StringIndented(indent string) string {
  659. if vals == nil {
  660. return "nil-ValidatorSet"
  661. }
  662. var valStrings []string
  663. vals.Iterate(func(index int, val *Validator) bool {
  664. valStrings = append(valStrings, val.String())
  665. return false
  666. })
  667. return fmt.Sprintf(`ValidatorSet{
  668. %s Proposer: %v
  669. %s Validators:
  670. %s %v
  671. %s}`,
  672. indent, vals.GetProposer().String(),
  673. indent,
  674. indent, strings.Join(valStrings, "\n"+indent+" "),
  675. indent)
  676. }
  677. //-------------------------------------
  678. // Implements sort for sorting validators by address.
  679. // Sort validators by address.
  680. type ValidatorsByAddress []*Validator
  681. func (valz ValidatorsByAddress) Len() int {
  682. return len(valz)
  683. }
  684. func (valz ValidatorsByAddress) Less(i, j int) bool {
  685. return bytes.Compare(valz[i].Address, valz[j].Address) == -1
  686. }
  687. func (valz ValidatorsByAddress) Swap(i, j int) {
  688. it := valz[i]
  689. valz[i] = valz[j]
  690. valz[j] = it
  691. }
  692. //----------------------------------------
  693. // for testing
  694. // RandValidatorSet returns a randomized validator set, useful for testing.
  695. // NOTE: PrivValidator are in order.
  696. // UNSTABLE
  697. func RandValidatorSet(numValidators int, votingPower int64) (*ValidatorSet, []PrivValidator) {
  698. valz := make([]*Validator, numValidators)
  699. privValidators := make([]PrivValidator, numValidators)
  700. for i := 0; i < numValidators; i++ {
  701. val, privValidator := RandValidator(false, votingPower)
  702. valz[i] = val
  703. privValidators[i] = privValidator
  704. }
  705. vals := NewValidatorSet(valz)
  706. sort.Sort(PrivValidatorsByAddress(privValidators))
  707. return vals, privValidators
  708. }
  709. ///////////////////////////////////////////////////////////////////////////////
  710. // safe addition/subtraction
  711. func safeAdd(a, b int64) (int64, bool) {
  712. if b > 0 && a > math.MaxInt64-b {
  713. return -1, true
  714. } else if b < 0 && a < math.MinInt64-b {
  715. return -1, true
  716. }
  717. return a + b, false
  718. }
  719. func safeSub(a, b int64) (int64, bool) {
  720. if b > 0 && a < math.MinInt64+b {
  721. return -1, true
  722. } else if b < 0 && a > math.MaxInt64+b {
  723. return -1, true
  724. }
  725. return a - b, false
  726. }
  727. func safeAddClip(a, b int64) int64 {
  728. c, overflow := safeAdd(a, b)
  729. if overflow {
  730. if b < 0 {
  731. return math.MinInt64
  732. }
  733. return math.MaxInt64
  734. }
  735. return c
  736. }
  737. func safeSubClip(a, b int64) int64 {
  738. c, overflow := safeSub(a, b)
  739. if overflow {
  740. if b > 0 {
  741. return math.MinInt64
  742. }
  743. return math.MaxInt64
  744. }
  745. return c
  746. }