You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

848 lines
26 KiB

10 years ago
10 years ago
10 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
7 years ago
7 years ago
  1. package types
  2. import (
  3. "bytes"
  4. "fmt"
  5. "math"
  6. "math/big"
  7. "sort"
  8. "strings"
  9. "github.com/tendermint/tendermint/crypto/merkle"
  10. cmn "github.com/tendermint/tendermint/libs/common"
  11. )
  12. // MaxTotalVotingPower - the maximum allowed total voting power.
  13. // It needs to be sufficiently small to, in all cases:
  14. // 1. prevent clipping in incrementProposerPriority()
  15. // 2. let (diff+diffMax-1) not overflow in IncrementProposerPriority()
  16. // (Proof of 1 is tricky, left to the reader).
  17. // It could be higher, but this is sufficiently large for our purposes,
  18. // and leaves room for defensive purposes.
  19. // PriorityWindowSizeFactor - is a constant that when multiplied with the total voting power gives
  20. // the maximum allowed distance between validator priorities.
  21. const (
  22. MaxTotalVotingPower = int64(math.MaxInt64) / 8
  23. PriorityWindowSizeFactor = 2
  24. )
  25. // ValidatorSet represent a set of *Validator at a given height.
  26. // The validators can be fetched by address or index.
  27. // The index is in order of .Address, so the indices are fixed
  28. // for all rounds of a given blockchain height.
  29. // On the other hand, the .ProposerPriority of each validator and
  30. // the designated .GetProposer() of a set changes every round,
  31. // upon calling .IncrementProposerPriority().
  32. // NOTE: Not goroutine-safe.
  33. // NOTE: All get/set to validators should copy the value for safety.
  34. type ValidatorSet struct {
  35. // NOTE: persisted via reflect, must be exported.
  36. Validators []*Validator `json:"validators"`
  37. Proposer *Validator `json:"proposer"`
  38. // cached (unexported)
  39. totalVotingPower int64
  40. }
  41. // NewValidatorSet initializes a ValidatorSet by copying over the
  42. // values from `valz`, a list of Validators. If valz is nil or empty,
  43. // the new ValidatorSet will have an empty list of Validators.
  44. // The addresses of validators in `valz` must be unique otherwise the
  45. // function panics.
  46. func NewValidatorSet(valz []*Validator) *ValidatorSet {
  47. vals := &ValidatorSet{}
  48. err := vals.updateWithChangeSet(valz, false)
  49. if err != nil {
  50. panic(fmt.Sprintf("cannot create validator set: %s", err))
  51. }
  52. if len(valz) > 0 {
  53. vals.IncrementProposerPriority(1)
  54. }
  55. return vals
  56. }
  57. // Nil or empty validator sets are invalid.
  58. func (vals *ValidatorSet) IsNilOrEmpty() bool {
  59. return vals == nil || len(vals.Validators) == 0
  60. }
  61. // Increment ProposerPriority and update the proposer on a copy, and return it.
  62. func (vals *ValidatorSet) CopyIncrementProposerPriority(times int) *ValidatorSet {
  63. copy := vals.Copy()
  64. copy.IncrementProposerPriority(times)
  65. return copy
  66. }
  67. // IncrementProposerPriority increments ProposerPriority of each validator and updates the
  68. // proposer. Panics if validator set is empty.
  69. // `times` must be positive.
  70. func (vals *ValidatorSet) IncrementProposerPriority(times int) {
  71. if vals.IsNilOrEmpty() {
  72. panic("empty validator set")
  73. }
  74. if times <= 0 {
  75. panic("Cannot call IncrementProposerPriority with non-positive times")
  76. }
  77. // Cap the difference between priorities to be proportional to 2*totalPower by
  78. // re-normalizing priorities, i.e., rescale all priorities by multiplying with:
  79. // 2*totalVotingPower/(maxPriority - minPriority)
  80. diffMax := PriorityWindowSizeFactor * vals.TotalVotingPower()
  81. vals.RescalePriorities(diffMax)
  82. vals.shiftByAvgProposerPriority()
  83. var proposer *Validator
  84. // call IncrementProposerPriority(1) times times:
  85. for i := 0; i < times; i++ {
  86. proposer = vals.incrementProposerPriority()
  87. }
  88. vals.Proposer = proposer
  89. }
  90. func (vals *ValidatorSet) RescalePriorities(diffMax int64) {
  91. if vals.IsNilOrEmpty() {
  92. panic("empty validator set")
  93. }
  94. // NOTE: This check is merely a sanity check which could be
  95. // removed if all tests would init. voting power appropriately;
  96. // i.e. diffMax should always be > 0
  97. if diffMax <= 0 {
  98. return
  99. }
  100. // Calculating ceil(diff/diffMax):
  101. // Re-normalization is performed by dividing by an integer for simplicity.
  102. // NOTE: This may make debugging priority issues easier as well.
  103. diff := computeMaxMinPriorityDiff(vals)
  104. ratio := (diff + diffMax - 1) / diffMax
  105. if ratio > 1 {
  106. for _, val := range vals.Validators {
  107. val.ProposerPriority /= ratio
  108. }
  109. }
  110. }
  111. func (vals *ValidatorSet) incrementProposerPriority() *Validator {
  112. for _, val := range vals.Validators {
  113. // Check for overflow for sum.
  114. newPrio := safeAddClip(val.ProposerPriority, val.VotingPower)
  115. val.ProposerPriority = newPrio
  116. }
  117. // Decrement the validator with most ProposerPriority:
  118. mostest := vals.getValWithMostPriority()
  119. // mind underflow
  120. mostest.ProposerPriority = safeSubClip(mostest.ProposerPriority, vals.TotalVotingPower())
  121. return mostest
  122. }
  123. // should not be called on an empty validator set
  124. func (vals *ValidatorSet) computeAvgProposerPriority() int64 {
  125. n := int64(len(vals.Validators))
  126. sum := big.NewInt(0)
  127. for _, val := range vals.Validators {
  128. sum.Add(sum, big.NewInt(val.ProposerPriority))
  129. }
  130. avg := sum.Div(sum, big.NewInt(n))
  131. if avg.IsInt64() {
  132. return avg.Int64()
  133. }
  134. // this should never happen: each val.ProposerPriority is in bounds of int64
  135. panic(fmt.Sprintf("Cannot represent avg ProposerPriority as an int64 %v", avg))
  136. }
  137. // compute the difference between the max and min ProposerPriority of that set
  138. func computeMaxMinPriorityDiff(vals *ValidatorSet) int64 {
  139. if vals.IsNilOrEmpty() {
  140. panic("empty validator set")
  141. }
  142. max := int64(math.MinInt64)
  143. min := int64(math.MaxInt64)
  144. for _, v := range vals.Validators {
  145. if v.ProposerPriority < min {
  146. min = v.ProposerPriority
  147. }
  148. if v.ProposerPriority > max {
  149. max = v.ProposerPriority
  150. }
  151. }
  152. diff := max - min
  153. if diff < 0 {
  154. return -1 * diff
  155. } else {
  156. return diff
  157. }
  158. }
  159. func (vals *ValidatorSet) getValWithMostPriority() *Validator {
  160. var res *Validator
  161. for _, val := range vals.Validators {
  162. res = res.CompareProposerPriority(val)
  163. }
  164. return res
  165. }
  166. func (vals *ValidatorSet) shiftByAvgProposerPriority() {
  167. if vals.IsNilOrEmpty() {
  168. panic("empty validator set")
  169. }
  170. avgProposerPriority := vals.computeAvgProposerPriority()
  171. for _, val := range vals.Validators {
  172. val.ProposerPriority = safeSubClip(val.ProposerPriority, avgProposerPriority)
  173. }
  174. }
  175. // Makes a copy of the validator list
  176. func validatorListCopy(valsList []*Validator) []*Validator {
  177. if valsList == nil {
  178. return nil
  179. }
  180. valsCopy := make([]*Validator, len(valsList))
  181. for i, val := range valsList {
  182. valsCopy[i] = val.Copy()
  183. }
  184. return valsCopy
  185. }
  186. // Copy each validator into a new ValidatorSet
  187. func (vals *ValidatorSet) Copy() *ValidatorSet {
  188. return &ValidatorSet{
  189. Validators: validatorListCopy(vals.Validators),
  190. Proposer: vals.Proposer,
  191. totalVotingPower: vals.totalVotingPower,
  192. }
  193. }
  194. // HasAddress returns true if address given is in the validator set, false -
  195. // otherwise.
  196. func (vals *ValidatorSet) HasAddress(address []byte) bool {
  197. idx := sort.Search(len(vals.Validators), func(i int) bool {
  198. return bytes.Compare(address, vals.Validators[i].Address) <= 0
  199. })
  200. return idx < len(vals.Validators) && bytes.Equal(vals.Validators[idx].Address, address)
  201. }
  202. // GetByAddress returns an index of the validator with address and validator
  203. // itself if found. Otherwise, -1 and nil are returned.
  204. func (vals *ValidatorSet) GetByAddress(address []byte) (index int, val *Validator) {
  205. idx := sort.Search(len(vals.Validators), func(i int) bool {
  206. return bytes.Compare(address, vals.Validators[i].Address) <= 0
  207. })
  208. if idx < len(vals.Validators) && bytes.Equal(vals.Validators[idx].Address, address) {
  209. return idx, vals.Validators[idx].Copy()
  210. }
  211. return -1, nil
  212. }
  213. // GetByIndex returns the validator's address and validator itself by index.
  214. // It returns nil values if index is less than 0 or greater or equal to
  215. // len(ValidatorSet.Validators).
  216. func (vals *ValidatorSet) GetByIndex(index int) (address []byte, val *Validator) {
  217. if index < 0 || index >= len(vals.Validators) {
  218. return nil, nil
  219. }
  220. val = vals.Validators[index]
  221. return val.Address, val.Copy()
  222. }
  223. // Size returns the length of the validator set.
  224. func (vals *ValidatorSet) Size() int {
  225. return len(vals.Validators)
  226. }
  227. // TotalVotingPower returns the sum of the voting powers of all validators.
  228. func (vals *ValidatorSet) TotalVotingPower() int64 {
  229. if vals.totalVotingPower == 0 {
  230. sum := int64(0)
  231. for _, val := range vals.Validators {
  232. // mind overflow
  233. sum = safeAddClip(sum, val.VotingPower)
  234. }
  235. if sum > MaxTotalVotingPower {
  236. panic(fmt.Sprintf(
  237. "Total voting power should be guarded to not exceed %v; got: %v",
  238. MaxTotalVotingPower,
  239. sum))
  240. }
  241. vals.totalVotingPower = sum
  242. }
  243. return vals.totalVotingPower
  244. }
  245. // GetProposer returns the current proposer. If the validator set is empty, nil
  246. // is returned.
  247. func (vals *ValidatorSet) GetProposer() (proposer *Validator) {
  248. if len(vals.Validators) == 0 {
  249. return nil
  250. }
  251. if vals.Proposer == nil {
  252. vals.Proposer = vals.findProposer()
  253. }
  254. return vals.Proposer.Copy()
  255. }
  256. func (vals *ValidatorSet) findProposer() *Validator {
  257. var proposer *Validator
  258. for _, val := range vals.Validators {
  259. if proposer == nil || !bytes.Equal(val.Address, proposer.Address) {
  260. proposer = proposer.CompareProposerPriority(val)
  261. }
  262. }
  263. return proposer
  264. }
  265. // Hash returns the Merkle root hash build using validators (as leaves) in the
  266. // set.
  267. func (vals *ValidatorSet) Hash() []byte {
  268. if len(vals.Validators) == 0 {
  269. return nil
  270. }
  271. bzs := make([][]byte, len(vals.Validators))
  272. for i, val := range vals.Validators {
  273. bzs[i] = val.Bytes()
  274. }
  275. return merkle.SimpleHashFromByteSlices(bzs)
  276. }
  277. // Remove deletes the validator with address. It returns the validator removed
  278. // and true. If returns nil and false if validator is not present in the set.
  279. func (vals *ValidatorSet) Remove(address []byte) (val *Validator, removed bool) {
  280. idx := sort.Search(len(vals.Validators), func(i int) bool {
  281. return bytes.Compare(address, vals.Validators[i].Address) <= 0
  282. })
  283. if idx >= len(vals.Validators) || !bytes.Equal(vals.Validators[idx].Address, address) {
  284. return nil, false
  285. }
  286. removedVal := vals.Validators[idx]
  287. newValidators := vals.Validators[:idx]
  288. if idx+1 < len(vals.Validators) {
  289. newValidators = append(newValidators, vals.Validators[idx+1:]...)
  290. }
  291. vals.Validators = newValidators
  292. // Invalidate cache
  293. vals.Proposer = nil
  294. vals.totalVotingPower = 0
  295. return removedVal, true
  296. }
  297. // Iterate will run the given function over the set.
  298. func (vals *ValidatorSet) Iterate(fn func(index int, val *Validator) bool) {
  299. for i, val := range vals.Validators {
  300. stop := fn(i, val.Copy())
  301. if stop {
  302. break
  303. }
  304. }
  305. }
  306. // Checks changes against duplicates, splits the changes in updates and removals, sorts them by address
  307. //
  308. // Returns:
  309. // updates, removals - the sorted lists of updates and removals
  310. // err - non-nil if duplicate entries or entries with negative voting power are seen
  311. //
  312. // No changes are made to 'origChanges'
  313. func processChanges(origChanges []*Validator) (updates, removals []*Validator, err error) {
  314. // Make a deep copy of the changes and sort by address
  315. changes := validatorListCopy(origChanges)
  316. sort.Sort(ValidatorsByAddress(changes))
  317. removals = make([]*Validator, 0, len(changes))
  318. updates = make([]*Validator, 0, len(changes))
  319. var prevAddr Address
  320. // Scan changes by address and append valid validators to updates or removals lists
  321. for _, valUpdate := range changes {
  322. if bytes.Equal(valUpdate.Address, prevAddr) {
  323. err = fmt.Errorf("duplicate entry %v in %v", valUpdate, changes)
  324. return nil, nil, err
  325. }
  326. if valUpdate.VotingPower < 0 {
  327. err = fmt.Errorf("voting power can't be negative %v", valUpdate)
  328. return nil, nil, err
  329. }
  330. if valUpdate.VotingPower == 0 {
  331. removals = append(removals, valUpdate)
  332. } else {
  333. updates = append(updates, valUpdate)
  334. }
  335. prevAddr = valUpdate.Address
  336. }
  337. return updates, removals, err
  338. }
  339. // Verifies a list of updates against a validator set, making sure the allowed
  340. // total voting power would not be exceeded if these updates would be applied to the set.
  341. // It also computes the total voting power of the set that would result after the updates but
  342. // before the removals.
  343. //
  344. // Returns:
  345. // updatedTotalVotingPower - the new total voting power if these updates would be applied
  346. // err - non-nil if the maximum allowed total voting power would be exceeded
  347. //
  348. // 'updates' should be a list of proper validator changes, i.e. they have been scanned
  349. // by processChanges for duplicates and invalid values.
  350. // No changes are made to the validator set 'vals'.
  351. func verifyUpdates(updates []*Validator, vals *ValidatorSet) (updatedTotalVotingPower int64, err error) {
  352. // Scan the updates, compute new total voting power, check for overflow
  353. updatedTotalVotingPower = vals.TotalVotingPower()
  354. for _, valUpdate := range updates {
  355. address := valUpdate.Address
  356. _, val := vals.GetByAddress(address)
  357. if val == nil {
  358. // new validator, add its voting power the the total
  359. updatedTotalVotingPower += valUpdate.VotingPower
  360. } else {
  361. // updated validator, add the difference in power to the total
  362. updatedTotalVotingPower += valUpdate.VotingPower - val.VotingPower
  363. }
  364. if updatedTotalVotingPower < 0 {
  365. err = fmt.Errorf(
  366. "failed to add/update validator with negative voting power %v",
  367. valUpdate)
  368. return 0, err
  369. }
  370. overflow := updatedTotalVotingPower > MaxTotalVotingPower
  371. if overflow {
  372. err = fmt.Errorf(
  373. "failed to add/update validator %v, total voting power would exceed the max allowed %v",
  374. valUpdate, MaxTotalVotingPower)
  375. return 0, err
  376. }
  377. }
  378. return updatedTotalVotingPower, nil
  379. }
  380. // Computes the proposer priority for the validators not present in the set based on 'updatedTotalVotingPower'
  381. // Leaves unchanged the priorities of validators that are changed.
  382. //
  383. // 'updates' parameter must be a list of unique validators to be added or updated.
  384. // No changes are made to the validator set 'vals'.
  385. func computeNewPriorities(updates []*Validator, vals *ValidatorSet, updatedTotalVotingPower int64) int {
  386. numNew := 0
  387. // Scan and update the proposerPriority for newly added and updated validators
  388. for _, valUpdate := range updates {
  389. address := valUpdate.Address
  390. _, val := vals.GetByAddress(address)
  391. if val == nil {
  392. // add val
  393. // Set ProposerPriority to -C*totalVotingPower (with C ~= 1.125) to make sure validators can't
  394. // un-bond and then re-bond to reset their (potentially previously negative) ProposerPriority to zero.
  395. //
  396. // Contract: updatedVotingPower < MaxTotalVotingPower to ensure ProposerPriority does
  397. // not exceed the bounds of int64.
  398. //
  399. // Compute ProposerPriority = -1.125*totalVotingPower == -(updatedVotingPower + (updatedVotingPower >> 3)).
  400. valUpdate.ProposerPriority = -(updatedTotalVotingPower + (updatedTotalVotingPower >> 3))
  401. numNew++
  402. } else {
  403. valUpdate.ProposerPriority = val.ProposerPriority
  404. }
  405. }
  406. return numNew
  407. }
  408. // Merges the vals' validator list with the updates list.
  409. // When two elements with same address are seen, the one from updates is selected.
  410. // Expects updates to be a list of updates sorted by address with no duplicates or errors,
  411. // must have been validated with verifyUpdates() and priorities computed with computeNewPriorities().
  412. func (vals *ValidatorSet) applyUpdates(updates []*Validator) {
  413. existing := make([]*Validator, len(vals.Validators))
  414. copy(existing, vals.Validators)
  415. merged := make([]*Validator, len(existing)+len(updates))
  416. i := 0
  417. for len(existing) > 0 && len(updates) > 0 {
  418. if bytes.Compare(existing[0].Address, updates[0].Address) < 0 {
  419. merged[i] = existing[0]
  420. existing = existing[1:]
  421. } else {
  422. merged[i] = updates[0]
  423. if bytes.Equal(existing[0].Address, updates[0].Address) {
  424. // validator present in both, advance existing
  425. existing = existing[1:]
  426. }
  427. updates = updates[1:]
  428. }
  429. i++
  430. }
  431. for j := 0; j < len(existing); j++ {
  432. merged[i] = existing[j]
  433. i++
  434. }
  435. for j := 0; j < len(updates); j++ {
  436. merged[i] = updates[j]
  437. i++
  438. }
  439. vals.Validators = merged[:i]
  440. vals.totalVotingPower = 0
  441. }
  442. // Checks that the validators to be removed are part of the validator set.
  443. // No changes are made to the validator set 'vals'.
  444. func verifyRemovals(deletes []*Validator, vals *ValidatorSet) error {
  445. for _, valUpdate := range deletes {
  446. address := valUpdate.Address
  447. _, val := vals.GetByAddress(address)
  448. if val == nil {
  449. return fmt.Errorf("failed to find validator %X to remove", address)
  450. }
  451. }
  452. return nil
  453. }
  454. // Removes the validators specified in 'deletes' from validator set 'vals'.
  455. // Should not fail as verification has been done before.
  456. func (vals *ValidatorSet) applyRemovals(deletes []*Validator) {
  457. for _, valUpdate := range deletes {
  458. address := valUpdate.Address
  459. _, removed := vals.Remove(address)
  460. if !removed {
  461. // Should never happen
  462. panic(fmt.Sprintf("failed to remove validator %X", address))
  463. }
  464. }
  465. }
  466. // UpdateWithChangeSet attempts to update the validator set with 'changes'
  467. // It performs the following steps:
  468. // - validates the changes making sure there are no duplicates and splits them in updates and deletes
  469. // - verifies that applying the changes will not result in errors
  470. // - computes the total voting power BEFORE removals to ensure that in the next steps the relative priorities
  471. // across old and newly added validators is fair
  472. // - computes the priorities of new validators against the final set
  473. // - applies the updates against the validator set
  474. // - applies the removals against the validator set
  475. // - performs scaling and centering of priority values
  476. // If error is detected during verification steps it is returned and the validator set
  477. // is not changed.
  478. func (vals *ValidatorSet) UpdateWithChangeSet(changes []*Validator) error {
  479. return vals.updateWithChangeSet(changes, true)
  480. }
  481. // main function used by UpdateWithChangeSet() and NewValidatorSet()
  482. // If 'allowDeletes' is false then delete operations are not allowed and must be reported if
  483. // present in 'changes'
  484. func (vals *ValidatorSet) updateWithChangeSet(changes []*Validator, allowDeletes bool) error {
  485. if len(changes) <= 0 {
  486. return nil
  487. }
  488. // Check for duplicates within changes, split in 'updates' and 'deletes' lists (sorted)
  489. updates, deletes, err := processChanges(changes)
  490. if err != nil {
  491. return err
  492. }
  493. if !allowDeletes && len(deletes) != 0 {
  494. err = fmt.Errorf("cannot process validators with voting power 0: %v", deletes)
  495. return err
  496. }
  497. // Verify that applying the 'deletes' against 'vals' will not result in error.
  498. if err := verifyRemovals(deletes, vals); err != nil {
  499. return err
  500. }
  501. // Verify that applying the 'updates' against 'vals' will not result in error.
  502. updatedTotalVotingPower, err := verifyUpdates(updates, vals)
  503. if err != nil {
  504. return err
  505. }
  506. // Compute the priorities for updates
  507. numNewValidators := computeNewPriorities(updates, vals, updatedTotalVotingPower)
  508. if len(vals.Validators)+numNewValidators <= len(deletes) {
  509. err = fmt.Errorf("applying the validator changes would result in empty set")
  510. return err
  511. }
  512. // Apply updates and removals
  513. vals.applyUpdates(updates)
  514. vals.applyRemovals(deletes)
  515. // Scale and center
  516. vals.RescalePriorities(PriorityWindowSizeFactor * vals.TotalVotingPower())
  517. vals.shiftByAvgProposerPriority()
  518. return nil
  519. }
  520. // Verify that +2/3 of the set had signed the given signBytes.
  521. func (vals *ValidatorSet) VerifyCommit(chainID string, blockID BlockID, height int64, commit *Commit) error {
  522. if err := commit.ValidateBasic(); err != nil {
  523. return err
  524. }
  525. if vals.Size() != len(commit.Precommits) {
  526. return fmt.Errorf("Invalid commit -- wrong set size: %v vs %v", vals.Size(), len(commit.Precommits))
  527. }
  528. if height != commit.Height() {
  529. return fmt.Errorf("Invalid commit -- wrong height: %v vs %v", height, commit.Height())
  530. }
  531. if !blockID.Equals(commit.BlockID) {
  532. return fmt.Errorf("Invalid commit -- wrong block id: want %v got %v",
  533. blockID, commit.BlockID)
  534. }
  535. talliedVotingPower := int64(0)
  536. for idx, precommit := range commit.Precommits {
  537. if precommit == nil {
  538. continue // OK, some precommits can be missing.
  539. }
  540. _, val := vals.GetByIndex(idx)
  541. // Validate signature.
  542. precommitSignBytes := commit.VoteSignBytes(chainID, precommit)
  543. if !val.PubKey.VerifyBytes(precommitSignBytes, precommit.Signature) {
  544. return fmt.Errorf("Invalid commit -- invalid signature: %v", precommit)
  545. }
  546. // Good precommit!
  547. if blockID.Equals(precommit.BlockID) {
  548. talliedVotingPower += val.VotingPower
  549. } else {
  550. // It's OK that the BlockID doesn't match. We include stray
  551. // precommits to measure validator availability.
  552. }
  553. }
  554. if talliedVotingPower > vals.TotalVotingPower()*2/3 {
  555. return nil
  556. }
  557. return errTooMuchChange{talliedVotingPower, vals.TotalVotingPower()*2/3 + 1}
  558. }
  559. // VerifyFutureCommit will check to see if the set would be valid with a different
  560. // validator set.
  561. //
  562. // vals is the old validator set that we know. Over 2/3 of the power in old
  563. // signed this block.
  564. //
  565. // In Tendermint, 1/3 of the voting power can halt or fork the chain, but 1/3
  566. // can't make arbitrary state transitions. You still need > 2/3 Byzantine to
  567. // make arbitrary state transitions.
  568. //
  569. // To preserve this property in the light client, we also require > 2/3 of the
  570. // old vals to sign the future commit at H, that way we preserve the property
  571. // that if they weren't being truthful about the validator set at H (block hash
  572. // -> vals hash) or about the app state (block hash -> app hash) we can slash
  573. // > 2/3. Otherwise, the lite client isn't providing the same security
  574. // guarantees.
  575. //
  576. // Even if we added a slashing condition that if you sign a block header with
  577. // the wrong validator set, then we would only need > 1/3 of signatures from
  578. // the old vals on the new commit, it wouldn't be sufficient because the new
  579. // vals can be arbitrary and commit some arbitrary app hash.
  580. //
  581. // newSet is the validator set that signed this block. Only votes from new are
  582. // sufficient for 2/3 majority in the new set as well, for it to be a valid
  583. // commit.
  584. //
  585. // NOTE: This doesn't check whether the commit is a future commit, because the
  586. // current height isn't part of the ValidatorSet. Caller must check that the
  587. // commit height is greater than the height for this validator set.
  588. func (vals *ValidatorSet) VerifyFutureCommit(newSet *ValidatorSet, chainID string,
  589. blockID BlockID, height int64, commit *Commit) error {
  590. oldVals := vals
  591. // Commit must be a valid commit for newSet.
  592. err := newSet.VerifyCommit(chainID, blockID, height, commit)
  593. if err != nil {
  594. return err
  595. }
  596. // Check old voting power.
  597. oldVotingPower := int64(0)
  598. seen := map[int]bool{}
  599. round := commit.Round()
  600. for idx, precommit := range commit.Precommits {
  601. if precommit == nil {
  602. continue
  603. }
  604. if precommit.Height != height {
  605. return cmn.NewError("Blocks don't match - %d vs %d", round, precommit.Round)
  606. }
  607. if precommit.Round != round {
  608. return cmn.NewError("Invalid commit -- wrong round: %v vs %v", round, precommit.Round)
  609. }
  610. if precommit.Type != PrecommitType {
  611. return cmn.NewError("Invalid commit -- not precommit @ index %v", idx)
  612. }
  613. // See if this validator is in oldVals.
  614. idx, val := oldVals.GetByAddress(precommit.ValidatorAddress)
  615. if val == nil || seen[idx] {
  616. continue // missing or double vote...
  617. }
  618. seen[idx] = true
  619. // Validate signature.
  620. precommitSignBytes := commit.VoteSignBytes(chainID, precommit)
  621. if !val.PubKey.VerifyBytes(precommitSignBytes, precommit.Signature) {
  622. return cmn.NewError("Invalid commit -- invalid signature: %v", precommit)
  623. }
  624. // Good precommit!
  625. if blockID.Equals(precommit.BlockID) {
  626. oldVotingPower += val.VotingPower
  627. } else {
  628. // It's OK that the BlockID doesn't match. We include stray
  629. // precommits to measure validator availability.
  630. }
  631. }
  632. if oldVotingPower <= oldVals.TotalVotingPower()*2/3 {
  633. return errTooMuchChange{oldVotingPower, oldVals.TotalVotingPower()*2/3 + 1}
  634. }
  635. return nil
  636. }
  637. //-----------------
  638. // ErrTooMuchChange
  639. func IsErrTooMuchChange(err error) bool {
  640. switch err_ := err.(type) {
  641. case cmn.Error:
  642. _, ok := err_.Data().(errTooMuchChange)
  643. return ok
  644. case errTooMuchChange:
  645. return true
  646. default:
  647. return false
  648. }
  649. }
  650. type errTooMuchChange struct {
  651. got int64
  652. needed int64
  653. }
  654. func (e errTooMuchChange) Error() string {
  655. return fmt.Sprintf("Invalid commit -- insufficient old voting power: got %v, needed %v", e.got, e.needed)
  656. }
  657. //----------------
  658. func (vals *ValidatorSet) String() string {
  659. return vals.StringIndented("")
  660. }
  661. // String
  662. func (vals *ValidatorSet) StringIndented(indent string) string {
  663. if vals == nil {
  664. return "nil-ValidatorSet"
  665. }
  666. var valStrings []string
  667. vals.Iterate(func(index int, val *Validator) bool {
  668. valStrings = append(valStrings, val.String())
  669. return false
  670. })
  671. return fmt.Sprintf(`ValidatorSet{
  672. %s Proposer: %v
  673. %s Validators:
  674. %s %v
  675. %s}`,
  676. indent, vals.GetProposer().String(),
  677. indent,
  678. indent, strings.Join(valStrings, "\n"+indent+" "),
  679. indent)
  680. }
  681. //-------------------------------------
  682. // Implements sort for sorting validators by address.
  683. // Sort validators by address
  684. type ValidatorsByAddress []*Validator
  685. func (valz ValidatorsByAddress) Len() int {
  686. return len(valz)
  687. }
  688. func (valz ValidatorsByAddress) Less(i, j int) bool {
  689. return bytes.Compare(valz[i].Address, valz[j].Address) == -1
  690. }
  691. func (valz ValidatorsByAddress) Swap(i, j int) {
  692. it := valz[i]
  693. valz[i] = valz[j]
  694. valz[j] = it
  695. }
  696. //----------------------------------------
  697. // For testing
  698. // RandValidatorSet returns a randomized validator set, useful for testing.
  699. // NOTE: PrivValidator are in order.
  700. // UNSTABLE
  701. func RandValidatorSet(numValidators int, votingPower int64) (*ValidatorSet, []PrivValidator) {
  702. valz := make([]*Validator, numValidators)
  703. privValidators := make([]PrivValidator, numValidators)
  704. for i := 0; i < numValidators; i++ {
  705. val, privValidator := RandValidator(false, votingPower)
  706. valz[i] = val
  707. privValidators[i] = privValidator
  708. }
  709. vals := NewValidatorSet(valz)
  710. sort.Sort(PrivValidatorsByAddress(privValidators))
  711. return vals, privValidators
  712. }
  713. ///////////////////////////////////////////////////////////////////////////////
  714. // Safe addition/subtraction
  715. func safeAdd(a, b int64) (int64, bool) {
  716. if b > 0 && a > math.MaxInt64-b {
  717. return -1, true
  718. } else if b < 0 && a < math.MinInt64-b {
  719. return -1, true
  720. }
  721. return a + b, false
  722. }
  723. func safeSub(a, b int64) (int64, bool) {
  724. if b > 0 && a < math.MinInt64+b {
  725. return -1, true
  726. } else if b < 0 && a > math.MaxInt64+b {
  727. return -1, true
  728. }
  729. return a - b, false
  730. }
  731. func safeAddClip(a, b int64) int64 {
  732. c, overflow := safeAdd(a, b)
  733. if overflow {
  734. if b < 0 {
  735. return math.MinInt64
  736. }
  737. return math.MaxInt64
  738. }
  739. return c
  740. }
  741. func safeSubClip(a, b int64) int64 {
  742. c, overflow := safeSub(a, b)
  743. if overflow {
  744. if b > 0 {
  745. return math.MinInt64
  746. }
  747. return math.MaxInt64
  748. }
  749. return c
  750. }