You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

845 lines
26 KiB

10 years ago
10 years ago
10 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
Normalize priorities to not exceed total voting power (#3049) * more proposer priority tests - test that we don't reset to zero when updating / adding - test that same power validators alternate * add another test to track / simulate similar behaviour as in #2960 * address some of Chris' review comments * address some more of Chris' review comments * temporarily pushing branch with the following changes: The total power might change if: - a validator is added - a validator is removed - a validator is updated Decrement the accums (of all validators) directly after any of these events (by the inverse of the change) * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * Fix 2960 by re-normalizing / scaling priorities to be in bounds of total power, additionally: - remove heap where it doesn't make sense - avg. only at the end of IncrementProposerPriority instead of each iteration - update (and slightly improve) TestAveragingInIncrementProposerPriorityWithVotingPower to reflect above changes * fix tests * add comment * update changelog pending & some minor changes * comment about division will floor the result & fix typo * Update TestLargeGenesisValidator: - remove TODO and increase large genesis validator's voting power accordingly * move changelog entry to P2P Protocol * Ceil instead of flooring when dividing & update test * quickly fix failing TestProposerPriorityDoesNotGetResetToZero: - divide by Ceil((maxPriority - minPriority) / 2*totalVotingPower) * fix typo: rename getValWitMostPriority -> getValWithMostPriority * test proposer frequencies * return absolute value for diff. keep testing * use for loop for div * cleanup, more tests * spellcheck * get rid of using floats: manually ceil where necessary * Remove float, simplify, fix tests to match chris's proof (#3157)
6 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
state: add more tests for block validation (#3674) * Expose priv validators for use in testing * Generalize block header validation test past height 1 * Remove ineffectual assignment * Remove redundant SaveState call * Reorder comment for clarity * Use the block executor ApplyBlock function instead of implementing a stripped-down version of it * Remove commented-out code * Remove unnecessary test The required tests already appear to be implemented (implicitly) through the TestValidateBlockHeader test. * Allow for catching of specific error types during TestValidateBlockCommit * Make return error testable * Clean up and add TestValidateBlockCommit code * Fix formatting * Extract function to create a new mock test app * Update comment for clarity * Fix comment * Add skeleton code for evidence-related test * Allow for addressing priv val by address * Generalize test beyond a single validator * Generalize TestValidateBlockEvidence past first height * Reorder code to clearly separate tests and utility code * Use a common constant for stop height for testing in state/validation_test.go * Refactor errors to resemble existing conventions * Fix formatting * Extract common helper functions Having the tests littered with helper functions makes them less easily readable imho, so I've pulled them out into a separate file. This also makes it easier to see what helper functions are available during testing, so we minimize the chance of duplication when writing new tests. * Remove unused parameter * Remove unused parameters * Add field keys * Remove unused height constant * Fix typo * Fix incorrect return error * Add field keys * Use separate package for tests This refactors all of the state package's tests into a state_test package, so as to keep any usage of the state package's internal methods explicit. Any internal methods/constants used by tests are now explicitly exported in state/export_test.go * Refactor: extract helper function to make, validate, execute and commit a block * Rename state function to makeState * Remove redundant constant for number of validators * Refactor mock evidence registration into TestMain * Remove extraneous nVals variable * Replace function-level TODOs with file-level TODO and explanation * Remove extraneous comment * Fix linting issues brought up by GolangCI (pulled in from latest merge from develop)
5 years ago
state: add more tests for block validation (#3674) * Expose priv validators for use in testing * Generalize block header validation test past height 1 * Remove ineffectual assignment * Remove redundant SaveState call * Reorder comment for clarity * Use the block executor ApplyBlock function instead of implementing a stripped-down version of it * Remove commented-out code * Remove unnecessary test The required tests already appear to be implemented (implicitly) through the TestValidateBlockHeader test. * Allow for catching of specific error types during TestValidateBlockCommit * Make return error testable * Clean up and add TestValidateBlockCommit code * Fix formatting * Extract function to create a new mock test app * Update comment for clarity * Fix comment * Add skeleton code for evidence-related test * Allow for addressing priv val by address * Generalize test beyond a single validator * Generalize TestValidateBlockEvidence past first height * Reorder code to clearly separate tests and utility code * Use a common constant for stop height for testing in state/validation_test.go * Refactor errors to resemble existing conventions * Fix formatting * Extract common helper functions Having the tests littered with helper functions makes them less easily readable imho, so I've pulled them out into a separate file. This also makes it easier to see what helper functions are available during testing, so we minimize the chance of duplication when writing new tests. * Remove unused parameter * Remove unused parameters * Add field keys * Remove unused height constant * Fix typo * Fix incorrect return error * Add field keys * Use separate package for tests This refactors all of the state package's tests into a state_test package, so as to keep any usage of the state package's internal methods explicit. Any internal methods/constants used by tests are now explicitly exported in state/export_test.go * Refactor: extract helper function to make, validate, execute and commit a block * Rename state function to makeState * Remove redundant constant for number of validators * Refactor mock evidence registration into TestMain * Remove extraneous nVals variable * Replace function-level TODOs with file-level TODO and explanation * Remove extraneous comment * Fix linting issues brought up by GolangCI (pulled in from latest merge from develop)
5 years ago
7 years ago
7 years ago
  1. package types
  2. import (
  3. "bytes"
  4. "fmt"
  5. "math"
  6. "math/big"
  7. "sort"
  8. "strings"
  9. "github.com/pkg/errors"
  10. "github.com/tendermint/tendermint/crypto/merkle"
  11. )
  12. // MaxTotalVotingPower - the maximum allowed total voting power.
  13. // It needs to be sufficiently small to, in all cases:
  14. // 1. prevent clipping in incrementProposerPriority()
  15. // 2. let (diff+diffMax-1) not overflow in IncrementProposerPriority()
  16. // (Proof of 1 is tricky, left to the reader).
  17. // It could be higher, but this is sufficiently large for our purposes,
  18. // and leaves room for defensive purposes.
  19. // PriorityWindowSizeFactor - is a constant that when multiplied with the total voting power gives
  20. // the maximum allowed distance between validator priorities.
  21. const (
  22. MaxTotalVotingPower = int64(math.MaxInt64) / 8
  23. PriorityWindowSizeFactor = 2
  24. )
  25. // ValidatorSet represent a set of *Validator at a given height.
  26. // The validators can be fetched by address or index.
  27. // The index is in order of .Address, so the indices are fixed
  28. // for all rounds of a given blockchain height - ie. the validators
  29. // are sorted by their address.
  30. // On the other hand, the .ProposerPriority of each validator and
  31. // the designated .GetProposer() of a set changes every round,
  32. // upon calling .IncrementProposerPriority().
  33. // NOTE: Not goroutine-safe.
  34. // NOTE: All get/set to validators should copy the value for safety.
  35. type ValidatorSet struct {
  36. // NOTE: persisted via reflect, must be exported.
  37. Validators []*Validator `json:"validators"`
  38. Proposer *Validator `json:"proposer"`
  39. // cached (unexported)
  40. totalVotingPower int64
  41. }
  42. // NewValidatorSet initializes a ValidatorSet by copying over the
  43. // values from `valz`, a list of Validators. If valz is nil or empty,
  44. // the new ValidatorSet will have an empty list of Validators.
  45. // The addresses of validators in `valz` must be unique otherwise the
  46. // function panics.
  47. func NewValidatorSet(valz []*Validator) *ValidatorSet {
  48. vals := &ValidatorSet{}
  49. err := vals.updateWithChangeSet(valz, false)
  50. if err != nil {
  51. panic(fmt.Sprintf("cannot create validator set: %s", err))
  52. }
  53. if len(valz) > 0 {
  54. vals.IncrementProposerPriority(1)
  55. }
  56. return vals
  57. }
  58. // Nil or empty validator sets are invalid.
  59. func (vals *ValidatorSet) IsNilOrEmpty() bool {
  60. return vals == nil || len(vals.Validators) == 0
  61. }
  62. // Increment ProposerPriority and update the proposer on a copy, and return it.
  63. func (vals *ValidatorSet) CopyIncrementProposerPriority(times int) *ValidatorSet {
  64. copy := vals.Copy()
  65. copy.IncrementProposerPriority(times)
  66. return copy
  67. }
  68. // IncrementProposerPriority increments ProposerPriority of each validator and updates the
  69. // proposer. Panics if validator set is empty.
  70. // `times` must be positive.
  71. func (vals *ValidatorSet) IncrementProposerPriority(times int) {
  72. if vals.IsNilOrEmpty() {
  73. panic("empty validator set")
  74. }
  75. if times <= 0 {
  76. panic("Cannot call IncrementProposerPriority with non-positive times")
  77. }
  78. // Cap the difference between priorities to be proportional to 2*totalPower by
  79. // re-normalizing priorities, i.e., rescale all priorities by multiplying with:
  80. // 2*totalVotingPower/(maxPriority - minPriority)
  81. diffMax := PriorityWindowSizeFactor * vals.TotalVotingPower()
  82. vals.RescalePriorities(diffMax)
  83. vals.shiftByAvgProposerPriority()
  84. var proposer *Validator
  85. // Call IncrementProposerPriority(1) times times.
  86. for i := 0; i < times; i++ {
  87. proposer = vals.incrementProposerPriority()
  88. }
  89. vals.Proposer = proposer
  90. }
  91. func (vals *ValidatorSet) RescalePriorities(diffMax int64) {
  92. if vals.IsNilOrEmpty() {
  93. panic("empty validator set")
  94. }
  95. // NOTE: This check is merely a sanity check which could be
  96. // removed if all tests would init. voting power appropriately;
  97. // i.e. diffMax should always be > 0
  98. if diffMax <= 0 {
  99. return
  100. }
  101. // Calculating ceil(diff/diffMax):
  102. // Re-normalization is performed by dividing by an integer for simplicity.
  103. // NOTE: This may make debugging priority issues easier as well.
  104. diff := computeMaxMinPriorityDiff(vals)
  105. ratio := (diff + diffMax - 1) / diffMax
  106. if diff > diffMax {
  107. for _, val := range vals.Validators {
  108. val.ProposerPriority /= ratio
  109. }
  110. }
  111. }
  112. func (vals *ValidatorSet) incrementProposerPriority() *Validator {
  113. for _, val := range vals.Validators {
  114. // Check for overflow for sum.
  115. newPrio := safeAddClip(val.ProposerPriority, val.VotingPower)
  116. val.ProposerPriority = newPrio
  117. }
  118. // Decrement the validator with most ProposerPriority.
  119. mostest := vals.getValWithMostPriority()
  120. // Mind the underflow.
  121. mostest.ProposerPriority = safeSubClip(mostest.ProposerPriority, vals.TotalVotingPower())
  122. return mostest
  123. }
  124. // Should not be called on an empty validator set.
  125. func (vals *ValidatorSet) computeAvgProposerPriority() int64 {
  126. n := int64(len(vals.Validators))
  127. sum := big.NewInt(0)
  128. for _, val := range vals.Validators {
  129. sum.Add(sum, big.NewInt(val.ProposerPriority))
  130. }
  131. avg := sum.Div(sum, big.NewInt(n))
  132. if avg.IsInt64() {
  133. return avg.Int64()
  134. }
  135. // This should never happen: each val.ProposerPriority is in bounds of int64.
  136. panic(fmt.Sprintf("Cannot represent avg ProposerPriority as an int64 %v", avg))
  137. }
  138. // Compute the difference between the max and min ProposerPriority of that set.
  139. func computeMaxMinPriorityDiff(vals *ValidatorSet) int64 {
  140. if vals.IsNilOrEmpty() {
  141. panic("empty validator set")
  142. }
  143. max := int64(math.MinInt64)
  144. min := int64(math.MaxInt64)
  145. for _, v := range vals.Validators {
  146. if v.ProposerPriority < min {
  147. min = v.ProposerPriority
  148. }
  149. if v.ProposerPriority > max {
  150. max = v.ProposerPriority
  151. }
  152. }
  153. diff := max - min
  154. if diff < 0 {
  155. return -1 * diff
  156. } else {
  157. return diff
  158. }
  159. }
  160. func (vals *ValidatorSet) getValWithMostPriority() *Validator {
  161. var res *Validator
  162. for _, val := range vals.Validators {
  163. res = res.CompareProposerPriority(val)
  164. }
  165. return res
  166. }
  167. func (vals *ValidatorSet) shiftByAvgProposerPriority() {
  168. if vals.IsNilOrEmpty() {
  169. panic("empty validator set")
  170. }
  171. avgProposerPriority := vals.computeAvgProposerPriority()
  172. for _, val := range vals.Validators {
  173. val.ProposerPriority = safeSubClip(val.ProposerPriority, avgProposerPriority)
  174. }
  175. }
  176. // Makes a copy of the validator list.
  177. func validatorListCopy(valsList []*Validator) []*Validator {
  178. if valsList == nil {
  179. return nil
  180. }
  181. valsCopy := make([]*Validator, len(valsList))
  182. for i, val := range valsList {
  183. valsCopy[i] = val.Copy()
  184. }
  185. return valsCopy
  186. }
  187. // Copy each validator into a new ValidatorSet.
  188. func (vals *ValidatorSet) Copy() *ValidatorSet {
  189. return &ValidatorSet{
  190. Validators: validatorListCopy(vals.Validators),
  191. Proposer: vals.Proposer,
  192. totalVotingPower: vals.totalVotingPower,
  193. }
  194. }
  195. // HasAddress returns true if address given is in the validator set, false -
  196. // otherwise.
  197. func (vals *ValidatorSet) HasAddress(address []byte) bool {
  198. idx := sort.Search(len(vals.Validators), func(i int) bool {
  199. return bytes.Compare(address, vals.Validators[i].Address) <= 0
  200. })
  201. return idx < len(vals.Validators) && bytes.Equal(vals.Validators[idx].Address, address)
  202. }
  203. // GetByAddress returns an index of the validator with address and validator
  204. // itself if found. Otherwise, -1 and nil are returned.
  205. func (vals *ValidatorSet) GetByAddress(address []byte) (index int, val *Validator) {
  206. idx := sort.Search(len(vals.Validators), func(i int) bool {
  207. return bytes.Compare(address, vals.Validators[i].Address) <= 0
  208. })
  209. if idx < len(vals.Validators) && bytes.Equal(vals.Validators[idx].Address, address) {
  210. return idx, vals.Validators[idx].Copy()
  211. }
  212. return -1, nil
  213. }
  214. // GetByIndex returns the validator's address and validator itself by index.
  215. // It returns nil values if index is less than 0 or greater or equal to
  216. // len(ValidatorSet.Validators).
  217. func (vals *ValidatorSet) GetByIndex(index int) (address []byte, val *Validator) {
  218. if index < 0 || index >= len(vals.Validators) {
  219. return nil, nil
  220. }
  221. val = vals.Validators[index]
  222. return val.Address, val.Copy()
  223. }
  224. // Size returns the length of the validator set.
  225. func (vals *ValidatorSet) Size() int {
  226. return len(vals.Validators)
  227. }
  228. // Force recalculation of the set's total voting power.
  229. func (vals *ValidatorSet) updateTotalVotingPower() {
  230. sum := int64(0)
  231. for _, val := range vals.Validators {
  232. // mind overflow
  233. sum = safeAddClip(sum, val.VotingPower)
  234. if sum > MaxTotalVotingPower {
  235. panic(fmt.Sprintf(
  236. "Total voting power should be guarded to not exceed %v; got: %v",
  237. MaxTotalVotingPower,
  238. sum))
  239. }
  240. }
  241. vals.totalVotingPower = sum
  242. }
  243. // TotalVotingPower returns the sum of the voting powers of all validators.
  244. // It recomputes the total voting power if required.
  245. func (vals *ValidatorSet) TotalVotingPower() int64 {
  246. if vals.totalVotingPower == 0 {
  247. vals.updateTotalVotingPower()
  248. }
  249. return vals.totalVotingPower
  250. }
  251. // GetProposer returns the current proposer. If the validator set is empty, nil
  252. // is returned.
  253. func (vals *ValidatorSet) GetProposer() (proposer *Validator) {
  254. if len(vals.Validators) == 0 {
  255. return nil
  256. }
  257. if vals.Proposer == nil {
  258. vals.Proposer = vals.findProposer()
  259. }
  260. return vals.Proposer.Copy()
  261. }
  262. func (vals *ValidatorSet) findProposer() *Validator {
  263. var proposer *Validator
  264. for _, val := range vals.Validators {
  265. if proposer == nil || !bytes.Equal(val.Address, proposer.Address) {
  266. proposer = proposer.CompareProposerPriority(val)
  267. }
  268. }
  269. return proposer
  270. }
  271. // Hash returns the Merkle root hash build using validators (as leaves) in the
  272. // set.
  273. func (vals *ValidatorSet) Hash() []byte {
  274. if len(vals.Validators) == 0 {
  275. return nil
  276. }
  277. bzs := make([][]byte, len(vals.Validators))
  278. for i, val := range vals.Validators {
  279. bzs[i] = val.Bytes()
  280. }
  281. return merkle.SimpleHashFromByteSlices(bzs)
  282. }
  283. // Iterate will run the given function over the set.
  284. func (vals *ValidatorSet) Iterate(fn func(index int, val *Validator) bool) {
  285. for i, val := range vals.Validators {
  286. stop := fn(i, val.Copy())
  287. if stop {
  288. break
  289. }
  290. }
  291. }
  292. // Checks changes against duplicates, splits the changes in updates and removals, sorts them by address.
  293. //
  294. // Returns:
  295. // updates, removals - the sorted lists of updates and removals
  296. // err - non-nil if duplicate entries or entries with negative voting power are seen
  297. //
  298. // No changes are made to 'origChanges'.
  299. func processChanges(origChanges []*Validator) (updates, removals []*Validator, err error) {
  300. // Make a deep copy of the changes and sort by address.
  301. changes := validatorListCopy(origChanges)
  302. sort.Sort(ValidatorsByAddress(changes))
  303. removals = make([]*Validator, 0, len(changes))
  304. updates = make([]*Validator, 0, len(changes))
  305. var prevAddr Address
  306. // Scan changes by address and append valid validators to updates or removals lists.
  307. for _, valUpdate := range changes {
  308. if bytes.Equal(valUpdate.Address, prevAddr) {
  309. err = fmt.Errorf("duplicate entry %v in %v", valUpdate, changes)
  310. return nil, nil, err
  311. }
  312. if valUpdate.VotingPower < 0 {
  313. err = fmt.Errorf("voting power can't be negative: %v", valUpdate)
  314. return nil, nil, err
  315. }
  316. if valUpdate.VotingPower > MaxTotalVotingPower {
  317. err = fmt.Errorf("to prevent clipping/ overflow, voting power can't be higher than %v: %v ",
  318. MaxTotalVotingPower, valUpdate)
  319. return nil, nil, err
  320. }
  321. if valUpdate.VotingPower == 0 {
  322. removals = append(removals, valUpdate)
  323. } else {
  324. updates = append(updates, valUpdate)
  325. }
  326. prevAddr = valUpdate.Address
  327. }
  328. return updates, removals, err
  329. }
  330. // Verifies a list of updates against a validator set, making sure the allowed
  331. // total voting power would not be exceeded if these updates would be applied to the set.
  332. //
  333. // Returns:
  334. // updatedTotalVotingPower - the new total voting power if these updates would be applied
  335. // numNewValidators - number of new validators
  336. // err - non-nil if the maximum allowed total voting power would be exceeded
  337. //
  338. // 'updates' should be a list of proper validator changes, i.e. they have been verified
  339. // by processChanges for duplicates and invalid values.
  340. // No changes are made to the validator set 'vals'.
  341. func verifyUpdates(updates []*Validator, vals *ValidatorSet) (updatedTotalVotingPower int64, numNewValidators int, err error) {
  342. updatedTotalVotingPower = vals.TotalVotingPower()
  343. for _, valUpdate := range updates {
  344. address := valUpdate.Address
  345. _, val := vals.GetByAddress(address)
  346. if val == nil {
  347. // New validator, add its voting power the the total.
  348. updatedTotalVotingPower += valUpdate.VotingPower
  349. numNewValidators++
  350. } else {
  351. // Updated validator, add the difference in power to the total.
  352. updatedTotalVotingPower += valUpdate.VotingPower - val.VotingPower
  353. }
  354. overflow := updatedTotalVotingPower > MaxTotalVotingPower
  355. if overflow {
  356. err = fmt.Errorf(
  357. "failed to add/update validator %v, total voting power would exceed the max allowed %v",
  358. valUpdate, MaxTotalVotingPower)
  359. return 0, 0, err
  360. }
  361. }
  362. return updatedTotalVotingPower, numNewValidators, nil
  363. }
  364. // Computes the proposer priority for the validators not present in the set based on 'updatedTotalVotingPower'.
  365. // Leaves unchanged the priorities of validators that are changed.
  366. //
  367. // 'updates' parameter must be a list of unique validators to be added or updated.
  368. // No changes are made to the validator set 'vals'.
  369. func computeNewPriorities(updates []*Validator, vals *ValidatorSet, updatedTotalVotingPower int64) {
  370. for _, valUpdate := range updates {
  371. address := valUpdate.Address
  372. _, val := vals.GetByAddress(address)
  373. if val == nil {
  374. // add val
  375. // Set ProposerPriority to -C*totalVotingPower (with C ~= 1.125) to make sure validators can't
  376. // un-bond and then re-bond to reset their (potentially previously negative) ProposerPriority to zero.
  377. //
  378. // Contract: updatedVotingPower < MaxTotalVotingPower to ensure ProposerPriority does
  379. // not exceed the bounds of int64.
  380. //
  381. // Compute ProposerPriority = -1.125*totalVotingPower == -(updatedVotingPower + (updatedVotingPower >> 3)).
  382. valUpdate.ProposerPriority = -(updatedTotalVotingPower + (updatedTotalVotingPower >> 3))
  383. } else {
  384. valUpdate.ProposerPriority = val.ProposerPriority
  385. }
  386. }
  387. }
  388. // Merges the vals' validator list with the updates list.
  389. // When two elements with same address are seen, the one from updates is selected.
  390. // Expects updates to be a list of updates sorted by address with no duplicates or errors,
  391. // must have been validated with verifyUpdates() and priorities computed with computeNewPriorities().
  392. func (vals *ValidatorSet) applyUpdates(updates []*Validator) {
  393. existing := vals.Validators
  394. merged := make([]*Validator, len(existing)+len(updates))
  395. i := 0
  396. for len(existing) > 0 && len(updates) > 0 {
  397. if bytes.Compare(existing[0].Address, updates[0].Address) < 0 { // unchanged validator
  398. merged[i] = existing[0]
  399. existing = existing[1:]
  400. } else {
  401. // Apply add or update.
  402. merged[i] = updates[0]
  403. if bytes.Equal(existing[0].Address, updates[0].Address) {
  404. // Validator is present in both, advance existing.
  405. existing = existing[1:]
  406. }
  407. updates = updates[1:]
  408. }
  409. i++
  410. }
  411. // Add the elements which are left.
  412. for j := 0; j < len(existing); j++ {
  413. merged[i] = existing[j]
  414. i++
  415. }
  416. // OR add updates which are left.
  417. for j := 0; j < len(updates); j++ {
  418. merged[i] = updates[j]
  419. i++
  420. }
  421. vals.Validators = merged[:i]
  422. }
  423. // Checks that the validators to be removed are part of the validator set.
  424. // No changes are made to the validator set 'vals'.
  425. func verifyRemovals(deletes []*Validator, vals *ValidatorSet) error {
  426. for _, valUpdate := range deletes {
  427. address := valUpdate.Address
  428. _, val := vals.GetByAddress(address)
  429. if val == nil {
  430. return fmt.Errorf("failed to find validator %X to remove", address)
  431. }
  432. }
  433. if len(deletes) > len(vals.Validators) {
  434. panic("more deletes than validators")
  435. }
  436. return nil
  437. }
  438. // Removes the validators specified in 'deletes' from validator set 'vals'.
  439. // Should not fail as verification has been done before.
  440. func (vals *ValidatorSet) applyRemovals(deletes []*Validator) {
  441. existing := vals.Validators
  442. merged := make([]*Validator, len(existing)-len(deletes))
  443. i := 0
  444. // Loop over deletes until we removed all of them.
  445. for len(deletes) > 0 {
  446. if bytes.Equal(existing[0].Address, deletes[0].Address) {
  447. deletes = deletes[1:]
  448. } else { // Leave it in the resulting slice.
  449. merged[i] = existing[0]
  450. i++
  451. }
  452. existing = existing[1:]
  453. }
  454. // Add the elements which are left.
  455. for j := 0; j < len(existing); j++ {
  456. merged[i] = existing[j]
  457. i++
  458. }
  459. vals.Validators = merged[:i]
  460. }
  461. // Main function used by UpdateWithChangeSet() and NewValidatorSet().
  462. // If 'allowDeletes' is false then delete operations (identified by validators with voting power 0)
  463. // are not allowed and will trigger an error if present in 'changes'.
  464. // The 'allowDeletes' flag is set to false by NewValidatorSet() and to true by UpdateWithChangeSet().
  465. func (vals *ValidatorSet) updateWithChangeSet(changes []*Validator, allowDeletes bool) error {
  466. if len(changes) == 0 {
  467. return nil
  468. }
  469. // Check for duplicates within changes, split in 'updates' and 'deletes' lists (sorted).
  470. updates, deletes, err := processChanges(changes)
  471. if err != nil {
  472. return err
  473. }
  474. if !allowDeletes && len(deletes) != 0 {
  475. return fmt.Errorf("cannot process validators with voting power 0: %v", deletes)
  476. }
  477. // Verify that applying the 'deletes' against 'vals' will not result in error.
  478. if err := verifyRemovals(deletes, vals); err != nil {
  479. return err
  480. }
  481. // Verify that applying the 'updates' against 'vals' will not result in error.
  482. updatedTotalVotingPower, numNewValidators, err := verifyUpdates(updates, vals)
  483. if err != nil {
  484. return err
  485. }
  486. // Check that the resulting set will not be empty.
  487. if numNewValidators == 0 && len(vals.Validators) == len(deletes) {
  488. return errors.New("applying the validator changes would result in empty set")
  489. }
  490. // Compute the priorities for updates.
  491. computeNewPriorities(updates, vals, updatedTotalVotingPower)
  492. // Apply updates and removals.
  493. vals.applyUpdates(updates)
  494. vals.applyRemovals(deletes)
  495. vals.updateTotalVotingPower()
  496. // Scale and center.
  497. vals.RescalePriorities(PriorityWindowSizeFactor * vals.TotalVotingPower())
  498. vals.shiftByAvgProposerPriority()
  499. return nil
  500. }
  501. // UpdateWithChangeSet attempts to update the validator set with 'changes'.
  502. // It performs the following steps:
  503. // - validates the changes making sure there are no duplicates and splits them in updates and deletes
  504. // - verifies that applying the changes will not result in errors
  505. // - computes the total voting power BEFORE removals to ensure that in the next steps the priorities
  506. // across old and newly added validators are fair
  507. // - computes the priorities of new validators against the final set
  508. // - applies the updates against the validator set
  509. // - applies the removals against the validator set
  510. // - performs scaling and centering of priority values
  511. // If an error is detected during verification steps, it is returned and the validator set
  512. // is not changed.
  513. func (vals *ValidatorSet) UpdateWithChangeSet(changes []*Validator) error {
  514. return vals.updateWithChangeSet(changes, true)
  515. }
  516. // Verify that +2/3 of the set had signed the given signBytes.
  517. func (vals *ValidatorSet) VerifyCommit(chainID string, blockID BlockID, height int64, commit *Commit) error {
  518. if err := commit.ValidateBasic(); err != nil {
  519. return err
  520. }
  521. if vals.Size() != len(commit.Precommits) {
  522. return NewErrInvalidCommitPrecommits(vals.Size(), len(commit.Precommits))
  523. }
  524. if height != commit.Height() {
  525. return NewErrInvalidCommitHeight(height, commit.Height())
  526. }
  527. if !blockID.Equals(commit.BlockID) {
  528. return fmt.Errorf("Invalid commit -- wrong block id: want %v got %v",
  529. blockID, commit.BlockID)
  530. }
  531. talliedVotingPower := int64(0)
  532. for idx, precommit := range commit.Precommits {
  533. if precommit == nil {
  534. continue // OK, some precommits can be missing.
  535. }
  536. _, val := vals.GetByIndex(idx)
  537. // Validate signature.
  538. precommitSignBytes := commit.VoteSignBytes(chainID, idx)
  539. if !val.PubKey.VerifyBytes(precommitSignBytes, precommit.Signature) {
  540. return fmt.Errorf("Invalid commit -- invalid signature: %v", precommit)
  541. }
  542. // Good precommit!
  543. if blockID.Equals(precommit.BlockID) {
  544. talliedVotingPower += val.VotingPower
  545. }
  546. // else {
  547. // It's OK that the BlockID doesn't match. We include stray
  548. // precommits to measure validator availability.
  549. // }
  550. }
  551. if talliedVotingPower > vals.TotalVotingPower()*2/3 {
  552. return nil
  553. }
  554. return errTooMuchChange{talliedVotingPower, vals.TotalVotingPower()*2/3 + 1}
  555. }
  556. // VerifyFutureCommit will check to see if the set would be valid with a different
  557. // validator set.
  558. //
  559. // vals is the old validator set that we know. Over 2/3 of the power in old
  560. // signed this block.
  561. //
  562. // In Tendermint, 1/3 of the voting power can halt or fork the chain, but 1/3
  563. // can't make arbitrary state transitions. You still need > 2/3 Byzantine to
  564. // make arbitrary state transitions.
  565. //
  566. // To preserve this property in the light client, we also require > 2/3 of the
  567. // old vals to sign the future commit at H, that way we preserve the property
  568. // that if they weren't being truthful about the validator set at H (block hash
  569. // -> vals hash) or about the app state (block hash -> app hash) we can slash
  570. // > 2/3. Otherwise, the lite client isn't providing the same security
  571. // guarantees.
  572. //
  573. // Even if we added a slashing condition that if you sign a block header with
  574. // the wrong validator set, then we would only need > 1/3 of signatures from
  575. // the old vals on the new commit, it wouldn't be sufficient because the new
  576. // vals can be arbitrary and commit some arbitrary app hash.
  577. //
  578. // newSet is the validator set that signed this block. Only votes from new are
  579. // sufficient for 2/3 majority in the new set as well, for it to be a valid
  580. // commit.
  581. //
  582. // NOTE: This doesn't check whether the commit is a future commit, because the
  583. // current height isn't part of the ValidatorSet. Caller must check that the
  584. // commit height is greater than the height for this validator set.
  585. func (vals *ValidatorSet) VerifyFutureCommit(newSet *ValidatorSet, chainID string,
  586. blockID BlockID, height int64, commit *Commit) error {
  587. oldVals := vals
  588. // Commit must be a valid commit for newSet.
  589. err := newSet.VerifyCommit(chainID, blockID, height, commit)
  590. if err != nil {
  591. return err
  592. }
  593. // Check old voting power.
  594. oldVotingPower := int64(0)
  595. seen := map[int]bool{}
  596. round := commit.Round()
  597. for idx, precommit := range commit.Precommits {
  598. if precommit == nil {
  599. continue
  600. }
  601. if precommit.Height != height {
  602. return errors.Errorf("Blocks don't match - %d vs %d", round, precommit.Round)
  603. }
  604. if precommit.Round != round {
  605. return errors.Errorf("Invalid commit -- wrong round: %v vs %v", round, precommit.Round)
  606. }
  607. if precommit.Type != PrecommitType {
  608. return errors.Errorf("Invalid commit -- not precommit @ index %v", idx)
  609. }
  610. // See if this validator is in oldVals.
  611. oldIdx, val := oldVals.GetByAddress(precommit.ValidatorAddress)
  612. if val == nil || seen[oldIdx] {
  613. continue // missing or double vote...
  614. }
  615. seen[oldIdx] = true
  616. // Validate signature.
  617. precommitSignBytes := commit.VoteSignBytes(chainID, idx)
  618. if !val.PubKey.VerifyBytes(precommitSignBytes, precommit.Signature) {
  619. return errors.Errorf("Invalid commit -- invalid signature: %v", precommit)
  620. }
  621. // Good precommit!
  622. if blockID.Equals(precommit.BlockID) {
  623. oldVotingPower += val.VotingPower
  624. }
  625. // else {
  626. // It's OK that the BlockID doesn't match. We include stray
  627. // precommits to measure validator availability.
  628. // }
  629. }
  630. if oldVotingPower <= oldVals.TotalVotingPower()*2/3 {
  631. return errTooMuchChange{oldVotingPower, oldVals.TotalVotingPower()*2/3 + 1}
  632. }
  633. return nil
  634. }
  635. //-----------------
  636. // ErrTooMuchChange
  637. func IsErrTooMuchChange(err error) bool {
  638. _, ok := errors.Cause(err).(errTooMuchChange)
  639. return ok
  640. }
  641. type errTooMuchChange struct {
  642. got int64
  643. needed int64
  644. }
  645. func (e errTooMuchChange) Error() string {
  646. return fmt.Sprintf("Invalid commit -- insufficient old voting power: got %v, needed %v", e.got, e.needed)
  647. }
  648. //----------------
  649. func (vals *ValidatorSet) String() string {
  650. return vals.StringIndented("")
  651. }
  652. // String
  653. func (vals *ValidatorSet) StringIndented(indent string) string {
  654. if vals == nil {
  655. return "nil-ValidatorSet"
  656. }
  657. var valStrings []string
  658. vals.Iterate(func(index int, val *Validator) bool {
  659. valStrings = append(valStrings, val.String())
  660. return false
  661. })
  662. return fmt.Sprintf(`ValidatorSet{
  663. %s Proposer: %v
  664. %s Validators:
  665. %s %v
  666. %s}`,
  667. indent, vals.GetProposer().String(),
  668. indent,
  669. indent, strings.Join(valStrings, "\n"+indent+" "),
  670. indent)
  671. }
  672. //-------------------------------------
  673. // Implements sort for sorting validators by address.
  674. // Sort validators by address.
  675. type ValidatorsByAddress []*Validator
  676. func (valz ValidatorsByAddress) Len() int {
  677. return len(valz)
  678. }
  679. func (valz ValidatorsByAddress) Less(i, j int) bool {
  680. return bytes.Compare(valz[i].Address, valz[j].Address) == -1
  681. }
  682. func (valz ValidatorsByAddress) Swap(i, j int) {
  683. it := valz[i]
  684. valz[i] = valz[j]
  685. valz[j] = it
  686. }
  687. //----------------------------------------
  688. // for testing
  689. // RandValidatorSet returns a randomized validator set, useful for testing.
  690. // NOTE: PrivValidator are in order.
  691. // UNSTABLE
  692. func RandValidatorSet(numValidators int, votingPower int64) (*ValidatorSet, []PrivValidator) {
  693. valz := make([]*Validator, numValidators)
  694. privValidators := make([]PrivValidator, numValidators)
  695. for i := 0; i < numValidators; i++ {
  696. val, privValidator := RandValidator(false, votingPower)
  697. valz[i] = val
  698. privValidators[i] = privValidator
  699. }
  700. vals := NewValidatorSet(valz)
  701. sort.Sort(PrivValidatorsByAddress(privValidators))
  702. return vals, privValidators
  703. }
  704. ///////////////////////////////////////////////////////////////////////////////
  705. // safe addition/subtraction
  706. func safeAdd(a, b int64) (int64, bool) {
  707. if b > 0 && a > math.MaxInt64-b {
  708. return -1, true
  709. } else if b < 0 && a < math.MinInt64-b {
  710. return -1, true
  711. }
  712. return a + b, false
  713. }
  714. func safeSub(a, b int64) (int64, bool) {
  715. if b > 0 && a < math.MinInt64+b {
  716. return -1, true
  717. } else if b < 0 && a > math.MaxInt64+b {
  718. return -1, true
  719. }
  720. return a - b, false
  721. }
  722. func safeAddClip(a, b int64) int64 {
  723. c, overflow := safeAdd(a, b)
  724. if overflow {
  725. if b < 0 {
  726. return math.MinInt64
  727. }
  728. return math.MaxInt64
  729. }
  730. return c
  731. }
  732. func safeSubClip(a, b int64) int64 {
  733. c, overflow := safeSub(a, b)
  734. if overflow {
  735. if b > 0 {
  736. return math.MinInt64
  737. }
  738. return math.MaxInt64
  739. }
  740. return c
  741. }