You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

319 lines
14 KiB

  1. # Proposer selection procedure in Tendermint
  2. This document specifies the Proposer Selection Procedure that is used in Tendermint to choose a round proposer.
  3. As Tendermint is “leader-based protocol”, the proposer selection is critical for its correct functioning.
  4. At a given block height, the proposer selection algorithm runs with the same validator set at each round .
  5. Between heights, an updated validator set may be specified by the application as part of the ABCIResponses' EndBlock.
  6. ## Requirements for Proposer Selection
  7. This sections covers the requirements with Rx being mandatory and Ox optional requirements.
  8. The following requirements must be met by the Proposer Selection procedure:
  9. ### R1: Determinism
  10. Given a validator set `V`, and two honest validators `p` and `q`, for each height `h` and each round `r` the following must hold:
  11. `proposer_p(h,r) = proposer_q(h,r)`
  12. where `proposer_p(h,r)` is the proposer returned by the Proposer Selection Procedure at process `p`, at height `h` and round `r`.
  13. ### R2: Fairness
  14. Given a validator set with total voting power P and a sequence S of elections. In any sub-sequence of S with length C*P, a validator v must be elected as proposer P/VP(v) times, i.e. with frequency:
  15. f(v) ~ VP(v) / P
  16. where C is a tolerance factor for validator set changes with following values:
  17. - C == 1 if there are no validator set changes
  18. - C ~ k when there are validator changes
  19. *[this needs more work]*
  20. ## Basic Algorithm
  21. At its core, the proposer selection procedure uses a weighted round-robin algorithm.
  22. A model that gives a good intuition on how/ why the selection algorithm works and it is fair is that of a priority queue. The validators move ahead in this queue according to their voting power (the higher the voting power the faster a validator moves towards the head of the queue). When the algorithm runs the following happens:
  23. - all validators move "ahead" according to their powers: for each validator, increase the priority by the voting power
  24. - first in the queue becomes the proposer: select the validator with highest priority
  25. - move the proposer back in the queue: decrease the proposer's priority by the total voting power
  26. Notation:
  27. - vset - the validator set
  28. - n - the number of validators
  29. - VP(i) - voting power of validator i
  30. - A(i) - accumulated priority for validator i
  31. - P - total voting power of set
  32. - avg - average of all validator priorities
  33. - prop - proposer
  34. Simple view at the Selection Algorithm:
  35. ```md
  36. def ProposerSelection (vset):
  37. // compute priorities and elect proposer
  38. for each validator i in vset:
  39. A(i) += VP(i)
  40. prop = max(A)
  41. A(prop) -= P
  42. ```
  43. ## Stable Set
  44. Consider the validator set:
  45. Validator | p1| p2
  46. ----------|---|---
  47. VP | 1 | 3
  48. Assuming no validator changes, the following table shows the proposer priority computation over a few runs. Four runs of the selection procedure are shown, starting with the 5th the same values are computed.
  49. Each row shows the priority queue and the process place in it. The proposer is the closest to the head, the rightmost validator. As priorities are updated, the validators move right in the queue. The proposer moves left as its priority is reduced after election.
  50. |Priority Run | -2| -1| 0 | 1| 2 | 3 | 4 | 5 | Alg step
  51. |--------------- |---|---|---- |---|---- |---|---|---|--------
  52. | | | |p1,p2| | | | | |Initialized to 0
  53. |run 1 | | | | p1| | p2| | |A(i)+=VP(i)
  54. | | | p2| | p1| | | | |A(p2)-= P
  55. |run 2 | | | | |p1,p2| | | |A(i)+=VP(i)
  56. | | p1| | | | p2| | | |A(p1)-= P
  57. |run 3 | | p1| | | | | | p2|A(i)+=VP(i)
  58. | | | p1| | p2| | | | |A(p2)-= P
  59. |run 4 | | | p1| | | | p2| |A(i)+=VP(i)
  60. | | | |p1,p2| | | | | |A(p2)-= P
  61. It can be shown that:
  62. - At the end of each run k+1 the sum of the priorities is the same as at end of run k. If a new set's priorities are initialized to 0 then the sum of priorities will be 0 at each run while there are no changes.
  63. - The max distance between priorites is (n-1) *P.*[formal proof not finished]*
  64. ## Validator Set Changes
  65. Between proposer selection runs the validator set may change. Some changes have implications on the proposer election.
  66. ### Voting Power Change
  67. Consider again the earlier example and assume that the voting power of p1 is changed to 4:
  68. Validator | p1| p2
  69. ----------|---| ---
  70. VP | 4 | 3
  71. Let's also assume that before this change the proposer priorites were as shown in first row (last run). As it can be seen, the selection could run again, without changes, as before.
  72. |Priority Run| -2 | -1 | 0 | 1 | 2 | Comment
  73. |--------------| ---|--- |------|--- |--- |--------
  74. | last run | | p2 | | p1 | |__update VP(p1)__
  75. | next run | | | | | p2 |A(i)+=VP(i)
  76. | | p1 | | | | p2 |A(p1)-= P
  77. However, when a validator changes power from a high to a low value, some other validator remain far back in the queue for a long time. This scenario is considered again in the Proposer Priority Range section.
  78. As before:
  79. - At the end of each run k+1 the sum of the priorities is the same as at run k.
  80. - The max distance between priorites is (n-1) * P.
  81. ### Validator Removal
  82. Consider a new example with set:
  83. Validator | p1 | p2 | p3 |
  84. --------- |--- |--- |--- |
  85. VP | 1 | 2 | 3 |
  86. Let's assume that after the last run the proposer priorities were as shown in first row with their sum being 0. After p2 is removed, at the end of next proposer selection run (penultimate row) the sum of priorities is -2 (minus the priority of the removed process).
  87. The procedure could continue without modifications. However, after a sufficiently large number of modifications in validator set, the priority values would migrate towards maximum or minimum allowed values causing truncations due to overflow detection.
  88. For this reason, the selection procedure adds another __new step__ that centers the current priority values such that the priority sum remains close to 0.
  89. |Priority Run |-3 | -2 | -1 | 0 | 1 | 2 | 4 |Comment
  90. |--------------- |--- | ---|--- |--- |--- |--- |---|--------
  91. | last run |p3 | | | | p1 | p2 | |__remove p2__
  92. | nextrun | | | | | | | |
  93. | __new step__ | | p3 | | | | p1 | |A(i) -= avg, avg = -1
  94. | | | | | | p3 | p1 | |A(i)+=VP(i)
  95. | | | | p1 | | p3 | | |A(p1)-= P
  96. The modified selection algorithm is:
  97. ```md
  98. def ProposerSelection (vset):
  99. // center priorities around zero
  100. avg = sum(A(i) for i in vset)/len(vset)
  101. for each validator i in vset:
  102. A(i) -= avg
  103. // compute priorities and elect proposer
  104. for each validator i in vset:
  105. A(i) += VP(i)
  106. prop = max(A)
  107. A(prop) -= P
  108. ```
  109. Observations:
  110. - The sum of priorities is now close to 0. Due to integer division the sum is an integer in (-n, n), where n is the number of validators.
  111. ### New Validator
  112. When a new validator is added, same problem as the one described for removal appears, the sum of priorities in the new set is not zero. This is fixed with the centering step introduced above.
  113. One other issue that needs to be addressed is the following. A validator V that has just been elected is moved to the end of the queue. If the validator set is large and/ or other validators have significantly higher power, V will have to wait many runs to be elected. If V removes and re-adds itself to the set, it would make a significant (albeit unfair) "jump" ahead in the queue.
  114. In order to prevent this, when a new validator is added, its initial priority is set to:
  115. ```md
  116. A(V) = -1.125 * P
  117. ```
  118. where P is the total voting power of the set including V.
  119. Curent implementation uses the penalty factor of 1.125 because it provides a small punishment that is efficient to calculate. See [here](https://github.com/tendermint/tendermint/pull/2785#discussion_r235038971) for more details.
  120. If we consider the validator set where p3 has just been added:
  121. Validator | p1 | p2 | p3
  122. ----------|--- |--- |---
  123. VP | 1 | 3 | 8
  124. then p3 will start with proposer priority:
  125. ```md
  126. A(p3) = -1.125 * (1 + 3 + 8) ~ -13
  127. ```
  128. Note that since current computation uses integer division there is penalty loss when sum of the voting power is less than 8.
  129. In the next run, p3 will still be ahead in the queue, elected as proposer and moved back in the queue.
  130. |Priority Run |-13 | -9 | -5 | -2 | -1 | 0 | 1 | 2 | 5 | 6 | 7 |Alg step
  131. |---------------|--- |--- |--- |----|--- |--- |---|---|---|---|---|--------
  132. |last run | | | | p2 | | | | p1| | | |__add p3__
  133. | | p3 | | | p2 | | | | p1| | | |A(p3) = -4
  134. |next run | | p3 | | | | | | p2| | p1| |A(i) -= avg, avg = -4
  135. | | | | | | p3 | | | | p2| | p1|A(i)+=VP(i)
  136. | | | | p1 | | p3 | | | | p2| | |A(p1)-=P
  137. ## Proposer Priority Range
  138. With the introduction of centering, some interesting cases occur. Low power validators that bind early in a set that includes high power validator(s) benefit from subsequent additions to the set. This is because these early validators run through more right shift operations during centering, operations that increase their priority.
  139. As an example, consider the set where p2 is added after p1, with priority -1.125 * 80k = -90k. After the selection procedure runs once:
  140. Validator | p1 | p2 | Comment
  141. ----------|-----|---- |---
  142. VP | 80k | 10 |
  143. A | 0 |-90k | __added p2__
  144. A |-45k | 45k | __run selection__
  145. Then execute the following steps:
  146. 1. Add a new validator p3:
  147. Validator | p1 | p2 | p3
  148. ----------|-----|--- |----
  149. VP | 80k | 10 | 10
  150. 2. Run selection once. The notation '..p'/'p..' means very small deviations compared to column priority.
  151. |Priority Run | -90k..| -60k | -45k | -15k| 0 | 45k | 75k | 155k | Comment
  152. |--------------|------ |----- |------- |---- |---|---- |----- |------- |---------
  153. | last run | p3 | | p2 | | | p1 | | | __added p3__
  154. | next run
  155. | *right_shift*| | p3 | | p2 | | | p1 | | A(i) -= avg,avg=-30k
  156. | | | ..p3| | ..p2| | | | p1 | A(i)+=VP(i)
  157. | | | ..p3| | ..p2| | | p1.. | | A(p1)-=P, P=80k+20
  158. 3. Remove p1 and run selection once:
  159. Validator | p3 | p2 | Comment
  160. ----------|----- |---- |--------
  161. VP | 10 | 10 |
  162. A |-60k |-15k |
  163. A |-22.5k|22.5k| __run selection__
  164. At this point, while the total voting power is 20, the distance between priorities is 45k. It will take 4500 runs for p3 to catch up with p2.
  165. In order to prevent these types of scenarios, the selection algorithm performs scaling of priorities such that the difference between min and max values is smaller than two times the total voting power.
  166. The modified selection algorithm is:
  167. ```md
  168. def ProposerSelection (vset):
  169. // scale the priority values
  170. diff = max(A)-min(A)
  171. threshold = 2 * P
  172. if diff > threshold:
  173. scale = diff/threshold
  174. for each validator i in vset:
  175. A(i) = A(i)/scale
  176. // center priorities around zero
  177. avg = sum(A(i) for i in vset)/len(vset)
  178. for each validator i in vset:
  179. A(i) -= avg
  180. // compute priorities and elect proposer
  181. for each validator i in vset:
  182. A(i) += VP(i)
  183. prop = max(A)
  184. A(prop) -= P
  185. ```
  186. Observations:
  187. - With this modification, the maximum distance between priorites becomes 2 * P.
  188. Note also that even during steady state the priority range may increase beyond 2 * P. The scaling introduced here helps to keep the range bounded.
  189. ## Wrinkles
  190. ### Validator Power Overflow Conditions
  191. The validator voting power is a positive number stored as an int64. When a validator is added the `1.125 * P` computation must not overflow. As a consequence the code handling validator updates (add and update) checks for overflow conditions making sure the total voting power is never larger than the largest int64 `MAX`, with the property that `1.125 * MAX` is still in the bounds of int64. Fatal error is return when overflow condition is detected.
  192. ### Proposer Priority Overflow/ Underflow Handling
  193. The proposer priority is stored as an int64. The selection algorithm performs additions and subtractions to these values and in the case of overflows and underflows it limits the values to:
  194. ```go
  195. MaxInt64 = 1 << 63 - 1
  196. MinInt64 = -1 << 63
  197. ```
  198. ## Requirement Fulfillment Claims
  199. __[R1]__
  200. The proposer algorithm is deterministic giving consistent results across executions with same transactions and validator set modifications.
  201. [WIP - needs more detail]
  202. __[R2]__
  203. Given a set of processes with the total voting power P, during a sequence of elections of length P, the number of times any process is selected as proposer is equal to its voting power. The sequence of the P proposers then repeats. If we consider the validator set:
  204. Validator | p1| p2
  205. ----------|---|---
  206. VP | 1 | 3
  207. With no other changes to the validator set, the current implementation of proposer selection generates the sequence:
  208. `p2, p1, p2, p2, p2, p1, p2, p2,...` or [`p2, p1, p2, p2`]*
  209. A sequence that starts with any circular permutation of the [`p2, p1, p2, p2`] sub-sequence would also provide the same degree of fairness. In fact these circular permutations show in the sliding window (over the generated sequence) of size equal to the length of the sub-sequence.
  210. Assigning priorities to each validator based on the voting power and updating them at each run ensures the fairness of the proposer selection. In addition, every time a validator is elected as proposer its priority is decreased with the total voting power.
  211. Intuitively, a process v jumps ahead in the queue at most (max(A) - min(A))/VP(v) times until it reaches the head and is elected. The frequency is then:
  212. ```md
  213. f(v) ~ VP(v)/(max(A)-min(A)) = 1/k * VP(v)/P
  214. ```
  215. For current implementation, this means v should be proposer at least VP(v) times out of k * P runs, with scaling factor k=2.