You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

250 lines
16 KiB

  1. # Interview Transcript with Tendermint core researcher, Zarko Milosevic, by Chjango
  2. **ZM**: Regarding leader election, it's round robin, but a weighted one. You
  3. take into account the amount of bonded tokens. Depending on how much weight
  4. they have of voting power, they would be elected more frequently. So we do
  5. rotate, but just the guys who are having more voting power would be elected
  6. more frequently. We are having 4 validators, and 1 of them have 2 times more
  7. voting power, they have 2 times more elected as a leader.
  8. **CC**: 2x more absolute voting power or probabilistic voting power?
  9. **ZM**: It's actually very deterministic. It's not probabilistic at all. See
  10. [Tendermint proposal election specification][1]. In Tendermint, there is no
  11. pseudorandom leader election. It's a deterministic protocol. So leader election
  12. is a built-in function in the code, so you know exactly—depending on the voting
  13. power in the validator set, you'd know who exactly would be the leader in round
  14. x, x + 1, and so on. There is nothing random there; we are not trying to hide
  15. who would be the leader. It's really well known. It's just that there is a
  16. function, it's a mathematical function, and it's just basically—it's kind of an
  17. implementation detail—it starts from the voting power, and when you are
  18. elected, you get decreased some number, and in each round you keep increasing
  19. depending on your voting power, so that you are elected after k rounds again.
  20. But knowing the validator set and the voting power, it's very simple function,
  21. you can calculate yourself to know exactly who would be next. For each round,
  22. this function will return you the leader for that round. In every round, we do
  23. this computation. It's all part of the same flow. It enforces the properties
  24. which are: proportional to your voting power, you will be elected, and we keep
  25. changing the leaders. So it can't happen to have one guy being more elected
  26. than other guys, if they have the same voting power. So one time it will be guy
  27. B, and next time it will be guy B1. So it's not random.
  28. **CC**: Assuming the validator set remains unchanged for a month, then if you
  29. run this function, are you able to know exactly who is going to go for that
  30. entire month?
  31. **ZM**: Yes.
  32. **CC**: What're the attack scenarios for this?
  33. **ZM**: This is something which is easily attacked by people who argue that
  34. Tendermint is not decentralized enough. They say that by knowing the leader,
  35. you can DDoS the leader. And by DDoSing the leader, you are able to stop the
  36. progress. Because it's true. If you would be able to DDoS the leader, the
  37. leader would not be able to propose and then effectively will not be making
  38. progress. How we are addressing this thing is Sentry Architecture. So the
  39. validator—or at least a proper validator—will never be available. You don't
  40. know the ip address of the validator. You are never able to open the connection
  41. to the validator. So validator is spawning sentry nodes and this is the single
  42. administration domain and there is only connection from validator in the sense
  43. of sentry nodes. And ip address of validator is not shared in the p2p network.
  44. It’s completely private. This is our answer to DDoS attack. By playing clever
  45. at this sentry node architecture and spawning additional sentry nodes in case,
  46. for ex your sentry nodes are being DDoS’d, bc your sentry nodes are public,
  47. then you will be able to connect to sentry nodes. this is where we will expect
  48. the validator to be clever enough that so that in case they are DDoS’d at the
  49. sentry level, they will spawn a different sentry node and then you communicate
  50. through them. We are in a sense pushing the responsibility on the validator.
  51. **CC**: So if I understand this correctly, the public identity of the validator
  52. doesn’t even matter because that entity can obfuscate where their real full
  53. nodes reside via a proxy through this sentry architecture.
  54. **ZM**: Exactly. So you do know what is the address or identity of the validator
  55. but you don’t know the network address of it; you’re not able to attack it
  56. because you don’t know where they are. They are completely obfuscated by the
  57. sentry nodes. There is now, if you really want to figure out….There is the
  58. Tendermint protocol, the structure of the protocol is not fully decentralized
  59. in the sense that the flow of information is going from the round proposer, or
  60. the round coordinator, to other nodes, and then after they receive this it’s
  61. basically like [inaudible: “O to 1”]. So by tracking where this information is
  62. coming from, you might be able to identify who are the sentry nodes behind it.
  63. So if you are doing some network analysis, you might be able to deduce
  64. something. If the thing would be completely stuck, where the validator would
  65. never change their sentry nodes or ip addresses of sentry nodes, it could be
  66. possible to deduce something. This is where economic game comes into play. We
  67. are doing an economics game there. We say that it’s a validator business. If
  68. they are not able to hide themselves well enough, they’ll be DDoS’d and they
  69. will be kicked out of the active validator set. So it’s in their interest.
  70. [Proposer Selection Procedure in Tendermint][1]. This is how it should work no
  71. matter what implementation.
  72. **CC**: Going back to the proposer, lets say the validator does get DDoS’d, then
  73. the proposer goes down. What happens?
  74. **ZM**: How the proposal mechanism works—there’s nothing special there—it goes
  75. through a sequence of rounds. Normal execution of Tendermint is that for each
  76. height, we are going through a sequence of rounds, starting from round 0, and
  77. then we are incrementing through the rounds. The nodes are moving through the
  78. rounds as part of normal procedure until they decide to commit. In case you
  79. have one proposer—the proposer of a single round—being DDoS’d, we will probably
  80. not decide in that round, because he will not be able to send his proposal. So
  81. we will go to the next round, and hopefully the next proposer will be able to
  82. communicate with the validators and then we’ll decide in the next round.
  83. **CC**: Are there timeouts between one round to another, if a round gets
  84. skipped?
  85. **ZM**: There are timeouts. It’s a bit more complex. I think we have 5 timeouts.
  86. We may be able to simplify this a bit. What is important to understand is: The
  87. only condition which needs to be satisfied so we can go to the next round is
  88. that your validator is able to communicate with more than 2/3rds of voting
  89. power. To be able to move to the next round, you need to receive more than
  90. 2/3rd of voting power equivalent of pre-commit messages.
  91. We have two kinds of messages: 1) Proposal: Where the current round proposer is
  92. suggesting how the next block should look like. This is first one. Every round
  93. starts with proposer sending a proposal. And then there are two more rounds of
  94. voting, where the validator is trying to agree whether they will commit the
  95. proposal or not. And the first of such vote messages is called `pre-vote` and
  96. the second one is `pre-commit`. Now, to be able to move between steps, between
  97. a `pre-vote` and `pre-commit` step, you need to receive enough number of
  98. messages where if message is sent by validator A, then also this message has a
  99. weight, or voting power which is equal to the voting power of the validator who
  100. sent this message. Before you receive more than 2/3 of voting power messages, you are not
  101. able to move to the higher round. Only when you receive more than 2/3 of
  102. messages, you actually start the timeout. The timeout is happening only after
  103. you receive enough messages. And it happens because of the asynchrony of the
  104. message communication so you give more time to guys with this timeout to
  105. receive some messages which are maybe delayed.
  106. **CC**: In this way that you just described via the whole network gossiping
  107. before we commit a block, that is what makes Tendermint BFT deterministic in a
  108. partially synchronous setting vs Bitcoin which has synchrony assumptions
  109. whereby blocks are first mined and then gossiped to the network.
  110. **ZM**: It's true that in Bitcoin, this is where the synchrony assumption comes
  111. to play because if they're not able to communicate timely, they are not able to
  112. converge to a single longest chain. Why are they not able to decrease timeout
  113. in Bitcoin? Because if they would decrease, there would be so many forks that
  114. they won't be able to converge to a single chain. By increasing this
  115. complexity and the block time, they're able to have not so many forks. This is
  116. effectively the timing assumption—the block duration in a sense because it's
  117. enough time so that the decided block is propagated through the network before
  118. someone else start deciding on the same block and creating forks. It's very
  119. different from the consensus algorithms in a distributed computing setup where
  120. Tendermint fits. In Tendermint, where we talk about the timing dependency, they
  121. are really part of this 3-communication step protocol I just explained. We have
  122. the following assumption: If the good guys are not able to communicate timely
  123. and reliably without having message loss within a round, the Tendermint will
  124. not make progress—it will not be making blocks. So if you are in a completely
  125. asynchronous network where messages get lost or delayed unpredictably,
  126. Tendermint will not make progress, it will not create forks, but it will not
  127. decide, it will not tell you what is the next block. For termination, it's a
  128. liveness property of consensus. It's a guarantee to decide. We do need timing
  129. assumptions. Within a round, correct validators are able to communicate to each
  130. other the consensus messages, not the transactions, but consensus messages.
  131. They need to communicate in a timely and reliable fashion. But this doesn't
  132. need to hold forever. It's just that what we are assuming when we say it's a
  133. partially synchronous system, we assume that the system will be going through a
  134. period of asynchrony, where we don't have this guarantee; the messages will be
  135. delayed or some will be lost and then will not make progress for some period of
  136. time, or we're not guaranteed to make progress. And the period of synchrony
  137. where these guarantees hold. And if we think about internet, internet is best
  138. described using such a model. Sometimes when we send a message to SF to
  139. Belgrade, it takes 100 ms, sometimes it takes 300 ms, sometimes it takes 1 s.
  140. But in most cases, it takes 100 ms or less than this.
  141. There is one thing which would be really nice if you understand it. In a global
  142. wide area network, we can't make assumption on the communication unless we are
  143. very conservative about this. If you want to be very fast, then we can't make
  144. assumption and say we'll be for sure communicating with 1 ms communication
  145. delay. Because of the complexity and various congestion issues on the network,
  146. it might happen that during a short period of time, this doesn't hold. If this
  147. doesn't hold and you depend on this for correctness of your protocol, you will
  148. have a fork. So the partially synchronous protocol, most of them like
  149. Tendermint, they don't depend on the timing assumption from the internet for
  150. correctness. This is where we state: safety always. So we never make a fork no
  151. matter how bad our estimates about the internet communication delays are. We'll
  152. never make a fork, but we do make some assumptions, and these assumptions are
  153. built-in our timeouts in our protocol which are actually adaptive. So we are
  154. adapting to the current condition and this is where we're saying...We do assume
  155. some properties, or some communication delays, to eventually hold on the
  156. network. During this period, we guarantee that we will be deciding and
  157. committing blocks. And we will be doing this very fast. We will be basically on
  158. the speed of the current network.
  159. **CC**: We make liveness assumptions based on the integrity of the validator
  160. businesses, assuming they're up and running fine.
  161. **ZM**: This is where we are saying, the protocol will be live if we have at
  162. most 1/3, or a bit less than 1/3, of faulty validators. Which means that all
  163. other guys should be online and available. This is also for liveness. This is
  164. related to the condition that we are not able to make progress in rounds if we
  165. don't receive enough messages. If half of our voting power, or half of our
  166. validators are down, we don't have enough messages, so the protocol is
  167. completely blocked. It doesn't make progress in a round, which means it's not
  168. able to be signed. So it's completely critical for Tendermint that we make
  169. progress in rounds. It's like breathing. Tendermint is breathing. If there is
  170. no progress, it's dead; it's blocked, we're not able to breathe, that's why
  171. we're not able to make progress.
  172. **CC**: How does Tendermint compare to other consensus algos?
  173. **ZM**: Tendermint is a very interesting protocol. From an academic point of
  174. view, I'm convinced that there is value there. Hopefully, we prove it by
  175. publishing it on some good conference. What is novel is, if we compare first
  176. Tendermint to this existing BFT problem, it's a continuation of academic
  177. research on BFT consensus. What is novel in Tendermint is that it somehow
  178. merges consensus protocol with gossip. This is completely novel idea.
  179. Originally, in BFT, people were assuming the single administration domain,
  180. small number of nodes, local area network, 4-7 nodes max. If you look at the
  181. research paper, 99% of them have this kind of setup. Wide area was studied but
  182. there is significantly less work in wide area networks. No one studied how to
  183. scale those protocols to hundreds or thousands of nodes before blockchain. It
  184. was always a single administration domain. So in Tendermint now, you are able
  185. to reach consensus among different administration domains which are potentially
  186. hundreds of them in wide area network. The system model is potentially harder
  187. because we have more nodes and wide area network. The second thing is that:
  188. normally, in bft protocols, the protocol itself are normally designed in a way
  189. that has two phases, or two parts. The one which is called normal case, which
  190. is normally quite simple, in this normal case. In spite of some failures, which
  191. are part of the normal execution of the protocol, like for example leader
  192. crashes or leader being DDoS'd, they need to go through a quite complex
  193. protocol, which is like being called view change or leader election or
  194. whatever. These two parts of the same protocol are having quite different
  195. complexity. And most of the people only understand this normal case. In
  196. Tendermint, there is no this difference. We have only one protocol, there are
  197. not two protocols. It's always the same steps and they are much closer to the
  198. normal case than this complex view change protocol.
  199. _This is a bit too technical but this is on a high level things to remember,
  200. that: The system it addresses it's harder than the others and the algorithm
  201. complexity in Tendermint is simpler._ The initial goal of Jae and Bucky which
  202. is inspired by Raft, is that it's simpler so normal engineers could understand.
  203. **CC**: Can you expand on the termination requirement?
  204. _Important point about Liveness in Tendermint_
  205. **ZM**: In Tendermint, we are saying, for termination, we are making assumption
  206. that the system is partially synchronous. And in a partially synchronous system
  207. model, we are able to mathematically prove that the protocol will make
  208. decisions; it will decide.
  209. **CC**: What is a persistent peer?
  210. **ZM**: It's a list of peer identities, which you will try to establish
  211. connection to them, in case connection is broken, Tendermint will automatically
  212. try to reestablish connection. These are important peers, you will really try
  213. persistently to establish connection to them. For other peers, you just drop it
  214. and try from your address book to connect to someone else. The address book is a
  215. list of peers which you discover that they exist, because we are talking about a
  216. very dynamic network—so the nodes are coming and going away—and the gossiping
  217. protocol is discovering new nodes and gossiping them around. So every node will
  218. keep the list of new nodes it discovers, and when you need to establish
  219. connection to a peer, you'll look to address book and get some addresses from
  220. there. There's categorization/ranking of nodes there.
  221. [1]: https://docs.tendermint.com/master/spec/reactors/consensus/proposer-selection.html