diff --git a/spec/consensus/light-client.md b/spec/consensus/light-client.md index 76ad8c823..1e4ed66f2 100644 --- a/spec/consensus/light-client.md +++ b/spec/consensus/light-client.md @@ -6,18 +6,18 @@ A lite client is a process that connects to Tendermint full nodes and then tries In order to make sure that full nodes have the incentive to follow the protocol, we have to address the following three Issues -1) The lite client needs a method to verify headers it obtains from full nodes according to trust assumptions -- this document. +1) The lite client needs a method to verify headers it obtains from a full node it connects to according to trust assumptions -- this document. -2) The lite client must be able to connect to one correct full node to detect and report on failures in the trust assumptions (i.e., conflicting headers) -- a future document. +2) The lite client must be able to connect to other full nodes to detect and report on failures in the trust assumptions (i.e., conflicting headers) -- a future document (see #4215). -3) In the event the trust assumption fails (i.e., a lite client is fooled by a conflicting header), the Tendermint fork accountability protocol must account for the evidence -- see #3840 +3) In the event the trust assumption fails (i.e., a lite client is fooled by a conflicting header), the Tendermint fork accountability protocol must account for the evidence -- a future document (see #3840). ## Problem statement We assume that the lite client knows a (base) header *inithead* it trusts (by social consensus or because the lite client has decided to trust the header before). The goal is to check whether another header *newhead* can be trusted based on the data in *inithead*. -The correctness of the protocol is based on the assumption that *inithead* was generated by an instance of Tendermint consensus. The term "trusting" above indicates that the correctness on the protocol depends on this assumption. It is in the responsibility of the user that runs the lite client to make sure that the risk of trusting a corrupted/forged *inithead* is negligible. +The correctness of the protocol is based on the assumption that *inithead* was generated by an instance of Tendermint consensus. The term "trusting" above indicates that the correctness of the protocol depends on this assumption. It is in the responsibility of the user that runs the lite client to make sure that the risk of trusting a corrupted/forged *inithead* is negligible. ## Definitions @@ -61,7 +61,7 @@ For the purpose of this lite client specification, we assume that the Tendermint ### Definitions -* *tp*: trusting period +* *TRUSTED_PERIOD*: trusting period * for realtime *t*, the predicate *correct(v,t)* is true if the validator *v* follows the protocol until time *t* (we will see about recovery later). @@ -70,11 +70,11 @@ For the purpose of this lite client specification, we assume that the Tendermint ### Tendermint Failure Model -If a block *h* is generated at time *bfttime* (and this time is stored in the block), then a set of validators that hold more than 2/3 of the voting power in h.Header.NextV is correct until time h.Header.bfttime + tp. +If a block *h* is generated at time *bfttime* (and this time is stored in the block), then a set of validators that hold more than 2/3 of the voting power in h.Header.NextV is correct until time h.Header.bfttime + TRUSTED_PERIOD. Formally, \[ -\sum_{(v,p) \in h.Header.NextV \wedge correct(v,h.Header.bfttime + tp)} p > +\sum_{(v,p) \in h.Header.NextV \wedge correct(v,h.Header.bfttime + TRUSTED_PERIOD)} p > 2/3 \sum_{(v,p) \in h.Header.NextV} p \] @@ -151,40 +151,39 @@ This can be used in several settings: ## Details -*Assumptions* - -1. *tp < unbonding period*. -2. *snh.Header.bfttime < now* -3. *snh.Header.bfttime < h.Header.bfttime+tp* -4. *trust(h)=true* - - **Observation 1.** If *h.Header.bfttime + tp > now*, we trust the old validator set *h.Header.NextV*. When we say we trust *h.Header.NextV* we do *not* trust that each individual validator in *h.Header.NextV* is correct, but we only trust the fact that at most 1/3 of them are faulty (more precisely, the faulty ones have at most 1/3 of the total voting power). - ### Functions The function *CanTrust* checks whether to trust header *h2* based on the trusted header *h1*. It does so by (potentially) building transitive trust relation between *h1* and *h2*, over some intermediate headers. For example, in case we cannot trust header *h2* based on the trusted header *h1*, the function *CanTrust* will try to find headers such that we can transition trust -from *h1* over intermediate headers to *h2*. We will give two implementations of *CanTrust* based on *CheckSupport*, the one based +from *h1* over intermediate headers to *h2*. We will give two implementations of *CanTrust*, the one based on bisection that is recursive and the other that is non-recursive. We give two implementations as recursive version might be easier to understand but non-recursive version might be simpler to formally express and verify using TLA+/TLC. -Both implementations of *CanTrust* function are based on *CheckSupport* function that implements the conditions under which we can trust a +Both implementations of *CanTrust* function are based on *CheckSupport* function that implements the skipping conditions under which we can trust a header *h2* given the trust in the header *h1* as a single step, i.e., it does not assume ensuring transitive trust relation between headers through some intermediate headers. +In order to incentivize correct behavior of validators that run Tendermint consensus protocol, fork detection protocol (it will be explained in different document) is executed in case of a fork (conflicting +headers are detected). As detecting conflicting headers, its propagation through the network (by the gossip protocol) and execution of the fork accountability +protocol on the chain takes time, the lite client logic assumes conservative value for trusted period. More precisely, in the context of lite client we always +operate with a smaller trusted period that we call *lite client trusted period* (LITE_CLIENT_TRUSTED_PERIOD). If we assume that upper bound +for fork detection, propagation and processing on the chain is denoted with *fork procession period* (FORK_PROCESSING_PERIOD), then the following formula +holds: +```LITE_CLIENT_TRUSTED_PERIOD + FORK_PROCESSING_PERIOD < TRUSTED_PERIOD```, where TRUSTED_PERIOD comes from the Tendermint Failure Model. + *Assumption*: In the following, we assume that *h2.Header.height > h1.Header.height*. We will quickly discuss the other case in the next section. We consider the following set-up: - the lite client communicates with one full node -- the lite client locally stores all the headers that has passed basic verification. In the pseudo code below we write *Store(header)* for this. If a header failed to verify, then +- the lite client locally stores all the headers that has passed basic verification and that are within lite client trust period. In the pseudo code below we write *Store(header)* for this. If a header failed to verify, then the full node we are talking to is faulty and we should disconnect from it and reinitialise lite client. - If *CanTrust* returns *error*, then the lite client has seen a forged header or the trusted header has expired (it is outside its trusted period). * In case of forged header, the full node is faulty so lite client should disconnect and reinitialise with new trusted header. @@ -193,17 +192,15 @@ the full node we are talking to is faulty and we should disconnect from it and r we will write ```totalVotingPower(V)``` for ```votingpower_in(V,V)```, which returns the total voting power in V. We further use the function ```signers(Commit)``` that returns the set of validators that signed the Commit. -**CheckSupport.** The following function checks whether we can trust the header h2 based on header h1 following the trusting period method. Time constraint is -captured by the ```hasExpired``` function that depends on trusted period (`tp`) and it returns true in case the header is outside its trusted period. +**CheckSupport.** The following function defines skipping condition under the Tendermint Failure model, i.e., it defines when we can trust the header h2 based on header h1. +Time validity of a header is captured by the ```isWithinTrustedPeriodWithin``` function that depends on lite client trusted period (`LITE_CLIENT_TRUSTED_PERIOD`) and it returns +true in case the header is within its lite client trusted period. ```verify``` function is capturing basic header verification, i.e., it ensures that the header is signed by more than 2/3 of the voting power of the corresponding validator set. ```go - // return true if header has expired, i.e., it is outside its trusted period; otherwise it returns false - func hasExpired(h, Delta) bool { - if h.Header.bfttime + tp - Delta < now { // Observation 1 - return true - } - return false + // return true if header is within its lite client trusted period; otherwise it returns false + func isWithinTrustedPeriod(h) bool { + return h.Header.bfttime + LITE_CLIENT_TRUSTED_PERIOD > now } // return true if header is correctly signed by 2/3+ voting power in the corresponding validator set; @@ -216,13 +213,18 @@ captured by the ```hasExpired``` function that depends on trusted period (`tp`) // Captures skipping condition. h1 and h2 has already passed basic validation (function `verify`). // returns nil in case h2 can be trusted based on h1, otherwise returns error. - // ErrHeaderExpired is used to signal that h1 has expired, + // ErrHeaderExpired is used to signal that h1 has expired with respect lite client trusted period, // ErrInvalidAdjacentHeaders that adjacent headers are not consistent and // ErrTooMuchChange that there is not enough intersection between validator sets to have skipping condition true. func CheckSupport(h1,h2,trustlevel) error { - assume h1.Header.Height < h2.header.Height + assume h1.Header.Height < h2.header.Height and h1.Header.bfttime < h2.Header.bfttime and h2.Header.bfttime < now + + if !isWithinTrustedPeriod(h1) return ErrHeaderNotWithinTrustedPeriod(h1) - if hasExpired(h1) return ErrHeaderExpired(h1) + // Although while executing the rest of CheckSupport function, h1 can expiry based on the lite client trusted period, this is not problem as + // lite client trusted period is smaller than trusted period of the header based on Tendermint Failure model, i.e., there is a significant + // time period (measure in days) during which validator set that has signed h1 can be trusted + // Furthermore, CheckSupport function is not doing expensive operation (neither rpc nor signature verification), so it should execute fast. // total sum of voting power of validators in h1.NextV vp_all := totalVotingPower(h1.Header.NextV) @@ -232,27 +234,14 @@ captured by the ```hasExpired``` function that depends on trusted period (`tp`) if h1.Header.NextV == h2.Header.V return nil return ErrInvalidAdjacentHeaders - } else { - // check for non-adjacent headers - if votingpower_in(signers(h2.Commit),h1.Header.NextV) > max(1/3,trustlevel) * vp_all - return nil - else - return ErrTooMuchChange(h1, h2) } + // check for non-adjacent headers + if votingpower_in(signers(h2.Commit),h1.Header.NextV) > max(1/3,trustlevel) * vp_all return nil + return ErrTooMuchChange + } ``` - *Remark*: Basic header verification must be done for *h2*. Similar checks are done in: - https://github.com/tendermint/tendermint/blob/master/types/validator_set.go#L591-L633 - - *Remark*: There are some sanity checks which are not in the code: - *h2.Header.height > h1.Header.height* and *h2.Header.bfttime > h1.Header.bfttime* and *h2.Header.bfttime < now*. - - *Remark*: ```return (votingpower_in(signers(h2.Commit),h1.Header.NextV) > max(1/3,trustlevel) * vp_all)``` may return false even if *h2* was properly generated by Tendermint consensus in the case of big changes in the validator sets. However, the check ```return (votingpower_in(signers(h2.Commit),h1.Header.NextV) > - 2/3 * vp_all)``` must return true if *h1* and *h2* were generated by Tendermint consensus. - -*Remark*: The 1/3 check differs from a previously proposed method that was based on intersecting validator sets and checking that the new validator set contains "enough" correct validators. We found that the old check is not suited for realistic changes in the validator sets. The new method is not only based on cardinalities, but also exploits that we can trust what is signed by a correct validator (i.e., signed by more than 1/3 of the voting power). - *Correctness arguments* Towards Lite Client Accuracy: @@ -270,83 +259,124 @@ Towards Lite Client Completeness: *Remark*: The variable *trustlevel* can be used if the user believes that relying on one correct validator is not sufficient. However, in case of (frequent) changes in the validator set, the higher the *trustlevel* is chosen, the more unlikely it becomes that CheckSupport returns true for non-adjacent headers. -**Bisection.** The following function uses CheckSupport in a recursion to find intermediate headers that allow to establish a sequence of trust. +**VerifyHeader.** The function *VerifyHeader* captures high level logic, i.e., application call to the lite client module to (optionally download) and +verify header for some height. The core verification logic is captured by *CanTrust* function that iteratively try to establish trust in given header +by relying on *CheckSupport* function. ```go -// return (true, nil) in case we can trust header h2 based on header h1; otherwise return (false, error) where error captures the nature of the error. -func CanTrust(h1,h2,trustlevel) (bool, error) { +func VerifyHeader(height, trustlevel) error { + if h2, exists := Store.Get(height); exists { + if isWithinTrustedPeriod(h2) return nil + return ErrHeaderNotWithinTrustedPeriod(h2) + } + else { + h2 := Commit(height) + if !verify(h2) return ErrInvalidHeader(h2) + if !isWithinTrustedPeriod(h2) return ErrHeaderNotWithinTrustedPeriod(h2) + } + + // get the highest trusted headers lower than h2 + h1 = Store.HighestTrustedSmallerThan(height) + if h1 == nil + return ErrNoTrustedHeader + + err = CanTrust(h1, h2, trustlevel) // or CanTrustBisection((h1, h2, trustlevel) + if err != nil return err + + if isWithinTrustedPeriod(h2) { + Store.add(h2) // we store only trusted headers, as we assume that only trusted headers are influencing end user business decisions. + return nil + } + return ErrHeaderNotTrusted(h2) +} + + +// return nil in case we can trust header h2 based on header h1; otherwise return error where error captures the nature of the error. +func CanTrust(h1,h2,trustlevel) error { assume h1.Header.Height < h2.header.Height - ok, err = CheckSupport(h1,h2,trustlevel) - if (ok or err != nil) return (ok, err) + err = CheckSupport(h1,h2,trustlevel) + if err == nil { + Store.Add(h2) + return nil + } + if err != ErrTooMuchChange return err // we cannot verify h2 based on h1, so we try to move trusted header closer to h2 so we can verify h2 th := h1 // th is trusted header - while th.Header.Height <= h2.Header.height - 1 do { - // try to move trusted header forward with stored headers - ih := th - // try to move trusted header forward - for h in stored headers s.t ih.Header.Height < h.Header.height < h2.Header.height do { - // we assume here that iteration is done in the order of header heights - ok, err = CheckSupport(th,h,trustlevel) - if err != nil { return (ok, err) } - if ok { - th = h - } - } - - // at this point we have potentially updated th based on stored headers so we try to verify h2 - // based on new trusted header - ok, err = CheckSupport(th,h2,trustlevel) - if (ok or err != nil) return (ok, err) + untrustedHeaders := [] - // we cannot verify h2 based on th, so we try to move trusted header closer to h2 by downloading header(s) between th and h2 - endHeight = h2.Header.height - foundPivot = false - while(!foundPivot) { + while true { + endHeight = h2.Header.height + foundPivot = false + while(!foundPivot) { pivot := (th.Header.height + endHeight) / 2 hp := Commit(pivot) - if !verify(hp) return (false, ErrInvalidHeader(hp)) - Store(hp) - + if !verify(hp) return ErrInvalidHeader(hp) // try to move trusted header forward to hp - ok, err = CheckSupport(th,hp,trustlevel) - if err != nil { return (ok, err) } - if ok { + err = CheckSupport(th,hp,trustlevel) + if (err != nil and err != ErrTooMuchChange) return err + if err == nil { th = hp + Store.Add(hp) foundPivot = true } + untrustedHeaders.add(hp) endHeight = pivot + } + + // try to move trusted header forward + for h in untrustedHeaders { + // we assume here that iteration is done in the order of header heights + err = CheckSupport(th,h,trustlevel) + if (err != nil and err != ErrTooMuchChange) return err + if err == nil { + th = h + Store.Add(h) + untrustedHeaders.Remove(h) + } + } + + // at this point we have potentially updated th based on stored headers so we try to verify h2 + // based on new trusted header + err = CheckSupport(h1,h2,trustlevel) + if err == nil { + Store.Add(h2) + return nil } + if err != ErrTooMuchChange return err } + return nil // this line should never be reached } ``` ```go -func CanTrustBisection(h1,h2,trustlevel) bool{ - if CheckSupport(h1,h2,trustlevel) { - return true - } - if h2.Header.height == h1.Header.height + 1 { - // we have adjacent headers that are not matching (failed - // the CheckSupport) - // we could submit evidence here - return false +func CanTrustBisection(h1,h2,trustlevel) error { + assume h1.Header.Height < h2.header.Height + + err = CheckSupport(h1,h2,trustlevel) + if err == nil { + Store.Add(h2) + return nil } + if err != ErrTooMuchChange return err + pivot := (h1.Header.height + h2.Header.height) / 2 hp := Commit(pivot) - // ask a full node for header of height pivot - Store(hp) - // store header hp locally - if Bisection(h1,hp,trustlevel) { - // only check right branch if hp is trusted - // (otherwise a lot of unnecessary computation may be done) - return Bisection(hp,h2,trustlevel) - } - else { - return false + if !verify(hp) return ErrInvalidHeader(hp) + + err = CanTrustBisection(h1,hp,trustlevel) + if err == nil { + Store.Add(hp) + err2 = CanTrustBisection(hp,h2,trustlevel) + if err2 == nil { + Store.Add(h2) + return nil + } + return err2 } + return err } ``` @@ -356,8 +386,8 @@ func CanTrustBisection(h1,h2,trustlevel) bool{ *Correctness arguments (sketch)* Lite Client Accuracy: -- Assume by contradiction that *h2* was not generated correctly and the lite client sets trust to true because Bisection returns true. -- Bisection returns true only if all calls to CheckSupport in the recursion return true. +- Assume by contradiction that *h2* was not generated correctly and the lite client sets trust to true because CanTrustBisection returns nil. +- CanTrustBisection returns true only if all calls to CheckSupport in the recursion return nil. - Thus we have a sequence of headers that all satisfied the CheckSupport - again a contradiction @@ -367,7 +397,7 @@ This is only ensured if upon *Commit(pivot)* the lite client is always provided *Stalling* -With Bisection, a faulty full node could stall a lite client by creating a long sequence of headers that are queried one-by-one by the lite client and look OK, before the lite client eventually detects a problem. There are several ways to address this: +With CanTrustBisection, a faulty full node could stall a lite client by creating a long sequence of headers that are queried one-by-one by the lite client and look OK, before the lite client eventually detects a problem. There are several ways to address this: * Each call to ```Commit``` could be issued to a different full node * Instead of querying header by header, the lite client tells a full node which header it trusts, and the height of the header it needs. The full node responds with the header along with a proof consisting of intermediate headers that the light client can use to verify. Roughly, Bisection would then be executed at the full node. * We may set a timeout how long bisection may take. @@ -380,111 +410,23 @@ In the use case where someone tells the lite client that application data that i *Remark.* For the case were the lite client trusts two headers *i* and *j* with *i < k < j*, we should discuss/experiment whether the forward or the backward method is more effective. ```go -func Backwards(h1,h2) bool { +func Backwards(h1,h2) error { assert (h2.Header.height < h1.Header.height) + if !isWithinTrustedPeriod(h1) return ErrHeaderNotTrusted(h1) + old := h1 for i := h1.Header.height - 1; i > h2.Header.height; i-- { new := Commit(i) - Store(new) if (hash(new) != old.Header.hash) { - return false + return ErrInvalidAdjacentHeaders } old := new + if !isWithinTrustedPeriod(h1) return ErrHeaderNotTrusted(h1) } - return (hash(h2) == old.Header.hash) + if hash(h2) == old.Header.hash return ErrInvalidAdjacentHeaders + return nil } ``` -```go -// return true if header has expired, i.e., it is outside its trusted period; otherwise it returns false -func hasExpired(h) bool { - if now - h.Header.bfttime > tp - Delta - return true // Observation 1 - return false -} - -// return true if header is correctly signed by 2/3+ voting power in the corresponding validator set; -// otherwise false. Additional checks should be done in the implementation -// to ensure header is well formed. -func verify(h) bool { - vp_all := totalVotingPower(h.Header.V) // total sum of voting power of validators in h - return votingpower_in(signers(h.Commit),h.Header.V) > 2/3 * vp_all -} - -// Captures skipping condition. h1 and h2 has already passed basic validation (function `verify`). -// returns (true, nil) in case h2 can be trusted based on h1, (false, nil) in case it cannot be trusted but no errors -// are observed during check and (false, error) in case an error is detected (for example adjacent -// headers are not consistent). -func CheckSupport(h1,h2,trustlevel) (bool, error) { - assume h1.Header.Height < h2.header.Height - - // check for adjacent headers - if h2.Header.height == h1.Header.height + 1 { - if h1.Header.NextV == h2.Header.V return nil - return false, ErrInvalidAdjacentHeaders - } - // total sum of voting power of validators in h1.NextV - vp_all := totalVotingPower(h1.Header.NextV) - // check for non-adjacent headers - return votingpower_in(signers(h2.Commit),h1.Header.NextV) > max(1/3,trustlevel) * vp_all, nil -} - -// return nil in case we can trust header h2 based on header h1, error otherwise -func Bisection(h1,h2,trustlevel) error { - assume h1.Header.Height < h2.header.Height - - th := h1 - pivot = h2.Header.height - untrustedHeaders = [] - - while h2 not in Store.Headers() { // could be replaced with while true and check if h2 is stored in the loop - // check support for pivot, bisect if no support until a higher trusted header is found - while pivot < h1.Header.height { - hp := Commit(pivot) - if !verify(hp) return ErrInvalidHeader(hp) - done, err = CheckSupport(th,hp,trustlevel) - if err != nil return err - if done { - th = hp - Store.Add(hp) - break - } - untrustedHeaders.add(hp) - pivot := (th.Header.height + endHeight) / 2 - } - - // try to move trusted header forward - for h in untrustedHeaders { - // we assume here that iteration is done in the order of header heights - done, err = CheckSupport(th,h,trustlevel) - if err != nil return err - if done { - th = h - Store.Add(h) - } - } - } - return nil -} - -func VerifyHeader(h2, trustlevel) error { - if h2 in Store.Headers() - return nil - - Store.DisableExpiration() - // get the highest trusted headers lower than h2 - h1 = Store.HighestTrusted(h2.Header.height) - if h1 == nil - return ErrNoTrustedHeader - - err = Bisection(h1, h2, trustlevel) - if err != nil return err - - // remove all expired headers and start the expiration timer - Store.EnableExpiration() - if h2 in Store.Headers() return nil - return ErrHeaderExpired -} -``` \ No newline at end of file