You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

466 lines
14 KiB

crypto: Use a different library for ed25519/sr25519 (#6526) At Oasis we have spend some time writing a new Ed25519/X25519/sr25519 implementation called curve25519-voi. This PR switches the import from ed25519consensus/go-schnorrkel, which should lead to performance gains on most systems. Summary of changes: * curve25519-voi is now used for Ed25519 operations, following the existing ZIP-215 semantics. * curve25519-voi's public key cache is enabled (hardcoded size of 4096 entries, should be tuned, see the code comment) to accelerate repeated Ed25519 verification with the same public key(s). * (BREAKING) curve25519-voi is now used for sr25519 operations. This is a breaking change as the current sr25519 support does something decidedly non-standard when going from a MiniSecretKey to a SecretKey and or PublicKey (The expansion routine is called twice). While I believe the new behavior (that expands once and only once) to be more "correct", this changes the semantics as implemented. * curve25519-voi is now used for merlin since the included STROBE implementation produces much less garbage on the heap. Side issues fixed: * The version of go-schnorrkel that is currently imported by tendermint has a badly broken batch verification implementation. Upstream has fixed the issue after I reported it, so the version should be bumped in the interim. Open design questions/issues: * As noted, the public key cache size should be tuned. It is currently backed by a trivial thread-safe LRU cache, which is not scan-resistant, but replacing it with something better is a matter of implementing an interface. * As far as I can tell, the only reason why serial verification on batch failure is necessary is to provide more detailed error messages (that are only used in some unit tests). If you trust the batch verification to be consistent with serial verification then the fallback can be eliminated entirely (the BatchVerifier provided by the new library supports an option that omits the fallback if this is chosen as the way forward). * curve25519-voi's sr25519 support could use more optimization and more eyes on the code. The algorithm unfortunately is woefully under-specified, and the implementation was done primarily because I got really sad when I actually looked at go-schnorrkel, and we do not use the algorithm at this time.
3 years ago
crypto: Use a different library for ed25519/sr25519 (#6526) At Oasis we have spend some time writing a new Ed25519/X25519/sr25519 implementation called curve25519-voi. This PR switches the import from ed25519consensus/go-schnorrkel, which should lead to performance gains on most systems. Summary of changes: * curve25519-voi is now used for Ed25519 operations, following the existing ZIP-215 semantics. * curve25519-voi's public key cache is enabled (hardcoded size of 4096 entries, should be tuned, see the code comment) to accelerate repeated Ed25519 verification with the same public key(s). * (BREAKING) curve25519-voi is now used for sr25519 operations. This is a breaking change as the current sr25519 support does something decidedly non-standard when going from a MiniSecretKey to a SecretKey and or PublicKey (The expansion routine is called twice). While I believe the new behavior (that expands once and only once) to be more "correct", this changes the semantics as implemented. * curve25519-voi is now used for merlin since the included STROBE implementation produces much less garbage on the heap. Side issues fixed: * The version of go-schnorrkel that is currently imported by tendermint has a badly broken batch verification implementation. Upstream has fixed the issue after I reported it, so the version should be bumped in the interim. Open design questions/issues: * As noted, the public key cache size should be tuned. It is currently backed by a trivial thread-safe LRU cache, which is not scan-resistant, but replacing it with something better is a matter of implementing an interface. * As far as I can tell, the only reason why serial verification on batch failure is necessary is to provide more detailed error messages (that are only used in some unit tests). If you trust the batch verification to be consistent with serial verification then the fallback can be eliminated entirely (the BatchVerifier provided by the new library supports an option that omits the fallback if this is chosen as the way forward). * curve25519-voi's sr25519 support could use more optimization and more eyes on the code. The algorithm unfortunately is woefully under-specified, and the implementation was done primarily because I got really sad when I actually looked at go-schnorrkel, and we do not use the algorithm at this time.
3 years ago
crypto: Use a different library for ed25519/sr25519 (#6526) At Oasis we have spend some time writing a new Ed25519/X25519/sr25519 implementation called curve25519-voi. This PR switches the import from ed25519consensus/go-schnorrkel, which should lead to performance gains on most systems. Summary of changes: * curve25519-voi is now used for Ed25519 operations, following the existing ZIP-215 semantics. * curve25519-voi's public key cache is enabled (hardcoded size of 4096 entries, should be tuned, see the code comment) to accelerate repeated Ed25519 verification with the same public key(s). * (BREAKING) curve25519-voi is now used for sr25519 operations. This is a breaking change as the current sr25519 support does something decidedly non-standard when going from a MiniSecretKey to a SecretKey and or PublicKey (The expansion routine is called twice). While I believe the new behavior (that expands once and only once) to be more "correct", this changes the semantics as implemented. * curve25519-voi is now used for merlin since the included STROBE implementation produces much less garbage on the heap. Side issues fixed: * The version of go-schnorrkel that is currently imported by tendermint has a badly broken batch verification implementation. Upstream has fixed the issue after I reported it, so the version should be bumped in the interim. Open design questions/issues: * As noted, the public key cache size should be tuned. It is currently backed by a trivial thread-safe LRU cache, which is not scan-resistant, but replacing it with something better is a matter of implementing an interface. * As far as I can tell, the only reason why serial verification on batch failure is necessary is to provide more detailed error messages (that are only used in some unit tests). If you trust the batch verification to be consistent with serial verification then the fallback can be eliminated entirely (the BatchVerifier provided by the new library supports an option that omits the fallback if this is chosen as the way forward). * curve25519-voi's sr25519 support could use more optimization and more eyes on the code. The algorithm unfortunately is woefully under-specified, and the implementation was done primarily because I got really sad when I actually looked at go-schnorrkel, and we do not use the algorithm at this time.
3 years ago
  1. package privval
  2. import (
  3. "bytes"
  4. "crypto/cipher"
  5. crand "crypto/rand"
  6. "crypto/sha256"
  7. "encoding/binary"
  8. "errors"
  9. "fmt"
  10. "io"
  11. "math"
  12. "net"
  13. "time"
  14. gogotypes "github.com/gogo/protobuf/types"
  15. pool "github.com/libp2p/go-buffer-pool"
  16. "github.com/oasisprotocol/curve25519-voi/primitives/merlin"
  17. "golang.org/x/crypto/chacha20poly1305"
  18. "golang.org/x/crypto/curve25519"
  19. "golang.org/x/crypto/hkdf"
  20. "golang.org/x/crypto/nacl/box"
  21. "github.com/tendermint/tendermint/crypto"
  22. "github.com/tendermint/tendermint/crypto/ed25519"
  23. cryptoenc "github.com/tendermint/tendermint/crypto/encoding"
  24. "github.com/tendermint/tendermint/internal/libs/protoio"
  25. tmsync "github.com/tendermint/tendermint/internal/libs/sync"
  26. "github.com/tendermint/tendermint/libs/async"
  27. tmprivval "github.com/tendermint/tendermint/proto/tendermint/privval"
  28. )
  29. // This code has been duplicated from p2p/conn prior to the P2P refactor.
  30. // It is left here temporarily until we migrate privval to gRPC.
  31. // https://github.com/tendermint/tendermint/issues/4698
  32. // 4 + 1024 == 1028 total frame size
  33. const (
  34. dataLenSize = 4
  35. dataMaxSize = 1024
  36. totalFrameSize = dataMaxSize + dataLenSize
  37. aeadSizeOverhead = 16 // overhead of poly 1305 authentication tag
  38. aeadKeySize = chacha20poly1305.KeySize
  39. aeadNonceSize = chacha20poly1305.NonceSize
  40. labelEphemeralLowerPublicKey = "EPHEMERAL_LOWER_PUBLIC_KEY"
  41. labelEphemeralUpperPublicKey = "EPHEMERAL_UPPER_PUBLIC_KEY"
  42. labelDHSecret = "DH_SECRET"
  43. labelSecretConnectionMac = "SECRET_CONNECTION_MAC"
  44. )
  45. var (
  46. ErrSmallOrderRemotePubKey = errors.New("detected low order point from remote peer")
  47. secretConnKeyAndChallengeGen = []byte("TENDERMINT_SECRET_CONNECTION_KEY_AND_CHALLENGE_GEN")
  48. )
  49. // SecretConnection implements net.Conn.
  50. // It is an implementation of the STS protocol.
  51. // See https://github.com/tendermint/tendermint/blob/0.1/docs/sts-final.pdf for
  52. // details on the protocol.
  53. //
  54. // Consumers of the SecretConnection are responsible for authenticating
  55. // the remote peer's pubkey against known information, like a nodeID.
  56. // Otherwise they are vulnerable to MITM.
  57. // (TODO(ismail): see also https://github.com/tendermint/tendermint/issues/3010)
  58. type SecretConnection struct {
  59. // immutable
  60. recvAead cipher.AEAD
  61. sendAead cipher.AEAD
  62. remPubKey crypto.PubKey
  63. conn io.ReadWriteCloser
  64. // net.Conn must be thread safe:
  65. // https://golang.org/pkg/net/#Conn.
  66. // Since we have internal mutable state,
  67. // we need mtxs. But recv and send states
  68. // are independent, so we can use two mtxs.
  69. // All .Read are covered by recvMtx,
  70. // all .Write are covered by sendMtx.
  71. recvMtx tmsync.Mutex
  72. recvBuffer []byte
  73. recvNonce *[aeadNonceSize]byte
  74. sendMtx tmsync.Mutex
  75. sendNonce *[aeadNonceSize]byte
  76. }
  77. // MakeSecretConnection performs handshake and returns a new authenticated
  78. // SecretConnection.
  79. // Returns nil if there is an error in handshake.
  80. // Caller should call conn.Close()
  81. // See docs/sts-final.pdf for more information.
  82. func MakeSecretConnection(conn io.ReadWriteCloser, locPrivKey crypto.PrivKey) (*SecretConnection, error) {
  83. var (
  84. locPubKey = locPrivKey.PubKey()
  85. )
  86. // Generate ephemeral keys for perfect forward secrecy.
  87. locEphPub, locEphPriv := genEphKeys()
  88. // Write local ephemeral pubkey and receive one too.
  89. // NOTE: every 32-byte string is accepted as a Curve25519 public key (see
  90. // DJB's Curve25519 paper: http://cr.yp.to/ecdh/curve25519-20060209.pdf)
  91. remEphPub, err := shareEphPubKey(conn, locEphPub)
  92. if err != nil {
  93. return nil, err
  94. }
  95. // Sort by lexical order.
  96. loEphPub, hiEphPub := sort32(locEphPub, remEphPub)
  97. transcript := merlin.NewTranscript("TENDERMINT_SECRET_CONNECTION_TRANSCRIPT_HASH")
  98. transcript.AppendMessage(labelEphemeralLowerPublicKey, loEphPub[:])
  99. transcript.AppendMessage(labelEphemeralUpperPublicKey, hiEphPub[:])
  100. // Check if the local ephemeral public key was the least, lexicographically
  101. // sorted.
  102. locIsLeast := bytes.Equal(locEphPub[:], loEphPub[:])
  103. // Compute common diffie hellman secret using X25519.
  104. dhSecret, err := computeDHSecret(remEphPub, locEphPriv)
  105. if err != nil {
  106. return nil, err
  107. }
  108. transcript.AppendMessage(labelDHSecret, dhSecret[:])
  109. // Generate the secret used for receiving, sending, challenge via HKDF-SHA2
  110. // on the transcript state (which itself also uses HKDF-SHA2 to derive a key
  111. // from the dhSecret).
  112. recvSecret, sendSecret := deriveSecrets(dhSecret, locIsLeast)
  113. const challengeSize = 32
  114. var challenge [challengeSize]byte
  115. transcript.ExtractBytes(challenge[:], labelSecretConnectionMac)
  116. sendAead, err := chacha20poly1305.New(sendSecret[:])
  117. if err != nil {
  118. return nil, errors.New("invalid send SecretConnection Key")
  119. }
  120. recvAead, err := chacha20poly1305.New(recvSecret[:])
  121. if err != nil {
  122. return nil, errors.New("invalid receive SecretConnection Key")
  123. }
  124. sc := &SecretConnection{
  125. conn: conn,
  126. recvBuffer: nil,
  127. recvNonce: new([aeadNonceSize]byte),
  128. sendNonce: new([aeadNonceSize]byte),
  129. recvAead: recvAead,
  130. sendAead: sendAead,
  131. }
  132. // Sign the challenge bytes for authentication.
  133. locSignature, err := signChallenge(&challenge, locPrivKey)
  134. if err != nil {
  135. return nil, err
  136. }
  137. // Share (in secret) each other's pubkey & challenge signature
  138. authSigMsg, err := shareAuthSignature(sc, locPubKey, locSignature)
  139. if err != nil {
  140. return nil, err
  141. }
  142. remPubKey, remSignature := authSigMsg.Key, authSigMsg.Sig
  143. if _, ok := remPubKey.(ed25519.PubKey); !ok {
  144. return nil, fmt.Errorf("expected ed25519 pubkey, got %T", remPubKey)
  145. }
  146. if !remPubKey.VerifySignature(challenge[:], remSignature) {
  147. return nil, errors.New("challenge verification failed")
  148. }
  149. // We've authorized.
  150. sc.remPubKey = remPubKey
  151. return sc, nil
  152. }
  153. // RemotePubKey returns authenticated remote pubkey
  154. func (sc *SecretConnection) RemotePubKey() crypto.PubKey {
  155. return sc.remPubKey
  156. }
  157. // Writes encrypted frames of `totalFrameSize + aeadSizeOverhead`.
  158. // CONTRACT: data smaller than dataMaxSize is written atomically.
  159. func (sc *SecretConnection) Write(data []byte) (n int, err error) {
  160. sc.sendMtx.Lock()
  161. defer sc.sendMtx.Unlock()
  162. for 0 < len(data) {
  163. if err := func() error {
  164. var sealedFrame = pool.Get(aeadSizeOverhead + totalFrameSize)
  165. var frame = pool.Get(totalFrameSize)
  166. defer func() {
  167. pool.Put(sealedFrame)
  168. pool.Put(frame)
  169. }()
  170. var chunk []byte
  171. if dataMaxSize < len(data) {
  172. chunk = data[:dataMaxSize]
  173. data = data[dataMaxSize:]
  174. } else {
  175. chunk = data
  176. data = nil
  177. }
  178. chunkLength := len(chunk)
  179. binary.LittleEndian.PutUint32(frame, uint32(chunkLength))
  180. copy(frame[dataLenSize:], chunk)
  181. // encrypt the frame
  182. sc.sendAead.Seal(sealedFrame[:0], sc.sendNonce[:], frame, nil)
  183. incrNonce(sc.sendNonce)
  184. // end encryption
  185. _, err = sc.conn.Write(sealedFrame)
  186. if err != nil {
  187. return err
  188. }
  189. n += len(chunk)
  190. return nil
  191. }(); err != nil {
  192. return n, err
  193. }
  194. }
  195. return n, err
  196. }
  197. // CONTRACT: data smaller than dataMaxSize is read atomically.
  198. func (sc *SecretConnection) Read(data []byte) (n int, err error) {
  199. sc.recvMtx.Lock()
  200. defer sc.recvMtx.Unlock()
  201. // read off and update the recvBuffer, if non-empty
  202. if 0 < len(sc.recvBuffer) {
  203. n = copy(data, sc.recvBuffer)
  204. sc.recvBuffer = sc.recvBuffer[n:]
  205. return
  206. }
  207. // read off the conn
  208. var sealedFrame = pool.Get(aeadSizeOverhead + totalFrameSize)
  209. defer pool.Put(sealedFrame)
  210. _, err = io.ReadFull(sc.conn, sealedFrame)
  211. if err != nil {
  212. return
  213. }
  214. // decrypt the frame.
  215. // reads and updates the sc.recvNonce
  216. var frame = pool.Get(totalFrameSize)
  217. defer pool.Put(frame)
  218. _, err = sc.recvAead.Open(frame[:0], sc.recvNonce[:], sealedFrame, nil)
  219. if err != nil {
  220. return n, fmt.Errorf("failed to decrypt SecretConnection: %w", err)
  221. }
  222. incrNonce(sc.recvNonce)
  223. // end decryption
  224. // copy checkLength worth into data,
  225. // set recvBuffer to the rest.
  226. var chunkLength = binary.LittleEndian.Uint32(frame) // read the first four bytes
  227. if chunkLength > dataMaxSize {
  228. return 0, errors.New("chunkLength is greater than dataMaxSize")
  229. }
  230. var chunk = frame[dataLenSize : dataLenSize+chunkLength]
  231. n = copy(data, chunk)
  232. if n < len(chunk) {
  233. sc.recvBuffer = make([]byte, len(chunk)-n)
  234. copy(sc.recvBuffer, chunk[n:])
  235. }
  236. return n, err
  237. }
  238. // Implements net.Conn
  239. func (sc *SecretConnection) Close() error { return sc.conn.Close() }
  240. func (sc *SecretConnection) LocalAddr() net.Addr { return sc.conn.(net.Conn).LocalAddr() }
  241. func (sc *SecretConnection) RemoteAddr() net.Addr { return sc.conn.(net.Conn).RemoteAddr() }
  242. func (sc *SecretConnection) SetDeadline(t time.Time) error { return sc.conn.(net.Conn).SetDeadline(t) }
  243. func (sc *SecretConnection) SetReadDeadline(t time.Time) error {
  244. return sc.conn.(net.Conn).SetReadDeadline(t)
  245. }
  246. func (sc *SecretConnection) SetWriteDeadline(t time.Time) error {
  247. return sc.conn.(net.Conn).SetWriteDeadline(t)
  248. }
  249. func genEphKeys() (ephPub, ephPriv *[32]byte) {
  250. var err error
  251. // TODO: Probably not a problem but ask Tony: different from the rust implementation (uses x25519-dalek),
  252. // we do not "clamp" the private key scalar:
  253. // see: https://github.com/dalek-cryptography/x25519-dalek/blob/34676d336049df2bba763cc076a75e47ae1f170f/src/x25519.rs#L56-L74
  254. ephPub, ephPriv, err = box.GenerateKey(crand.Reader)
  255. if err != nil {
  256. panic("Could not generate ephemeral key-pair")
  257. }
  258. return
  259. }
  260. func shareEphPubKey(conn io.ReadWriter, locEphPub *[32]byte) (remEphPub *[32]byte, err error) {
  261. // Send our pubkey and receive theirs in tandem.
  262. var trs, _ = async.Parallel(
  263. func(_ int) (val interface{}, abort bool, err error) {
  264. lc := *locEphPub
  265. _, err = protoio.NewDelimitedWriter(conn).WriteMsg(&gogotypes.BytesValue{Value: lc[:]})
  266. if err != nil {
  267. return nil, true, err // abort
  268. }
  269. return nil, false, nil
  270. },
  271. func(_ int) (val interface{}, abort bool, err error) {
  272. var bytes gogotypes.BytesValue
  273. _, err = protoio.NewDelimitedReader(conn, 1024*1024).ReadMsg(&bytes)
  274. if err != nil {
  275. return nil, true, err // abort
  276. }
  277. var _remEphPub [32]byte
  278. copy(_remEphPub[:], bytes.Value)
  279. return _remEphPub, false, nil
  280. },
  281. )
  282. // If error:
  283. if trs.FirstError() != nil {
  284. err = trs.FirstError()
  285. return
  286. }
  287. // Otherwise:
  288. var _remEphPub = trs.FirstValue().([32]byte)
  289. return &_remEphPub, nil
  290. }
  291. func deriveSecrets(
  292. dhSecret *[32]byte,
  293. locIsLeast bool,
  294. ) (recvSecret, sendSecret *[aeadKeySize]byte) {
  295. hash := sha256.New
  296. hkdf := hkdf.New(hash, dhSecret[:], nil, secretConnKeyAndChallengeGen)
  297. // get enough data for 2 aead keys, and a 32 byte challenge
  298. res := new([2*aeadKeySize + 32]byte)
  299. _, err := io.ReadFull(hkdf, res[:])
  300. if err != nil {
  301. panic(err)
  302. }
  303. recvSecret = new([aeadKeySize]byte)
  304. sendSecret = new([aeadKeySize]byte)
  305. // bytes 0 through aeadKeySize - 1 are one aead key.
  306. // bytes aeadKeySize through 2*aeadKeySize -1 are another aead key.
  307. // which key corresponds to sending and receiving key depends on whether
  308. // the local key is less than the remote key.
  309. if locIsLeast {
  310. copy(recvSecret[:], res[0:aeadKeySize])
  311. copy(sendSecret[:], res[aeadKeySize:aeadKeySize*2])
  312. } else {
  313. copy(sendSecret[:], res[0:aeadKeySize])
  314. copy(recvSecret[:], res[aeadKeySize:aeadKeySize*2])
  315. }
  316. return
  317. }
  318. // computeDHSecret computes a Diffie-Hellman shared secret key
  319. // from our own local private key and the other's public key.
  320. func computeDHSecret(remPubKey, locPrivKey *[32]byte) (*[32]byte, error) {
  321. shrKey, err := curve25519.X25519(locPrivKey[:], remPubKey[:])
  322. if err != nil {
  323. return nil, err
  324. }
  325. var shrKeyArray [32]byte
  326. copy(shrKeyArray[:], shrKey)
  327. return &shrKeyArray, nil
  328. }
  329. func sort32(foo, bar *[32]byte) (lo, hi *[32]byte) {
  330. if bytes.Compare(foo[:], bar[:]) < 0 {
  331. lo = foo
  332. hi = bar
  333. } else {
  334. lo = bar
  335. hi = foo
  336. }
  337. return
  338. }
  339. func signChallenge(challenge *[32]byte, locPrivKey crypto.PrivKey) ([]byte, error) {
  340. signature, err := locPrivKey.Sign(challenge[:])
  341. if err != nil {
  342. return nil, err
  343. }
  344. return signature, nil
  345. }
  346. type authSigMessage struct {
  347. Key crypto.PubKey
  348. Sig []byte
  349. }
  350. func shareAuthSignature(sc io.ReadWriter, pubKey crypto.PubKey, signature []byte) (recvMsg authSigMessage, err error) {
  351. // Send our info and receive theirs in tandem.
  352. var trs, _ = async.Parallel(
  353. func(_ int) (val interface{}, abort bool, err error) {
  354. pbpk, err := cryptoenc.PubKeyToProto(pubKey)
  355. if err != nil {
  356. return nil, true, err
  357. }
  358. _, err = protoio.NewDelimitedWriter(sc).WriteMsg(&tmprivval.AuthSigMessage{PubKey: pbpk, Sig: signature})
  359. if err != nil {
  360. return nil, true, err // abort
  361. }
  362. return nil, false, nil
  363. },
  364. func(_ int) (val interface{}, abort bool, err error) {
  365. var pba tmprivval.AuthSigMessage
  366. _, err = protoio.NewDelimitedReader(sc, 1024*1024).ReadMsg(&pba)
  367. if err != nil {
  368. return nil, true, err // abort
  369. }
  370. pk, err := cryptoenc.PubKeyFromProto(pba.PubKey)
  371. if err != nil {
  372. return nil, true, err // abort
  373. }
  374. _recvMsg := authSigMessage{
  375. Key: pk,
  376. Sig: pba.Sig,
  377. }
  378. return _recvMsg, false, nil
  379. },
  380. )
  381. // If error:
  382. if trs.FirstError() != nil {
  383. err = trs.FirstError()
  384. return
  385. }
  386. var _recvMsg = trs.FirstValue().(authSigMessage)
  387. return _recvMsg, nil
  388. }
  389. //--------------------------------------------------------------------------------
  390. // Increment nonce little-endian by 1 with wraparound.
  391. // Due to chacha20poly1305 expecting a 12 byte nonce we do not use the first four
  392. // bytes. We only increment a 64 bit unsigned int in the remaining 8 bytes
  393. // (little-endian in nonce[4:]).
  394. func incrNonce(nonce *[aeadNonceSize]byte) {
  395. counter := binary.LittleEndian.Uint64(nonce[4:])
  396. if counter == math.MaxUint64 {
  397. // Terminates the session and makes sure the nonce would not re-used.
  398. // See https://github.com/tendermint/tendermint/issues/3531
  399. panic("can't increase nonce without overflow")
  400. }
  401. counter++
  402. binary.LittleEndian.PutUint64(nonce[4:], counter)
  403. }