You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

773 lines
21 KiB

9 years ago
9 years ago
9 years ago
9 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
7 years ago
9 years ago
9 years ago
9 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
9 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
7 years ago
9 years ago
9 years ago
7 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
8 years ago
9 years ago
9 years ago
9 years ago
9 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
9 years ago
9 years ago
9 years ago
8 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
7 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: introduce peerConn to simplify peer creation (#1226) * expose AuthEnc in the P2P config if AuthEnc is true, dialed peers must have a node ID in the address and it must match the persistent pubkey from the secret handshake. Refs #1157 * fixes after my own review * fix docs * fix build failure ``` p2p/pex/pex_reactor_test.go:288:88: cannot use seed.NodeInfo().NetAddress() (type *p2p.NetAddress) as type string in array or slice literal ``` * p2p: introduce peerConn to simplify peer creation * Introduce `peerConn` containing the known fields of `peer` * `peer` only created in `sw.addPeer` once handshake is complete and NodeInfo is checked * Eliminates some mutable variables and makes the code flow better * Simplifies the `newXxxPeer` funcs * Use ID instead of PubKey where possible. * SetPubKeyFilter -> SetIDFilter * nodeInfo.Validate takes ID * remove peer.PubKey() * persistent node ids * fixes from review * test: use ip_plus_id.sh more * fix invalid memory panic during fast_sync test ``` 2018-02-21T06:30:05Z box887.localdomain docker/local_testnet_4[14907]: panic: runtime error: invalid memory address or nil pointer dereference 2018-02-21T06:30:05Z box887.localdomain docker/local_testnet_4[14907]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x98dd3e] 2018-02-21T06:30:05Z box887.localdomain docker/local_testnet_4[14907]: 2018-02-21T06:30:05Z box887.localdomain docker/local_testnet_4[14907]: goroutine 3432 [running]: 2018-02-21T06:30:05Z box887.localdomain docker/local_testnet_4[14907]: github.com/tendermint/tendermint/p2p.newOutboundPeerConn(0xc423fd1380, 0xc420933e00, 0x1, 0x1239a60, 0 xc420128c40, 0x2, 0x42caf6, 0xc42001f300, 0xc422831d98, 0xc4227951c0, ...) 2018-02-21T06:30:05Z box887.localdomain docker/local_testnet_4[14907]: #011/go/src/github.com/tendermint/tendermint/p2p/peer.go:123 +0x31e 2018-02-21T06:30:05Z box887.localdomain docker/local_testnet_4[14907]: github.com/tendermint/tendermint/p2p.(*Switch).addOutboundPeerWithConfig(0xc4200ad040, 0xc423fd1380, 0 xc420933e00, 0xc423f48801, 0x28, 0x2) 2018-02-21T06:30:05Z box887.localdomain docker/local_testnet_4[14907]: #011/go/src/github.com/tendermint/tendermint/p2p/switch.go:455 +0x12b 2018-02-21T06:30:05Z box887.localdomain docker/local_testnet_4[14907]: github.com/tendermint/tendermint/p2p.(*Switch).DialPeerWithAddress(0xc4200ad040, 0xc423fd1380, 0x1, 0x 0, 0x0) 2018-02-21T06:30:05Z box887.localdomain docker/local_testnet_4[14907]: #011/go/src/github.com/tendermint/tendermint/p2p/switch.go:371 +0xdc 2018-02-21T06:30:05Z box887.localdomain docker/local_testnet_4[14907]: github.com/tendermint/tendermint/p2p.(*Switch).reconnectToPeer(0xc4200ad040, 0x123e000, 0xc42007bb00) 2018-02-21T06:30:05Z box887.localdomain docker/local_testnet_4[14907]: #011/go/src/github.com/tendermint/tendermint/p2p/switch.go:290 +0x25f 2018-02-21T06:30:05Z box887.localdomain docker/local_testnet_4[14907]: created by github.com/tendermint/tendermint/p2p.(*Switch).StopPeerForError 2018-02-21T06:30:05Z box887.localdomain docker/local_testnet_4[14907]: #011/go/src/github.com/tendermint/tendermint/p2p/switch.go:256 +0x1b7 ```
7 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
9 years ago
p2p: seed mode refactoring (#3011) ListOfKnownAddresses is removed panic if addrbook size is less than zero CrawlPeers does not attempt to connect to existing or peers we're currently dialing various perf. fixes improved tests (though not complete) move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716) * addrbook: preallocate memory when saving addrbook to file * addrbook: remove oldestFirst struct and check for ID * oldestFirst replaced with sort.Slice * ID is now mandatory, so no need to check * addrbook: remove ListOfKnownAddresses GetSelection is used instead in seed mode. * addrbook: panic if size is less than 0 * rewrite addrbook#saveToFile to not use a counter * test AttemptDisconnects func * move IsDialingOrExistingAddress check into DialPeerWithAddress * save and cleanup crawl peer data * get rid of DefaultSeedDisconnectWaitPeriod * make linter happy * fix TestPEXReactorSeedMode * fix comment * add a changelog entry * Apply suggestions from code review Co-Authored-By: melekes <anton.kalyaev@gmail.com> * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress * lowercase errors * do not persist seed data pros: - no extra files - less IO cons: - if the node crashes, seed might crawl a peer too soon * fixes after Ethan's review * add a changelog entry * we should only consult Switch about peers checking addrbook size does not make sense since only PEX reactor uses it for dialing peers! https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875
6 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
  1. package pex
  2. import (
  3. "fmt"
  4. "reflect"
  5. "sync"
  6. "time"
  7. "github.com/pkg/errors"
  8. amino "github.com/tendermint/go-amino"
  9. cmn "github.com/tendermint/tendermint/libs/common"
  10. "github.com/tendermint/tendermint/p2p"
  11. "github.com/tendermint/tendermint/p2p/conn"
  12. )
  13. type Peer = p2p.Peer
  14. const (
  15. // PexChannel is a channel for PEX messages
  16. PexChannel = byte(0x00)
  17. // over-estimate of max NetAddress size
  18. // hexID (40) + IP (16) + Port (2) + Name (100) ...
  19. // NOTE: dont use massive DNS name ..
  20. maxAddressSize = 256
  21. // NOTE: amplificaiton factor!
  22. // small request results in up to maxMsgSize response
  23. maxMsgSize = maxAddressSize * maxGetSelection
  24. // ensure we have enough peers
  25. defaultEnsurePeersPeriod = 30 * time.Second
  26. // Seed/Crawler constants
  27. // minTimeBetweenCrawls is a minimum time between attempts to crawl a peer.
  28. minTimeBetweenCrawls = 2 * time.Minute
  29. // check some peers every this
  30. crawlPeerPeriod = 30 * time.Second
  31. maxAttemptsToDial = 16 // ~ 35h in total (last attempt - 18h)
  32. // if node connects to seed, it does not have any trusted peers.
  33. // Especially in the beginning, node should have more trusted peers than
  34. // untrusted.
  35. biasToSelectNewPeers = 30 // 70 to select good peers
  36. )
  37. type errMaxAttemptsToDial struct {
  38. }
  39. func (e errMaxAttemptsToDial) Error() string {
  40. return fmt.Sprintf("reached max attempts %d to dial", maxAttemptsToDial)
  41. }
  42. type errTooEarlyToDial struct {
  43. backoffDuration time.Duration
  44. lastDialed time.Time
  45. }
  46. func (e errTooEarlyToDial) Error() string {
  47. return fmt.Sprintf(
  48. "too early to dial (backoff duration: %d, last dialed: %v, time since: %v)",
  49. e.backoffDuration, e.lastDialed, time.Since(e.lastDialed))
  50. }
  51. // PEXReactor handles PEX (peer exchange) and ensures that an
  52. // adequate number of peers are connected to the switch.
  53. //
  54. // It uses `AddrBook` (address book) to store `NetAddress`es of the peers.
  55. //
  56. // ## Preventing abuse
  57. //
  58. // Only accept pexAddrsMsg from peers we sent a corresponding pexRequestMsg too.
  59. // Only accept one pexRequestMsg every ~defaultEnsurePeersPeriod.
  60. type PEXReactor struct {
  61. p2p.BaseReactor
  62. book AddrBook
  63. config *PEXReactorConfig
  64. ensurePeersPeriod time.Duration // TODO: should go in the config
  65. // maps to prevent abuse
  66. requestsSent *cmn.CMap // ID->struct{}: unanswered send requests
  67. lastReceivedRequests *cmn.CMap // ID->time.Time: last time peer requested from us
  68. seedAddrs []*p2p.NetAddress
  69. attemptsToDial sync.Map // address (string) -> {number of attempts (int), last time dialed (time.Time)}
  70. // seed/crawled mode fields
  71. crawlPeerInfos map[p2p.ID]crawlPeerInfo
  72. }
  73. func (r *PEXReactor) minReceiveRequestInterval() time.Duration {
  74. // NOTE: must be less than ensurePeersPeriod, otherwise we'll request
  75. // peers too quickly from others and they'll think we're bad!
  76. return r.ensurePeersPeriod / 3
  77. }
  78. // PEXReactorConfig holds reactor specific configuration data.
  79. type PEXReactorConfig struct {
  80. // Seed/Crawler mode
  81. SeedMode bool
  82. // We want seeds to only advertise good peers. Therefore they should wait at
  83. // least as long as we expect it to take for a peer to become good before
  84. // disconnecting.
  85. SeedDisconnectWaitPeriod time.Duration
  86. // Seeds is a list of addresses reactor may use
  87. // if it can't connect to peers in the addrbook.
  88. Seeds []string
  89. }
  90. type _attemptsToDial struct {
  91. number int
  92. lastDialed time.Time
  93. }
  94. // NewPEXReactor creates new PEX reactor.
  95. func NewPEXReactor(b AddrBook, config *PEXReactorConfig) *PEXReactor {
  96. r := &PEXReactor{
  97. book: b,
  98. config: config,
  99. ensurePeersPeriod: defaultEnsurePeersPeriod,
  100. requestsSent: cmn.NewCMap(),
  101. lastReceivedRequests: cmn.NewCMap(),
  102. crawlPeerInfos: make(map[p2p.ID]crawlPeerInfo),
  103. }
  104. r.BaseReactor = *p2p.NewBaseReactor("PEXReactor", r)
  105. return r
  106. }
  107. // OnStart implements BaseService
  108. func (r *PEXReactor) OnStart() error {
  109. err := r.book.Start()
  110. if err != nil && err != cmn.ErrAlreadyStarted {
  111. return err
  112. }
  113. numOnline, seedAddrs, err := r.checkSeeds()
  114. if err != nil {
  115. return err
  116. } else if numOnline == 0 && r.book.Empty() {
  117. return errors.New("Address book is empty and couldn't resolve any seed nodes")
  118. }
  119. r.seedAddrs = seedAddrs
  120. // Check if this node should run
  121. // in seed/crawler mode
  122. if r.config.SeedMode {
  123. go r.crawlPeersRoutine()
  124. } else {
  125. go r.ensurePeersRoutine()
  126. }
  127. return nil
  128. }
  129. // OnStop implements BaseService
  130. func (r *PEXReactor) OnStop() {
  131. r.book.Stop()
  132. }
  133. // GetChannels implements Reactor
  134. func (r *PEXReactor) GetChannels() []*conn.ChannelDescriptor {
  135. return []*conn.ChannelDescriptor{
  136. {
  137. ID: PexChannel,
  138. Priority: 1,
  139. SendQueueCapacity: 10,
  140. },
  141. }
  142. }
  143. // AddPeer implements Reactor by adding peer to the address book (if inbound)
  144. // or by requesting more addresses (if outbound).
  145. func (r *PEXReactor) AddPeer(p Peer) {
  146. if p.IsOutbound() {
  147. // For outbound peers, the address is already in the books -
  148. // either via DialPeersAsync or r.Receive.
  149. // Ask it for more peers if we need.
  150. if r.book.NeedMoreAddrs() {
  151. r.RequestAddrs(p)
  152. }
  153. } else {
  154. // inbound peer is its own source
  155. addr, err := p.NodeInfo().NetAddress()
  156. if err != nil {
  157. r.Logger.Error("Failed to get peer NetAddress", "err", err, "peer", p)
  158. return
  159. }
  160. // Make it explicit that addr and src are the same for an inbound peer.
  161. src := addr
  162. // add to book. dont RequestAddrs right away because
  163. // we don't trust inbound as much - let ensurePeersRoutine handle it.
  164. err = r.book.AddAddress(addr, src)
  165. r.logErrAddrBook(err)
  166. }
  167. }
  168. // RemovePeer implements Reactor by resetting peer's requests info.
  169. func (r *PEXReactor) RemovePeer(p Peer, reason interface{}) {
  170. id := string(p.ID())
  171. r.requestsSent.Delete(id)
  172. r.lastReceivedRequests.Delete(id)
  173. }
  174. func (r *PEXReactor) logErrAddrBook(err error) {
  175. if err != nil {
  176. switch err.(type) {
  177. case ErrAddrBookNilAddr:
  178. r.Logger.Error("Failed to add new address", "err", err)
  179. default:
  180. // non-routable, self, full book, private, etc.
  181. r.Logger.Debug("Failed to add new address", "err", err)
  182. }
  183. }
  184. }
  185. // Receive implements Reactor by handling incoming PEX messages.
  186. func (r *PEXReactor) Receive(chID byte, src Peer, msgBytes []byte) {
  187. msg, err := decodeMsg(msgBytes)
  188. if err != nil {
  189. r.Logger.Error("Error decoding message", "src", src, "chId", chID, "msg", msg, "err", err, "bytes", msgBytes)
  190. r.Switch.StopPeerForError(src, err)
  191. return
  192. }
  193. r.Logger.Debug("Received message", "src", src, "chId", chID, "msg", msg)
  194. switch msg := msg.(type) {
  195. case *pexRequestMessage:
  196. // NOTE: this is a prime candidate for amplification attacks,
  197. // so it's important we
  198. // 1) restrict how frequently peers can request
  199. // 2) limit the output size
  200. // If we're a seed and this is an inbound peer,
  201. // respond once and disconnect.
  202. if r.config.SeedMode && !src.IsOutbound() {
  203. id := string(src.ID())
  204. v := r.lastReceivedRequests.Get(id)
  205. if v != nil {
  206. // FlushStop/StopPeer are already
  207. // running in a go-routine.
  208. return
  209. }
  210. r.lastReceivedRequests.Set(id, time.Now())
  211. // Send addrs and disconnect
  212. r.SendAddrs(src, r.book.GetSelectionWithBias(biasToSelectNewPeers))
  213. go func() {
  214. // In a go-routine so it doesn't block .Receive.
  215. src.FlushStop()
  216. r.Switch.StopPeerGracefully(src)
  217. }()
  218. } else {
  219. // Check we're not receiving requests too frequently.
  220. if err := r.receiveRequest(src); err != nil {
  221. r.Switch.StopPeerForError(src, err)
  222. return
  223. }
  224. r.SendAddrs(src, r.book.GetSelection())
  225. }
  226. case *pexAddrsMessage:
  227. // If we asked for addresses, add them to the book
  228. if err := r.ReceiveAddrs(msg.Addrs, src); err != nil {
  229. r.Switch.StopPeerForError(src, err)
  230. return
  231. }
  232. default:
  233. r.Logger.Error(fmt.Sprintf("Unknown message type %v", reflect.TypeOf(msg)))
  234. }
  235. }
  236. // enforces a minimum amount of time between requests
  237. func (r *PEXReactor) receiveRequest(src Peer) error {
  238. id := string(src.ID())
  239. v := r.lastReceivedRequests.Get(id)
  240. if v == nil {
  241. // initialize with empty time
  242. lastReceived := time.Time{}
  243. r.lastReceivedRequests.Set(id, lastReceived)
  244. return nil
  245. }
  246. lastReceived := v.(time.Time)
  247. if lastReceived.Equal(time.Time{}) {
  248. // first time gets a free pass. then we start tracking the time
  249. lastReceived = time.Now()
  250. r.lastReceivedRequests.Set(id, lastReceived)
  251. return nil
  252. }
  253. now := time.Now()
  254. minInterval := r.minReceiveRequestInterval()
  255. if now.Sub(lastReceived) < minInterval {
  256. return fmt.Errorf(
  257. "peer (%v) sent next PEX request too soon. lastReceived: %v, now: %v, minInterval: %v. Disconnecting",
  258. src.ID(),
  259. lastReceived,
  260. now,
  261. minInterval,
  262. )
  263. }
  264. r.lastReceivedRequests.Set(id, now)
  265. return nil
  266. }
  267. // RequestAddrs asks peer for more addresses if we do not already have a
  268. // request out for this peer.
  269. func (r *PEXReactor) RequestAddrs(p Peer) {
  270. id := string(p.ID())
  271. if r.requestsSent.Has(id) {
  272. return
  273. }
  274. r.Logger.Debug("Request addrs", "from", p)
  275. r.requestsSent.Set(id, struct{}{})
  276. p.Send(PexChannel, cdc.MustMarshalBinaryBare(&pexRequestMessage{}))
  277. }
  278. // ReceiveAddrs adds the given addrs to the addrbook if theres an open
  279. // request for this peer and deletes the open request.
  280. // If there's no open request for the src peer, it returns an error.
  281. func (r *PEXReactor) ReceiveAddrs(addrs []*p2p.NetAddress, src Peer) error {
  282. id := string(src.ID())
  283. if !r.requestsSent.Has(id) {
  284. return errors.New("unsolicited pexAddrsMessage")
  285. }
  286. r.requestsSent.Delete(id)
  287. srcAddr, err := src.NodeInfo().NetAddress()
  288. if err != nil {
  289. return err
  290. }
  291. srcIsSeed := false
  292. for _, seedAddr := range r.seedAddrs {
  293. if seedAddr.Equals(srcAddr) {
  294. srcIsSeed = true
  295. break
  296. }
  297. }
  298. for _, netAddr := range addrs {
  299. // NOTE: we check netAddr validity and routability in book#AddAddress.
  300. err = r.book.AddAddress(netAddr, srcAddr)
  301. if err != nil {
  302. r.logErrAddrBook(err)
  303. // XXX: should we be strict about incoming data and disconnect from a
  304. // peer here too?
  305. continue
  306. }
  307. // If this address came from a seed node, try to connect to it without
  308. // waiting (#2093)
  309. if srcIsSeed {
  310. r.Logger.Info("Will dial address, which came from seed", "addr", netAddr, "seed", srcAddr)
  311. go func(addr *p2p.NetAddress) {
  312. err := r.dialPeer(addr)
  313. if err != nil {
  314. switch err.(type) {
  315. case errMaxAttemptsToDial, errTooEarlyToDial, p2p.ErrCurrentlyDialingOrExistingAddress:
  316. r.Logger.Debug(err.Error(), "addr", addr)
  317. default:
  318. r.Logger.Error(err.Error(), "addr", addr)
  319. }
  320. }
  321. }(netAddr)
  322. }
  323. }
  324. return nil
  325. }
  326. // SendAddrs sends addrs to the peer.
  327. func (r *PEXReactor) SendAddrs(p Peer, netAddrs []*p2p.NetAddress) {
  328. p.Send(PexChannel, cdc.MustMarshalBinaryBare(&pexAddrsMessage{Addrs: netAddrs}))
  329. }
  330. // SetEnsurePeersPeriod sets period to ensure peers connected.
  331. func (r *PEXReactor) SetEnsurePeersPeriod(d time.Duration) {
  332. r.ensurePeersPeriod = d
  333. }
  334. // Ensures that sufficient peers are connected. (continuous)
  335. func (r *PEXReactor) ensurePeersRoutine() {
  336. var (
  337. seed = cmn.NewRand()
  338. jitter = seed.Int63n(r.ensurePeersPeriod.Nanoseconds())
  339. )
  340. // Randomize first round of communication to avoid thundering herd.
  341. // If no peers are present directly start connecting so we guarantee swift
  342. // setup with the help of configured seeds.
  343. if r.nodeHasSomePeersOrDialingAny() {
  344. time.Sleep(time.Duration(jitter))
  345. }
  346. // fire once immediately.
  347. // ensures we dial the seeds right away if the book is empty
  348. r.ensurePeers()
  349. // fire periodically
  350. ticker := time.NewTicker(r.ensurePeersPeriod)
  351. for {
  352. select {
  353. case <-ticker.C:
  354. r.ensurePeers()
  355. case <-r.Quit():
  356. ticker.Stop()
  357. return
  358. }
  359. }
  360. }
  361. // ensurePeers ensures that sufficient peers are connected. (once)
  362. //
  363. // heuristic that we haven't perfected yet, or, perhaps is manually edited by
  364. // the node operator. It should not be used to compute what addresses are
  365. // already connected or not.
  366. func (r *PEXReactor) ensurePeers() {
  367. var (
  368. out, in, dial = r.Switch.NumPeers()
  369. numToDial = r.Switch.MaxNumOutboundPeers() - (out + dial)
  370. )
  371. r.Logger.Info(
  372. "Ensure peers",
  373. "numOutPeers", out,
  374. "numInPeers", in,
  375. "numDialing", dial,
  376. "numToDial", numToDial,
  377. )
  378. if numToDial <= 0 {
  379. return
  380. }
  381. // bias to prefer more vetted peers when we have fewer connections.
  382. // not perfect, but somewhate ensures that we prioritize connecting to more-vetted
  383. // NOTE: range here is [10, 90]. Too high ?
  384. newBias := cmn.MinInt(out, 8)*10 + 10
  385. toDial := make(map[p2p.ID]*p2p.NetAddress)
  386. // Try maxAttempts times to pick numToDial addresses to dial
  387. maxAttempts := numToDial * 3
  388. for i := 0; i < maxAttempts && len(toDial) < numToDial; i++ {
  389. try := r.book.PickAddress(newBias)
  390. if try == nil {
  391. continue
  392. }
  393. if _, selected := toDial[try.ID]; selected {
  394. continue
  395. }
  396. if r.Switch.IsDialingOrExistingAddress(try) {
  397. continue
  398. }
  399. // TODO: consider moving some checks from toDial into here
  400. // so we don't even consider dialing peers that we want to wait
  401. // before dialling again, or have dialed too many times already
  402. r.Logger.Info("Will dial address", "addr", try)
  403. toDial[try.ID] = try
  404. }
  405. // Dial picked addresses
  406. for _, addr := range toDial {
  407. go func(addr *p2p.NetAddress) {
  408. err := r.dialPeer(addr)
  409. if err != nil {
  410. switch err.(type) {
  411. case errMaxAttemptsToDial, errTooEarlyToDial:
  412. r.Logger.Debug(err.Error(), "addr", addr)
  413. default:
  414. r.Logger.Error(err.Error(), "addr", addr)
  415. }
  416. }
  417. }(addr)
  418. }
  419. if r.book.NeedMoreAddrs() {
  420. // 1) Pick a random peer and ask for more.
  421. peers := r.Switch.Peers().List()
  422. peersCount := len(peers)
  423. if peersCount > 0 {
  424. peer := peers[cmn.RandInt()%peersCount]
  425. r.Logger.Info("We need more addresses. Sending pexRequest to random peer", "peer", peer)
  426. r.RequestAddrs(peer)
  427. }
  428. // 2) Dial seeds if we are not dialing anyone.
  429. // This is done in addition to asking a peer for addresses to work-around
  430. // peers not participating in PEX.
  431. if len(toDial) == 0 {
  432. r.Logger.Info("No addresses to dial. Falling back to seeds")
  433. r.dialSeeds()
  434. }
  435. }
  436. }
  437. func (r *PEXReactor) dialAttemptsInfo(addr *p2p.NetAddress) (attempts int, lastDialed time.Time) {
  438. _attempts, ok := r.attemptsToDial.Load(addr.DialString())
  439. if !ok {
  440. return
  441. }
  442. atd := _attempts.(_attemptsToDial)
  443. return atd.number, atd.lastDialed
  444. }
  445. func (r *PEXReactor) dialPeer(addr *p2p.NetAddress) error {
  446. attempts, lastDialed := r.dialAttemptsInfo(addr)
  447. if attempts > maxAttemptsToDial {
  448. // TODO(melekes): have a blacklist in the addrbook with peers whom we've
  449. // failed to connect to. Then we can clean up attemptsToDial, which acts as
  450. // a blacklist currently.
  451. // https://github.com/tendermint/tendermint/issues/3572
  452. r.book.MarkBad(addr)
  453. return errMaxAttemptsToDial{}
  454. }
  455. // exponential backoff if it's not our first attempt to dial given address
  456. if attempts > 0 {
  457. jitterSeconds := time.Duration(cmn.RandFloat64() * float64(time.Second)) // 1s == (1e9 ns)
  458. backoffDuration := jitterSeconds + ((1 << uint(attempts)) * time.Second)
  459. sinceLastDialed := time.Since(lastDialed)
  460. if sinceLastDialed < backoffDuration {
  461. return errTooEarlyToDial{backoffDuration, lastDialed}
  462. }
  463. }
  464. err := r.Switch.DialPeerWithAddress(addr)
  465. if err != nil {
  466. if _, ok := err.(p2p.ErrCurrentlyDialingOrExistingAddress); ok {
  467. return err
  468. }
  469. markAddrInBookBasedOnErr(addr, r.book, err)
  470. switch err.(type) {
  471. case p2p.ErrSwitchAuthenticationFailure:
  472. // NOTE: addr is removed from addrbook in markAddrInBookBasedOnErr
  473. r.attemptsToDial.Delete(addr.DialString())
  474. default:
  475. r.attemptsToDial.Store(addr.DialString(), _attemptsToDial{attempts + 1, time.Now()})
  476. }
  477. return errors.Wrapf(err, "dialing failed (attempts: %d)", attempts+1)
  478. }
  479. // cleanup any history
  480. r.attemptsToDial.Delete(addr.DialString())
  481. return nil
  482. }
  483. // checkSeeds checks that addresses are well formed.
  484. // Returns number of seeds we can connect to, along with all seeds addrs.
  485. // return err if user provided any badly formatted seed addresses.
  486. // Doesn't error if the seed node can't be reached.
  487. // numOnline returns -1 if no seed nodes were in the initial configuration.
  488. func (r *PEXReactor) checkSeeds() (numOnline int, netAddrs []*p2p.NetAddress, err error) {
  489. lSeeds := len(r.config.Seeds)
  490. if lSeeds == 0 {
  491. return -1, nil, nil
  492. }
  493. netAddrs, errs := p2p.NewNetAddressStrings(r.config.Seeds)
  494. numOnline = lSeeds - len(errs)
  495. for _, err := range errs {
  496. switch e := err.(type) {
  497. case p2p.ErrNetAddressLookup:
  498. r.Logger.Error("Connecting to seed failed", "err", e)
  499. default:
  500. return 0, nil, errors.Wrap(e, "seed node configuration has error")
  501. }
  502. }
  503. return numOnline, netAddrs, nil
  504. }
  505. // randomly dial seeds until we connect to one or exhaust them
  506. func (r *PEXReactor) dialSeeds() {
  507. perm := cmn.RandPerm(len(r.seedAddrs))
  508. // perm := r.Switch.rng.Perm(lSeeds)
  509. for _, i := range perm {
  510. // dial a random seed
  511. seedAddr := r.seedAddrs[i]
  512. err := r.Switch.DialPeerWithAddress(seedAddr)
  513. switch err.(type) {
  514. case nil, p2p.ErrCurrentlyDialingOrExistingAddress:
  515. return
  516. }
  517. r.Switch.Logger.Error("Error dialing seed", "err", err, "seed", seedAddr)
  518. }
  519. // do not write error message if there were no seeds specified in config
  520. if len(r.seedAddrs) > 0 {
  521. r.Switch.Logger.Error("Couldn't connect to any seeds")
  522. }
  523. }
  524. // AttemptsToDial returns the number of attempts to dial specific address. It
  525. // returns 0 if never attempted or successfully connected.
  526. func (r *PEXReactor) AttemptsToDial(addr *p2p.NetAddress) int {
  527. lAttempts, attempted := r.attemptsToDial.Load(addr.DialString())
  528. if attempted {
  529. return lAttempts.(_attemptsToDial).number
  530. }
  531. return 0
  532. }
  533. //----------------------------------------------------------
  534. // Explores the network searching for more peers. (continuous)
  535. // Seed/Crawler Mode causes this node to quickly disconnect
  536. // from peers, except other seed nodes.
  537. func (r *PEXReactor) crawlPeersRoutine() {
  538. // If we have any seed nodes, consult them first
  539. if len(r.seedAddrs) > 0 {
  540. r.dialSeeds()
  541. } else {
  542. // Do an initial crawl
  543. r.crawlPeers(r.book.GetSelection())
  544. }
  545. // Fire periodically
  546. ticker := time.NewTicker(crawlPeerPeriod)
  547. for {
  548. select {
  549. case <-ticker.C:
  550. r.attemptDisconnects()
  551. r.crawlPeers(r.book.GetSelection())
  552. r.cleanupCrawlPeerInfos()
  553. case <-r.Quit():
  554. return
  555. }
  556. }
  557. }
  558. // nodeHasSomePeersOrDialingAny returns true if the node is connected to some
  559. // peers or dialing them currently.
  560. func (r *PEXReactor) nodeHasSomePeersOrDialingAny() bool {
  561. out, in, dial := r.Switch.NumPeers()
  562. return out+in+dial > 0
  563. }
  564. // crawlPeerInfo handles temporary data needed for the network crawling
  565. // performed during seed/crawler mode.
  566. type crawlPeerInfo struct {
  567. Addr *p2p.NetAddress `json:"addr"`
  568. // The last time we crawled the peer or attempted to do so.
  569. LastCrawled time.Time `json:"last_crawled"`
  570. }
  571. // crawlPeers will crawl the network looking for new peer addresses.
  572. func (r *PEXReactor) crawlPeers(addrs []*p2p.NetAddress) {
  573. now := time.Now()
  574. for _, addr := range addrs {
  575. peerInfo, ok := r.crawlPeerInfos[addr.ID]
  576. // Do not attempt to connect with peers we recently crawled.
  577. if ok && now.Sub(peerInfo.LastCrawled) < minTimeBetweenCrawls {
  578. continue
  579. }
  580. // Record crawling attempt.
  581. r.crawlPeerInfos[addr.ID] = crawlPeerInfo{
  582. Addr: addr,
  583. LastCrawled: now,
  584. }
  585. err := r.dialPeer(addr)
  586. if err != nil {
  587. switch err.(type) {
  588. case errMaxAttemptsToDial, errTooEarlyToDial, p2p.ErrCurrentlyDialingOrExistingAddress:
  589. r.Logger.Debug(err.Error(), "addr", addr)
  590. default:
  591. r.Logger.Error(err.Error(), "addr", addr)
  592. }
  593. continue
  594. }
  595. peer := r.Switch.Peers().Get(addr.ID)
  596. if peer != nil {
  597. r.RequestAddrs(peer)
  598. }
  599. }
  600. }
  601. func (r *PEXReactor) cleanupCrawlPeerInfos() {
  602. for id, info := range r.crawlPeerInfos {
  603. // If we did not crawl a peer for 24 hours, it means the peer was removed
  604. // from the addrbook => remove
  605. //
  606. // 10000 addresses / maxGetSelection = 40 cycles to get all addresses in
  607. // the ideal case,
  608. // 40 * crawlPeerPeriod ~ 20 minutes
  609. if time.Since(info.LastCrawled) > 24*time.Hour {
  610. delete(r.crawlPeerInfos, id)
  611. }
  612. }
  613. }
  614. // attemptDisconnects checks if we've been with each peer long enough to disconnect
  615. func (r *PEXReactor) attemptDisconnects() {
  616. for _, peer := range r.Switch.Peers().List() {
  617. if peer.Status().Duration < r.config.SeedDisconnectWaitPeriod {
  618. continue
  619. }
  620. if peer.IsPersistent() {
  621. continue
  622. }
  623. r.Switch.StopPeerGracefully(peer)
  624. }
  625. }
  626. func markAddrInBookBasedOnErr(addr *p2p.NetAddress, book AddrBook, err error) {
  627. // TODO: detect more "bad peer" scenarios
  628. switch err.(type) {
  629. case p2p.ErrSwitchAuthenticationFailure:
  630. book.MarkBad(addr)
  631. default:
  632. book.MarkAttempt(addr)
  633. }
  634. }
  635. //-----------------------------------------------------------------------------
  636. // Messages
  637. // PexMessage is a primary type for PEX messages. Underneath, it could contain
  638. // either pexRequestMessage, or pexAddrsMessage messages.
  639. type PexMessage interface{}
  640. func RegisterPexMessage(cdc *amino.Codec) {
  641. cdc.RegisterInterface((*PexMessage)(nil), nil)
  642. cdc.RegisterConcrete(&pexRequestMessage{}, "tendermint/p2p/PexRequestMessage", nil)
  643. cdc.RegisterConcrete(&pexAddrsMessage{}, "tendermint/p2p/PexAddrsMessage", nil)
  644. }
  645. func decodeMsg(bz []byte) (msg PexMessage, err error) {
  646. if len(bz) > maxMsgSize {
  647. return msg, fmt.Errorf("Msg exceeds max size (%d > %d)", len(bz), maxMsgSize)
  648. }
  649. err = cdc.UnmarshalBinaryBare(bz, &msg)
  650. return
  651. }
  652. /*
  653. A pexRequestMessage requests additional peer addresses.
  654. */
  655. type pexRequestMessage struct {
  656. }
  657. func (m *pexRequestMessage) String() string {
  658. return "[pexRequest]"
  659. }
  660. /*
  661. A message with announced peer addresses.
  662. */
  663. type pexAddrsMessage struct {
  664. Addrs []*p2p.NetAddress
  665. }
  666. func (m *pexAddrsMessage) String() string {
  667. return fmt.Sprintf("[pexAddrs %v]", m.Addrs)
  668. }