You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

234 lines
6.3 KiB

add support for block pruning via ABCI Commit response (#4588) * Added BlockStore.DeleteBlock() * Added initial block pruner prototype * wip * Added BlockStore.PruneBlocks() * Added consensus setting for block pruning * Added BlockStore base * Error on replay if base does not have blocks * Handle missing blocks when sending VoteSetMaj23Message * Error message tweak * Properly update blockstore state * Error message fix again * blockchain: ignore peer missing blocks * Added FIXME * Added test for block replay with truncated history * Handle peer base in blockchain reactor * Improved replay error handling * Added tests for Store.PruneBlocks() * Fix non-RPC handling of truncated block history * Panic on missing block meta in needProofBlock() * Updated changelog * Handle truncated block history in RPC layer * Added info about earliest block in /status RPC * Reorder height and base in blockchain reactor messages * Updated changelog * Fix tests * Appease linter * Minor review fixes * Non-empty BlockStores should always have base > 0 * Update code to assume base > 0 invariant * Added blockstore tests for pruning to 0 * Make sure we don't prune below the current base * Added BlockStore.Size() * config: added retain_blocks recommendations * Update v1 blockchain reactor to handle blockstore base * Added state database pruning * Propagate errors on missing validator sets * Comment tweaks * Improved error message Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * use ABCI field ResponseCommit.retain_height instead of retain-blocks config option * remove State.RetainHeight, return value instead * fix minor issues * rename pruneHeights() to pruneBlocks() * noop to fix GitHub borkage Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
5 years ago
abci: Refactor tagging events using list of lists (#3643) ## PR This PR introduces a fundamental breaking change to the structure of ABCI response and tx tags and the way they're processed. Namely, the SDK can support more complex and aggregated events for distribution and slashing. In addition, block responses can include duplicate keys in events. Implement new Event type. An event has a type and a list of KV pairs (ie. list-of-lists). Typical events may look like: "rewards": [{"amount": "5000uatom", "validator": "...", "recipient": "..."}] "sender": [{"address": "...", "balance": "100uatom"}] The events are indexed by {even.type}.{even.attribute[i].key}/.... In this case a client would subscribe or query for rewards.recipient='...' ABCI response types and related types now include Events []Event instead of Tags []cmn.KVPair. PubSub logic now publishes/matches against map[string][]string instead of map[string]string to support duplicate keys in response events (from #1385). A match is successful if the value is found in the slice of strings. closes: #1859 closes: #2905 ## Commits: * Implement Event ABCI type and updates responses to use events * Update messages_test.go * Update kvstore.go * Update event_bus.go * Update subscription.go * Update pubsub.go * Update kvstore.go * Update query logic to handle slice of strings in events * Update Empty#Matches and unit tests * Update pubsub logic * Update EventBus#Publish * Update kv tx indexer * Update godocs * Update ResultEvent to use slice of strings; update RPC * Update more tests * Update abci.md * Check for key in validateAndStringifyEvents * Fix KV indexer to skip empty keys * Fix linting errors * Update CHANGELOG_PENDING.md * Update docs/spec/abci/abci.md Co-Authored-By: Federico Kunze <31522760+fedekunze@users.noreply.github.com> * Update abci/types/types.proto Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update docs/spec/abci/abci.md Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update libs/pubsub/query/query.go Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update match function to match if ANY value matches * Implement TestSubscribeDuplicateKeys * Update TestMatches to include multi-key test cases * Update events.go * Update Query interface godoc * Update match godoc * Add godoc for matchValue * DRY-up tx indexing * Return error from PublishWithEvents in EventBus#Publish * Update PublishEventNewBlockHeader to return an error * Fix build * Update events doc in ABCI * Update ABCI events godoc * Implement TestEventBusPublishEventTxDuplicateKeys * Update TestSubscribeDuplicateKeys to be table-driven * Remove mod file * Remove markdown from events godoc * Implement TestTxSearchDeprecatedIndexing test
5 years ago
  1. package coretypes
  2. import (
  3. "encoding/json"
  4. "time"
  5. abci "github.com/tendermint/tendermint/abci/types"
  6. "github.com/tendermint/tendermint/crypto"
  7. "github.com/tendermint/tendermint/libs/bytes"
  8. "github.com/tendermint/tendermint/p2p"
  9. "github.com/tendermint/tendermint/types"
  10. )
  11. // List of blocks
  12. type ResultBlockchainInfo struct {
  13. LastHeight int64 `json:"last_height"`
  14. BlockMetas []*types.BlockMeta `json:"block_metas"`
  15. }
  16. // Genesis file
  17. type ResultGenesis struct {
  18. Genesis *types.GenesisDoc `json:"genesis"`
  19. }
  20. // Single block (with meta)
  21. type ResultBlock struct {
  22. BlockID types.BlockID `json:"block_id"`
  23. Block *types.Block `json:"block"`
  24. }
  25. // Commit and Header
  26. type ResultCommit struct {
  27. types.SignedHeader `json:"signed_header"`
  28. CanonicalCommit bool `json:"canonical"`
  29. }
  30. // ABCI results from a block
  31. type ResultBlockResults struct {
  32. Height int64 `json:"height"`
  33. TxsResults []*abci.ResponseDeliverTx `json:"txs_results"`
  34. BeginBlockEvents []abci.Event `json:"begin_block_events"`
  35. EndBlockEvents []abci.Event `json:"end_block_events"`
  36. ValidatorUpdates []abci.ValidatorUpdate `json:"validator_updates"`
  37. ConsensusParamUpdates *abci.ConsensusParams `json:"consensus_param_updates"`
  38. }
  39. // NewResultCommit is a helper to initialize the ResultCommit with
  40. // the embedded struct
  41. func NewResultCommit(header *types.Header, commit *types.Commit,
  42. canonical bool) *ResultCommit {
  43. return &ResultCommit{
  44. SignedHeader: types.SignedHeader{
  45. Header: header,
  46. Commit: commit,
  47. },
  48. CanonicalCommit: canonical,
  49. }
  50. }
  51. // Info about the node's syncing state
  52. type SyncInfo struct {
  53. LatestBlockHash bytes.HexBytes `json:"latest_block_hash"`
  54. LatestAppHash bytes.HexBytes `json:"latest_app_hash"`
  55. LatestBlockHeight int64 `json:"latest_block_height"`
  56. LatestBlockTime time.Time `json:"latest_block_time"`
  57. EarliestBlockHash bytes.HexBytes `json:"earliest_block_hash"`
  58. EarliestAppHash bytes.HexBytes `json:"earliest_app_hash"`
  59. EarliestBlockHeight int64 `json:"earliest_block_height"`
  60. EarliestBlockTime time.Time `json:"earliest_block_time"`
  61. CatchingUp bool `json:"catching_up"`
  62. }
  63. // Info about the node's validator
  64. type ValidatorInfo struct {
  65. Address bytes.HexBytes `json:"address"`
  66. PubKey crypto.PubKey `json:"pub_key"`
  67. VotingPower int64 `json:"voting_power"`
  68. }
  69. // Node Status
  70. type ResultStatus struct {
  71. NodeInfo p2p.NodeInfo `json:"node_info"`
  72. SyncInfo SyncInfo `json:"sync_info"`
  73. ValidatorInfo ValidatorInfo `json:"validator_info"`
  74. }
  75. // Is TxIndexing enabled
  76. func (s *ResultStatus) TxIndexEnabled() bool {
  77. if s == nil {
  78. return false
  79. }
  80. return s.NodeInfo.Other.TxIndex == "on"
  81. }
  82. // Info about peer connections
  83. type ResultNetInfo struct {
  84. Listening bool `json:"listening"`
  85. Listeners []string `json:"listeners"`
  86. NPeers int `json:"n_peers"`
  87. Peers []Peer `json:"peers"`
  88. }
  89. // Log from dialing seeds
  90. type ResultDialSeeds struct {
  91. Log string `json:"log"`
  92. }
  93. // Log from dialing peers
  94. type ResultDialPeers struct {
  95. Log string `json:"log"`
  96. }
  97. // A peer
  98. type Peer struct {
  99. NodeInfo p2p.NodeInfo `json:"node_info"`
  100. IsOutbound bool `json:"is_outbound"`
  101. ConnectionStatus p2p.ConnectionStatus `json:"connection_status"`
  102. RemoteIP string `json:"remote_ip"`
  103. }
  104. // Validators for a height.
  105. type ResultValidators struct {
  106. BlockHeight int64 `json:"block_height"`
  107. Validators []*types.Validator `json:"validators"`
  108. // Count of actual validators in this result
  109. Count int `json:"count"`
  110. // Total number of validators
  111. Total int `json:"total"`
  112. }
  113. // ConsensusParams for given height
  114. type ResultConsensusParams struct {
  115. BlockHeight int64 `json:"block_height"`
  116. ConsensusParams types.ConsensusParams `json:"consensus_params"`
  117. }
  118. // Info about the consensus state.
  119. // UNSTABLE
  120. type ResultDumpConsensusState struct {
  121. RoundState json.RawMessage `json:"round_state"`
  122. Peers []PeerStateInfo `json:"peers"`
  123. }
  124. // UNSTABLE
  125. type PeerStateInfo struct {
  126. NodeAddress string `json:"node_address"`
  127. PeerState json.RawMessage `json:"peer_state"`
  128. }
  129. // UNSTABLE
  130. type ResultConsensusState struct {
  131. RoundState json.RawMessage `json:"round_state"`
  132. }
  133. // CheckTx result
  134. type ResultBroadcastTx struct {
  135. Code uint32 `json:"code"`
  136. Data bytes.HexBytes `json:"data"`
  137. Log string `json:"log"`
  138. Codespace string `json:"codespace"`
  139. Hash bytes.HexBytes `json:"hash"`
  140. }
  141. // CheckTx and DeliverTx results
  142. type ResultBroadcastTxCommit struct {
  143. CheckTx abci.ResponseCheckTx `json:"check_tx"`
  144. DeliverTx abci.ResponseDeliverTx `json:"deliver_tx"`
  145. Hash bytes.HexBytes `json:"hash"`
  146. Height int64 `json:"height"`
  147. }
  148. // ResultCheckTx wraps abci.ResponseCheckTx.
  149. type ResultCheckTx struct {
  150. abci.ResponseCheckTx
  151. }
  152. // Result of querying for a tx
  153. type ResultTx struct {
  154. Hash bytes.HexBytes `json:"hash"`
  155. Height int64 `json:"height"`
  156. Index uint32 `json:"index"`
  157. TxResult abci.ResponseDeliverTx `json:"tx_result"`
  158. Tx types.Tx `json:"tx"`
  159. Proof types.TxProof `json:"proof,omitempty"`
  160. }
  161. // Result of searching for txs
  162. type ResultTxSearch struct {
  163. Txs []*ResultTx `json:"txs"`
  164. TotalCount int `json:"total_count"`
  165. }
  166. // List of mempool txs
  167. type ResultUnconfirmedTxs struct {
  168. Count int `json:"n_txs"`
  169. Total int `json:"total"`
  170. TotalBytes int64 `json:"total_bytes"`
  171. Txs []types.Tx `json:"txs"`
  172. }
  173. // Info abci msg
  174. type ResultABCIInfo struct {
  175. Response abci.ResponseInfo `json:"response"`
  176. }
  177. // Query abci msg
  178. type ResultABCIQuery struct {
  179. Response abci.ResponseQuery `json:"response"`
  180. }
  181. // Result of broadcasting evidence
  182. type ResultBroadcastEvidence struct {
  183. Hash []byte `json:"hash"`
  184. }
  185. // empty results
  186. type (
  187. ResultUnsafeFlushMempool struct{}
  188. ResultUnsafeProfile struct{}
  189. ResultSubscribe struct{}
  190. ResultUnsubscribe struct{}
  191. ResultHealth struct{}
  192. )
  193. // Event data from a subscription
  194. type ResultEvent struct {
  195. Query string `json:"query"`
  196. Data types.TMEventData `json:"data"`
  197. Events map[string][]string `json:"events"`
  198. }