You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

322 lines
8.6 KiB

libs/pubsub: relax tx querying (#4070) Some linting/cleanup missed from the initial events refactor Don't panic; instead, return false, error when matching breaks unexpectedly Strip non-numeric chars from values when attempting to match against query values Have the server log during send upon error * cleanup/lint Query#Conditions and do not panic * cleanup/lint Query#Matches and do not panic * cleanup/lint matchValue and do not panic * rever to panic in Query#Conditions * linting * strip alpha chars when attempting to match * add pending log entries * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * build: update variable names * update matchValue to return an error * update Query#Matches to return an error * update TestMatches * log error in send * Fix tests * Fix TestEmptyQueryMatchesAnything * fix linting * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update libs/pubsub/pubsub.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * add missing errors pkg import * update Query#Conditions to return an error * update query pkg unit tests * update TxIndex#Search * update pending changelog
5 years ago
libs/pubsub: relax tx querying (#4070) Some linting/cleanup missed from the initial events refactor Don't panic; instead, return false, error when matching breaks unexpectedly Strip non-numeric chars from values when attempting to match against query values Have the server log during send upon error * cleanup/lint Query#Conditions and do not panic * cleanup/lint Query#Matches and do not panic * cleanup/lint matchValue and do not panic * rever to panic in Query#Conditions * linting * strip alpha chars when attempting to match * add pending log entries * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * build: update variable names * update matchValue to return an error * update Query#Matches to return an error * update TestMatches * log error in send * Fix tests * Fix TestEmptyQueryMatchesAnything * fix linting * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update libs/pubsub/pubsub.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * add missing errors pkg import * update Query#Conditions to return an error * update query pkg unit tests * update TxIndex#Search * update pending changelog
5 years ago
abci: Refactor tagging events using list of lists (#3643) ## PR This PR introduces a fundamental breaking change to the structure of ABCI response and tx tags and the way they're processed. Namely, the SDK can support more complex and aggregated events for distribution and slashing. In addition, block responses can include duplicate keys in events. Implement new Event type. An event has a type and a list of KV pairs (ie. list-of-lists). Typical events may look like: "rewards": [{"amount": "5000uatom", "validator": "...", "recipient": "..."}] "sender": [{"address": "...", "balance": "100uatom"}] The events are indexed by {even.type}.{even.attribute[i].key}/.... In this case a client would subscribe or query for rewards.recipient='...' ABCI response types and related types now include Events []Event instead of Tags []cmn.KVPair. PubSub logic now publishes/matches against map[string][]string instead of map[string]string to support duplicate keys in response events (from #1385). A match is successful if the value is found in the slice of strings. closes: #1859 closes: #2905 ## Commits: * Implement Event ABCI type and updates responses to use events * Update messages_test.go * Update kvstore.go * Update event_bus.go * Update subscription.go * Update pubsub.go * Update kvstore.go * Update query logic to handle slice of strings in events * Update Empty#Matches and unit tests * Update pubsub logic * Update EventBus#Publish * Update kv tx indexer * Update godocs * Update ResultEvent to use slice of strings; update RPC * Update more tests * Update abci.md * Check for key in validateAndStringifyEvents * Fix KV indexer to skip empty keys * Fix linting errors * Update CHANGELOG_PENDING.md * Update docs/spec/abci/abci.md Co-Authored-By: Federico Kunze <31522760+fedekunze@users.noreply.github.com> * Update abci/types/types.proto Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update docs/spec/abci/abci.md Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update libs/pubsub/query/query.go Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update match function to match if ANY value matches * Implement TestSubscribeDuplicateKeys * Update TestMatches to include multi-key test cases * Update events.go * Update Query interface godoc * Update match godoc * Add godoc for matchValue * DRY-up tx indexing * Return error from PublishWithEvents in EventBus#Publish * Update PublishEventNewBlockHeader to return an error * Fix build * Update events doc in ABCI * Update ABCI events godoc * Implement TestEventBusPublishEventTxDuplicateKeys * Update TestSubscribeDuplicateKeys to be table-driven * Remove mod file * Remove markdown from events godoc * Implement TestTxSearchDeprecatedIndexing test
5 years ago
abci: Refactor tagging events using list of lists (#3643) ## PR This PR introduces a fundamental breaking change to the structure of ABCI response and tx tags and the way they're processed. Namely, the SDK can support more complex and aggregated events for distribution and slashing. In addition, block responses can include duplicate keys in events. Implement new Event type. An event has a type and a list of KV pairs (ie. list-of-lists). Typical events may look like: "rewards": [{"amount": "5000uatom", "validator": "...", "recipient": "..."}] "sender": [{"address": "...", "balance": "100uatom"}] The events are indexed by {even.type}.{even.attribute[i].key}/.... In this case a client would subscribe or query for rewards.recipient='...' ABCI response types and related types now include Events []Event instead of Tags []cmn.KVPair. PubSub logic now publishes/matches against map[string][]string instead of map[string]string to support duplicate keys in response events (from #1385). A match is successful if the value is found in the slice of strings. closes: #1859 closes: #2905 ## Commits: * Implement Event ABCI type and updates responses to use events * Update messages_test.go * Update kvstore.go * Update event_bus.go * Update subscription.go * Update pubsub.go * Update kvstore.go * Update query logic to handle slice of strings in events * Update Empty#Matches and unit tests * Update pubsub logic * Update EventBus#Publish * Update kv tx indexer * Update godocs * Update ResultEvent to use slice of strings; update RPC * Update more tests * Update abci.md * Check for key in validateAndStringifyEvents * Fix KV indexer to skip empty keys * Fix linting errors * Update CHANGELOG_PENDING.md * Update docs/spec/abci/abci.md Co-Authored-By: Federico Kunze <31522760+fedekunze@users.noreply.github.com> * Update abci/types/types.proto Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update docs/spec/abci/abci.md Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update libs/pubsub/query/query.go Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update match function to match if ANY value matches * Implement TestSubscribeDuplicateKeys * Update TestMatches to include multi-key test cases * Update events.go * Update Query interface godoc * Update match godoc * Add godoc for matchValue * DRY-up tx indexing * Return error from PublishWithEvents in EventBus#Publish * Update PublishEventNewBlockHeader to return an error * Fix build * Update events doc in ABCI * Update ABCI events godoc * Implement TestEventBusPublishEventTxDuplicateKeys * Update TestSubscribeDuplicateKeys to be table-driven * Remove mod file * Remove markdown from events godoc * Implement TestTxSearchDeprecatedIndexing test
5 years ago
abci: Refactor tagging events using list of lists (#3643) ## PR This PR introduces a fundamental breaking change to the structure of ABCI response and tx tags and the way they're processed. Namely, the SDK can support more complex and aggregated events for distribution and slashing. In addition, block responses can include duplicate keys in events. Implement new Event type. An event has a type and a list of KV pairs (ie. list-of-lists). Typical events may look like: "rewards": [{"amount": "5000uatom", "validator": "...", "recipient": "..."}] "sender": [{"address": "...", "balance": "100uatom"}] The events are indexed by {even.type}.{even.attribute[i].key}/.... In this case a client would subscribe or query for rewards.recipient='...' ABCI response types and related types now include Events []Event instead of Tags []cmn.KVPair. PubSub logic now publishes/matches against map[string][]string instead of map[string]string to support duplicate keys in response events (from #1385). A match is successful if the value is found in the slice of strings. closes: #1859 closes: #2905 ## Commits: * Implement Event ABCI type and updates responses to use events * Update messages_test.go * Update kvstore.go * Update event_bus.go * Update subscription.go * Update pubsub.go * Update kvstore.go * Update query logic to handle slice of strings in events * Update Empty#Matches and unit tests * Update pubsub logic * Update EventBus#Publish * Update kv tx indexer * Update godocs * Update ResultEvent to use slice of strings; update RPC * Update more tests * Update abci.md * Check for key in validateAndStringifyEvents * Fix KV indexer to skip empty keys * Fix linting errors * Update CHANGELOG_PENDING.md * Update docs/spec/abci/abci.md Co-Authored-By: Federico Kunze <31522760+fedekunze@users.noreply.github.com> * Update abci/types/types.proto Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update docs/spec/abci/abci.md Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update libs/pubsub/query/query.go Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update match function to match if ANY value matches * Implement TestSubscribeDuplicateKeys * Update TestMatches to include multi-key test cases * Update events.go * Update Query interface godoc * Update match godoc * Add godoc for matchValue * DRY-up tx indexing * Return error from PublishWithEvents in EventBus#Publish * Update PublishEventNewBlockHeader to return an error * Fix build * Update events doc in ABCI * Update ABCI events godoc * Implement TestEventBusPublishEventTxDuplicateKeys * Update TestSubscribeDuplicateKeys to be table-driven * Remove mod file * Remove markdown from events godoc * Implement TestTxSearchDeprecatedIndexing test
5 years ago
abci: Refactor tagging events using list of lists (#3643) ## PR This PR introduces a fundamental breaking change to the structure of ABCI response and tx tags and the way they're processed. Namely, the SDK can support more complex and aggregated events for distribution and slashing. In addition, block responses can include duplicate keys in events. Implement new Event type. An event has a type and a list of KV pairs (ie. list-of-lists). Typical events may look like: "rewards": [{"amount": "5000uatom", "validator": "...", "recipient": "..."}] "sender": [{"address": "...", "balance": "100uatom"}] The events are indexed by {even.type}.{even.attribute[i].key}/.... In this case a client would subscribe or query for rewards.recipient='...' ABCI response types and related types now include Events []Event instead of Tags []cmn.KVPair. PubSub logic now publishes/matches against map[string][]string instead of map[string]string to support duplicate keys in response events (from #1385). A match is successful if the value is found in the slice of strings. closes: #1859 closes: #2905 ## Commits: * Implement Event ABCI type and updates responses to use events * Update messages_test.go * Update kvstore.go * Update event_bus.go * Update subscription.go * Update pubsub.go * Update kvstore.go * Update query logic to handle slice of strings in events * Update Empty#Matches and unit tests * Update pubsub logic * Update EventBus#Publish * Update kv tx indexer * Update godocs * Update ResultEvent to use slice of strings; update RPC * Update more tests * Update abci.md * Check for key in validateAndStringifyEvents * Fix KV indexer to skip empty keys * Fix linting errors * Update CHANGELOG_PENDING.md * Update docs/spec/abci/abci.md Co-Authored-By: Federico Kunze <31522760+fedekunze@users.noreply.github.com> * Update abci/types/types.proto Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update docs/spec/abci/abci.md Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update libs/pubsub/query/query.go Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update match function to match if ANY value matches * Implement TestSubscribeDuplicateKeys * Update TestMatches to include multi-key test cases * Update events.go * Update Query interface godoc * Update match godoc * Add godoc for matchValue * DRY-up tx indexing * Return error from PublishWithEvents in EventBus#Publish * Update PublishEventNewBlockHeader to return an error * Fix build * Update events doc in ABCI * Update ABCI events godoc * Implement TestEventBusPublishEventTxDuplicateKeys * Update TestSubscribeDuplicateKeys to be table-driven * Remove mod file * Remove markdown from events godoc * Implement TestTxSearchDeprecatedIndexing test
5 years ago
abci: Refactor tagging events using list of lists (#3643) ## PR This PR introduces a fundamental breaking change to the structure of ABCI response and tx tags and the way they're processed. Namely, the SDK can support more complex and aggregated events for distribution and slashing. In addition, block responses can include duplicate keys in events. Implement new Event type. An event has a type and a list of KV pairs (ie. list-of-lists). Typical events may look like: "rewards": [{"amount": "5000uatom", "validator": "...", "recipient": "..."}] "sender": [{"address": "...", "balance": "100uatom"}] The events are indexed by {even.type}.{even.attribute[i].key}/.... In this case a client would subscribe or query for rewards.recipient='...' ABCI response types and related types now include Events []Event instead of Tags []cmn.KVPair. PubSub logic now publishes/matches against map[string][]string instead of map[string]string to support duplicate keys in response events (from #1385). A match is successful if the value is found in the slice of strings. closes: #1859 closes: #2905 ## Commits: * Implement Event ABCI type and updates responses to use events * Update messages_test.go * Update kvstore.go * Update event_bus.go * Update subscription.go * Update pubsub.go * Update kvstore.go * Update query logic to handle slice of strings in events * Update Empty#Matches and unit tests * Update pubsub logic * Update EventBus#Publish * Update kv tx indexer * Update godocs * Update ResultEvent to use slice of strings; update RPC * Update more tests * Update abci.md * Check for key in validateAndStringifyEvents * Fix KV indexer to skip empty keys * Fix linting errors * Update CHANGELOG_PENDING.md * Update docs/spec/abci/abci.md Co-Authored-By: Federico Kunze <31522760+fedekunze@users.noreply.github.com> * Update abci/types/types.proto Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update docs/spec/abci/abci.md Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update libs/pubsub/query/query.go Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update match function to match if ANY value matches * Implement TestSubscribeDuplicateKeys * Update TestMatches to include multi-key test cases * Update events.go * Update Query interface godoc * Update match godoc * Add godoc for matchValue * DRY-up tx indexing * Return error from PublishWithEvents in EventBus#Publish * Update PublishEventNewBlockHeader to return an error * Fix build * Update events doc in ABCI * Update ABCI events godoc * Implement TestEventBusPublishEventTxDuplicateKeys * Update TestSubscribeDuplicateKeys to be table-driven * Remove mod file * Remove markdown from events godoc * Implement TestTxSearchDeprecatedIndexing test
5 years ago
abci: Refactor tagging events using list of lists (#3643) ## PR This PR introduces a fundamental breaking change to the structure of ABCI response and tx tags and the way they're processed. Namely, the SDK can support more complex and aggregated events for distribution and slashing. In addition, block responses can include duplicate keys in events. Implement new Event type. An event has a type and a list of KV pairs (ie. list-of-lists). Typical events may look like: "rewards": [{"amount": "5000uatom", "validator": "...", "recipient": "..."}] "sender": [{"address": "...", "balance": "100uatom"}] The events are indexed by {even.type}.{even.attribute[i].key}/.... In this case a client would subscribe or query for rewards.recipient='...' ABCI response types and related types now include Events []Event instead of Tags []cmn.KVPair. PubSub logic now publishes/matches against map[string][]string instead of map[string]string to support duplicate keys in response events (from #1385). A match is successful if the value is found in the slice of strings. closes: #1859 closes: #2905 ## Commits: * Implement Event ABCI type and updates responses to use events * Update messages_test.go * Update kvstore.go * Update event_bus.go * Update subscription.go * Update pubsub.go * Update kvstore.go * Update query logic to handle slice of strings in events * Update Empty#Matches and unit tests * Update pubsub logic * Update EventBus#Publish * Update kv tx indexer * Update godocs * Update ResultEvent to use slice of strings; update RPC * Update more tests * Update abci.md * Check for key in validateAndStringifyEvents * Fix KV indexer to skip empty keys * Fix linting errors * Update CHANGELOG_PENDING.md * Update docs/spec/abci/abci.md Co-Authored-By: Federico Kunze <31522760+fedekunze@users.noreply.github.com> * Update abci/types/types.proto Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update docs/spec/abci/abci.md Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update libs/pubsub/query/query.go Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update match function to match if ANY value matches * Implement TestSubscribeDuplicateKeys * Update TestMatches to include multi-key test cases * Update events.go * Update Query interface godoc * Update match godoc * Add godoc for matchValue * DRY-up tx indexing * Return error from PublishWithEvents in EventBus#Publish * Update PublishEventNewBlockHeader to return an error * Fix build * Update events doc in ABCI * Update ABCI events godoc * Implement TestEventBusPublishEventTxDuplicateKeys * Update TestSubscribeDuplicateKeys to be table-driven * Remove mod file * Remove markdown from events godoc * Implement TestTxSearchDeprecatedIndexing test
5 years ago
libs/pubsub: relax tx querying (#4070) Some linting/cleanup missed from the initial events refactor Don't panic; instead, return false, error when matching breaks unexpectedly Strip non-numeric chars from values when attempting to match against query values Have the server log during send upon error * cleanup/lint Query#Conditions and do not panic * cleanup/lint Query#Matches and do not panic * cleanup/lint matchValue and do not panic * rever to panic in Query#Conditions * linting * strip alpha chars when attempting to match * add pending log entries * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * build: update variable names * update matchValue to return an error * update Query#Matches to return an error * update TestMatches * log error in send * Fix tests * Fix TestEmptyQueryMatchesAnything * fix linting * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update libs/pubsub/query/query.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * Update libs/pubsub/pubsub.go Co-Authored-By: Anton Kaliaev <anton.kalyaev@gmail.com> * add missing errors pkg import * update Query#Conditions to return an error * update query pkg unit tests * update TxIndex#Search * update pending changelog
5 years ago
abci: Refactor tagging events using list of lists (#3643) ## PR This PR introduces a fundamental breaking change to the structure of ABCI response and tx tags and the way they're processed. Namely, the SDK can support more complex and aggregated events for distribution and slashing. In addition, block responses can include duplicate keys in events. Implement new Event type. An event has a type and a list of KV pairs (ie. list-of-lists). Typical events may look like: "rewards": [{"amount": "5000uatom", "validator": "...", "recipient": "..."}] "sender": [{"address": "...", "balance": "100uatom"}] The events are indexed by {even.type}.{even.attribute[i].key}/.... In this case a client would subscribe or query for rewards.recipient='...' ABCI response types and related types now include Events []Event instead of Tags []cmn.KVPair. PubSub logic now publishes/matches against map[string][]string instead of map[string]string to support duplicate keys in response events (from #1385). A match is successful if the value is found in the slice of strings. closes: #1859 closes: #2905 ## Commits: * Implement Event ABCI type and updates responses to use events * Update messages_test.go * Update kvstore.go * Update event_bus.go * Update subscription.go * Update pubsub.go * Update kvstore.go * Update query logic to handle slice of strings in events * Update Empty#Matches and unit tests * Update pubsub logic * Update EventBus#Publish * Update kv tx indexer * Update godocs * Update ResultEvent to use slice of strings; update RPC * Update more tests * Update abci.md * Check for key in validateAndStringifyEvents * Fix KV indexer to skip empty keys * Fix linting errors * Update CHANGELOG_PENDING.md * Update docs/spec/abci/abci.md Co-Authored-By: Federico Kunze <31522760+fedekunze@users.noreply.github.com> * Update abci/types/types.proto Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update docs/spec/abci/abci.md Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update libs/pubsub/query/query.go Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update match function to match if ANY value matches * Implement TestSubscribeDuplicateKeys * Update TestMatches to include multi-key test cases * Update events.go * Update Query interface godoc * Update match godoc * Add godoc for matchValue * DRY-up tx indexing * Return error from PublishWithEvents in EventBus#Publish * Update PublishEventNewBlockHeader to return an error * Fix build * Update events doc in ABCI * Update ABCI events godoc * Implement TestEventBusPublishEventTxDuplicateKeys * Update TestSubscribeDuplicateKeys to be table-driven * Remove mod file * Remove markdown from events godoc * Implement TestTxSearchDeprecatedIndexing test
5 years ago
abci: Refactor tagging events using list of lists (#3643) ## PR This PR introduces a fundamental breaking change to the structure of ABCI response and tx tags and the way they're processed. Namely, the SDK can support more complex and aggregated events for distribution and slashing. In addition, block responses can include duplicate keys in events. Implement new Event type. An event has a type and a list of KV pairs (ie. list-of-lists). Typical events may look like: "rewards": [{"amount": "5000uatom", "validator": "...", "recipient": "..."}] "sender": [{"address": "...", "balance": "100uatom"}] The events are indexed by {even.type}.{even.attribute[i].key}/.... In this case a client would subscribe or query for rewards.recipient='...' ABCI response types and related types now include Events []Event instead of Tags []cmn.KVPair. PubSub logic now publishes/matches against map[string][]string instead of map[string]string to support duplicate keys in response events (from #1385). A match is successful if the value is found in the slice of strings. closes: #1859 closes: #2905 ## Commits: * Implement Event ABCI type and updates responses to use events * Update messages_test.go * Update kvstore.go * Update event_bus.go * Update subscription.go * Update pubsub.go * Update kvstore.go * Update query logic to handle slice of strings in events * Update Empty#Matches and unit tests * Update pubsub logic * Update EventBus#Publish * Update kv tx indexer * Update godocs * Update ResultEvent to use slice of strings; update RPC * Update more tests * Update abci.md * Check for key in validateAndStringifyEvents * Fix KV indexer to skip empty keys * Fix linting errors * Update CHANGELOG_PENDING.md * Update docs/spec/abci/abci.md Co-Authored-By: Federico Kunze <31522760+fedekunze@users.noreply.github.com> * Update abci/types/types.proto Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update docs/spec/abci/abci.md Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update libs/pubsub/query/query.go Co-Authored-By: Ethan Buchman <ethan@coinculture.info> * Update match function to match if ANY value matches * Implement TestSubscribeDuplicateKeys * Update TestMatches to include multi-key test cases * Update events.go * Update Query interface godoc * Update match godoc * Add godoc for matchValue * DRY-up tx indexing * Return error from PublishWithEvents in EventBus#Publish * Update PublishEventNewBlockHeader to return an error * Fix build * Update events doc in ABCI * Update ABCI events godoc * Implement TestEventBusPublishEventTxDuplicateKeys * Update TestSubscribeDuplicateKeys to be table-driven * Remove mod file * Remove markdown from events godoc * Implement TestTxSearchDeprecatedIndexing test
5 years ago
  1. // Package query implements the custom query format used to filter event
  2. // subscriptions in Tendermint.
  3. //
  4. // Query expressions describe properties of events and their attributes, using
  5. // strings like:
  6. //
  7. // abci.invoice.number = 22 AND abci.invoice.owner = 'Ivan'
  8. //
  9. // Query expressions can handle attribute values encoding numbers, strings,
  10. // dates, and timestamps. The complete query grammar is described in the
  11. // query/syntax package.
  12. //
  13. package query
  14. import (
  15. "fmt"
  16. "regexp"
  17. "strconv"
  18. "strings"
  19. "time"
  20. "github.com/tendermint/tendermint/abci/types"
  21. "github.com/tendermint/tendermint/internal/pubsub/query/syntax"
  22. )
  23. // All is a query that matches all events.
  24. var All *Query
  25. // A Query is the compiled form of a query.
  26. type Query struct {
  27. ast syntax.Query
  28. conds []condition
  29. }
  30. // New parses and compiles the query expression into an executable query.
  31. func New(query string) (*Query, error) {
  32. ast, err := syntax.Parse(query)
  33. if err != nil {
  34. return nil, err
  35. }
  36. return Compile(ast)
  37. }
  38. // MustCompile compiles the query expression into an executable query.
  39. // In case of error, MustCompile will panic.
  40. //
  41. // This is intended for use in program initialization; use query.New if you
  42. // need to check errors.
  43. func MustCompile(query string) *Query {
  44. q, err := New(query)
  45. if err != nil {
  46. panic(err)
  47. }
  48. return q
  49. }
  50. // Compile compiles the given query AST so it can be used to match events.
  51. func Compile(ast syntax.Query) (*Query, error) {
  52. conds := make([]condition, len(ast))
  53. for i, q := range ast {
  54. cond, err := compileCondition(q)
  55. if err != nil {
  56. return nil, fmt.Errorf("compile %s: %w", q, err)
  57. }
  58. conds[i] = cond
  59. }
  60. return &Query{ast: ast, conds: conds}, nil
  61. }
  62. // Matches reports whether q matches the given events. If q == nil, the query
  63. // matches any non-empty collection of events.
  64. func (q *Query) Matches(events []types.Event) bool {
  65. if q == nil {
  66. return true
  67. }
  68. for _, cond := range q.conds {
  69. if !cond.matchesAny(events) {
  70. return false
  71. }
  72. }
  73. return len(events) != 0
  74. }
  75. // String matches part of the pubsub.Query interface.
  76. func (q *Query) String() string {
  77. if q == nil {
  78. return "<empty>"
  79. }
  80. return q.ast.String()
  81. }
  82. // Syntax returns the syntax tree representation of q.
  83. func (q *Query) Syntax() syntax.Query {
  84. if q == nil {
  85. return nil
  86. }
  87. return q.ast
  88. }
  89. // A condition is a compiled match condition. A condition matches an event if
  90. // the event has the designated type, contains an attribute with the given
  91. // name, and the match function returns true for the attribute value.
  92. type condition struct {
  93. tag string // e.g., "tx.hash"
  94. match func(s string) bool
  95. }
  96. // findAttr returns a slice of attribute values from event matching the
  97. // condition tag, and reports whether the event type strictly equals the
  98. // condition tag.
  99. func (c condition) findAttr(event types.Event) ([]string, bool) {
  100. if !strings.HasPrefix(c.tag, event.Type) {
  101. return nil, false // type does not match tag
  102. } else if len(c.tag) == len(event.Type) {
  103. return nil, true // type == tag
  104. }
  105. var vals []string
  106. for _, attr := range event.Attributes {
  107. fullName := event.Type + "." + attr.Key
  108. if fullName == c.tag {
  109. vals = append(vals, attr.Value)
  110. }
  111. }
  112. return vals, false
  113. }
  114. // matchesAny reports whether c matches at least one of the given events.
  115. func (c condition) matchesAny(events []types.Event) bool {
  116. for _, event := range events {
  117. if c.matchesEvent(event) {
  118. return true
  119. }
  120. }
  121. return false
  122. }
  123. // matchesEvent reports whether c matches the given event.
  124. func (c condition) matchesEvent(event types.Event) bool {
  125. vs, tagEqualsType := c.findAttr(event)
  126. if len(vs) == 0 {
  127. // As a special case, a condition tag that exactly matches the event type
  128. // is matched against an empty string. This allows existence checks to
  129. // work for type-only queries.
  130. if tagEqualsType {
  131. return c.match("")
  132. }
  133. return false
  134. }
  135. // At this point, we have candidate values.
  136. for _, v := range vs {
  137. if c.match(v) {
  138. return true
  139. }
  140. }
  141. return false
  142. }
  143. func compileCondition(cond syntax.Condition) (condition, error) {
  144. out := condition{tag: cond.Tag}
  145. // Handle existence checks separately to simplify the logic below for
  146. // comparisons that take arguments.
  147. if cond.Op == syntax.TExists {
  148. out.match = func(string) bool { return true }
  149. return out, nil
  150. }
  151. // All the other operators require an argument.
  152. if cond.Arg == nil {
  153. return condition{}, fmt.Errorf("missing argument for %v", cond.Op)
  154. }
  155. // Precompile the argument value matcher.
  156. argType := cond.Arg.Type
  157. var argValue interface{}
  158. switch argType {
  159. case syntax.TString:
  160. argValue = cond.Arg.Value()
  161. case syntax.TNumber:
  162. argValue = cond.Arg.Number()
  163. case syntax.TTime, syntax.TDate:
  164. argValue = cond.Arg.Time()
  165. default:
  166. return condition{}, fmt.Errorf("unknown argument type %v", argType)
  167. }
  168. mcons := opTypeMap[cond.Op][argType]
  169. if mcons == nil {
  170. return condition{}, fmt.Errorf("invalid op/arg combination (%v, %v)", cond.Op, argType)
  171. }
  172. out.match = mcons(argValue)
  173. return out, nil
  174. }
  175. // TODO(creachadair): The existing implementation allows anything number shaped
  176. // to be treated as a number. This preserves the parts of that behavior we had
  177. // tests for, but we should probably get rid of that.
  178. var extractNum = regexp.MustCompile(`^\d+(\.\d+)?`)
  179. func parseNumber(s string) (float64, error) {
  180. return strconv.ParseFloat(extractNum.FindString(s), 64)
  181. }
  182. // A map of operator ⇒ argtype ⇒ match-constructor.
  183. // An entry does not exist if the combination is not valid.
  184. //
  185. // Disable the dupl lint for this map. The result isn't even correct.
  186. //nolint:dupl
  187. var opTypeMap = map[syntax.Token]map[syntax.Token]func(interface{}) func(string) bool{
  188. syntax.TContains: {
  189. syntax.TString: func(v interface{}) func(string) bool {
  190. return func(s string) bool {
  191. return strings.Contains(s, v.(string))
  192. }
  193. },
  194. },
  195. syntax.TEq: {
  196. syntax.TString: func(v interface{}) func(string) bool {
  197. return func(s string) bool { return s == v.(string) }
  198. },
  199. syntax.TNumber: func(v interface{}) func(string) bool {
  200. return func(s string) bool {
  201. w, err := parseNumber(s)
  202. return err == nil && w == v.(float64)
  203. }
  204. },
  205. syntax.TDate: func(v interface{}) func(string) bool {
  206. return func(s string) bool {
  207. ts, err := syntax.ParseDate(s)
  208. return err == nil && ts.Equal(v.(time.Time))
  209. }
  210. },
  211. syntax.TTime: func(v interface{}) func(string) bool {
  212. return func(s string) bool {
  213. ts, err := syntax.ParseTime(s)
  214. return err == nil && ts.Equal(v.(time.Time))
  215. }
  216. },
  217. },
  218. syntax.TLt: {
  219. syntax.TNumber: func(v interface{}) func(string) bool {
  220. return func(s string) bool {
  221. w, err := parseNumber(s)
  222. return err == nil && w < v.(float64)
  223. }
  224. },
  225. syntax.TDate: func(v interface{}) func(string) bool {
  226. return func(s string) bool {
  227. ts, err := syntax.ParseDate(s)
  228. return err == nil && ts.Before(v.(time.Time))
  229. }
  230. },
  231. syntax.TTime: func(v interface{}) func(string) bool {
  232. return func(s string) bool {
  233. ts, err := syntax.ParseTime(s)
  234. return err == nil && ts.Before(v.(time.Time))
  235. }
  236. },
  237. },
  238. syntax.TLeq: {
  239. syntax.TNumber: func(v interface{}) func(string) bool {
  240. return func(s string) bool {
  241. w, err := parseNumber(s)
  242. return err == nil && w <= v.(float64)
  243. }
  244. },
  245. syntax.TDate: func(v interface{}) func(string) bool {
  246. return func(s string) bool {
  247. ts, err := syntax.ParseDate(s)
  248. return err == nil && !ts.After(v.(time.Time))
  249. }
  250. },
  251. syntax.TTime: func(v interface{}) func(string) bool {
  252. return func(s string) bool {
  253. ts, err := syntax.ParseTime(s)
  254. return err == nil && !ts.After(v.(time.Time))
  255. }
  256. },
  257. },
  258. syntax.TGt: {
  259. syntax.TNumber: func(v interface{}) func(string) bool {
  260. return func(s string) bool {
  261. w, err := parseNumber(s)
  262. return err == nil && w > v.(float64)
  263. }
  264. },
  265. syntax.TDate: func(v interface{}) func(string) bool {
  266. return func(s string) bool {
  267. ts, err := syntax.ParseDate(s)
  268. return err == nil && ts.After(v.(time.Time))
  269. }
  270. },
  271. syntax.TTime: func(v interface{}) func(string) bool {
  272. return func(s string) bool {
  273. ts, err := syntax.ParseTime(s)
  274. return err == nil && ts.After(v.(time.Time))
  275. }
  276. },
  277. },
  278. syntax.TGeq: {
  279. syntax.TNumber: func(v interface{}) func(string) bool {
  280. return func(s string) bool {
  281. w, err := parseNumber(s)
  282. return err == nil && w >= v.(float64)
  283. }
  284. },
  285. syntax.TDate: func(v interface{}) func(string) bool {
  286. return func(s string) bool {
  287. ts, err := syntax.ParseDate(s)
  288. return err == nil && !ts.Before(v.(time.Time))
  289. }
  290. },
  291. syntax.TTime: func(v interface{}) func(string) bool {
  292. return func(s string) bool {
  293. ts, err := syntax.ParseTime(s)
  294. return err == nil && !ts.Before(v.(time.Time))
  295. }
  296. },
  297. },
  298. }