zolfa
/
tendermint
1
0
Fork 0
Code Issues 0 Pull Requests 0 Projects 0 Releases 221 Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
243 Commits
183 Branches
291 MiB
Tree: a6a06f820f
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from 'a6a06f820f'
${ noResults }
tendermint/pubsub/query/parser_test.go

88 lines
2.6 KiB
Raw Normal View History

new events package query parser use parser compiler to generate query parser I used https://github.com/pointlander/peg which has a nice API and seems to be the most popular Golang compiler parser using PEG on Github. More about PEG: - https://en.wikipedia.org/wiki/Parsing_expression_grammar - https://github.com/PhilippeSigaud/Pegged/wiki/PEG-Basics - https://github.com/PhilippeSigaud/Pegged/wiki/Grammar-Examples rename implement query match function match function uncomment test lines add more test cases for query#Matches fix int case rename events to pubsub add comment about cache assertReceive helper to not block on receive in tests fix bug with multiple conditions uncomment benchmark first results: ``` Benchmark10Clients-2 1000 1305493 ns/op 3957519 B/op 355 allocs/op Benchmark100Clients-2 100 12278304 ns/op 39571751 B/op 3505 allocs/op Benchmark1000Clients-2 10 124120909 ns/op 395714004 B/op 35005 allocs/op ``` 124ms to publish message to 1000 clients. A lot. use AST from query.peg.go separate pubsub and query packages by using Query interface in pubsub wrote docs and refactor code updates from Frey's review refactor type assertion to use type switch cleanup during shutdown subscriber should create output channel, not the server overflow strategies, server buffer capacity context as the first argument for Publish log error introduce Option type update NewServer comment move helpers into pubsub_test increase assertReceive timeout add query.MustParse add more false tests for parser add more false tests for query.Matches parse numbers as int64 / float64 try our best to convert from other types add number to panic output add more comments save commit introduce client argument as first argument to Subscribe > Why we do not specify buffer size on the output channel in Subscribe? The choice of buffer size of N here depends on knowing the number of messages server will receive and the number of messages downstream subscribers will consume. This is fragile: if we publish an additional message, or if one of the downstream subscribers reads any fewer messages, we will again have blocked goroutines. save commit remove reference counting fix test test client resubscribe test UnsubscribeAll client options [pubsub/query] fuzzy testing do not print msg as it creates data race!
8 years ago
​
​
​
​
​
​
​
​
​
​
​
​
​
​
[pubsub/query] quote values using single quotes This fixes the problem with base-16 encoded values which may start with digits: 015AB.... In such cases, the parser recognizes them as numbers but fails to parse because of the follow-up characters (AB). ``` failed to parse tm.events.type=Tx AND hash=136E18F7E4C348B780CF873A0BF43922E5BAFA63: parse error near digit (line 1 symbol 31 - line 1 symbol 32): "6" ``` So, from now on we should quote any values. This seems to be the way Postgresql has chosen.
8 years ago
​
​
​
​
​
​
​
new events package query parser use parser compiler to generate query parser I used https://github.com/pointlander/peg which has a nice API and seems to be the most popular Golang compiler parser using PEG on Github. More about PEG: - https://en.wikipedia.org/wiki/Parsing_expression_grammar - https://github.com/PhilippeSigaud/Pegged/wiki/PEG-Basics - https://github.com/PhilippeSigaud/Pegged/wiki/Grammar-Examples rename implement query match function match function uncomment test lines add more test cases for query#Matches fix int case rename events to pubsub add comment about cache assertReceive helper to not block on receive in tests fix bug with multiple conditions uncomment benchmark first results: ``` Benchmark10Clients-2 1000 1305493 ns/op 3957519 B/op 355 allocs/op Benchmark100Clients-2 100 12278304 ns/op 39571751 B/op 3505 allocs/op Benchmark1000Clients-2 10 124120909 ns/op 395714004 B/op 35005 allocs/op ``` 124ms to publish message to 1000 clients. A lot. use AST from query.peg.go separate pubsub and query packages by using Query interface in pubsub wrote docs and refactor code updates from Frey's review refactor type assertion to use type switch cleanup during shutdown subscriber should create output channel, not the server overflow strategies, server buffer capacity context as the first argument for Publish log error introduce Option type update NewServer comment move helpers into pubsub_test increase assertReceive timeout add query.MustParse add more false tests for parser add more false tests for query.Matches parse numbers as int64 / float64 try our best to convert from other types add number to panic output add more comments save commit introduce client argument as first argument to Subscribe > Why we do not specify buffer size on the output channel in Subscribe? The choice of buffer size of N here depends on knowing the number of messages server will receive and the number of messages downstream subscribers will consume. This is fragile: if we publish an additional message, or if one of the downstream subscribers reads any fewer messages, we will again have blocked goroutines. save commit remove reference counting fix test test client resubscribe test UnsubscribeAll client options [pubsub/query] fuzzy testing do not print msg as it creates data race!
8 years ago
[pubsub/query] quote values using single quotes This fixes the problem with base-16 encoded values which may start with digits: 015AB.... In such cases, the parser recognizes them as numbers but fails to parse because of the follow-up characters (AB). ``` failed to parse tm.events.type=Tx AND hash=136E18F7E4C348B780CF873A0BF43922E5BAFA63: parse error near digit (line 1 symbol 31 - line 1 symbol 32): "6" ``` So, from now on we should quote any values. This seems to be the way Postgresql has chosen.
8 years ago
new events package query parser use parser compiler to generate query parser I used https://github.com/pointlander/peg which has a nice API and seems to be the most popular Golang compiler parser using PEG on Github. More about PEG: - https://en.wikipedia.org/wiki/Parsing_expression_grammar - https://github.com/PhilippeSigaud/Pegged/wiki/PEG-Basics - https://github.com/PhilippeSigaud/Pegged/wiki/Grammar-Examples rename implement query match function match function uncomment test lines add more test cases for query#Matches fix int case rename events to pubsub add comment about cache assertReceive helper to not block on receive in tests fix bug with multiple conditions uncomment benchmark first results: ``` Benchmark10Clients-2 1000 1305493 ns/op 3957519 B/op 355 allocs/op Benchmark100Clients-2 100 12278304 ns/op 39571751 B/op 3505 allocs/op Benchmark1000Clients-2 10 124120909 ns/op 395714004 B/op 35005 allocs/op ``` 124ms to publish message to 1000 clients. A lot. use AST from query.peg.go separate pubsub and query packages by using Query interface in pubsub wrote docs and refactor code updates from Frey's review refactor type assertion to use type switch cleanup during shutdown subscriber should create output channel, not the server overflow strategies, server buffer capacity context as the first argument for Publish log error introduce Option type update NewServer comment move helpers into pubsub_test increase assertReceive timeout add query.MustParse add more false tests for parser add more false tests for query.Matches parse numbers as int64 / float64 try our best to convert from other types add number to panic output add more comments save commit introduce client argument as first argument to Subscribe > Why we do not specify buffer size on the output channel in Subscribe? The choice of buffer size of N here depends on knowing the number of messages server will receive and the number of messages downstream subscribers will consume. This is fragile: if we publish an additional message, or if one of the downstream subscribers reads any fewer messages, we will again have blocked goroutines. save commit remove reference counting fix test test client resubscribe test UnsubscribeAll client options [pubsub/query] fuzzy testing do not print msg as it creates data race!
8 years ago
[pubsub/query] quote values using single quotes This fixes the problem with base-16 encoded values which may start with digits: 015AB.... In such cases, the parser recognizes them as numbers but fails to parse because of the follow-up characters (AB). ``` failed to parse tm.events.type=Tx AND hash=136E18F7E4C348B780CF873A0BF43922E5BAFA63: parse error near digit (line 1 symbol 31 - line 1 symbol 32): "6" ``` So, from now on we should quote any values. This seems to be the way Postgresql has chosen.
8 years ago
​
new events package query parser use parser compiler to generate query parser I used https://github.com/pointlander/peg which has a nice API and seems to be the most popular Golang compiler parser using PEG on Github. More about PEG: - https://en.wikipedia.org/wiki/Parsing_expression_grammar - https://github.com/PhilippeSigaud/Pegged/wiki/PEG-Basics - https://github.com/PhilippeSigaud/Pegged/wiki/Grammar-Examples rename implement query match function match function uncomment test lines add more test cases for query#Matches fix int case rename events to pubsub add comment about cache assertReceive helper to not block on receive in tests fix bug with multiple conditions uncomment benchmark first results: ``` Benchmark10Clients-2 1000 1305493 ns/op 3957519 B/op 355 allocs/op Benchmark100Clients-2 100 12278304 ns/op 39571751 B/op 3505 allocs/op Benchmark1000Clients-2 10 124120909 ns/op 395714004 B/op 35005 allocs/op ``` 124ms to publish message to 1000 clients. A lot. use AST from query.peg.go separate pubsub and query packages by using Query interface in pubsub wrote docs and refactor code updates from Frey's review refactor type assertion to use type switch cleanup during shutdown subscriber should create output channel, not the server overflow strategies, server buffer capacity context as the first argument for Publish log error introduce Option type update NewServer comment move helpers into pubsub_test increase assertReceive timeout add query.MustParse add more false tests for parser add more false tests for query.Matches parse numbers as int64 / float64 try our best to convert from other types add number to panic output add more comments save commit introduce client argument as first argument to Subscribe > Why we do not specify buffer size on the output channel in Subscribe? The choice of buffer size of N here depends on knowing the number of messages server will receive and the number of messages downstream subscribers will consume. This is fragile: if we publish an additional message, or if one of the downstream subscribers reads any fewer messages, we will again have blocked goroutines. save commit remove reference counting fix test test client resubscribe test UnsubscribeAll client options [pubsub/query] fuzzy testing do not print msg as it creates data race!
8 years ago
​
[pubsub/query] quote values using single quotes This fixes the problem with base-16 encoded values which may start with digits: 015AB.... In such cases, the parser recognizes them as numbers but fails to parse because of the follow-up characters (AB). ``` failed to parse tm.events.type=Tx AND hash=136E18F7E4C348B780CF873A0BF43922E5BAFA63: parse error near digit (line 1 symbol 31 - line 1 symbol 32): "6" ``` So, from now on we should quote any values. This seems to be the way Postgresql has chosen.
8 years ago
new events package query parser use parser compiler to generate query parser I used https://github.com/pointlander/peg which has a nice API and seems to be the most popular Golang compiler parser using PEG on Github. More about PEG: - https://en.wikipedia.org/wiki/Parsing_expression_grammar - https://github.com/PhilippeSigaud/Pegged/wiki/PEG-Basics - https://github.com/PhilippeSigaud/Pegged/wiki/Grammar-Examples rename implement query match function match function uncomment test lines add more test cases for query#Matches fix int case rename events to pubsub add comment about cache assertReceive helper to not block on receive in tests fix bug with multiple conditions uncomment benchmark first results: ``` Benchmark10Clients-2 1000 1305493 ns/op 3957519 B/op 355 allocs/op Benchmark100Clients-2 100 12278304 ns/op 39571751 B/op 3505 allocs/op Benchmark1000Clients-2 10 124120909 ns/op 395714004 B/op 35005 allocs/op ``` 124ms to publish message to 1000 clients. A lot. use AST from query.peg.go separate pubsub and query packages by using Query interface in pubsub wrote docs and refactor code updates from Frey's review refactor type assertion to use type switch cleanup during shutdown subscriber should create output channel, not the server overflow strategies, server buffer capacity context as the first argument for Publish log error introduce Option type update NewServer comment move helpers into pubsub_test increase assertReceive timeout add query.MustParse add more false tests for parser add more false tests for query.Matches parse numbers as int64 / float64 try our best to convert from other types add number to panic output add more comments save commit introduce client argument as first argument to Subscribe > Why we do not specify buffer size on the output channel in Subscribe? The choice of buffer size of N here depends on knowing the number of messages server will receive and the number of messages downstream subscribers will consume. This is fragile: if we publish an additional message, or if one of the downstream subscribers reads any fewer messages, we will again have blocked goroutines. save commit remove reference counting fix test test client resubscribe test UnsubscribeAll client options [pubsub/query] fuzzy testing do not print msg as it creates data race!
8 years ago
​
​
[pubsub/query] quote values using single quotes This fixes the problem with base-16 encoded values which may start with digits: 015AB.... In such cases, the parser recognizes them as numbers but fails to parse because of the follow-up characters (AB). ``` failed to parse tm.events.type=Tx AND hash=136E18F7E4C348B780CF873A0BF43922E5BAFA63: parse error near digit (line 1 symbol 31 - line 1 symbol 32): "6" ``` So, from now on we should quote any values. This seems to be the way Postgresql has chosen.
8 years ago
​
new events package query parser use parser compiler to generate query parser I used https://github.com/pointlander/peg which has a nice API and seems to be the most popular Golang compiler parser using PEG on Github. More about PEG: - https://en.wikipedia.org/wiki/Parsing_expression_grammar - https://github.com/PhilippeSigaud/Pegged/wiki/PEG-Basics - https://github.com/PhilippeSigaud/Pegged/wiki/Grammar-Examples rename implement query match function match function uncomment test lines add more test cases for query#Matches fix int case rename events to pubsub add comment about cache assertReceive helper to not block on receive in tests fix bug with multiple conditions uncomment benchmark first results: ``` Benchmark10Clients-2 1000 1305493 ns/op 3957519 B/op 355 allocs/op Benchmark100Clients-2 100 12278304 ns/op 39571751 B/op 3505 allocs/op Benchmark1000Clients-2 10 124120909 ns/op 395714004 B/op 35005 allocs/op ``` 124ms to publish message to 1000 clients. A lot. use AST from query.peg.go separate pubsub and query packages by using Query interface in pubsub wrote docs and refactor code updates from Frey's review refactor type assertion to use type switch cleanup during shutdown subscriber should create output channel, not the server overflow strategies, server buffer capacity context as the first argument for Publish log error introduce Option type update NewServer comment move helpers into pubsub_test increase assertReceive timeout add query.MustParse add more false tests for parser add more false tests for query.Matches parse numbers as int64 / float64 try our best to convert from other types add number to panic output add more comments save commit introduce client argument as first argument to Subscribe > Why we do not specify buffer size on the output channel in Subscribe? The choice of buffer size of N here depends on knowing the number of messages server will receive and the number of messages downstream subscribers will consume. This is fragile: if we publish an additional message, or if one of the downstream subscribers reads any fewer messages, we will again have blocked goroutines. save commit remove reference counting fix test test client resubscribe test UnsubscribeAll client options [pubsub/query] fuzzy testing do not print msg as it creates data race!
8 years ago
​
​
[pubsub/query] quote values using single quotes This fixes the problem with base-16 encoded values which may start with digits: 015AB.... In such cases, the parser recognizes them as numbers but fails to parse because of the follow-up characters (AB). ``` failed to parse tm.events.type=Tx AND hash=136E18F7E4C348B780CF873A0BF43922E5BAFA63: parse error near digit (line 1 symbol 31 - line 1 symbol 32): "6" ``` So, from now on we should quote any values. This seems to be the way Postgresql has chosen.
8 years ago
​
​
​
​
new events package query parser use parser compiler to generate query parser I used https://github.com/pointlander/peg which has a nice API and seems to be the most popular Golang compiler parser using PEG on Github. More about PEG: - https://en.wikipedia.org/wiki/Parsing_expression_grammar - https://github.com/PhilippeSigaud/Pegged/wiki/PEG-Basics - https://github.com/PhilippeSigaud/Pegged/wiki/Grammar-Examples rename implement query match function match function uncomment test lines add more test cases for query#Matches fix int case rename events to pubsub add comment about cache assertReceive helper to not block on receive in tests fix bug with multiple conditions uncomment benchmark first results: ``` Benchmark10Clients-2 1000 1305493 ns/op 3957519 B/op 355 allocs/op Benchmark100Clients-2 100 12278304 ns/op 39571751 B/op 3505 allocs/op Benchmark1000Clients-2 10 124120909 ns/op 395714004 B/op 35005 allocs/op ``` 124ms to publish message to 1000 clients. A lot. use AST from query.peg.go separate pubsub and query packages by using Query interface in pubsub wrote docs and refactor code updates from Frey's review refactor type assertion to use type switch cleanup during shutdown subscriber should create output channel, not the server overflow strategies, server buffer capacity context as the first argument for Publish log error introduce Option type update NewServer comment move helpers into pubsub_test increase assertReceive timeout add query.MustParse add more false tests for parser add more false tests for query.Matches parse numbers as int64 / float64 try our best to convert from other types add number to panic output add more comments save commit introduce client argument as first argument to Subscribe > Why we do not specify buffer size on the output channel in Subscribe? The choice of buffer size of N here depends on knowing the number of messages server will receive and the number of messages downstream subscribers will consume. This is fragile: if we publish an additional message, or if one of the downstream subscribers reads any fewer messages, we will again have blocked goroutines. save commit remove reference counting fix test test client resubscribe test UnsubscribeAll client options [pubsub/query] fuzzy testing do not print msg as it creates data race!
8 years ago
[pubsub/query] quote values using single quotes This fixes the problem with base-16 encoded values which may start with digits: 015AB.... In such cases, the parser recognizes them as numbers but fails to parse because of the follow-up characters (AB). ``` failed to parse tm.events.type=Tx AND hash=136E18F7E4C348B780CF873A0BF43922E5BAFA63: parse error near digit (line 1 symbol 31 - line 1 symbol 32): "6" ``` So, from now on we should quote any values. This seems to be the way Postgresql has chosen.
8 years ago
new events package query parser use parser compiler to generate query parser I used https://github.com/pointlander/peg which has a nice API and seems to be the most popular Golang compiler parser using PEG on Github. More about PEG: - https://en.wikipedia.org/wiki/Parsing_expression_grammar - https://github.com/PhilippeSigaud/Pegged/wiki/PEG-Basics - https://github.com/PhilippeSigaud/Pegged/wiki/Grammar-Examples rename implement query match function match function uncomment test lines add more test cases for query#Matches fix int case rename events to pubsub add comment about cache assertReceive helper to not block on receive in tests fix bug with multiple conditions uncomment benchmark first results: ``` Benchmark10Clients-2 1000 1305493 ns/op 3957519 B/op 355 allocs/op Benchmark100Clients-2 100 12278304 ns/op 39571751 B/op 3505 allocs/op Benchmark1000Clients-2 10 124120909 ns/op 395714004 B/op 35005 allocs/op ``` 124ms to publish message to 1000 clients. A lot. use AST from query.peg.go separate pubsub and query packages by using Query interface in pubsub wrote docs and refactor code updates from Frey's review refactor type assertion to use type switch cleanup during shutdown subscriber should create output channel, not the server overflow strategies, server buffer capacity context as the first argument for Publish log error introduce Option type update NewServer comment move helpers into pubsub_test increase assertReceive timeout add query.MustParse add more false tests for parser add more false tests for query.Matches parse numbers as int64 / float64 try our best to convert from other types add number to panic output add more comments save commit introduce client argument as first argument to Subscribe > Why we do not specify buffer size on the output channel in Subscribe? The choice of buffer size of N here depends on knowing the number of messages server will receive and the number of messages downstream subscribers will consume. This is fragile: if we publish an additional message, or if one of the downstream subscribers reads any fewer messages, we will again have blocked goroutines. save commit remove reference counting fix test test client resubscribe test UnsubscribeAll client options [pubsub/query] fuzzy testing do not print msg as it creates data race!
8 years ago
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
 
  1. package query_test
  2. 

  3. import (
  4. 	"testing"
  5. 

  6. 	"github.com/stretchr/testify/assert"
  7. 	"github.com/tendermint/tmlibs/pubsub/query"
  8. )
  9. 

  10. // TODO: fuzzy testing?

  11. func TestParser(t *testing.T) {
  12. 	cases := []struct {
  13. 		query string
  14. 		valid bool
  15. 	}{
  16. 		{"tm.events.type='NewBlock'", true},
  17. 		{"tm.events.type = 'NewBlock'", true},
  18. 		{"tm.events.name = ''", true},
  19. 		{"tm.events.type='TIME'", true},
  20. 		{"tm.events.type='DATE'", true},
  21. 		{"tm.events.type='='", true},
  22. 		{"tm.events.type='TIME", false},
  23. 		{"tm.events.type=TIME'", false},
  24. 		{"tm.events.type==", false},
  25. 		{"tm.events.type=NewBlock", false},
  26. 		{">==", false},
  27. 		{"tm.events.type 'NewBlock' =", false},
  28. 		{"tm.events.type>'NewBlock'", false},
  29. 		{"", false},
  30. 		{"=", false},
  31. 		{"='NewBlock'", false},
  32. 		{"tm.events.type=", false},
  33. 

  34. 		{"tm.events.typeNewBlock", false},
  35. 		{"tm.events.type'NewBlock'", false},
  36. 		{"'NewBlock'", false},
  37. 		{"NewBlock", false},
  38. 		{"", false},
  39. 

  40. 		{"tm.events.type='NewBlock' AND abci.account.name='Igor'", true},
  41. 		{"tm.events.type='NewBlock' AND", false},
  42. 		{"tm.events.type='NewBlock' AN", false},
  43. 		{"tm.events.type='NewBlock' AN tm.events.type='NewBlockHeader'", false},
  44. 		{"AND tm.events.type='NewBlock' ", false},
  45. 

  46. 		{"abci.account.name CONTAINS 'Igor'", true},
  47. 

  48. 		{"tx.date > DATE 2013-05-03", true},
  49. 		{"tx.date < DATE 2013-05-03", true},
  50. 		{"tx.date <= DATE 2013-05-03", true},
  51. 		{"tx.date >= DATE 2013-05-03", true},
  52. 		{"tx.date >= DAT 2013-05-03", false},
  53. 		{"tx.date <= DATE2013-05-03", false},
  54. 		{"tx.date <= DATE -05-03", false},
  55. 		{"tx.date >= DATE 20130503", false},
  56. 		{"tx.date >= DATE 2013+01-03", false},
  57. 		// incorrect year, month, day

  58. 		{"tx.date >= DATE 0013-01-03", false},
  59. 		{"tx.date >= DATE 2013-31-03", false},
  60. 		{"tx.date >= DATE 2013-01-83", false},
  61. 

  62. 		{"tx.date > TIME 2013-05-03T14:45:00+07:00", true},
  63. 		{"tx.date < TIME 2013-05-03T14:45:00-02:00", true},
  64. 		{"tx.date <= TIME 2013-05-03T14:45:00Z", true},
  65. 		{"tx.date >= TIME 2013-05-03T14:45:00Z", true},
  66. 		{"tx.date >= TIME2013-05-03T14:45:00Z", false},
  67. 		{"tx.date = IME 2013-05-03T14:45:00Z", false},
  68. 		{"tx.date = TIME 2013-05-:45:00Z", false},
  69. 		{"tx.date >= TIME 2013-05-03T14:45:00", false},
  70. 		{"tx.date >= TIME 0013-00-00T14:45:00Z", false},
  71. 		{"tx.date >= TIME 2013+05=03T14:45:00Z", false},
  72. 

  73. 		{"account.balance=100", true},
  74. 		{"account.balance >= 200", true},
  75. 		{"account.balance >= -300", false},
  76. 		{"account.balance >>= 400", false},
  77. 		{"account.balance=33.22.1", false},
  78. 	}
  79. 

  80. 	for _, c := range cases {
  81. 		_, err := query.New(c.query)
  82. 		if c.valid {
  83. 			assert.NoError(t, err, "Query was '%s'", c.query)
  84. 		} else {
  85. 			assert.Error(t, err, "Query was '%s'", c.query)
  86. 		}
  87. 	}
  88. }