Unified query converter for Visibility #8307
Open
+10,047
−2,146
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changed?
Unify Visibility query converters.
The new query converter is the struct
query.QueryConverter[ExprT]
. The generic parameter specify the output type of the converter (in ES iselastic.Query
, in SQL issqlparser.Expr
).The query converter has a
query.StoreQueryConverter[ExprT]
. This is the Visibility store specific implementation for building the output query.For example, the store needs to implement
BuildAndExpr(exprs)
to tell the query converter how to build theAND
expression. In SQL, it's*sqlparser.AndExpr{exprs}
. In Elasticsearch, it'selastic.NewBoolQuery().Filter(exprs)
.The store query converter for SQL needs a plugin query converter called
sqlplugin.VisibilityQueryConverter
which is implemented by each plugin (MySQL, PostgreSQL, SQLite). This interface is needed to build DB specific syntax forKeywordList
andText
searches as well as to build the finalSELECT
statements.Added dynamic config
system.VisibilityEnableUnifiedQueryConverter
that acts as switch between the legacy and the unified query converters.Why?
There are two query converters: one for Elasticsearch, and another for SQL.
They are somewhat similar, and have quite a few code duplications for validation, modifying the query, etc.
Unifying the query converter ensures that we provide the exact same behavior across different Visibility stores.
How did you test it?
I've added a lot of unit tests for the new code with coverage close to 100%.
Note that there has been basically no changes to functional tests, which suggests that the change works well.
Potential risks
There might be some edge cases that either query converters that this unified query converter might deal differently.