Skip to content

Conversation

rodrigozhou
Copy link
Contributor

@rodrigozhou rodrigozhou commented Sep 9, 2025

What changed?

Unify Visibility query converters.

The new query converter is the struct query.QueryConverter[ExprT]. The generic parameter specify the output type of the converter (in ES is elastic.Query, in SQL is sqlparser.Expr).

The query converter has a query.StoreQueryConverter[ExprT]. This is the Visibility store specific implementation for building the output query.

For example, the store needs to implement BuildAndExpr(exprs) to tell the query converter how to build the AND expression. In SQL, it's *sqlparser.AndExpr{exprs}. In Elasticsearch, it's elastic.NewBoolQuery().Filter(exprs).

The store query converter for SQL needs a plugin query converter called sqlplugin.VisibilityQueryConverter which is implemented by each plugin (MySQL, PostgreSQL, SQLite). This interface is needed to build DB specific syntax for KeywordList and Text searches as well as to build the final SELECT statements.

Added dynamic config system.VisibilityEnableUnifiedQueryConverter that acts as switch between the legacy and the unified query converters.

Why?

There are two query converters: one for Elasticsearch, and another for SQL.
They are somewhat similar, and have quite a few code duplications for validation, modifying the query, etc.
Unifying the query converter ensures that we provide the exact same behavior across different Visibility stores.

How did you test it?

  • built
  • run locally and tested manually
  • covered by existing tests
  • added new unit test(s)
  • added new functional test(s)

I've added a lot of unit tests for the new code with coverage close to 100%.

Note that there has been basically no changes to functional tests, which suggests that the change works well.

Potential risks

There might be some edge cases that either query converters that this unified query converter might deal differently.

@rodrigozhou rodrigozhou force-pushed the rodrigozhou/refactor-query-converter branch 4 times, most recently from 944ea45 to a9c3f44 Compare September 10, 2025 18:22
@rodrigozhou rodrigozhou marked this pull request as ready for review September 10, 2025 18:31
@rodrigozhou rodrigozhou requested a review from a team as a code owner September 10, 2025 18:31
@rodrigozhou rodrigozhou changed the base branch from main to rodrigozhou/rename-legacy-qc September 16, 2025 23:04
@rodrigozhou rodrigozhou force-pushed the rodrigozhou/refactor-query-converter branch from a9c3f44 to 43351c5 Compare September 16, 2025 23:04
@rodrigozhou rodrigozhou changed the title Refactor visibility query converter Unified query converter for Visibility Sep 16, 2025
@rodrigozhou rodrigozhou force-pushed the rodrigozhou/refactor-query-converter branch 2 times, most recently from c70c5b9 to 9c750df Compare September 16, 2025 23:33
@rodrigozhou rodrigozhou force-pushed the rodrigozhou/rename-legacy-qc branch from e15f659 to 7084e98 Compare September 16, 2025 23:34
@rodrigozhou rodrigozhou changed the base branch from rodrigozhou/rename-legacy-qc to main September 16, 2025 23:38
@rodrigozhou rodrigozhou force-pushed the rodrigozhou/refactor-query-converter branch from 9c750df to 33e3f76 Compare September 17, 2025 00:28
@rodrigozhou rodrigozhou force-pushed the rodrigozhou/refactor-query-converter branch from 33e3f76 to ff59665 Compare September 17, 2025 02:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant