REP-6503 Avoid $type in partition queries unless needed #129
+572
−503
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tdq45gj
approved these changes
Aug 20, 2025
| @@ -91,7 +91,7 @@ var bsonTypeString = map[bsontype.Type]string{ | |||
| // This is kind of like strings.Cut() but against the sort-ordered list of BSON | |||
| // types, except that if the given value is a number or string-like, then other | |||
| // “like” types will not be in the returned slices. | |||
| func getTypeBracketExcludedBSONTypes(val any) ([]string, []string, error) { | |||
| func getTypeBracketExcludedBSONTypes(val any) ([]bsontype.Type, []bsontype.Type, error) { | |||
There was a problem hiding this comment.
The comment for this function still says it returns string slices.
internal/partitions/partition.go
Outdated
| } | ||
|
|
||
| // filterWithExplicitTypeChecks compensates for the server’s type bracketing | ||
| // getExplicitTypeCheckPredicates compensates for the server’s type bracketing | ||
| // by matching _id types between the partition’s min & max boundaries. | ||
| // It should yield the same result as filterWithExpr. |
There was a problem hiding this comment.
Note that this comment needs to be updated.
| // < 5.0, will not use indexes and thus will be very slow | ||
| func (p *Partition) filterWithExpr() bson.D { | ||
| // < 5.0, will not use indexes and thus will be very slow. | ||
| func getExprPredicates(lower, upper any) ([]bson.D, error) { |
There was a problem hiding this comment.
We're okay with $expr because the query cannot use PROJECTION_COVERED IXSCAN and would need to FETCH anyway?
There was a problem hiding this comment.
This changeset addresses a performance problem that surfaced specifically with the non-$expr queries.
If there are improvements to make to the $expr queries we could probably do those separately.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The $type checks that REP-6129 (PR #118) added to fix type bracketing have impeded performance against 4.4 clusters.
This changeset makes partition queries use $type only when the range requires it, which should only be true for a handful of partitions anyhow. The queries are simplified, too, so that instead of ANDing together predicates like
_id >= 12 OR $type in ["string", "symbol", ...])andid <= 99 OR $type in ["null", "minkey"]we just have a single $type invocation (if it’s even there, which it usually won’t be now).The existing checks for type-bracketing accommodation are expanded a bit here as well.