v1/topdown/graphql: Cache GraphQL schema parse results (#5377) #7457

robmyersrobmyers · 2025-03-20T16:03:08Z

This commit stores parsed GraphQL schemas to the cache, which improves the performance of GraphQL operations that parse the schema more than once.

Queries are not cached.

pkg: github.com/open-policy-agent/opa/v1/topdown
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_string-16	                   15519            100178 ns/op
BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_string-16               371311              3383 ns/op
BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_object-16                            2133            542355 ns/op
BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_object-16                 3471            528579 ns/op
BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_AST_object-16                        7105            193325 ns/op
BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_AST_object-16            66594             18093 ns/op
BenchmarkGraphQLParseSchema/Trivial_Schema_-_string-16                              6429            173773 ns/op
BenchmarkGraphQLParseSchema/Trivial_Schema_with_cache_-_string-16                   6523            170819 ns/op
BenchmarkGraphQLParseQuery/Trivial_Query_-_string-16                               16352             72777 ns/op
BenchmarkGraphQLParseQuery/Trivial_Query_with_cache_-_string-16                    16083             73548 ns/op
BenchmarkGraphQLIsValid/Trivial_Schema_-_string-16                                 14320             83589 ns/op
BenchmarkGraphQLIsValid/Trivial_Schema_with_cache_-_string-16                      71486             15463 ns/op
BenchmarkGraphQLParse/Trivial_Schema_-_string-16                                    3380            321490 ns/op
BenchmarkGraphQLParse/Trivial_Schema_with_cache_-_string-16                        13909             87633 ns/op
BenchmarkGraphQLParseAndVerify/Trivial_Schema_-_string-16                           3435            327646 ns/op
BenchmarkGraphQLParseAndVerify/Trivial_Schema_with_cache_-_string-16               13844             85213 ns/op
PASS
ok      github.com/open-policy-agent/opa/v1/topdown     112.465s

Resolves: #5377

Why the changes in this PR are needed?

GraphQL schema parsing can be an expensive operation, so caching can help speed things up.

What are the changes in this PR?

This PR leverages the InterQueryBuiltinValueCache to store parsed GraphQL schema data.

Notes to assist PR review:

The performance improvements are more dramatic on more complex schemas, but the complex schemas were omitted from the included test cases because they are quite large. Those numbers are captured in the issue thread.

Further comments:

The GraphQL builtin code has some repeated patterns which could probably be cleaned up, but I felt that was out of scope for adding a caching layer.

netlify · 2025-03-20T16:04:48Z

✅ Deploy Preview for openpolicyagent ready!

Name	Link
🔨 Latest commit	`f9a0b8f`
🔍 Latest deploy log	https://app.netlify.com/sites/openpolicyagent/deploys/67fe99956cf4cc0008f6e848
😎 Deploy Preview	https://deploy-preview-7457--openpolicyagent.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

johanfylling

Thank you for working on this! 😃

Just a couple of questions/comments.

v1/topdown/graphql.go

johanfylling · 2025-03-27T10:11:30Z

v1/topdown/graphql.go

+
+	if k, keyOk := cacheKeyWithPrefix(bctx, operands, "gql_schema_ast"); keyOk {
+		key = k
+		if val, ok := bctx.InterQueryBuiltinValueCache.Get(ast.StringTerm(key).Value); ok {


I'd recommend using a named value cache here. The benefits are:

Caching is isolated to the GraphQL built-in, and entries aren't "competing" with other built-ins

~~No need to prefix the key, for above reason~~ (looks like we need internal differentiation, though)

Named caches can be individually configured (complete with individual default settings)

See the io.jwt.* built-ins for inspiration.

@robmyersrobmyers, what's your thoughts about using a named value cache? Do you want us to help making those changes?

I'm working on it now, but seem to be having trouble initializing the cache. Right now I'm looking at 3 different named value caches for the different types, but that seems a bit inelegant, so maybe I should create a single named cache and use the cache key to differentiate the different types? Let me know your preference and I'll try and get it in close shape before I get your help to take it over the finish line. :)

Seems to be working; feedback welcome. Not sure people will love exposing 3 different named cache configurations.

Three separate configuration options might be overkill, like you say 👍. A single named cache with differentiating cache keys for the "sub-types" sounds like a good solution to me.

Sounds good - I'll resurrect the cache key stuff and consolidate to one named cache.

I think it's ready - please take a look and let me know if we missed anything.

Updated bench results with a few less trivial inputs:

BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_string-10 23740 51418 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_string-10 965624 1187 ns/op BenchmarkGraphQLSchemaIsValid/Schema_w/_1000_types_-_string-10 751 1578718 ns/op BenchmarkGraphQLSchemaIsValid/Schema_w/_1000_types_with_cache_-_string-10 33801 34915 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_AST_object-10 17491 67845 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_AST_object-10 124808 9801 ns/op BenchmarkGraphQLParseSchema/Trivial_Schema_-_string-10 9914 116169 ns/op BenchmarkGraphQLParseSchema/Trivial_Schema_with_cache_-_string-10 678207 1640 ns/op BenchmarkGraphQLParseQuery/Trivial_Query_-_string-10 26994 44328 ns/op BenchmarkGraphQLParseQuery/Trivial_Query_with_cache_-_string-10 26922 44386 ns/op BenchmarkGraphQLIsValid/Trivial_Schema_-_string-10 21441 55487 ns/op BenchmarkGraphQLIsValid/Trivial_Schema_with_cache_-_string-10 152800 7384 ns/op BenchmarkGraphQLIsValid/Schema_w/_1000_types_-_string-10 758 1568036 ns/op BenchmarkGraphQLIsValid/Schema_w/_1000_types_with_cache_-_string-10 28108 43051 ns/op BenchmarkGraphQLParse/Trivial_Schema_-_string-10 5119 216610 ns/op BenchmarkGraphQLParse/Trivial_Schema_with_cache_-_string-10 24186 49551 ns/op BenchmarkGraphQLParseAndVerify/Trivial_Schema_-_string-10 5241 217419 ns/op BenchmarkGraphQLParseAndVerify/Trivial_Schema_with_cache_-_string-10 23911 50004 ns/op

v1/topdown/graphql.go

v1/topdown/graphql_bench_test.go

…gent#5377) This commit stores parsed GraphQL schemas to the cache, which improves the performance of GraphQL operations that parse the schema more than once. Queries are not cached. BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_string-10 23740 51418 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_string-10 965624 1187 ns/op BenchmarkGraphQLSchemaIsValid/Schema_w/_1000_types_-_string-10 751 1578718 ns/op BenchmarkGraphQLSchemaIsValid/Schema_w/_1000_types_with_cache_-_string-10 33801 34915 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_AST_object-10 17491 67845 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_AST_object-10 124808 9801 ns/op BenchmarkGraphQLParseSchema/Trivial_Schema_-_string-10 9914 116169 ns/op BenchmarkGraphQLParseSchema/Trivial_Schema_with_cache_-_string-10 678207 1640 ns/op BenchmarkGraphQLParseQuery/Trivial_Query_-_string-10 26994 44328 ns/op BenchmarkGraphQLParseQuery/Trivial_Query_with_cache_-_string-10 26922 44386 ns/op BenchmarkGraphQLIsValid/Trivial_Schema_-_string-10 21441 55487 ns/op BenchmarkGraphQLIsValid/Trivial_Schema_with_cache_-_string-10 152800 7384 ns/op BenchmarkGraphQLIsValid/Schema_w/_1000_types_-_string-10 758 1568036 ns/op BenchmarkGraphQLIsValid/Schema_w/_1000_types_with_cache_-_string-10 28108 43051 ns/op BenchmarkGraphQLParse/Trivial_Schema_-_string-10 5119 216610 ns/op BenchmarkGraphQLParse/Trivial_Schema_with_cache_-_string-10 24186 49551 ns/op BenchmarkGraphQLParseAndVerify/Trivial_Schema_-_string-10 5241 217419 ns/op BenchmarkGraphQLParseAndVerify/Trivial_Schema_with_cache_-_string-10 23911 50004 ns/op Resolves: open-policy-agent#5377 Signed-off-by: Rob Myers <[email protected]>

robmyersrobmyers force-pushed the 5377_cache_gql_parse branch 2 times, most recently from 77cd7a2 to c8871c7 Compare March 20, 2025 16:20

johanfylling reviewed Mar 27, 2025

View reviewed changes

robmyersrobmyers force-pushed the 5377_cache_gql_parse branch from 49b0a83 to c8871c7 Compare April 1, 2025 23:33

robmyersrobmyers force-pushed the 5377_cache_gql_parse branch 4 times, most recently from 13dbd37 to 32ad5f0 Compare April 12, 2025 06:56

robmyersrobmyers mentioned this pull request Apr 14, 2025

Add JSON annotation in internal/gqlparser/ast to Position fields #7509

Open

robmyersrobmyers force-pushed the 5377_cache_gql_parse branch 2 times, most recently from fad8810 to d88bd17 Compare April 14, 2025 14:44

robmyersrobmyers force-pushed the 5377_cache_gql_parse branch from d88bd17 to f9a0b8f Compare April 15, 2025 17:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v1/topdown/graphql: Cache GraphQL schema parse results (#5377) #7457

v1/topdown/graphql: Cache GraphQL schema parse results (#5377) #7457

robmyersrobmyers commented Mar 20, 2025 •

edited

Loading

netlify bot commented Mar 20, 2025 •

edited

Loading

johanfylling left a comment

johanfylling Mar 27, 2025

johanfylling Apr 11, 2025

robmyersrobmyers Apr 11, 2025

robmyersrobmyers Apr 11, 2025

johanfylling Apr 15, 2025

robmyersrobmyers Apr 15, 2025

robmyersrobmyers Apr 15, 2025

v1/topdown/graphql: Cache GraphQL schema parse results (#5377) #7457

Are you sure you want to change the base?

v1/topdown/graphql: Cache GraphQL schema parse results (#5377) #7457

Conversation

robmyersrobmyers commented Mar 20, 2025 • edited Loading

Why the changes in this PR are needed?

What are the changes in this PR?

Notes to assist PR review:

Further comments:

netlify bot commented Mar 20, 2025 • edited Loading

✅ Deploy Preview for openpolicyagent ready!

johanfylling left a comment

Choose a reason for hiding this comment

johanfylling Mar 27, 2025

Choose a reason for hiding this comment

johanfylling Apr 11, 2025

Choose a reason for hiding this comment

robmyersrobmyers Apr 11, 2025

Choose a reason for hiding this comment

robmyersrobmyers Apr 11, 2025

Choose a reason for hiding this comment

johanfylling Apr 15, 2025

Choose a reason for hiding this comment

robmyersrobmyers Apr 15, 2025

Choose a reason for hiding this comment

robmyersrobmyers Apr 15, 2025

Choose a reason for hiding this comment

robmyersrobmyers commented Mar 20, 2025 •

edited

Loading

netlify bot commented Mar 20, 2025 •

edited

Loading