Mutations became much slower after upgrade to 2.4.15

[AlenaSviridenko]

👋 Hi there from GitHub!

Looking for some help or ideas on a strange behaviour we observed after upgrading 2.3.22 to 2.4.15:
Following the upgrade, we noticed that mutations became much slower, P50 grew from ~290ms to ~470ms:

However there is no same trend for queries, and we are puzzled by this difference. We didn't adapt any new features due to a bigger scope of work, and for the time being we just enforced old behaviour by adding the following to our Schema class:

use GraphQL::Schema::Warden
GraphQL::Schema.did_you_mean(nil)

For now we will rollback the gem to resolve the performance issues, but eventually we'd like to upgrade to a newest version. Perhaps you have any ideas on why it could get slower for mutations, and if that's something that can be fixed on a gem side? Thank you so much for looking into it! 🙏

[rmosolgo]

Hey, sorry for the trouble and thanks for reporting this. I'm not sure what would have slowed down just Mutations (or what would have slowed down anything...), but I definitely want to get to the bottom of this.

The first thing that comes to mind would be a "before" and "after" trace of a mutation which got slower. For example, I use stackprof for this, which would be hooked up like this:

Stackprof..run(mode: :cpu, out: 'tmp/graphql-mutation-before.dump') do # <- or "graphql-mutation-after"
  # run mutation here 
end 

That produces a .dump file which shows where the program spent its time. If you could share those .dump files (here or by email), that would almost certainly point to lines of code which were slower on the new version than on the old version. Alternatively, you could use stackprof to show the top 50 or so entries, and sanitize those as needed, and share the result (from before and after).

If you have another profiling setup ready to go, I'm sure the output of that would do the job, too. Let me know what you find and we'll get fixing!

[AlenaSviridenko]

Thank you for suggesting that idea! We will run the profiler and will get back as soon as possible

[rmosolgo]

Just a little ping here. I'd love to investigate this if you get a chance to run some profiling!

[AlenaSviridenko]

Hi! Sorry, I've been a bit too busy past couple weeks, but today I have planned time to run profiling, so hoping to send you some results by the end of this week!

[AlenaSviridenko]

I just shared profiling results via email!