Best number of `total_tokens` on the default `CapacityLimiter`

[Kludex]

Hi there 👋

Yesterday, I asked on trio's Gitter channel if there was a reason for the number of total_tokens on the default CapacityLimiter to be 40. @njsmith reply was:

And I guess it's a good number, as it's been there for five years. 😅 But... As he mentioned, it would be interesting to find an optimal number there.

The part of the code I'm talking about:

trio/trio/_threads.py

Lines 30 to 32 in f2e2dd2

	# I pulled this number out of the air; it isn't based on anything. Probably we
	# should make some kind of measurements to pick a good value.
	DEFAULT_LIMIT = 40

I'm creating this issue mainly to see if we can come up with some ideas to have this optimal number there.

Ideas? 😅

Gitter conversation

This is just for reference.

Link to the Gitter conversation

References:

• Gunicorn recommends 2-4 * NUM_CORES threads.

[Sxderp]

It all depends on /what/ you're using it for I suppose. Might be a good idea to sample code and see how many instances use the default and how many that don't.

For example, I've always provided a number. I didn't even know a default was available, which I guess it makes sense as the builtin Process/Thread pools do. My use cases generally involve web scraping so I tend to keep my capacity fairly low < 10 as to not overwhelm servers.