Open
Description
What feature would you like to see?
I am using DSPY version 2.6.5.
I set up LM with the "other providers" approach, as shown in the getting started material.
One issue that I encounter is that there is a limit of the number of requests that can make per minute, like 1000 requests per minute. When I run optimizers or evalaution with multiple threads or even 1 thread, it easily goes over that limit and the whole program run stops due to the connection error.
Is there any way to resolve this issue?
For example, by sending a batch completion call or handling the number of requests explicitly?
If there's a way to resolve this issue already, it would be great to know.
Would you like to contribute?
- Yes, I'd like to help implement this.
- No, I just want to request it.
Additional Context
No response