-
Notifications
You must be signed in to change notification settings - Fork 14
Open
Description
Improve timeout and retry handling with better error management and configuration options.
Acceptance Criteria:
- Add APITimeoutError exception handling
- Improve retry logic with exponential backoff
- Add configurable retry attempts
- Add timeout configuration per request
- Add comprehensive tests for timeout and retry scenarios
- Update documentation with timeout and retry examples
Technical Requirements:
- Implement proper timeout error handling
- Support configurable retry strategies
- Add exponential backoff for retries
- Maintain performance with retries
Example Usage:
llm = ChatGradient(
model="llama3.3-70b-instruct",
timeout=30.0,
max_retries=3
)
try:
response = llm.invoke("Hello world!")
except APITimeoutError:
print("Request timed out")
References: