Skip to content

Conversation

timvw
Copy link

@timvw timvw commented Aug 22, 2025

Summary

This PR introduces an HttpClient trait abstraction that allows users to provide custom HTTP client implementations, enabling middleware support for automatic instrumentation, logging, retry logic, and more.

Motivation

Currently, async-openai is tightly coupled to reqwest::Client, which prevents users from:

  • Adding middleware for tracing, logging, or retry logic
  • Using alternative HTTP clients
  • Mocking HTTP calls for testing
  • Implementing custom request/response handling

This is particularly important for production applications that need automatic OpenTelemetry instrumentation without manually wrapping every API call.

Changes

  1. New http_client module (src/http_client.rs):

    • Defines HttpClient trait for abstract HTTP operations
    • HttpError and HttpResponse types for trait methods
    • Default implementation for reqwest::Client
    • BoxedHttpClient type alias for convenience
  2. New ClientWithTrait struct (src/client.rs):

    • Alternative client that accepts any HttpClient implementation
    • Maintains backward compatibility - existing Client unchanged
    • Enables dependency injection of HTTP clients

Benefits

  • Middleware Support: Use reqwest-middleware for automatic OpenTelemetry/tracing
  • Testing: Easy mocking with custom HttpClient implementations
  • Flexibility: Support for any HTTP client library
  • Backward Compatible: Existing code continues to work unchanged

Example Usage

use async_openai::{ClientWithTrait, http_client::HttpClient};
use reqwest_middleware::{ClientBuilder, ClientWithMiddleware};
use reqwest_tracing::TracingMiddleware;

// Create HTTP client with middleware
let client = ClientBuilder::new(reqwest::Client::new())
    .with(TracingMiddleware::default())
    .build();

// Use with async-openai
let openai = ClientWithTrait::new_with_http_client(client, config);

Testing

The changes compile successfully and maintain full backward compatibility. The new trait has been tested with reqwest-middleware for OpenTelemetry instrumentation.

Next Steps

If this abstraction is accepted, future work could:

  • Migrate internal API implementations to use the trait
  • Add built-in middleware implementations
  • Provide testing utilities with mock clients

@timvw timvw marked this pull request as draft August 22, 2025 15:37
…eqwest::Client

- Replace hardcoded reqwest::Client with HttpClient trait throughout
- Client struct now accepts any HttpClient implementation
- Remove duplicate ClientWithTrait and ChatWithTrait structures
- All API methods (get, post, delete) now use the trait
- Backward compatibility: forms and streaming still use reqwest directly (TODO)
- Default implementation uses reqwest::Client for zero breaking changes
- This enables middleware injection for tracing, logging, etc.
timvw added 3 commits August 22, 2025 18:42
… support

- Extended HttpClient trait with request_multipart() for file uploads
- Extended HttpClient trait with request_stream() for SSE streaming
- Added MultipartForm struct and conversion helper from reqwest forms
- Added SseEvent struct for streaming responses
- Updated post_form_raw to use HttpClient trait instead of reqwest directly
- Updated streaming methods to use HttpClient trait
- Removed all outdated comments about multipart forms not being supported
- Added uuid dependency for multipart boundary generation
- Eliminated all direct reqwest usage except for eventsource compatibility

Note: One method (post_stream_mapped_raw_events) still uses reqwest directly
for eventsource_stream::Event compatibility, documented with TODO
- Removed unused EventSource import in http_client.rs
- Removed unused stream() function in client.rs (replaced by stream_from_sse)
- Fixed formatting in responses-stream example
- Reordered http_client module export in lib.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant