Skip to content

Added support for http_client,extra_headers and extra_query in openai #23

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

daichi-m
Copy link

This PR contains changes to support custom HTTP client, default headers and default queries in using OpenAITextGenerator.

Why is this change required

We want to pass custom headers to our internal proxy layer which is built on top of AzureOpenAI for our internal audit and metrics. Currently this is not supported on llmx and subsequently, not available via lida. This change attempts to add support for a custom http_client along with additional default_headers and default_query parameters.

How is this tested

We have done an internal testing with our proxy layer. Sample code:

headers = {"X-Custom-Header": "Custom-Val"}
client = httpx.Client(headers=headers)

llm_inst = llm(provider="openai",  api_type="azure", azure_endpoint="https://openaiproxy.prod.walmart.com",
               api_key=api_key, api_version="2024-02-01", model="gpt-35-turbo", http_client=client)
config = TextGenerationConfig(n=1, temperature=0.2, max_tokens=100)

msgs = [
    {"role": "system", "content": "You are a helpful assistant that can explain concepts clearly to a 6 year old child."},
    {"role": "user", "content": "What is  gravity?"}
]
response = llm_inst.generate(messages=msgs, config=config)

@daichi-m
Copy link
Author

@victordibia Can this be reviewed and merged?

@daichi-m
Copy link
Author

daichi-m commented May 7, 2024

@victordibia Can we move forward with this?

@prati04
Copy link

prati04 commented May 30, 2024

@victordibia Can we merge these changes?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants