OpenRouter compatible with SemanticKernel

[sebastianrosch]

Hi, we're using SemanticKernel in C# with plugins and function calling and would like to switch to OpenRouter. I am getting an exception from the OpenAI SDK which SemanticKernel uses internally when parsing the response from OpenRouter.

System.NullReferenceException: Object reference not set to an instance of an object.
   at OpenAI.Chat.ChatCompletion.get_Refusal()
   at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.GetChatCompletionMetadata(ChatCompletion completions)
   at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.CreateChatMessageContent(ChatCompletion completion, String targetModel)

I assume it comes from this line: https://github.com/openai/openai-dotnet/blob/c046c34ad06f7ca050b2f8044cfc8372d8cf0e76/src/Custom/Chat/ChatCompletion.cs#L82

Is there any special configuration required to make this work with SemanticKernel?

[tntwist]

Hello there,

the issue is that OpenRouter returns a 200er response when there are errors on provider side. Something like this:

{
    "error": {
        "message": "Provider returned error",
        "code": 400,
        "metadata": {
            "raw": "{\n  \"error\": {\n    \"code\": 400,\n    \"message\": \"Invalid JSON payload received. Unknown name \\\"additionalProperties\\\" at 'generation_config.response_schema.properties[0].value.items': Cannot find field.\",\n    \"status\": \"INVALID_ARGUMENT\",\n    \"details\": [\n      {\n        \"@type\": \"type.googleapis.com/google.rpc.BadRequest\",\n        \"fieldViolations\": [\n          {\n            \"field\": \"generation_config.response_schema.properties[0].value.items\",\n            \"description\": \"Invalid JSON payload received. Unknown name \\\"additionalProperties\\\" at 'generation_config.response_schema.properties[0].value.items': Cannot find field.\"\n          }\n        ]\n      }\n    ]\n  }\n}\n",
            "provider_name": "Google AI Studio"
        }
    },
    "user_id": "*******"
}

As the response code of OpenRouter is 200 it is treated as a successfull response and the OpenAi Connector throws a null reference exception when trying to read the models response.

There are two options to resolve this issue:

1. Semantic Kernel builds a dedicated connector for OpenRouter and fails when an error comes back like this from OpenRouter.
2. OpenRouter changes their api to return the status code that where given by the provider (like 400 in the example). With this the SemanticKernel connector for OpenAi would just fail and throw an exception that can be handled by consumers.

[yogasanas]

@tntwist We now return a lot of errors with the correct error code from the provider especially for errors such as the 400 error shown above.

When did you reproduce this error? Would love to work with your team to be able to normalize midstream errors for your clients.

[maxwell-01]

Please see the ticket on the semantic kernal repo here, they are recommending that a 200 response is not returned where there is an error: microsoft/semantic-kernel#10530

I am also experiencing this issue. We would like to switch to using openrouter but as semantic kernal does not handle the errors its creating a bit of a blocker

[ThimoDEV]

Hello, I just came from the semantic kernel issue also. Is this something the OpenRouter team is willing to change or have a solution for? I reallly would want to use OpenRouter with semantic kernel. So maybe a community maintained Connector (by OpenRouter) could be a solution?

[kacisse]

If that can help anyone, it works like that:

            `var httpClient = CreateHttpClient();
            httpClient.DefaultRequestHeaders.Clear();
            httpClient.DefaultRequestHeaders.Add("Authorization", $"Bearer {openRouterConfig.ApiKey}");
            httpClient.DefaultRequestHeaders.Add("HTTP-Referer", "referer.com");
            httpClient.DefaultRequestHeaders.Add("X-Title", "MyApp");

            // Add OpenAI chat completion service with OpenRouter endpoint
            builder.AddOpenAIChatCompletion(
                modelId: model.ModelId,
                apiKey: openRouterConfig.ApiKey,
                httpClient: httpClient,
                serviceId: $"openrouter_{model.ModelId.Replace("/", "_")}",
                endpoint: new Uri(openRouterConfig.BaseUrl));`

            var kernel = builder.Build();

My bad if I misunderstood the issue.