[Bug]: tool_calls.id is Missing in Streaming Responses (stream=true) but Present in Non-Streaming Responses

[Kalle12345]

🐛 Describe the bug

When making an API call to the chat completions endpoint with tools and stream=true, the tool_calls objects within the streamed chunks (delta.tool_calls) do not include the id field for each tool call. However, when stream=false, the tool_calls.id field is correctly present in the response.

This inconsistency makes it difficult to uniquely identify and track tool calls when processing streamed responses, which is crucial for many applications that rely on tool usage. The OpenAI API, which vLLM aims to be compatible with, includes the tool_call_id (or equivalent id) in streaming chunks.

Steps to Reproduce:

Make a request with stream=false:

Request Body:

{
    "model": "Qwen/Qwen3-14B-AWQ",
    "messages": [
        {
            "role": "user",
            "content": "What is the weather like in Boston today?"
        }
    ],
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "get_current_weather",
                "description": "Get the current weather in a given location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA"
                        },
                        "unit": {
                            "type": "string",
                            "enum": [
                                "celsius",
                                "fahrenheit"
                            ]
                        }
                    },
                    "required": [
                        "location"
                    ]
                }
            }
        }
    ],
    "tool_choice": {
        "type": "function",
        "function": {
            "name": "get_current_weather"
        }
    },
    "stream": false
}

Response :

{
    "id": "chatcmpl-26cfd57cdd4c4574b6cfe2d488d6caf0",
    "object": "chat.completion",
    "created": 1747748271,
    "model": "default",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "tool_calls": [
                    {
                        "id": "chatcmpl-tool-c562052db72a4e3c94b844712b4f38e1", // <--- ID IS PRESENT
                        "type": "function",
                        "function": {
                            "name": "get_current_weather",
                            "arguments": "{\"location\": \"Boston\", \"unit\": \"fahrenheit\"}"
                        }
                    }
                ]
            },
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "prompt_tokens": 196,
        "total_tokens": 210,
        "completion_tokens": 14
    }
}

Make a request with stream=true:

Request Body:

{
    "model": "Qwen/Qwen3-14B-AWQ",
    "messages": [
        {
            "role": "user",
            "content": "What is the weather like in Boston today?"
        }
    ],
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "get_current_weather",
                "description": "Get the current weather in a given location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA"
                        },
                        "unit": {
                            "type": "string",
                            "enum": [
                                "celsius",
                                "fahrenheit"
                            ]
                        }
                    },
                    "required": [
                        "location"
                    ]
                }
            }
        }
    ],
    "tool_choice": {
        "type": "function",
        "function": {
            "name": "get_current_weather"
        }
    },
    "stream": true
}

Response Chunks:

data: {"id":"chatcmpl-f2a93065f04041d08a6af9fb23cf22d6","object":"chat.completion.chunk","created":1747748383,"model":"default","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-f2a93065f04041d08a6af9fb23cf22d6","object":"chat.completion.chunk","created":1747748383,"model":"default","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"name":"get_current_weather","arguments":"{\""}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-f2a93065f04041d08a6af9fb23cf22d6","object":"chat.completion.chunk","created":1747748383,"model":"default","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"name":"get_current_weather","arguments":"location"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-f2a93065f04041d08a6af9fb23cf22d6","object":"chat.completion.chunk","created":1747748383,"model":"default","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"name":"get_current_weather","arguments":"\":"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-f2a93065f04041d08a6af9fb23cf22d6","object":"chat.completion.chunk","created":1747748383,"model":"default","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"name":"get_current_weather","arguments":" \""}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-f2a93065f04041d08a6af9fb23cf22d6","object":"chat.completion.chunk","created":1747748383,"model":"default","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"name":"get_current_weather","arguments":"Boston"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-f2a93065f04041d08a6af9fb23cf22d6","object":"chat.completion.chunk","created":1747748383,"model":"default","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"name":"get_current_weather","arguments":"\","}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-f2a93065f04041d08a6af9fb23cf22d6","object":"chat.completion.chunk","created":1747748383,"model":"default","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"name":"get_current_weather","arguments":" \""}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-f2a93065f04041d08a6af9fb23cf22d6","object":"chat.completion.chunk","created":1747748383,"model":"default","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"name":"get_current_weather","arguments":"unit"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-f2a93065f04041d08a6af9fb23cf22d6","object":"chat.completion.chunk","created":1747748383,"model":"default","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"name":"get_current_weather","arguments":"\":"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-f2a93065f04041d08a6af9fb23cf22d6","object":"chat.completion.chunk","created":1747748383,"model":"default","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"name":"get_current_weather","arguments":" \""}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-f2a93065f04041d08a6af9fb23cf22d6","object":"chat.completion.chunk","created":1747748383,"model":"default","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"name":"get_current_weather","arguments":"f"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-f2a93065f04041d08a6af9fb23cf22d6","object":"chat.completion.chunk","created":1747748383,"model":"default","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"name":"get_current_weather","arguments":"ahrenheit"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-f2a93065f04041d08a6af9fb23cf22d6","object":"chat.completion.chunk","created":1747748383,"model":"default","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"name":"get_current_weather","arguments":"\"}"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-f2a93065f04041d08a6af9fb23cf22d6","object":"chat.completion.chunk","created":1747748383,"model":"default","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"name":"get_current_weather","arguments":""}}]},"logprobs":null,"finish_reason":"stop","stop_reason":null}]}

data: [DONE]

Notice that in the chunks containing delta.tool_calls, the id field for the tool call is missing.

Expected Behavior:
When stream=true, streamed chunks containing delta.tool_calls should include an id field for each tool call. This id must be present in the first chunk that introduces that specific tool call (identified by its index), and may also be present in all subsequent chunks pertaining to that same tool call. This behavior is necessary for consistency with the non-streaming response and for OpenAI API compatibility.

For example, a chunk might look like:

data: {"id":"chatcmpl-xxxxxxxx","object":"chat.completion.chunk","created":1747748383,"model":"default","choices":[{"index":0,"delta":{"tool_calls":[{"index":0, "id": "call_xxxxxxxxxxxx", "type": "function", "function":{"name":"get_current_weather","arguments":"{""}}]},"logprobs":null,"finish_reason":null}]}

The id for a specific tool call must be provided in its introductory chunk. If it is also included in subsequent chunks for that same tool call, its value must remain consistent.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[calvin0327]

/assign

[princepride]

ChatCompletionChunk(id='chatcmpl-BZgkS9g5fmfDWr6abBzaXKbabTYAL', choices=[Choice(delta=ChoiceDelta(content=None, function_call=None, refusal=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=0, id='call_Dcuer5y2RowXl7Ig47gCS9Ft', function=ChoiceDeltaToolCallFunction(arguments='', name='get_current_weather'), type='function')]), finish_reason=None, index=0, logprobs=None)], created=1747845504, model='gpt-4o-2024-08-06', object='chat.completion.chunk', service_tier='default', system_fingerprint='fp_76544d79cb', usage=None)
ChatCompletionChunk(id='chatcmpl-BZgkS9g5fmfDWr6abBzaXKbabTYAL', choices=[Choice(delta=ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='{"', name=None), type=None)]), finish_reason=None, index=0, logprobs=None)], created=1747845504, model='gpt-4o-2024-08-06', object='chat.completion.chunk', service_tier='default', system_fingerprint='fp_76544d79cb', usage=None)
ChatCompletionChunk(id='chatcmpl-BZgkS9g5fmfDWr6abBzaXKbabTYAL', choices=[Choice(delta=ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='location', name=None), type=None)]), finish_reason=None, index=0, logprobs=None)], created=1747845504, model='gpt-4o-2024-08-06', object='chat.completion.chunk', service_tier='default', system_fingerprint='fp_76544d79cb', usage=None)
ChatCompletionChunk(id='chatcmpl-BZgkS9g5fmfDWr6abBzaXKbabTYAL', choices=[Choice(delta=ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='":"', name=None), type=None)]), finish_reason=None, index=0, logprobs=None)], created=1747845504, model='gpt-4o-2024-08-06', object='chat.completion.chunk', service_tier='default', system_fingerprint='fp_76544d79cb', usage=None)
ChatCompletionChunk(id='chatcmpl-BZgkS9g5fmfDWr6abBzaXKbabTYAL', choices=[Choice(delta=ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='Paris', name=None), type=None)]), finish_reason=None, index=0, logprobs=None)], created=1747845504, model='gpt-4o-2024-08-06', object='chat.completion.chunk', service_tier='default', system_fingerprint='fp_76544d79cb', usage=None)
ChatCompletionChunk(id='chatcmpl-BZgkS9g5fmfDWr6abBzaXKbabTYAL', choices=[Choice(delta=ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='","', name=None), type=None)]), finish_reason=None, index=0, logprobs=None)], created=1747845504, model='gpt-4o-2024-08-06', object='chat.completion.chunk', service_tier='default', system_fingerprint='fp_76544d79cb', usage=None)
ChatCompletionChunk(id='chatcmpl-BZgkS9g5fmfDWr6abBzaXKbabTYAL', choices=[Choice(delta=ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='unit', name=None), type=None)]), finish_reason=None, index=0, logprobs=None)], created=1747845504, model='gpt-4o-2024-08-06', object='chat.completion.chunk', service_tier='default', system_fingerprint='fp_76544d79cb', usage=None)
ChatCompletionChunk(id='chatcmpl-BZgkS9g5fmfDWr6abBzaXKbabTYAL', choices=[Choice(delta=ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='":"', name=None), type=None)]), finish_reason=None, index=0, logprobs=None)], created=1747845504, model='gpt-4o-2024-08-06', object='chat.completion.chunk', service_tier='default', system_fingerprint='fp_76544d79cb', usage=None)
ChatCompletionChunk(id='chatcmpl-BZgkS9g5fmfDWr6abBzaXKbabTYAL', choices=[Choice(delta=ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='c', name=None), type=None)]), finish_reason=None, index=0, logprobs=None)], created=1747845504, model='gpt-4o-2024-08-06', object='chat.completion.chunk', service_tier='default', system_fingerprint='fp_76544d79cb', usage=None)
ChatCompletionChunk(id='chatcmpl-BZgkS9g5fmfDWr6abBzaXKbabTYAL', choices=[Choice(delta=ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='elsius', name=None), type=None)]), finish_reason=None, index=0, logprobs=None)], created=1747845504, model='gpt-4o-2024-08-06', object='chat.completion.chunk', service_tier='default', system_fingerprint='fp_76544d79cb', usage=None)
ChatCompletionChunk(id='chatcmpl-BZgkS9g5fmfDWr6abBzaXKbabTYAL', choices=[Choice(delta=ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='"}', name=None), type=None)]), finish_reason=None, index=0, logprobs=None)], created=1747845504, model='gpt-4o-2024-08-06', object='chat.completion.chunk', service_tier='default', system_fingerprint='fp_76544d79cb', usage=None)
ChatCompletionChunk(id='chatcmpl-BZgkS9g5fmfDWr6abBzaXKbabTYAL', choices=[Choice(delta=ChoiceDelta(content=None, function_call=None, refusal=None, role=None, tool_calls=None), finish_reason='tool_calls', index=0, logprobs=None)], created=1747845504, model='gpt-4o-2024-08-06', object='chat.completion.chunk', service_tier='default', system_fingerprint='fp_76544d79cb', usage=None)

I noticed the stream output format of gpt is like this, whether we should follow the format?

[princepride]

@aarnphm do we need adjust streaming output format as openai

[calvin0327]

this is my test results by examples/online_serving/openai_chat_completion_client_with_tools.py

ChoiceDeltaToolCall(index=0, id='chatcmpl-tool-3ba87fd45f4c4c4a997766224e24b0b0', function=ChoiceDeltaToolCallFunction(arguments=None, name='get_current_weather'), type='function')
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='{"city": "', name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='Dallas', name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='",', name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=' "', name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='state', name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='":', name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=' "', name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='TX', name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='",', name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=' "', name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='unit', name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='":', name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments=' "', name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='f', name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='ahrenheit', name=None), type=None)
ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='"}', name=None), type=None)

Except for the fisrt returned data which has a tool call ID, the remaining ones indeed do not, I wonder if this complies with the specification?

[princepride]

I think this format is completely up to standard.

[chaunceyjiang]

This is compliant with the specification.
Please refer to the OpenAI API documentation.

https://platform.openai.com/docs/guides/function-calling?api-mode=responses#streaming