LiteLLM streaming not truly asynchronous - returns batch processed chunks

[DennySORA]

Describe the bug
The streaming implementation in lite_llm.py contains an async/await usage error that causes a "fake streaming" phenomenon. The current implementation waits for all content to be fully returned before chunking it into Events, rather than providing true real-time streaming.

To Reproduce
Steps to reproduce the behavior:

1. Install google-adk==1.2.1
2. Use LiteLLM model with streaming calls
3. Observe the response behavior
4. Notice that streaming is not real-time but rather batch processing disguised as streaming

Expected behavior
Should implement true real-time streaming where each chunk is returned immediately as it's generated, rather than waiting for all content to complete before batch processing.

Root Cause Analysis
The issue occurs in lite_llm.py#L682-L683:

Current incorrect implementation:

for part in self.llm_client.completion(**completion_args):
    for chunk, finish_reason in model_response_to_chunk(part):

Correct implementation should be:

async for part in await self.llm_client.acompletion(**completion_args):
    for chunk, finish_reason in model_response_to_chunk(part):

Technical Explanation:

1. Synchronous Issue: The current completion() method is synchronous and blocks the main thread in streaming mode until all data is received before starting iteration
2. Fake Streaming Phenomenon: Due to synchronous blocking, all chunks get buffered and processed at once, creating an illusion of streaming when it's actually batch processing
3. Proper Async Implementation: Using acompletion() with async for enables true non-blocking streaming

Python Async Principles:

• async for allows execution to pause at each iteration, giving other coroutines a chance to execute
• await ensures proper waiting for async functions without blocking the event loop
• True streaming requires non-blocking iterators, which can only be achieved through async for
• Synchronous for loops wait for the entire iterator to complete before continuing, losing the real-time nature of streaming

Desktop (please complete the following information):

• OS: macOS
• Python version: 3.13
• ADK version: 1.2.1

Additional context
Reference to LiteLLM's official documentation for correct async streaming implementation:
https://docs.litellm.ai/docs/completion/stream#async-streaming

Video demonstration: https://youtu.be/nF1G4leSbSI

This issue affects all applications that rely on true streaming responses, particularly real-time interactive chat applications.

Proposed Fix
The fix requires changing the synchronous iteration to asynchronous iteration and ensuring the method calling this code is properly marked as async and awaited appropriately throughout the call stack.

[simonwei97]

same for version 1.3.0

[avidelatm]

@DennySORA are you submitting a PR to fix this ? or are you hoping for someone else to do it ? Just asking cause I'm also wanting for my personal project Litellm to do correct streaming instead of the current batch behaviour, and I'm happy to try to submit a PR with your changes if you are too busy/can't do it for any reason

[growmuye]

I recently discovered a phenomenon that seems to be related to the feedback you provided.

I also have the same problem, I am stress-testing the agent, and I have found a strange phenomenon: [10 concurrents]

Not Srteam

Stream

I tested 1 concurrent, 2 concurrent, and 10 concurrent. Under 1 concurrent, it took 7 seconds; under 2 concurrent, it took 14 seconds; under 10 concurrent, it was 10 times.

[DennySORA]

@avidelatm Thank you for your pull request. But I originally thought this needed to be checked and fixed by the ADK Office. I don’t know how big of an impact this change has. But it looks like the office doesn’t care about the issue.