Tools Calling not working with fake streaming

[codedpro]

Hi,

I'm encountering an issue when trying to connect tool calls to my standalone custom provider. As you know, I'm using LLaMA 3.1, which doesn't support streaming. To work around this, I'm using the fakeStream option for messages.
I've recently added tool support to my custom provider, and it's now correctly sending the array of tools as defined in the AI SDK (as shown below). However, I'm still facing issues with the integration.

type CoreAssistantMessage = {
    role: 'assistant';
    content: AssistantContent;
    /**
  Additional provider-specific metadata. They are passed through
  to the provider from the AI SDK and enable provider-specific
  functionality that can be fully encapsulated in the provider.
   */
    providerOptions?: ProviderOptions;
    /**
  @deprecated Use `providerOptions` instead.
  */
    experimental_providerMetadata?: ProviderMetadata;
};

interface ToolCallPart {
    type: 'tool-call';
    /**
  ID of the tool call. This ID is used to match the tool call with the tool result.
   */
    toolCallId: string;
    /**
  Name of the tool that is being called.
   */
    toolName: string;
    /**
  Arguments of the tool call. This is a JSON-serializable object that matches the tool's input schema.
     */
    args: unknown;
    /**
  Additional provider-specific metadata. They are passed through
  to the provider from the AI SDK and enable provider-specific
  functionality that can be fully encapsulated in the provider.
   */
    providerOptions?: ProviderOptions;
    /**
  @deprecated Use `providerOptions` instead.
   */
    experimental_providerMetadata?: ProviderMetadata;
}

type AssistantContent = string | Array<TextPart | FilePart | ReasoningPart | RedactedReasoningPart | ToolCallPart>;

currently with normal strings of the AssistantContent streaming works, and also it is fine.
but as you can see in ai sdk you defined that we can use an array of that Parts too, as i tried to do,
but i will get zod errors in line of :
result.mergeIntoDataStream(dataStream, { sendReasoning: true }); //
which is trying to display my content to the user and will get errors as it expects the string, im a little confused that why it needs an string but we have types that clearly display string or an array of parts so can you help me to how to implant my tools calling well in here, and i mention again that llama models are not capable of streaming so i have to use fakestream in my case of usage, and following are some logs, attachments and my modified codes

server console logs of the api :

Received POST request.
Parsed request JSON: {
  id: '4e1a03de-d31b-4d9d-a887-1b445b098478',
  selectedChatModel: 'chat-model',
  messagesCount: 1
}
Session retrieved: {
  user: { email: '[email protected]', id: 'c8f94275-cf12-4572-af12-a5cb8049810e' },
  expires: '2025-04-29T14:50:32.898Z'
}
Extracted most recent user message: {
  role: 'user',
  content: 'What is the weather in San Francisco?',
  id: '91e03cce-a01d-4049-a237-a67bb24d22b1',
  createdAt: '2025-03-30T14:50:31.752Z',
  parts: [ { type: 'text', text: 'What is the weather in San Francisco?' } ]
}
Chat lookup result: undefined
No existing chat found. Generating title for new chat...
Generated chat title: San Francisco Weather Inquiry
New chat saved with ID: 4e1a03de-d31b-4d9d-a887-1b445b098478
Saving user message...
User message saved successfully.
Preparing data stream response...
Executing data stream response...
Starting stream consumption...
Merging result into data stream with reasoning enabled...
 GET /api/history 200 in 124ms
An error occurred during the data stream execution. [Error [AI_APICallError]: Invalid JSON response] {
  url: 'http://10.223.188.42:3001/process_message/chat',
  requestBodyValues: [Object],
  statusCode: 200,
  responseHeaders: [Object],
  responseBody: '{"created_at":"2025-03-31T01:30:41.553911","done":true,"eval_count":1,"eval_duration":4.3209333419799805,"message":{"role":"assistant","content":[{"type":"tool-call","toolCallId":"121499a8-af13-4344-9ec1-2910cfaf2a1c","toolName":"getWeather","args":{"latitude":37.7749,"longitude":-122.4194},"providerOptions":null}],"providerOptions":null},"model":"llama3.1","total_duration":4.3209333419799805}',
  isRetryable: false,
  data: undefined,
  [cause]: [Error [AI_TypeValidationError]: Type validation failed: Value: {"created_at":"2025-03-31T01:30:41.553911","done":true,"eval_count":1,"eval_duration":4.3209333419799805,"message":{"role":"assistant","content":[{"type":"tool-call","toolCallId":"121499a8-af13-4344-9ec1-2910cfaf2a1c","toolName":"getWeather","args":{"latitude":37.7749,"longitude":-122.4194},"providerOptions":null}],"providerOptions":null},"model":"llama3.1","total_duration":4.3209333419799805}.
  Error message: [
    {
      "code": "invalid_type",
      "expected": "string",
      "received": "array",
      "path": [
        "message",
        "content"
      ],
      "message": "Expected string, received array"
    }
  ]] {
    value: {
      created_at: '2025-03-31T01:30:41.553911',
      done: true,
      eval_count: 1,
      eval_duration: 4.3209333419799805,
      message: [Object],
      model: 'llama3.1',
      total_duration: 4.3209333419799805
  ]] {
    value: {
      created_at: '2025-03-31T01:30:41.553911',
      done: true,
      eval_count: 1,
      eval_duration: 4.3209333419799805,
      message: [Object],
      model: 'llama3.1',
      total_duration: 4.3209333419799805
    value: {
      created_at: '2025-03-31T01:30:41.553911',
      done: true,
      eval_count: 1,
      eval_duration: 4.3209333419799805,
      message: [Object],
      model: 'llama3.1',
      total_duration: 4.3209333419799805
      created_at: '2025-03-31T01:30:41.553911',
      done: true,
      eval_count: 1,
      eval_duration: 4.3209333419799805,
      message: [Object],
      model: 'llama3.1',
      total_duration: 4.3209333419799805
      eval_count: 1,
      eval_duration: 4.3209333419799805,
      message: [Object],
      model: 'llama3.1',
      total_duration: 4.3209333419799805
    },
    [cause]: [Error [ZodError]: [
      {
      eval_duration: 4.3209333419799805,
      message: [Object],
      model: 'llama3.1',
      total_duration: 4.3209333419799805
    },
    [cause]: [Error [ZodError]: [
      {
        "code": "invalid_type",
        "expected": "string",
      total_duration: 4.3209333419799805
    },
    [cause]: [Error [ZodError]: [
      {
        "code": "invalid_type",
        "expected": "string",
        "received": "array",
        "path": [
    },
    [cause]: [Error [ZodError]: [
      {
        "code": "invalid_type",
        "expected": "string",
        "received": "array",
        "path": [
          "message",
        "code": "invalid_type",
        "expected": "string",
        "received": "array",
        "path": [
          "message",
        "received": "array",
        "path": [
          "message",
          "message",
          "content"
        ],
        "message": "Expected string, received array"
      }
    ]] {
      issues: [ [Object] ],
      addIssue: [Function (anonymous)],
      addIssues: [Function (anonymous)]
    }
  }
}

Page error msg:

modified (chat)/api/chat/route.ts:

import {
  UIMessage,
  appendResponseMessages,
  createDataStreamResponse,
  smoothStream,
  streamText,
} from 'ai';
import { auth } from '@/app/(auth)/auth';
import { systemPrompt } from '@/lib/ai/prompts';
import {
  deleteChatById,
  getChatById,
  saveChat,
  saveMessages,
} from '@/lib/db/queries';
import {
  generateUUID,
  getMostRecentUserMessage,
  getTrailingMessageId,
} from '@/lib/utils';
import { generateTitleFromUserMessage } from '../../actions';
import { createDocument } from '@/lib/ai/tools/create-document';
import { updateDocument } from '@/lib/ai/tools/update-document';
import { requestSuggestions } from '@/lib/ai/tools/request-suggestions';
import { getWeather } from '@/lib/ai/tools/get-weather';
import { isProductionEnvironment } from '@/lib/constants';
import { myProvider } from '@/lib/ai/providers';

export const maxDuration = 60;

export async function POST(request: Request) {
  try {
    console.log("Received POST request.");
    const {
      id,
      messages,
      selectedChatModel,
    }: {
      id: string;
      messages: Array<UIMessage>;
      selectedChatModel: string;
    } = await request.json();
    console.log("Parsed request JSON:", { id, selectedChatModel, messagesCount: messages.length });

    const session = await auth();
    console.log("Session retrieved:", session);

    if (!session || !session.user || !session.user.id) {
      console.error("Unauthorized: Session or user details missing.");
      return new Response('Unauthorized', { status: 401 });
    }

    const userMessage = getMostRecentUserMessage(messages);
    console.log("Extracted most recent user message:", userMessage);

    if (!userMessage) {
      console.error("No user message found in the request.");
      return new Response('No user message found', { status: 400 });
    }

    const chat = await getChatById({ id });
    console.log("Chat lookup result:", chat);

    if (!chat) {
      console.log("No existing chat found. Generating title for new chat...");
      const title = await generateTitleFromUserMessage({
        message: userMessage,
      });
      console.log("Generated chat title:", title);
      await saveChat({ id, userId: session.user.id, title });
      console.log("New chat saved with ID:", id);
    } else {
      if (chat.userId !== session.user.id) {
        console.error("Unauthorized: Chat does not belong to current user.");
        return new Response('Unauthorized', { status: 401 });
      }
    }

    console.log("Saving user message...");
    await saveMessages({
      messages: [
        {
          chatId: id,
          id: userMessage.id,
          role: 'user',
          parts: userMessage.parts,
          attachments: userMessage.experimental_attachments ?? [],
          createdAt: new Date(),
        },
      ],
    });
    console.log("User message saved successfully.");

    console.log("Preparing data stream response...");
    return createDataStreamResponse({
      execute: (dataStream) => {
        console.log("Executing data stream response...");
        const result = streamText({
          model: myProvider.languageModel(selectedChatModel),
          system: systemPrompt({ selectedChatModel }),
          messages,
          maxSteps: 5,
          experimental_activeTools:
            selectedChatModel === 'chat-model-reasoning'
              ? []
              : [
                  'getWeather',
                  'createDocument',
                  'updateDocument',
                  'requestSuggestions',
                ],
          experimental_transform: smoothStream({ chunking: 'word' }),
          experimental_generateMessageId: generateUUID,
          tools: {
            getWeather,
            createDocument: createDocument({ session, dataStream }),
            updateDocument: updateDocument({ session, dataStream }),
            requestSuggestions: requestSuggestions({
              session,
              dataStream,
            }),
          },
          onFinish: async ({ response }) => {
            console.log("Stream finished. Processing response...");
            if (session.user?.id) {
              try {
                const assistantId = getTrailingMessageId({
                  messages: response.messages.filter(
                    (message) => message.role === 'assistant'
                  ),
                });
                console.log("Retrieved assistant ID:", assistantId);

                if (!assistantId) {
                  throw new Error('No assistant message found!');
                }

                const [, assistantMessage] = appendResponseMessages({
                  messages: [userMessage],
                  responseMessages: response.messages,
                });
                console.log("Assistant message to be saved:", assistantMessage);

                await saveMessages({
                  messages: [
                    {
                      id: assistantId,
                      chatId: id,
                      role: assistantMessage.role,
                      parts: assistantMessage.parts,
                      attachments:
                        assistantMessage.experimental_attachments ?? [],
                      createdAt: new Date(),
                    },
                  ],
                });
                console.log("Assistant message saved successfully.");
              } catch (error) {
                console.error("Failed to save assistant message:", error);
              }
            }
          },
          experimental_telemetry: {
            isEnabled: isProductionEnvironment,
            functionId: 'stream-text',
          },
        });

        console.log("Starting stream consumption...");
        result.consumeStream();

        console.log("Merging result into data stream with reasoning enabled...");
        result.mergeIntoDataStream(dataStream, { sendReasoning: true });      // 
      },
      onError: (error) => {
        console.error("An error occurred during the data stream execution.", error);
        return 'Oops, an error occured!';
      },
    });
  } catch (error) {
    console.error("Error in POST handler:", error);
    return new Response('An error occurred while processing your request!', {
      status: 404,
    });
  }
}

export async function DELETE(request: Request) {
  const { searchParams } = new URL(request.url);
  const id = searchParams.get('id');

  if (!id) {
    return new Response('Not Found', { status: 404 });
  }

  const session = await auth();

  if (!session || !session.user) {
    return new Response('Unauthorized', { status: 401 });
  }

  try {
    const chat = await getChatById({ id });

    if (chat.userId !== session.user.id) {
      return new Response('Unauthorized', { status: 401 });
    }

    await deleteChatById({ id });

    return new Response('Chat deleted', { status: 200 });
  } catch (error) {
    return new Response('An error occurred while processing your request!', {
      status: 500,
    });
  }
}

modified provider.ts:

import {
  customProvider,
  extractReasoningMiddleware,
  wrapLanguageModel,
} from "ai";
import { groq } from "@ai-sdk/groq";
import { xai } from "@ai-sdk/xai";

import { createOllama } from "ollama-ai-provider";

const ollama = createOllama({
  baseURL: "http://10.223.188.42:3001/process_message",
});
export const myProvider =  customProvider({
      languageModels: {
        "chat-model": ollama("llama3.1", { simulateStreaming: true }),
        "chat-model-reasoning": wrapLanguageModel({
          model: groq("deepseek-r1-distill-llama-70b"),
          middleware: extractReasoningMiddleware({ tagName: "think" }),
        }),
        "title-model": ollama("llama3.1", { simulateStreaming: true }),
        "artifact-model": ollama("llama3.1", { simulateStreaming: true }),
      },
      imageModels: {
        "small-model": xai.image("grok-2-image"),
      },
    });

response of the self hosted Llama provider :

{
  "created_at": "2025-03-31T01:22:12.358127",
  "done": true,
  "eval_count": 1,
  "eval_duration": 4.34537148475647,
  "message": {
    "role": "assistant",
    "content": [
      {
        "type": "tool-call",
        "toolCallId": "ed7c2ebf-fe60-440b-83d0-b5a055ade0b2",
        "toolName": "getWeather",
        "args": {
          "latitude": 37.7749,
          "longitude": -122.4194
        }
      }
    ]
  },
  "model": "llama3.1",
  "total_duration": 4.34537148475647
}

[codedpro]

can anybody help out?