fluent-ai

Warning

This project is in beta. The API is subject to changes and may break.

fluent-ai is a lightweight, type-safe AI toolkit that seamlessly integrates multiple AI providers. It features structured outputs, streaming capabilities, and job serialization support.

Installation

Zod is a popular type of validation library for TypeScript and JavaScript that allows developers to define and validate data schemas in a concise and type-safe manner. fluent-ai is built upon zod.

npm install fluent-ai zod

AI Service provider support

fluent-ai includes support for multiple AI providers and modalities.

provider	chat completion	embedding	image generation	list models	text to speech
anthropic	✅			✅
elevenlabs					✅
fal			✅
google	✅
ollama	✅	✅		✅
openai	✅	✅	✅	✅	✅
voyage		✅

By default, API keys for providers are read from environment variable (process.env) following the format <PROVIDER>_API_KEY (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY).

You can also initialize a provider with manual API key settings:

import { openai } from "fluent-ai";

openai({ apiKey: "<key>" });

For more examples with different AI providers, check out the examples directory.

Don't see your AI providers? Feel free to open an issue or start a discussion to request support. Join our Discord community

Job API

Each request to AI providers is wrapped in a Job. which can also serialized and deserialized. A fluent API with method chaining help create jobs easily.

Method chaining

import { openai, user } from "fluent-ai";

const job = openai()
  .chat("gpt-4o-mini")
  .messages([user("Hi")])
  .temperature(0.5)
  .maxTokens(1024);

Declaration

Alternatively, fluent-ai supports declarative job creation using JSON objects, with full TypeScript autocompletion support.

import { load } from "fluent-ai";

const job = load({
  provider: "openai",
  type: "chat",
  input: {
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "hi" }],
    temperature: 0.5,
  },
});

fluent-ai provides built-in TypeScript type definitions and schema validation for jobs:

import { type Job } from "fluent-ai"; // TypeScript type
import { JobSchema } from "fluent-ai"; // Zod schema
import { jobJSONSchema } from "fluent-ai"; // JSON Schema

Job serialization and deserialization

To serialize a job to a JSON object, use the dump method:

const payload = job.dump();

This allows you to save the job's state for later use, such as storing it in a queue or database. To recreate and execute a job from the JSON object, use the load function:

import { load } from "fluent-ai";

const job = load(payload);
await job.run();

Chat completions

Chat completion, such as ChatGPT, is the most common AI service. It generates responses in a conversational format based on given inputs, also knows as prompts.

Text generation

import { openai, system, user, text } from "fluent-ai";

const job = openai()
  .chat("gpt-4o-mini")
  .messages([system("You are a helpful assistant"), user("Hi")]);

const result = await job.run();
console.log(text(result));

Function calling (tool calling)

Function calling (or tool calling) is an advanced functionality in chat completions that enhances their ability to interact with external systems and perform specific tasks.

Here's how to create a tool:

import * as z from "zod";
import { tool } from "fluent-ai";

const weatherTool = tool("get_current_weather")
  .description("Get the current weather in a given location")
  .input(
    z.object({
      location: z.string(),
      unit: z.enum(["celsius", "fahrenheit"]).optional(),
    }),
  );

To use the tool, add it to a chat job with a function-calling-enabled model, such as gpt-4o-mini from openai.

const job = openai().chat("gpt-4o-mini").tool(weatherTool);

await job.messages([user("What is the weather in San Francisco?")]).run();

Streaming support

Rather than waiting for the complete response, streaming enables the model to return portions of the response as they're generated. fluent-ai provides built-in streaming support for text, objects, and tools in chat models.

const job = openai()
  .chat("gpt-4o-mini")
  .messages([system("You are a helpful assistant"), user("Hi")])
  .stream();

for await (const event of await job.run()) {
  console.log(text(event));
}

fluent-ai supports streaming text, object and tool calls on demand. For more details, see the streaming docs.

Embedding

import { openai } from "fluent-ai";

const job = openai().embedding("text-embedding-3-small").value("hello");
const result = await job.run();

Image generation

import { openai } from "fluent-ai";

const job = openai().image("dalle-2").prompt("a cat").n(1).size("512x512");
const result = await job.run();

List models

fluent-ai provides an easy way to retrieve all available models from supported providers (openai, anthropic, ollama).

import { openai } from "fluent-ai";

const models = await openai().models().run();

Text to Speech

import { openai } from "fluent-ai";

const job = openai().model("tts-1").text("hi");
const result = await job.run();

Support

Feel free to open an issue or start a discussion if you have any questions. If you would like to request support for a new AI provider, please create an issue with details about the provider's API. Join our Discord community for help and updates.

License

fluent-ai is licensed under Apache 2.0 as found in the LICENSE file.