feature/track costs general

CHANGELOG.md

@@ -6,6 +6,19 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 ## [Unreleased]
 
+### Added
+- Automatic cost tracking for OpenAI, Google, and OpenRouter providers:
+  - Fetches real-time pricing from provider APIs (models.dev for OpenAI/Google, OpenRouter API).
+  - `CostInfo` struct in responses with token usage and cost breakdowns.
+  - Support for automatic and manual pricing configuration.
+- `Pricing` module for cost calculations.
+- Supporting modules for OpenRouter pricing and tracking.
+
+### Changed
+- Enhanced `LlmResponse` to include cost information.
+- Updated provider implementations (OpenAI, Google, OpenRouter) to support cost tracking.
+- Updated README with examples for cost tracking.
+
 ## [0.11.2] - 2025-09-25
 
 - Updated README a bit more docs and examples. Included in new release for hex docs too.

README.md

@@ -27,7 +27,8 @@
   - [Provider Router Simple](#provider-router-simple)
   - [Cost Tracking](#cost-tracking)
     - [Requirements](#requirements)
-    - [Basic Cost Tracking Example](#basic-cost-tracking-example)
+    - [Automatic Cost Tracking](#automatic-cost-tracking)
+    - [Manual Pricing](#manual-pricing)
     - [Starting Cache in a Supervision Tree](#starting-cache-in-a-supervision-tree)
     - [Dependencies Setup](#dependencies-setup)
   - [Additional Features](#additional-features)
@@ -74,6 +75,7 @@ The following table shows which features are supported by each provider:
 | Function Calls | ✅ | ✅ | ❌ | ❌ | ✅ |
 | Auto Function Execution | ✅ | ✅ | ❌ | ❌ | ✅ |
 | Structured Outputs | ✅ | ✅ | ❌ | ❌ | ✅ |
+| Cost Tracking | ✅ | ✅ | ❌ | ❌ | ✅ |
 | Fallback Models | ❌ | ✅ | ❌ | ❌ | ❌ |
 | Provider Routing | ❌ | ✅ | ❌ | ❌ | ❌ |
 
@@ -276,6 +278,47 @@ LlmComposer.Message.new(
 )
 ```
 
+#### Automatic Cost Tracking with OpenRouter
+
+OpenRouter supports automatic cost tracking without manual pricing configuration. When `track_costs: true` is enabled, LlmComposer automatically fetches real-time pricing data from OpenRouter's API and calculates costs based on actual token usage.
+
+```elixir
+# Configure the OpenRouter API key
+Application.put_env(:llm_composer, :open_router, api_key: "<your openrouter api key>")
+
+defmodule MyOpenRouterAutoCostChat do
+  @settings %LlmComposer.Settings{
+    providers: [
+      {LlmComposer.Providers.OpenRouter,
+       [
+         model: "anthropic/claude-3-haiku:beta",
+         track_costs: true  # Enables automatic cost tracking
+       ]}
+    ],
+    system_prompt: "You are a helpful assistant."
+  }
+
+  def simple_chat(msg) do
+    LlmComposer.simple_chat(@settings, msg)
+  end
+end
+
+{:ok, res} = MyOpenRouterAutoCostChat.simple_chat("Explain quantum computing")
+
+# Access automatic cost information
+IO.puts("Input cost: #{Decimal.to_string(res.cost_info.input_cost, :normal)}$")
+IO.puts("Output cost: #{Decimal.to_string(res.cost_info.output_cost, :normal)}$")
+IO.puts("Total cost: #{Decimal.to_string(res.cost_info.total_cost, :normal)}$")
+```
+
+**Features:**
+- **Automatic Pricing**: Fetches up-to-date prices from OpenRouter API without manual configuration
+- **Granular Tracking**: Provides breakdown of input vs. output costs
+- **Caching**: Prices are cached for 24 hours to reduce API calls
+- **Error Handling**: Gracefully falls back if pricing data is unavailable
+
+**Note:** Automatic cost tracking requires the Decimal package: `{:decimal, "~> 2.3"}`
+
 ### Using AWS Bedrock
 
 LlmComposer also integrates with [Bedrock](https://aws.amazon.com/es/bedrock/) via its Converse API. This allows you to use Bedrock as a provider with any of its supported models.
@@ -777,7 +820,9 @@ This mechanism ensures high availability and resilience by automatically failing
 
 ### Cost Tracking
 
-LlmComposer provides built-in cost tracking functionality, for **OpenRouter backend only**, to monitor token usage and associated costs across different providers. This feature helps you keep track of API expenses and optimize your usage.
+LlmComposer provides built-in cost tracking functionality for **OpenAI**, **OpenRouter**, and **Google** providers to monitor token usage and associated costs. This feature helps you keep track of API expenses and optimize your usage by providing detailed cost breakdowns per request.
+
+**OpenAI**, **OpenRouter**, and **Google** support automatic cost tracking that fetches real-time pricing data from their respective APIs (models.dev API for OpenAI/Google, OpenRouter API for OpenRouter), eliminating the need for manual pricing configuration.
 
 #### Requirements
 
@@ -786,61 +831,156 @@ To use cost tracking, you need:
 1. **Decimal package**: Add `{:decimal, "~> 2.0"}` to your dependencies in `mix.exs`
 2. **Cache backend**: A cache implementation for storing cost data (LlmComposer provides an ETS-based cache by default, or you can implement a custom one using `LlmComposer.Cache.Behaviour`)
 
-#### Basic Cost Tracking Example
+#### Automatic Cost Tracking
+
+**OpenAI** and **Google** support automatic cost tracking via the models.dev dataset. When you enable `track_costs: true` and do not supply explicit prices, LlmComposer will fetch pricing from models.dev and cache it (ETS). Start the cache before using automatic pricing.
+
+```elixir
+# OpenAI automatic pricing example
+Application.put_env(:llm_composer, :open_ai, api_key: "<your openai api key>")
+
+# Start the cache backend (required for automatic pricing fetching)
+{:ok, _} = LlmComposer.Cache.Ets.start_link()
+
+defmodule MyOpenAICostTrackingChat do
+  @settings %LlmComposer.Settings{
+    providers: [
+      {LlmComposer.Providers.OpenAI,
+       [
+         model: "gpt-4o-mini",
+         track_costs: true
+         # these are optional, if not provided, this data is fetched from models.dev api and cached with ets
+         # input_price_per_million: "0.150",
+         # output_price_per_million: "0.600"
+       ]}
+    ],
+    system_prompt: "You are a helpful assistant."
+  }
+
+  def run_chat_with_costs() do
+    messages = [
+      %LlmComposer.Message{type: :user, content: "Explain quantum computing briefly"}
+    ]
+
+    {:ok, res} = LlmComposer.run_completion(@settings, messages)
+
+    # Access cost information from the response
+    IO.puts("Provider: #{res.cost_info.provider_name}")
+    IO.puts("Model: #{res.cost_info.provider_model}")
+    IO.puts("Total tokens: #{res.cost_info.total_tokens}")
+    IO.puts("Total cost: #{Decimal.to_string(res.cost_info.total_cost, :normal)}$")
+
+    res
+  end
+end
+
+MyOpenAICostTrackingChat.run_chat_with_costs()
+```
+
+**Google** uses the same setup (replace provider with `LlmComposer.Providers.Google` and model with e.g. `"gemini-2.5-flash"`).
+
+
+#### Automatic Cost Tracking (OpenRouter)
+
+OpenRouter also supports automatic pricing via its API. Start the cache and enable `track_costs: true`; prices will be fetched and cached if not provided.
 
 ```elixir
 Application.put_env(:llm_composer, :open_router, api_key: "<your openrouter api key>")
 
-defmodule MyCostTrackingChat do
+# Start the cache backend (required for automatic pricing fetching)
+{:ok, _} = LlmComposer.Cache.Ets.start_link()
+
+defmodule MyOpenRouterCostTrackingChat do
   @settings %LlmComposer.Settings{
     providers: [
-      {LlmComposer.Providers.OpenRouter, [model: "meta-llama/llama-3.2-3b-instruct"]}
+      {LlmComposer.Providers.OpenRouter,
+       [
+         model: "anthropic/claude-3-haiku:beta",
+         track_costs: true,
+         # these are optional, if not provided, this data is fetched from openrouter api and cached with ets
+         # input_price_per_million: "0.250",
+         # output_price_per_million: "1.250"
+       ]}
     ],
-    system_prompt: "You are a helpful assistant.",
-    track_costs: true
+    system_prompt: "You are a helpful assistant."
+  }
+
+  def run_chat_with_costs() do
+    messages = [
+      %LlmComposer.Message{type: :user, content: "Explain quantum computing briefly"}
+    ]
+
+    {:ok, res} = LlmComposer.run_completion(@settings, messages)
+
+    # Access cost information from the response
+    IO.puts("Provider: #{res.cost_info.provider_name}")
+    IO.puts("Model: #{res.cost_info.provider_model}")
+    IO.puts("Total tokens: #{res.cost_info.total_tokens}")
+    IO.puts("Total cost: #{Decimal.to_string(res.cost_info.total_cost, :normal)}$")
+
+    res
+  end
+end
+
+MyOpenRouterCostTrackingChat.run_chat_with_costs()
+```
+
+#### Manual Pricing (Explicit Prices — no cache required)
+
+If you prefer to provide explicit prices for input/output tokens, you can do so and no ETS cache is required. This is useful when you already know your pricing or want to override automatic fetching.
+
+```elixir
+Application.put_env(:llm_composer, :google, api_key: "<your google api key>")
+
+defmodule MyManualCostTrackingChat do
+  @settings %LlmComposer.Settings{
+    providers: [
+      {LlmComposer.Providers.Google,
+       [
+         model: "gemini-2.5-flash",
+         track_costs: true,
+         input_price_per_million: "0.075",
+         output_price_per_million: "0.300"
+       ]}
+    ],
+    system_prompt: "You are a helpful assistant."
   }
 
   def run_chat_with_costs() do
     messages = [
       %LlmComposer.Message{type: :user, content: "How much is 1 + 1?"}
     ]
-    
+
     {:ok, res} = LlmComposer.run_completion(@settings, messages)
-    
+
     # Access cost information from the response
     IO.puts("Input tokens: #{res.input_tokens}")
     IO.puts("Output tokens: #{res.output_tokens}")
-    IO.puts("Total cost: #{Decimal.to_string(res.metadata.total_cost, :normal)}$")
-
+    IO.puts("Input cost: #{Decimal.to_string(res.cost_info.input_cost, :normal)}$")
+    IO.puts("Output cost: #{Decimal.to_string(res.cost_info.output_cost, :normal)}$")
+    IO.puts("Total cost: #{Decimal.to_string(res.cost_info.total_cost, :normal)}$")
+
     res
   end
 end
 
-# Start the cache backend (required for cost tracking)
-# The default ETS cache can be overridden by configuring a custom cache module:
-#
-# config :llm_composer, cache_mod: MyCustomCache
-#
-# Your custom cache module must implement the LlmComposer.Cache.Behaviour
-# which defines callbacks for get/1, put/3, and delete/1 operations.
-{:ok, _} = LlmComposer.Cache.Ets.start_link()
-
-MyCostTrackingChat.run_chat_with_costs()
+MyManualCostTrackingChat.run_chat_with_costs()
 ```
 
+
 #### Starting Cache in a Supervision Tree
 
-For production applications, you should start the cache as part of your application's supervision tree:
+For production applications, add the ETS cache to your supervision tree to ensure it's available for automatic pricing fetching:
 
 ```elixir
-# In your application.ex file
+# In your application.ex
 defmodule MyApp.Application do
   use Application
 
   def start(_type, _args) do
     children = [
-      # Other supervisors/workers...
-      LlmComposer.Cache.Ets,
+      # Other children...
+      LlmComposer.Cache.Ets,  # Add this for automatic pricing caching
       # ... rest of your children
     ]
 
@@ -850,6 +990,13 @@ defmodule MyApp.Application do
 end
 ```
 
+You can also start it manually in development or testing:
+
+```elixir
+# Start the cache manually
+{:ok, _} = LlmComposer.Cache.Ets.start_link()
+```
+
 #### Dependencies Setup
 
 Add the decimal dependency to your `mix.exs`:
@@ -863,7 +1010,7 @@ def deps do
 end
 ```
 
-**Note:** Cost tracking calculates expenses based on the provider's pricing model and token usage. The cache backend stores pricing information to avoid repeated lookups and improve performance.
+**Note:** Automatic pricing requires starting the ETS cache (see Starting Cache section). Manual pricing does not require the cache. If automatic pricing sources (models.dev or OpenRouter API) are unavailable, token counts are still tracked, but costs will be nil. All cost information is available in the `CostInfo` struct within responses.
 
 ### Additional Features
 * Auto Function Execution: Automatically executes predefined functions, reducing manual intervention.