-
Couldn't load subscription status.
- Fork 0
Description
Would be nice to have a costs calculation for all calls, for each provider
To return an struct that holds costs information, whenever we set the track_costs: true flag.
This could be setup like this example
# Cost tracking enabled with automatic calculation
@settings %LlmComposer.Settings{
providers: [
{LlmComposer.Providers.OpenAI, [
model: "gpt-4o-mini",
track_costs: true, # <- here and next two settings
input_price_per_million: "0.150",
output_price_per_million: "0.600"
]}
]
}For the providers that doesn't allow retrieving the cost information via API, only OpenRouter has an api that says the prices. OpenAi and Google have the pricing in their "pricing" pages.
When receiving the response, we may include in the LlmResponse struct, a new key with CostInfo proposed struct:
defmodule LlmComposer.CostInfo do
@moduledoc """
Struct for tracking costs and token usage of LLM API calls.
This module provides a standardized way to track both token usage and financial costs
across different LLM providers. It supports both streaming and non-streaming responses
and can automatically calculate costs when provided with pricing information.
## Fields
* `:input_tokens` - Number of tokens in the input/prompt
* `:output_tokens` - Number of tokens in the output/completion
* `:total_tokens` - Total tokens used (input + output)
* `:input_cost` - Cost for input tokens (using Decimal for precision)
* `:output_cost` - Cost for output tokens (using Decimal for precision)
* `:total_cost` - Total cost for the request (using Decimal for precision)
* `:currency` - Currency code (e.g., "USD", "EUR")
* `:provider_model` - The actual model used by the provider
* `:provider_name` - The provider that served the request
* `:input_price_per_million` - Price per million input tokens
* `:output_price_per_million` - Price per million output tokens
* `:metadata` - Additional provider-specific cost information
## Examples
# Basic token tracking
%LlmComposer.CostInfo{
input_tokens: 150,
output_tokens: 75,
total_tokens: 225
}
# With pricing information - costs calculated automatically
%LlmComposer.CostInfo{
input_tokens: 150_000,
output_tokens: 75_000,
input_price_per_million: Decimal.new("1.0"),
output_price_per_million: Decimal.new("3.0"),
currency: "USD",
provider_model: "gpt-4o-mini"
}
# Will automatically calculate:
# input_cost: 0.15, output_cost: 0.225, total_cost: 0.375
# Direct cost specification (bypasses automatic calculation)
%LlmComposer.CostInfo{
input_tokens: 150,
output_tokens: 75,
input_cost: Decimal.new("0.0015"),
output_cost: Decimal.new("0.0030"),
total_cost: Decimal.new("0.0045"),
currency: "USD"
}
"""
endStreaming responses case
https://github.com/doofinder/llm_composer?tab=readme-ov-file#streaming-responses
For this case, depending of the provider, we may have the tokens info at last chunk or not (openai case). Maybe we need to expose the construct of the CostInfo struct, so implementors does use that after having the input/output tokens information and the related provider, model and prices