Skip to content

remove messages that got limit in the history #201

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

guitavano
Copy link
Contributor

Currently, when a message with too many tokens is sent or received, the chat fails because it always tries to include the full thread history.

not sure about use the number 200.000

https://www.loom.com/share/ccb8f8f1b7374e1bad7177d6ccffe27a

@viktormarinho
Copy link
Contributor

viktormarinho commented May 10, 2025

some considerations:

  • string length is not the same thing as llm tokens
  • the server also includes lots of tokens on the context, like instruction on the agent configuration and tools, so maybe we're better off calculating it there. actually, i believe we already have it or almost, you should check it there on the actor. ask me if you need help on locating it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants