Question / feature idea

[schneiderpy]

I came accross your package {searcher} on the Slack channel (where you've mentioned it) and was asking myself if it is - or would be - possible to store the external results coming from ask_*() locally (or cloud) for later use, e.g. (to train) a local personal assistent (kind of Knowledge base)?

[coatless]

@schneiderpy unfortunately, we can't easily access this information for all AI service providers since this would invalidate API use cases if permitted.

To support something like this, we'd likely need to have more control over the browser than simply opening a URL. That would imply writing and keeping up-to-date several custom chromote web scraping scripts. Specifically, we'd need to launch a controlled browser session with the logged in user account, and, then, parse from R the web page response after sending the prompt.

Alternatively, one could use tampermonkey and have a custom script per AI Service provider (example: {claude-downloader}) to permit downloading the chat when given the chat URL inside of R or by directly clicking on the chat.

That said, many service providers do allow for a user's entire chat history (across all conversations):

1. OpenAI's ChatGPT: https://help.openai.com/en/articles/7260999-how-do-i-export-my-chatgpt-history-and-data
2. Anthropic's Claude: https://support.anthropic.com/en/articles/9450526-how-can-i-export-my-claude-ai-data
3. Mistral: https://help.mistral.ai/en/articles/323761-how-do-i-export-my-data-from-le-chat
4. Microsoft (Bing) Copilot: https://account.microsoft.com/privacy/copilot
5. xAI's Grok: https://x.ai/legal/faq#how-do-i-select-whether-my-content-is-used-for-model-training

Missing: Perplexity & MetaAI

[schneiderpy]

Thank you @coatless for your prompt reply.
Actually I'm more interested if this could be feasible for an Open Source solution, however, I don't have the technical know-how. In other words, I would like to keep the AI-generated information (that solved any problem) for myself in an Open Source framework since I have Ollama, Deepseek (and some others) installed.

[coatless]

@schneiderpy so, something like ollama and deepseek likely would have this history stored locally in some kind of postgres database. Are you accessing these LLMs via {ellmer}?

[schneiderpy]

@coatless I don't use {ellmer} .. I didn't know about {ellmer}. I use Msty (or AnythingLLM) with Ollama, deepseek, qwen, Mistral ... or other external LLMs. It seems that {ellmer} is similar to Msty or AnythingLLM.

[coatless]

@schneiderpy from those names you mentioned, note:

• Msty's export history (if subscribed): https://docs.msty.app/features/export-chat
• AnythingLLM export history: https://docs.anythingllm.com/features/chat-logs

As we're getting into localized LLMs and how to export their chats, I'm going to close this issue as it's outside the scope of searcher for now.