Replace providers like ollama, lm studio, etc. with pure llama.cpp server or equivalent?

[wbste]

Hi all!
Tired of subpar performance due to wrappers (ollama) and waiting for updates (LM Studio). I would love to only rely on one thing llama.cpp for my needs, which is as follows (all compat. with OpenAI endpoints.)

• Support image(s) as input for vision models.
• Support native tool calling.
• Support structured output.
• Dynamic loading/unloading of models based on demand.
• Endpoint to list models.

The top three are done, but the bottom two aren't available in the server from what I gather. I see llama-cpp-python but seems out of date (no current vision models listed in readme and doesn't seem to grab latest release here).

This must be a solved problem, but not having luck finding it. Thanks for the excellent project and support!

[ITankForCAD]

Llama-swap is what you're looking for.

[wbste]

I can't believe I couldn't find that. Thanks so much!