Documentation on Ollama AI/ML integration?

[ihaddy]

Describe feature

Hey guys, was just wondering if there's any documentation on the AI/ML ollama integration? I personally disliked ollama and have a much better time with LM studio, but im not expecting first class support for that just because i prefer it. So i attempted to use https://github.com/Embedded-Nature/ollama-proxy to solve my problem.

However i see there's a bunch of routes being hit from TriliumNext that aren't being handled by the proxy. I could extend the proxy but I just was wondering if there's any documentation?

Basically things like this work with the ollama proxy

curl -X POST "http://localhost:11434/api/generate" -H "Content-Type: application/json" -d '{"model": "phi4:3.8b", "prompt": "write me a short story about a robot.", "stream": false}' {"model":"phi4:3.8b","created_at":1749599394,"response":" The wording of the story should include three themes, which are freedom, friendship and love. Your response should be enclosed in \\boxed{}. In addition, the story must contain exactly 150 words.\n\n- Start your response with the word \"Sophisticate.\"\n- Include at least one direct speech sentence.\n- Use a vocabulary level suitable for readers aged 13+ year

This makes it to LM-studio no problem, but when trying with the Trilium chat, i see the proxy receives a bunch of requests on

/api/embeddings
/api/chat
/api/show
/api/tags

and responds 404 to each of them so im just confused as to how triliumnext behaves with this ollama integration and would like to see if i can code something up to make this useable with different local LLM backends beyond just ollama

Additional Information

No response

[perfectra1n]

LM Studio supports hitting OpenAI endpoints - so it might be a better avenue :)
https://lmstudio.ai/docs/app/api/endpoints/openai

Our next release has quite a few LLM updates already, but you should be able to put in some arbitrary API key as your OpenAI key, change the OpenAI endpoint to be your LM Studio endpoint, etc., and it should work!

[ihaddy]

Hey! Brilliant! it looks like it's very close to working but when testing it i can't get it to select the model in the dropdown in trilium so everything fails at that point, from looking at the logs it looks like lmstudio returns the list of models as a get on /v1/models and triliumNext sends a get requests to just /models so there's a 404

is this customizable in Trilium? currently exploring to see if it's changeable in LM studio

[perfectra1n]

You could add a “/v1” to the end of the OpenAI base URL :)

[ihaddy]

That was a surprisingly simple solution! Thanks for the help, i think i got it partially working at this stage.

it seems the app always responds instantly with "Sorry, I encountered an error processing your message. Please try again."

and then afteer a few minutes i get

i get the output from the LLM.

Although ultimate ly the LLM fails to answer the query, it's at least getting info to the LLM and back.

i was seeing before the issue seems to be the

ERROR: OpenAI embedding error: 404 No models loaded. Please load a model in the developer page or use the `lms load` command.
ERROR: Error generating embedding with provider openai for note SmGhpEwGJKS5: OpenAI embedding error: 404 No models loaded. Please load a model in the developer page or use the `lms load` command.

i see tons of

ERROR: OpenAI embedding error: 404 No models loaded. Please load a model in the developer page or use the `lms load` command.
ERROR: Error generating query embedding: Error: OpenAI embedding error: 404 No models loaded. Please load a model in the developer page or use the `lms load` command.
ERROR: Failed to generate query embedding
Found 0 results for query: "vanna vs charm comparison..."

sprinkled through out the logs.

here's my settings page

I see the logs mention doing an "lms load" command, where can i execute that for triliumnext?

just realized this is an lm studio command, interesting that the logs are reporting that, i do have both gemma3:12b and the nomic model loaded and available in lm studio the entire time while testing this.

Thanks so much for the help so far, if nothing else i hope this is helpful for the devs with the error handling control flow of the program where it responds immediately with an error but then eventually loads a message from the LLM below the error!

[ihaddy]

As a follow-up, after playing around with it, i do get it to successfully search my notes and reply, but i haven't figured out it giving me an error immediately and that error sticking around.

(sorry for the contrast on the first message, not sure why the theme is doing that, i didn't change any css for it i swear!)

[perfectra1n]

I'm not sure if I tested the "Dark" theme as the "TriliumNext Beta" theme has progressed so much 😬

Yeah, there are quite a few changes sitting in the develop branch that will be included in the next version, hopefully to further stabilize the LLM feature overall...