feat(build): add llms.txt and llms-full.txt support

[YoSev]

Add vitepress-plugin-llms for LLM support.

This plugin only works after build - not in dev mode according to the author!

[System-Glitch]

I don't understand exactly what it does and how that works just by reading the plugin's readme. Could you explain the benefits? Is it a widespread practice? Are most LLMs compatible with it?

The docs website uses several markdown plugins (for callouts for example), would that still work well?

[YoSev]

Its more like a wide spread adopted proposel.
Most LLMs check sites for /llms.txt to understand the content better - and they prefer markdown.

Quite some famous websites use it as of today and myself has made good experience:

• https://docs.stripe.com/llms.txt
• https://docs.ffmate.io/llms.txt >> https://docs.ffmate.io/llms-full.txt

When LLMs "index" your website, the results are simply cleaner. It also help on the fly indexing when pasting a URL (for example to docs) into the prompt.

[System-Glitch]

I'll think about it. Thanks for the suggestion! In the meantime, can you rewrite your PR to remove the formatting changes (quotes, semi-colon, spacing, etc) please?

[YoSev]

I can disable auto-formatting on save in my IDE but a common formatting library like prettier would be a better solution if you want people to contribute without having formatting issues.

[YoSev]

I'll think about it. Thanks for the suggestion! In the meantime, can you rewrite your PR to remove the formatting changes (quotes, semi-colon, spacing, etc) please?

Done

[System-Glitch]

I looked into it a bit more and decided not to add this to the docs website. At least yet.

• This is not a standard yet and still very much just a proposal, without any proof of adoption by the LLM providers. There is no real specification as to how LLM providers need to use or index the file.
• The user experience might be degraded if the llms.txt file is referenced instead of the actual page, since source links will probably point towards the markdown file.
• Experts on the topic say that this is completely unnecessary, and my guts told me the same initially. There are already plenty of SEO tools that LLMs take advantage of like search engines (sitemaps, semantic HTML, meta tags, ...).
• Some sneaky people already found loopholes to trick SEO and LLMs using llms.txt. This may lead to worse indexing because search engines would lower the site's score if there is such file, just like they do for all other usual shady SEO tricks.
• Overall this is still a very immature and hypothetical system.

My sources:

• https://llmstxt.org/
• https://www.reddit.com/r/SEO/comments/1nnocpu/semrush_and_txt_file/
• https://www.searchenginejournal.com/llms-txt-for-ai-seo/556576/
• https://www.reddit.com/r/TechSEO/comments/1k0kcx9/llmtxt_where_are_we_at/