diff --git a/ai-ingestion.mdx b/ai-ingestion.mdx index d48fa049..3f358142 100644 --- a/ai-ingestion.mdx +++ b/ai-ingestion.mdx @@ -12,40 +12,144 @@ export const PreviewButton = ({ children, href }) => { ) } -Documentation pages are content rich and can be a great source of information for LLMs for AI tools. +## What is AI Ingestion? -Today, you can use your documentation to fine-tune a chatbot, answer questions about your product, or use it to generate code. +AI ingestion is the process of making your documentation accessible and consumable by Large Language Models (LLMs) and other AI tools. By properly formatting and exposing your documentation, you can: -## Supported Shortcuts + + + Enable AI tools to better understand and process your documentation content + + + Make your content more accessible to various AI-powered tools and platforms + + + Allow users to interact with your documentation through AI-powered interfaces + + + Optimize how AI tools consume and analyze your documentation + + -We support a variety of shortcuts and configurations to make it easier to use your documentation in AI tools. +## Available Formats and Shortcuts -### /llms.txt +Mintlify automatically generates multiple formats of your documentation to support different AI ingestion needs. Here's a comprehensive guide to each format: -An [industry standard](https://llmstxt.org) that helps general-purpose LLMs index more efficiently (e.g. a sitemap for AI). +### 1. llms.txt Format -Every documentation site automatically hosts a **/llms.txt** file at the root that lists all available pages in your documentation. + + The llms.txt format follows the [industry standard](https://llmstxt.org) specification for helping LLMs efficiently index content, similar to how sitemaps work for search engines. + + +Every documentation site automatically hosts a **/llms.txt** file at the root that provides a structured list of all available pages. Open llms.txt for this site -### /llms-full.txt +Example structure of llms.txt: +```txt +https://your-docs.com/getting-started +https://your-docs.com/api-reference +https://your-docs.com/guides/authentication +``` + +### 2. llms-full.txt Format -A markdown file of all your content is automatically available at **/llms-full.txt**. + + This format is ideal when you need to provide complete documentation context to AI tools for comprehensive analysis or training. + -llms-full.txt contains the entire content of your docs, and is a great way to provide a complete context to AI tools. +The **/llms-full.txt** endpoint provides a complete markdown version of your entire documentation in a single file. Open llms-full.txt for this site -### .md extension +Use cases include: +- Training custom AI models +- Building documentation-aware chatbots +- Creating comprehensive search indices -Add **.md** to an individual docs page to see a markdown version of that page. Helps load individual pages into AI tools with better token efficiency & inference time. +### 3. Markdown Version (.md) -Open quickstart.md + + Adding `.md` to any page URL provides a markdown version of that specific page, optimized for AI processing. + -### contextual menu +Simply append **.md** to any documentation URL to access its markdown version: -You can enable a [contextual menu](settings#param-contextual) to allow users to copy markdown source, view markdown source, or open ChatGPT with the page content. +```plaintext +https://your-docs.com/api-reference.md +https://your-docs.com/getting-started.md +``` -### ⌘ \+ c shortcut +Open quickstart.md + +### 4. Quick Access Features + +#### Contextual Menu +Enable the [contextual menu](settings#param-contextual) to provide users with quick actions: +- Copy markdown source +- View markdown source +- Open content directly in ChatGPT + +#### Keyboard Shortcut +Use **⌘ + c** (or **Ctrl + c** on Windows) to quickly copy the markdown source of any page. + +## Best Practices for AI Ingestion + +Follow these guidelines to maximize the effectiveness of AI ingestion: + + + + - Use clear headings and subheadings + - Maintain consistent formatting + - Include relevant metadata + - Use appropriate tags and categories + + + + - Keep content concise and clear + - Use standard markdown formatting + - Avoid complex custom HTML + - Include code examples with proper language tags + + + + - Keep content up to date + - Remove deprecated information + - Regularly verify external links + - Monitor AI tool feedback + + + + + Avoid including sensitive information in your documentation as it will be accessible through these AI ingestion endpoints. + + +## Implementation Examples + +Here's how you might use these formats in different scenarios: + + +```python Python +import requests + +# Fetch markdown version of a specific page +response = requests.get('https://your-docs.com/api-reference.md') +markdown_content = response.text + +# Process with an AI tool +ai_response = process_with_ai(markdown_content) +``` + +```javascript JavaScript +// Fetch complete documentation +async function fetchFullDocs() { + const response = await fetch('https://your-docs.com/llms-full.txt'); + const content = await response.text(); + return content; +} -Copy markdown source of any page using command \+ c. +// Use in your AI application +const docs = await fetchFullDocs(); +const aiResult = await processWithAI(docs); +``` + \ No newline at end of file diff --git a/docs.json b/docs.json index 35f1a866..220c08e5 100644 --- a/docs.json +++ b/docs.json @@ -24,7 +24,8 @@ "index", "quickstart", "installation", - "editor" + "editor", + "ai-ingestion" ] }, {