llms.txt guide
llms.txt: what it is, and how to write one.
What is an AI text file for LLMs? What the proposal says, how to create the file, and what Google says about using it.
Short answer
What is an AI text file for LLMs?
The llms.txt file is a proposed markdown file at a site's root. It gives language models a curated overview: a short summary and links to your key pages. Google Search says it does not use it.
In brief
Five things to know first.
A proposal, not a rule
The AI text file is a community proposal for a markdown document that gives language models a curated view of your site.
Only one part is required
A valid file needs just an H1 with the site name. Summary, notes and link lists are optional.
It complements, not replaces
A sitemap lists all pages and robots.txt sets access. The AI text file adds a curated overview.
Google does not use it
Google says you don't need AI text files to appear in Google Search, including its generative AI features.
Adoption is strongest in docs
Coding agents follow it to API references. Developer documentation is where it is used most.
The basics
What an llms.txt file is
Track how often AI engines cite your pages before and after launching the file. Google Search does not use it. LLMReach sets it up with schema and crawler access. Start with the free AI audit.
Jeremy Howard proposed it in 2024. The proposal's v2 draws on two years of adoption, which its author says includes thousands of sites publishing a file.
A markdown file
It lives at your site root, written in plain markdown.
A curated overview
A short summary plus links to the pages that matter most.
Read on demand
The authors expected it to help with inference, not training.
The format
llms.txt format and specification
The specification keeps the structure small. Only the H1 is required. Everything after it is optional, and the order is fixed.
The proposal uses Markdown instead of XML because language models and agents read these files, yet standard tools can still parse them.
| Section | Required? | What it holds |
|---|---|---|
| H1 | Yes | Name of the project or site |
| Blockquote | No | Short summary with key information |
| Body text | No | Details in paragraphs or lists, no headings |
| H2 file lists | No | Links as [name](url), with optional notes |
| Optional H2 | No | Secondary links an agent can skip |
How it compares
llms.txt vs robots.txt and sitemaps
The proposal says this file complements existing ones. It does not set access rules or list every URL.
Mintlify, a docs platform vendor, puts it this way: sitemap.xml lists what exists, robots.txt controls access, and the AI text file signals what matters most.
| File | What it does | |
|---|---|---|
| robots.txt | robots.txt | Tells tools what access is acceptable |
| sitemap.xml | sitemap.xml | Lists all pages for search engines |
| llms.txt | AI text file | Gives LLMs a curated overview, read on demand |
Want to see how AI engines read your site?
Our free AI audit shows where your brand appears in ChatGPT, Claude, Perplexity and Gemini answers, and what blocks it.
How to create it
How to create llms.txt: 5 steps
You can write the file by hand in any text editor. Keep it short and curated, not a copy of your sitemap.
Name the site
Start with an H1 holding the name of your project or site. This is the only required part.Write a short summary
Add a blockquote with the key facts a reader needs to understand the rest of the file.List your key pages
Under H2 headings, add links as [name](url), each with a short note about the page.Mark secondary links
Put lower-priority links under an Optional H2, so agents can skip them for a shorter context.Publish at the root
Upload the file to your site root. The proposal also allows files at subpaths; the most specific one applies.
Going deeper
llms-full.txt and .md pages
The proposal suggests pages also offer a clean markdown version: add .md to the URL or swap the extension for .md.
Some companies pair a slim index with a larger llms-full.txt export. Anthropic publishes both for its docs.
| AI text file | llms-full.txt | |
|---|---|---|
| Content | A slim index with links | The full documentation in markdown |
| Best for | Real-time assistants such as ChatGPT and Claude | Ingestion pipelines, IDE indexing and RAG |
SEO and Google
Does Google use llms.txt?
Google's guide says: “You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search.” It adds that Google Search itself doesn't use them.
For SEO, Google's recap says that for Google Search you can ignore tactics like creating unnecessary AI text files. Google adds that SEO best practices stay relevant.
Google Search does not use it
Google says Search, including its generative AI features, does not use these files.
Foundations still apply
Its AI features are rooted in core Search ranking and quality systems.
Other engines may differ
These sources do not show how other assistants treat the file. Test rather than assume.
Fix the foundation first
We handle AI text files, schema, crawler access and rendering as part of technical AEO, built from your audit findings.
Adoption
Who publishes llms.txt today
OpenAI, Anthropic and Gemini publish AI text files for their developer docs. Chrome's Lighthouse also audits sites for one as part of its agentic browsing checks.
The file is used most for software documentation, where coding agents follow it to API references. It can also serve businesses, personal sites and schools.
Who publishes AI text files today
OpenAI, Anthropic and Gemini publish AI text files for their developer docs. Chrome's Lighthouse also audits sites for one as part of its agentic browsing checks.
FAQ
llms.txt: common questions.
Is an AI text file worth creating?
The AI text file is cheap to write and read by some tools, but Google says Search doesn't use it. It helps most where agents read documentation. Treat it as a small addition to crawlable, useful pages, not a way to earn citations.
Does Google use an AI text file?
Google does not use AI text files in Search. Its guide says you don't need AI text files or Markdown to appear in Google Search, including its generative AI features. Its recap tells site owners to ignore unnecessary files like these.
What is the AI text file, and how do I add it to WordPress?
WordPress sites can add the AI text file by uploading it to the site root, or through a plugin. The llmstxt.org proposal lists Yoast SEO and AIOSEO as plugins that generate the file automatically. Check the result at your domain's root path.
Do I need llms-full.txt, and how do I add it on Shopify?
Shopify does not give direct access to a store's root folder, and these sources include no Shopify setup guide. Check your theme or app options, or ask a developer to serve the file at your site root. Then confirm it loads in a browser.
What is the difference between the index file and llms-full.txt?
The basic AI text file is a slim index with a summary and links. Llms-full.txt is a larger export holding the full documentation in markdown. Anthropic publishes both: the index suits real-time assistants, the full file suits ingestion pipelines such as RAG systems.
Is the AI text file an official standard?
The AI text file is a proposal published at llmstxt.org, not a ratified standard. No search engine is required to follow it, and Google says it doesn't use it. Some AI labs publish files for their own docs, and some tools read them.
Does this AI text file replace robots.txt?
The file does not replace robots.txt. The proposal says it complements it: robots.txt tells tools what access is acceptable, while this file gives context for allowed content. Neither file guarantees that an AI assistant will read or cite your pages.
Free AI Audit
Find out how AI engines read and cite your site.
You leave knowing where the gap is, what is causing it, and which changes would matter first, whether we work together or not.
What to expect
Audit
Before the call, we run your buyers' questions through ChatGPT, Claude, Perplexity and Gemini.
Map
On the call, we show you where competitors are cited and you are not, and what's causing it.
Decide
An honest read on whether the Citation Stack fits. If it doesn't, you'll hear it on the call.
Don't see a time that works? Email Karim directly: contact@llmreach.ai