What is llms.txt and do you need one in 2026?

Updated September 2026 · 5 min read

llms.txt is a plain Markdown file at the root of a website, yoursite.com/llms.txt, that describes the site and links to its most useful pages in a form written for language models rather than browsers. Jeremy Howard of Answer.AI proposed it in September 2024. You do not need one to appear in ChatGPT, Claude, Perplexity or Google: none of them has said it uses the file to decide what to cite, Google has said outright that Search ignores it, and an Ahrefs study of 137,000 domains in May 2026 found that 97 percent of published llms.txt files received no requests at all. It is useful in one situation, when an agent is already on your site and needs a map, and it takes about an hour, so publish one if you like. Just do not expect it to change your visibility.

What the file is

The specification at llmstxt.org asks for a Markdown file with a fixed shape:

1. An H1 with the name of the site or project. This is the only required line.
2. A blockquote with a one-paragraph summary.
3. Any number of paragraphs of context.
4. H2 sections, each a list of links in the form - [Title](url): one-line description.
5. An optional section called Optional for links an agent can skip when it is short on context.

The idea is that a model reading a site should not have to parse navigation, cookie banners and scripts to learn what the site is. It reads one clean file instead. A companion convention, llms-full.txt, puts the entire documentation of a project into one file for the same reason.

Who reads it

Documentation tools took it up first, because the use case is clearest there. Mintlify, Anthropic's own documentation, Cloudflare's developer docs and a long list of developer products publish one. AI coding assistants that fetch documentation can save tokens by reading it instead of the rendered site.

Outside developer documentation, the picture is thin. Google's John Mueller and Gary Illyes have both said Google Search does not use it and is not planning to, comparing it to the keywords meta tag. Google's 2026 guidance on generative AI features lists llms.txt among the things you do not need to appear in AI Overviews or AI Mode. OpenAI, Anthropic and Perplexity have published documentation on their crawlers and robots.txt, and none of it mentions llms.txt.

The Ahrefs study is the most direct evidence: across 137,000 domains, 28 percent had published the file, and 97 percent of those files logged zero requests in the month studied.

What it will not do

• It will not get you cited. Citations come from the crawlers reading your actual pages and the models judging them relevant.
• It will not fix a blocked crawler or a page whose text only exists after JavaScript runs. Those problems are covered in how to make your website readable by AI crawlers.
• It will not affect Google rankings in either direction.

When it is worth an hour

• You have documentation or a help centre. Coding agents and support bots fetch documentation directly, and a clean map helps them land on the right page.
• Your name is ambiguous. A file that states plainly what you are, and what you are not, is one more place an agent can find the disambiguation. Our own llms.txt at getclerion.com opens with the fact that Clerion is not Celerion or Clarion, because assistants kept mixing them up.
• You already maintain a summary of your product for other purposes. The pricing page, feature list and FAQ can be pasted into the format in an afternoon.

How to write one

Keep it factual, current and short. Write it the way you would brief a new colleague.

# Acme Analytics

> Acme Analytics is privacy-first web analytics for small teams. No cookies, no consent banner, one script tag. Plans from $9 a month.

Acme is spelled A-C-M-E and lives at acme.example. It is not Acme Corp, the industrial supplier.

## Product
- [Features](https://acme.example/features): everything the product does, one line each
- [Pricing](https://acme.example/pricing): plans, limits and what counts as a pageview
- [Get started](https://acme.example/get-started): the script tag and platform guides

## Compare
- [Acme vs Google Analytics](https://acme.example/vs/google-analytics): what changes when you switch

## Optional
- [Changelog](https://acme.example/changelog): release notes

Three rules that keep it useful. Every fact in the file must match the page it links to, so update prices in both places. Put the disambiguation and the pricing near the top, because those are the two facts assistants most often get wrong. Do not stuff it with keywords; a model reading a summary written for a model can tell.

How to check yours

Open yoursite.com/llms.txt in a browser. It should return the file as plain text, not a redirect to your homepage or a 404 page dressed as 200. The free AI crawler checker includes llms.txt as one of its seven checks, alongside the ones that matter more: robots.txt rules for the assistant crawlers and how much of your text is readable without JavaScript.

Frequently asked questions

Does llms.txt help SEO?

No. Google has said Search ignores it, and no other search or answer engine has said it uses the file for ranking or citation decisions. It neither helps nor hurts.

What is the difference between llms.txt and robots.txt?

robots.txt tells crawlers which pages they may fetch, and every major crawler reads it. llms.txt describes the site for a model that has already arrived, and few systems read it. One is access control that works today; the other is a courtesy summary.

Should llms.txt include my whole documentation?

That is what llms-full.txt is for. Keep llms.txt as the short map with links, and put the complete text in the full file if agents fetching your documentation is a real use case for you.

Is there a generator?

Several exist, and most documentation platforms can emit the file automatically. The generated result is only as good as the pages it links to. A hand-written one page long is usually better than a generated one that lists 400 URLs.