llms.txt: What It Is, Does It Work, and Do You Need One?

llms.txt is a proposed plain-text file at your site root that gives AI models a curated, Markdown summary of your most important pages. Does it get you cited by AI engines? There is no evidence yet that it does: when we checked 13,772 URLs that AI engines actually retrieved while answering real questions, not one was an llms.txt file.

This guide covers what the file is, what our data shows, who should still add one, how to create it in a few minutes, and what actually gets websites cited instead.

What Is llms.txt?

llms.txt is a convention proposed by Jeremy Howard in September 2024. It asks websites to publish a Markdown file at /llms.txt that summarises the site and links to its key pages, so a language model can find the right content without parsing complex HTML, navigation and scripts.

The reasoning is simple: models have limited context windows, and a clean, curated file is easier to use than a full website. Documentation platforms adopted the idea quickly, because developer docs map neatly onto a list of linked pages. Much of what you read about llms.txt comes from tools that generate or host it, which is worth keeping in mind: their guides describe how to adopt the file, not whether it changes anything.

What llms.txt Does and Does Not Do

llms.txt is a voluntary convention, not a standard that any major AI engine has committed to reading when it answers questions. It is easy to confuse with the files you already know, so here is the difference:

What llms.txt does not do: it does not control crawling, it does not block AI training, it does not get pages indexed, and it does not guarantee citations. If you want to control which AI crawlers can reach your site, that is a job for robots.txt; our guide to robots.txt optimization covers the rules.

For which AI crawlers to allow or block, see AI crawlers and robots.txt.

Does llms.txt Work? What Our Data Shows

This is where opinion stops and evidence starts. GetCited platform data covers 26,639 source retrievals of 13,772 distinct URLs, pulled for 17 brands' tracked questions between July 19 and October 18, 2026. These are the pages AI engines retrieved and cited while answering real prompts.

Of those 13,772 URLs, none was an llms.txt file. No .txt file of any kind appeared. For contrast, 127 of the retrieved URLs were PDFs, so AI engines will use unusual formats when the content is worth it. llms.txt simply was not part of what they retrieved.

We also checked the same 17 brands' live websites on October 19, 2026. Eleven serve an llms.txt file and six do not. Several Shopify stores serve near-identical files of about the same size, which suggests the file is increasingly an automatic template output rather than something a team wrote on purpose.

What the data can and cannot tell you: with 17 brands split 11 to 6, the sample is too small to show any difference in how often AI engines link to sites with and without the file, in either direction. Retrieval logs show what answers cite, not what crawlers fetch, so an engine could read the file and still never cite it. This is not proof that llms.txt is harmful. It is evidence that nobody has shown it helps.

Do You Need an llms.txt File?

The right answer depends on your site, not a universal rule. The file is cheap to add; the question is whether it deserves your time before other work.

Low cost, low proven benefit. Treat it as a five-minute task, not a strategy.

How to Create an llms.txt File

The format is plain Markdown:

1. An H1 with your site or product name.
2. A one-paragraph summary in a blockquote.
3. Sections (H2) with lists of links to your most important pages, each with a short description.

A copy-paste template (illustrative):

# Acme

> Acme is project management software for remote teams of 5 to 200 people.

## Product
- [Features](https://acme.example/features): what the product does
- [Pricing](https://acme.example/pricing): plans and what each includes

## Docs
- [Getting started](https://acme.example/docs/start): setup in 10 minutes
- [API reference](https://acme.example/docs/api): every endpoint

## Company
- [About](https://acme.example/about): who we are
- [Contact](https://acme.example/contact): sales and support

Host it at the root of your domain so it loads at yourdomain.com/llms.txt as plain text, and update it when you add or remove key pages. If you would rather not write it by hand, our free llms.txt generator builds one from your site's existing pages.

What Actually Gets Sites Cited by AI Engines

If llms.txt is not in the citation path, what is? The same dataset shows that the brand's own website accounted for just 4.4% of retrievals. Editorial and publisher sites made up 23.5%, user-generated content such as YouTube, Reddit and forums 11.0%, and competitors' sites 8.2%.

By page type, articles drove 16.2% of retrievals and listicles 11.4%, far ahead of homepages (6.7%), category pages (3.6%) and product pages (2.8%).

The engines also behave differently. Across 2,577 tracked AI answers for 15 brands (September 19 to October 18, 2026), brands were named in 29% of answers but their own site was linked as a source in only 9%. Google AI Mode and Perplexity, which search the web and show sources, linked the brand's site in about 26% of answers. Claude, DeepSeek and Meta AI, answering mostly from model memory, named brands about as often but almost never linked. No file format changes that mechanic.

What moves citations, based on this pattern:

• Crawler access: make sure AI crawlers can reach your pages and get server-rendered HTML. See our technical AI search optimization guide.
• Answer-first content: articles and comparison-style pages that answer the question directly. See AI search content optimization.
• Structured data: schema that describes your pages and organisation. See schema markup for AI engines.
• Third-party presence: editorial coverage, reviews, community discussion and listicles, where most retrieved sources live. See AI trust signals.

How to Test Whether llms.txt Helps Your Site

Generate your own evidence instead of relying on vendor claims:

1. Check your server logs for requests to /llms.txt and note which user agents fetch it. Our guide to log file analysis for SEO and AI crawlers shows how.
2. Record a baseline of AI mention and citation rates across the engines and prompts you track.
3. Publish the file and note the date. Change nothing else for the test period if you can.
4. Re-measure on the same prompts after several weeks, and compare per engine.

If nothing moves and no AI crawler requests the file, you have your answer for your site. Tracking per engine matters here; our guide on how to track AI citations covers the method.

Where llms.txt Fits in Your Priority List

llms.txt is not harmful and it costs little. But as of October 2026, it did not appear once in 13,772 URLs that AI engines retrieved, while articles, listicles, editorial coverage and community discussion made up most of what they cited. Put it below crawler access, answer-first content, structured data and third-party presence, and add it when those are in hand.

Frequently Asked Questions

What is llms.txt used for?

llms.txt is a proposed Markdown file at a site's root that gives AI models and agents a curated summary of the site and links to its key pages. It was designed to help language models use a site's content without parsing full HTML pages. No major AI engine has said it reads the file when generating answers.

Does llms.txt improve AI citations?

There is no evidence that it does. In GetCited platform data covering 13,772 URLs that AI engines retrieved for 17 brands between July and October 2026, none was an llms.txt file. A check of the same 17 websites found 11 with the file and 6 without, and the sample is too small to show any difference either way.

Is llms.txt the same as robots.txt?

No. robots.txt tells crawlers which parts of a site they may access, and crawlers that respect it follow those rules. llms.txt does not allow or block anything; it is a summary file that tools may choose to read. To control AI crawler access, use robots.txt.

Should I add an llms.txt file to my website?

Add one if you run developer documentation or a docs-heavy SaaS product, because it is cheap and agents are likely readers. For most other sites it is optional and low priority. Keep an auto-generated file if your platform already serves one, but do not expect it to change how often AI engines cite you.

How do I create an llms.txt file?

Write a Markdown file with your site name as an H1, a one-paragraph summary in a blockquote, and H2 sections listing your key pages as links with short descriptions. Save it as llms.txt at the root of your domain. A free generator can build it from your existing pages.

How can I tell if AI crawlers read my llms.txt?

Check your server or CDN logs for requests to /llms.txt and look at the user agents making them. Then compare your AI mention and citation rates before and after publishing the file, per engine, over several weeks.