What is llms.txt, and does your business actually need one?
llms.txt has been pitched as the new robots.txt for AI. The honest answer is more limited than the hype: it doesn't drive ChatGPT or Perplexity citations, but it does have one narrower, real use — here's what it actually does.
llms.txt is a plain-text file, usually Markdown, that sits at a website's root directory and lists a site's most important pages for AI systems to reference. It loosely resembles robots.txt, but it works as a routing suggestion, not an access-control rule — it can't block or restrict any crawler. No major LLM provider has committed to systematically crawling it as of 2026. Neither OpenAI, Google, Anthropic, nor Perplexity has publicly promised to crawl llms.txt the way Googlebot crawls sitemap.xml, and Google's own John Mueller has said directly that it isn't a ranking factor.
So if your goal is showing up more often in ChatGPT or Perplexity answers, llms.txt isn't where the evidence points. It does have a real, narrower value for a different audience: third-party developers building AI tools or agents on top of your content. A clean llms.txt helps those tools parse and cite your site more reliably. Adding one costs little and won't hurt you — just don't mistake it for a visibility strategy.
What llms.txt actually is
llms.txt is a community-proposed convention. Site owners write it as a plain-text file, typically Markdown, and place it at a site's root (yoursite.com/llms.txt). It usually opens with an H1 for the site name and a short blockquote summary. Below that sit H2 sections linking out to the pages the owner considers most important — documentation, key articles, pricing, and so on. The idea is to hand a language model a curated map of the site instead of making it crawl and infer structure from scratch.
It's worth being precise about what it isn't. robots.txt is an access-control file — it tells a crawler what it's allowed and not allowed to fetch. llms.txt does nothing like that. It carries no enforcement mechanism and no backing from any recognized standards body, not the W3C, not the IETF. Any AI system can read it, partially read it, or ignore it entirely.
Does llms.txt actually help you get cited by ChatGPT or Perplexity?
The honest answer, backed by the available evidence, is no — not in the way most of the marketing around it implies. A few specific findings worth knowing:
- No major provider systematically crawls it. As of 2026, OpenAI, Google, and Microsoft have made no public statement committing to regular llms.txt ingestion, unlike the well-established, decades-old convention around sitemap.xml.
- Google has said directly it isn't a ranking factor. Google's own guidance treats llms.txt as outside its indexing and ranking systems entirely — it has no bearing on traditional search rankings or AI Overviews.
- OpenAI, Anthropic and Perplexity's own crawler guidance doesn't mention it. Each of these companies publishes documentation about their crawlers and user agents. None of them list llms.txt as required, recommended, or used in citation decisions.
Suppose the goal is appearing more often when someone asks ChatGPT or Perplexity a question relevant to your business. The priorities that actually move that needle are the same ones covered elsewhere on this site: schema markup, answer-optimized content, and consistent entity information. A routing file that most AI systems aren't reading on any schedule won't do that job.
So what does llms.txt actually do?
It has one real, narrower use case: supporting AI products and agents built directly on top of your content. These are tools that use retrieval-augmented generation (RAG) to pull from your site programmatically, rather than a consumer chatbot answering a one-off question. For that specific audience, a clean, well-structured llms.txt can make your most important pages easier to find and parse. That's a genuinely different job than "getting cited by ChatGPT," and conflating the two is where most of the hype goes wrong.
Should you add one anyway?
If it takes fifteen minutes and creates no downside, there's a reasonable case for adding one. Just calibrate your expectations first. It won't move your citation rate in ChatGPT or Perplexity, and it won't affect your Google rankings or AI Overviews eligibility. Treat it as a low-priority, low-cost addition, not a substitute for schema markup, content depth, or entity consistency — the three things with actual evidence behind them.
How to build a basic llms.txt correctly
- Start with an H1 — your site or organization name.
- Add a short blockquote — one or two sentences summarizing what the business does.
- List your most important pages under H2 sections — documentation, key service pages, your best reference articles — as a simple Markdown link list.
- Keep it current. A stale llms.txt pointing at outdated or removed pages is worse than none at all.
- Place it at the root — yoursite.com/llms.txt, publicly accessible, no authentication required.
Frequently Asked Questions
What is llms.txt in simple terms?
A plain-text, usually Markdown file placed at a website's root directory that lists a site's most important pages for AI systems to reference. It's modeled loosely on robots.txt but works as a routing suggestion, not an access-control rule.
Does llms.txt help my business get cited by ChatGPT or Perplexity?
The evidence says no. No major AI provider has committed to systematically crawling llms.txt, Google has stated directly it isn't a ranking factor, and OpenAI, Anthropic and Perplexity's own crawler guidance doesn't reference it as part of citation decisions.
Is llms.txt the same thing as robots.txt?
No. robots.txt is an access-control file that tells crawlers what they're allowed to fetch. llms.txt has no enforcement mechanism at all — it can't block or restrict any crawler, and following it is entirely optional for any AI system.
Does llms.txt do anything useful at all?
Yes, for a narrower audience than most of the hype suggests: third-party developers building AI tools or agents that pull from your content via retrieval-augmented generation (RAG). For that specific use, a clean llms.txt can help those tools parse your site's key pages more reliably.
Should I still add an llms.txt file to my website?
There's no real downside to adding one if it takes minimal effort, but it shouldn't be a priority investment. Schema markup, answer-optimized content, and consistent entity information across your web presence are the three things with actual evidence behind them for AI citation.
What should a basic llms.txt file include?
An H1 with your organization's name, a short blockquote summarizing what you do, and H2 sections linking to your most important pages as a simple Markdown link list — kept current, and placed publicly at yoursite.com/llms.txt.
Sources & References
- Webyes: Does llms.txt Help? What John Mueller Says — Google's own position that llms.txt is not a ranking factor.
- Jacob Tyler: llms.txt and AI Search Visibility — The Honest Answer
- Peec AI: llms.txt & .md Files — Important AI Visibility Helper or Hoax? — the RAG/third-party-developer use case.
- We-Optimizz: Stop Adding llms.txt. ChatGPT Isn't Reading It.
- aeo.press: The State of llms.txt in 2026 — adoption status and lack of standards-body backing.
Wondering which AI-visibility investments actually move the needle?
Run the free AI Search Scorecard to check your real citation signals in five minutes — or book a visibility audit for a full breakdown.
Get the free scorecard → See the AI Visibility Audit