llms.txt is a proposed plain-text (Markdown) file, placed at yoursite.com/llms.txt, that gives AI assistants like ChatGPT, Claude, and Perplexity a short, curated summary of your site and links to the pages that matter most. Think of it as sitting alongside robots.txt and sitemap.xml, but solving a different problem: robots.txt controls whether crawlers can access your content, and a sitemap lists every URL - llms.txt tells an AI system which pages are actually worth reading, in a format built for a language model rather than a search engine index.
Why It Exists
Large language models work with a limited context window and often can't (or won't) crawl an entire website the way a search engine does. A long, JavaScript-heavy homepage full of navigation, ads, and marketing copy is expensive and unreliable for an AI system to parse for the handful of facts it actually needs. llms.txt gives it a shortcut: a clean, curated, plain-Markdown index of your most important pages - what they're about and where to find them - without needing to render and parse the full site first.
The Basic Format
A minimal llms.txt looks like this:
# Acme Co
> Acme Co makes widget-fitting software for small manufacturers.
## Docs
- [Getting Started](https://acme.co/docs/getting-started): Setup and first steps
- [API Reference](https://acme.co/docs/api): Full endpoint documentation
## Optional
- [Pricing](https://acme.co/pricing): Plans and pricing tiers
- An H1 with your site or product name.
- A blockquote summary - one or two sentences on what the site/business actually does, giving an AI system fast context before it reads anything else.
- One or more
##sections grouping links by topic (Docs, Guides, Products - whatever fits your site), each link with a short description of what's on that page. - An
## Optionalsection for genuinely secondary content - a signal that these pages can be skipped first if the AI system is working with a limited budget for how much it reads.
What to Actually Include
- Your core product/service pages and what problem each solves.
- Key documentation, guides, or FAQ content - anything that answers questions people (or AI assistants, on their behalf) are likely to ask.
- Skip marketing fluff, legal boilerplate, and anything duplicated elsewhere - the whole point is a curated list, not a full sitemap restated in Markdown.
Common Mistakes
- Publishing an empty or placeholder file just to have one - a lower-value signal than not having one at all if it's clearly out of date or contentless.
- Listing pages blocked by robots.txt. If a page is disallowed for crawlers, pointing an AI system at it in llms.txt is a contradictory signal - see our robots.txt guide and AI crawler accessibility checklist to make sure your crawler rules and llms.txt agree with each other.
- Treating it as a replacement for good page content. llms.txt points an AI system at your best pages faster; it doesn't fix pages that are thin, unclear, or missing the information an AI assistant (or a person) is actually looking for.
The Fast Way to Check Yours
Writing a valid llms.txt by hand is easy to get subtly wrong - a missing blockquote, broken links, or sections in the wrong format. AuditCrow's free llms.txt checker validates yours instantly and flags what's missing, alongside the rest of the AI-readiness checks covered in does ChatGPT recommend your business.