llms.txt: What It Is, Whether It Works, and How to Create Yours

llms.txt is a proposed standard: a plain markdown file at your website's root (yoursite.com/llms.txt) that gives large language models a concise, machine-friendly summary of who you are and links to your most important content. Think of it as a curated reading list for AI tools — related to robots.txt and sitemap.xml in spirit, but designed for LLMs that need clean context instead of crawl rules.

We generate a starter llms.txt file in every free AI visibility scan we run, and we publish one on our own site. So to be clear up front: we think it's worth twenty minutes. But this guide will also be honest about the debate — including the fact that major AI companies haven't formally committed to reading it.

What problem does llms.txt solve?

Websites are built for browsers, not for language models. A typical business homepage is a few kilobytes of actual information wrapped in hundreds of kilobytes of HTML, navigation, scripts, and styling. When an AI tool with a limited context window tries to understand your site, it burns most of its budget parsing that wrapper — and modern JavaScript-heavy pages often hide the substance entirely.

The llms.txt proposal (introduced by Jeremy Howard of Answer.AI in September 2024) addresses this with a simple convention: one markdown file, at a predictable location, containing your site's essentials in plain text. Human-readable, trivially parseable, no rendering required.

llms.txt vs robots.txt vs sitemap.xml

The file is easiest to understand by contrast with the two standards it resembles:

robots.txt says "you may enter." sitemap.xml says "here's every room." llms.txt says "here's who lives here and which rooms are worth your time." It grants no permissions and blocks nothing — if you want to control AI crawler access, that still happens in robots.txt.

What does an llms.txt file look like?

The specification is deliberately minimal. An H1 with the site name, a blockquote summary, then sections of links with one-line descriptions:

# Coastal Roofing Co.

> Family-owned roofing contractor serving Sarasota and Bradenton, Florida
> since 2004. Licensed (CCC1330000), insured, specializing in tile and
> metal roof replacement. Free inspections, financing available.

## Services

- [Roof Replacement](https://coastalroofing.com/replacement): Tile, metal,
  and shingle replacement; typical Sarasota cost ranges and timelines
- [Storm Damage Repair](https://coastalroofing.com/storm): Insurance claim
  documentation and emergency tarping

## Company

- [Reviews](https://coastalroofing.com/reviews): 340+ verified customer reviews
- [Service Area & Contact](https://coastalroofing.com/contact): Phone,
  hours, and neighborhoods served

## Optional

- [Blog](https://coastalroofing.com/blog): Maintenance guides and cost articles

The ## Optional section has a defined meaning in the spec: content an AI can skip when its context budget is tight. Some sites also publish expanded companions (llms-full.txt) with entire documentation inlined — that pattern comes from developer-tool sites like Anthropic's docs, and matters less for a typical business.

Does llms.txt actually work? The honest answer

Here's the part most articles skip. As of mid-2026:

• No major AI company has formally adopted the standard. Google's John Mueller has publicly compared it to the old keywords meta tag, and Google says its systems don't use it. OpenAI, Anthropic, and Perplexity crawl llms.txt files with varying enthusiasm but haven't committed to privileging them.
• Server logs (ours included) show AI crawlers do fetch it. Requesting a file isn't the same as weighting it, but the file is demonstrably being read by GPTBot, ClaudeBot, and PerplexityBot among others.
• Adoption is real on the supply side. Thousands of sites — including Anthropic, Stripe, Cloudflare, and Zapier — publish one. Proposed standards become standards through exactly this dynamic.

So our position, based on running AI visibility tests daily: llms.txt is a low-cost, no-downside hedge — not a ranking lever. It will not, by itself, make ChatGPT recommend your business. The signals that actually move recommendations are structured answer-first content, consistent entity data, and third-party corroboration. But when an AI agent does land on your domain looking for clean context — and agentic browsing keeps growing — the file guarantees the summary it finds is the one you wrote. Twenty minutes, once, with quarterly touch-ups.

Anyone selling llms.txt as an "AI SEO secret weapon" is overclaiming. Anyone refusing to spend twenty minutes on it is being stubborn about a free option.

How to create your llms.txt file

1. Write the summary blockquote first. Two or three sentences: what you do, where, for whom, and one or two proof points (license, years, review count). This is the highest-value text in the file — assume it's the only part that gets read.
2. List 5–15 pages, not everything. Your services, pricing/cost pages, reviews, contact, and about. Curation is the point; a dump of every URL recreates the problem the file exists to solve. One-line descriptions should state what a page contains, not slogans.
3. Keep it in sync. Stale llms.txt (old prices, dead links, discontinued services) is worse than none — you'd be feeding AI tools wrong facts in their most digestible format. Calendar a quarterly review.
4. Upload to your root directory so it resolves at yoursite.com/llms.txt, alongside robots.txt and sitemap.xml. Plain text file, UTF-8, no HTML.
5. Don't contradict your site. The file supplements your pages; AI systems cross-check. Numbers, service names, and locations must match what's published elsewhere — consistency is the whole game in AEO.

If you'd rather not start from a blank page: our free llms.txt generator drafts the file from your homepage in seconds, and the free AI visibility scan generates a starter llms.txt alongside the actual evidence of what AI currently says about you.

Common llms.txt mistakes

• Blocking AI crawlers in robots.txt while publishing llms.txt. The file can't help crawlers you've banned. Decide your AI access policy first; make the two files agree.
• Marketing copy instead of facts. "We're passionate about excellence" gives a language model nothing. Services, locations, prices, credentials do.
• Linking to JavaScript-dependent pages. If the linked page renders empty without a browser, the link wastes an AI's fetch. Server-rendered text wins.
• Set-and-forget. The file is a standing claim about your business. Treat it like your Google Business Profile: reviewed, current, accurate.

Frequently asked questions

Is llms.txt an official standard?

No — it's a proposed standard from September 2024 with meaningful voluntary adoption but no formal endorsement from Google, OpenAI, or Anthropic. It may become de facto standard through usage, the way sitemaps did before formal adoption. Publishing one is a hedge, not a compliance requirement.

Does llms.txt help SEO?

Not directly — Google states its search systems don't use it, so expect no ranking effect. Its value sits in the AI layer: giving LLM-based tools and agents clean, curated context about your site. The broader work of answer engine optimization is what affects visibility in AI answers.

Where does llms.txt go?

In your website's root directory, so it's reachable at https://yoursite.com/llms.txt — same location pattern as robots.txt and sitemap.xml. It's a plain markdown text file; no registration, tag, or search-console submission involved.

What's the difference between llms.txt and llms-full.txt?

llms.txt is the curated index: summary plus links with descriptions. llms-full.txt is an optional companion that inlines full content (common for developer documentation) so an AI can ingest everything in one fetch. Most local businesses only need llms.txt.

Should I block AI crawlers instead?

That's a business decision, made in robots.txt rather than llms.txt. For local businesses that want customers, blocking AI crawlers is usually self-defeating — invisible businesses don't get recommended. Check what AI currently says about you before deciding what to feed or starve it.