Skip to content
crawlcoach.
Get my free report

llms.txt validator and generator

Check an llms.txt file against the llmstxt.org format, line by line. Test the file a site serves, paste one in, or draft a first version from your sitemap.

Check a sitePaste a fileDraft a file

Free, no sign-up. Reads /llms.txt at the site root, or the .txt address you enter, and opens up to 10 of its links.

What llms.txt is, and who reads it

llms.txt is a Markdown file at the root of a site, at /llms.txt. It gives AI tools a short summary of the site and a list of links to its key pages. Jeremy Howard proposed it at llmstxt.org in September 2024. It is a proposal, not an adopted web standard.

No major AI search engine has said it uses llms.txt to rank or cite pages. Here is where each one stands:

  • Google. Its guide to AI features says: "You don't need to create new machine readable files, AI text files, or markup to appear in these features." John Mueller of Google wrote in June 2025: "FWIW no AI system currently uses llms.txt." Chrome's Lighthouse does have an llms.txt audit. It only flags a server error when it fetches the file, and marks a missing file as not applicable.
  • OpenAI, Anthropic, and Perplexity. Their crawler pages explain how to control their bots with robots.txt. None says their crawlers read llms.txt on other sites. Each publishes an llms.txt for its own developer docs.

So publish one if it's cheap for you, and don't expect traffic from it. If you want AI search to find your pages, two things matter more: robots.txt must let the AI search crawlers in (see which AI crawlers to allow), and your pages must be in Google's index.

The format

The proposal sets this order. Only the H1 is required.

  1. One H1 with the name of the site or project.
  2. A blockquote with a short summary.
  3. Any other Markdown, such as paragraphs or lists, but no headings.
  4. H2 sections, each with a "file list": a Markdown list where each item has "a required markdown hyperlink [name](url), then optionally a : and notes about the file."

A section named Optional has a special meaning. It holds "links an agent can skip when a shorter context is needed." This is the proposal's own outline:

The llmstxt.org format
# Title

> Optional description goes here

Optional details go here

## Section name

- [Link title](https://link_url): Optional link details

## Optional

- [Link title](https://link_url)

A real example: CrawlCoach publishes its own at crawlcoach.com/llms.txt. It is built from the same data as the site's pages, so its prices and guide links stay in step with the site.

Best practices

  • Serve it at /llms.txt with HTTP 200, as text/plain or text/markdown. A site that answers every address with its app returns HTML instead. This validator catches that.
  • Use full addresses that start with https://. The file is often read on its own, away from your site.
  • Keep it short. List the pages that explain what you do, and put legal pages and older material under ## Optional.
  • Give each link a plain name and a short note that says what the page answers.
  • Build it from the same data as your site, so it doesn't go stale when pages move.

Questions

What is llms.txt?

A Markdown file at /llms.txt that gives AI tools a short summary of a site and links to its key pages. Jeremy Howard proposed it at llmstxt.org in September 2024. It is a proposal, not an adopted web standard.

Does Google use llms.txt?

Google Search doesn't. Google's guide to AI features says you don't need new machine readable files or AI text files to appear in AI Overviews or AI Mode. Google's John Mueller wrote in June 2025 that no AI system currently uses llms.txt.

Is llms.txt the same as robots.txt?

No. robots.txt tells crawlers what they may fetch, and the major AI crawlers follow it. llms.txt is a reading list. It can't allow or block any crawler, so keep your AI crawler rules in robots.txt.

What is llms-full.txt?

A file some documentation sites publish with the full text of their docs in one place. It is not part of the llmstxt.org proposal. This validator reports whether a site has one and how big it is, but checks only llms.txt.

Related guides

Sources

Checked on October 8, 2026. Vendors change their tools and docs, so check the source if something looks different.

See which of your pages Google left out

A page must be in Google's index to show up in AI Overviews. Sign in with Google, and CrawlCoach checks up to 500 pages for free and shows the fix for each one Google skipped.

Get my free report

Free for up to 500 pages. No card needed. Sign in with the Google account you use for Search Console.