llms.txt: What It Is and Whether Your Site Needs One
llms.txt is a proposed Markdown file that gives AI agents an overview of your site. Here is its status, who uses it, and how to create one.
llms.txt is a Markdown text file placed at the root of a website that gives language models and AI agents a short, curated overview of the site's most important content. The file is a proposal β not an adopted web standard β and it pays to separate what it was designed for from what it is actually used for.
The short version: llms.txt is cheap to create and can be useful for AI agents that fetch content while working on a task. But Google does not use it in Search, and none of the major AI search services has documented that the file affects who gets cited. The status in this article is as of October 2026.
What is llms.txt?
The proposal was published in September 2024 by Jeremy Howard at llmstxt.org. The reasoning is practical: a language model has a limited context window, and a typical web page is full of navigation, scripts and layout that make it hard to extract what matters. llms.txt is meant as a shortcut β one place where the site itself points to the content worth reading, ideally in a clean Markdown version.
The specification was updated to version 2 in August 2026. It now describes usage like this: an agent views or searches llms.txt to find what it needs, then follows the relevant links. The file is aimed at the moment an AI answers a specific question β not at model training.
llms.txt, robots.txt and sitemap.xml
The three files do different jobs and do not replace each other:
- robots.txt controls which crawlers may fetch what.
- sitemap.xml lists every indexable page.
- llms.txt is a curated selection with short descriptions, sized for a language model's context window.
A common misunderstanding is that llms.txt can be used to block AI crawlers. It cannot. Access is still controlled in robots.txt.
How an llms.txt file is structured
The format is plain Markdown in a fixed order:
- An H1 with the name of the site or project. This is the only required element.
- A blockquote with a short summary.
- Optional sections with more detailed information.
- One or more H2 sections containing lists of links. Each item is a Markdown link, optionally followed by a colon and a short note.
A section named Optional is used, by convention, for secondary links an agent can skip when it needs a shorter context.
# Example Ltd
> Example Ltd builds booking software for clinics. The site contains product descriptions, pricing, documentation and guides.
Important notes:
- Prices are listed excluding VAT
- The documentation covers version 4 and later
## Product
- [Features](https://www.example.com/features.md): Overview of all features
- [Pricing](https://www.example.com/pricing.md): Plans and terms
## Documentation
- [Getting started](https://www.example.com/docs/getting-started.md): Step-by-step setup
- [API reference](https://www.example.com/docs/api.md): Endpoints and authentication
## Optional
- [About the company](https://www.example.com/about.md): History and team
Markdown versions of your pages
The proposal also recommends that pages with useful content are available in a clean Markdown version β either by appending .md to the URL or by replacing the file extension. Version 2 additionally describes how a page can point to its Markdown version with rel="alternate" type="text/markdown", as a link element in the HTML or as an HTTP header.
You may also come across a file called llms-full.txt, where the entire content is collected in a single file. It is not part of the specification at llmstxt.org but a convention used by some documentation platforms.
Status: who actually supports llms.txt?
It is necessary to separate three things here: who publishes the file, who generates it, and who reads it.
Who publishes and generates it
According to llmstxt.org, thousands of sites publish the file and several platforms generate it automatically. The specification mentions Mintlify, GitBook and Wix, as well as WordPress plugins such as Yoast SEO and AIOSEO. It also notes that OpenAI, Anthropic and Google's Gemini team publish llms.txt files for their developer documentation.
The pattern is worth noticing: adoption is strongest on documentation and developer sites, where AI coding assistants and agents actually look things up.
Google does not use it
Google's position is documented and unambiguous. Google's guide to optimising for generative AI features in Search (last updated in July 2026) states that you do not need to create new machine readable files, AI text files, markup or Markdown to appear in Google Search. The documentation on AI features says the same about AI Overviews and AI Mode: there are no additional requirements beyond being indexed and eligible to be shown with a snippet.
The fact that a Google team publishes an llms.txt for its API documentation does not change that. It is a service for developers and agents, not a search signal. If you want to optimise for Google's AI answers, the ordinary SEO fundamentals apply β we cover them in our guide to Google AI Overviews.
OpenAI and other AI search services
OpenAI's FAQ for publishers describes robots.txt and noindex as the mechanisms that control whether a site can appear in ChatGPT search. llms.txt is not mentioned. We cover what actually controls visibility in our guide to ChatGPT SEO. Nor have we found documentation from Microsoft, Perplexity or Anthropic stating that the file is used as a signal for ranking or citation.
That does not mean nobody reads the file. Agents and coding assistants can fetch it when they work with a site, and that is exactly the use version 2 of the specification describes. But claims that llms.txt gives you "better visibility in ChatGPT" are not backed by any of the providers' own documentation.
llms.txt and structured data
The two are often mentioned together, but they work at different levels:
- Scope: llms.txt is one document for the whole site. Structured data is markup on each individual page.
- Content: llms.txt is a curated list of links with descriptions. Structured data is facts about entities β who, what and how they relate.
- Status: llms.txt is a proposal. Structured data builds on an established vocabulary, schema.org.
- Google: llms.txt is not used by Google. Structured data is used for rich results, among other things.
llms.txt says where the important content lives. Structured data says what the content is and who is behind it. The first is a table of contents; the second is facts in a format machines can read without guessing.
For most business websites the order is therefore clear: get correct, consistent schema markup in place first, then consider llms.txt as a supplement. We go deeper into how AI agents use the markup in structured data for AI agents.
How to create an llms.txt file
- Choose the content. Pick the pages that best explain what you do: products or services, pricing, documentation, terms and contact. It is a selection, not a copy of your sitemap.
- Write the file in Markdown. Start with the H1 and a blockquote containing a precise description. Group links under H2 headings and give each link a short note.
- Put the file at the root. It should be available at
https://yourdomain.com/llms.txt. Version 2 clarifies that a file covers the pages under its path, so a file can also live in a subdirectory. - Point to clean content. Links should preferably lead to Markdown versions or pages without heavy navigation. If you have no Markdown versions, link to the regular pages.
- Test. Open the URL in a browser, check that the file is shown as plain text and that every link works.
- Maintain it. An llms.txt with dead links or outdated descriptions is worse than none.
On WordPress you can let a plugin generate the file. According to llmstxt.org, both Yoast SEO and AIOSEO include a generator. Review the result before you rely on it: an automatic list of every post is not a curated overview.
Common mistakes
- Assuming the file blocks anything. It provides no access control.
- Pasting in the whole sitemap. The point is selection.
- Describing something other than what is on the pages. Descriptions must match the content they point to.
- Expecting an effect in Google. There is none.
Frequently asked questions
Is llms.txt an official standard?
No. llms.txt is a proposal from 2024 that is maintained at llmstxt.org and was updated to version 2 in August 2026. It has not been adopted by a standards body, and no search engine requires it.
Does Google use llms.txt?
No. Google's documentation states that you do not need to create new machine readable files, AI text files or Markdown to appear in Google Search, including AI Overviews and AI Mode. The file neither helps nor hurts in Google.
Does llms.txt improve visibility in ChatGPT?
That is not documented. OpenAI's guidance for publishers covers robots.txt and access for OAI-SearchBot, but not llms.txt. The file can be useful for agents fetching content from your site, but it is not a confirmed factor for citations.
Can I block AI crawlers with llms.txt?
No. llms.txt is an overview, not an access control. If you want to decide which AI crawlers may fetch your content, do it in robots.txt using the relevant user agents.
What is the difference between llms.txt and structured data?
llms.txt is a single file with a curated list of the site's most important pages. Structured data is machine-readable facts on each individual page, written with the schema.org vocabulary. Structured data is used by Google and Bing; llms.txt is not.
Does my website need an llms.txt file?
If you have documentation, an API or a product that developers and AI agents look things up in, it makes good sense. For an ordinary business website it is a low-priority supplement: it costs little to create, but should not come before crawlability, content and structured data.
Summary
llms.txt is a Markdown file at the root of your site that gives language models a curated overview of your most important content. It is still a proposal, widely adopted on documentation sites in particular and supported by several CMS platforms and plugins. Google does not use it, and none of the major AI search services has documented that it affects citations. By all means create one β but as a supplement to a crawlable site with correct, consistent structured data, not as a replacement.