Jeremy Howard, cofounder of Answer.AI, proposed the llms.txt file on September 3, 2024 on llmstxt.org. It's a Markdown document you place at the root of your site to tell large language models (ChatGPT, Claude, Gemini, Perplexity) what you do and which pages to read first. robots.txt sets rules for what crawlers can't touch, while llms.txt hands them a reading guide. Anthropic, Stripe and Cloudflare publish one on their developer docs. Google, for its part, has stated that its search engine ignores it. This guide covers the definition, the format with an example, setup on WordPress, Webflow, Shopify and modern frameworks, and what the evidence actually shows in 2026.
What is the llms.txt file?
llms.txt defined
The llms.txt file is a Markdown text file served at votresite.com/llms.txt that introduces your site to language models: what you offer, your key pages and how they're organized. Jeremy Howard (Answer.AI) proposed it in September 2024 as a README for AI, written by you.
The llmstxt.org spec requires only one element: an H1 title with the name of the site or project. The intended readers are conversational AI tools like OpenAI's ChatGPT, Anthropic's Claude, Gemini and Perplexity, along with coding assistants like Cursor and GitHub Copilot. When one of these tools reads your llms.txt, it gets in a few lines what it would otherwise have to piece together by crawling your whole site.
llms.txt vs robots.txt vs sitemap.xml
Three files, three jobs:
- robots.txt: tells Google's crawlers which pages not to crawl. It's a restriction file.
- sitemap.xml: tells Google's crawlers which pages exist and how your site organizes them. It's a site map.
- llms.txt: tells AI what your site does and which pages are most useful for answering questions. It's a reading guide.
A common mix-up is thinking llms.txt exists to "block AI" the way robots.txt blocks crawlers. llms.txt opens the door: it points AI toward your best content. To keep GPTBot or ClaudeBot off a page, robots.txt is still the right tool.
Who already uses it?
The early adopters are tech companies whose documentation serves developers who code with AI assistants. As of September 2026, three public examples are still live:
- Anthropic publishes an llms.txt on its developer docs (docs.claude.com, formerly docs.anthropic.com).
- Stripe publishes one on docs.stripe.com, with a section titled "Instructions for Large Language Model Agents".
- Cloudflare publishes a root llms.txt on developers.cloudflare.com that links out to a separate llms.txt for each product.
Stripe's case is worth a closer look. Its file lists pages and also gives AI agents instructions on how to integrate Stripe correctly, such as checking the latest npm package version instead of relying on a memorized number. At that point the format becomes an instruction channel between an API provider and the AI tools writing code for its customers. Cloudflare illustrates another rule in the spec: an llms.txt can live at the root or in a subfolder, in which case it covers the pages under that path.
Why your startup should have an llms.txt
GEO (Generative Engine Optimization) in 2026
SEO optimizes your site to show up in Google results. GEO (Generative Engine Optimization) optimizes your content so AI tools cite it. Both disciplines rest on the same foundation, useful content and clean technical setup, which Amandine Bart's SEO guide for startups covers in detail.
The difference is fundamental. With SEO, you're aiming for a blue link on a results page. With GEO, you're aiming for a citation in an answer generated by ChatGPT, Perplexity or Gemini. Traffic no longer comes from a click on a link; it comes from a mention inside an AI answer the user reads directly.
In a February 2024 press release, Gartner predicted a 25% drop in traditional search volume by 2026 as users shift to chatbots and AI agents. For an early-stage startup, that means your growth strategy can't stop at classic SEO: you need to treat GEO as an acquisition channel in its own right.
Showing up in ChatGPT, Perplexity and Gemini answers
When a user asks Perplexity "What invoicing tool should a small business in France use?", the AI gathers sources, generates an answer and cites its references. A clear llms.txt that presents your product as an invoicing tool for small French businesses gives the AI a reliable summary of your offer, if it comes to read it.
Keep that "if it comes to read it" in mind: no major consumer AI provider has confirmed that it reads llms.txt when building answers. The section on limitations covers this in detail.
In practice, AI search tools like Perplexity take a question, browse relevant pages, then synthesize an answer with citations. An llms.txt can help during that second step by flagging which pages on your site are useful for the question. Without it, the AI crawls your site blind. With it, the AI has a map.
llms.txt as a low-cost bet at the early stage
Implementation cost is low: one Markdown file and half an hour of work for a simple site. For a startup with an API or SDK, the most concrete payoff comes from coding assistants, which read documentation at the moment a developer is integrating your product. For a marketing site, treat it as an add-on with no guaranteed effect, alongside the core work on your content.
File structure and format
The Markdown syntax explained
An llms.txt file follows a simple structure:
# Your Startup Name
> Short description of what your product/service does (1-2 sentences).
Additional context if needed.
## Documentation
- [Getting started guide](https://votresite.com/docs/guide): How to get started
- [API Reference](https://votresite.com/docs/api): Full API documentation
## Blog
- [Key article 1](https://votresite.com/blog/article-1): Main topic
- [Key article 2](https://votresite.com/blog/article-2): Another topic
## Optional
- [Legal notice](https://votresite.com/legal): Legal information
The H1 title is the only required element. The blockquote (>) gives the essential context. H2 sections group links by category. The "Optional" section tells AI tools these resources are secondary.
llms.txt vs llms-full.txt: which one should you use?
llms.txt is an index: it points to your important pages with short descriptions. It's lightweight and quick for an AI to read.
llms-full.txt is a full dump: it contains the complete text of all your pages in a single file. It's heavier, but lets the AI read everything without navigating.
For a startup, start with llms.txt. It's easier to maintain and enough in most cases. Move to llms-full.txt if your documentation is dense and technical (API, SDK, integrations) and you want AI tools like Cursor or Copilot to understand your stack in depth.
A real example for a startup site
# Factolib
> Online invoicing software for small businesses and freelancers in France. Compliant with 2026 French tax regulations.
## Product
- [Features](https://factolib.fr/fonctionnalites): Invoicing, quotes, payment tracking
- [Pricing](https://factolib.fr/tarifs): Free tier + paid plans
## Resources
- [Getting started guide](https://factolib.fr/guide): Create your first invoice in 5 min
- [Blog](https://factolib.fr/blog): Invoicing and small business management tips
## Optional
- [Terms of sale](https://factolib.fr/cgv): General terms and conditions
Setting up llms.txt on your site
Manual method (any CMS)
- Create an
llms.txtfile in a text editor (VS Code, Notepad, TextEdit) - Write the content following the Markdown structure above
- Upload the file to the root of your site (the same folder as index.html or robots.txt)
- Check that it's accessible by visiting
votresite.com/llms.txtin your browser
That's it: no plugin, no server configuration, no code. Just a text file at the root. Jeremy Howard wanted it this simple and designed the format to be as easy to create as a README.md on GitHub. If you can write Markdown (and if you're reading this article, you probably can), you can create an llms.txt.
WordPress: Yoast, Rank Math or a dedicated plugin
On WordPress, your SEO plugin probably already generates the file:
- Yoast SEO: in June 2025, Yoast launched a free built-in llms.txt generator. Turn it on under Yoast SEO, Settings, Site features, in the AI tools section, with the llms.txt option. Yoast builds the file from your pages and posts.
- Rank Math: the plugin offers an llms.txt setting in its general settings (Rank Math SEO, General Settings, Edit llms.txt), where you choose which content to include.
- Website LLMs.txt: a dedicated plugin available on wordpress.org that creates the file and regenerates it whenever you publish. It's a good fit if your SEO plugin doesn't handle llms.txt.
Turn on only one source at a time: two plugins that each generate an llms.txt will fight over the same URL. Once it's active, open votresite.com/llms.txt and review the list. An automatic generator pulls in every page you have, whereas a good llms.txt keeps a curated selection.
Webflow: upload it in site settings
Webflow now serves llms.txt natively. In Site settings, you upload your file in the dedicated section, and Webflow serves it at the root once you publish. The Webflow help center describes the process and which hosting plans support it.
Webflow doesn't write the content for you. You draft the file locally following the structure above, upload it, then republish the site. Every time you add an important page or a pillar article, replace the file and republish, or your llms.txt will describe an outdated version of your site.
Shopify: a default file you can customize
Shopify automatically generates an llms.txt for every store. Since May 29, 2026, according to Shopify's developer changelog, you can replace it with your own: under Online Store, Themes, Edit code, add an llms.txt.liquid template. Shopify then serves your version at /llms.txt in place of the default. Review the generated file before deciding whether it needs a rewrite.
Astro / Next.js / modern frameworks
On modern frameworks:
- Astro: create a
public/llms.txtfile, and Astro copies it to the site root at build time. - Next.js: same thing in the
public/folder. Or use an API route to generate the file dynamically. - Nuxt: put the file in
public/llms.txt(thestatic/folder on Nuxt 2).
For frameworks that build static sites, consider adding a step to your CI/CD pipeline that generates llms.txt automatically from your content. A script that lists your published pages and formats them as Markdown takes about an hour to write, and you'll never have to update the file by hand again.
Does it actually work?
The state of the evidence in 2026
The case for: Anthropic, Stripe and Cloudflare maintain their llms.txt files on their docs. Platforms are building it in: Yoast and Rank Math on WordPress, native upload on Webflow, a default generated file on Shopify. AI agents that browse the web to complete tasks, and coding assistants that read API documentation, get a short entry point from a well-built llms.txt.
The case against: a 2025 SE Ranking study found no measurable impact of llms.txt on visibility in conversational AI answers. French outlet Journal du Net describes its usefulness as "still uncertain". Kai Spriestersbach went as far as titling a Medium article "The llms.txt is dead". Skeptics add that today's AI tools can read an entire site without a guide file.
Google's official position
Google has weighed in several times. Gary Illyes explained that Google doesn't read llms.txt files and that standard SEO work is enough to show up in AI Overviews, according to Search Engine Land. John Mueller compared the file to the meta keywords tag, which search engines have ignored for years because every site can write whatever it wants there.
In 2026, Google made this position part of its official documentation. Its guide to optimizing for generative AI in Google Search states that Google Search doesn't use llms.txt, that you don't need one to appear in its results or generative AI features, and that such a file neither helps nor hurts your rankings. Google also notes you can keep it for other services that do use it.
The limitations
Three limitations to keep in mind:
-
No major engine has confirmed using the file. Google explicitly rules it out. OpenAI hasn't published any statement indicating that GPTBot or ChatGPT uses it to rank or cite sources.
-
Correlation isn't causation. An llms.txt guarantees you no AI citations. The quality of your content, your domain authority and the relevance of your pages matter far more.
-
The standard is still a proposal. Answer.AI publishes it on llmstxt.org, outside any standards body like the W3C. The format may change, or another format may replace it.
Our verdict for startups
Set it up if your tool does it in one click or if your startup publishes technical documentation, and expect no effect on Google. The cost is low (half an hour for a simple site) and the risk is zero, since Google confirms an llms.txt doesn't hurt your rankings. Two use cases justify the effort:
- Technical documentation: if your startup has an API or SDK, llms.txt helps assistants like Cursor and Copilot find the right pages in your docs. It's the most concrete use case today.
- Structured editorial content: if you publish guides and analysis, llms.txt gives the AI tools that read it a clean index of your reference pages.
Worst case: you lose half an hour and the file does nothing. Best case: the agents and assistants that read the format find your key pages before your competitors'.
How to measure the impact
Watch for AI bots in your server logs
AI crawlers identify themselves by User-Agent:
- GPTBot (OpenAI)
- ClaudeBot (Anthropic)
- PerplexityBot (Perplexity)
Google-Extended, often included in these lists, won't show up in your logs: Google designed it as a robots.txt token that allows or blocks the use of your pages for Gemini, and Googlebot is still the crawler that actually visits your site.
Search your server logs for these identifiers. If you see requests to /llms.txt, the bots are reading it. If the number of requests rises after you set it up, that's a positive signal.
For startups on simple hosting (Vercel, Netlify), traffic stats can surface bot hits through function logs or edge analytics.
Track AI citations (GEO tracking)
GEO tracking is still emerging, but a few methods exist:
- Search for your brand on Perplexity, ChatGPT and Gemini: ask questions related to your field and check whether you show up in the answers.
- GEO tracking tools: platforms like Otterly.AI and SEMrush are starting to offer AI citation tracking. The market is young but moving fast.
- Rising "direct" referral traffic: if you see an increase in direct traffic you can't explain through other channels, part of it may come from AI citations (users copy and paste the URL from an AI answer).
GEO tracking is still a hands-on affair in 2026. That's also an opportunity: startups that set up these metrics now will have historical data their competitors won't have once the market matures.
FAQ
What is the llms.txt file?
The llms.txt file is a Markdown text file served at votresite.com/llms.txt that summarizes for language models what your site does and which pages to read first. Jeremy Howard, cofounder of Answer.AI, proposed it on September 3, 2024 on llmstxt.org.
Where should I put the llms.txt file?
At the root of your domain, so it responds at votresite.com/llms.txt, in the same folder as robots.txt. The llmstxt.org spec also allows a file in a subfolder, for example /docs/llms.txt, which then covers the pages under that path. On Astro or Next.js, drop it in the public folder; on Webflow, upload it in site settings; on Shopify, create an llms.txt.liquid template.
llms.txt and SEO: what's the effect?
None on Google. Google states in its Search Central documentation that Google Search doesn't use llms.txt and that such a file neither helps nor hurts your rankings. The file can be useful to coding assistants and AI agents that read it, a channel separate from classic SEO.
Should I block AI with robots.txt or guide it with llms.txt?
They're two opposite choices. robots.txt blocks crawlers from certain pages. llms.txt guides AI toward your best pages. For a startup looking for visibility, the logical strategy is to guide rather than block. Use robots.txt to protect sensitive pages (admin, private data) and llms.txt to showcase your public content.
How long does it take to set up an llms.txt?
30 minutes for a simple site. Create the file, list your 5-10 most important pages, upload it to the root. On WordPress, Yoast SEO and Rank Math generate it from their settings in a few clicks.
My site is on Shopify. Can I do it?
Yes, and Shopify already does it for you: the platform generates a default llms.txt for every store. Since May 29, 2026, you can replace it with your own version by adding an llms.txt.liquid template to your theme code (Online Store, Themes, Edit code). Shopify then serves your file at /llms.txt.
