The llms.txt file is a Markdown document placed at the root of your site that helps large language models (ChatGPT, Claude, Gemini, Perplexity) understand your content. It isn't a blocking tool like robots.txt. It's a reading guide for AI. Introduced by Jeremy Howard (Answer.AI) in September 2024, it's already used by Anthropic, Stripe, Cloudflare, Perplexity, Cursor, and Vercel. With Gartner predicting a 25% drop in traditional search volume by 2026, making your site readable by AI is no longer optional. This guide covers the format, installation, the honest limitations, and how to measure the impact for your startup.
What is llms.txt?
Definition: a file for AI, not for Google
It's a Markdown text file placed at yoursite.com/llms.txt that tells language models what your site does, which pages matter, and how they're organized. Think of it as a README for AI.
Google doesn't use llms.txt (it has its own indexing crawlers). The intended "readers" are conversational AI: OpenAI's ChatGPT, Anthropic's Claude, Google's Gemini, Perplexity, and coding assistants like Cursor and GitHub Copilot. When a user asks one of these tools a question, the AI can consult your llms.txt to quickly understand what you offer, without parsing your entire site.
llms.txt vs robots.txt vs sitemap.xml
Three files, three jobs:
- robots.txt: tells Google's crawlers which pages not to crawl. It's a restriction file.
- sitemap.xml: tells Google's crawlers which pages exist and how they're structured. It's a site map.
- llms.txt: tells AI what your site does and which pages are most useful for answering questions. It's a reading guide.
A common mix-up: assuming llms.txt exists to "block AI" the way robots.txt blocks crawlers. It doesn't. llms.txt is a tool for openness, not restriction. You're not blocking AI, you're guiding it toward your best content.
Who already uses it?
The early adopters are tech companies that build or use AI every day:
- Anthropic (docs.anthropic.com/llms.txt): the creator of Claude, so it makes sense they'd adopt the format.
- Stripe: goes further with an "instructions" section in its llms.txt that guides AI on how to use its API documentation.
- Cloudflare: llms.txt for its developer documentation.
- Perplexity: the search AI uses an llms.txt itself.
- Cursor and Vercel: developer tools that support the format.
The common thread: these are companies whose documentation is critical. The better AI understands their content, the more relevant answers their users find.
Stripe is a particularly interesting case. Their llms.txt doesn't just list pages. It includes an "instructions" section that explicitly tells AI how to interpret their documentation: which pages to favor for integration questions, which pages for pricing questions. It's a glimpse of what the format could become: no longer a simple index, but a genuine communication protocol between your site and AI.
Why your startup should have an llms.txt
GEO (Generative Engine Optimization) in 2026
SEO optimizes your site to appear in Google's results. GEO (Generative Engine Optimization) optimizes your content to be cited by AI.
The difference is fundamental. With SEO, you're aiming for a blue link on a results page. With GEO, you're aiming for a citation in an answer generated by ChatGPT, Perplexity, or Gemini. Traffic no longer comes from a click on a link, it comes from a mention in an AI answer that the user reads directly.
Gartner predicts a 25% drop in traditional search volume by 2026 in favor of chatbots and AI agents. For an early-stage startup, that means your growth strategy can no longer be limited to classic SEO. GEO is becoming an acquisition channel in its own right.
Getting cited by ChatGPT, Perplexity, and Gemini
When a user asks Perplexity "Which billing tool for a small business in France?", the AI compiles sources, generates an answer, and cites its references. If your site has a clear llms.txt explaining that you're a billing tool for small French businesses, you improve your chances of appearing in that answer.
The key phrase is "improve your chances." Not "guarantee." More on that in the limitations section.
In practice, search AI like Perplexity works in three steps: it receives a question, crawls relevant pages, then synthesizes an answer with citations. Your llms.txt comes in at the second step: it helps the AI quickly identify which pages of your site are relevant to the question. Without llms.txt, the AI has to guess by navigating your site. With it, the AI has a map.
llms.txt as an early-stage competitive edge
In 2026, most sites still don't have an llms.txt file. That's an advantage for startups that move fast: while your competitors haven't yet grasped what GEO is, you're already structuring your content for AI. The implementation cost is near zero (a Markdown file, 30 minutes of work). The visibility upside is disproportionate to the effort.
Think back to robots.txt adoption in the 2000s. The first sites to implement it correctly gained an SEO edge that latecomers took years to catch up on. llms.txt may be to GEO what robots.txt was to SEO: an early signal of attention to discoverability.
Structure and file format
The Markdown syntax explained
An llms.txt file follows a simple structure:
# Your Startup Name
> Short description of what your product/service does (1-2 sentences).
Additional context if needed.
## Documentation
- [Getting started guide](https://yoursite.com/docs/guide): How to get started
- [API Reference](https://yoursite.com/docs/api): Complete API documentation
## Blog
- [Key article 1](https://yoursite.com/blog/article-1): Main topic
- [Key article 2](https://yoursite.com/blog/article-2): Another topic
## Optional
- [Legal notice](https://yoursite.com/legal): Legal information
The H1 title is the only mandatory element. The blockquote (>) gives the essential context. The H2 sections organize links by category. The "Optional" section signals to AI that those resources are secondary.
llms.txt vs llms-full.txt: which one to use?
llms.txt is an index: it points to your important pages with short descriptions. Lightweight and fast for an AI to read.
llms-full.txt is a complete dump: it contains the full text of all your pages in a single file. Heavier, but it lets the AI read everything without navigating.
For a startup, start with llms.txt. It's simpler to maintain and sufficient in 90% of cases. Move to llms-full.txt if your documentation is dense and technical (API, SDK, integrations) and you want AI tools like Cursor or Copilot to understand your stack in depth.
A concrete example for a startup site
# Factolib
> Online billing software for small businesses and freelancers in France. Compliant with 2026 French tax regulations.
## Product
- [Features](https://factolib.fr/fonctionnalites): Invoicing, quotes, payment tracking
- [Pricing](https://factolib.fr/tarifs): Free plan + paid plans
## Resources
- [Getting started guide](https://factolib.fr/guide): Create your first invoice in 5 min
- [Blog](https://factolib.fr/blog): Billing and small-business management tips
## Optional
- [Terms of sale](https://factolib.fr/cgv): General terms and conditions
Installing llms.txt on your site
Manual method (any CMS)
- Create an
llms.txtfile in a text editor (VS Code, Notepad, TextEdit) - Write the content following the Markdown structure above
- Upload the file to the root of your site (the same folder as index.html or robots.txt)
- Verify access by visiting
yoursite.com/llms.txtin your browser
That's it. No plugin, no server configuration, no code. A text file at the root. The simplicity is deliberate: Jeremy Howard designed the format to be as easy to create as a README.md on GitHub. If you can write Markdown (and if you're reading this article, you probably can), you can create an llms.txt.
WordPress: AIOSEO and Yoast plugins
On WordPress, you have two options:
- AIOSEO: builds llms.txt management into its SEO settings. Automatic generation based on your pages and posts.
- Website LLMs plugin: a dedicated plugin that automatically creates and updates your llms.txt.
With both, the file is updated automatically whenever you publish new content.
Astro / Next.js / modern frameworks
For startups on modern frameworks (and there are many of you):
- Astro: create a
public/llms.txtfile and it will be served automatically at the root. - Next.js: same thing in the
public/folder. Or use an API route to generate the file dynamically. - Nuxt: a file in
static/llms.txt.
For frameworks that build static sites, consider adding a step to your CI/CD pipeline that generates the llms.txt automatically from your content. A script that lists your published pages and formats them in Markdown takes an hour to write, and you'll never have to update the file manually again.
Does it actually work?
What the data says
Let's be honest. The question deserves a straight answer.
The case for it: major tech companies (Anthropic, Stripe, Cloudflare) are adopting it. AI development tools (Cursor, Copilot) actively use it to understand codebases and generate relevant code. The number of sites with an llms.txt is growing steadily. And above all, the ecosystem of autonomous AI agents (which navigate the web to complete tasks) is expanding rapidly. These agents benefit directly from a well-structured llms.txt.
The case against it: a 2025 study by SE Ranking found no measurable impact of llms.txt on AI visibility in conversational answers. Journal du Net calls its usefulness "still uncertain." And a February 2026 article by Kai Spriestersbach goes so far as to declare "llms.txt is dead." Skeptics argue that modern AI is perfectly capable of parsing a full website without needing a guide file.
The honest limitations
Three limitations to be aware of:
-
OpenAI and Google haven't officially supported the format. GPTBot and Googlebot don't specifically look for your llms.txt (for now). Adoption is driven mainly by development tools and search AI like Perplexity.
-
Correlation isn't causation. Having an llms.txt doesn't guarantee you'll be cited by AI. The quality of your content, your domain authority, and the relevance of your pages matter far more.
-
The standard isn't stabilized yet. It's a proposal, not a W3C norm. The format could evolve or be replaced by something else.
Our verdict for startups
Install it. The cost is near zero (30 minutes of work), the risk is zero (a text file can't break your site), and the potential is real for the following use cases:
- Technical documentation: if your startup has an API or an SDK, llms.txt helps tools like Cursor and Copilot understand your stack. That's the most concrete use case today.
- Structured editorial content: if you publish quality content (guides, analyses), llms.txt helps search AI like Perplexity index you correctly.
- A signal of modernity: for investors and tech partners, having an llms.txt shows that you understand the AI ecosystem. That counts for something.
Worst case: you lose 30 minutes and the file does nothing. Best case: you get cited by AI before your competitors do. The effort-to-upside ratio is unbeatable.
How to measure the impact
Monitor AI bots in your server logs
The AI that crawls your site identifies itself through its User-Agent:
- GPTBot (OpenAI)
- ClaudeBot (Anthropic)
- PerplexityBot (Perplexity)
- Google-Extended (Gemini)
Look for these identifiers in your server logs. If you see requests to /llms.txt, the bots are consulting it. If the number of requests rises after you set it up, that's a positive signal.
For startups on simpler hosting (Vercel, Netlify), traffic analytics can show bot access through function logs or edge analytics.
Track AI citations (GEO tracking)
GEO tracking is still emerging, but a few methods exist:
- Search for your brand on Perplexity, ChatGPT, and Gemini: ask questions related to your field and check whether you appear in the answers.
- GEO tracking tools: platforms like Otterly.AI and SEMrush are starting to offer AI citation tracking. The market is young but moving fast.
- A rise in "direct" referral traffic: if you see an increase in direct traffic you can't explain through other channels, some of it may come from AI citations (users copy-paste the URL from an AI answer).
GEO tracking is still a hands-on craft in 2026. But that's also an opportunity: startups that set up these metrics now will have a data history their competitors won't have when the market matures.
FAQ
Does llms.txt improve Google rankings?
No, not directly. Google uses its own indexing crawlers and hasn't announced support for llms.txt. That said, llms.txt can improve your visibility in the answers of conversational AI (ChatGPT, Perplexity, Gemini), which is a traffic channel distinct from classic SEO.
Should I block AI with robots.txt or guide it with llms.txt?
These are two opposite choices. robots.txt blocks crawler access to certain pages. llms.txt guides AI toward your best pages. For a startup seeking visibility, the logical strategy is to guide, not block. Use robots.txt to protect sensitive pages (admin, private data) and llms.txt to showcase your public content.
How long does it take to install an llms.txt?
30 minutes for a simple site. Create the file, list your 5-10 most important pages, and upload it to the root. No plugin or complex configuration needed.
My site is on Shopify, is it possible?
Yes, but it's more constrained than on WordPress or a custom framework. On Shopify, you can't simply upload a file to the root. You have to create a custom page template in the theme editor (Liquid) that serves the content in the right Markdown format with the right Content-Type. Specific tutorials for this setup exist in the Shopify community. Alternative: use a subdomain (docs.yoursite.com) hosted on a service that allows direct uploads (Vercel, Netlify) and place your llms.txt there.
