What is llms.txt and should your website have one?

As more people ask AI assistants for advice, website owners want those assistants to understand their sites correctly. One idea that has spread quickly is a file called llms.txt: a small file at the root of your website that summarizes what you do and links to your most important pages. This guide explains what llms.txt is, what goes in it, how to create one and, just as importantly, what it will not do.

What is llms.txt?

llms.txt is a proposal, first published in 2024, for a Markdown file placed at yoursite.com/llms.txt. Large language models (LLMs) have limited room to read a page, and websites are full of menus, scripts and ads. The file gives them a clean, short summary instead. The proposal and its format are documented at llmstxt.org.

llms.txt vs robots.txt vs sitemap.xml

What goes in an llms.txt file

The format is simple Markdown:

1. An H1 heading with your company or site name.
2. A short summary in a blockquote: who you are and what you do, in one or two sentences.
3. Optional plain paragraphs with key facts, such as plans, locations or contact details.
4. H2 sections with lists of links to your most important pages, each with a short description.

Here is a shortened example of the structure:

# Example Plumbing Co.
> Licensed plumbers serving Denver with same-day repairs.

## Services
- [Emergency plumbing](https://example.com/emergency/): 24/7 call-outs
- [Water heater repair](https://example.com/water-heaters/): repair and replacement

You can see a real example on our own site at siliconhybrids.com/llms.txt.

How to create the file

1. Write a one or two sentence summary of your business in plain words.
2. List your 10 to 30 most important pages: services, pricing, about, contact and your best guides.
3. Add a short, factual description to each link.
4. Save it as a plain text file with that name and upload it to the root of your site.
5. Update it whenever you add or remove important pages. Some sites generate it automatically on each build.

What llms.txt will not do

It is important to be realistic. The file is not an official standard, and Google and other search engines have not said they use it to rank or cite pages. AI answers are built mainly from pages that are crawlable, trusted and clearly written. So the file will not:

• Replace SEO or make a weak page rank.
• Force any AI tool to mention your business.
• Control how AI companies use your content. That is what robots.txt and the AI crawler rules are for.

What it can do is give AI tools and agents that do read it a clear, accurate summary at almost no cost. That is why we include one in every AEO and GEO project, alongside the work that matters more.

Should every business add llms.txt?

For most businesses the answer is yes, as long as expectations stay realistic. It takes under an hour to write, costs nothing to host and does no harm. Sites with lots of useful content, such as software documentation, guides and service catalogues, benefit most, because a clean list of key pages is genuinely helpful to any tool that reads it.

The businesses that should not spend much time on it are the ones whose basics are not in place yet. If your pages are slow, thin or not indexed, that work comes first.

Keeping the file up to date automatically

An out-of-date file that links to deleted pages is worse than none. If your site has a build process or a CMS, generate the file automatically from your list of published pages, the same way sitemaps are generated. On WordPress, some SEO plugins can now create it for you. On our own site, the file is rebuilt every time we publish, so new services and guides appear in it without anyone remembering to edit it.

Common mistakes with the file

• Listing every page. The point is a short, curated map. Hundreds of links defeat the purpose.
• Marketing language. Plain, factual descriptions are more useful to AI tools than slogans.
• Broken links. Pages get renamed or removed, so check the links whenever your site changes.
• Conflicting facts. Prices, services and contact details should match your website exactly.
• Expecting instant results. The file is a small signal that some tools may read. It does not trigger rankings or citations by itself.

AI crawlers and robots.txt

If you want AI tools to read and cite your pages, they first need to be allowed to crawl them. Many AI companies use named crawlers, such as GPTBot for OpenAI, PerplexityBot for Perplexity and ClaudeBot for Anthropic, and Google uses the Google-Extended token for some AI uses. You control these in robots.txt, not in the summary file. A common setup is to allow search and AI crawlers to read public pages while blocking admin areas, internal search results and private sections.

Check your robots.txt and any firewall or bot protection rules together. Security tools sometimes block AI crawlers by default, which can quietly stop your content being cited.

Summary

In short: the file is a short Markdown summary of your site for AI assistants. It is quick to write and does no harm, but it is not an official standard and will not replace SEO or good content.

Where llms.txt fits in AI search

For the full picture, read What is AEO and GEO? and our checklist for getting cited in Google AI Overviews.

Frequently asked questions

What is llms.txt?

llms.txt is a proposed Markdown file at the root of a website that gives AI assistants a short summary of the site and links to its most important pages.

Does Google use llms.txt?

Google has not said it uses llms.txt for Search rankings or AI Overviews. Treat it as an optional extra for AI tools that choose to read it.

Is llms.txt the same as robots.txt?

No. robots.txt tells crawlers what they may access. llms.txt only summarizes your site and does not allow or block anything.

Where do I put llms.txt?

At the root of your domain, so it loads at yoursite.com/llms.txt, the same place as robots.txt.

How long should an llms.txt file be?

Short. A summary plus 10 to 30 key links with one-line descriptions is enough for most businesses.

Does llms.txt help with ChatGPT or Perplexity?

It may help tools that choose to read it, but none of the major AI companies has promised to use it. Being crawlable, clear and trusted matters far more for being cited.

Can llms.txt block AI from using my content?

No. llms.txt only describes your site. To allow or block specific AI crawlers, use rules in robots.txt for their user agents, such as GPTBot or Google-Extended.

How often should I update my AI summary file?

Whenever you add, rename or remove important pages. If your site has a build process, generate it automatically.

Can the file hurt my SEO?

No. It is a plain text file that search engines do not use for ranking, so it does not affect your SEO either way.

Is the AI summary file required?

No. It is entirely optional. Search engines and AI tools can still read your site without it.

What format should the file use?

Plain Markdown: an H1 title, a short summary in a blockquote, then sections of links with one-line descriptions.

Should I include every blog post in the file?

No. Include your most useful pages and best guides. A short, curated list is more helpful than a long one.

Who created the llms.txt proposal?

It was proposed in 2024 by Jeremy Howard of Answer.AI and is documented publicly so anyone can adopt it.