So, is it actually worth your time?

In this guide, we'll look at what llms.txt does, how to create one for your website, and how to check that it's set up correctly. We'll also cover what we know about its impact on AI search visibility so far.

What Is llms.txt?

Think of llms.txt as a reading list for AI tools.

It's a simple Markdown file that lives at the root of your website, usually at yourdomain.com/llms.txt. Instead of making AI systems navigate through your entire website, the file points them toward pages you consider important.

These might include product information, documentation, FAQs, tutorials, or detailed guides.

The idea was introduced by Jeremy Howard of fast.ai and Answer.AI in September 2024.

Websites already use robots.txt to tell crawlers which areas they can access. llms.txt takes a different approach. It suggests which pages an AI system might find useful when looking for information about your website.

There are two commonly used formats:

• llms.txt: A short, organized list of important pages, with a brief description of each.
• llms-full.txt: A longer document containing the full text of selected pages.

You don't need both. For most websites, starting with a basic llms.txt file is enough to experiment with the idea.

How Is llms.txt Different From robots.txt?

It's easy to confuse the two because they both sit in your website's root directory. But they serve different purposes.

The key difference is that llms.txt doesn't control access to your website.

Adding one won't prevent AI crawlers from visiting certain pages or using your content for training.

If you want to manage crawler access, robots.txt is still the place to do that, although its rules depend on crawlers choosing to follow them.

What Should You Include in llms.txt?

You don't need anything complicated here.

A good llms.txt file should help someone understand what your website offers and where to find the most useful information.

The proposed format includes:

1. Website name: Start with a heading that identifies your website or project.
2. Short introduction: Explain what your business, product, or website is about.
3. Content sections: Group related pages under headings such as Documentation, Product Guides, or FAQs.
4. Important links: Add direct URLs with a short explanation of what each page covers.
5. Optional resources: Include less important pages, such as changelogs or supplementary references, in a separate section.

For example, a software company might organize its file around Getting Started, API Documentation, Troubleshooting, and Product Updates.

One thing worth remembering: llms.txt isn't supposed to be another sitemap.

You don't need to include every page on your website. Start with around 10 to 30 useful pages and adjust that number based on how much content you have.

The goal is to make the file helpful, not comprehensive.

How to Create an llms.txt File

You don't need development experience to put together a basic file. Here's how to get started.

Step 1: Choose Your Most Useful Pages

Start by identifying the pages that explain your business, products, or services best.

Good candidates include FAQs, product documentation, detailed guides, and other pages that answer common customer questions.

Avoid adding pages just because they exist. If a page offers little useful information, it probably doesn't belong in your llms.txt file.

Step 2: Write Your File

Open a plain text editor and create a file named llms.txt.

Add your website name, a short introduction, and links organized into relevant sections.

Keep the descriptions straightforward. A sentence explaining what someone will find on each page is usually enough.

Step 3: Publish It on Your Website

Upload the file to your website's root directory.

Once published, it should be accessible at:

https://yourdomain.com/llms.txt

This is similar to where you'd find your robots.txt or sitemap.

Step 4: Keep It Updated

Your website won't stay the same forever, and neither should your llms.txt file.

When you publish important documentation, update product information, or remove old pages, review the file too.

Broken links and outdated descriptions defeat the purpose of creating it.

Want a Faster Way to Get Started?

You can also use a free llms.txt generator to create an initial version from your website URL.

It's a useful starting point, especially if you have several pages to organize. Just review the generated file before publishing it. You'll still want to make sure the right pages are included and the descriptions are accurate.

How to Check If Your llms.txt File Is Working

Once you've published the file, there are a few simple checks worth doing.

• Open the URL: Visit yourdomain.com/llms.txt and make sure the file loads without an error.
• Check the formatting: Your website name should appear at the top, followed by clearly organized sections.
• Test the links: Open each URL to confirm it leads to the intended page.
• Look for redirects: Where possible, use final destination URLs instead of links that redirect elsewhere.
• Review the descriptions: Make sure they accurately describe the linked pages.

If everything looks right, your file is accessible and formatted as intended.

But there's an important distinction here.

A working llms.txt file doesn't necessarily mean AI platforms are reading or using it.

These checks only confirm that you've published the file correctly. They don't tell you whether it's helping your AI search visibility.

Does llms.txt Actually Improve AI Search Visibility?

This is where things get interesting.

There's plenty of discussion around llms.txt, but the evidence supporting its SEO or AI visibility benefits is still limited.

The format is a proposal, not an established web standard. Major AI search providers haven't confirmed that simply adding an llms.txt file improves your chances of appearing in AI-generated answers.

And while some developers and documentation platforms have started experimenting with the format, that doesn't mean every AI system uses it.

So why are people still talking about it?

Because the underlying idea makes sense. If AI tools can easily find well-organized information about your website, that could make certain content easier to access in systems that support the format.

The missing piece is evidence that this leads to more mentions, citations, or traffic across major AI platforms.

For now, there isn't enough reliable evidence to treat llms.txt as a proven AI SEO ranking factor.

That doesn't mean you should ignore it completely.

If you run a documentation-heavy website, creating the file is relatively simple and may be worthwhile. Just don't expect it to solve bigger visibility problems.

Clear page structure, accessible content, useful information, and regularly updated pages should still come first.

Common Misconceptions About llms.txt

A few claims about llms.txt come up repeatedly. Here's what you should know.

"Adding llms.txt will help my website appear in ChatGPT."

Not necessarily. There's no guarantee that ChatGPT or other AI platforms will use your file when generating answers. A ChatGPT rank tracker can show whether it makes a difference.

"I can use llms.txt to stop AI bots from crawling my website."

No. llms.txt doesn't contain crawler-blocking instructions. Use robots.txt and other access controls for that purpose.

"llms.txt is a replacement for traditional SEO."

Not at all. Your website still needs crawlable pages, useful content, sensible internal linking, and sound technical SEO.

"Only large websites need llms.txt."

Website size isn't the deciding factor. Even a small website can benefit from having its important resources organized clearly, particularly if it contains detailed documentation or guides.

"Once I've added llms.txt, I don't need to do anything else."

Publishing the file is only one small step. You'll still need to maintain your content and measure whether your website is actually appearing in AI-generated answers.

Should You Add llms.txt to Your Website?

If your website has helpful guides, documentation, FAQs, or other reference content, creating an llms.txt file is a reasonable experiment.

It doesn't take much effort, and keeping a curated list of your best resources can be useful in its own right.

But if your goal is to increase visibility in ChatGPT, Perplexity, Gemini, or other AI search platforms, don't make llms.txt your main priority.

Start with the content itself. Make sure your important pages are accessible, answer real questions, and provide information that AI systems can understand and reference.

Then consider llms.txt as an additional step, not a shortcut.

And remember, publishing the file and measuring AI visibility are two different things.

If you want to know whether AI platforms are actually mentioning your brand or citing your website, you'll need to track those results separately. A free AI visibility checker is an easy place to start.

Disclosure: I work on Track My Visibility, a platform that helps businesses track how their brands appear in AI-generated answers. This article is adapted from our original guide to llms.txt.

\