Group your pages under headings, add a one-line description for each, and generate a valid llms.txt file — a structured index that helps AI models like ChatGPT, Claude, and Perplexity understand what your site is about.
One page per line: Section | Title | URL | description — description is optional.
llms.txt is a proposed convention — not an official web standard, but one that's been adopted widely enough to matter — for giving AI models a clean, structured summary of a site's content at a predictable location: yoursite.com/llms.txt. Where a normal page is full of navigation, scripts, and layout markup that a model has to wade through, an llms.txt file is plain markdown: a name, a short description, and a categorized list of links, each with its own one-line summary.
robots.txt controls whether a crawler is allowed to fetch a page. sitemap.xml lists which pages exist and roughly when they changed. Neither one describes what a page is actually about — that's the gap llms.txt fills, specifically for AI models trying to understand a site's content well enough to answer questions about it or cite it accurately, rather than for traditional search engine crawling. See this post for more on how the first two differ from each other, and this one for why "just add the AI crawlers to robots.txt" isn't the same job as publishing an llms.txt — one is a permission file, the other is a description file, and neither substitutes for the other.
Upload the file to your site root so it's reachable at exactly yoursite.com/llms.txt. There's no submission step the way there is with a sitemap in Search Console — AI crawlers that support the convention check for the file directly when they visit.
In May 2026, Google shipped a new "Agentic Browsing" audit category in Chrome Lighthouse that includes a check for llms.txt. It's worth understanding exactly what that check does, because it's narrower than the headlines about it suggest: Lighthouse flags a genuine server error if it tries to fetch your llms.txt and something breaks, but if the file simply doesn't exist — a normal 404 — the audit is marked Not Applicable rather than failed, since providing the file is still explicitly optional. It's an infrastructure check for AI agents browsing the live web, not a content-quality grade.
That's a separate question from search ranking, and the two shouldn't be conflated. The same month, Google's own Search Central guidance was explicit that llms.txt files don't help a site appear in AI Overviews or other AI-generated search results — a different team at Google addressing a different question than the one Lighthouse asks. Chrome checking whether the file exists for an AI agent visiting your site directly is not the same as Google Search using it as a ranking or citation signal, and as of this writing no major AI company, Google included, has said llms.txt factors into what their systems choose to cite. Add it because it gives an AI agent a cheap, structured summary instead of making it parse your HTML — not because it will move your search rankings, since it won't.
llms.txt and llms-full.txt solve two different problems. llms.txt is the index — a short summary plus a categorized list of links with one-line descriptions, which is exactly the format this tool produces. llms-full.txt is a much bigger file: the complete text content of every listed page, concatenated into a single document, so a model or coding agent can read an entire site in one fetch instead of following each link individually. Producing it means actually crawling and extracting every page's content — a heavier job than summarizing a list you already typed out — so generators handle it inconsistently: some gate llms-full.txt or full-site coverage behind a paid plan or an email capture, while others now emit both files free for a capped number of pages, and documentation platforms like Mintlify and GitBook publish both automatically on every build. If you need one, check what page limit a given generator applies before assuming it covered your whole site.
That's also why this tool doesn't generate llms-full.txt: it works from a list you paste in yourself — section, title, URL, one-line description — not from crawling and extracting full page content, so there's no page text here to concatenate in the first place. Building one yourself for a small site isn't hard: paste the rendered text or markdown source of each page you listed above into a single .txt file, in the same order as your llms.txt, with a heading marking where each page starts so a model can still tell them apart. For a handful of pages that's a manual copy-paste job; past a dozen or so, a crawler-based generator that extracts page content automatically becomes the more practical route.
Worth calibrating expectations either way: how much AI agents actually fetch either file today is genuinely unsettled. One widely cited data point from a documentation-hosting platform found agents pulling llms-full.txt more than twice as often as llms.txt on the sites it tracks — but broader measurement across large numbers of domains has found that most published llms.txt files receive no AI requests at all, with a large share of the traffic they do get coming from SEO auditing tools rather than actual models. Publish it because it's a cheap, honest description of your site, not because either file is a guaranteed traffic lever.
No — it's one input among many, not a ranking mechanism. It makes your content easier for a supporting model to parse accurately if it does decide to reference your site, the same way a sitemap makes a page easier to discover without guaranteeing it gets indexed.
Support varies and is still evolving — this is a relatively new, community-driven convention rather than an enforced standard, so treat it as a reasonable-effort addition rather than something every AI crawler is guaranteed to read.
A sitemap is a bare list of URLs with no description of what each page contains — a crawler still has to fetch and parse every page to know what it's about. llms.txt front-loads that summary in plain text at a single, small file, which is a much cheaper way for a model with a limited context window to get an overview before deciding what (if anything) to fetch in more depth.
No. The Lighthouse Agentic Browsing audit only fails you for a genuine server error when it tries to fetch llms.txt — not having the file at all is marked Not Applicable, not a failure, since the file is still optional. It's also unrelated to Google Search ranking; Google's own Search Central guidance says llms.txt doesn't affect AI search visibility. It's a reasonable, low-cost thing to add for AI agents browsing your live site, but there's no compliance requirement or ranking penalty for skipping it.
Only if you have a specific reason to. Documentation sites that developers point coding assistants at are the clearest case where it earns its keep, since the assistant can read an entire doc set in one fetch instead of crawling page by page. For a general marketing or blog site, the evidence that it moves AI citations or traffic is thin so far, so treat it as optional polish rather than something to chase.
The convention's file name is llms.txt (plural — the file describes content for large language models in general, not one specific model), served at your site's root just like robots.txt. "llm.txt" without the s is a common typo of the same idea, not a separate file — this generator outputs the correct llms.txt name and format.