llms.txt, honestly: what it does and does not do
llms.txt is a markdown file at your site root that lists your most important pages for AI agents. Adoption is voluntary and no major engine promises to read it, so treat it as cheap insurance rather than a ranking lever. It takes half an hour and costs nothing.
What the file is
A single H1 with your site name, an optional one-line blockquote summary, then H2 sections containing markdown link lists in the form '- [Title](/path): description'. That is the whole spec.
It is not robots.txt. It grants nothing and blocks nothing. It is a curated map for an agent that already has access.
Why we weight it at 8% and not 20%
Because we have no evidence that any major assistant currently uses it as a retrieval input. Weighting it higher would be inventing a finding. Weighting it at zero ignores that agentic tools built on these models do fetch it, and that the cost of publishing one is trivial.
Common mistakes
Listing every route including /login, /admin and API callbacks. Publishing a two-line stub with no descriptions. Serving an HTML 404 page at /llms.txt, which is the failure we see most often — check the actual response body, not just the status code.
Want the version of this measured against your own site? Run a free audit.