llms.txt is a text file in a site’s root directory giving AI systems and answer engines an ordered, readable overview of its most important content. The idea is to supply what an HTML page with navigation, scripts and layout does not readily yield: a compact map of what this site is about.
Its status: a proposal, not a standard
Important for context is that llms.txt is an emerging proposal, not a binding specification. There is no guarantee any particular system reads or considers the file. Anyone creating one does so as a low-cost measure with an open return — not as a precondition for being found at all.
What belongs in it
What works is a short outline of what the organisation does, followed by thematically grouped sections listing the most important pages. Each entry consists of a title, an address, and a sentence describing the content. The selection is what matters: an llms.txt with fifty well-described pages is more useful than one with five hundred uncommented links, because it conveys a judgement.
How it differs from robots.txt and the sitemap
The three files solve different tasks and complement each other. robots.txt governs what may be fetched — it is an access rule. The sitemap lists exhaustively what exists — it is a directory. llms.txt describes what is important and what it is about — it is an orientation. None replaces another; creating an llms.txt leaves the need for a sitemap and robots.txt unchanged.
Maintainability is the real problem
A hand-maintained llms.txt is outdated within months, and an outdated file is worse than none, because it points at content that no longer exists. It therefore makes sense to generate it from the same data as the site itself — then it stays current automatically and new content appears without intervention.
What it does not do
The file does not make weak content citable and does not replace structured data. Nor does it govern whether an AI system may use content — robots.txt and its treatment of the relevant crawlers do that. Its value lies in easing comprehension for a system that is reading the site anyway.
Practical consequence
An llms.txt is a small investment with an uncertain but plausibly positive return. It makes sense when it is generated automatically and makes a genuine selection. Maintaining it by hand, or writing everything into it, creates effort without effect.