What the file is

llms.txt is a proposed convention for providing a concise overview and links to useful documentation. Its presence does not prove that an engine reads or prioritizes it. Support must be established for a particular consumer rather than assumed for all AI systems.

Google's documented position

Google says that it does not use llms.txt files for Search and that maintaining such a file neither helps nor harms visibility or rankings there. This includes its generative search features. Do not describe the file as a Google ranking factor.

What it does not replace

MechanismActual role
robots.txtCommunicates crawler access rules
Public HTML and linksMake content accessible and discoverable
Canonical and status handlingDescribe preferred and available pages
SitemapLists URLs for discovery
llms.txtOptional overview for consumers that choose to use it

Search and training are separate

OAI-SearchBot and GPTBot have distinct documented purposes. A file listing pages does not override access rules or establish whether material can appear in search. Check the crawler's actual documentation.

When maintaining it can be reasonable

A documentation publisher may find a concise index useful for a known consumer or internal workflow. Keep links current, avoid publishing private material, and treat the file as an additional representation of existing information. Do not promise citation lift or claim its link order controls a search engine.

What to fix first

If an important page returns an error, lacks usable HTML, or is isolated from navigation, address that problem. An optional text overview is not a substitute for a useful, technically accessible page.

Sources and scope