llms.txt: the complete guide for 2026, hype separated from setup
More than half of web traffic is now bots, agents, and AI crawlers. llms.txt is the web's first attempt at a reading guide for those visitors. It costs an hour to implement and the internet cannot agree on whether it matters. Both facts are true. Here is the guide without the evangelism. ## What it is llms.txt is a Markdown file at your domain root. It tells AI systems what your site is and where the important content lives. The format, from the original proposal: An H1 with the site name. A blockquote summary of what the site does. Sections of links to key pages, each with a one-line description. Optionally, a companion llms-full.txt containing expanded page content for systems that want depth without crawling. The design logic: AI systems work under context limits. A curated index lets them spend their budget on your important pages instead of parsing navigation and footers. ## The honest state of adoption Publisher adoption is real: thousands of sites ship one, and among developer-tool companies it approached standard practice through 2025 and 2026. Reader adoption is the murky half. Some AI tools fetch it. No major vendor documents guaranteed behavior. Nobody credible can show a controlled before-and-after. So the case is asymmetric rather than proven: one hour of cost, plausible upside, alignment with where machine-readable conventions are heading. That is why our recommendation stays what we wrote in [the agentic SEO explainer](/blog/agentic-seo-explained): add o