Introducing llms.txt — Controlling AI Access to Your Content

The web was built for people first and robots second. For decades, the implicit contract between publishers and search engines was clear: search crawlers.