llms.txt
llms.txt is a proposed web standard for a Markdown file placed at a site's root, at /llms.txt, that gives large language models a concise summary of the site and links to its most useful pages. Jeremy Howard of Answer.AI proposed it in September 2024. Support among AI companies is still uneven.
Example
A software company's /llms.txt opens with a one-paragraph product summary, followed by links to its docs, pricing and API reference.
Why it matters for a website chatbot
An llms.txt file is cheap to add and gives AI tools a clean summary of what you do. ChatBento publishes its own at chatbento.com/llms.txt, with plans and prices kept in sync with the pricing page.
See it in practice: Docs and help-center search
Related terms
- Answer engine optimization (AEO)Answer engine optimization (AEO) is the practice of structuring web content so AI answer engines, such as ChatGPT, Perplexity and Google AI Overviews, can find, understand and cite it. It favors direct answers near the top of a page, clear headings, accurate facts, structured data and pages that crawlers can read.
- AI OverviewsAI Overviews are AI-generated summaries that Google Search shows above the regular results for some queries, with links to the web pages the summary draws on. Google launched them broadly in the United States in May 2024, and they answer many questions directly on the results page.
- Web crawlerA web crawler is a program that visits web pages automatically, reads their content and follows links or sitemaps to discover more pages. Search engines, AI companies and site tools use crawlers. Well-behaved crawlers identify themselves and respect a site's robots.txt file and noindex rules.