Does llms.txt Help SEO? Google's Answer, and What Next

No. Google Search does not read llms.txt, and since June 2026 it says so in writing: creating one "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them." The data on other consumers is nearly as blunt — Ahrefs checked 137,210 domains and found that of the 28% publishing a valid llms.txt, 97% received zero requests to that file in May 2026. The file is still worth shipping under one narrow condition: your build generates it automatically, so it costs you nothing to keep accurate.

If a site audit flagged your missing llms.txt with a scary warning, that is the whole answer. The rest is the reasoning, the one way the file can actually cost you something, and a decision rule you can apply in ten minutes.

What Google actually changed in June 2026

Google's AI optimization guide has a section titled "Mythbusting generative AI search: what you don't need to do." Under it sits the load-bearing sentence: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them."

That part is not new. What changed on June 15, 2026 was the addition Google logged in its documentation changelog as "Added a note to the AI optimization guide clarifying Google Search's usage of llms.txt files." The note reads: "It's completely fine if you decide to create and maintain LLMS.txt files (or other similar files) for other services or systems that use these files."

Read carefully, that is a narrowing of scope rather than a reversal. Google previously discouraged special machine-readable files broadly enough that people read it as "don't bother." The June note limits the discouragement to Google Search specifically and explicitly blesses the file for use elsewhere — as Search Engine Journal noted at the time, Google moved from discouraging the practice to declaring itself neutral about it.

Neutral is useful information: the downside risk in Search is zero. You will not be demoted for having the file, and you will not rank for it either. Every remaining argument has to be made on non-Google ground.

Why your SEO tool said something different

Because tools are describing a different promise. Roger Montti's audit of tool messaging for Search Engine Journal found Semrush warning that a site lacking a clear llms.txt "risks being misrepresented by AI systems," and Rank Math claiming the file "increases your chances of being cited properly." Yoast hedged with "can." Squirrly told its own users the opposite: "LLMs txt will not help you magically appear in AI search. There is currently zero proof that it helps."

Four tools, four positions, one file. None is lying about Google — Google is not the system they are describing. They are describing a hypothetical future consumer, and the honest phrasing for that is "may," not "risks."

Then why does Chrome audit it?

Here is the genuinely confusing part: Chrome shipped an llms.txt audit in Lighthouse 13.3 in May 2026, inside a new Agentic Browsing category — days after Search published guidance saying you do not need the file. Both are Google.

They are answering different questions. Search is talking about discovery — how your pages get found and cited. Chrome is talking about functionality once an agent is already on your site. Chrome's own documentation for the audit puts it plainly: "Without this file, agents may spend more time crawling the site to understand its high-level structure and primary content." That is a latency-and-tokens argument, not a ranking argument, and the category is explicitly experimental and separate from the SEO audits.

The asymmetry that actually matters

This is the one detail that can make shipping the file worse than skipping it.

From Chrome's docs: "If the file is not provided by the server (resulting in a 404), the audit is marked as Not Applicable (N/A), as providing the file is optional at the moment." But Lighthouse flags the page if a server error occurs when it tries to retrieve the file.

Restated for a repo owner: no file is a shrug, a broken file is a failure. Absence never hurts you; a stale, malformed or half-deployed file can cost you an audit result you would have passed by default.

The audit also applies validation that the docs do not fully describe. Slobodan Manic ran Lighthouse CLI against his own site and got the error "File does not appear to contain any links" on a file that was full of URLs — they were just plain-text URLs. Converting them to Markdown [text](url) syntax moved his Agentic Browsing score from 0.67 to 1.0. That rule is not in the documentation; he found it by running the tool.

The audit itself has been shaky, too. Lighthouse issue #17082, filed in June 2026, reports "Fetch of llms.txt failed" on a spec-compliant file returning HTTP 200 — with Cloudflare edge logs showing Google's PageSpeed Insights runner successfully fetching it twice on June 18, 2026. The reporter also flagged a version mismatch: PSI was running headless Chrome 146 while the docs require Chrome 150 or later. It was labelled P1 and has since been closed with a linked fix. Good outcome, fair warning: do not architect your build around this score.

Who actually reads llms.txt

Three independent datasets point the same way. They measure different things, so keep them separate.

Ahrefs, June 2026. Louise Linehan and Xibeijia Guan checked 137,210 domains that received traffic in May 2026. 28% — about 38,360 domains — published a valid llms.txt. Of those files, 97% received zero requests during the month; roughly 1,100 domains' files got any traffic at all. Of the requests that did happen, 96% were bots and 4% humans. The largest single channel was SEO audit tools at 21.7% — bigger than any AI category. All AI-related categories combined came to 19.5%.

Otterly.ai, February 2026. A 90-day server-log study found 84 of more than 62,100 AI bot visits went to /llms.txt — about 0.1%, roughly a third of the traffic an average content page drew on the same site.

Limy, 2026. The largest sample found: 408 requests out of 515,382,577 monitored LLM-bot events over 90 days touched /llms.txt, tracking GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot and Google-Extended. It is a vendor blog rather than a peer-reviewed study, so weigh it as directional.

On the citation question specifically, SE Ranking analysed roughly 300,000 domains and found no relationship between having llms.txt and how often a domain is cited in major LLM answers. Removing the llms.txt variable from their predictive model actually improved its accuracy.

One more Ahrefs finding deserves its own line, because it quietly determines whether shipping the file does anything at all: AI bots do not probe for missing llms.txt files. Requests to non-existent files were overwhelmingly human — SEOs checking competitors. An agent reads your llms.txt only if it already knows the file exists, so a file nothing links to is a file nothing reads. So much for "publish it and the crawlers will come."

What the file is genuinely good for

Two things, both real, neither of them ranking.

Documentation for coding agents. In Ahrefs' data, GPTBot was the top individual requester at 4.51%, with Claude-Code close behind as the leading agentic tool — out-fetching every AI search bot except GPTBot. That is the actual constituency: developer tooling working inside a session, not answer engines building an index. If your site is docs, an API reference or a dev tool, your users' agents are the ones fetching this file. If it is a marketing blog, they are not.

The exhibit is Anthropic's own file at code.claude.com/llms.txt. It is an H1, a blockquote description, one ## Docs heading, and a few hundred flat bullets in - [Title](URL): description form. No topical grouping, no cleverness. Per the spec Jeremy Howard proposed in September 2024, that is all it was ever meant to be: a curated Markdown index whose purpose is to "provide information to help LLMs use a website at inference time."

The spec also has a second half most write-ups skip: it proposes serving a clean Markdown version of each page at the same URL with .md appended — exactly what Anthropic's entries link to. The index file is half the proposal, and arguably the less interesting half.

A build artefact for your own pipeline. This is where the file earns its commit. For a static site it should be generated at build time from your source, never hand-maintained: docusaurus-plugin-llms, starlight-llms-txt for Astro Starlight, mkdocs-llmstxt, vitepress-plugin-llms and sphinx-llms-txt all do it, and Hugo can do it with a custom output format. Hosted docs platforms like Mintlify generate it with zero configuration.

For teams assembling that pipeline, a practical walkthrough on building an AI content agent explains the six parts beyond the prompt and how they slot into build-time generation.

Generated means it cannot drift. Hand-maintained means it will, and drift is the failure mode Ahrefs lists first among the costs — a file describing pages you renamed six months ago is a confidently wrong map. Same discipline as automated link verification in a publishing pipeline: an artefact nobody re-checks is quietly rotting. Our own agent will not save a draft until every cited URL resolves, for exactly that reason.

The thing worth doing alongside it

If the goal is "make my site cheap for an agent to read," content negotiation has more demonstrable uptake than the index file.

Stefan Judis at Checkly tested seven coding agents in February 2026 and found three of them send Accept: text/markdown: Claude Code v2.1.38, Cursor 2.4.28 and OpenCode 1.2.5. OpenAI Codex, Gemini CLI, GitHub Copilot and Windsurf sent standard browser Accept headers or */*. Three of seven is not universal — but it is measurable behaviour from named clients, which is more than llms.txt discovery can show.

Cloudflare shipped Markdown for Agents on February 12, 2026, converting HTML to Markdown at the edge when a client sends that header and returning a token-count header alongside it. Cloudflare claims up to 80% token reduction versus HTML. For a static site behind Cloudflare, it is a dashboard toggle rather than a repo change.

Two caveats. First, machine-specific content raises a cloaking question — SEO consultants at launch called it a second candidate version of reality — and a Markdown build that drifts from your HTML has the same problem as a stale llms.txt. Second, none of this is access control. Whether bots may use your content is robots.txt's job, and that is the surface standards bodies are extending: the IETF's AI Preferences working group builds on RFC 8615 well-known URIs and the RFC 9309 robots exclusion protocol, with standards-track drafts due to the IESG by August 31, 2026. llms.txt has no standards body, and its spec deliberately declines to say how the file should be processed.

The ten-minute decision

A checklist you can run against your own repo:

1. Does your generator make one for free? If a plugin or platform emits it at build time, ship it. The marginal cost is a config line and it stays accurate on its own.
2. Would you write it by hand? Then skip it. A hand-maintained index is guaranteed to drift, and a broken file is the only version of this that costs you anything.
3. Is your audience developers using coding agents? Then the file has a real constituency. Otherwise the honest expected value is near zero.
4. Link to it. Nothing probes for a missing file. Reference it from your docs, your README, or your footer, or it will not be found.
5. Use Markdown link syntax. - [Title](https://example.com/page): description, not bare URLs — that alone is the difference between a passing and failing Lighthouse audit.
6. Do not chase the Agentic Browsing score. The category is experimental and its audits have had real bugs. Passing is nice; refactoring your build to pass is not.
7. Re-check in six months. Adoption moved from 10.13% of domains in SE Ranking's November 2025 sample to 28% in Ahrefs' May 2026 sample — different samples, not a like-for-like series, but a clear enough direction that the answer could change.

The honest summary: a ten-minute commit with a near-zero downside and a near-zero upside, and the time you would spend agonising over it is better spent on pages worth citing. Ahrefs closed its study with the line that captures the moment: "Either we're watching the early scaffolding of a real standard, or we're watching the SEO industry prove it can productize anything."

FAQ

Does llms.txt help SEO? No. Google Search ignores the file and states that maintaining one "will neither harm nor help your site's visibility or rankings in Google Search." Nor has an independent study linked it to AI citations — SE Ranking's analysis of roughly 300,000 domains found no correlation.

Will adding llms.txt hurt my rankings? No. Google's June 2026 guidance is explicit that the file neither harms nor helps Search visibility. The only measurable downside is operational: Lighthouse marks a missing file "Not Applicable" but flags one that errors on fetch, so a broken or stale file is worse than no file at all.

Do ChatGPT, Claude or Perplexity read llms.txt? No major model provider has publicly committed to consuming llms.txt in production. Server logs show mostly tooling traffic — SEO audit tools were the largest requester category in Ahrefs' data at 21.7%, with coding agents like Claude Code the most active AI-side consumers. Bots also never request the file speculatively, so an unlinked file goes unread.

Should a static site ship an llms.txt file? Only if your build generates it. Plugins exist for Docusaurus, Astro Starlight, MkDocs, VitePress and Sphinx, and Hugo can emit one via a custom output format. If shipping it means hand-writing and hand-updating a Markdown index, the maintenance cost outweighs any realistic benefit.

What should I do instead? Publish clean, well-structured HTML with accurate titles and headings, keep robots.txt correct — that is where access control and the IETF's standards-track AIPREF work live — and consider serving Markdown via content negotiation if your audience uses coding agents.

The part that actually compounds

Everything above resolves to a config line. What decides whether an agent finds anything worth reading on your domain is what it always was: pages that answer real questions, with claims a reader can check.

That is the slower work, and it is what Magic Share's drafting and verification pipeline keeps moving — the agent studies your site, drafts a post, checks every cited link before the draft is saved, and opens a pull request you review over a coffee. Nothing publishes without you. Our close reading of what Google's scaled content abuse policy actually bans runs the same exercise as this post, and more of it sits in Notes from the garden.

Ship the llms.txt if your build makes it for free. Then go write something worth linking to — and if you would rather that happened while you sleep, start free.