Clients have asked us this several times this year, usually as "should we build an AI-friendly version of the site?"
For Google, the answer is no. Google's documentation says a well-built HTML site already works for Search, AI Overviews and AI Mode, and extra AI files are ignored. Google cannot speak for ChatGPT, Perplexity and other assistants, so treat this as guidance about Google's systems, not a universal rule.
Key takeaways
Google ignores llms.txt for rankings and does not need markdown or AI-only files.
John Mueller's sharper view (shared personally, not as Google policy): if special formats helped AI answers, AI companies would say so loudly.
Bing's Fabrice Canel: a second version doubles crawling, and bot-only versions are often neglected and broken.
The risk is drift. A markdown copy is not cloaking by itself, but it becomes a problem when it stops matching the page.
Build one clear site instead: clean headings, direct answers, server-rendered content and accurate structured data.
Why does this question keep coming up?
AI crawlers now make up a large and growing share of web traffic. Cloudflare's July 2026 report says more than half of internet traffic is now non-human, and the purpose of that crawling has shifted fast:
- Spring 202522%
- June 202652%
Source: Cloudflare, 'Content Independence Day, one year on', 1 July 2026
With that much automated traffic, it is reasonable to wonder whether a leaner, text-only copy of your site would help bots understand it or cite it. The evidence says it would not, at least for Google.
What does Google actually say?
Google's guide to AI features and your website is unusually direct:
You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search.
On llms.txt, the guide says such files "neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them." If Google happens to index one, "this doesn't mean that the file is treated in a special way."
That does not make llms.txt harmful. If another tool you use reads it, keep it. It just does nothing for Google.
What did John Mueller say?
This is Mueller's personal view, posted on Bluesky on 23 November 2025, not published Google policy. Responding to the idea that AI systems prefer particular file formats, he wrote:
If those creating and running these systems knew they could create better responses from sites with specific file formats, I expect they would be very vocal about that. AI companies aren't really known for being shy.
What does Bing think?
Fabrice Canel from Microsoft Bing made a more practical point on 5 February 2026. A separate bot version does not replace crawling your real pages. It adds to it:
We'll crawl anyway to check similarity. Non-user versions (crawlable AJAX and like) are often neglected, broken.
He finished with "Less is more in SEO!"
Here is how the three positions line up:
| Source | Date | Position | Formal policy? |
|---|---|---|---|
| Google Search documentation | Current | AI files and markdown are not required, llms.txt is ignored | Yes |
| John Mueller, Google | 23 Nov 2025 | Special formats would be publicised if they helped | No, personal view |
| Fabrice Canel, Bing | 5 Feb 2026 | Bing crawls both versions, bot-only copies tend to break | No, personal view |
Is a markdown version a cloaking risk?
Not automatically. Cloaking, under Google's spam policies, means showing search engines different content from what people see in order to manipulate rankings. An accurate markdown copy of the same content is not that.
The risk is drift. Someone updates the page and forgets the copy. Now bots and people see different things, even though nobody intended it. That maintenance failure is exactly what Canel describes, and it is why a second version is a liability rather than an asset.
Do AI crawlers even send you visitors?
Far fewer than they crawl. When Cloudflare first published crawl-to-referral ratios in July 2025, Anthropic's crawlers made roughly 70,900 page requests for every visit referred back (19 to 26 June 2025). Cloudflare notes this may overstate the gap, because visits from native apps often arrive without a referrer.
In July 2026, Cloudflare also reported that some of the most heavily crawled categories saw human traffic fall by as much as 40% in under a year.
So the case for a second site is weak on two fronts. Google does not use it, and the crawlers it would serve send back very little traffic.
What should you build instead?
Nothing extra. Make the one version everyone sees easier to read, for people and machines alike.
One clear H1 and a logical H2 and H3 structure
The answer in the first paragraph, then the detail
Structured data that matches what visitors can see
Server-rendered content, so the text exists before JavaScript runs
Descriptive link text instead of "click here"
Fast, stable pages on a mid-range phone
You can check the server-rendered part in a few seconds. This fetches the raw HTML the way a simple crawler does, without running JavaScript:
# If your H1 and opening paragraph do not appear here, some crawlers will not see them either.
curl -s https://www.example.com/your-page | grep -i -A2 "<h1"And this is the shape of a page that already works for search engines and AI systems:
<article>
<h1>Does my site need a separate AI version?</h1>
<p>No, not for Google. A well-built HTML page already works for Search and AI features.</p>
<h2>What does Google actually say?</h2>
<p>...</p>
</article>None of this is AI-specific. It is the same foundation behind planning a website that scales. To see whether your pages appear in Google's AI features, read how to track AI Overview visibility in Search Console.
What should you do next?
Do not build the second site. Spend that budget making the first one clearer, faster and better structured.
If someone is selling you an AI mirror of your website, send us the pitch and we will tell you plainly whether it is worth anything. You can also see our AI search visibility work.
Frequently asked questions
Does an llms.txt file do anything at all?
It can help if another platform or internal tool reads it. It has no effect on Google Search visibility or rankings.
Does this apply to ChatGPT, Perplexity and other assistants?
Not by confirmation. Google's statements cover Google Search, AI Overviews and AI Mode only. Other companies have not published an equivalent position.
Will a lightweight AI-only version ever make sense?
Not as a Google ranking tactic. It can make sense for a genuinely different job, such as feeding an internal tool, as long as it does not drift from the real page.
Is this the same as having a separate mobile site?
No. A responsive site serves the same content to every visitor and bot. A markdown copy only becomes a similar concern once it stops matching what the HTML page says.
Sources
Cloudflare: Content Independence Day, one year on (1 July 2026)
Cloudflare: The crawl before the fall of referrals (1 July 2025)
