AIFindUsMeasured, not guessed

question

Why does my site look empty to AI assistants when it works fine in a browser?

Most AI crawlers do not run JavaScript. If your text arrives via the browser, they see an empty page — and that is measurable in seconds.

author Roydell Clarkepublished 3 min read3 verified sources

Because most AI crawlers do not run JavaScript. Your browser executes your site's code and paints the words on screen. A crawler that only reads the HTML your server sent gets whatever was in that first response — and on many modern sites, that is an empty shell.

You can see this yourself in about thirty seconds, and the result is often a surprise.

How do I check what a crawler sees?

Fetch your own page the way a simple crawler does, and read what comes back:

curl -s https://example.com/ | wc -c
curl -s https://example.com/ | grep -o "your headline text"

If the byte count is small and your headline is missing, the text is arriving by JavaScript. In a browser the page looks perfect. To a text-only crawler it is close to blank.

Disabling JavaScript in your browser's developer tools shows the same thing more vividly. Whatever disappears is what a non-rendering crawler never had.

Doesn't Google render JavaScript?

Google does render, with caveats worth knowing, and other crawlers are a separate question. Google Search Central notes one limit that catches sites out: "Google Search won't render JavaScript from blocked files or on blocked pages."

That is easy to trigger by accident. A Disallow on a build directory blocks the very bundle that produces your content, so the crawler fetches the page, cannot fetch the script, and indexes the shell. The robots.txt line looks like tidy housekeeping and behaves like a content deletion.

Rendering also does not make you eligible on its own. Google Search Central states the requirement: "To be eligible to be shown as a supporting link in AI Overviews or AI Mode, a page must be indexed and eligible to be shown in Google Search with a snippet, fulfilling the Search technical requirements."

And crucially, rendering behaviour differs by crawler. Several answer-time retrieval bots fetch HTML and do not execute scripts at all. For those, your rendered output is irrelevant; only the server's response counts.

What is the fix?

Serve the content in the HTML. In practice that means one of three things.

  1. Server-render the pages that matter. Every modern framework supports it. Your

marketing pages, documentation and answers are exactly the pages that should not depend on the client.

  1. Pre-render at build time. For content that changes rarely, static output is simplest

and fastest.

  1. Stop blocking your own assets. Check that nothing in robots.txt disallows the

scripts, styles or data your pages need.

A useful habit: make does this text exist with JavaScript off a release check rather than an audit finding. It is cheap to verify and expensive to discover late.

Does structured data help here?

It helps a different problem, and it is not a substitute. Markup states plainly what a page is about — Google Search Central frames it as: "You can help us by providing explicit clues about the meaning of a page to Google by including structured data on the page."

But if the markup itself is injected by JavaScript, a non-rendering crawler misses it too. Explicit clues in the HTML help. Explicit clues that only exist after hydration do not.

What this does not tell you

It does not tell you which specific assistants render and which do not, because that behaviour changes without announcement and nobody publishes a reliable matrix. The safe assumption is the conservative one: if the text matters, put it in the HTML. That way you do not have to track anyone's rendering roadmap.

sources

Each passage was fetched from the publisher’s own page and checked to appear there word for word before this post was written. Open any link and search for the sentence — that is the whole point of printing it.

  1. Google Search Central

    01
    Google Search won't render JavaScript from blocked files or on blocked pages.

    https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basicsretrieved 2026-08-04

  2. Google Search Central

    02
    To be eligible to be shown as a supporting link in AI Overviews or AI Mode, a page must be indexed and eligible to be shown in Google Search with a snippet, fulfilling the Search technical requirements.

    https://developers.google.com/search/docs/appearance/ai-featuresretrieved 2026-08-04

  3. Google Search Central

    03
    You can help us by providing explicit clues about the meaning of a page to Google by including structured data on the page.

    https://developers.google.com/search/docs/appearance/structured-data/intro-structured-dataretrieved 2026-08-04

how this was written

The sources above were fetched and verified by AIFindUs, then drafted with claude-opus-5 from those passages only, and reviewed and published by Roydell Clarke. A draft that cannot cite a primary source for a claim is not published.

read next