Which top websites block AI assistants?

Name: AI crawler blocking on the world's most popular websites
Creator: Adnave
License: https://creativecommons.org/licenses/by/4.0/

A live study of the web's most popular sites, measured from each site's own published rules and updated as we re-check them.

21%

block at least one AI assistant

115 of 551 sites

11%

block all five we check

59 of 551 sites

79%

let every assistant in

436 of 551 sites

When first measured on November 12, 2026 across 66 sites, the share blocking at least one assistant was 33%. It is now 21%. The set of sites has grown since, so part of any change is the wider sample.

Policy tracker: which sites changed their rules, checked daily

How often is each assistant blocked?

ChatGPT

19% (103)

ChatGPT search

12% (64)

Claude

18% (99)

Perplexity

15% (82)

Google Gemini

16% (90)

Which sites block AI assistants?

Each links to its full audit.

How was this measured?

For each of the most popular websites, we read its robots.txt file, the public file where a site tells crawlers what they may visit, and checked whether it shuts out the crawlers behind ChatGPT (GPTBot and OAI-SearchBot), Claude (ClaudeBot), Perplexity (PerplexityBot) and Google Gemini (Google-Extended). A site counts as blocking an assistant when it disallows that crawler from the whole site. Sites that refused our audit or have no web page are left out. Last measured November 13, 2026; each site is re-checked about once a month.

Blocking is a choice, not a fault. Publishers often block AI crawlers to protect their content, while businesses that want to be recommended by assistants usually should not.

Loading the figures…

Put this figure on your own page

The picture is drawn fresh from our data, so it stays up to date wherever you place it. Free to use; the code links back to this page as the source.

<a href="https://adnave.com/research/ai-crawlers"><img src="https://adnave.com/embed/ai-crawlers.svg" alt="Share of the world's top websites that block AI assistants. Live figure by Adnave." width="480" height="200" loading="lazy"></a>

Use this data

Free to quote, chart and republish with a link to this page. The full data behind the study is one file:

Download the data (CSV)

Suggested citation: Adnave, “Which top websites block AI assistants?”, https://adnave.com/research/ai-crawlers

Does your site block AI assistants?

Many do without knowing. The free audit checks it, along with 40 other things.

Guide: how to be found by AI assistants