Guide · Cloudflare

Block fake AI bots on Cloudflare

A rule that blocks the “GPTBot” user agent also blocks the real GPTBot and lets an impostor with any other name through. If your site is behind Cloudflare, you don't need IP lists: Cloudflare already checks known bots for you and exposes the result as cf.client.bot. One custom rule blocks every request that claims to be an AI crawler but isn't one. In our log study at least 43% of AI-bot requests were impostors.

1. Create the rule

1. Cloudflare dashboard → your domain → Security → WAF → Custom rules → Create rule.
2. Name it “Fake AI bots”, click Edit expression and paste the expression below.
3. Action: Block. Deploy.

(
  lower(http.user_agent) contains "gptbot" or
  lower(http.user_agent) contains "oai-searchbot" or
  lower(http.user_agent) contains "chatgpt-user" or
  lower(http.user_agent) contains "oai-adsbot" or
  lower(http.user_agent) contains "claudebot" or
  lower(http.user_agent) contains "claude-searchbot" or
  lower(http.user_agent) contains "claude-user" or
  lower(http.user_agent) contains "perplexitybot" or
  lower(http.user_agent) contains "perplexity-user" or
  lower(http.user_agent) contains "google-extended" or
  lower(http.user_agent) contains "bingbot" or
  lower(http.user_agent) contains "applebot" or
  lower(http.user_agent) contains "ccbot" or
  lower(http.user_agent) contains "duckassistbot" or
  lower(http.user_agent) contains "mistralai-user" or
  lower(http.user_agent) contains "google-cloudvertexbot" or
  lower(http.user_agent) contains "googlebot"
) and not cf.client.bot

Uses only lower(), contains and cf.client.bot, so it needs no regex. Cloudflare's custom rules are on every plan (Free: 5 rules, regex only from Business); this uses one.

2. Check it works

From your own computer, pretend to be GPTBot (403), then make a normal request (200):

curl -s -o /dev/null -w "%{http_code}\n" -A "GPTBot/1.2" https://example.com/   # 403
curl -s -o /dev/null -w "%{http_code}\n" https://example.com/                     # 200

Then watch Security → Events for a few days. Every block shows the user agent and IP; paste suspicious ones into our log checker to see whether the IP is on the operator's official list.

How it works

Cloudflare documents cf.client.bot as “whether the request originated from a known good bot or crawler”. A bot gets verified only if it identifies itself through a Web Bot Auth signature, a published IP list, or reverse DNS. The rule blocks the overlap: user agent names an AI bot, Cloudflare says it isn't a verified one.

Tokens like Google-Extended are robots.txt switches only: no real request carries them, so anything that does is fake. To stop the real bots, use robots.txt (our robots.txt generator) or Cloudflare's own AI-crawler settings.

Bots this rule checks

• GPTBot (OpenAI)
• OAI-SearchBot (OpenAI)
• ChatGPT-User (OpenAI)
• OAI-AdsBot (OpenAI)
• ClaudeBot (Anthropic)
• Claude-SearchBot (Anthropic)
• Claude-User (Anthropic)
• PerplexityBot (Perplexity)
• Perplexity-User (Perplexity)
• Google-Extended (Google)
• Bingbot (Microsoft)
• Applebot (Apple)
• CCBot (Common Crawl)
• DuckAssistBot (DuckDuckGo)
• MistralAI-User (Mistral AI)
• Google-CloudVertexBot (Google)
• Googlebot (Google)

On WordPress?

The Verified AI Bot Analytics plugin does the same check against the operators' IP lists (refreshed daily) and shows which AI bots actually visit you, also behind Cloudflare. More options in the WordPress guide.