Claude-SearchBot

Anthropic · AI search indexer

Navigates the web to improve search result quality for Claude users.

User-agent contains

Claude-SearchBot

robots.txt token

Claude-SearchBot

Respects robots.txt

Yes, per operator docs

Official IP ranges

14 prefixes

How to verify Claude-SearchBot

Matching the user agent isn't enough, because any client can send it. A real Claude-SearchBot request comes from an IP address inside Anthropic's published list:

• https://claude.com/crawling/bots.json
  14 prefixes · list dated 2026-10-20 · fetched 2026-10-21

The quickest way to check a whole log is our free log checker. It runs in your browser and splits Claude-SearchBot requests into real and fake.

216.73.216.0/22
34.162.230.222/32
34.162.244.71/32
34.162.191.81/32
34.162.86.102/32
136.79.143.251/32
136.83.20.215/32
34.162.111.78/32
34.162.90.102/32
34.162.27.34/32
34.152.124.140/32
34.153.21.190/32
34.186.252.4/32
34.162.153.71/32

Block or allow Claude-SearchBot in robots.txt

User-agent: Claude-SearchBot
Disallow: /

Use Allow: / instead to explicitly permit it, or build a full file with the AI robots.txt generator. robots.txt is a request, not a lock: it only works for bots that honor it, and only real Claude-SearchBot traffic would.

Source: Anthropic documentation