PerplexityBot: user agent, robots.txt, and IP ranges

What it does

Perplexity says PerplexityBot surfaces and links websites in its search results.

Perplexity says it is not used to crawl content for AI foundation models.

User agent

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)

Perplexity gives this as the full user-agent string.

robots.txt

• Perplexity lists PerplexityBot as a robots.txt tag for managing how your site works with Perplexity.
• Perplexity says changes can take up to 24 hours to show.

User-agent: PerplexityBot
Allow: /

User-agent: PerplexityBot
Disallow: /

Allowed is the default. You only need an allow group if another rule blocks the crawler. A group that names a crawler replaces your User-agent: * rules for it, so copy into it any Disallow lines you still want, such as your admin pages.

What blocking it costs you

• To appear in Perplexity's search results, Perplexity recommends allowing PerplexityBot in robots.txt and allowing its published IP ranges. Blocking it works against that.

How to verify real traffic

• Check the request's IP against Perplexity's published list for PerplexityBot.
• PerplexityBot IP ranges (JSON)

Firewalls and CDNs

robots.txt is only half of it. A firewall, CDN, or bot rule can turn a crawler away before it reads robots.txt.

• Perplexity says you may need to allow its bots in your firewall. Its advice: match the user agent and the IP together, use its JSON lists as the source of truth, and refresh them often.
• In Cloudflare, Perplexity suggests a custom rule under Security > WAF: user agent contains PerplexityBot, and the IP is in its ranges, with the action Allow.
• If you use Cloudflare, open Security Settings > Configure AI bot policies. Set Search to Allow (do not block). Cloudflare's docs say that since September 15, 2026, blocking Training also blocks crawlers that do both search and training.

With CrawlCoach

On every paid plan, CrawlCoach reads your robots.txt and flags rules that block PerplexityBot from your public pages on Google. It also asks for one public page as PerplexityBot and as a plain crawler, and tells you when a firewall likely refuses PerplexityBot.

Questions

Related guides

• All AI crawlers compared
• Perplexity-User (Visits a page a person asked for)
• OAI-SearchBot (ChatGPT search)
• Claude-SearchBot (Claude's search results)

Sources

Checked on October 8, 2026. Vendors change their tools and docs, so check the source if something looks different.

• Perplexity: Perplexity Crawlers
• Cloudflare docs: Block AI bots and AI bot policies