Skip to content
crawlcoach.
Get my free report

OAI-SearchBot: should you allow it, and how

OAI-SearchBot finds pages for ChatGPT search. Block it and ChatGPT search won't show your pages in its answers. Here is how to allow it.

OpenAISearch crawlerAllow it to show up in AI answersChecked on October 8, 2026

By Bruno Amorim · ·

What it does

OAI-SearchBot is OpenAI's search crawler. OpenAI says it is used to surface websites in search results in ChatGPT's search features.

It is not the training crawler. Training is GPTBot. OpenAI treats each setting on its own, so you can allow OAI-SearchBot and block GPTBot.

If you allow both, OpenAI says it may use one crawl for both uses, to avoid crawling twice.

User agent

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; robots.txt; +https://openai.com/searchbot

OpenAI calls these examples: the version number may change. The second one has a robots.txt marker. OpenAI uses it when it fetches your robots.txt file, so you can tell those requests apart in your logs. Match on OAI-SearchBot, not the full string.

robots.txt

  • OpenAI lists OAI-SearchBot as a robots.txt tag that site owners use to manage how their sites work with ChatGPT search.
  • OpenAI says its systems can take about 24 hours to adjust after you change robots.txt.
To allow it
User-agent: OAI-SearchBot
Allow: /
To block it
User-agent: OAI-SearchBot
Disallow: /

Allowed is the default. You only need an allow group if another rule blocks the crawler. A group that names a crawler replaces your User-agent: * rules for it, so copy into it any Disallow lines you still want, such as your admin pages.

What blocking it costs you

  • OpenAI says sites that opt out of OAI-SearchBot will not be shown in ChatGPT search answers. They can still appear as navigational links.
  • To appear in ChatGPT search, OpenAI recommends you allow OAI-SearchBot in robots.txt and allow requests from its published IP ranges.

How to verify real traffic

  • Check that the request's IP is in OpenAI's published list for OAI-SearchBot. Anyone can copy the user agent. Only OpenAI's addresses prove it.
  • OAI-SearchBot IP ranges (JSON)

Firewalls and CDNs

robots.txt is only half of it. A firewall, CDN, or bot rule can turn a crawler away before it reads robots.txt.

  • A firewall, CDN, or bot rule can block OAI-SearchBot even when robots.txt allows it. OpenAI asks you to allow both: the robots.txt rule and requests from its IP ranges.
  • If you use Cloudflare, open Security Settings > Configure AI bot policies. Set Search to Allow (do not block). Cloudflare's docs say that since September 15, 2026, blocking Training also blocks crawlers that do both search and training.

With CrawlCoach

On every paid plan, CrawlCoach reads your robots.txt and flags rules that block OAI-SearchBot from your public pages on Google. It also asks for one public page as OAI-SearchBot and as a plain crawler. If only the OAI-SearchBot request gets a 403, a challenge, or a similar refusal, the fix list tells you a firewall rule likely blocks it.

Questions

Should I allow OAI-SearchBot?

Yes, if you want your pages in ChatGPT search answers. OpenAI says sites that block it will not be shown in those answers. Allowing it does not let OpenAI train on your pages. That is GPTBot, which you can block on its own.

What is the difference between OAI-SearchBot and ChatGPT-User?

OAI-SearchBot crawls on its own to find pages for ChatGPT search, and it follows robots.txt. ChatGPT-User visits a page only when a person in ChatGPT asks for it, and OpenAI says robots.txt may not apply to it. OpenAI says ChatGPT-User does not decide whether you appear in search. Use OAI-SearchBot in robots.txt for that.

How do I get my website to show up in ChatGPT?

Let OAI-SearchBot in, through robots.txt and your firewall. Then make pages that answer real questions clearly, with the text in the HTML your server sends. No setting guarantees a mention. ChatGPT picks what to show.

Related guides

Sources

Checked on October 8, 2026. Vendors change their tools and docs, so check the source if something looks different.

Check that AI search can reach your pages

Every paid plan checks your robots.txt for the ChatGPT, Claude, Perplexity, Bing, and Apple search crawlers, and tests whether a firewall turns away OAI-SearchBot, Claude-SearchBot, or PerplexityBot. Start with a free report on Google: up to 500 pages.

Get my free report

Free for up to 500 pages. No card needed. Sign in with the Google account you use for Search Console.

Other AI crawlers