Skip to content
crawlcoach.
Get my free report

Bingbot: the crawler behind Bing and Microsoft Copilot

Bingbot crawls for Bing search, and Bing's guidelines say Copilot uses the same crawling and index. Block it and you lose both.

MicrosoftSearch crawlerAllow it to show up in AI answersChecked on October 8, 2026

By Bruno Amorim · ·

What it does

Bing says Bingbot is its standard crawler and handles most of its crawling.

Bing's guidelines say Bing and Copilot search experiences rely on the same crawling, indexing, and ranking. A page Bingbot can't crawl can't be cited in Copilot answers.

Bing Webmaster Tools has an AI Performance report that shows which of your pages Copilot and Bing's AI summaries cite.

User agent

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/W.X.Y.Z Safari/537.36
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)

Bing says W.X.Y.Z is the current Microsoft Edge version Bingbot uses, and it changes. Match on bingbot.

robots.txt

  • Bing says robots.txt tells its crawlers how to interact with your site.
  • Bingbot follows Crawl-delay, and Bing says that line always wins over the Crawl Control setting in Bing Webmaster Tools.
  • There is no separate Bing token for AI use. Bing's guidelines say to use noindex to keep a page out of Bing search and Copilot, and that noarchive keeps content out of Copilot answers.
To allow it
User-agent: bingbot
Allow: /
To block it
User-agent: bingbot
Disallow: /

Allowed is the default. You only need an allow group if another rule blocks the crawler. A group that names a crawler replaces your User-agent: * rules for it, so copy into it any Disallow lines you still want, such as your admin pages.

What blocking it costs you

  • Blocking Bingbot keeps your pages from being crawled for Bing search. Bing's guidelines list blocking Bingbot in robots.txt as something to avoid.
  • Copilot answers rely on the same crawl, so you lose those citations too.

How to verify real traffic

  • Do a reverse DNS lookup on the IP. The name should end in search.msn.com. Then do a forward lookup on that name and check it returns the same IP.
  • Or paste the IP into Verify Bingbot in Bing Webmaster Tools, or match it against Bing's published IP list. Bing says to refresh that list daily.
  • Bingbot IP addresses (JSON)

Firewalls and CDNs

robots.txt is only half of it. A firewall, CDN, or bot rule can turn a crawler away before it reads robots.txt.

  • Strict bot protection and "block AI bots" settings can catch Bingbot. Look for bingbot in your firewall's event log, and allow it when the IP checks out.
  • To slow Bingbot instead of blocking it, use Crawl Control in Bing Webmaster Tools.

With CrawlCoach

On every paid plan, CrawlCoach reads your robots.txt and flags rules that block Bingbot from your public pages on Google. Connect Bing Webmaster Tools, and CrawlCoach also lists the pages on Google that Bing has not crawled. On Pro and Agency, it sends those pages to Bing for you. CrawlCoach does not test Bingbot through your firewall, since Bing verifies its crawler by address.

Questions

Does blocking Bingbot remove me from Microsoft Copilot?

Bing's guidelines say Copilot relies on the same crawling and index as Bing search. A page Bingbot can't crawl can't be in either.

How do I keep a page out of Copilot but stay on Bing?

Bing's guidelines say noarchive keeps content out of Copilot answers and grounding results, and nocache limits Copilot to the URL, title, and snippet. noindex removes the page from both Bing and Copilot.

Why is Bingbot not crawling my site?

Check robots.txt for a Crawl-delay or a bingbot group, your firewall for blocks, and Bing Webmaster Tools for a submitted sitemap. The Bing guides cover each status Bing shows.

Related guides

Sources

Checked on October 8, 2026. Vendors change their tools and docs, so check the source if something looks different.

Check that AI search can reach your pages

Every paid plan checks your robots.txt for the ChatGPT, Claude, Perplexity, Bing, and Apple search crawlers, and tests whether a firewall turns away OAI-SearchBot, Claude-SearchBot, or PerplexityBot. Start with a free report on Google: up to 500 pages.

Get my free report

Free for up to 500 pages. No card needed. Sign in with the Google account you use for Search Console.

Other AI crawlers