Bing crawl server errors (403 and 5xx): fixes

Q: How do I check that a visitor is really Bingbot?

Do a reverse DNS lookup on its IP. The name should end in search.msn.com. Then look up that name and check it returns the same IP. Bing's Verify Bingbot tool does this for you.

Bingbot got a 403 or a 5xx from your server. Often a firewall blocks Bingbot. Sometimes the server is too slow. Here is how to tell and fix it.

Where you see it

In Bing Webmaster Tools, open Site Explorer and apply the filter "URLs with crawling issues (server errors)". Bing's help says it includes HTTP 403 and 5xx.

What it means

Your server refused Bingbot (403) or failed to answer (500 to 599). Bing can't read a page it can't load.

Bing's help says that if it can't crawl a site consistently, it may choose not to show it. So repeated errors cost you more than the pages that failed.

Common causes

Most likely first.

A firewall, CDN, or bot protection rule blocks Bingbot or challenges it, and returns 403.
A security plugin blocks unknown bots, or blocks by country.
The server is slow or overloaded when Bingbot visits, and returns 500, 502, 503, or 504.
A rate limit treats Bingbot as an attack.
A page throws an error only for some requests.

How to tell which cause applies

Run the Live URL test in URL Inspection. If it fails while the page loads in your browser, something treats Bingbot differently.
Open your firewall or CDN's event log. Filter by user agent bingbot and look for blocks or challenges.
Check that blocked requests really came from Bing. Do a reverse DNS lookup on the IP: a real Bingbot name ends in search.msn.com. Or paste the IP into Verify Bingbot in Bing Webmaster Tools.
Check your server's error log at the times of the errors in Site Explorer.

How to fix it

Allow verified Bingbot in your firewall, CDN, and security plugin. Match the user agent and the IP, using Bing's published Bingbot IP list or a reverse DNS check.
Turn off country blocks and rate limits for verified search engine crawlers.
If your server struggles, use Crawl Control in Bing Webmaster Tools to slow Bingbot at your busy hours. Don't block it.
Fix the code or hosting problem behind any 5xx errors.
Run the Live URL test again, then click Request indexing for your key pages.

When you can ignore it

Ignore a 403 on pages that should be private, such as admin or account pages. Better still, block those paths in robots.txt so Bingbot does not try them.

With CrawlCoach

CrawlCoach flags links on your site to pages that return a server error. It does not test Bingbot through your firewall: a test from our address can't show what Bing's real crawler gets, since a firewall may check the address too. Use your firewall's log and Verify Bingbot for that.

Questions

Why does my site work in a browser but return 403 to Bingbot?

Firewalls and bot rules often treat crawlers differently from people. A rule that blocks "bots" or "AI crawlers" can catch Bingbot too. Check your firewall's log for bingbot and allow it.

How do I check that a visitor is really Bingbot?

Do a reverse DNS lookup on its IP. The name should end in search.msn.com. Then look up that name and check it returns the same IP. Bing's Verify Bingbot tool does this for you.

Can I slow Bingbot down instead of blocking it?

Yes. Use Crawl Control in Bing Webmaster Tools to set a slower rate for your busy hours. A Crawl-delay line in robots.txt also works, but it is the same for every hour, and it overrides Crawl Control.

Related guides

Bing blockers people miss
"URLs with crawling issues (deadlinks)" in Bing
Bingbot (Microsoft's crawler)
"Server error (5xx)" (the Google Search Console reason)
"Blocked due to access forbidden (403)" (the Google Search Console reason)

Sources

Checked on October 8, 2026. Vendors change their tools and docs, so check the source if something looks different.

Bing "URLs with crawling issues (server errors)": 403 and 5xx