Applebot and Applebot-Extended: robots.txt guide

Q: How do I verify Applebot?

Do a reverse DNS lookup on the IP. It should end in applebot.apple.com. Then look up that name and check it returns the same IP. Or match the IP against Apple's JSON list.

Applebot crawls for Siri, Spotlight, and Safari. Applebot-Extended is a robots.txt token that opts you out of Apple's model training.

What it does

Apple says Applebot's data powers search features across its products, including Spotlight, Siri, and Safari. Allowing Applebot lets your content appear in search results for Apple users.

Apple says Applebot's data may also help train Apple's foundation models, and may give AI answers in Siri and Search up-to-date context, with links to sources.

Applebot-Extended does not crawl pages. Apple says it only decides how data that Applebot crawled may be used. Disallow it to opt out of training Apple's foundation models.

User agent

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot)

Mozilla/5.0 (iPhone; CPU iPhone OS 17_4_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4.1 Mobile/15E148 Safari/604.1 (Applebot/0.1; +http://www.apple.com/go/applebot)

Apple gives these as examples. The string always contains Applebot, and Apple updates the browser version in it from time to time. Applebot-Extended has no user agent of its own.

robots.txt

Apple says Applebot follows robots.txt rules aimed at Applebot.
If your robots.txt does not mention Applebot but does mention Googlebot, Apple says Applebot follows the Googlebot rules.
Apple says Applebot does not follow Crawl-delay.
To keep content out of AI answers in Siri and Search without leaving search, Apple says to use the nosnippet robots meta tag.

To allow it

User-agent: Applebot
Allow: /

To block it

# Opt out of Apple model training only.
User-agent: Applebot-Extended
Disallow: /

# Also leave Apple search. Not advised.
User-agent: Applebot
Disallow: /

Allowed is the default. You only need an allow group if another rule blocks the crawler. A group that names a crawler replaces your User-agent: * rules for it, so copy into it any Disallow lines you still want, such as your admin pages.

What blocking it costs you

Blocking Applebot removes you from Siri, Spotlight, and Safari search results.
Blocking Applebot-Extended only opts you out of Apple's model training. Apple says those pages can still be in search results, and the rule is not a ranking factor.

How to verify real traffic

Apple says to check traffic with reverse DNS: real Applebot hosts end in applebot.apple.com.
Or match the IP against Apple's published CIDR list.
Applebot IP ranges (JSON)

Firewalls and CDNs

robots.txt is only half of it. A firewall, CDN, or bot rule can turn a crawler away before it reads robots.txt.

Apple says Applebot may render your pages in a browser. If robots.txt or a firewall blocks your CSS or JavaScript, it may not see the page properly.
If you use Cloudflare, open Security Settings > Configure AI bot policies. Set Search to Allow (do not block). Cloudflare's docs say that since September 15, 2026, blocking Training also blocks crawlers that do both search and training.

With CrawlCoach

On every paid plan, CrawlCoach reads your robots.txt and flags rules that block Applebot from your public pages on Google. It does not test Applebot through your firewall, since Apple verifies its crawler by address and a test from our address proves nothing. Applebot-Extended is your call, so CrawlCoach does not flag it.

Questions

What is Applebot-Extended?

A robots.txt token. Apple says it does not crawl. Disallowing it tells Apple not to use your content to train its foundation models. Your pages stay in Apple's search results.

Does Applebot follow my Googlebot rules?

Yes, when your robots.txt has no Applebot group. Apple says Applebot then follows the Googlebot instructions.

How do I verify Applebot?

Do a reverse DNS lookup on the IP. It should end in applebot.apple.com. Then look up that name and check it returns the same IP. Or match the IP against Apple's JSON list.

Related guides

All AI crawlers compared
Google-Extended (Controls Gemini training and grounding)
Bingbot (Bing search and Microsoft Copilot)
OAI-SearchBot (ChatGPT search)

Sources

Checked on October 8, 2026. Vendors change their tools and docs, so check the source if something looks different.

Applebot and Applebot-Extended: what each one controls