PerplexityBot and Perplexity-User: robots.txt rules to stay citable

To stay citable in Perplexity, let PerplexityBot crawl your public pages: it is the crawler that surfaces and links websites in Perplexity's search results, and Perplexity says it is not used to train AI models. Allow it in robots.txt and in your firewall, because Perplexity recommends both, and leave Perplexity-User alone, since it fetches pages for users and generally ignores robots.txt anyway.

The quotes below come from Perplexity's crawler documentation, checked in September 2026.

What is PerplexityBot?

PerplexityBot is Perplexity's search crawler. Perplexity says it is "designed to surface and link websites in search results on Perplexity" and that "it is not used to crawl content for AI foundation models." It is the agent that decides whether your pages are in the pool Perplexity can cite.

Its full user-agent string is:

Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)

Perplexity's own recommendation is short: "To ensure your site appears in search results, we recommend allowing PerplexityBot in your site's robots.txt file and permitting requests from our published IP ranges." Both halves matter. The robots.txt rule is a permission; the IP ranges are what your firewall needs to let the permission through.

Because Perplexity says PerplexityBot is not used for model training, blocking it does not protect your content from training. It only removes you from Perplexity's results. That makes it a different decision from blocking GPTBot or ClaudeBot, which are training crawlers.

What is Perplexity-User, and does it follow robots.txt?

Perplexity-User visits a page when a user's question needs it. Perplexity says it "supports user actions within Perplexity," may "visit a web page to help provide an accurate answer and include a link to the page in its response," and is not used for crawling or for training. On robots.txt, the documentation is direct: it "generally ignores robots.txt rules."

The reason Perplexity gives is that "a user requested the fetch." OpenAI takes the same position for ChatGPT-User. The practical effect is that a User-agent: Perplexity-User Disallow line is not a dependable control. If you need to keep it out of a path, that has to happen at the server or firewall, matched against its published address list.

Its user-agent string:

Mozilla/5.0 (compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)

For a product site, blocking it rarely makes sense. When a buyer asks Perplexity about your pricing, Perplexity-User is what reads your pricing page and links it in the answer.

What robots.txt rules keep you citable in Perplexity?

Give PerplexityBot and Perplexity-User a group that allows your public pages and repeats your private-path rules. Under RFC 9309, a crawler that finds a group naming it ignores the User-agent: * group, so any path you want kept private must be listed again inside the Perplexity group.

User-agent: PerplexityBot
User-agent: Perplexity-User
Allow: /
Disallow: /app/
Disallow: /account/
Disallow: /api/

If your * group already allows everything public, you do not need a Perplexity group at all. You need one when a broader rule would catch PerplexityBot, which is how sites end up blocking Perplexity without meaning to.

Matching on the token is case-insensitive, so perplexitybot works, but the spelling must be exact. After any change, Perplexity says "it may take up to 24 hours for our systems to reflect changes." The robots.txt guide for AI bots has a complete file covering every operator.

Which rules fit which goal?

Decide what you want from Perplexity first, then write the smallest set of rules that does it. Because Perplexity says neither agent is used for model training, there is no "block training, keep citations" split to make here, unlike with OpenAI or Anthropic. The choice is between being a source and not being one.

The last row is the only one that needs the firewall to do real work, because robots.txt alone does not reliably stop Perplexity-User. Anything that must never be fetched by any agent belongs behind authentication, not behind a Disallow line: robots.txt rules, in the words of RFC 9309, "are not a form of access authorization."

How do you verify PerplexityBot by IP?

Check the source address against Perplexity's published lists: https://www.perplexity.com/perplexitybot.json for PerplexityBot and https://www.perplexity.com/perplexity-user.json for Perplexity-User. Each file holds a prefixes array of ipv4Prefix entries. A request that says PerplexityBot from any other address did not come from Perplexity.

python3 - <<'EOF'
import ipaddress, json, urllib.request
ip = ipaddress.ip_address("203.0.113.7")  # the address from your log
data = json.load(urllib.request.urlopen("https://www.perplexity.com/perplexitybot.json"))
nets = [ipaddress.ip_network(p["ipv4Prefix"]) for p in data["prefixes"] if "ipv4Prefix" in p]
print("PerplexityBot" if any(ip in n for n in nets) else "not PerplexityBot")
EOF

Perplexity advises combining both checks in firewall rules: user-agent contains PerplexityBot or Perplexity-User, and source IP is in the published ranges. Matching the user-agent alone lets anyone who copies the string through; matching the IP alone breaks when Perplexity adds addresses. Perplexity also recommends automating updates to the lists, since the ranges change.

Why can a firewall hide you from Perplexity when robots.txt is open?

A firewall answers the request before your site does, so robots.txt never gets a say. If a WAF rule, a bot-protection mode or a CDN setting returns a 403 or a challenge page to PerplexityBot, Perplexity gets no content from you, even though your robots.txt allows it.

On Cloudflare, three settings can cause it. Bot Fight Mode issues challenges to traffic matching known bot patterns, and Cloudflare's documentation says it "cannot be customized, adjusted, or reconfigured via WAF custom rules," so a skip rule will not exempt PerplexityBot from it. The AI bot policies under Security Settings let you block the Search, Agent and Training behaviours, and Cloudflare defines Search as crawlers that "collect or index your content to answer questions about it later," which is PerplexityBot's job. And from September 15, 2026, new Cloudflare domains block Agent traffic on pages that display ads by default, and Cloudflare's Agent category includes "chat fetch bots" of the kind Perplexity-User is.

Test what Perplexity's agents actually receive:

1. Request a page as PerplexityBot and print the status: curl -s -o /dev/null -w "%{http_code}\n" -A "Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)" https://yoursite.com/.
2. Repeat with the Perplexity-User string.
3. Check headers with curl -sI for cf-mitigated: challenge, which Cloudflare sets on every challenge page.
4. Read your firewall or security event log for requests from Perplexity's published addresses and the action applied.

Your laptop is not on Perplexity's address list, so a correctly written rule that allows only verified Perplexity addresses may still block your curl test. The firewall log is the real answer. Perplexity's documentation walks through the rule for Cloudflare and AWS WAF, and the guide to Cloudflare blocking AI crawlers covers every Cloudflare setting involved.

Does PerplexityBot render JavaScript?

No, by Vercel's measurement. Vercel's study of AI crawler traffic listed PerplexityBot among the crawlers that do not render JavaScript and counted 24.4 million PerplexityBot fetches across its network in one month. A page whose content appears only after scripts run reaches Perplexity as an empty shell.

Search the raw HTML for a sentence from your page: curl -s https://yoursite.com/pricing | grep -c "per seat". A 0 means Perplexity cannot quote it. Server rendering, static generation or prerendering fixes it. Run the same test for your <title>, your meta description and any JSON-LD block: if a script injects them after load, a crawler that does not execute JavaScript never sees them either, and Perplexity has to describe your page from whatever text is left.

How much does Perplexity send back?

More than other AI platforms per page crawled. Cloudflare's 2025 Radar review found Perplexity had the lowest crawl-to-refer ratios of the major AI platforms, with peaks generally below 400 crawls per referral and below 200 from September 2025 onwards. Allowing PerplexityBot costs you crawl load and returns visits at a better rate than the other platforms Cloudflare measured. To see those visits yourself, filter your analytics for referrals from perplexity.ai and compare them month by month with the PerplexityBot requests in your server logs.

Once access is right, the guide to getting cited by Perplexity covers the content side, and the list of AI crawler user agents has the tokens and IP lists for every other operator.

Check whether Perplexity's crawler can reach you

A robots.txt line is easy to read; a firewall rule that challenges one user agent is not. Run the free scan on LaunchScaler: it takes a URL, needs no account and runs 156 checks across 6 of its 7 categories at no cost. Its AI visibility checks flag robots.txt rules that block PerplexityBot, OAI-SearchBot or Claude-SearchBot, fail a wildcard block that catches them, request your page with a retrieval bot's user agent to catch a 403 or a challenge that a browser never sees, and check whether your main content is in the raw HTML Perplexity reads.