Perplexity Access Problems: Diagnose the Block Before Changing the Firewall

Trace a failed public-page request through policy, edge security and the application. Preserve crawler identity, evidence and a narrow recovery plan.

Share
Blue Ethernet cables connected to a network switch.
Illustrative image. Photo by Scott Rodgerson / Unsplash

The page opens in a browser. The agency's crawler test also succeeds. Yet the client has a log entry showing a Perplexity-labelled request was blocked.

Those observations can all be true. They describe different requests, possibly handled by different rules. The investigation needs a request trail rather than another general “AI readiness” score.

Start by asking the infrastructure owner for permission to inspect the relevant logs. This is a public-page access investigation, not authority to disable security protections.

First identify what was actually requested

PerplexityBot supports search discovery; Perplexity-User handles user-triggered visits. Perplexity documents separate IP lists and recommends matching each identity with its own ranges for WAF rules. Perplexity crawler documentation.

Put the identity on the incident record. “Perplexity blocked” is too broad to guide a change. A user-agent label alone should be recorded as an unverified claim until the technical owner has checked the request against the provider's current published information.

Also preserve the requested hostname and path. A successful visit to the homepage tells you little about a failure on a service page, asset host or redirect destination.

Build a request trail

For one relevant event, ask for the following fields where available:

FieldWhy the investigation needs it
Timestamp and timezoneAlign evidence from different systems
Hostname, path and methodIdentify the exact resource involved
User agent and source identity checksSeparate claimed identity from verified traffic
Response status and response typeDistinguish content from a block or challenge
Rule identifier and actionLocate the decision that affected the request
Origin evidenceEstablish whether the request reached the application

Avoid distributing full logs containing customer or security-sensitive data through an account-management chat. The agency usually needs a redacted event and the infrastructure owner's interpretation, not a copy of everything the server recorded.

A request that never appears at the origin may need investigation at an earlier layer. That is a working hypothesis, not proof of a particular CDN fault.

A hypothetical diagnosis

Consider an agency investigating a public installation guide. The robots policy reflects the client's intended access. A browser receives the guide. One verified crawler request receives a challenge at the edge, and the corresponding request does not appear in the origin evidence available to the team.

The next task is to inspect the rule responsible for that challenge. Rewriting the guide, adding schema or changing an unrelated robots directive would not address the evidence collected so far.

Now change the example: the request reaches the application, which redirects to an authentication page. The security team may be functioning exactly as intended. The commercial question becomes whether the guide was meant to be public, not which firewall control to bypass.

These are illustrative scenarios, not claims about how frequently either problem occurs.

Keep any exception narrower than the incident

Have the infrastructure owner propose the smallest justified adjustment. The review should specify the identity checks, affected resources, exact control involved, rollback condition and person responsible for maintaining it.

Do not turn off protection for every request claiming to be an AI crawler. Do not broaden the task to private resources because the public page happened to share a host.

Provider-specific behaviour matters. Cloudflare documents that a WAF skip rule cannot skip Bot Fight Mode, and that logging settings affect whether matching requests appear in Security Events. Cloudflare skip options.

Read the documentation for the actual deployed service and plan before proposing a rule. A screenshot from another client's dashboard is not sufficient configuration evidence.

Verify both access and retained protections

Use a short before-and-after record. Show the proposed policy, approved change and observed response. Check that unrelated protections still behave as intended. Preserve a route to restore the earlier configuration if the change has unexpected effects.

If you cannot observe an authentic request after the change, say so. A configuration review and a synthetic test may justify keeping the investigation open; they should not be relabelled as a confirmed crawler visit.

Even a successful visit leaves a separate editorial question: does the page offer accurate, useful information that supports the buyer's decision? Access diagnosis removes one possible obstacle. It cannot promise a citation or substitute for credible content.

For the page-level work that follows, create a kloovy account and evaluate one bounded task. Keep CDN and WAF changes under the client's infrastructure approval process.

Sources

  1. Perplexity: Crawlers and WAF configuration
  2. Cloudflare: Available skip options