How to optimize for AI search: one checklist for every AI engine

Optimizing for AI search comes down to four checks, in order: access (can each engine's retrieval bot fetch your pages), readability (is your content in the HTML those bots receive), citability (does the page hold passages worth quoting), and corroboration (do other sites describe you the same way). Pass all four, then measure by asking the same buyer questions across the engines every month.

Each check below has a pass/fail test you can run yourself, and links to the full guide for each engine and each fix.

How do ChatGPT, Perplexity, Claude, Gemini and AI Overviews find pages?

Each AI answer engine retrieves pages from its own crawler or search index, then writes an answer and cites some of the pages it read. So the first question is which crawler each engine uses, and what its owner says happens if you block it.

The per-engine guides go deeper: ranking in ChatGPT, getting cited by Perplexity, getting mentioned by Claude, getting cited by Gemini, getting cited in Google AI Overviews and getting cited by Microsoft Copilot.

Access: can AI retrieval bots fetch your pages?

Pass when every engine's search or retrieval bot gets a 200 and your real page: robots.txt allows OAI-SearchBot, Claude-SearchBot and PerplexityBot (plus Googlebot and Bingbot), and no firewall answers them with a 403 or a challenge. Fail when either layer blocks them. The firewall is the one people miss, because it overrides a robots.txt Allow.

The engines separate search bots from training bots, and so should your robots.txt. OpenAI says each setting "is independent of the others," so a site can allow OAI-SearchBot and disallow GPTBot. Anthropic makes the same split between Claude-SearchBot and ClaudeBot, and Perplexity says PerplexityBot "is not used to crawl content for AI foundation models." A blanket User-agent: * / Disallow: /, or a "block AI" rule that names every bot, takes you out of the answers along with the training sets.

Firewalls changed recently. Cloudflare's documentation says that from September 15, 2026, new domains get updated defaults: bots classified as Training or as Agent are blocked on pages that display ads, Search stays allowed, and mixed-purpose crawlers that combine Search and Training are blocked by every setting that blocks AI training. Agent covers "chat fetch bots," the user-triggered fetchers such as ChatGPT-User and Perplexity-User. Check the policy under Security Settings > Configure AI bot policies.

Test it:

curl -s -o /dev/null -w '%{http_code}\n' -A 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot' https://example.com/pricing
curl -s -o /dev/null -w '%{http_code}\n' -A 'Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)' https://example.com/pricing

A 200 for both passes. A 403, a 503 or a challenge page fails. Some firewalls verify bots by IP, so a spoofed user agent from your laptop can be blocked even when the real bot is allowed; confirm in your firewall's event log, where the real bots' requests show up with the IP ranges OpenAI and Perplexity publish.

Guides: robots.txt for AI bots, OAI-SearchBot vs GPTBot, Cloudflare blocking AI crawlers and checking whether AI bots crawl your site.

Readability: can the bots parse your content?

Pass when your main content, headings and product facts are in the HTML the server sends, before any JavaScript runs. Fail when the initial HTML is an empty app shell. Vercel's December 2024 analysis of AI crawler traffic found that none of the major AI crawlers it measured render JavaScript, so client-rendered content is invisible to them.

Vercel's study named OpenAI's OAI-SearchBot, ChatGPT-User and GPTBot, Anthropic's ClaudeBot, and PerplexityBot among the crawlers that don't execute JavaScript. ChatGPT's and Claude's crawlers do fetch JavaScript files (11.50% and 23.84% of their requests) but don't run them. Gemini, which uses Google's infrastructure, and AppleBot do render. So a React or Vue app that fills <div id="root"> in the browser can rank in Google and still be blank to ChatGPT, Claude and Perplexity.

Test it:

curl -s https://example.com/pricing | grep -c "a sentence from your pricing page"

A count of 1 or more passes. Zero means the text arrives only after JavaScript. The fix is server-side rendering, static generation or prerendering for the pages you want quoted; client-side rendering is fine for widgets and interactive extras.

Two more readability failures come from the same study. ChatGPT's crawler spent 34.82% of its fetches on 404 pages and 14.36% following redirects, against 8.22% and 1.49% for Googlebot. Keep URLs stable, redirect old ones in one hop, and keep your sitemap current. Google's AI features guide adds that you should make important content "available in textual form," and that you don't need new machine-readable files or AI text files to appear in AI Overviews or AI Mode, which puts llms.txt in the optional column.

Guides: do AI crawlers render JavaScript and llms.txt explained.

Citability: will AI engines quote your pages?

Pass when your key pages answer specific questions in self-contained passages, carry figures with their sources, and show when they were written and updated. Fail when the answer is spread across a page of marketing copy with no dates or numbers. The GEO study found that adding statistics, quotations and cited sources raised a page's visibility in generated answers by 30 to 40%.

That study, "GEO: Generative Engine Optimization" by Aggarwal and colleagues, tested content changes on a benchmark of queries and on Perplexity. Its best methods (Cite Sources, Quotation Addition and Statistics Addition) improved visibility by 30 to 40% on its main metric, and it measured improvements of up to 37% on Perplexity. For keyword stuffing, a classic SEO move, the authors found such methods "offer little to no improvement."

What that looks like on a page:

1. Put questions your buyers ask in the headings, and answer each one in the first 40 to 60 words under it, so the passage stands alone when an engine lifts it.
2. State what the product is, who it is for, and what it costs, in plain sentences, on pages crawlers can read.
3. Replace vague claims with numbers, and link the source of each one.
4. Show a visible "Updated" date, and match it with datePublished and dateModified in your Article structured data.
5. Use tables for comparisons and lists for steps. Bing's AI Performance guidance names "descriptive headings, concise sections, tables, and FAQ-style content."

Freshness matters most on Perplexity. A study by AI+Automation comparing the top three cited sources found Perplexity's had a median age of 32.5 days for SaaS, tech and e-commerce queries, against 108.2 days for Google. Update pages when the facts change, not just the date.

Snippet controls also decide citability. Google's robots meta specification says nosnippet prevents content from being used as a direct input for AI Overviews and AI Mode, and Bing's guidelines say noarchive keeps content out of Copilot responses. Check your pages don't carry them by accident.

Guides: writing content AI answers cite and content freshness and AI citations.

Corroboration: do other sites vouch for you?

Pass when third-party pages your buyers trust (directories and listings in your category, review sites, community threads, comparison articles) describe your product consistently with your own site. Fail when the only page that says what you do is yours. Engines lean heavily on third-party sources, and each engine leans on different ones.

The studies differ in method, so read each on its own terms:

• Yext analyzed 6.8 million citations across 1.6 million responses, from location-based consumer queries in finance, food, healthcare and retail. It found 48.73% of ChatGPT's citations came from third-party listings such as Yelp, TripAdvisor and MapQuest, 52.15% of Gemini's came from brand-owned websites, and Perplexity drew 24% of citations for unbranded subjective queries from industry-specific directories.
• Profound analyzed 680 million citations from August 2024 to June 2025. Within each engine's ten most-cited sources, Wikipedia had 47.9% of ChatGPT's share and Reddit had 46.7% of Perplexity's. Across all of Perplexity's citations, Reddit was 6.6%.

The practical list is short. Get listed accurately in the directories and review sites for your category, keep the facts consistent everywhere, answer real questions in the communities your buyers use (without advertising), and earn mentions in comparison and "best of" articles. The guide to why AI cites third-party sites and the Reddit and AI citations guide cover each channel.

How do you measure whether AI search optimization is working?

Write at least 10 questions a buyer would ask about your category, not your brand name, and ask each in ChatGPT, Perplexity, Claude, Gemini, Copilot and Google's AI features. Record for each answer whether your site is cited, whether you are named, and which sources are cited instead. Repeat monthly with identical wording, because answers vary run to run.

Supporting data from the platforms:

• Bing Webmaster Tools' AI Performance report shows citations of your pages in Copilot and Bing's AI summaries, with grounding queries and cited URLs. Bing notes a citation is not a click.
• OpenAI says ChatGPT adds utm_source=chatgpt.com to referral URLs, so ChatGPT visits show up in your analytics.
• Google says sites appearing in AI Overviews and AI Mode are included in the overall search traffic in Search Console.

The measuring AI visibility guide gives a tracking sheet and how to read the results.

What is on the AI search checklist?

Check the access and readability layers now

The first two layers are technical and testable today. Run the free scan on LaunchScaler with just your address and no account: it runs 156 checks across 6 of its 7 categories at no cost, including robots.txt rules that block OAI-SearchBot, Claude-SearchBot or PerplexityBot, a blanket rule that catches them with the training bots, a firewall that returns 403 or a challenge to AI retrieval bots, main content missing from the server HTML, malformed JSON-LD, a sitemap without lastmod, and broken links and redirect chains that waste AI crawls. The free run doesn't query the engines; the $19 full audit adds that panel, putting 10 questions to 7 answer engines and reading all 70 answers.