How To Investigate Missing AI Citations

An AI answer that cites a competitor for a prompt you care about is a symptom, not a diagnosis. The five-step workflow below narrows the cause step by step, from access to the engine's choice of sources, and each cause needs a different fix.

This workflow applies when a page you own is relevant to a tracked prompt and an AI answer cites other sources, or nothing from your domain, across repeated runs. No check guarantees a citation. Google states that meeting its requirements does not guarantee indexing or serving, and AI answers change between runs, so judge results over a panel of prompts, not one answer.

What A Missing AI Citation Actually Means

A missing AI citation means an AI answer to a prompt you care about lists other pages as its sources and not yours, which is a different problem from your brand going unnamed, and Ahrefs Brand Radar separates a third state, pages an engine retrieved but did not cite.

Getting the definition right decides where the investigation starts. It is the first question in any piece of AI search visibility work, because a mention, a citation and a retrieval each point to a different cause.

Ahrefs Brand Radar uses three states that map neatly onto a diagnosis.

• Mention. The brand appears at least once in the response text, with or without a source link.
• Citation. A page from your domain appears at least once as a cited source.
• Found but not cited. The engine checked the page while building the answer but did not cite it. Ahrefs notes that AI does not cite every page it checks.

The third state is the useful one. A page that is found but not cited has already passed access and retrieval, so the problem is the engine's choice between sources. A page that never shows up at all may have an access or relevance problem further up the chain.

Mentions and citations are separate signals and should be reported separately. A brand can be named without a link, and a page can be cited in an answer that never names the brand. This post deals with the citation side only, so for the classification workflow, see the playbook on recording AI brand mentions and AI citations as two separate signals.

Why Is My Page Not Cited By AI When It Ranks?

Ranking is a weak predictor of AI citation, since only 12% of the links cited by ChatGPT, Gemini and Copilot appeared in Google's top 10 for the same prompt across 15,000 prompts in Ahrefs' study, while Perplexity's overlap was higher at 28.6%. A page that ranks has passed Google's ranking tests, not each engine's own choice of sources.

The same Ahrefs article reports earlier Ahrefs research in which 76% of URLs cited in Google AI Overviews came from pages in the top 10. AI Overviews therefore appear to follow rankings more closely than ChatGPT, Gemini or Copilot, though the two studies used different methods. Both studies come from Ahrefs, a vendor that sells AI visibility tracking, and they describe overlap, not cause.

One published breakdown of why relevant pages go uncited comes from AgentGEO, a March 2026 paper by researchers at Virginia Tech and Zhejiang University. In its experiments, 43% of topically relevant webpages received no citation under baseline conditions. The researchers sorted the failures into four categories.

These figures come from a simulated experiment, not from live ChatGPT or Google rates. They show how failures split in the researchers' setup. Use them as an order of where to look, not as a forecast for your site.

The decision rule follows from that split. Semantic alignment dominated and technical integrity was a minority. So check access quickly, because it is binary and cheap to rule out, then spend most of the time on prompt relevance and source selection.

How To Investigate Missing AI Citations, Step By Step

Investigating a missing AI citation works best in five steps, in this order, confirm the gap across repeated runs, check crawler access, check that the answer exists in the original HTML, test prompt relevance section by section, and compare the pages the engine cites instead.

• Confirm the gap. Is the page absent in most runs of a small prompt panel? Record each run's status and the URLs cited instead.
• Check crawler access. Can each engine's search crawler reach the page, and is it indexed and snippet-eligible in Google? Record robots.txt rules, CDN actions and URL Inspection results.
• Check rendering. Is the answering passage in the original HTML? Record whether it appears before and after JavaScript runs.
• Check prompt relevance. Does one section answer the prompt in the prompt's own nouns within two sentences? Record the section and the failure type.
• Run a source selection diagnosis. What do the cited pages give the engine that yours does not? Record each cited page's type and the gap.

The order runs from the cheapest, most binary checks to the judgement-heavy ones. Access and rendering are yes or no. Prompt relevance and source selection need someone who knows the buyer and the content.

An access failure stops the investigation. If ChatGPT's search crawler is blocked or Google cannot index the page, nothing downstream matters until that is fixed and the prompt panel is re-run.

Each step below follows the same shape, namely what it rules out, the tool, the access needed, the evidence to record and its limitation. Five tools cover the work. Promptwatch handles steps 1, 2, 4 and 5. Ahrefs Brand Radar handles steps 1 and 5. Google Search Console and Cloudflare handle step 2, and Screaming Frog handles steps 2 and 3.

Permissions And Access You Need Before You Start

Most of this investigation needs only read access, since a restricted Search Console user can view Performance data and fetch URLs in URL Inspection, but the crawler checks need someone with access to the CDN, such as Cloudflare's AI Crawl Control for the site's zone.

Here is the access each tool needs, and what a lower level leaves out:

In Google Search Console, owners have full control, including adding and removing users. Full users see all data and use most tools, including URL Inspection. Restricted users can view most data and the Performance report, and can use URL Inspection in fetch-only mode. The robots.txt report exists only on domain-level properties, so a URL-prefix property will not show it.

Cloudflare's AI Crawl Control is on every Cloudflare plan. Entry-level plans detect crawlers by user agent and show the past 24 hours, while Enterprise with Bot Management adds advanced detection. Customising the response a blocked crawler receives needs a paid plan.

Promptwatch's crawler logs need a key created for each project in its settings and an integration with the CDN or host, such as Akamai, CloudFront, Fastly, Vercel or Cloudflare Logpush. Netlify log drains are an Enterprise feature on Netlify's side.

Step 1 — Confirm The Citation Is Actually Missing

Before diagnosing anything, run the same prompt and two or three rephrasings across each engine you care about and log whether your page is cited, mentioned, found but not cited, or absent, because one AI answer is a sample, not a verdict.

Build the panel from the prompt as the buyer would type it, then add two or three rephrasings that keep the same intent. Run the panel on each engine that matters to the business, and run it more than once. Record which pages are cited each time, not only whether yours is.

What it rules out: A false alarm. Answers and cited sources vary between runs and between engines, so one answer that leaves your page out proves very little. A gap that repeats across most runs on one engine is worth investigating, while a gap that appears once is noise.

Tool: Promptwatch tracks citations across ChatGPT, Claude, Gemini, Perplexity, Google AI Overviews, Grok and other engines, and shows which pages are read, cited and recommended by AI. Ahrefs Brand Radar's AI Responses report lists answers that mention and don't mention your brand next to the prompts that produced them, for each selected AI platform.

Access needed: A Promptwatch project with the panel's prompts, or an Ahrefs account with Brand Radar. Brand Radar's AI Visibility Index covers AI Overviews, AI Mode, ChatGPT, Perplexity, Gemini and Copilot. Claude sits in Custom Prompts, a separate product, which also lets you add your own panel prompts.

Evidence to record: For each run, the status of your page using Brand Radar's definitions of cited, mentioned and found but not cited, plus absent, and the full list of cited URLs. A simple log keeps the runs comparable.

Prompt | {{prompt}}
Rephrasing | {{rephrasing-number}}
Engine | {{engine}}
Run date | {{run-date}}
Our status | {{our-status}} (cited / mentioned / found but not cited / absent)
Our page | {{our-url}}
Cited instead | {{cited-urls}}
Notes | {{notes}}

Limitation: Trackers sample prompts rather than seeing every answer a buyer gets. Brand Radar's AI Share of Voice is comparative and can change a lot depending on how entities are set up, so two tools can disagree about the same brand. If the numbers do not match between platforms, read why AI visibility scores differ between platforms before acting on either.

The decision rule is simple. Move to step 2 only if your page is absent, or found but not cited, in most runs of the panel on the engine you care about. If the page is cited in most runs, the gap was noise and nothing needs fixing.

Step 2 — Can AI Crawlers Access My Page?

Check the search crawler, not the training crawler, because OpenAI says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, while blocking GPTBot only signals that content should not be used for model training. Each engine has its own crawler and rules, so check ChatGPT search, Google and the CDN separately.

What it rules out: A page the engine cannot fetch or is not allowed to show. OpenAI runs three agents with different jobs.

• OAI-SearchBot surfaces websites in ChatGPT search. A site opted out of it can still appear as a navigational link, but not in search answers.
• GPTBot crawls content that may be used to train foundation models.
• ChatGPT-User handles actions a user starts. It is not used for automatic crawling or to decide search inclusion.

OpenAI says it can take about 24 hours from a robots.txt update for its search systems to adjust. For Google AI Overviews and AI Mode, Google's rule is narrower than most checklists suggest. A page needs to be indexed and eligible to appear in Google Search with a snippet, and there are no extra technical requirements. The nosnippet, data-nosnippet, max-snippet and noindex rules limit what Google can show, while Google-Extended controls use in Google's other AI systems.

CDN rules can block crawlers without anyone editing robots.txt. From 1 July 2025, Cloudflare automatically blocks AI crawlers for new customers who sign up. Existing customers were not blocked automatically but can block with one click, so a site that moved to Cloudflare recently may be blocking AI crawlers by default.

Tool: Several tools cover the different layers.

• Google Search Console URL Inspection shows indexed status, the Google-selected canonical, crawl allowed, page fetch and indexing allowed. Its robots.txt report shows the files Google found for the top 20 hosts, with fetch status and parsing issues.
• Cloudflare AI Crawl Control lists AI crawlers hitting the zone, with Allow or Block for each one in the Crawlers tab, and metrics by crawler, operator, status code, hostname or path. Its robots.txt tab shows file health, whether files include Content Signals, and crawlers that requested disallowed paths.
• Promptwatch crawler logs show which AI crawlers, including GPTBot, ClaudeBot, PerplexityBot and Google's AI crawlers, requested which pages.
• Screaming Frog can crawl as an AI crawler user agent or a custom one, as a quick spot check.

Access needed: A full user in Search Console for URL Inspection beyond fetch-only, a domain-level property for the robots.txt report, and access to the zone's AI Crawl Control in Cloudflare.

Evidence to record: For each engine, the robots.txt rule that applies, the CDN action for its crawler, and the URL Inspection results for the page. Note the Google-selected canonical, because a different canonical means Google treats another URL as the page.

Limitation: Search Console's robots.txt report describes how Google fetches and parses the file, not how other crawlers read it. The URL Inspection live test fetches in real time but does not check sitemaps, duplicate status or manual actions, and inspections have daily limits per property.

This step checks access for one page. For the full retrieval check across a site, from robots.txt tokens to log proof, follow the guide on how to audit pages for AI retrieval.

The decision rule is strict. If any engine you care about is blocked, or the page is not indexed and snippet-eligible in Google, fix that first and re-run the panel before going further.

Step 3 — Check The Answer Exists Without JavaScript

If the passage that answers the prompt appears only after JavaScript runs, treat that as a risk worth removing as a precaution, and confirm it by crawling the original and rendered HTML side by side in Screaming Frog's SEO Spider.

What it rules out: An answer that depends on client-side rendering. JavaScript and dynamic-content failures are one of the technical integrity sub-modes in the AgentGEO taxonomy, alongside access blocking, unparseable content and a low signal-to-noise ratio. Moving the answer into the server response is a precaution grounded in that sub-mode, not a fix for a known crawler behaviour.

Tool: Screaming Frog's SEO Spider. Set Configuration > Spider > Rendering to JavaScript, and the crawler renders pages in headless Chrome. Since version 16 it stores both the original and the rendered HTML, so content or links that exist only client-side stand out, and the two versions can be viewed side by side.

Access needed: A licensed copy of the SEO Spider. Screaming Frog restricts JavaScript rendering and crawl configuration without a licence, so plan on one for this step.

Evidence to record: For the page, whether the answering passage, its heading and its key figures appear in the original HTML, in the rendered HTML only, or in neither. Record any internal links that exist only after rendering.

Limitation: Analytics will not confirm this for you. Promptwatch points out that AI crawlers do not execute JavaScript tracking scripts, so script-based analytics miss their visits entirely. Proof that a crawler fetched the page has to come from server or CDN logs, as in step 2.

The decision rule is direct. If the passage that answers the prompt appears only in the rendered HTML, move it into the server response before working on anything else.

Step 4 — How To Check Prompt Relevance

Prompt relevance was the largest failure category in the AgentGEO study, since semantic misalignment accounted for about 62% of citation failures in its simulated experiments, ahead of content quality and technical problems.

What it rules out: A page that is about the topic but does not answer the prompt. AgentGEO splits semantic alignment into four sub-modes, and they are worth testing in order.

The method takes one page and one prompt at a time. Paste the prompt, then find the single section that should answer it. Read only the first two sentences of that section. They should answer the prompt in the prompt's own nouns, without leaning on the heading or the paragraphs around them. If they need context from elsewhere to make sense, an engine that lifts passages will struggle to use them.

Tool: Promptwatch's answer gap view maps your content against AI responses to show where you cover the answer and where you don't. Treat it as an input that points to likely gaps, never as the verdict, because the judgement about intent belongs to someone who knows the buyer.

Access needed: No special system access beyond the tracker from step 1. What this step needs is a reviewer who understands the buyer's situation and has the authority to change the page.

Evidence to record: The prompt, the section that should answer it, the sub-mode that fits, and a one-line note on what the first two sentences say against what the prompt asks.

Limitation: The AgentGEO shares come from a simulated setting. Its authors report that their repairs lifted citation rates by over 40% in relative terms while changing about 5% of the content, but that is a result inside the experiment, not a promise for a live engine.

The decision rule has two branches. If no section answers the prompt directly, write that section. If one does but buries the answer, rewrite its opening so the answer comes first.

Step 5 — How To Run A Source Selection Diagnosis

A source selection diagnosis compares your page with the pages an AI engine cited for the same prompt, and when Ahrefs Brand Radar marks your page found but not cited, the engine retrieved it and preferred another source, so the gap is specificity or structure, not access.

What it rules out: A page that is accessible and relevant but loses to better sources. This is where the content quality and systemic exclusion categories from AgentGEO come in.

Tool: Two reports give you the competing set. Brand Radar's Cited Pages report shows the top sources cited in AI answers and can be filtered to domains that include competitors while excluding your own. Promptwatch finds pages AI cites for your prompts that don't mention you.

Access needed: The same tracker access as step 1. Brand Radar's Cited Domains and Cited Pages reports sit inside Brand Radar, and Promptwatch's view needs the prompts tracked in a project.

Evidence to record: For each cited page, its type and what it gives the engine that your page does not. Use the AgentGEO content quality sub-modes as the checklist.

• Information scarcity. Your page lacks the specifics, numbers or steps the cited page has.
• Content fragmentation. The answer is split across sections or pages.
• Excessive verbosity. The answer is buried in padding.
• Unstructured layout. No table, list or clear heading carries the answer.

Add systemic exclusion when your page adds nothing the cited set lacks. Competitive redundancy means the engine already has the same point from another source, and window truncation means relevant evidence was cut from the input.

Then match the type of cited page to the fix:

Limitation: Brand Radar reports that most AI citations for a brand come from third-party sites. That means many gaps cannot be closed on your own page, and the fix sits in how others write about you.

The decision rule follows from the cited set. If most cited pages are third-party, the fix is off-site. If they are competitor pages, the fix is on-page. If your page shows found but not cited in Brand Radar, the engine already retrieved it and preferred another source, so work on specificity and structure rather than access.

AI Visibility Workflow Tools For Each Step

Three of the five checks need no AI tracker at all, because Google Search Console, Cloudflare's AI Crawl Control and Screaming Frog's SEO Spider cover crawler access, indexing and rendering, while confirming the gap and comparing cited pages needs a tracker such as Promptwatch or Ahrefs Brand Radar.

Each tool covers part of the workflow:

Not every team needs every tool. A team on Cloudflare without a tracker can complete steps 2 to 4 with Search Console, Cloudflare and Screaming Frog, and run step 1 by hand by typing the panel into each engine and logging the results in the template above.

A team that needs repeatable panels across several engines needs a tracker, either Promptwatch or Brand Radar. Promptwatch puts crawler logs, citations and answer gaps in one place, which suits teams that want access and content checks side by side. Brand Radar suits teams already working in Ahrefs that want cited pages and found but not cited states next to their other data.

The manual route is slower and samples fewer runs, which matters most for step 1. For a fuller comparison of trackers on citation depth and engine coverage, see the roundup of AI citation tracking tools.

How To Measure Whether The Fix Worked

Measure a fix by re-running the same prompt panel on the same engines and comparing each page's status, because Google counts AI feature traffic within the Web search type in Search Console, so its Performance report cannot isolate AI Overview citations.

Use the step 1 log as the baseline. A fix is working when statuses move in the right direction across most runs, from absent to found but not cited, and from found but not cited to cited. Change the prompts or the engines and the comparison stops meaning anything.

Timing depends on the fix. After a robots.txt change, OpenAI says its search systems can take about 24 hours to adjust, so wait at least that long before re-running ChatGPT prompts. Content changes need the engine to fetch the page again before they can count.

Crawler logs confirm that the search crawler came back. Promptwatch crawler logs and Cloudflare's AI Crawl Control metrics show when OAI-SearchBot or another AI crawler requested the page after an access fix. Promptwatch says pages with higher crawl rates from citation-oriented bots correlate with higher citation rates, but it gives no figure. Treat a crawl return as a precondition for a citation, not as proof of one.

Search Console still has a role. It shows the page's overall Google performance, so a drop or rise there tells you about the page in Google, not about AI Overview citations.

Frequently Asked Questions

Sources