Why AI Visibility Scores Differ Between Platforms

Two AI visibility reports can put the same brand at very different scores in the same week. Before treating the gap as a win or a loss, it helps to know what each number counts.

This applies to any brand-mention or citation score from AI visibility tools across ChatGPT, Gemini, Perplexity, Claude, Copilot, AI Overviews and AI Mode. A consistent setup removes measurement noise, not platform drift, so scores still move when the engines themselves change.

Why Do AI Visibility Scores Differ Between Platforms?

AI visibility scores differ between platforms because each tool samples different prompts, runs them a different number of times and divides by a different total. A nine-platform methodology audit by AuthorityTech, a PR firm, found three platforms publishing conflicting definitions of their own headline metric.

The practical consequence is simple. Two scores for one brand usually measure two different things, so a gap between them says more about the tools than about the brand. Anyone reporting on AI search visibility needs to know which of these causes sits behind a number before acting on it.

The five causes, in the order to check them:

• Prompt selection: Tools score different questions, from search-backed databases of millions of prompts to a short list the team writes.
• Sampling: AI answers rarely repeat, so the number of runs per prompt decides how far a score swings.
• Metric definitions: One tool divides by all answers, another by answers naming any tracked brand, another by modelled impressions.
• Engine differences: AI Overviews and AI Mode cite different URLs for the same query, and a blended score depends on which engines it mixes.
• Collection path: An API call and the consumer interface can search the web differently for the same prompt.

Each cause can move a score on its own, and they stack. Two tools that differ on prompts, runs and denominator can disagree widely while both report accurately.

That makes a gap between two tools a measurement question first and a performance question second. The sections below take each cause in turn, then set out which score to trust and what to ask a vendor.

How This Guide Separates Evidence From Opinion

This guide treats a figure as a measured finding only when its study publishes a sample size, such as SparkToro's 2,961 runs by 600 volunteers, and labels vendor-run research and practitioner views as such.

Every claim in the post carries one of four labels:

• Measured finding: A result from a study that states its sample, such as SparkToro's volunteer study or the arXiv paper on repeat runs.
• Vendor definition: How a tool's own documentation says its score is calculated.
• Vendor research: A study run by a company that sells AI visibility tracking, here Profound, Ahrefs and Genezio.
• Practitioner view: An opinion or recommendation, quoted as the person's own.

How Does Prompt Selection Change An AI Visibility Score?

Prompt selection changes an AI visibility score because database tools such as Ahrefs Brand Radar score a brand across more than 525 million search-backed prompts, while tracked-prompt tools such as Peec AI score only the questions a team writes itself. The two models answer different questions about one brand.

Database prompts. Ahrefs models Brand Radar's prompts on real search behaviour from People Also Ask and its keyword database rather than writing them by hand. Semrush builds its database of over 317 million prompts and responses from AI search clickstream data and Google keyword data. A database score shows how often a brand appears across the questions a market asks at volume.

Tracked prompts. Peec AI tracks a set number of prompts the team writes. Rankscale scores search terms the team writes, with no cap on how many. Brand Radar also lets teams add custom prompts alongside its database. A tracked-prompt score covers only the questions on the list, so its value depends on building a prompt tracking list that mirrors how buyers ask.

The same brand scores differently across the two models for three reasons:

• Different questions: A database follows what people search at volume, while a tracked list holds the buying questions a team chooses.
• Different intent mix: A brand strong on early research questions can look weak on a list built from comparison prompts, and the reverse.
• Different competitors: Each question set surfaces its own rivals, so any score measured against competitors shifts with the set.

Ahrefs states the limits of its own approach plainly. Brand Radar's coverage is strongest in English, its set covers high-demand questions and may miss long-tail prompts, and its metrics are directional. Because a search-backed set can miss long-tail buyer prompts, a tracked list is the way to cover the specific questions a team needs to win.

Decision rule: Use a database score for market-share context across many questions, and a tracked-prompt score for the buyer-intent questions the team needs to win.

How Do Sampling Differences Affect AI Visibility Scores?

Sampling differences move AI visibility scores because AI tools rarely repeat themselves, and across 2,961 runs of 12 prompts in ChatGPT, Claude and Google's AI Overview, SparkToro's study of 600 volunteers found the same brand list appeared less than once in 100 runs.

The instability held whatever the question. List length ranged from two or three names to ten or more, so a brand can drop out of an answer simply because the engine named fewer brands that time. The volunteers used each engine's default experience, which is what a buyer sees.

Fixed settings do not remove the randomness either. In an arXiv paper, Atil and colleagues set temperature to zero with a fixed seed and ran eight tasks ten times each on GPT-3.5 Turbo, GPT-4o, Llama-3-70B, Llama-3-8B and Mixtral-8x7B. They found accuracy varied by up to 15% across runs, and none of the models gave repeatable accuracy across all tasks.

A score built from one run per prompt per engine is therefore one noisy sample. Tools differ in how many samples they take:

• Peec AI runs each tracked prompt daily on each of its models, and its monthly answer allowance reflects that count.
• Rankscale lets the team schedule runs anywhere from hourly to monthly.
• Ahrefs Brand Radar checks each custom prompt daily on every selected platform.
• Profound runs its prompts once a day.

Daily runs over weeks pool many samples per prompt, which narrows the swing. A one-off audit that runs each prompt once does not, and its result can change completely on a second pass.

Position-weighted scores carry an extra risk. Rankscale's formula discounts a brand's appearances by its average position in the answer. When the same list in the same order appears less than once in 1,000 runs, a score that weights by position inherits that instability. This is a judgement drawn from the order finding, not a measured result, but it means movement in a position-weighted score is best read as direction rather than precise change.

Decision rule: Check how many runs sit behind each prompt and engine before trusting any single reading, and prefer pooled readings over weeks to a one-off snapshot.

How Do Metric Definitions Differ Between AI Visibility Tools?

AI visibility tools define the same-sounding metric differently, so Peec AI divides brand-mentioning responses by all responses, Rankscale discounts appearances by average position, and Ahrefs Brand Radar weights share of voice by modelled search impressions, while the nine-platform audit found Profound publishing two denominators for one Visibility Score.

The AuthorityTech audit lists the denominators in use. They include all responses, brand-mentioning responses, mentions, citations and estimated impressions. Scores also differ by unit, sampling, aggregation, deduplication, prompt set, engine, locale and cadence.

The audit's own illustration shows how far one choice moves the number. Take 100 answers, 50 of which name the brand and 60 of which name any brand. Divide by all answers and the brand scores 50%. Divide by answers that name any brand and it scores 83.3%. Both figures are correct, and they measure different things.

The five tools' published definitions, side by side:

Peec AI and Rankscale publish formulas a team can rebuild. Peec AI's Visibility Score is responses mentioning the brand divided by total responses, times 100, with no weighting. Rankscale's simplified formula multiplies appearances over total runs by a position discount, so a lower average position reduces the score. Rankscale counts one answer as one mention even when the brand is named several times.

Two of the definitions cannot be rebuilt from the documentation. Semrush's metric page describes its 0-100 score as a benchmark of how often a brand appears in AI answers compared with competitors, and its data page says the score combines Topic Coverage and Mention Consistency, without giving a calculation. Profound's documentation calls its visibility score the fraction of answers a brand appears in, traffic-weighted, without publishing the weighting.

Ahrefs is open about its estimate. It defines AI share of voice as responses that mention a brand divided by responses that mention any tracked brand. Brand Radar then weights prompts by AI-adjusted search volume, so its share becomes estimated impressions over total impressions. In Ahrefs' words, "Impressions are a modeled estimate of exposure, not a count of people who saw an answer."

That is also why AI share of voice and AI visibility are not the same metric. Visibility usually divides by all answers, while share of voice divides by answers or impressions that name any tracked brand.

Same name, different metric. Compare definitions before comparing numbers.

Why Do Scores Differ Between AI Engines?

Scores differ between AI engines because the engines cite different sources for the same query, and Ahrefs research on 540,000 query pairs found Google's AI Overviews and AI Mode cited the same URLs only 13.7% of the time. That is vendor research, since Ahrefs sells Brand Radar.

The study used September 2025 US data across 730,000 response pairs. It found:

• AI Overviews and AI Mode cited the same URLs 13.7% of the time.
• Their answers still averaged 86% semantic similarity, so the two said similar things from different sources.
• AI Mode responses included 2.5 times more people and brand entities.
• 32.8% of responses named no person or brand at all.

Those gaps sit inside two products from one company. A brand can be cited in one and absent from the other on the same query, and an engine that names more brands per answer gives each brand more chances to appear.

A multi-engine score therefore depends on which engines it includes and how it weights them. Ahrefs notes that in its impression weighting ChatGPT carries far more weight than Gemini or Perplexity. A change on ChatGPT moves Brand Radar's blended share more than the same change on Perplexity.

Decision rule: Read the score for each engine before the blended score, and check which engines and weights sit inside any blend.

Does API Or Interface Collection Change AI Visibility Results?

Collection path changes AI visibility results because the API and the consumer interface search the web differently, and Genezio's study of 3,645 UK banking conversations found zero shared search queries between the GPT-5.2 API and ChatGPT.com. Genezio sells AI visibility tracking, so this is vendor research.

Over 27 days, the two paths diverged on sources as well as queries:

• The API touched more unique URLs, 3,562 against 2,843 for the interface.
• ChatGPT.com relied on comparison and aggregator content for 32.2% of source uses, against 11.0% via the API.
• The API cited bank-owned domains about 4.4 times more often.

A tool that collects through the API can therefore report a different brand picture from one that reads the consumer interface, even on the same prompt. For a brand that depends on comparison sites, the gap is large, because the interface leans on that content nearly three times as much.

Some vendors state their path. Ahrefs runs Brand Radar's prompts through free public web interfaces, not APIs. Semrush says it captures responses from real requests, not LLM APIs.

Decision rule: Ask every vendor which collection path it uses for each engine, and do not compare an API-based score with an interface-based one.

AI Visibility Expert Insights On How Many Runs Are Enough

Practitioners agree a single AI answer proves little but disagree on how many runs are enough, with SparkToro's Rand Fishkin recommending at least 60 to 100 runs per prompt and Profound economist Jennifer Zou reporting that one run a day lands within about 2 percentage points of ten runs a day.

Independent practitioner view: Fishkin and O'Donnell conclude that "visibility % across dozens to hundreds of prompts run multiple times is a reasonable metric." They recommend running prompts usually at least 60 to 100 times and averaging the results. Fishkin is blunter about position metrics.

That is Fishkin's opinion. The measured finding behind it is that the same list in the same order appeared less than once in 1,000 runs.

Vendor researcher view: Zou analysed 753 prompts across 7 US platforms, 5,271 prompt configurations, comparing about 129,000 runs made once a day with 860,000 made ten times a day. Once a day landed within about 2 percentage points of the ten-times-a-day visibility reading, 78.7% against 80.4%. Ten runs a day cut day-to-day noise in citation share by roughly 40% but improved visibility precision by only about 10%.

Profound runs its own prompts once a day, so the study tests the cadence its product uses. That interest is worth knowing when weighing the result.

Where they agree: Both treat single answers as unreliable, and both measure visibility as a percentage across many runs. They differ on how many runs are enough.

The positions are closer than they sound. Daily runs over two to three months pool 60 to 90 samples per prompt and engine, inside the range Fishkin recommends.

AI Visibility Trends And Why Scores Move Over Time

AI visibility scores move over time even when a brand changes nothing, because the engines drift from day to day and, as Profound's run-frequency analysis of 753 prompts found, that drift sets a precision floor that extra same-day runs cannot remove.

Three things move a score with no change on the brand's side:

• Platform drift: Profound's study found day-to-day drift does not shrink with more same-day runs.
• Refresh windows: Brand Radar updates ChatGPT, Perplexity, Gemini and Copilot data monthly over a 90-day window, while AI Overviews and AI Mode update continuously. Semrush runs Prompt Tracking daily and Brand Performance weekly.
• Setup changes: Ahrefs notes that changing prompts, platforms, competitors, location or timing changes the percentage, which breaks like-for-like comparison.

Refresh windows also explain why two tools can show the same change in different weeks. A monthly update over a 90-day window smooths a drop that a daily tracker shows at once.

Reading trends this way is part of why AEO and GEO run on a different scoreboard from classic SEO.

Decision rule: Treat a move as a trend only when it persists across several periods on an unchanged setup, which is also Ahrefs' advice.

Which AI Visibility Score Should You Trust?

The AI visibility score to trust is the trend from one tool on an unchanged prompt set, engine list and competitor list, because as Ahrefs puts it, two tools can report different figures for the same brand "without either being wrong." A raw cross-tool comparison is not a trend.

Use these rules before reporting a number:

• Report one tool's trend on a fixed prompt set, fixed engines and fixed competitors.
• Compare across tools only after matching prompt set, engines, locale, cadence and denominator.
• Prefer metrics whose formula the vendor publishes.
• Read each engine's score before any blended score.
• Treat position-weighted metrics as directional.
• Count a change as real only when it holds across several periods.

Before buying or reporting, ask each vendor these questions, built on the dimensions in the AuthorityTech audit:

• What does the score divide by?
• Does it count answers, mentions or citations?
• How many times is each prompt run on each engine?
• How are runs combined across days and engines?
• Are repeat mentions in one answer counted once?
• Where do the prompts come from?
• Is each engine read through the API or the consumer interface?

Market context: For how a brand shows up across the many questions a market asks, a database score such as Ahrefs Brand Radar or Semrush's AI Visibility score fits. Read it with Ahrefs' own caution that search-backed sets can miss long-tail prompts.

Buyer questions: For the specific questions that lead to a purchase, a tracked-prompt score such as Peec AI or Rankscale fits, because it scores only the prompts the team writes. Peec AI's unweighted formula is the easier of the two to reconcile, while Rankscale's position discount is best read as direction.

For the workflow that turns these rules into a report, see how to measure AI visibility without double-counting.

To weigh tools on engine coverage and data collection before buying, compare AI visibility platforms side by side.

FAQ

Sources