In this article (8 sections)
If ChatGPT didn’t mention your company just now, the most useful next question isn’t “why not” — it’s “would it mention me if I asked again, right now, with the exact same words?” Usually, the honest answer is: maybe, maybe not. AI answers aren’t a stable ranking you either hold or don’t.
They’re generated fresh each time, and research on exactly this question has found that the same prompt, asked repeatedly, rarely produces the same list of brands twice. Understanding why changes how you should think about “AI visibility” entirely — and gives you a test you can run yourself instead of guessing.
The short answer
ChatGPT (and Perplexity, and Google’s AI Overviews) don’t maintain a fixed leaderboard of businesses per topic the way a search engine maintains a ranking. Each answer is generated by a language model sampling from probabilities, drawing on whatever set of sources it retrieved for that specific request.
Ask the same question twice and you can get two different answers — not because something changed about your business between the two asks, but because the process that generates the answer is not deterministic by design.
That doesn’t mean visibility is random luck with nothing you can do about it. It means the right question isn’t “am I mentioned or not,” it’s “how often, across many tries, do I show up — and how does that compare to my competitors’ rate.”
The misconception this corrects
A lot of advice about “AI SEO” or “getting cited by ChatGPT” implicitly treats AI visibility like a search ranking: optimize the right things, and you’ll hold position #1 (or #3, or wherever) reliably. That framing comes from decades of SEO experience where rankings, while they do fluctuate, are comparatively stable day to day for a given query.
Independent research on how consistent AI recommendations actually are tells a very different story. A large-scale study published by the marketing research firm SparkToro in January 2026 ran nearly 3,000 combined queries across ChatGPT, Claude and Google’s AI-generated results, using 600 volunteers testing a fixed set of prompts across categories like kitchen knives, headphones and cloud computing.
The headline finding: there was less than a 1-in-100 chance that any two runs of the same question returned an identical list of recommended brands, and roughly a 1-in-1,000 chance the list came back in the same order. Coverage of the same study by Search Engine Land summarized it plainly — recommendation lists essentially don’t repeat.
That’s the correction worth internalizing: if your business showed up in someone’s ChatGPT answer last week and doesn’t show up in yours today, that’s not necessarily a sign anything got worse. It might be the same underlying probability distribution producing a different sample.
Why this happens, in plain terms
Three separate mechanisms combine to produce this variability, and a business owner doesn’t need to understand the math behind any of them to use the idea — just what each one means practically.
1. Sampling and “temperature.” When a language model generates text, it isn’t picking the single most likely next word every time — it’s sampling from a probability distribution over plausible next words, with some controlled randomness built in on purpose. This is what makes the model’s writing feel natural instead of robotic and repetitive, but the same mechanism means that when it’s deciding which three or four companies to mention in an answer, it isn’t deterministically selecting “the best three” from a fixed ranked list. It’s drawing from a distribution where several candidates have meaningfully similar probability, and small random variation tips which ones get named this time.
2. Retrieval variability. Before the model writes anything, most AI answer engines first retrieve a set of relevant pages or passages — similar in spirit to a search engine’s results, but usually a different, changing subset. Which pages get retrieved can shift between requests based on infrastructure-level factors (load balancing across servers, minor index updates, caching) that have nothing to do with content quality. If the retrieval step hands the model a slightly different set of source pages, the model can only mention what it was given.
3. Context and phrasing sensitivity. The exact wording of a question, prior conversation turns in the same session, and even small changes like adding “near me” or a follow-up clarification can shift both what gets retrieved and how the model summarizes it. Two people asking “what’s a good CRM for a small business” and “what CRM should a small business use” are asking functionally the same question, but there’s no guarantee the engine treats them identically.
None of these three are bugs to be fixed — they’re how these systems are built to work. Sampling variation is deliberate, retrieval systems are built for freshness and load, not perfect repeatability, and context-sensitivity is what makes conversational answers feel responsive rather than canned. The practical takeaway is simply that a single answer is a sample, not a measurement.
“But I asked it twice and got the same answer”
This is a common objection, and it’s worth addressing directly: if you’ve personally tried asking ChatGPT about your industry a couple of times and gotten a consistent answer, that doesn’t contradict the research above — it’s exactly the pattern it predicts.
A handful of casual tries isn’t enough runs to reveal the underlying variability; you need the sample size (the SparkToro study used 60-100 runs per platform per question for this reason) to see the spread.
It’s the same reason flipping a coin twice and getting heads both times doesn’t tell you the coin is unfair — you need enough flips before the pattern becomes visible. Two or three manual checks in a browser tab will usually look consistent purely by chance, even for a business with a middling presence rate.
There’s also a session-memory effect worth knowing about: if you ask a follow-up question in the same conversation, or if you’ve discussed your own business with that AI assistant earlier in the session, the model has that context available and may lean on it — which is a different thing from what a first-time customer, with no prior context, would see.
That’s exactly why the test method below insists on a fresh conversation for every single run.
A test you can run yourself
You don’t need special tools or a paid tracking platform to see this pattern with your own business. This is a method, not a promise of any particular result — run it and see what your own numbers actually look like.
What you’ll need: a notes document or spreadsheet, and 20–30 minutes.
Step 1 — Pick your question(s). Choose one to three questions a real customer would plausibly type into ChatGPT when looking for a business like yours. Use natural phrasing, not your target keyword — “who’s a good plumber in [your city],” not “best plumbing company [city] 2026.”
Step 2 — Ask each question 10 to 20 times. Open a fresh chat each time (not a follow-up in the same conversation, which can carry context from earlier turns) and ask the exact same question, worded identically. If your platform allows it, space some of the runs across different times of day or different days, since retrieval and model versions can shift over that window too.
Step 3 — Log every response. For each run, record: was your business named, was anyone else named, and in what order. A simple table works:
| Run | Your company mentioned? | Other companies mentioned | Notes |
|---|---|---|---|
| 1 | Yes | Competitor A, Competitor B | Accurate description |
| 2 | No | Competitor A, Competitor C, Competitor D | — |
| 3 | Yes | Competitor B | Named but service list outdated |
Step 4 — Calculate your presence rate. Divide the number of runs where you were mentioned by the total number of runs. That percentage — not any single answer — is your actual visibility figure for that question, on that day. Do the same for your two or three biggest named competitors using the same log, so you have a comparison point rather than a number in isolation.
Step 5 — Repeat monthly. Because the underlying system changes (model updates, index refreshes, your own site changes), a single month’s presence rate is one data point. The trend across several months, alongside whatever content or technical changes you make, is what tells you whether you’re actually moving the needle.
Reading your own results honestly
A few things to keep in mind once you have your numbers:
- 0% across 20 runs is a real, useful data point — a company that never appears in 20 tries with a directly relevant question has a genuine visibility gap worth investigating, starting with whether AI crawlers can even reach and parse the site’s content.
- A moderate, inconsistent presence rate (say, 20-50%) is normal, not a failure. Given the SparkToro findings above, even well-known brands in the study appeared in a majority but not all of relevant answers; perfect, universal mentions aren’t a realistic bar for most businesses.
- Being named inaccurately is a different problem than not being named at all, and your log will show you this distinction — note it separately, since fixing outdated or wrong information (stale service lists, a closed location, an old phone number) is usually a faster fix than trying to increase raw presence rate.
- Don’t over-read a single low run. If one out of twenty runs came back empty, that’s within the range of ordinary sampling noise the research above describes, not a signal something broke.
What actually moves the presence rate
Because retrieval still pulls from indexed, crawlable content, the levers are largely the same fundamentals GEO always comes back to:
- making sure AI crawlers can actually reach your content (checking what GPTBot, ClaudeBot and PerplexityBot receive, not just what a browser renders — the crawler traffic data in Vercel’s December 2024 analysis is a useful reference point for how much of this traffic exists and how it behaves),
- keeping facts about your business consistent everywhere they appear, and
- writing the specific facts a model would need to answer the question directly and self-containedly, rather than requiring inference across several paragraphs.
None of that guarantees a specific presence rate, and treat any promise of a guaranteed number with the same skepticism you’d apply to a guaranteed #1 Google ranking. What the test above gives you is something more useful than a guarantee: an honest, repeatable baseline for your own business, measured the way these systems actually behave rather than the way a fixed ranking would.
Frequently asked questions
Is there any way to guarantee my business gets mentioned by ChatGPT? No, and treat anyone who claims otherwise with real skepticism. Because the answer generation process has deliberate randomness built into it and depends on a retrieval step you don’t control, “guaranteed AI mentions” is about as credible as “guaranteed #1 Google ranking” — both promise certainty over a system that, by design, doesn’t produce it.
Should I stop caring about AI visibility if it’s this inconsistent? No — inconsistency at the level of a single answer doesn’t mean the underlying presence rate can’t be influenced or measured. It means the right target is a rate across many tries, not a single result, the same way a batting average matters more than any one at-bat.
Does asking ChatGPT to “remember” my business change future answers? Only within your own account and conversations, and only for you personally — it has no effect on what other users, asking from fresh sessions, will see. The test method above deliberately uses fresh sessions to avoid this exact confound.
Is this specific to ChatGPT, or does it apply to Perplexity and Google’s AI Overviews too? The research cited above covered ChatGPT, Claude, and Google’s AI-generated results together, and found meaningful inconsistency across all of them, though the degree varied by platform and query category. The test method in this article works the same way regardless of which engine you’re checking — just log each engine’s runs separately, since presence rates can differ noticeably between them.
If you want help auditing the technical and content side of your own AI visibility, see our GEO service.