How to Set an AI Visibility Baseline That Stays Comparable
An AI visibility baseline is only useful if you can rerun it the same way. How to pick prompts, platforms, and sample sizes, and log results.
An AI visibility baseline is a recorded snapshot of how often AI assistants mention your brand, which sources they cite, and how they describe you, measured across a fixed set of buyer prompts with repeated runs. To set one, freeze the prompt set, platforms, and testing conditions, then document them so every later run is comparable.
By the end of this process you will have a versioned prompt set, a run log with every answer recorded, the share of runs in which your brand appears for each prompt and platform, and a methodology document that lets anyone repeat the audit later. Before you start, gather a list of your main competitors, every variant of your brand name (including common misspellings and product names), access to the AI assistants you plan to test, a spreadsheet, and raw buyer language from sales calls, support tickets, and review sites. Budget a few working sessions spread across a week, since spreading the runs out is part of the method.
How do you set an AI visibility baseline?
You set one by fixing what you test and how you test it, sampling each prompt several times on each platform, and logging every answer in the same structure. The steps:
Collect buyer questions: Pull the actual wording prospects use from call transcripts, support tickets, Search Console queries, Reddit threads, and review site category pages.
Build the prompt set: Turn those questions into prompts, tag each one by intent and persona, and freeze the list as version one.
Choose the platforms: Pick the assistants your buyers actually use and decide which interface you will test on each.
Set the testing conditions: Decide account state, location, language, and whether memory or chat history is on, then keep those identical for every run.
Run each prompt repeatedly: Ask every prompt several times per platform in fresh sessions, spread across different days.
Log every answer: Record mentions, position in any list, cited URLs, sentiment, and factual errors for each run in its own row.
Calculate appearance by prompt: For each prompt and platform, count how many runs mentioned you out of the total runs.
Write the methodology document: Capture every decision from the steps above so a colleague could repeat the audit without asking you anything.
Steps 2, 3, 5, and 6 need the most judgment, so the sections below cover them in detail.
How many prompts belong in a baseline prompt set?
Enough to cover every way a buyer might discover, compare, and vet you, which for most brands lands somewhere between a few dozen and a couple hundred prompts. Coverage of intents matters more than the raw count. A set of two hundred variations on "best CRM" tells you less than fifty prompts that span the whole buying journey.
Group prompts into intent types and keep each type visible in your reporting:
Discovery: "What payroll software works for a restaurant with tip pooling?"
Comparison: "Gusto vs a restaurant-specific payroll tool for a three-location group"
Alternatives: "What are alternatives to [competitor] for small restaurants?"
Problem-led: "How do I handle payroll for servers who work at two locations?"
Brand-direct: "Is [your brand] good for restaurants with seasonal staff?"
Keep brand-direct prompts in their own bucket. If you ask an assistant about your brand by name, it will almost always mention you, so mixing those prompts into a headline number makes visibility look better than it is. The discovery and problem-led prompts show whether assistants bring you up when nobody asked about you.
Write prompts the way buyers talk, with the constraints they mention. Real questions include company size, industry, budget, and the tool they are leaving. "Best project management software" is a search engine habit. "Which project management tool should a ten-person design agency use if we hate Jira" is closer to how people type into ChatGPT, and it produces a different shortlist.
Tag each prompt with an ID, intent type, persona, funnel stage, and whether it names any brand. Those tags let you slice results later without rewriting anything.
Which AI platforms should you test?
Test the assistants your buyers use, and record how each one built its answer, because the platforms draw on different mechanisms. Ask your sales team which tools prospects mention, and check your analytics for referral traffic from domains like chatgpt.com and perplexity.ai, which shows where people are already clicking through from AI answers.
Platform | How it builds answers | What to record beyond mentions |
|---|---|---|
ChatGPT | Draws on training knowledge and runs a live web search when it judges one useful | Whether a search ran, and the linked sources if it did |
Gemini | Can ground answers in Google Search results | Sources shown and whether they differ from the organic results |
Perplexity | Searches the web by default and lists numbered sources | The full source list, including third-party pages that name you |
Claude | Draws on training knowledge and can search the web when that capability is on | Whether a search ran, which changes how you interpret a miss |
Microsoft Copilot | Searches the web and cites sources alongside its answer | Linked sources and any shopping or product panels |
Google AI Overviews and AI Mode | Built from Google's index, shown within search results, and AI Overviews do not appear for every query | Whether an AI Overview appeared at all, plus the linked sources |
The "did it search" column matters more than it looks. When an assistant answers without searching, your presence or absence reflects what the model learned in training, which is the territory of Generative Engine Optimization (GEO): Wikipedia, Reddit, Quora, and publications that repeat the same names over time. When it searches, your presence reflects retrieval, which is Answer Engine Optimization (AEO): whether pages that rank for the question today mention or link to you. Logging this lets you route each gap to the right fix later.
Choose an interface per platform and stick with it. Consumer apps can add web search, memory, and their own instructions, so answers from an API often differ from what a buyer sees in the app. For a baseline that reflects buyer experience, test the consumer interface in a logged-out session or a clean account with memory and history off.
How do you account for response variability?
You account for it by sampling each prompt several times and reporting how often you appear, never a single answer. AI answers are non-deterministic. The same prompt can return a different shortlist an hour later because of sampling randomness, small changes in what a search retrieves, or a quiet model update. A brand does not hold a fixed rank in these answers, so a baseline built on one run per prompt is a coin flip recorded as fact.
Five runs per prompt per platform is a reasonable floor for a first baseline. For the prompts closest to revenue, run more, since extra runs tighten your sense of how often you really appear. Spread runs across at least three different days, at different times, so a single bad retrieval day does not define your starting point.
Record each run as its own row, then summarize. A prompt where you appeared in four of five ChatGPT runs and zero of five Perplexity runs tells you something specific. A prompt where you appeared in two of five runs on every platform tells you that you are on the edge of the shortlist and that small gains in third-party coverage could move it. Position within a list can be logged too, but treat it as noisy. Mention order shifts between runs even when the set of names stays the same.
What should you record for each answer?
Record whether you were mentioned, whether you were cited, how you were described, and whether the description was accurate. Mentions and citations are separate signals. A mention is your brand named in the answer text. A citation is a linked source. Assistants can cite your blog post for a definition without naming you as an option, and they can recommend you by name with no link at all, usually because the recommendation came from training.
For sentiment, write coding rules before you look at any answers, so you do not drift as you go. A simple scale works: positive when the answer recommends you or describes you favorably, neutral when you are listed without judgment, negative when the answer warns against you or leads with drawbacks, and mixed when it does both. Paste the exact phrase that drove the code into the log. "Good for small teams but limited reporting" is more useful in six months than the word "mixed".
Add an accuracy flag. If an assistant states the wrong pricing model, names a discontinued product, or confuses you with a similarly named company, record the claim verbatim. Those errors are often fixable at the source and are worth tracking as their own line.
The baseline worksheet structure
Build the worksheet as one spreadsheet with three tabs: a prompt register, a run log, and a methodology tab. Copy the structure below into Google Sheets or Excel and you have the template.
The prompt register holds one row per prompt with these columns: prompt ID, exact prompt text, intent type, persona, funnel stage, brand named (yes or no), and prompt set version.
The run log holds one row per run:
Column | What to record | Example entry |
|---|---|---|
Run ID | Unique ID for the run | R-0142 |
Prompt ID | Links back to the prompt register | P-017 |
Platform and interface | Assistant plus app, web, or API | Perplexity, web app |
Run date and time | When the prompt was asked | Logged with time zone |
Model shown | Any model name the interface displays | As displayed |
Search used | Yes, no, or unclear | Yes |
Brand mentioned | Yes or no | Yes |
List position | Order among named brands, if a list | Third of five |
Competitors mentioned | Every other brand named | Gusto, Toast |
Cited URLs | Every linked source | Paste all links |
Own domain cited | Yes or no | No |
Sentiment | Positive, neutral, negative, mixed | Mixed |
Descriptor quote | Exact phrase describing you | "Strong on tips, weak on reporting" |
Accuracy issue | Any false claim, verbatim | Wrong pricing model |
Full answer | Pasted text or screenshot link | Link to screenshot |
The methodology tab is covered in the next section. A summary tab with a pivot of appearance by prompt, intent type, and platform is optional, but build it from the run log so the numbers trace back to raw answers.
Saving the full answer text feels like overkill until you need it. When results shift next quarter, the stored answers let you see whether the shortlist changed, the sources changed, or the description changed.
How do you document the methodology so later comparisons stay valid?
Write down every condition that could change an answer, in enough detail that someone else could repeat the audit without you. A later run is only comparable if the prompts, platforms, and conditions match, or if every difference is recorded. Document at least:
The prompt set version and where the frozen copy lives
Each platform, the interface used, and any model name displayed
Account state, including logged in or out, memory and history settings, and any custom instructions
Location, VPN use, and language settings
Number of runs per prompt and how runs were spread across days
The date range of the audit
Brand name variants and competitor list used for coding
Sentiment coding rules and who did the coding
Keep a change log at the bottom. Assistants update models and search behavior without notice, so note anything you observe, such as a new model name appearing in the interface or citations suddenly showing up where they did not before. When you add prompts later, keep the original set as a fixed core and report it separately from new additions. Otherwise a rise in visibility might just mean you added easier prompts.
Manual sampling at this scale is slow, which is why many teams automate it. Tools such as nonBot AI run a brand's buyer prompts across ChatGPT, Gemini, Perplexity, Claude, and Google AI Overviews and show where the brand appears and which pages get cited. Whether you automate or not, the methodology document is what keeps the comparison honest.
How can you tell the baseline worked?
A baseline works when it is repeatable and when it points to specific gaps. Test repeatability directly: have a colleague rerun a sample of prompts using only the methodology tab. Their appearance counts will not match yours exactly, because answers vary, but they should land in the same neighborhood. If they come back wildly different, a condition is undocumented, usually account state or location.
Usefulness is the second check. You should be able to name the prompts where you are absent, the platforms where the gap is widest, and the third-party pages assistants cite instead of you. If the baseline only produces one blended number, it will not tell anyone what to work on.
Common ways a baseline goes wrong
Three mistakes break most baselines, and all three are easy to avoid at the start.
Editing prompts without versioning: Someone rewords a prompt to sound more natural, and the next quarter's comparison quietly measures a different question. Freeze the text, and treat any rewrite as a new prompt with a new ID.
Testing in a personalized session: Running prompts in your own logged-in ChatGPT account, where memory knows you work at the brand, inflates mentions. Use logged-out sessions or clean accounts, and record which.
Reporting single runs as results: One screenshot of an assistant recommending you, or ignoring you, gets passed around as the verdict. Report how often you appeared across runs and keep the screenshots as examples, never as the metric.
Before you share the baseline with anyone, pick five prompts from the register and have a colleague rerun them from the methodology tab alone, then fix whatever they had to ask you about.
FAQ
How often should I rerun the baseline?
Quarterly works for most brands, with a monthly rerun of a small core set if you are actively publishing or earning coverage. Rerunning too often mostly captures noise from response variability. Always rerun after a major site change or a big press moment so you can see the effect against a known starting point.
Can I use the API instead of the chat apps to save time?
You can, but label it clearly and do not mix the two in one comparison. API responses often differ from the consumer app because the app may add web search, memory, and its own instructions. If your goal is to see what buyers see, the consumer interface is the closer match.
Should competitors be part of the baseline?
Yes. Log every brand named in each answer so you can see who appears where you do not. Competitor data also gives context for your own numbers, since a prompt where nobody appears consistently is a different problem from one where a single competitor dominates.
What if a platform changes its model between my baseline and the next run?
Record the change in the methodology log and compare anyway, but read the shift with caution. A model update can move results on its own, so look for consistency across platforms before crediting your own work. If visibility rose on every platform, your changes are a more plausible cause than one platform's update.
Does referral traffic from AI tools replace prompt testing?
No, the two measure different things. Referral visits from domains like chatgpt.com and perplexity.ai show people clicking cited links, which only happens when you are cited and the reader clicks. Prompt testing shows mentions without links, sentiment, and the answers where you never appeared, which traffic data cannot reveal.
Measure Your Brand's AI Visibility
See how often AI assistants like ChatGPT and Perplexity recommend your business.
Free analysis • No credit card required
About nonBot AI: We help brands optimize their visibility across AI platforms—both retrieval-based and training-based. Our AI Visibility tool tracks your presence across ChatGPT, Perplexity, Claude, and more. If you're ready to build a real AIO strategy, talk to an expert.
