2
Yes
No
01
02
Are you now
tracking AI
prompts alongside
organic rankings?
POLL
3
If you're still measuring
success the way you did
two years ago, you’ve got
it all wrong.
4
AI search
diverges from
SEO, but they
have the same
foundation.
5
Search
& rankings
Reframe how you approach search
FROM
Conversations
& visibility
TO
6
The AI search disruption
isn’t coming — it’s already
in your data.
7
Between Sept 8–12, Google disabled num=100.
Here’s what happened next.
8
Impressions growth dropped 67%
Source: Botify E-Commerce & Retail 2025 Data — bit.ly/googlenum100
Impressions YoY growth, retail/ecommerce
75%
50%
25%
0%
25%
Jan Mar May Jul Sep
Sept 10th
9
Clicks have stayed flat
Clicks YoY growth, retail/ecommerce
20%
15%
0%
10%
20%
Sept 10th
Jan Mar May Jul Sep
Source: Botify E-Commerce & Retail 2025 Data — bit.ly/googlenum100
10
CTR grew 150%
CTR YoY growth, retail/ecommerce
20%
0%
20%
40%
Sept 10th
Jan Mar May Jul Sep
Source: Botify E-Commerce & Retail 2025 Data — bit.ly/googlenum100
11
Average position improved 366%
Average position YoY growth, retail/ecommerce
50%
25%
0%
25%
50%
Sept 10th
Jan Mar May Jul Sep
Source: Botify E-Commerce & Retail 2025 Data — bit.ly/googlenum100
12
AI bot activity has increased 75% since July
Daily logs volume, AI bots
80M
60M
40M
20M
0
May Jun Jul Aug Oct
Sep
Source: Botify E-Commerce & Retail 2025 Data — bit.ly/googlenum100
13
How much of the data
you relied on was
synthetic noise?
14
How do I
measure
success?
What actions
should I take?
1 2
Answer two questions:
15
AI vs Traditional Search
16
AI search compresses the customer journey:
Fewer touchpoints, fewer interfaces
FROM TO
17
What’s the difference?
Traditional search
AI search
Deterministic
Content is king
Probabilistic
Context is king
User signals
Session awareness
Intent + query fan out
VS
18
You don’t “rank”
in AI search.
AI platforms are not search engines.
They don’t rank content — they
answer questions.
Ranking has no meaning in this context.
19
Conversations don’t
happen in a vacuum.
Share of voice only tracks one prompt.
But consumers have entire conversations.
These conversations are personalized.
Your brand may only appear at step #15 —
there’s no tracking for that.
20
Measuring Success
21
Traditional search engines rely on crawling and indexing...
bit.ly/how-ai-search-works
22
And so do AI search platforms!
bit.ly/how-ai-search-works
23
Log data is your
strongest source of
truth for how you
show up in AI search.
focus on collecting data for
LLM training.
Training bots
create and maintain a
search index.
Indexing bots
RAG bots
supplement pre-trained
models with real-time data.
bit.ly/ai-crawler-bots
24
Benefits
● 100 % of bot hits recorded server‑side
● Shows depth & frequency of crawls
Key metrics to monitor
● Bot hits by user-agent
● Bot hits by page type
● User hits referred by AI platforms
Log files: Your best
source of truth
Source: Botify
Important!
The best proxy for a Chat GPT impression is
a ChatGPT-User or OAI-Searchbot request
25
GPTBot is used to train future
models (like ChatGPT-6)
OAI-Searchbot is your “traditional”
indexing bot.
→ Triggered by users asking
questions that need context
→ Retrieves live information to use
as context for answers
→ Most important bot for
performance measurement.
GPTBot OAI-Searchbot ChatGPT-User
ChatGPT-User visiting your sites
means ChatGPT is using your
content.
→ Crawls websites and indexes
content for OpenAI's platforms
→ Similar to crawlers used by
traditional search engines, like
Googlebot or Bingbot
→ Training bot, powering large
language models (LLMs) like
OpenAI's GPT
→ Crawls sites to use their
content to train the next model
→ Only used in the future, not
now.
bit.ly/ai-crawler-bots
26
Combine log data &
directional data to
measure AI visibility
27
Used to make technical
recommendations & understand
bot behavior on your website.
Helps you analyze:
→ Bot behavior
→ RAG impressions
→ Training / indexing / RAG
Cross-reference log data with
analytics data to understand:
→ How much traffic you’re getting
from AI search
→ What people are interested in
based on URL & topics
→ How much revenue you’re
getting from AI platforms.
Directional data is further away;
you’ll choose one of several
options as new info surfaces.
Ask:
→ What questions do you need
answered?
→ What data answers them?
Log file analysis Analytics platforms Visibility dashboard
28
What to
measure
Percentage of answers linking
to website / top citation sources
Citation share
Percentage positive / neutral /
negative; flag answer quality issues
Sentiment mix
Model coverage
The models surfacing the brand across
ChatGPT, Perplexity, Gemini, etc.
Percentage of answers that
mention brand or URLs
Presence rate
Visibility across awareness,
research, and decision intents
Journey coverage
AI visibility
29
Actions to Take
30
Normalize your data
Source: Botify E-Commerce & Retail 2025 Data
Impressions YoY growth, retail/ecommerce
75%
50%
25%
0%
25%
Jan Mar May Jul Sep
Sept 10th
Set September 2025 as your
new baseline
1
Estimate synthetic data percentage
2
Apply that discount to past data
3
Report both raw and adjusted
numbers going forward
4
Compare YoY with adjusted data
5
31
Build AI-ready tech foundations
Determine your AI bot
governance plan (robots.txt)
1
Analyze AI bot behavior and make
action plan (inlinks, depth, etc.)
2
Look into JavaScript reliance, what
can bots see?
3
Look into structured data, what can
you improve?
4
If possible, use server-side
rendering for AI bots
5
32
Build an AI performance framework
Build an AI performance framework
1
Look at data by page type,
language, etc.
2
Create AI-specific analytics
dashboard (referrer + UTM)
3
Determine prompts to track
(keywords, intent, PAA)
4
Create AI visibility / SoV dashboard
5
33
Final Thoughts
34
Q&A
Thank you!

How Do You Track What Doesn’t Rank? Measuring Visibility in AI Search.

  • 2.
    2 Yes No 01 02 Are you now trackingAI prompts alongside organic rankings? POLL
  • 3.
    3 If you're stillmeasuring success the way you did two years ago, you’ve got it all wrong.
  • 4.
    4 AI search diverges from SEO,but they have the same foundation.
  • 5.
    5 Search & rankings Reframe howyou approach search FROM Conversations & visibility TO
  • 6.
    6 The AI searchdisruption isn’t coming — it’s already in your data.
  • 7.
    7 Between Sept 8–12,Google disabled num=100. Here’s what happened next.
  • 8.
    8 Impressions growth dropped67% Source: Botify E-Commerce & Retail 2025 Data — bit.ly/googlenum100 Impressions YoY growth, retail/ecommerce 75% 50% 25% 0% 25% Jan Mar May Jul Sep Sept 10th
  • 9.
    9 Clicks have stayedflat Clicks YoY growth, retail/ecommerce 20% 15% 0% 10% 20% Sept 10th Jan Mar May Jul Sep Source: Botify E-Commerce & Retail 2025 Data — bit.ly/googlenum100
  • 10.
    10 CTR grew 150% CTRYoY growth, retail/ecommerce 20% 0% 20% 40% Sept 10th Jan Mar May Jul Sep Source: Botify E-Commerce & Retail 2025 Data — bit.ly/googlenum100
  • 11.
    11 Average position improved366% Average position YoY growth, retail/ecommerce 50% 25% 0% 25% 50% Sept 10th Jan Mar May Jul Sep Source: Botify E-Commerce & Retail 2025 Data — bit.ly/googlenum100
  • 12.
    12 AI bot activityhas increased 75% since July Daily logs volume, AI bots 80M 60M 40M 20M 0 May Jun Jul Aug Oct Sep Source: Botify E-Commerce & Retail 2025 Data — bit.ly/googlenum100
  • 13.
    13 How much ofthe data you relied on was synthetic noise?
  • 14.
    14 How do I measure success? Whatactions should I take? 1 2 Answer two questions:
  • 15.
  • 16.
    16 AI search compressesthe customer journey: Fewer touchpoints, fewer interfaces FROM TO
  • 17.
    17 What’s the difference? Traditionalsearch AI search Deterministic Content is king Probabilistic Context is king User signals Session awareness Intent + query fan out VS
  • 18.
    18 You don’t “rank” inAI search. AI platforms are not search engines. They don’t rank content — they answer questions. Ranking has no meaning in this context.
  • 19.
    19 Conversations don’t happen ina vacuum. Share of voice only tracks one prompt. But consumers have entire conversations. These conversations are personalized. Your brand may only appear at step #15 — there’s no tracking for that.
  • 20.
  • 21.
    21 Traditional search enginesrely on crawling and indexing... bit.ly/how-ai-search-works
  • 22.
    22 And so doAI search platforms! bit.ly/how-ai-search-works
  • 23.
    23 Log data isyour strongest source of truth for how you show up in AI search. focus on collecting data for LLM training. Training bots create and maintain a search index. Indexing bots RAG bots supplement pre-trained models with real-time data. bit.ly/ai-crawler-bots
  • 24.
    24 Benefits ● 100 %of bot hits recorded server‑side ● Shows depth & frequency of crawls Key metrics to monitor ● Bot hits by user-agent ● Bot hits by page type ● User hits referred by AI platforms Log files: Your best source of truth Source: Botify Important! The best proxy for a Chat GPT impression is a ChatGPT-User or OAI-Searchbot request
  • 25.
    25 GPTBot is usedto train future models (like ChatGPT-6) OAI-Searchbot is your “traditional” indexing bot. → Triggered by users asking questions that need context → Retrieves live information to use as context for answers → Most important bot for performance measurement. GPTBot OAI-Searchbot ChatGPT-User ChatGPT-User visiting your sites means ChatGPT is using your content. → Crawls websites and indexes content for OpenAI's platforms → Similar to crawlers used by traditional search engines, like Googlebot or Bingbot → Training bot, powering large language models (LLMs) like OpenAI's GPT → Crawls sites to use their content to train the next model → Only used in the future, not now. bit.ly/ai-crawler-bots
  • 26.
    26 Combine log data& directional data to measure AI visibility
  • 27.
    27 Used to maketechnical recommendations & understand bot behavior on your website. Helps you analyze: → Bot behavior → RAG impressions → Training / indexing / RAG Cross-reference log data with analytics data to understand: → How much traffic you’re getting from AI search → What people are interested in based on URL & topics → How much revenue you’re getting from AI platforms. Directional data is further away; you’ll choose one of several options as new info surfaces. Ask: → What questions do you need answered? → What data answers them? Log file analysis Analytics platforms Visibility dashboard
  • 28.
    28 What to measure Percentage ofanswers linking to website / top citation sources Citation share Percentage positive / neutral / negative; flag answer quality issues Sentiment mix Model coverage The models surfacing the brand across ChatGPT, Perplexity, Gemini, etc. Percentage of answers that mention brand or URLs Presence rate Visibility across awareness, research, and decision intents Journey coverage AI visibility
  • 29.
  • 30.
    30 Normalize your data Source:Botify E-Commerce & Retail 2025 Data Impressions YoY growth, retail/ecommerce 75% 50% 25% 0% 25% Jan Mar May Jul Sep Sept 10th Set September 2025 as your new baseline 1 Estimate synthetic data percentage 2 Apply that discount to past data 3 Report both raw and adjusted numbers going forward 4 Compare YoY with adjusted data 5
  • 31.
    31 Build AI-ready techfoundations Determine your AI bot governance plan (robots.txt) 1 Analyze AI bot behavior and make action plan (inlinks, depth, etc.) 2 Look into JavaScript reliance, what can bots see? 3 Look into structured data, what can you improve? 4 If possible, use server-side rendering for AI bots 5
  • 32.
    32 Build an AIperformance framework Build an AI performance framework 1 Look at data by page type, language, etc. 2 Create AI-specific analytics dashboard (referrer + UTM) 3 Determine prompts to track (keywords, intent, PAA) 4 Create AI visibility / SoV dashboard 5
  • 33.
  • 34.
  • 35.