How AI picks who gets cited
Ask ChatGPT, Perplexity or Google a question and the answer names a few websites. This guide walks through how those few get picked, one step at a time.
Step 1: a bot downloads your page
Why this matters
visits clicked a source inside Google's AI summary.
Pew Research, 2025, about 69,000 searchesvisibility in AI answers after adding statistics, quotes and sources.
GEO study, Princeton et al., 2023 (lab test)major AI crawlers ran JavaScript. Script-only pages look blank to them. Google is the exception.
Vercel crawler study, 2024Search results vs AI answers
A results page shows ten links and lets you choose. An AI answer reads the pages for you and names only a few.
Every result on the page has a chance at a click.
For most desk workers, yes, if you switch between sitting and standing1. The calorie burn is small, so it does not replace exercise2.
Two sites are named. The others are not mentioned at all.
What competes
What winning looks like
Where trust comes from
What carries over
What do SEO, GEO and AEO actually mean?
SEO (search engine optimisation) is the craft of ranking pages in classic search results. GEO (generative engine optimisation) and AEO (answer engine optimisation) are newer names for getting your content used and cited inside AI-written answers from ChatGPT, Perplexity, Google AI Overviews and AI Mode, Gemini, Copilot and Claude.
They are not rivals. Many AI engines start from a classic search index, so SEO decides whether you are even a candidate. GEO decides whether you get picked and quoted. Measuring GEO is harder, because answers change by engine, wording, location and day.
Build a page to test
Say you wrote a blog post called "Is a standing desk worth it?" Make seven choices about it, then follow it through each step below to see where it gets dropped. The engine in this example works like ChatGPT search or Perplexity.
Is a standing desk worth it?
When I was a kid, my grandfather never sat down at work. Years later, hunched over a laptop, I started to wonder...
Does the AI search at all?
An AI answers either from what it learned in training, or by searching the web first. Only the search path cites anyone.
How it really works
Answering from memory uses patterns learned from a large snapshot of text, months or years old. These answers usually show no sources.
Search first is called retrieval-augmented generation (RAG): fetch fresh pages, read them, write from them, and cite them. This is where AI visibility is won day to day.
Memory still matters slowly: if many pages mention your brand, a future model may simply know it. Many assistants now search for most factual questions, and behaviour differs by product.
Pick a question. Which way does it go?
From memory
Training data. No sources.
Search, then answer
Fresh pages. Cited.
Advice that depends on current studies and products. Assistants usually search, and citations appear.
Crawling: a bot downloads your page
Each AI company sends crawlers to download pages. Your robots.txt decides who gets in, and none of the major AI bots run JavaScript.
Meet the bots
| Bot | Company | Job |
|---|---|---|
| GPTBot | OpenAI | Collects training data |
| OAI-SearchBot | OpenAI | Index that ChatGPT search cites from |
| ChatGPT-User | OpenAI | Live fetch during a user's chat |
| ClaudeBot | Anthropic | Training |
| Claude-SearchBot | Anthropic | Search index |
| PerplexityBot | Perplexity | Search index |
| Googlebot | Search, which also feeds AI Overviews and AI Mode | |
| Google-Extended | A robots.txt switch for Gemini training, not a bot. Does not affect AI Overviews |
Trap 1: blocking every AI bot to stop training also removes you from AI answers. Block the training bots only. Trap 2: if your text appears only after JavaScript runs, AI crawlers see an empty page. Googlebot is the exception and renders JavaScript.
Watch the bots visit
Same switches as the builder. Flip them and watch the log.
# robots.txt User-agent: * Allow: / # who visited GPTBot ✓ 200 full text OAI-SearchBot ✓ 200 full text ClaudeBot ✓ 200 full text Claude-SearchBot ✓ 200 full text PerplexityBot ✓ 200 full text Googlebot ✓ 200 full text
Is a standing desk worth it?
When I was a kid, my grandfather never sat down at work. Years later, hunched over a laptop, I started to wonder...
1,900 words, no headings
Every AI bot can read your page, and the text is right there in the HTML.
Chunking: the page is split into passages
Pages are sliced into short passages, and each one competes alone. A passage that starts with "It..." has lost what "it" means.
How it really works
Each chunk is turned into an embedding: a long list of numbers that captures its meaning. Similar meanings get similar numbers, which is how the engine later finds passages that match a question.
Real systems cut by length, by headings, or both, and some attach a bit of surrounding context to each chunk. Clear headings and passages that name their subject make every chunk stronger.
Try different chunk sizes
Yellow words point to something the chunk no longer contains.
A wall of text gets cut anywhere. Chunks mix topics and lose context.
Fan-out: one question becomes several searches
The AI quietly rewrites your question into several narrower searches. Pages that cover the follow-up questions get more chances.
How it really works
Google calls this query fan-out and describes it as the core of AI Mode: the question is broken into subtopics and many searches run at once. Results from all of them are pooled before anything is ranked.
See the hidden searches
- 6 hidden searches waiting…
Press "Fan it out" to reveal the hidden searches the AI runs.
Retrieval: finding passages that match
For each search, the engine pulls the passages closest in meaning, not just the ones sharing words. Try keyword mode and see what sneaks in.
How it really works
Engines usually blend two methods: keyword search (shared words) and semantic search (nearby embeddings). The map here is a 2D simplification; real embeddings have hundreds or thousands of dimensions.
This is also where classic SEO pays off. Many engines ask a normal web index first and only read its top results. A page nobody links to or talks about may never enter the pool.
The meaning map
Pick one of the hidden searches
Pulled in for this search
- 3Trial: sit-stand desks cut sitting by an hour
- 5Standing over 4 hours a day causes discomfort
- 9Leg fatigue from standing too long
The red target is the search. The 3 passages closest to it in meaning are pulled in. Tap any dot to read it.
Its answers are buried in one long block, it opens with a story instead of an answer, and nothing links to it, so it never reached the shortlist.
Re-ranking: picking the best few
A stricter model re-scores every candidate on five signals. Pages that answer early and bring evidence climb the podium.
The five signals
- Relevance: does it answer this exact question?
- Authority: is the source trusted, linked and talked about?
- Freshness: is it current?
- Evidence: numbers, sources, quotes, first-hand data. The 2023 GEO study found these raised visibility in AI answers by up to about 40%.
- Directness: is the answer stated plainly and early?
No engine publishes its formula. The weights and scores in this toy are illustrative.
Change what the ranker cares about
Scores are out of 10: each signal is scored 0 to 10, then averaged using these weights. These are the default weights, and the rest of the page uses them.
- 1HealthWikiStanding desks: what the research says7.9
- 2Lab Notes DailyStudy: sit-stand desks cut sitting by an hour a day7.4
- 3r/WorkFromHomeOne year with a standing desk, AMA6.6
- only the top 3 get cited
- 4DeskMartWhy you NEED a standing desk (sale)4.8
- 5OfficeChairForumstanding desks worth it?? (2014)4.2
- your page is missing: it was never retrieved
Dropped at an earlier step.
Citing: writing the answer and adding sources
The model writes from the winners and pins sources to sentences. Unique facts, like your own survey, keep their link best.
How it really works
Generic advice gets blended across sources and is often credited to the biggest name. A specific number that only you published is hard to use without crediting you, which is why original data is the strongest position a small site can hold.
Tap a number to see what it supports
For most desk workers, yes, as long as you alternate between sitting and standing. The extra calorie burn is small, about 10 calories an hour, so it does not replace exercise.
In workplace trials, sit-stand desks cut daily sitting time by roughly an hour. Long-term users say the habit sticks best when you switch positions often.
Dropped at an earlier step.
Clicks: most people never visit
When the answer is good enough, people stop reading. Being named still matters: people see your name, and the few who click usually know what they want.
About the data
Pew Research Center tracked about 69,000 real Google searches by 900 US adults in March 2025. With an AI summary, people also ended their browsing session more often: 26% versus 16% without one.
Clicks per 100 Google visits
15 clickswith no AI summary
8 + 1 clickswith an AI summary: 8 on a regular result, 1 on a source inside the summary
Dropped at an earlier step.
How your page did
Change your page above and this updates straight away.
Dropped at retrieve
Its answers are buried in one long block, it opens with a story instead of an answer, and nothing links to it, so it never reached the shortlist.
Fix: lead with the answer, add headings for follow-up questions, earn a few mentions.
What to do on your own site
One or two changes for each step.
Let the right bots in
- Allow search bots even if you block training bots.
- Serve real text in the HTML (server rendering or static pages).
- Check your CDN or firewall isn't blocking AI bots.
Write passages that stand alone
- One idea per section, under a question-style heading.
- Repeat the subject instead of "it" or "this".
- Lists and tables for facts that belong together.
Cover the next questions
- Answer the follow-ups: how long, which one, any downsides.
- Add a short FAQ with real questions people ask.
Be findable
- Keep classic SEO healthy: indexing, internal links, backlinks.
- Get mentioned where engines look: Reddit, reviews, news.
Give a reason to pick you
- Lead with the answer in the first two sentences.
- Add numbers, sources and quotes.
- Show a real update date, and actually update.
Own something quotable
- Publish facts only you have: surveys, tests, prices.
- Ask AI engines your customers' questions regularly and note who gets cited.
- Track mentions, not only clicks.
Common questions
What is GEO (generative engine optimisation)?
GEO is the work of getting your content used and cited in AI-generated answers from tools like ChatGPT, Perplexity and Google AI Overviews. It builds on SEO rather than replacing it.
How do AI search engines decide which websites to cite?
Most follow the same pipeline: crawl pages, split them into passages, rewrite the question into several searches, retrieve the passages closest in meaning, re-rank them on relevance, authority, freshness, evidence and directness, and cite the few that support the final answer.
Does blocking GPTBot remove my site from ChatGPT search?
No. GPTBot collects training data. ChatGPT search relies on OAI-SearchBot, and live fetches during a chat use ChatGPT-User. You can block GPTBot and still allow the search bots.
Can AI crawlers read pages built with JavaScript?
Mostly not. In Vercel's 2024 study, the major AI crawlers did not run JavaScript; Googlebot is the exception. Put your main text in the HTML your server sends.
Does SEO still matter for AI search?
Yes. Many AI engines start from a regular search index, so pages that are crawlable, indexed and linked are more likely to become candidates in the first place.
How often do people click the sources in AI answers?
Rarely. Pew Research found that when Google showed an AI summary, people clicked a source inside it in about 1% of visits (March 2025, about 69,000 searches).
Why I made this
Most explanations of AI search are either vague or buried in jargon. I wanted something you could poke at: change a page, then watch where it drops out.
Everything runs in your browser. There are no cookies and no tracking. The websites in the standing-desk example are made up; the mechanics and the numbers are real, and the sources are listed below.
Glossary and sources
Glossary
- LLM
- Large language model, the AI that writes the answer.
- RAG
- Retrieval-augmented generation: search first, then answer from what was found.
- AI Overviews / AI Mode
- Google's AI answers, above results or as a full chat-style search.
- Crawler
- A bot that downloads web pages.
- robots.txt
- A file telling bots what they may crawl.
- Chunk
- A short passage cut from a page; the unit that gets retrieved and quoted.
- Embedding
- Numbers representing meaning. Close numbers, similar meaning.
- Query fan-out
- One question split into many narrower searches.
- Re-ranking
- A second, stricter scoring of the candidates.
- Zero-click
- A search that ends without visiting any site.
- llms.txt
- A proposed summary file for AI tools. Use by major engines is unconfirmed.
Sources
- GEO: Generative Engine Optimization, Aggarwal et al., 2023
- Google users are less likely to click on links when an AI summary appears, Pew Research Center, 2025
- The rise of the AI crawler, Vercel, 2024
- Overview of OpenAI crawlers, OpenAI
- Google's common crawlers, Google Search Central
- AI Mode and query fan-out, Google, 2025
The websites in the standing-desk story are fictional. The mechanics are real; the scoring toy is illustrative.