How AI picks who gets cited
Ask ChatGPT, Perplexity or Google a question and the answer names a few websites. This guide walks through how those few get picked, one step at a time.
- Crawl: a bot downloads your page
- Chunk: your page is cut into passages
- Fan-out: one question becomes many searches
- Retrieve: the closest passages get pulled in
- Re-rank: only the top few make it
- Cite: the answer links its sources
- Click: few people click the sources
Why this matters
Google visits that showed an AI summary ended with a click on a source inside it.
Pew Research, March 2025: 900 US adults, about 69,000 searches. Google only, not ChatGPT or Perplexity.visibility in generative-engine answers after adding statistics, quotes and sources, in experiments.
GEO study, Aggarwal et al., 2023. Experimental research; gains varied by method and topic and do not predict your site's results.of the major AI crawlers Vercel measured ran JavaScript, so script-only pages looked blank to them. Googlebot and AppleBot render it.
Vercel crawler study, December 2024Search results vs AI answers
A results page shows ten links and lets you choose. An AI answer reads the pages for you and names only a few.
Every result on the page has a chance at a click.
For most desk workers, yes, if you switch between sitting and standing1. The calorie burn is small, so it does not replace exercise2.
Two sites are named. The others are not mentioned at all.
What competes
What winning looks like
Where trust comes from
What carries over
What do SEO, GEO and AEO actually mean?
SEO (search engine optimisation) is the craft of ranking pages in classic search results. GEO (generative engine optimisation) and AEO (answer engine optimisation) are newer names for getting your content used and cited inside AI-written answers from ChatGPT, Perplexity, Google AI Overviews and AI Mode, Gemini, Copilot and Claude.
They are not rivals. Many AI engines start from a classic search index, so SEO decides whether you are even a candidate. GEO decides whether you get picked and quoted. Measuring GEO is harder, because answers change by engine, wording, location and day.
Try the citation simulator
This uses a made-up blog post, "Is a standing desk worth it?", not your own site. Change its seven settings, then follow it through each step below to see where it gets dropped. The simulated engine is modelled loosely on ChatGPT search and Perplexity.
Simulation. The signals, weights, thresholds and scores here are invented teaching rules. Real engines don't publish theirs, so none of these numbers predict how a real page will rank.
Is a standing desk worth it?
When I was a kid, my grandfather never sat down at work. Years later, hunched over a laptop, I started to wonder...
Does the AI search at all?
This tour follows one simplified web-search workflow, modelled loosely on ChatGPT search and Perplexity. Real systems vary and change often.
An AI answers either from what it learned in training, or by searching the web first. In this model, only the search path cites sources.
How it really works
Answering from memory uses patterns learned from a large snapshot of text, months or years old. These answers usually show no sources.
Search first is called retrieval-augmented generation (RAG): fetch fresh pages, read them, write from them, and cite them. This is where AI visibility is won day to day.
Memory still matters slowly: if many pages mention your brand, a future model may simply know it. Many assistants now search for most factual questions, and behaviour differs by product.
Separate example. These questions don't change your test page.
Pick a question. Which way does it go?
From memory
Training data. No sources.
Search, then answer
Fresh pages. Cited.
Advice that depends on current studies and products. Assistants usually search, and citations appear.
Crawling: a bot downloads your page
Each AI company sends crawlers to download pages. Your robots.txt decides who gets in, and in Vercel's late-2024 study none of the major AI bots ran JavaScript.
Meet the bots
| Bot | Company | Job |
|---|---|---|
| GPTBot | OpenAI | Collects training data |
| OAI-SearchBot | OpenAI | Index that ChatGPT search cites from |
| ChatGPT-User | OpenAI | Live fetch during a user's chat |
| ClaudeBot | Anthropic | Training |
| Claude-SearchBot | Anthropic | Search index |
| PerplexityBot | Perplexity | Search index |
| Googlebot | Search, which also feeds AI Overviews and AI Mode | |
| Google-Extended | A robots.txt switch for Gemini training, not a bot. Does not affect AI Overviews |
Trap 1: blocking every AI bot to stop training also removes you from ChatGPT search, Claude and Perplexity answers. Block the training bots only. Google works differently: AI Overviews and AI Mode are part of Search, so Googlebot's robots.txt rules control them, and Google-Extended doesn't. Trap 2: if your text appears only after JavaScript runs, the major AI crawlers in Vercel's December 2024 study saw an empty page. Googlebot, which also serves Gemini, renders JavaScript.
Watch the bots visit
Change your robots.txt or how the text loads, and watch the log.
# robots.txt User-agent: * Allow: / # who visited GPTBot ✓ 200 full text OAI-SearchBot ✓ 200 full text ClaudeBot ✓ 200 full text Claude-SearchBot ✓ 200 full text PerplexityBot ✓ 200 full text Googlebot ✓ 200 full text
Is a standing desk worth it?
When I was a kid, my grandfather never sat down at work. Years later, hunched over a laptop, I started to wonder...
1,900 words, no headings
Every AI bot can read your page, and the text is right there in the HTML.
Chunking: the page is split into passages
Pages are sliced into short passages, and each one competes alone. A passage that starts with "It..." has lost what "it" means.
How it really works
Each chunk is turned into an embedding: a long list of numbers that captures its meaning. Similar meanings get similar numbers, which is how the engine later finds passages that match a question.
Real systems cut by length, by headings, or both, and some attach a bit of surrounding context to each chunk. Clear headings and passages that name their subject make every chunk stronger.
Watch your page get cut up
Your layout decides where the cuts land. The chunk size is set by the engine, so try a few.
With no headings, cuts land anywhere: some chunks lose what they refer to (yellow words), others mix topics. Switch the layout to Headings + FAQ to compare.
A wall of text gets cut anywhere. Chunks mix topics and lose context. Switch Layout to Headings + FAQ above to fix it.
Fan-out: one question becomes several searches
The AI quietly rewrites your question into several narrower searches. Pages that cover the follow-up questions get more chances.
How it really works
Google calls this query fan-out and describes it as the core of AI Mode: the question is broken into subtopics and many searches run at once. Results from all of them are pooled before anything is ranked.
See the hidden searches
- 6 hidden searches waiting…
Press "Fan it out" to reveal the hidden searches the AI runs.
It touches 4 of the 6 hidden searches, but the answers sit inside one long block, so every match is weak. Switch Layout to Headings + FAQ to turn them into clear matches.
Retrieval: finding passages that match
For each search, the engine pulls the passages closest in meaning, not just the ones sharing words. Try keyword mode and see what sneaks in.
How it really works
Engines usually blend two methods: keyword search (shared words) and semantic search (nearby embeddings). The map here is a 2D simplification; real embeddings have hundreds or thousands of dimensions.
This is also where classic SEO pays off. Many engines ask a normal web index first and only read its top results. A page nobody links to or talks about may never enter the pool.
The meaning map
Pick one of the hidden searches
Pulled in for this search
- 3Trial: sit-stand desks cut sitting by an hour
- 5Standing over 4 hours a day causes discomfort
- 9Leg fatigue from standing too long
The red target is the search. The 3 passages closest to it in meaning are pulled in. Tap any dot to read it.
All 14 passages as a list
Its answers are buried in one long block, it opens with a story instead of an answer, and nothing links to it, so it never reached the shortlist. Fixing any one of these gets it pulled in: Layout to Headings + FAQ, First paragraph to Answer first, or Reputation to Talked about on Reddit.
Re-ranking: picking the best few
A stricter model re-scores the candidates. In this example it uses five signals, and pages that answer early and bring evidence climb the podium.
The five signals
- Relevance: does it answer this exact question?
- Authority: is the source trusted, linked and talked about?Authority compounds: sites that get cited earn more links and get cited again. This is where the Matthew effect starts (see step 6).
- Freshness: is the information current? This matters most for things that change, like prices and products. For settled facts, an old but correct page can still be the best source, and a new date on unchanged text is not new evidence.
- Evidence: numbers, sources, quotes, first-hand data. In 2023 experiments, the GEO study found these raised visibility in generative-engine answers by up to about 40%, with results varying by method and topic.
- Directness: is the answer stated plainly and early?
Each signal only works while it reflects real quality. Fake dates, stuffed keywords and invented statistics get discounted: Goodhart's law (see the playbook).
No engine publishes its formula. The weights and scores in this toy are illustrative.
Separate experiment. These weights change only this ranking. Your page's journey and result keep the default weights.
Change what the ranker cares about
Invented weights for teaching. Real engines don't publish theirs, and the number of sources an answer cites varies; here it's always 3.
Each page gets 0 to 10 on five signals (shown under each page below). Its score is the weighted average, so what matters is each signal's share: Relevance 29%, Authority 19%, Freshness 14%, Evidence 19%, Directness 19%. These are the default weights, and the rest of the page uses them.
- 1HealthWikiStanding desks: what the research says7.9Rel 8Auth 9Fresh 6Evid 8Direct 8
- 2Lab Notes DailyStudy: sit-stand desks cut sitting by an hour a day7.4Rel 7Auth 8Fresh 7Evid 9Direct 6
- 3r/WorkFromHomeOne year with a standing desk, AMA6.6Rel 8Auth 6Fresh 9Evid 4Direct 6
- in this example, only the top 3 get cited
- 4DeskMartWhy you NEED a standing desk (sale)4.8Rel 6Auth 5Fresh 8Evid 2Direct 3
- 5OfficeChairForumstanding desks worth it?? (2014)4.2Rel 7Auth 3Fresh 1Evid 3Direct 5
- your page is missing: it was never retrieved
Where your page's signals come from: relevance from First paragraph and Layout, authority from Reputation, freshness from Last updated, evidence from Evidence, directness from First paragraph. The other pages' signals are fixed. Ties go to the more relevant page.
Your page was dropped at step 4 (Retrieve), so it never got here.
Citing: writing the answer and adding sources
The model writes from the winners and pins sources to sentences. Unique facts, like your own survey, keep their link best.
How it really works
Generic advice gets blended across sources and is often credited to the biggest name. A specific number that only you published is hard to use without crediting you, which is why original data is the strongest position a small site can hold.
Tap a number to see what it supports
For most desk workers, yes, as long as you alternate between sitting and standing. The extra calorie burn is small, about 10 calories an hour, so it does not replace exercise.
In workplace trials, sit-stand desks cut daily sitting time by roughly an hour. Long-term users say the habit sticks best when you switch positions often.
Who gets the credit?
Two pages make the same kind of point. The answer cites one of them for it.
Same point, nothing of your own behind it. For this point the answer credits HealthWiki, the bigger name. (This demo assumes your page made the shortlist, which it hasn't yet.)
Your page was dropped at step 4 (Retrieve), so it never got here.
Clicks: most people never visit
When the answer is good enough, people stop reading. Being named still matters: people see your name, and the few who click usually know what they want.
About the data
Pew Research Center tracked about 69,000 real Google searches by 900 US adults in March 2025. With an AI summary, people also ended their browsing session more often: 26% versus 16% without one.
Clicks per 100 Google visits
15 clickswith no AI summary
8 + 1 clickswith an AI summary: 8 on a regular result, 1 on a source inside the summary
Your page was dropped at step 4 (Retrieve), so it never got here.
How your page did
Change your page above and this updates straight away.
Simulated result. It shows how the pipeline behaves under invented rules. It is not a measurement or a prediction for a real page.
Dropped at retrieve
Its answers are buried in one long block, it opens with a story instead of an answer, and nothing links to it, so it never reached the shortlist.
- Lead with the answer.
- Add headings for follow-up questions.
- Earn mentions on sites like Reddit.
What to do on your own site
One or two changes for each step. Earn the signals, don't fake them.
Let the right bots in
- Allow search bots even if you block training bots.
- Serve real text in the HTML (server rendering or static pages).
- Check your CDN or firewall isn't blocking AI bots.
Write passages that stand alone
- One idea per section, under a question-style heading.
- Repeat the subject instead of "it" or "this".
- Lists and tables for facts that belong together.
Cover the next questions
- Answer the follow-ups: how long, which one, any downsides.
- Add a short FAQ with real questions people ask.
Be findable
- Keep classic SEO healthy: indexing, internal links, backlinks.
- Get mentioned where engines look: Reddit, reviews, news.
Give a reason to pick you
- Lead with the answer in the first two sentences.
- Add numbers, sources and quotes.
- Show a real update date, and actually update.
Own something quotable
◆ Beats the Matthew effect- Publish facts only you have: surveys, tests, prices.
- Ask AI engines your customers' questions regularly and note who gets cited.
- Track mentions, not only clicks.
Common questions
What is GEO (generative engine optimisation)?
GEO is the work of getting your content used and cited in AI-generated answers from tools like ChatGPT, Perplexity and Google AI Overviews. It builds on SEO rather than replacing it.
How do AI search engines decide which websites to cite?
Each system works differently, and none publishes its formula. A common web-search workflow is: crawl pages, split them into passages, rewrite the question into several searches, retrieve the passages closest in meaning, re-rank the candidates, and cite a few that support the answer. The five ranking signals in this guide (relevance, authority, freshness, evidence and directness) are a simplified teaching model, not a known formula.
Does blocking GPTBot remove my site from ChatGPT search?
No. GPTBot collects training data. ChatGPT search relies on OAI-SearchBot, and live fetches during a chat use ChatGPT-User. You can block GPTBot and still allow the search bots.
Can AI crawlers read pages built with JavaScript?
Often not. In Vercel's December 2024 study, the major AI crawlers it measured, from OpenAI, Anthropic and Perplexity, did not run JavaScript. Googlebot, which also serves Gemini, and AppleBot did. Put your main text in the HTML your server sends.
Does SEO still matter for AI search?
Yes. Many AI search tools draw on a regular search index, so pages that are crawlable, indexed and linked are more likely to become candidates in the first place.
How often do people click the sources in AI answers?
Rarely, in the data available. Pew Research found that when Google showed an AI summary, people clicked a source inside it in about 1% of visits (March 2025, 900 US adults, about 69,000 searches). That covers Google only, not ChatGPT or Perplexity.
Do big websites always win AI citations?
Often, for generic advice. When several pages say the same thing, answers tend to credit the best-known source, a pattern sociologists call the Matthew effect. Smaller sites break through with something only they published: original data, tests or first-hand experience.
Why we made this
Most explanations of AI search are either vague or buried in jargon. We wanted something you could poke at: change a page, then watch where it drops out.
The pipeline stages are real. The standing-desk websites, scores, weights and thresholds are invented for teaching. The research figures are real and sourced below, each with its limits. Everything runs in your browser. If you allow it, Google Analytics counts visits and which parts of the page get used. No ads and no personal profiles.
Glossary and sources
Glossary
- LLM
- Large language model, the AI that writes the answer.
- RAG
- Retrieval-augmented generation: search first, then answer from what was found.
- AI Overviews / AI Mode
- Google's AI answers, above results or as a full chat-style search.
- Crawler
- A bot that downloads web pages.
- robots.txt
- A file telling bots what they may crawl.
- Chunk
- A short passage cut from a page; the unit that gets retrieved and quoted.
- Embedding
- Numbers representing meaning. Close numbers, similar meaning.
- Query fan-out
- One question split into many narrower searches.
- Re-ranking
- A second, stricter scoring of the candidates.
- Zero-click
- A search that ends without visiting any site.
- Matthew effect
- Credit flows to whoever is already well known. Named by Robert Merton in 1968.
- Goodhart's law
- When a measure becomes a target, it stops being a good measure.
- llms.txt
- A proposed summary file for AI tools. Use by major engines is unconfirmed.
Sources
- GEO: Generative Engine Optimization, Aggarwal et al., 2023
- Google users are less likely to click on links when an AI summary appears, Pew Research Center, 2025
- The rise of the AI crawler, Vercel, 2024
- Overview of OpenAI crawlers, OpenAI
- Google's common crawlers, Google Search Central
- AI Mode and query fan-out, Google, 2025
- The Matthew Effect in Science, Robert K. Merton, Science, 1968
The websites in the standing-desk story are fictional. The pipeline stages are real; the scoring, weights and thresholds are invented for teaching.