How AI Engines Select Sources

1. What Happens When You Ask AI a Question

AI assistants are powered by technology called large language models (LLMs). These are software programs that have been trained to understand and produce human language. They do not think or reason the way people do. Instead, they predict what words are most likely to come next in a response, based on patterns they learned during a training process. [Source: Jurafsky & Martin, Speech and Language Processing, 3rd edition, Stanford University — web.stanford.edu/~jurafsky/slp3/]

When you type a question, the AI generates an answer using one of two general approaches — or a combination of both:

Approach A — Recall from Memory. The AI draws on knowledge it absorbed during training. It does not look anything up in real time. It generates the answer entirely from what it already "knows." Think of a student answering an exam question from memory — they cannot open any books or check the internet, they can only work with what they studied beforehand.

Approach B — Search, Then Answer. The AI searches the internet for current information before writing its response. It reads web pages in real time and builds its answer from what it finds. Think of a student taking an open-book exam — they still need to understand the subject, but they can look things up to make sure their answer is accurate and current.

This distinction is the single most important concept in AI visibility. Some engines rely entirely on memory. Some always search first. Some can do both depending on the situation. The rest of this article explains each approach in detail, so you understand exactly what is happening behind the scenes when AI decides whether to mention your brand.

2. Training Data: The AI's Built-in Knowledge

What Training Data Is

Training data is the information an AI model studied before it was released to the public. Think of it as everything the AI "read" during its education. This data forms the foundation of every response the model generates — even when the model also has the ability to search the web.

The training process works like this: AI companies collect massive amounts of text from across the internet and other sources. They feed this text into the model, which processes it over weeks or months using powerful computers. During this process, the model learns language patterns, factual associations, and relationships between concepts. [Source: Brown, T. et al., "Language Models are Few-Shot Learners," 2020, OpenAI — arxiv.org/abs/2005.14165]

Where Training Data Comes From

While exact training data compositions vary by company and are not always fully disclosed, published research papers from major AI labs describe the following common sources:

Web pages — Billions of pages collected through large-scale web crawling projects. The most well-known is Common Crawl, a nonprofit organization that continuously archives publicly accessible web content and makes it available for research and AI training. Common Crawl's archive contains petabytes of data collected since 2008. [Source: Common Crawl Foundation — commoncrawl.org]

Wikipedia — The full text of Wikipedia is included in virtually every major training dataset, according to published model documentation. Wikipedia's structured, factual, well-sourced format makes it one of the most influential sources in training data. When AI recommends a brand or explains a concept, Wikipedia's description often shapes the language the AI uses. [Source: Brown et al., 2020 — arxiv.org/abs/2005.14165]

Books and academic papers — Collections of published books, textbooks, and peer-reviewed research papers provide depth on specialized topics. Academic content carries high authority because it has been vetted through editorial and peer-review processes.

News articles — Content from major news outlets, wire services (such as AP and Reuters), and established online publications provides coverage of current events, company news, product launches, and public figures.

Forums and community discussions — Publicly available posts from platforms like Reddit, Stack Overflow, and Quora capture real-world opinions, product experiences, and community knowledge. Reddit in particular has been documented as a significant source in training datasets. [Source: Brown et al., 2020 — arxiv.org/abs/2005.14165]

Code repositories — Open-source code from platforms like GitHub helps models understand and generate programming languages. This is less relevant for brand visibility but important context for understanding the breadth of training data.

Government and institutional sites — Content from .gov and .edu domains carries inherently high authority in training data due to the editorial standards and credibility associated with these institutions.

A Helpful Way to Think About It

Imagine hiring a new employee. Before their first day, they spend six months reading every document, report, customer review, news article, and forum post about your entire industry. On day one, they cannot look anything up — but they can draw on everything they read. If your brand appeared frequently and positively across what they studied, they will naturally mention and recommend you when asked. If your brand was absent or barely mentioned, they will recommend your competitors instead.

That is how training-data-only AI works. Its answers are shaped entirely by what it absorbed during its study period.

How Models Learn from This Data

During training, the AI does not memorize web pages word for word. Instead, it learns patterns, relationships, and associations across all the content it processes. [Source: Jurafsky & Martin, Speech and Language Processing, 3rd edition — web.stanford.edu/~jurafsky/slp3/]

Here is what that means in practice:

• If your brand is mentioned alongside words like "reliable," "recommended," or "industry-leading" across many different independent sources, the model learns that positive association. When someone later asks for a recommendation, the model is more likely to mention your brand with positive language.
• If your competitors are mentioned frequently in response to a particular type of question and your brand is not, the model learns to recommend them instead of you — even if your product is objectively better. The model only knows what it has seen.
• If a Wikipedia article, three news articles, and two review sites all describe your product in consistent terms, that description becomes strongly embedded. Consistency across sources reinforces the message.
• If different sources say conflicting things about your brand — one says you specialize in enterprise, another says small business — the model may produce confused or inaccurate descriptions.

This is a critical point: the AI does not maintain a database of facts it can look up. It absorbs patterns and probabilities. When it generates an answer, it is predicting what a helpful response would look like based on everything it absorbed during training.

Not All Sources Carry Equal Weight

While AI companies do not publish exact weighting formulas, research on training data composition and analysis of model behavior indicates that certain types of content carry more influence: [Source: Longpre, S. et al., "A Pretrainer's Guide to Training Data," 2023 — arxiv.org/abs/2305.13169]

The key takeaway: what others say about you matters more than what you say about yourself. Third-party mentions from authoritative sources carry significantly more weight than content on your own website. This principle runs through every article in this series — see Wikipedia & Knowledge Graphs, Review Platforms & Ratings, Industry Publications & PR, and Third-Party Validation for specific strategies.

What the AI Does NOT Learn From

Certain types of content are largely invisible to training data collection:

• Content behind login walls — If a page requires signing in to view, web crawlers typically cannot access it. Your gated whitepapers, member-only content, and internal documentation are not part of training data.
• Private databases and intranets — Internal company systems, CRM data, and proprietary databases are completely invisible.
• Content blocked by robots.txt — Website owners can instruct web crawlers not to index certain pages. Some major publishers have opted out of AI training entirely using these mechanisms.
• PDFs and images without extractable text — Content locked in scanned document images or infographics is difficult for crawlers to process. Text-based HTML pages are far more accessible to AI training pipelines.
• JavaScript-rendered content — Pages that require JavaScript to execute before their content appears may not be fully captured by all web crawlers. If your key product information only loads after user interaction (clicking tabs, scrolling, expanding sections), it may be partially or completely missed.

3. Knowledge Cutoffs: When the Learning Stops

What a Knowledge Cutoff Is

Every AI model that relies on training data has a knowledge cutoff — a specific date after which it has no awareness of new events, updated content, or changed information. This exists because training is a process that happens once (or periodically at intervals of months), not continuously.

A simple way to understand this: a knowledge cutoff is like the last day a student studied before an exam. Everything published before that date might be in their notes. Everything published after that date does not exist in their world — no matter how important, no matter how widely covered, no matter how relevant to the question being asked.

Current Cutoff Dates by Engine

AI companies publish knowledge cutoff dates in their official documentation. These dates change when new model versions are released. Below are approximate cutoffs as of early 2026 — always check each provider's current documentation for the most up-to-date information:

Important: A Cutoff Does Not Mean Your Brand Cannot Be Cited

A common misconception is that if your website or brand was created after an engine's knowledge cutoff date, that engine can never mention you. This is not true for hybrid engines like ChatGPT and Gemini.

These engines have two ways of finding information: their training data (which has a cutoff) and their real-time browse or search capabilities (which do not). When ChatGPT activates its browse mode — something it does automatically when it determines current information is needed — it searches the live web just like a regular search engine. This means it can discover, read, and cite websites and brands that were created well after its training cutoff date.

For example, if you launched your company in March 2025 and ChatGPT's training data only goes to late 2024, ChatGPT can still find and recommend your business through its browse mode — provided your website is publicly accessible, well-optimized, and mentioned across authoritative sources. The same applies to Gemini, which can "ground" its responses using live Google Search results.

The engines where the cutoff truly matters are training-data-only engines like Claude, which do not search the web at all. For these engines, if your brand was not captured in the training data, it will not appear in responses until the next model update.

See Section 4: Real-Time Retrieval (RAG) below for a full explanation of how real-time search works, and the engine comparison table showing which engines have browse capabilities.

Why Knowledge Cutoffs Still Matter for Your Brand

Even with browse capabilities, knowledge cutoffs have direct, practical consequences for how AI represents your business — particularly for engines that rely heavily on training data:

New products or services launched after the cutoff date will not exist in the training-based portion of AI responses. For hybrid engines like ChatGPT, this means the engine may still find your product through browsing, but its underlying model will have no deep familiarity with it. For training-only engines like Claude, the product simply will not exist in responses until the next model update.

Rebranding that happened recently will not be reflected in training data. Hybrid engines may pick up the new branding through browse mode, but may inconsistently switch between old and new names depending on whether the response draws from training data or live search results.

Negative press or incidents that occurred before the cutoff will remain in the model's training knowledge even if the situation has been fully resolved. Browse mode may surface more recent positive coverage, but the underlying model still "remembers" the older narrative.

Competitor changes create gaps. If a competitor shut down, was acquired, or significantly changed their offering after the cutoff, training-based AI may still recommend them as if nothing changed — unless browse mode is activated and finds current information.

Updated pricing, features, and policies will not be reflected in training-based responses. Hybrid engines may find current information through browsing, but there is no guarantee browse mode will activate for every query.

How and When Models Get Updated

When an AI company releases a new version of its model — for example, when OpenAI moves from one GPT version to a newer one — the new version is trained on a more recent dataset. This is the moment when recent content, brand mentions, and updated information become part of the model's knowledge.

The retraining process typically involves collecting a new and more recent dataset, training the model on it (which takes weeks to months and costs millions of dollars in computing resources), evaluating the model for accuracy and safety, and then releasing it to users. [Source: OpenAI, "GPT-4 Technical Report," 2023 — arxiv.org/abs/2303.08774]

This means the work you do today to build brand authority — getting press coverage, earning review site listings, updating your Wikipedia presence — may not appear in ChatGPT or Claude responses for three to twelve months. This is why AI visibility is fundamentally a long-term strategy, and why sustained, consistent authority building is more effective than one-time campaigns. Content that persists across authoritative sources over time is more likely to be captured in multiple training cycles.

4. Real-Time Retrieval (RAG): Searching as You Ask

What RAG Is

Not all AI engines rely solely on what they learned during training. Some search the internet in real time before generating a response. This approach is called Retrieval-Augmented Generation, commonly shortened to RAG.

The concept was introduced in a 2020 research paper by Patrick Lewis and colleagues at what was then Facebook AI Research (now Meta AI). Their work demonstrated that combining a search/retrieval system with a language model produced more accurate, more current, and more verifiable responses than using either approach alone. [Source: Lewis, P. et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks," 2020 — arxiv.org/abs/2005.11401]

A Helpful Way to Think About It

If training-data-only AI is like a closed-book exam, RAG is like an open-book exam. The student still has foundational knowledge from their studies (the training data), but before answering each question, they flip through reference materials (the internet) to find the most relevant, current information. Their final answer combines background understanding with the specific sources they just looked up.

How RAG Works, Step by Step

Here is what happens when you ask a question to a RAG-based AI engine like Perplexity:

Step 1 — You ask a question. You type your query, just as you would with any search engine or AI assistant. For example: "What are the best CRM tools for small businesses in 2026?"

Step 2 — The AI reformulates your question for search. The model takes your question and converts it into one or more optimized search queries. Your single question might become several searches — one for CRM feature comparisons, one for recent reviews, one for pricing information — to gather comprehensive results.

Step 3 — Search results are retrieved. The system queries a search index (its own or a third-party one like Bing or Google) and retrieves a set of potentially relevant web pages. Depending on the engine, this is typically 5 to 20 pages.

Step 4 — Pages are read and evaluated. The AI reads the actual content of the retrieved pages and assesses which ones are most relevant, authoritative, and useful for answering the specific question. This is where ranking signals such as authority, freshness, and relevance come into play.

Step 5 — A response is generated with citations. The AI synthesizes information from the highest-quality sources and writes a natural language response. Most RAG-based engines include numbered citations or clickable source links so you can verify the information yourself.

Which Engines Use RAG

Why RAG Matters for Your Brand

RAG-based engines are dramatically more responsive to changes in your online presence compared to training-data-only engines:

Speed of impact — New content, updated pages, and fresh press coverage can appear in Perplexity and Google AI Overview responses within hours to days of being published and indexed. You do not have to wait months for a model retraining cycle.

Content freshness counts — RAG engines actively prefer recent, up-to-date content. A product page updated last week will generally be favored over a competitor's page that has not been touched in two years. See Freshness & Update Strategy for details.

Strong SEO correlation — Because RAG systems pull from search results, pages that already rank well in traditional Google search are more likely to be retrieved and cited. Your existing SEO investments directly benefit your AI visibility on these engines.

Competitive agility — You can actively and quickly improve your RAG visibility by publishing new content, updating existing pages, and earning fresh third-party mentions. Results are measurable in weeks rather than months.

How to Optimize for RAG-Based Engines

• Make your content crawlable. Use text-based HTML pages. Avoid putting critical information inside PDFs, images, or behind JavaScript interactions that require clicks to reveal content.
• Answer questions directly. Place a clear, concise answer in the first paragraph of each page. RAG systems favor content that gets straight to the point and directly addresses the query.
• Keep content fresh. Regularly update key pages with current dates, data, and information. An "Updated February 2026" timestamp signals active maintenance.
• Use structured data markup. Schema.org markup for FAQ, Product, and Organization types helps AI understand your content structure. See Technical Optimization for AI for implementation guidance.
• Ensure fast page loads. RAG systems have retrieval time limits. If your page takes too long to load, it may be skipped entirely in favor of a faster competitor.
• Build traditional SEO authority. Strong search rankings directly improve your chances of being retrieved by RAG systems. Backlinks, domain authority, and content quality all carry over. See Backlink Authority Building.

5. Grounding: Anchoring AI to Facts

What Grounding Is

Grounding is the process of connecting an AI model's response to verifiable, external information. Its primary purpose is to reduce a well-known problem called hallucination — when an AI confidently generates information that is incorrect, made up, or entirely fabricated.

Without grounding, an AI might produce an answer that sounds perfectly convincing but contains invented statistics, fictitious sources, or inaccurate descriptions of your product. Grounding forces the model to anchor its claims to real, checkable data — making responses more accurate and trustworthy.

Types of Grounding

Different AI engines use different grounding approaches:

Web Search Grounding — The AI searches the web and anchors its response to the pages it finds. This is the most common type and overlaps significantly with RAG. Used by Perplexity, Google AI Overview, Gemini, and ChatGPT (when browsing is active).

Knowledge Graph Grounding — The AI cross-references its response against a structured database of verified facts, such as Google's Knowledge Graph or Wikidata. This provides confirmed entity information — things like a company's founding year, headquarters location, CEO name, or official website URL. Used primarily by Google AI Overview and Gemini.

Document Grounding — The AI bases its response on specific documents the user has uploaded or linked to. This is less relevant for brand visibility but important to understand. Supported by ChatGPT, Claude, and Gemini.

The Difference Between Grounding and RAG

These terms are related but not identical. RAG is a specific technique — search for information, then use it to generate a response. Grounding is a broader concept — ensuring AI responses are connected to verifiable facts by any means, which may include RAG but also includes knowledge graph lookups, fact-checking steps, and document referencing.

A simple rule: all RAG is a form of grounding, but not all grounding uses RAG.

Why Grounding Matters for Your Brand

Knowledge Graphs determine your entity identity. If your brand has a clear, accurate entry in Google's Knowledge Graph or Wikidata, grounded AI responses will use this verified information when describing you — your founding date, location, what you do, your official website. If you do not have a clear entity presence, the AI may confuse your brand with similarly named companies, products, or people. See Wikipedia & Knowledge Graphs for how to establish and maintain your entity identity.

Web search grounding means your live online content shapes AI responses. When Gemini or ChatGPT ground a response using web search, they pull from the same pages that appear in search results. Your website content, your review platform profiles, your press coverage — all of it directly influences what the AI says about you in that moment.

Inconsistent information causes problems. If your website states one price, your G2 profile lists another, and a news article from last year mentions a third, a grounded AI response may present confused or contradictory information about your brand. Maintaining consistent, accurate information across all your online properties is essential — not just for customers browsing those sites, but because AI engines are reading and synthesizing all of them simultaneously.

6. How Each Engine Handles Source Selection

Now that you understand the four core mechanisms — training data, knowledge cutoffs, RAG, and grounding — here is a summary of how they come together in each major AI engine:

Quickest to influence: Perplexity and Google AI Overview. Both use real-time search, so content changes can appear in their responses within hours to days.

Slowest to influence: Claude. Training-data only with no web search capability. Your content only reaches Claude when Anthropic releases a new model version trained on more recent data, which happens at intervals of months.

Middle ground: ChatGPT and Gemini. Both have training data foundations but also search the web in certain situations. Improvements to your searchable web presence can have relatively quick effects when these engines choose to browse; long-term authority building improves their training-data-based responses over time.

7. What This Means for Your Brand

Understanding how AI selects sources leads to a clear strategic framework. You need two parallel efforts running at the same time:

Short-Term: Optimize for Real-Time Engines

Timeframe: Results visible in days to weeks.

These actions improve your visibility on Perplexity, Google AI Overview, and the browsing modes of ChatGPT and Gemini:

• Ensure your website content is crawlable and text-based (not locked in PDFs, images, or JavaScript)
• Optimize for traditional search engine rankings — SEO directly feeds RAG-based AI
• Keep key product and service pages updated with current information and dates
• Implement structured data markup (Schema.org) on important pages
• Answer common customer questions directly in the first paragraph of each relevant page
• Monitor your citations on Perplexity and Google AI Overview to measure progress

Long-Term: Build for Training Data Inclusion

Timeframe: Results visible in 3 to 12 months.

These actions improve your visibility on ChatGPT (default mode), Claude, and the training-data foundations of Gemini:

• Get your brand mentioned in authoritative third-party sources (news, industry publications, review sites)
• Establish or improve your Wikipedia presence if your brand meets their notability requirements
• Earn coverage in industry publications, trade media, and mainstream press
• Maintain consistent brand messaging and descriptions across every online property
• Create high-quality content that other people and organizations cite and reference
• Build relationships with journalists, analysts, and industry experts who produce content that ends up in training data

The Compounding Effect

These two strategies are not independent — they reinforce each other powerfully. Content that ranks well in search today gets cited by Perplexity and Google AI Overview immediately. That same content, as it persists over time and gets referenced by other authoritative sources, becomes increasingly likely to be captured in future training data updates for ChatGPT and Claude. Your short-term SEO wins plant seeds for long-term training data inclusion.

This compounding dynamic is covered throughout the Tactical Layer articles — start with Content That AI Trusts for content strategy, and Backlink Authority Building for building the external signals that both real-time and training-based engines rely on.