Quick answer
RAG (retrieval-augmented generation) makes an AI answer from your documents: a search step fetches the relevant passages, then the model writes its reply from them. NVIDIA credits a 2020 research paper with coining the term. Vendors say it helps with current, private information, and their pages also warn about stale files and wrong passages.
Key takeaways
- RAG = a lookup step before the answer: the system searches your documents, then the model writes from the passages it found.
- You meet it as 'chat with your documents', a connected knowledge base, or file search; you choose a tool that does it and do not build it.
- Vendor pages describe benefits (current, private information, citations) and state conditions on them: stale data, irrelevant passages, access control.
- This site's questions for a seller: which documents, how often refreshed, are sources shown, what happens when nothing is found, who can see the files.
What is RAG in AI?
RAG (retrieval-augmented generation) is a way to make an AI answer from your own documents: a search step first fetches the relevant passages, then the model writes its reply using them and your question. NVIDIA's page credits a 2020 research paper with coining the term. Its abstract describes "a general-purpose fine-tuning recipe for retrieval-augmented generation (RAG)" and notes that, for large pre-trained models, "providing provenance for their decisions and updating their world knowledge remain open research problems."
In that paper the lookup source was "a dense vector index of Wikipedia". After fine-tuning and evaluating on "a wide range of knowledge-intensive NLP tasks", the authors report that their models "set the state-of-the-art on three open domain QA tasks" and, for language generation, produced "more specific, diverse and factual language than a state-of-the-art parametric-only seq2seq baseline." Those are the authors' 2020 results on research tasks, not a forecast of what a chatbot will do with your price list.
How does a chatbot answer from my own documents?
It searches first and writes second. AWS describes the order this way: the system pulls information from a new data source, and "The user query and the relevant information are both given to the LLM." The model then writes the reply with that text in front of it.
Here is an illustration, not a description of any named product. You upload a price list and a refund policy. A customer asks, "Can I return a custom-cut item?" The search step finds the refund-policy paragraph on custom orders and passes it, with the question, to the model, which writes a short answer from it.
This site's analogy, not a vendor's: think of a new employee with a binder. They do not memorize it; they find the right page and answer in their own words. If the binder is out of date or they open the wrong tab, the reply is wrong, however politely written.
Where will I meet RAG without the name?
You meet it as "chat with your documents", a connected knowledge base, or file search inside an assistant. Each description below is the vendor's own sentence, read October 7, 2026, from pages written for different audiences (developers, organizations, end users).
- A cloud platform that connects a knowledge base (AWS, for organizations building apps): "With knowledge bases for Amazon Bedrock, you can connect FMs to your data sources for RAG in just a few clicks."
- Search over uploaded files (OpenAI, API developer documentation): "It enables models to retrieve information in a knowledge base of previously uploaded files through semantic and keyword search."
- A project of uploaded files in an assistant (Anthropic, help article for paid plans): "When RAG is enabled for your project, Claude uses a project knowledge search tool to retrieve relevant information from your uploaded documents."
- Any application needing fresh, private or specialized data (Google Cloud, overview page): "RAG and grounding can be integrated into any LLM application or agent which needs access to fresh, private, or specialized data."
- A managed knowledge layer (Microsoft Learn, developer documentation): Azure AI Search "also underpins Foundry IQ, the managed knowledge layer that transforms enterprise content into reusable, permission-aware knowledge bases for agents in the Microsoft Foundry portal."
- Business manuals as a knowledge base (NVIDIA): "almost any business can turn its technical or policy manuals, videos or logs into resources called knowledge bases that can enhance LLMs."
For a website chatbot that answers from your pages, see how to add an AI chatbot to your website without coding. For pulling fields out of invoices or forms instead, see AI document data extraction for small business.
What do the vendors' own pages say RAG is, and where does it fall short?
All six pages describe a similar lookup-then-answer idea, and each states a condition or availability note. The table quotes each page; the last column is the condition or hedge on that page, not a verdict on the product.
| Page (read Oct 7, 2026) | How it describes RAG | A condition, hedge or availability note it states |
|---|---|---|
| AWS (explainer) | "the process of optimizing the output of a large language model, so it references an authoritative knowledge base outside of its training data sources before generating a response" | "what if the external data becomes stale?" It suggests updating through "automated real-time processes or periodic batch processing." |
| Google Cloud (overview page) | "an AI framework that combines the strengths of traditional information retrieval systems (such as search and databases) with the capabilities of generative large language models (LLMs)" | "If your retrieved information is irrelevant, your generation could be grounded but off-topic or incorrect." |
| Microsoft Learn (developer documentation) | "a pattern that extends LLM capabilities by grounding responses in your proprietary content" | "Opening private content to LLMs requires granular access control." |
| NVIDIA (blog explainer) | "a technique for enhancing the accuracy and reliability of generative AI models with information fetched from specific and relevant data sources" | "Getting the best performance for RAG workflows requires massive amounts of memory and compute to move and process data." |
| OpenAI (feature documentation) | "It enables models to retrieve information in a knowledge base of previously uploaded files through semantic and keyword search." | Cutting the number of results retrieved "may come at the cost of reduced answer quality." |
| Anthropic (help article) | "a technology that allows your projects to store and access significantly more knowledge than before" | "RAG for projects is available on paid plans (Pro, Max, Team, and Enterprise)." |
Anthropic's is the only one of the six pages that names paid plans for the feature. OpenAI says file search "is a tool available in the Responses API", and Microsoft marks LLM-based query planning in its agentic retrieval as preview.
Does RAG stop an AI from making things up?
NVIDIA and Google both stop short of saying RAG removes made-up answers. NVIDIA writes that RAG "also reduces the possibility that a model will give a very plausible but incorrect answer." Google's page states the dependency: "If your retrieved information is irrelevant, your generation could be grounded but off-topic or incorrect." In plain words, a model that reads the wrong paragraph can still answer from it.
Neither page gives a percentage, and this page supplies none. The 2020 paper reports a benchmark result for one research setup. A vendor's claim of better accuracy for its own product needs its own evaluation, with the task, the model and the year named.
What should I ask a seller who says "we use RAG"?
Ask five questions about your documents, not about the technology. The questions are this site's suggestion. Each is tied, where a source exists, to a limit a vendor page states.
- Which documents does it read, and who chooses them? NVIDIA describes the sources as "knowledge bases" made from manuals, videos or logs, so someone has to decide what goes in. Ask for the list, and whether a retired price list can be removed.
- How often are they refreshed? AWS asks "what if the external data becomes stale?" and says updates can run in real time or in periodic batches. Ask which one this product does, and who triggers it when you change a policy.
- Does each answer show its source? AWS says "The output can include citations or references to sources," and NVIDIA says "potentially citing." OpenAI's documentation says output text carries "annotations (references to files)" but "the file search call will not return search results by default." Ask to see an answer with the file named, and whether you can open the passage.
- What happens when nothing relevant is found? Google warns about irrelevant passages. Microsoft lists "minimum thresholds" among Azure AI Search relevance settings but does not say what a chatbot replies. Ask what it tells a customer then: a handoff to you, "I don't know", or its best guess.
- Who can see the documents? Microsoft's example: "Finance data should only be accessible to finance team, even when an executive asks the chatbot." Ask whether staff-only files can be kept out of a public chatbot, and who at the vendor can read them. That second part is not addressed on the pages we read.
Do I need to know this, or can I ignore it?
You do not need to build or tune RAG; you need to know it is there so you can ask the five questions. Anthropic's help article says of its paid-plan feature, "No setup or configuration is required," and also that "RAG activation is handled automatically based on the size of your project knowledge", so you may not be able to switch it on or off yourself.
For assistants that take actions rather than answer, see AI agents for small businesses.
How was this page made?
The bold definition is this site's plain-words version; the Lewis et al. abstract (arXiv 2005.11401) supplies the term and research context. Vendor statements come from the AWS, Google Cloud, Microsoft Learn, NVIDIA, OpenAI and Anthropic pages named in the table, all opened October 7, 2026, and are attributed rather than endorsed. This page was drafted with Claude, an AI model made by Anthropic, and checked against each vendor's published material. Anthropic is one of the six vendors covered; its entries were kept to the same kinds of statement and about the same space as the others.
Next step: pick one tool you are considering and ask its seller the five questions in writing. A seller who can answer all five about your own files has told you something; one who only says "RAG" has not.
FAQ
Is ChatGPT a RAG model?
RAG is a pattern around a model, not a model. OpenAI's developer documentation describes file search as a tool: "When the model decides to use it, it will automatically call the tool, retrieve information from your files, and return an output." Whether a particular chat uses such a tool depends on the feature in use; that page is API documentation, not a description of the consumer app (read October 7, 2026).
What is the difference between an LLM and RAG?
An LLM is the language model that writes the text; RAG is the lookup step added around it. AWS says RAG "extends the already powerful capabilities of LLMs to specific domains or an organization's internal knowledge base, all without the need to retrain the model."
Is RAG cheaper than fine-tuning a model?
Two vendors say so about retraining, and neither page gives a figure that applies to your business. AWS: "RAG is a more cost-effective approach to introducing new data to the LLM." NVIDIA: "That makes the method faster and less expensive than retraining a model with additional datasets." Treat these as vendor statements; the cost of a product is on its own pricing page.
Is RAG the same as document data extraction?
No, in this site's reading they do different jobs. RAG answers a question using passages found in your documents. Extraction pulls fixed fields, such as an invoice total, out of documents into a spreadsheet. The extraction guide linked below covers the second.