Retrieval-Augmented Generation (RAG) is a technique that lets AI answer questions using your own documents and data — not just its training knowledge.

The Problem RAG Solves

Language models know a lot, but they don't know:

• Your company's internal documentation
• Your product specifications and pricing
• Your customer data and history
• Information created after their training cutoff

RAG bridges this gap by retrieving relevant information from your documents and injecting it into the prompt.

How RAG Works

1. Index — Split your documents into chunks and convert them to vector embeddings (numerical representations)
2. Store — Save embeddings in a vector database (Pinecone, Weaviate, pgvector, Chroma)
3. Query — When a user asks a question, convert it to an embedding and find the most similar document chunks
4. Augment — Insert the retrieved chunks into the AI's prompt as context
5. Generate — The AI answers based on the retrieved information

Simple Example

Without RAG:

User: "What's our refund policy?" AI: "I don't have information about your specific refund policy."

With RAG:

User: "What's our refund policy?" System retrieves: [refund-policy.pdf, section 3.2] AI: "According to your policy, customers can request a full refund within 30 days of purchase. After 30 days, a 15% restocking fee applies..."

RAG Implementation Options

No-Code:

• Google NotebookLM — Upload documents, ask questions grounded in them
• ChatGPT with file uploads — Attach files for per-conversation RAG
• Claude Projects — Upload documents to Project Knowledge for persistent RAG

Low-Code:

• LangChain + vector DB — Python framework for building RAG pipelines
• LlamaIndex — Specialized framework for document Q&A
• Supabase pgvector — PostgreSQL-based vector storage with familiar SQL

Enterprise:

• Azure AI Search — Microsoft's managed RAG infrastructure
• Google Vertex AI Search — Google's enterprise search and RAG
• AWS Bedrock Knowledge Bases — Amazon's managed RAG service

Chunking Strategies

How you split documents matters enormously:

• Fixed-size chunks (500-1000 tokens) — Simple but may split context
• Semantic chunking — Split at paragraph/section boundaries
• Recursive chunking — Split at headers, then paragraphs, then sentences
• Overlapping chunks — Include some overlap between chunks to preserve context

Common RAG Pitfalls

• Chunk size too large — Dilutes relevant information with noise
• Chunk size too small — Loses necessary context
• Poor embedding model — Use modern embedding models (text-embedding-3, Gemini embeddings)
• No metadata — Include source, date, and section info for better retrieval
• Stale data — Set up re-indexing pipelines when documents change