8. RAG and Embeddings

8.1 What is RAG?

RAG means Retrieval-Augmented Generation.

A RAG system retrieves relevant information and provides it to the model.

Documents
   |
   v
Index
   |
   v
User question
   |
   v
Retriever
   |
   v
Relevant context
   |
   v
LLM
   |
   v
Answer

8.2 Embeddings

An embedding maps information into a numerical vector representation.

Conceptually:

"Python API testing"
        |
        v
[0.21, -0.18, 0.73, ...]

The vector can be used for semantic similarity.

8.3 Chunking

Long documents are often divided into smaller chunks.

Bad chunking can hurt retrieval.

Good chunks should preserve enough context to be useful without becoming unnecessarily large.

8.4 RAG vs agents

RAG answers:

What information should the model retrieve?

Agents answer:

What should the AI do?

They can work together:

Agent
 |
 +--> Search documentation
 |
 +--> Retrieve relevant information
 |
 +--> Read source
 |
 +--> Run tools
 |
 v
Complete task

Exercise

Design a RAG system for a college programming curriculum.

Define:

• Documents
• Chunking strategy
• Metadata
• Retrieval query
• Answer generation
• Evaluation method