Agentic RAG, one station at a time

Why RAG exists

A smart model that has never seen your documents is still guessing about them.

A Large Language Model (LLM) like DeepSeek learned from a huge amount of public text. So it knows a lot of general things: what finance is, how Python works, what a Transformer is.

But it has never seen your private documents: your company's internal financial report, your HR policy, your client contracts, your class notes. Those were not in its training data. If you ask about them, it either says "I don't know" or, worse, confidently makes something up.

RAG stands for Retrieval-Augmented Generation. Read the three words backwards and you have the whole idea:

• Generation: the LLM writes an answer…
• Augmented: …but we first add extra information to its prompt…
• Retrieval: …which we fetched (retrieved) from our own documents.

No retraining, no fine-tuning. We simply hand the model the right pages at the moment it needs them.

Traditional RAG, step by step

Two phases: prepare the library once, then answer questions from it many times.

Step 1: Load

Read the documents. They can be PDFs, Word files, web pages, text files. In this project we use one PDF: the research paper that introduced the Transformer architecture (the "T" in GPT).

Step 2: Split into chunks

We cut the long text into small pieces called chunks, for example 1,000–2,000 characters each. Small pieces let us later pick only the parts that matter. There are different splitting techniques: by fixed size, by paragraph, by heading, and so on. Often consecutive chunks overlap a little so a sentence cut at the edge is not lost.

Step 3: Embed (turn text into numbers)

Computers can't compare meanings directly, but they are excellent at comparing numbers. An embedding model converts each chunk into a list of numbers called a vector. The model we use, BAAI/bge-small-en-v1.5, turns every chunk into 384 numbers.

The magic: texts with similar meaning get similar numbers, even if they use different words. "How do I cancel my order?" and "Steps to stop a purchase" land close together.

You see this idea every day: a music app puts songs that "feel" alike next to each other, so when you like one romantic Hindi song it suggests ten more like it. It isn't matching song titles; it's matching how close they are in its number-space. Embeddings do the same for text.

Step 4: Store in a vector database

We save every vector (plus its original text) in a vector database. It is built for one job: given a new vector, quickly find the most similar stored vectors. We use ChromaDB, which runs on your laptop and saves to disk, so indexing happens only once.

Then, for every question: retrieve and generate

1. Embed the user's question with the same embedding model.
2. Ask the vector database for the top-k most similar chunks (k = 4 in our code).
3. Put the question and those chunks into a prompt and send it to the LLM.
4. The LLM writes an answer using those chunks.

Where plain RAG struggles

It follows the same fixed path every time, even when that path is wrong.

• Retrieval can miss. The top-k chunks may be only loosely related. The LLM then answers from the wrong material.
• It always searches the documents. Ask "What's the weather today?" and it still digs through your PDF, finds nothing useful, and answers badly.
• It can still hallucinate. RAG reduces made-up answers a lot, but the LLM can still add facts that are not in the chunks.
• Nobody checks the work. Whatever comes out goes straight to the user.

Agentic RAG: the big idea

Replace the fixed pipeline with a small team of AI agents that decide, fetch, and double-check.

An agent is an LLM given a role, a goal, and permission to use tools. Instead of one model doing everything blindly, we split the work across five specialists. Each is powered by an LLM, and each does one narrow job well.

Two tools

• RAG tool: the whole traditional RAG pipeline from chapter 2, packed into one function the agents can call.
• Web search tool: searches the internet and returns the top 3 results.

Because tools are just functions, you can keep adding more later: a SQL database tool, a graph database tool for customer relationships, a calculator, a company API. That is what makes this design powerful and easy to extend.

Two LLMs

The system uses two models with different jobs:

• Main LLM (DeepSeek, through OpenRouter): the brain of all five agents. It orchestrates the whole workflow.
• RAG LLM (fast model on Groq): sits inside the RAG tool and turns the retrieved chunks into a short answer. Speed matters here, so we use Groq, which is famous for very fast responses.

Follow one question, end to end

Like a relay race: each runner does one lap and passes the baton.

Here is exactly what each agent receives and hands over for the question "What is self-attention?". The texts are shortened, but this is the real shape of what you'll see in your terminal.

1. Routerthe reception desk
   ReceivesThe question + a one-line note: "our database holds a paper about Transformers, attention, positional encoding…"

   Hands overvectorstore
   Self-attention is a Transformer topic, so the PDF should know.
2. Retriever + RAG toolthe librarian
   ReceivesThe word vectorstore + the question. Calls rag_tool("What is self-attention?").

   Hands overSOURCE: document. "Self-attention relates different positions of one sentence to build its meaning…" + the 4 chunks it came from.
3. Answer Graderthe teacher
   ReceivesThe question + the retrieved answer.

   Hands overyes
   The answer talks about self-attention, which is what was asked.
4. Hallucination Graderthe auditor
   ReceivesThe answer + the 4 source chunks + the relevance grade.

   Hands overyes
   Every sentence in the answer can be traced to a chunk.
5. Answer Generatorthe news anchor
   ReceivesThe answer + both grades (yes, yes).

   Hands overA clean, short final answer for the user. No mention of routing or grading.

Same team, different question

Ask "Who won yesterday's cricket match?" and the baton takes the other track:

1. Router
   ReceivesThe question + the note about the PDF.

   Hands overwebsearch. A research paper can't know yesterday's score.
2. Retriever + web tool
   Receiveswebsearch. Calls web_search_tool(...).

   Hands overSOURCE: web. Top 3 results with titles, links and text.
3. Graders, then Answer Generator
   ReceivesThe web answer + the snippets.

   Hands overBoth checks pass → a short answer built from the search results.

What one question costs

Real-world use cases

Same pattern everywhere: private documents for the library line, live data for the web line, checks because mistakes are costly.

Plain RAG or agentic RAG? How to choose

The toolbox we use

Every name you will see in the code, in one line each.

Agent, Task, Tool, Crew

Four words. Learn them once and every CrewAI project becomes readable.

An agent has three key settings

• role: the job title. "Router".
• goal: what success means. "Route user questions to the vectorstore or web search."
• backstory: who this agent is and how it thinks. "You are an expert at deciding…". You talk to the agent as if it were a person with expertise.

Plus llm (its brain), allow_delegation (may it hand work to another agent? we say no), and verbose (print its thinking so we can learn from it).

A task has four key settings

• description: the instructions, with a {question} placeholder.
• expected_output: the exact shape of the result ("one word: yes or no").
• agent: who does it.
• context: which earlier tasks' outputs it may read. This is how information flows down the line.

Prompt engineering is the real work

The backstories and descriptions are prompts. Getting them right takes several rounds of trial and error. Three fixes that came from real testing of this exact project:

1. The Router kept matching keywords instead of understanding the question. Adding "make routing decisions based on the semantic meaning of the question, not just keyword matching" fixed it.
2. Graders wrote long explanations, which confused the next agent. Adding "exactly one word: yes or no, no explanation" fixed it.
3. The final answer explained how it was found instead of just answering. Adding "the final answer contains only the answer, not how you found it" fixed it.

Set up the project

About 10 minutes, most of it waiting for installs.

Folder structure

code/
├── step0_simulate_flow.py   # run first: no keys, no installs
├── step1_plain_rag.py       # run second: classic RAG, no agents
├── config.py                # all settings in one place
├── llms.py                  # the two LLMs
├── tools.py                 # rag_tool + web_search_tool
├── agents.py                # the 5 agents
├── tasks.py                 # the 5 tasks
├── main.py                  # wires everything and runs it
├── requirements.txt         # libraries to install
├── .env.example             # template for your secret keys
├── README.md
├── data/                    # the PDF is downloaded here
└── answers_log.md           # created on first run: every Q&A is saved

Step 0: feel the flow first (no keys needed)

Before any AI, run the simulator. It does the same five steps with simple Python rules, so you can see the flow clearly, including the hallucination grader catching a made-up fact.

$ cd code
$ python step0_simulate_flow.py

QUESTION: How does self-attention work in a Transformer?
  1. Router            -> vectorstore
  2. Retriever         -> source=document
        chunk (similarity 0.34): Self-attention lets each word in a sentence look at...
  3. Answer Grader     -> relevant? yes
  4. Hallucination Grd -> grounded? yes
  5. Answer Generator  -> Self-attention lets each word in a sentence look at every other word...

QUESTION: What is multi-head attention?
  1. Router            -> vectorstore
     (demo: the retriever added a made-up fact!)
  2. Retriever         -> source=document
  3. Answer Grader     -> relevant? yes
  4. Hallucination Grd -> grounded? no
  5. Answer Generator  -> Web result: Multi-head attention splits attention into several 'heads'...

Step 1: a virtual environment

A virtual environment is a private box of Python libraries just for this project, so it never clashes with other projects.

$ python -m venv venv

# activate it (pick your system)
$ venv\Scripts\Activate.ps1          # Windows PowerShell
$ source venv/Scripts/activate      # Windows Git Bash
$ source venv/bin/activate          # Mac / Linux

You'll see (venv) at the start of the terminal line once it's active. Use Python 3.10 to 3.13.

Step 2: install the libraries

# Agentic RAG project - created by Shivank Agrawal
# Python 3.10, 3.11, 3.12 or 3.13
crewai[tools]>=1.15,<2      # agents, tasks, crew + PDFSearchTool + ChromaDB (tested on 1.15)
sentence-transformers>=3    # runs the Hugging Face embedding model locally (installs PyTorch)
tavily-python>=0.7          # web search for AI agents
python-dotenv>=1.0          # reads the secret keys from the .env file

$ pip install -r requirements.txt

CrewAI pulls in many dependencies, and the embedding library installs PyTorch, so the first install can take several minutes and a few hundred MB. Start it, then get your keys while it runs.

Step 3: choose how to run the brain

The five agents need one "brain" (LLM). Pick one of three ways with the LLM_MODE setting:

Step 4: get your API keys

Step 5: put the keys in .env

Copy .env.example to a new file named exactly .env and paste your keys after the = signs.

# ============================================================
# Agentic RAG project - created by Shivank Agrawal
# 1) Copy this file and rename the copy to:  .env
# 2) Paste your own keys after the = sign (no quotes, no spaces)
# 3) NEVER share or upload your .env file
# ============================================================

# --- API keys -------------------------------------------------
OPENROUTER_API_KEY=
GROQ_API_KEY=
TAVILY_API_KEY=

# --- Main brain for the 5 agents --------------------------------
# openrouter = DeepSeek in the cloud (tiny pay-as-you-go cost)
# groq       = free-tier practice model (not DeepSeek)
# ollama     = DeepSeek on your laptop (free, slow, ~8 GB RAM)
LLM_MODE=openrouter
OPENROUTER_MODEL=deepseek/deepseek-v4-flash
OLLAMA_MODEL=deepseek-r1:8b
OLLAMA_BASE_URL=http://localhost:11434

# --- Fast LLM inside the RAG tool -------------------------------
# groq = fast cloud model   |   same = reuse the main brain
RAG_LLM_MODE=groq
GROQ_MODEL=llama-3.3-70b-versatile

# --- Embeddings + document ---------------------------------------
EMBEDDING_MODEL=BAAI/bge-small-en-v1.5
PDF_PATH=data/attention_is_all_you_need.pdf
# PDF_TOPIC=one line describing what is inside YOUR pdf

# --- Limits -------------------------------------------------------
TOP_K=4
MAX_RPM=10

Plain RAG first

Before building the team, see what one person working alone can and can't do.

step1_plain_rag.py is the traditional pipeline from chapter 2 in a few lines: search the PDF, make one LLM call, print the answer. No router, no web, no checks.

def plain_rag(question: str) -> str:
    # 1. INDEXING (only slow the first time; the vectors are saved on disk)
    library = create_pdf_search_tool(config.PDF_PATH)

    # 2. RETRIEVAL: get the top-k most similar chunks
    chunks = library.run(query=question)
    print("\n----- CHUNKS FOUND (first 600 characters) -----")
    print(chunks[:600], "...")

    # 3. GENERATION: one LLM call with question + chunks
    llm = get_rag_llm()
    answer = clean_text(llm.call(RAG_PROMPT.format(context=chunks, question=question)))
    return answer

1. It reuses the same library builder as the agentic version, so you're comparing like with like.
2. It prints the chunks it found, so you can see exactly what the LLM was given.
3. One LLM call, then done. Fast and cheap.

$ python step1_plain_rag.py "What is multi-head attention?"
$ python step1_plain_rag.py "What is the weather in Delhi today?"

The first question gets a good answer. The second can't: the PDF knows nothing about Delhi's weather, so the best plain RAG can do is say NOT_FOUND_IN_DOCUMENT. There is no other place it can look.

Now run the same two questions through main.py (after the next chapters) and compare the answers, the time taken, and the journey line.

config.py and llms.py

One place for settings, one place for the two brains.

config.py: read settings from .env

# Read the .env file and put every line into "environment variables".
load_dotenv()

# ---------------------------------------------------------------------------
# 1) API KEYS  (see the study guide for where to get each one)
# ---------------------------------------------------------------------------
OPENROUTER_API_KEY = os.getenv("OPENROUTER_API_KEY", "")  # DeepSeek in the cloud
GROQ_API_KEY = os.getenv("GROQ_API_KEY", "")              # very fast cloud models
TAVILY_API_KEY = os.getenv("TAVILY_API_KEY", "")          # web search

# ---------------------------------------------------------------------------
# 2) WHICH "BRAIN" DO THE 5 AGENTS USE?
#    "openrouter" -> DeepSeek in the cloud (default). Pay-as-you-go, but a
#                    question costs a tiny fraction of a rupee.
#    "groq"       -> a fast open model on Groq's free tier (not DeepSeek).
#                    Good for practising without any payment.
#    "ollama"     -> DeepSeek running on your own laptop. Free and private,
#                    but slow, and needs about 8 GB of free RAM.
# ---------------------------------------------------------------------------
LLM_MODE = os.getenv("LLM_MODE", "openrouter").strip().lower()

# DeepSeek model on OpenRouter. It must support "tools" (function calling),
# because our Retriever agent calls tools. Check openrouter.ai/models.
OPENROUTER_MODEL = os.getenv("OPENROUTER_MODEL", "deepseek/deepseek-v4-flash")

# DeepSeek on your laptop via Ollama  (run:  ollama pull deepseek-r1:8b)
OLLAMA_MODEL = os.getenv("OLLAMA_MODEL", "deepseek-r1:8b")
OLLAMA_BASE_URL = os.getenv("OLLAMA_BASE_URL", "http://localhost:11434")

1. load_dotenv() reads your .env file and makes each line available through os.getenv(...).
2. os.getenv("NAME", "default") means: use the value from .env if present, otherwise use the default. That's why the code still runs if you skip optional settings.
3. LLM_MODE is a three-way switch: openrouter (cloud DeepSeek), groq (free practice model) or ollama (DeepSeek on your laptop).
4. The model must support tools (also called function calling), because the Retriever agent has to call our tools. Not every model can.

PDF_PATH = os.getenv("PDF_PATH", "data/attention_is_all_you_need.pdf")
PDF_URL = os.getenv("PDF_URL", "https://arxiv.org/pdf/1706.03762")

# One line that tells the Router agent WHAT is inside the PDF.
# The router never reads the PDF itself - it decides using this description.
# If you swap in your own PDF, update this line too!
PDF_TOPIC = os.getenv(
    "PDF_TOPIC",
    "the Transformer neural-network architecture: attention, self-attention, "
    "multi-head attention, positional encoding, encoder-decoder design and "
    "how it was trained for machine translation",
)

1. PDF_TOPIC is a one-line description of what's inside the PDF. The Router never opens the PDF; it decides from this description and the question. If you use your own PDF, update this line.

# ---------------------------------------------------------------------------
# 6) SAFETY LIMITS
# ---------------------------------------------------------------------------
WEB_RESULTS = int(os.getenv("WEB_RESULTS", "3"))   # how many web results to fetch
TOP_K = int(os.getenv("TOP_K", "4"))               # how many PDF chunks to retrieve
MAX_RPM = int(os.getenv("MAX_RPM", "10"))          # max LLM calls per minute (free tiers are strict)
ANSWERS_LOG = os.getenv("ANSWERS_LOG", "answers_log.md")  # every Q&A is saved here

1. TOP_K: how many chunks to bring back. MAX_RPM: at most this many LLM calls per minute, so free tiers don't block you. Like a speed governor on a school bus.
2. ANSWERS_LOG: every question, answer and journey gets saved to this file, so you can compare runs later.

"""
config.py  -  The "settings room" of our project
=================================================
Created by Shivank Agrawal

Every secret key and every model name lives here, in ONE place.
The other files simply do:   import config

Why?  Think of the main switchboard in your house. You don't rewire every
room to turn off the lights - you flip one switch. Same here: change a model
in ONE place, and the whole project follows.

The actual secret values are NOT written in this file.
They are read from a file called `.env` (copy `.env.example` -> `.env`).
Never share your `.env` file or push it to GitHub.
"""

import os

from dotenv import load_dotenv

# Read the .env file and put every line into "environment variables".
load_dotenv()

# ---------------------------------------------------------------------------
# 1) API KEYS  (see the study guide for where to get each one)
# ---------------------------------------------------------------------------
OPENROUTER_API_KEY = os.getenv("OPENROUTER_API_KEY", "")  # DeepSeek in the cloud
GROQ_API_KEY = os.getenv("GROQ_API_KEY", "")              # very fast cloud models
TAVILY_API_KEY = os.getenv("TAVILY_API_KEY", "")          # web search

# ---------------------------------------------------------------------------
# 2) WHICH "BRAIN" DO THE 5 AGENTS USE?
#    "openrouter" -> DeepSeek in the cloud (default). Pay-as-you-go, but a
#                    question costs a tiny fraction of a rupee.
#    "groq"       -> a fast open model on Groq's free tier (not DeepSeek).
#                    Good for practising without any payment.
#    "ollama"     -> DeepSeek running on your own laptop. Free and private,
#                    but slow, and needs about 8 GB of free RAM.
# ---------------------------------------------------------------------------
LLM_MODE = os.getenv("LLM_MODE", "openrouter").strip().lower()

# DeepSeek model on OpenRouter. It must support "tools" (function calling),
# because our Retriever agent calls tools. Check openrouter.ai/models.
OPENROUTER_MODEL = os.getenv("OPENROUTER_MODEL", "deepseek/deepseek-v4-flash")

# DeepSeek on your laptop via Ollama  (run:  ollama pull deepseek-r1:8b)
OLLAMA_MODEL = os.getenv("OLLAMA_MODEL", "deepseek-r1:8b")
OLLAMA_BASE_URL = os.getenv("OLLAMA_BASE_URL", "http://localhost:11434")

# ---------------------------------------------------------------------------
# 3) THE SECOND, FAST LLM - used only inside the RAG tool to turn the
#    retrieved PDF chunks into a short answer.
#    "groq"  -> very fast cloud inference (needs GROQ_API_KEY)
#    "same"  -> reuse the main brain above (no Groq key needed)
# ---------------------------------------------------------------------------
RAG_LLM_MODE = os.getenv("RAG_LLM_MODE", "groq").strip().lower()

# One Groq model name, used for the RAG LLM (and for the brain if LLM_MODE=groq)
GROQ_MODEL = os.getenv("GROQ_MODEL", "llama-3.3-70b-versatile")
GROQ_BASE_URL = "https://api.groq.com/openai/v1"

# ---------------------------------------------------------------------------
# 4) EMBEDDINGS - the model that turns text into numbers (vectors).
#    Runs locally on your laptop. Downloaded once from Hugging Face.
# ---------------------------------------------------------------------------
EMBEDDING_MODEL = os.getenv("EMBEDDING_MODEL", "BAAI/bge-small-en-v1.5")

# ---------------------------------------------------------------------------
# 5) OUR KNOWLEDGE BASE (the private document)
# ---------------------------------------------------------------------------
PDF_PATH = os.getenv("PDF_PATH", "data/attention_is_all_you_need.pdf")
PDF_URL = os.getenv("PDF_URL", "https://arxiv.org/pdf/1706.03762")

# One line that tells the Router agent WHAT is inside the PDF.
# The router never reads the PDF itself - it decides using this description.
# If you swap in your own PDF, update this line too!
PDF_TOPIC = os.getenv(
    "PDF_TOPIC",
    "the Transformer neural-network architecture: attention, self-attention, "
    "multi-head attention, positional encoding, encoder-decoder design and "
    "how it was trained for machine translation",
)

# ---------------------------------------------------------------------------
# 6) SAFETY LIMITS
# ---------------------------------------------------------------------------
WEB_RESULTS = int(os.getenv("WEB_RESULTS", "3"))   # how many web results to fetch
TOP_K = int(os.getenv("TOP_K", "4"))               # how many PDF chunks to retrieve
MAX_RPM = int(os.getenv("MAX_RPM", "10"))          # max LLM calls per minute (free tiers are strict)
ANSWERS_LOG = os.getenv("ANSWERS_LOG", "answers_log.md")  # every Q&A is saved here


def check_keys() -> list[str]:
    """Return a list of friendly warnings for settings or keys that are missing."""
    problems = []
    if LLM_MODE not in ("openrouter", "groq", "ollama"):
        problems.append(f"LLM_MODE='{LLM_MODE}' is not valid. Use openrouter, groq or ollama.")
    if RAG_LLM_MODE not in ("groq", "same"):
        problems.append(f"RAG_LLM_MODE='{RAG_LLM_MODE}' is not valid. Use groq or same.")
    if LLM_MODE == "openrouter" and not OPENROUTER_API_KEY:
        problems.append("OPENROUTER_API_KEY is empty (or choose LLM_MODE=groq / ollama).")
    if (LLM_MODE == "groq" or RAG_LLM_MODE == "groq") and not GROQ_API_KEY:
        problems.append("GROQ_API_KEY is empty (or set RAG_LLM_MODE=same).")
    if not TAVILY_API_KEY:
        problems.append("TAVILY_API_KEY is empty - web search will not work.")
    return problems

llms.py: build the two brains

@lru_cache(maxsize=1)  # build it once, then reuse the same object
def get_main_llm() -> LLM:
    """The brain shared by every agent."""
    if config.LLM_MODE == "ollama":
        # DeepSeek running on YOUR laptop. Before running:
        #   1) install Ollama from ollama.com
        #   2) in a terminal:   ollama pull deepseek-r1:8b
        return LLM(
            model=f"ollama/{config.OLLAMA_MODEL}",
            base_url=config.OLLAMA_BASE_URL,
            temperature=0.1,
        )

    if config.LLM_MODE == "groq":
        # Free-tier practice mode: a fast open model on Groq (not DeepSeek).
        return _groq_llm()

    # Default: DeepSeek in the cloud through OpenRouter.
    # The "openrouter/" prefix tells CrewAI which provider to call.
    return LLM(
        model=f"openrouter/{config.OPENROUTER_MODEL}",
        api_key=config.OPENROUTER_API_KEY,
        temperature=0.1,  # low temperature = calm, consistent answers
    )

1. LLM(...) is CrewAI's universal model connector, like a universal charger with different tips. The text before the first slash in model tells it which provider to call: openrouter/…, ollama/… or openai/….
2. temperature=0.1 keeps answers calm and consistent. Higher values make the model more creative, which we don't want for routing and grading.
3. @lru_cache(maxsize=1) means "build this once and reuse it". All five agents share one brain object.

def get_rag_llm() -> LLM:
    """The fast helper LLM used only inside the RAG tool."""
    if config.RAG_LLM_MODE == "groq":
        return _groq_llm()
    # "same" -> reuse the main brain
    return get_main_llm()

@lru_cache(maxsize=1)
def _groq_llm() -> LLM:
    # Groq speaks the same "language" (API format) as OpenAI,
    # so we use the OpenAI connector and just point it at Groq's address.
    return LLM(
        model=f"openai/{config.GROQ_MODEL}",
        base_url=config.GROQ_BASE_URL,
        api_key=config.GROQ_API_KEY,
        temperature=0.0,
    )

1. Groq speaks the same API "language" as OpenAI, so we use the openai/ connector and just change base_url to Groq's address. Same plug, different socket.
2. No Groq key? Set RAG_LLM_MODE=same and the RAG tool reuses the main brain.

def clean_text(text) -> str:
    """Reasoning models like DeepSeek-R1 sometimes print their private
    'thinking' inside <think> ... </think>. We only want the final answer."""
    text = str(text)
    text = re.sub(r"<think>.*?</think>", "", text, flags=re.DOTALL)
    return text.strip()

1. Reasoning models like DeepSeek-R1 sometimes write their private "rough work" between <think> tags, like a student's rough page in an exam. We keep only the final answer.

tools.py

The hands of the agents: one searches our PDF, one searches the web.

The library: create_pdf_search_tool

def create_pdf_search_tool(pdf_path: str) -> PDFSearchTool:
    """Build the 'library': load the PDF, chunk it, embed it, store it."""
    # One "shelf" (collection) per PDF + embedding model, so switching to a
    # different PDF or model never mixes old vectors with new ones.
    shelf = f"{Path(pdf_path).stem}_{config.EMBEDDING_MODEL.split('/')[-1]}"
    shelf = re.sub(r"[^a-zA-Z0-9_-]", "_", shelf)[:60]

    return PDFSearchTool(
        pdf=pdf_path,
        collection_name=shelf,
        limit=config.TOP_K,  # bring back the top-k most similar chunks
        config={
            # The embedding model: turns text into numbers (vectors).
            # It runs locally and is downloaded once from Hugging Face.
            "embedding_model": {
                "provider": "sentence-transformer",
                "config": {"model_name": config.EMBEDDING_MODEL},
            },
            # The vector database where the vectors are stored.
            "vectordb": {"provider": "chromadb", "config": {}},
        },
    )

1. PDFSearchTool from CrewAI does the whole indexing phase for us: load the PDF, split into chunks, embed, store in ChromaDB.
2. collection_name is the "shelf" in the vector database. We name it after the PDF and the embedding model so that switching either one never mixes old vectors with new ones.
3. limit=config.TOP_K is our top-k: bring back the 4 most similar chunks (change it in .env).
4. "sentence-transformer" means the embedding model runs locally on your laptop. It's downloaded once from Hugging Face (about 130 MB).

The RAG tool: retrieval + fast generation

def create_rag_tool(pdf_path: str):
    """Wrap the PDF search + the fast RAG LLM into ONE tool the agents can call."""
    pdf_search = create_pdf_search_tool(pdf_path)  # indexing happens here
    rag_llm = get_rag_llm()

    @tool("rag_tool")
    def rag_tool(question: str) -> str:
        """Search the private PDF knowledge base and return an answer grounded
        in the document, followed by the exact text chunks that were used.
        Input must be the user's full question as plain text."""
        # Step A - RETRIEVAL: find the most similar chunks in the vector DB
        chunks = pdf_search.run(query=question)

        if not chunks or "No relevant content found" in chunks:
            return "NOT_FOUND_IN_DOCUMENT"

        # Step B - GENERATION: the fast LLM writes an answer from those chunks
        answer = clean_text(rag_llm.call(RAG_PROMPT.format(context=chunks, question=question)))

        if "NOT_FOUND_IN_DOCUMENT" in answer:
            return "NOT_FOUND_IN_DOCUMENT"
        return f"ANSWER FROM DOCUMENT:\n{answer}\n\nCHUNKS USED:\n{chunks}"

    return rag_tool

1. We build the library once (pdf_search) and the fast LLM once (rag_llm) outside the inner function, so they are reused on every call.
2. @tool("rag_tool") turns the inner function into something an agent can call. The agent learns what the tool does by reading its docstring, so the docstring is an instruction, not decoration.
3. Step A, retrieval: pdf_search.run(query=question) returns the most similar chunks.
4. Step B, generation: the fast LLM reads only those chunks and writes an answer. The prompt says "use ONLY the context" and "reply NOT_FOUND_IN_DOCUMENT if it isn't there".
5. clean_text removes any <think> rough work. If the fast LLM itself says the answer isn't there, the tool says NOT_FOUND_IN_DOCUMENT so the Retriever knows to try the web.
6. We return both the answer and the chunks. The chunks are the evidence the Hallucination Grader needs later, like attaching receipts to an expense claim.

RAG_PROMPT = """You are a careful assistant answering questions from a document.

Use ONLY the context below. Do not use outside knowledge.
If the context does not contain the answer, reply with exactly: NOT_FOUND_IN_DOCUMENT

Context from the document:
-----
{context}
-----

Question: {question}

Answer in 3 to 6 clear sentences:"""

The web search tool

@tool("web_search_tool")
def web_search_tool(question: str) -> str:
    """Search the internet for fresh or general information that is NOT in the
    private PDF. Input must be the user's full question as plain text.
    Returns the top results with their title, link and content."""
    try:
        client = TavilyClient(api_key=config.TAVILY_API_KEY)
        response = client.search(query=question, max_results=config.WEB_RESULTS)
    except Exception as error:  # no internet, wrong key, limit reached ...
        # Return a readable message instead of crashing the whole crew.
        return f"WEB_SEARCH_FAILED: {error}"

    results = response.get("results", [])
    if not results:
        return "NO_WEB_RESULTS"

    lines = []
    for number, item in enumerate(results, start=1):
        lines.append(
            f"[{number}] {item.get('title', '')}\n"
            f"Link: {item.get('url', '')}\n"
            f"{item.get('content', '')}\n"
        )
    return "\n".join(lines)

1. The plain @tool decorator makes this ordinary Python function an agent tool.
2. TavilyClient(...).search(query=..., max_results=3) runs the search and returns clean results.
3. The try / except catches problems like no internet or a wrong key, and returns a readable message instead of crashing the whole team. One broken tool shouldn't stop the whole office.
4. We format each result as title, link and content so the agents (and you) can see where facts came from.

agents.py

Five job descriptions. Same structure, different personality.

The Router

def create_router_agent() -> Agent:
    return Agent(
        role="Router",
        goal="Route each user question to either the vectorstore (our PDF) "
        "or a web search, based on what the question is really about.",
        backstory=(
            "You are an expert at deciding whether a question can be answered "
            "from the information stored in our vector database or needs a web search. "
            f"The vector database contains a detailed document about {config.PDF_TOPIC}. "
            "You make routing decisions based on the semantic MEANING of the "
            "question, not just on matching keywords."
        ),
        allow_delegation=False,
        max_iter=5,
        verbose=True,
        llm=get_main_llm(),  # the DeepSeek brain (same one for all agents)
    )

1. The backstory tells the Router what's inside the vector database (from PDF_TOPIC). Without this it would have no idea what the PDF contains.
2. The last sentence, "semantic meaning, not keyword matching", was added after testing showed the Router was just matching words.
3. allow_delegation=False: the Router decides alone; it can't pass the job to another agent.
4. max_iter=5: at most 5 think-and-act rounds, then it must answer. A safety brake so an agent can't go round in circles and burn your credits.

The Retriever

def create_retriever_agent() -> Agent:
    return Agent(
        role="Retriever",
        goal="Use the tool chosen by the router to fetch the information "
        "needed to answer the question.",
        backstory=(
            "You are an assistant for question-answering tasks. You call exactly "
            "the tool you are told to call, pass it the user's full question, and "
            "report what the tool returned. Use the information present in the "
            "retrieved context to answer the question. Provide a clear, concise answer "
            "and never add facts that are not in the retrieved context."
        ),
        allow_delegation=False,
        max_iter=5,
        verbose=True,
        llm=get_main_llm(),  # the DeepSeek brain (same one for all agents)
    )

Notice what's not here: tools. In our design, tools are attached to the task, not the agent, so the same agent could be reused for other tasks with different tools.

def create_answer_grader_agent() -> Agent:
    return Agent(
        role="Answer Grader",
        goal="Filter out wrong or irrelevant retrievals.",
        backstory=(
            "You are a strict grader. You read the question and the retrieved answer "
            "and judge only one thing: does this answer actually address the question? "
            "If it contains the key ideas related to the question, it is relevant."
        ),
        allow_delegation=False,
        max_iter=5,
        verbose=True,
        llm=get_main_llm(),  # the DeepSeek brain (same one for all agents)
    )

def create_hallucination_grader_agent() -> Agent:
    return Agent(
        role="Hallucination Grader",
        goal="Catch made-up facts before they reach the user.",
        backstory=(
            "You are a fact checker. You compare the answer with the source text it "
            "came from and decide whether every important claim is supported by that "
            "source. An answer that invents facts, numbers or names is a hallucination."
        ),
        allow_delegation=False,
        max_iter=5,
        verbose=True,
        llm=get_main_llm(),  # the DeepSeek brain (same one for all agents)
    )

def create_answer_generator_agent() -> Agent:
    return Agent(
        role="Answer Generator",
        goal="Give the user one clear, correct and concise final answer.",
        backstory=(
            "You are the final editor. You receive the retrieved answer together with "
            "the relevance and hallucination checks. If the checks pass, you polish the "
            "answer. If they fail, you run a fresh web search and answer from that. "
            "You never explain your internal process to the user."
        ),
        allow_delegation=False,
        max_iter=5,
        verbose=True,
        llm=get_main_llm(),  # the DeepSeek brain (same one for all agents)
    )

tasks.py

The to-do list, and the wiring that passes each result to the next agent.

Task 1: route

def create_router_task(router_agent: Agent) -> Task:
    return Task(
        description=(
            "Analyse this question: '{question}'.\n"
            "Decide where its answer is most likely to be found:\n"
            "- Answer 'vectorstore' if the question is about a topic covered by our "
            "document (even if the exact words are different).\n"
            "- Answer 'websearch' if the question needs current or real-time "
            "information (news, prices, scores, today's events) or is about a general "
            "topic that our document does not cover.\n"
            "Decide using the semantic meaning of the question, not keyword matching."
        ),
        expected_output=(
            "Exactly one word: 'vectorstore' or 'websearch'. "
            "No explanation, no punctuation, nothing else."
        ),
        agent=router_agent,
    )

1. '{question}' is a placeholder. When we start the crew with inputs={"question": ...}, CrewAI fills in the real question everywhere.
2. The description gives clear rules: topics in the document go to the vectorstore; current or real-time information goes to the web.
3. expected_output forces a single word. The next agent then has an easy, unambiguous signal.

Task 2: retrieve

def create_retriever_task(retriever_agent: Agent, router_task: Task,
                          rag_tool, web_search_tool) -> Task:
    return Task(
        description=(
            "Read the router's decision from the context.\n"
            "- If it is 'vectorstore', call the rag_tool and pass it the question: "
            "'{question}'.\n"
            "- If it is 'websearch', call the web_search_tool and pass it the "
            "question: '{question}'.\n"
            "- If the rag_tool returns NOT_FOUND_IN_DOCUMENT, fall back to the "
            "web_search_tool with the same question.\n"
            "Always pass the user's full question to the tool."
        ),
        expected_output=(
            "First line: 'SOURCE: document' or 'SOURCE: web'.\n"
            "Then the answer to '{question}' based only on what the tool returned.\n"
            "Then the supporting text (document chunks or web snippets) copied as is."
        ),
        agent=retriever_agent,
        context=[router_task],
        tools=[rag_tool, web_search_tool],
    )

1. context=[router_task]: the Retriever reads the Router's one-word decision.
2. tools=[rag_tool, web_search_tool]: it may use either tool, and the description tells it which one to pick.
3. If the PDF has nothing useful (NOT_FOUND_IN_DOCUMENT), it falls back to the web on its own.

Tasks 3 and 4: the two checks

def create_grader_task(grader_agent: Agent, retriever_task: Task) -> Task:
    return Task(
        description=(
            "Look at the retrieved answer in the context and the question "
            "'{question}'. Decide whether the retrieved answer is RELEVANT to the "
            "question, i.e. it actually addresses what was asked."
        ),
        expected_output=(
            "Exactly one word: 'yes' if the retrieved answer is relevant to the "
            "question, or 'no' if it is not. No explanation."
        ),
        agent=grader_agent,
        context=[retriever_task],
    )

def create_hallucination_task(hallucination_agent: Agent, retriever_task: Task,
                              grader_task: Task) -> Task:
    return Task(
        description=(
            "Using the context, compare the retrieved answer with the supporting "
            "text (document chunks or web snippets) it came from. Decide whether "
            "the answer is grounded in / supported by that text for the question "
            "'{question}'. Also note the relevance grade from the previous check."
        ),
        expected_output=(
            "Exactly one word: 'yes' if the answer is supported by the source text "
            "and is useful (NO hallucination), or 'no' if it contains claims that the "
            "source text does not support (hallucination). No explanation."
        ),
        agent=hallucination_agent,
        context=[retriever_task, grader_task],
    )

1. Relevance and grounding are different questions. An answer can be on-topic but invented, or true but off-topic. That's why we check both.
2. Careful with the meaning of "yes": for the hallucination task, yes means safe (grounded, no hallucination).

Task 5: the final answer

def create_answer_task(answer_agent: Agent, retriever_task: Task, grader_task: Task,
                       hallucination_task: Task, web_search_tool) -> Task:
    return Task(
        description=(
            "Write the final reply to the question: '{question}'.\n"
            "Read the context: the retrieved answer, the relevance grade and the "
            "hallucination grade.\n"
            "- If BOTH grades are 'yes', return a clear and concise answer based on "
            "the retrieved answer.\n"
            "- If EITHER grade is 'no', call the web_search_tool with the question "
            "'{question}' and answer from those results instead.\n"
            "Make sure the final answer contains only the answer to the question, "
            "not an explanation of how you found it."
        ),
        expected_output=(
            "A clear and concise answer to '{question}' in plain language "
            "(about 3-6 sentences)."
        ),
        agent=answer_agent,
        context=[retriever_task, grader_task, hallucination_task],
        tools=[web_search_tool],
    )

1. This task reads all three earlier results: the answer and both grades.
2. Both "yes" → polish and return the answer. Any "no" → run a fresh web search and answer from it.
3. The last instruction stops the model from narrating its process in the final reply.

main.py

No intelligence here. It just builds the team and presses start.

def build_crew() -> Crew:
    # ---- 1. TOOLS ---------------------------------------------------------
    rag_tool = create_rag_tool(config.PDF_PATH)  # builds the vector DB (indexing)

    # ---- 2. AGENTS --------------------------------------------------------
    router = create_router_agent()
    retriever = create_retriever_agent()
    grader = create_answer_grader_agent()
    hallucination_grader = create_hallucination_grader_agent()
    answer_generator = create_answer_generator_agent()

    # ---- 3. TASKS (each one can read the output of the ones before it) ----
    router_task = create_router_task(router)
    retriever_task = create_retriever_task(retriever, router_task, rag_tool, web_search_tool)
    grader_task = create_grader_task(grader, retriever_task)
    hallucination_task = create_hallucination_task(hallucination_grader, retriever_task, grader_task)
    answer_task = create_answer_task(
        answer_generator, retriever_task, grader_task, hallucination_task, web_search_tool
    )

    # ---- 4. CREW ----------------------------------------------------------
    # Process.sequential = tasks run one after another, in the list order.
    # max_rpm = at most this many LLM calls per minute (protects free tiers).
    return Crew(
        agents=[router, retriever, grader, hallucination_grader, answer_generator],
        tasks=[router_task, retriever_task, grader_task, hallucination_task, answer_task],
        process=Process.sequential,
        max_rpm=config.MAX_RPM,
        verbose=True,
    )

1. Creating the RAG tool triggers indexing (first run only builds the vector database; later runs reuse it).
2. We create the five agents, then the five tasks, passing each task the earlier tasks it depends on.
3. Process.sequential: tasks run one after another, in list order. Simple and easy to debug.
4. max_rpm=config.MAX_RPM: the whole team may make at most 10 LLM calls a minute. If they go faster, CrewAI waits.
5. verbose=True: print everything each agent thinks and does. Turn it off later for a cleaner output.

def ask(crew: Crew, question: str) -> str:
    start = time.time()
    # kickoff() starts the team. `inputs` fills every {question} placeholder.
    result = crew.kickoff(inputs={"question": question})
    seconds = time.time() - start
    answer = clean_text(result)

    route, _, relevant, grounded, _ = (one_word(t) for t in crew.tasks)
    summary = f"route={route}  relevant={relevant}  grounded={grounded}  time={seconds:.0f}s"

    print("\n" + "=" * 70)
    print("QUESTION :", question)
    print("ANSWER   :", answer)
    print("JOURNEY  :", summary)
    print("=" * 70)

    # Keep a diary of every question, so you can compare runs later.
    with open(config.ANSWERS_LOG, "a", encoding="utf-8") as log:
        log.write(f"## {datetime.now():%Y-%m-%d %H:%M}  {question}\n\n{answer}\n\n`{summary}`\n\n")
    return answer

1. crew.kickoff(inputs={"question": question}) starts the team and fills every {question} placeholder.
2. After the run, every task's result is still available. We read the one-word results of the Router and both graders to print a journey line, like a train ticket showing the stations you passed.
3. Every question is appended to answers_log.md: your diary of experiments.

def main() -> None:
    problems = config.check_keys()
    if problems:
        print("\nPlease fix your .env file first:")
        for p in problems:
            print("  -", p)
        sys.exit(1)

    download_pdf(config.PDF_URL, config.PDF_PATH)
    crew = build_crew()

    if "--chat" in sys.argv:
        print("\nChat mode. Type a question and press Enter. Type 'exit' to stop.")
        while True:
            question = input("\nYou: ").strip()
            if question.lower() in ("exit", "quit", ""):
                break
            ask(crew, question)
        return

    question = " ".join(sys.argv[1:]).strip() or DEFAULT_QUESTION
    ask(crew, question)

1. config.check_keys() gives a friendly message if a key or setting is wrong, before anything expensive runs.
2. --chat keeps the team ready and lets you ask question after question. The PDF is indexed only once.
3. Otherwise the question comes from the command line, or falls back to a default.

def download_pdf(url: str, save_path: str) -> None:
    """Download the PDF only if we don't already have it."""
    if os.path.exists(save_path):
        print(f"PDF already present: {save_path}")
        return
    os.makedirs(os.path.dirname(save_path) or ".", exist_ok=True)
    print(f"Downloading PDF from {url} ...")
    request = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0"})
    with urllib.request.urlopen(request, timeout=60) as response, open(save_path, "wb") as f:
        f.write(response.read())
    print(f"Saved to {save_path}")

Run it and read the output

Ask one question the PDF can answer and one it can't.

$ python main.py
$ python main.py "What is multi-head attention?"
$ python main.py "What is the latest stable version of Python?"
$ python main.py --chat

What you'll see, in order

1. First run only: the PDF downloads, the embedding model downloads, and chunks are inserted into ChromaDB.
2. Router prints its task and a final answer: vectorstore for the Transformer question.
3. Retriever calls rag_tool. You'll see the tool input (the question) and the tool output (answer + chunks from the paper).
4. Answer Grader: yes, relevant.
5. Hallucination Grader: yes, grounded.
6. Answer Generator prints the final answer.
7. Finally, a summary box with the question, the answer and the journey line:

======================================================================
QUESTION : What is the self-attention mechanism in Transformers?
ANSWER   : Self-attention lets every word in a sentence look at every other
           word and decide how much attention to pay to it, using query, key
           and value vectors...
JOURNEY  : route=vectorstore  relevant=yes  grounded=yes  time=48s
======================================================================

For the Python version question, the Router should say websearch and the Retriever calls web_search_tool instead.

Why is it slow?

One question means roughly 7–8 LLM calls (5 agents, a tool call, and the RAG LLM). On fast providers the whole flow takes roughly 15–30 seconds. On free or overloaded models it can take several minutes; on a slow free model it once took about 10 minutes. On a laptop with Ollama, expect minutes too. Speed depends on the provider, its rate limits, and your hardware.

Retries are normal

Sometimes an agent hits an error (a tool call in the wrong format, a temporary API failure) and you'll see it try again. CrewAI retries automatically. Only worry if the same error repeats until the run stops.

Run DeepSeek on your laptop

No API key for the brain, full privacy, but slower and less accurate on small hardware.

1. Install Ollama from ollama.com and open it.
2. Download DeepSeek-R1 8B (about 5 GB; your laptop needs roughly 8 GB of free RAM):

$ ollama pull deepseek-r1:8b
$ ollama run deepseek-r1:8b      # optional: chat with it to test, type /bye to exit

3. In .env set:

LLM_MODE=ollama
OLLAMA_MODEL=deepseek-r1:8b
RAG_LLM_MODE=same          # optional: use DeepSeek inside the RAG tool too

Now python main.py uses DeepSeek running on http://localhost:11434. Only Tavily (web search) still needs the internet.

When things break

Read the error. It almost always tells you which of these it is.

Questions learners ask

Answers to the questions that come up every time this is taught.

How does the Router know whether the answer is in the document, if it never reads it?

It judges the meaning of the question against what it has been told the vector database contains (the PDF_TOPIC line in its backstory). It's an educated guess, not a lookup. The safety nets are the Retriever's fallback (if the RAG tool returns nothing, go to the web) and the two graders. A stronger design gives the Router a quick "peek" tool to check the vector database before deciding.

What exactly does the RAG tool do?

Everything from chapter 2 in one function. When created, it does the indexing (load, chunk, embed, store). When called, it does retrieval (find similar chunks) and a quick generation (the fast LLM writes an answer from them). The agents around it decide when to use it and check what it returns.

What is a vector database, and why not use SQL?

Text is first split into tokens, and models represent meaning as long lists of numbers. Large models use thousands of numbers per token; our small embedding model uses 384 per chunk. A normal SQL database is great at exact matches ("find customer 42") but bad at "find the 4 rows whose 384 numbers are closest to these 384 numbers". A vector database is built and indexed for exactly that similarity search.

How are Agents, Tasks and Tools related?

The task says what to do, the agent does it, and the tool helps the agent reach outside itself (a PDF, the web, an API). You give an agent a task and a tool to achieve that task's goal. They complement each other.

Can an agent pull data from a website or a company API?

Yes. Write a Python function that calls the API or scrapes the page (for example with requests and BeautifulSoup, or a browser tool like Selenium), decorate it with @tool, give it a clear docstring, and add it to a task. There's practically no limit to the tools you can add.

Where does this code run: front end or back end?

Back end. It's Python. Here we run it from the command line; in a real product you'd wrap crew.kickoff() in an API (for example with FastAPI) and a front end would call that API.

How would I deploy this for real users?

• Simple start: put the API on a cloud server such as an AWS EC2 machine. Add a GPU only if you host the LLM yourself.
• Many users at once: run several copies behind a load balancer, and use a queue for long-running jobs, since each answer takes seconds to minutes.
• Models: APIs are fastest to ship; self-hosting helps with privacy and cost at scale but needs more maintenance. It depends on your timeline and requirements.

Can I use this for my company's documents?

Yes. Put your PDF in data/, set PDF_PATH and a good PDF_TOPIC. For many documents, point the tool at a folder or build one library per department. Before real use, test with 20–30 real questions from your users and check the answers by hand. If the documents are confidential, run the brain locally with Ollama or use a provider your company has approved.

Why two different LLMs?

Different jobs need different strengths. The orchestrator needs good reasoning. The RAG step is simple ("summarise these chunks") but happens in the middle of the flow, so a very fast model keeps the whole run snappy.

Practice and self-check

Do the exercises in order. Each one teaches a different part of the system.

Exercises

1. Compare plain vs agentic. Ask the same 3 questions with step1_plain_rag.py and main.py. Note answer quality, time and cost in a small table.
2. Watch the routing. Run 3 questions about the paper and 3 about current events. Note which line each takes. Did any go to the wrong line? Why?
3. Break the Router on purpose. Delete the "semantic meaning" sentence from the Router's backstory. Ask "How do Transformers know the order of words?" Compare.
4. Change top-k. Set limit=2 and then limit=8 in tools.py. How do the answers and the evidence change?
5. Use your own PDF. Add a PDF to data/, set PDF_PATH and PDF_TOPIC in .env, and ask questions about it.
6. Add a third tool. Write a @tool that returns today's date, give it to the Answer Generator's task, and ask "What is today's date and what is self-attention?"
7. Compare speed. Time the same question with RAG_LLM_MODE=groq and RAG_LLM_MODE=same, and with LLM_MODE=groq.
8. Build a mini use case. Pick one case from the "Real-world use cases" chapter, find a public PDF for it (for example a bank's published fee schedule), and write 5 questions: 3 the PDF should answer, 2 that need the web. Check that the journey line matches your expectation every time.

Quick quiz

Glossary