Dockerize Everything: ML → LLM → Agents

This is a revision document, not a video transcript. It works best if you go through it three times, with your terminal open beside it.

• Pass 1 — Read Skim top to bottom without typing anything. Goal: recognise the vocabulary. Image, container, volume, compose, service name.
• Pass 2 — Build Rebuild all three projects from scratch, copying commands from here. Use the self-check lists as you go.
• Pass 3 — Recall Close this page. Review the questions and answers and interview questions at the bottom from memory. Whatever you miss, that's your revision list.

Every command block has a Copy button. Every command block is followed by a decoder that explains each flag — don't skip those, they're where the details are.

What you'll get: a clear mental model of Docker before you write a single command, plus a working setup on your Mac. By the end of Port 0 you should have run hello-world and be able to explain what happened.

Picture two kinds of housing:

• Virtual Machine = a standalone bungalow. Every app gets an entire house: its own kitchen, bathroom, and security guard (a full operating system). Heavy, slow to start, expensive.
• Docker = apartment flats. One building (the host operating system's core) is shared, but every flat (container) has its own lock, its own belongings, its own privacy. Lightweight, starts in seconds.

The punchline: a VM takes minutes to boot; a container takes milliseconds. That's why at Swiggy/Zomato scale you don't run VMs — you run containers. Traffic spike? Open 50 new flats in seconds.

Open Terminal (press Cmd + Space , type "Terminal", press Enter). If the terminal is new to you: it's just a way to talk to your computer with text instead of clicks. You type a command, press Enter, the computer replies.

brew install --cask docker

docker --version
docker compose version
docker run hello-world

⚠️ The step most people miss: after installing, open the Docker Desktop app once (Cmd+Space → "Docker" → Enter). A whale 🐳 icon appears in the menu bar at the top of the screen. Docker commands only work while that whale is there — the app runs the Docker "engine" in the background. No whale = every command fails with "Cannot connect to the Docker daemon" . ("Daemon" is just an old Unix word for a background program.)

When hello-world prints "Hello from Docker!" — you have just pulled an image from Docker Hub and turned it into a running container. First tiffin delivered. 🎉

What you'll build: a sklearn model (a food-delivery ETA predictor) served via FastAPI, packed into a container. This is where the core fundamentals live: Dockerfile anatomy, layers, caching, port mapping, .dockerignore.

Build the skeleton first:

mkdir quickbite-eta && cd quickbite-eta
touch train.py app.py requirements.txt Dockerfile .dockerignore

scikit-learn==1.5.2
pandas==2.2.3
fastapi==0.115.6
uvicorn==0.34.0
joblib==1.4.2

import pandas as pd, numpy as np, joblib
from sklearn.ensemble import RandomForestRegressor

# ① make the fake training data reproducible
np.random.seed(42)
n = 5000
# ② generate sample orders with the same features the API will receive
df = pd.DataFrame({
    "distance_km": np.random.uniform(0.5, 12, n),
    "prep_time_min": np.random.uniform(5, 30, n),
    "rider_available": np.random.randint(0, 2, n),
    "is_raining": np.random.randint(0, 2, n),
})
# ③ compute a noisy ETA target so the model has a pattern to learn
# ETA = base + distance*3 + prep + rain penalty + rider penalty + noise
df["eta_min"] = (8 + df.distance_km*3 + df.prep_time_min*0.7
                 + df.is_raining*9 + (1-df.rider_available)*6
                 + np.random.normal(0, 2, n))

# ④ train the model to predict eta_min from the order features
X, y = df.drop(columns=["eta_min"]), df["eta_min"]
model = RandomForestRegressor(n_estimators=60, random_state=42).fit(X, y)
# ⑤ save the trained model so the API can load it without retraining
joblib.dump(model, "eta_model.pkl")
print("Model saved: eta_model.pkl ✅")

from fastapi import FastAPI
from pydantic import BaseModel
import joblib, pandas as pd

# ① create the API and load the trained model once at startup
app = FastAPI(title="QuickBite ETA")
model = joblib.load("eta_model.pkl")

# ② describe the order fields every prediction request must send
class Order(BaseModel):
    distance_km: float
    prep_time_min: float
    rider_available: int
    is_raining: int

@app.get("/")
def health():
    return {"status": "QuickBite ETA is live 🛵"}

@app.post("/predict")
def predict(order: Order):
    # ① convert validated JSON into the dataframe shape the model expects
    X = pd.DataFrame([order.model_dump()])
    # ② run the model and round the ETA for display
    eta = round(float(model.predict(X)[0]), 1)
    # ③ send the prediction back as JSON for the caller
    return {"eta_minutes": eta, "message": f"Your food arrives in {eta} min 🍔"}

__pycache__/
*.pyc
venv/
.venv/
.git/
.env
*.ipynb
eta_model.pkl
data/raw/

docker ps
docker ps -a
docker logs -f eta-service
docker exec -it eta-service bash
docker stop eta-service
docker rm eta-service

docker exec makes the idea concrete: you are standing inside a tiny, separate Linux world living inside your Mac.

What you'll build: a RAG chatbot (FastAPI + OpenAI API + ChromaDB). New concepts: secrets/env vars, docker compose, volumes, multi-container networking, startup readiness . This code is tested — it includes fixes for two real bugs that show up every time.

mkdir scalergpt && cd scalergpt
mkdir docs
touch app.py ingest.py requirements.txt Dockerfile docker-compose.yml .env.example .dockerignore .gitignore

fastapi==0.115.6
uvicorn==0.34.0
openai==1.59.7
chromadb-client==0.6.3
python-dotenv==1.0.1

This is the corrected code. The naive version crashes at startup — the next card explains exactly why, because that bug is the best lesson in the whole guide.

import os, sys, time
import chromadb
from chromadb.utils import embedding_functions
from fastapi import FastAPI, HTTPException
from openai import OpenAI
from pydantic import BaseModel

app = FastAPI(title="ScalerGPT")

# ① fail loudly if the key is missing instead of showing a cryptic traceback
# Fail LOUDLY and clearly if the key is missing - not with a cryptic traceback
API_KEY = os.getenv("OPENAI_API_KEY", "").strip()
if not API_KEY or API_KEY.startswith("sk-paste"):
    sys.exit("[ScalerGPT] OPENAI_API_KEY missing. Put a real key in .env")

llm = OpenAI(api_key=API_KEY)

# ② create the embedder Chroma will use for similarity search
# The thin client has no built-in embedder - we must supply one explicitly
openai_ef = embedding_functions.OpenAIEmbeddingFunction(
    api_key=API_KEY, model_name="text-embedding-3-small")

# ③ read the Chroma network address from environment variables
CHROMA_HOST = os.getenv("CHROMA_HOST", "localhost")
CHROMA_PORT = int(os.getenv("CHROMA_PORT", "8000"))

def connect_to_chroma(retries=30, delay=2):
    # ① retry because Chroma may be started before it is ready
    # Chroma takes a few seconds to boot. depends_on only waits for its
    # container to START, not to be READY - so we knock politely and retry.
    for attempt in range(1, retries + 1):
        try:
            # ② create a client and prove the service answers a heartbeat
            client = chromadb.HttpClient(host=CHROMA_HOST, port=CHROMA_PORT)
            client.heartbeat()
            print(f"[ScalerGPT] Connected to chroma at {CHROMA_HOST}:{CHROMA_PORT}", flush=True)
            # ③ return the ready client so the app can continue startup
            return client
        except Exception as e:
            # ④ wait before trying again so startup races can self-heal
            print(f"[ScalerGPT] Waiting for chroma ({attempt}/{retries}): {type(e).__name__}", flush=True)
            time.sleep(delay)
    # ⑤ stop with a clear message if Chroma never becomes reachable
    sys.exit(f"[ScalerGPT] Could not reach chroma at {CHROMA_HOST}:{CHROMA_PORT}")

# ④ connect to Chroma and open the notes collection
chroma = connect_to_chroma()
collection = chroma.get_or_create_collection(name="notes", embedding_function=openai_ef)

# ⑤ define the request body shape for /ask
class Question(BaseModel):
    query: str

@app.get("/")
def health():
    return {"status": "ScalerGPT is live 📚", "docs_indexed": collection.count(),
            "chroma_host": CHROMA_HOST, "chroma_port": CHROMA_PORT}

@app.post("/ask")
def ask(q: Question):
    # ① stop early if no documents were indexed
    if collection.count() == 0:
        raise HTTPException(status_code=400,
            detail="No documents indexed. Run: docker compose exec app python ingest.py")
    # ② RETRIEVE - find the 3 most relevant chunks
    hits = collection.query(query_texts=[q.query], n_results=3)
    documents = hits.get("documents") or [[]]
    context = "\n\n---\n\n".join(documents[0])
    # ③ AUGMENT - paste those chunks into the prompt
    system_prompt = ("You are ScalerGPT, a helpful teaching assistant. "
        "Answer using ONLY the context below. If it does not contain the answer, "
        f"say you don't know.\n\nCONTEXT:\n{context}")
    # ④ GENERATE - let the LLM write the final answer
    resp = llm.chat.completions.create(model="gpt-4o-mini",
        messages=[{"role": "system", "content": system_prompt},
                  {"role": "user", "content": q.query}])
    # ⑤ return the model answer plus how many chunks supported it
    return {"question": q.query, "answer": resp.choices[0].message.content,
            "sources_used": len(documents[0])}

You'll also need ingest.py — it reads every file in docs/ , splits them into paragraph chunks, and loads them into Chroma, using the same retry pattern.

What you'll build: a 3-container agentic system — an agent service (LLM + tool-calling loop), a tools service (separate microservice), and Redis (conversation memory). New concepts: microservice separation, private networking (no published port!), and one compose file running it all.

mkdir deskbuddy && cd deskbuddy
mkdir agent tools
touch docker-compose.yml .env.example
touch agent/app.py agent/requirements.txt agent/Dockerfile
touch tools/app.py tools/requirements.txt tools/Dockerfile

from fastapi import FastAPI
from pydantic import BaseModel
import datetime

# ① create a tiny API that exposes utility tools
app = FastAPI(title="DeskBuddy Tools")

# ② describe the calculator request body
class Calc(BaseModel):
    expression: str

@app.post("/calculator")
def calculator(c: Calc):
    try:
        # ① evaluate the expression in a stripped-down namespace for the demo
        # demo only - never use eval in production!
        return {"result": eval(c.expression, {"__builtins__": {}})}
    except Exception as e:
        # ② report tool errors as data so the agent can continue
        return {"error": str(e)}

@app.get("/datetime")
def now():
    # ① return the current timestamp for the agent's clock tool
    return {"now": datetime.datetime.now().isoformat()}

# ① connect to shared Redis memory and the private tools service
r = redis.Redis(host=os.getenv("REDIS_HOST", "redis"), port=6379, decode_responses=True)
TOOLS_URL = os.getenv("TOOLS_URL", "http://tools:7000")

@app.post("/chat")
def chat(req: Chat):
    # ① load this session's memory and append the new user message
    key = f"history:{req.session_id}"
    history = [json.loads(m) for m in r.lrange(key, 0, -1)]   # load memory
    history.append({"role": "user", "content": req.message})

    # ② let the model decide whether it needs a tool, up to five rounds
    for _ in range(5):                                  # the agent loop
        # ③ ask the model for the next assistant message or tool call
        resp = llm.chat.completions.create(
            model="gpt-4o-mini", messages=history, tools=TOOL_DEFS)
        msg = resp.choices[0].message
        if not msg.tool_calls:                            # no tool needed?
            break                                         # then we're done
        # ④ save the assistant tool request before running the tool
        history.append(msg.model_dump(exclude_none=True))
        for tc in msg.tool_calls:                        # run each requested tool
            # ⑤ execute each requested tool and feed its result back to the model
            result = call_tool(tc.function.name, json.loads(tc.function.arguments))
            history.append({"role": "tool", "tool_call_id": tc.id,
                            "content": json.dumps(result)})

    # ⑥ save memory back to Redis, then return the final answer
    # save memory back to redis, return final answer

Look at the two addresses: http://tools:7000 and host="redis" . Again — service names, not IPs. By project 3 this should feel natural.

This is the section to keep open on a second screen while you work. Use the full cheatsheet, follow the debug tree when something breaks, and re-read the golden rules before any interview.

When something breaks, don't guess. Walk this:

Something is broken.
├─ Does docker ps show my container?
│  ├─ NO → run docker ps -a (or compose ps -a)
│  │  ├─ Status "Exited" → docker logs <name> ← the answer is in here
│  │  │  ├─ "Connection refused" → started ≠ ready, or wrong port/host
│  │  │  ├─ "KeyError / missing key" → .env not loaded → --force-recreate
│  │  │  └─ Python traceback → your code, not Docker. Fix, then --build
│  │  └─ Not listed at all → the build failed. Scroll up in the build output.
│  └─ YES → keep going ↓
├─ Can I reach it from my Mac (curl localhost:PORT)?
│  ├─ NO → check three things, in order:
│  │  1. Is there a -p / ports: line at all? (no ports = private, by design)
│  │  2. Is the LEFT number the one I'm curling? (host:container)
│  │  3. Does the app bind to 0.0.0.0, not 127.0.0.1?
│  └─ YES → keep going ↓
├─ Can container A reach container B?
│  ├─ NO → am I using the SERVICE NAME as host (not localhost)?
│  │       am I using the INTERNAL port (not the published one)?
│  │       are both services in the same compose file?
│  └─ YES → keep going ↓
└─ Is my code change showing up?
   ├─ NO → you rebuilt? docker compose up -d --build
   │       edited .env? docker compose up -d --force-recreate
   └─ YES → it's a logic bug now. Now debug the app logic.

1. Warm-up — Push the QuickBite ETA image to Docker Hub ( docker tag + docker push ).
2. Medium — Replace ScalerGPT's retry loop with a compose healthcheck on chroma + depends_on: condition: service_healthy . Write two lines on which approach you'd pick and why.
3. Hard — Add a weather tool to DeskBuddy by updating ONLY the tools service — no agent rebuild. This proves you understood microservices.
4. Advanced — Multi-stage builds on all three projects; cut image sizes by 40%. Record before/after from docker images .
5. Bonus — Break something on purpose (wrong port, missing .env, edit without rebuild), then fix it using only the debug tree above. This is a fast way to remember the steps.

Read each question. Say your answer out loud before reading the notes. Recall builds memory better than re-reading. Anything you get wrong, go back to that Port.

Image

A read-only, layered package of your app plus everything it needs to run.

Container

A running instance of an image, isolated from the host and from other containers.

Layer

The filesystem diff produced by one Dockerfile instruction. Cached and reused across builds.

Build context

The folder you pass to docker build (the lonely . ). Everything in it, minus .dockerignore, is sent to the Docker engine.

Registry / Docker Hub

The remote store where images are pushed and pulled from.

Tag

The human-readable name:version label on an image.

Daemon

The background engine that actually does the work. Docker Desktop starts it; the whale icon means it's alive.

Port publishing

Mapping a host port to a container port so the outside world can reach in ( -p host:container ).

Volume

Docker-managed storage that outlives containers. For anything you can't afford to lose.

Bind mount

Mapping a host folder straight into a container. Great for live-reload in development, avoided in production.

Compose

A tool that reads one YAML file and runs a whole multi-container application.

Service

One entry in a compose file — and also the DNS name other containers use to reach it.

Healthcheck

A command Docker runs periodically to decide whether a container is actually ready, not just started.

Multi-stage build

Using one image to build and a second, smaller one to run — the standard way to shrink production images.

Orchestrator

The system that runs containers across many machines (Kubernetes, ECS). Compose is the single-machine version of the same idea.

"Writing code is half the job. Making it run anywhere in the world — that's engineering. Docker is the box that carries your work to the world."

Where to go next: Kubernetes — for when you have 10,000 boxes to manage instead of three. Everything you learned here (images, ports, volumes, service names, readiness) maps directly onto it. You already understand the core ideas. 🐳