Teaching Agentic AI at Carnegie Mellon
I teach the Guided Virtual Labs and run office hours for the Agentic AI Program from Carnegie Mellon University’s School of Computer Science Executive Education, delivered in collaboration with Emeritus.
The program runs seven weeks, live and online. Each week has one faculty-led live session and two hands-on labs, and it ends with a capstone. The program is built for working engineers: AI and ML engineers, backend and platform engineers, data scientists. They already know Python. They want to know how agents actually work, and they want to build one.
That is fourteen labs. I built the teaching layer for all of them: a cell-by-cell instructor guide for every lab, and a live, animated slide deck for every lab.
Office hours are the other half of the job. Students bring their notebooks, their error messages and their capstone ideas, and we work through them together. It is where I learn which parts of a lab actually landed.
All fourteen labs
Module 1: Foundations
- 1.1 Introduction to Large Language Models: Benchmarking quality, cost and latency; few-shot and native chain of thought; hallucination probes; scaling from Qwen3 0.6B to 4B; tokenization.
- 1.2 Model Context Protocol: Host, client and server; JSON-RPC 2.0 and transports; tools, resources and prompts; sampling, roots and elicitation; scope-based auth.
Module 2: Reasoning and tools
- 2.1 Building a ReAct Agent: Thought, action and observation loops; when to think versus act; recovering from partial failures.
- 2.2 Tool Calling in Agents: Tool schemas and input/output contracts; validated, safe dispatch; behavior under noisy tool outputs.
Module 3: Retrieval
- 3.1 Domain-Specific GPT with RAG: Building and indexing a domain corpus; grounding generation in retrieval with open-source models.
- 3.2 Maximal Marginal Relevance: The relevance and redundancy tradeoff; MMR from scratch; measuring faithfulness and coverage.
Module 4: Deliberate reasoning
- 4.1 Tree of Thoughts: Thought generation and evaluation; BFS over reasoning states; replicating IO vs CoT vs ToT on the Game of 24.
- 4.2 Agentic Harnesses: System prompt, tool suite, agent loop, context management, permission gate and sandbox; Claude Code, Codex CLI and opencode.
Module 5: Adaptation and multi-agent
- 5.1 Multi-Agent Coordination: Roles, protocols and shared context; pipeline, supervisor, swarm and ensemble topologies; CrewAI and LangGraph.
- 5.2 LLM Fine-Tuning: Dataset preparation; LoRA; monitoring training; measuring gains and regressions.
Module 6: Reliability
- 6.1 Guardrailing Hallucinations: Hallucination taxonomy; guardrails before, during and after generation; ablations.
- 6.2 Monitoring Trajectories for Sabotage: Trajectory logging; suspicion monitors calibrated to a false-positive budget; ROC and AUC; human escalation.
Module 7: Shipping
- 7.1 Building a Plugin: MCP server, skill, agent and hook packaged for Claude Code and Gemini CLI.
- 7.2 Capstone: A modular web-research agent, graded on the quality of its report.
An agent is just a loop around an LLM. That is the kind of plain-language framing my guides open with, and almost everything in the course is a variation on that loop.
How I built the labs to be taught
In a lab, students have a notebook open, and the slides have to keep pace with the code without giving the answers away.
So the decks are built around reveal on keypress. Nothing appears until I press a key. Students predict first, then we look.
Diagrams build one step at a time instead of appearing finished. A finished figure is something you glance at. A diagram that assembles itself in front of you is something you follow.
Each lab is its own deck, so any lab can stand on its own. Every lab opens with a plain-language big picture and every deck closes with the same wrap-up shape: a handful of takeaways, a few discussion questions, and a one-sentence bridge to the next lab. Speaker notes, a clickable outline and fullscreen are built in.
The guides are the other half. Each one has a time budget for every section, “ask the class” prompts placed at specific moments, notes on what students should expect to see, and notes on the mistakes to watch for. While teaching I run two screens: the notebook on one, the guide on the other. The guides also double as onboarding for new lab instructors and TAs.
Two principles run through the whole thing.
Build it by hand before you use the framework. A ReAct loop with no library. A harness in a couple dozen lines. MMR from scratch. Then the same idea at production scale.
Failure is the lesson. Most labs run on small local open models on purpose, so the class can watch scale effects and breakdowns directly. Where things break is usually where the understanding is.
The clips below are the concept diagrams from the decks, one or two per module, recorded straight from the live slides.
Module 1: Foundations of Agentic AI and LLM Capabilities
The course starts by taking the magic out. An LLM predicts the next token. We compare plain prompting, few-shot chain of thought and native thinking modes across a range of small open models, and treat model choice as what it really is: a tradeoff between quality, cost and latency.
Then tokenization. The classic “is 9.11 bigger than 9.9” failure is the best argument I know for why agents need tools. A model that can write working Python can still get that wrong, and the reason lives in how the text was cut up before the model ever saw it.
Lab 1.1: following “9.11 is larger…” from raw text to subword tokens, token IDs, and embedding plus position vectors on the way into the model.
The second lab is the Model Context Protocol. Without a shared protocol, M applications and N tools means M times N custom integrations. With one, it is M plus N. We cover tools, resources and prompts, the transports, and authentication.
Lab 1.2: the MCP architecture. A host embeds the LLM and runs one client per server, and every message is JSON-RPC 2.0.
Module 2: Agent Architecture: Memory, Tools, and Reasoning Loops
This is where the loop gets built. Students write a ReAct agent by hand, Thought, Action, Observation, repeat, with no framework underneath, and then watch where it breaks.
Lab 2.1: the ReAct cycle step by step, and which parts the model writes versus which parts our code writes.
The second lab gives the same tools to a model four different ways: hand-written ReAct, LangChain with a local model, native tool calling, and an MCP server. The tools never change. Only the way they are described and parsed does. Once that clicks, every agent framework looks a lot less mysterious.
Module 3: RAG Agents and Vector Databases
Retrieval is the first real memory an agent gets. Students build a small dense passage retriever, look at how retrieved passages get combined into an answer, and weigh long context against RAG against MemGPT-style memory.
Lab 3.1: assembling retrieval-augmented generation. A DPR retriever pulls the top-K passages from a non-parametric index, and a seq2seq generator combines them into the answer.
The second lab is Maximal Marginal Relevance, built from scratch, to trade relevance against redundancy. The predict-then-run moment here is a good one: which five documents will come back, and why did the duplicates survive? The takeaway is that the right setting depends on the geometry of your embeddings, so it has to be tuned rather than copied.
Module 4: Tree-of-Thought Reasoning and Multi-Agent Concepts
The first lab rebuilds Tree of Thoughts (Yao et al., NeurIPS 2023) on the Game of 24: a proposer, an evaluator and a search over partial solutions, benchmarked against plain prompting and chain of thought. Part of the lesson is counting how many more model calls that better reasoning takes.
Lab 4.1: from IO prompting to chain of thought, self-consistency voting, and finally Tree of Thoughts, which searches, evaluates and backtracks over partial reasoning.
The second lab is about harnesses, the loop at the core of the coding agents many students already use without looking under the hood. We write a minimal agent loop over a tool registry, compare Claude Code, Codex CLI, Antigravity CLI and opencode, and drive opencode headlessly while reading its event stream. The anchor paper is SWE-agent.
Lab 4.2: coding agents stripped down to the loop they share. The transcript goes to a stateless model, the harness runs any tool calls, appends the results, and repeats until a final answer comes back.
Module 5: Multi-Agent Workflows With CrewAI and LangGraph
Adding agents adds cost. In LangGraph, students build shared state, conditional routing, capped retry loops, failover and a tool adapter, and they measure what each coordination choice costs.
Lab 5.1: four design questions for multi-agent systems and four standard topologies: linear pipeline, supervisor, peer-to-peer swarm, and ensemble vote.
The second lab is fine-tuning with LoRA on a small open model: domain-adaptive pretraining, then instruction tuning, while training a tiny fraction of the weights. The line I want everyone to leave with is that training loss is not evaluation.
Lab 5.2: LoRA freezes the pretrained W0 and trains only a small low-rank update BA, about 56x fewer weights for a single Qwen projection.
Module 6: Evaluation, Guardrails, Logging, and Observability
The first lab builds guardrails before, during and after generation for a medical-safety agent, chains them into one pipeline, and runs an ablation to see what each layer actually buys. Then we break every cheap guardrail on purpose, including showing how easily a chain-of-thought monitor can be evaded.
Lab 6.1: where hallucinations enter an agent pipeline, and why guardrails before, during and after generation catch more than a single check at the end.
The second lab rebuilds a small SHADE-Arena-style loop: log the agent’s trajectory, score it with a suspicion monitor calibrated to a false-positive budget, and escalate to a human. The uncomfortable result is that an attacker that hides its intent in its hidden reasoning gets past a monitor that only watches actions. The readings include Anthropic’s SHADE-Arena and recent work on chain-of-thought obfuscation and monitorability.
Watching only what an agent does is not the same as knowing what it is doing.
Module 7: Capstone Project
The last module turns what students built into something they can ship. In the first lab, one MCP server plus a skill, a sub-agent and session hooks become a single plugin that installs in both Claude Code and Gemini CLI. The protocol is the portability layer.
Lab 7.1: the host (Claude Code or Gemini CLI) runs the agent loop while a plugin supplies the parts it calls: an MCP tool, a skill, an agent persona and a lifecycle hook.
The capstone sample is a modular web-research agent, and it is judged on the quality of its report, not on whether tests pass. Before we read any code I ask the class to write down three criteria for a good research report, and we grade the output against them at the end. A report can be beautifully formatted and still be about the wrong thing. That is a better lesson about evaluating agents than any metric I could hand them.
Why I teach this
I build ML systems at OctaPulse. Teaching is a different discipline. You cannot hand-wave a step when someone is about to run it, and you find out very quickly which parts you actually understand.
The thread through all seven weeks is simple. A bare model guesses text, so it needs tools. Tools need a loop. The loop needs memory, search and coordination. And once it can act, it needs guardrails and someone watching. The goal is for a room of engineers to go from “what makes this an agent and not a chatbot?” to shipping one.