Phase 14: Agent Engineering

Phase 14: Agent Engineering

> The core of modern AI engineering. Build agents from first principles.

Start this phase on GitHub

Prerequisites: Phase 11 LLM Engineering and Phase 13 Tools and Protocols.

First lesson: The Agent Loop

Two focused routes are available when you do not need the full phase:

• Agent-Assisted Engineering covers task framing, repository evidence, harnesses, isolation, verification, review, and durable feedback.
• Product Judgment and Delivery covers outcomes, workflow discovery, assumptions, slices, specifications, metrics, staged release, and feedback ownership.

Run this command from the repository root:

python3 phases/14-agent-engineering/01-the-agent-loop/code/main.py

Keep the command, exit code, thought-action-observation trace, final answer,
turn count, and the exact stop condition that ended the run.

Next action: Change the scripted task or turn budget, predict the effect,
then continue to ReWOO and Plan-and-Execute.

Browse the full Phase 14 lesson list or the
cross-phase roadmap.

Lessons in this phase

• The Agent Loop: Observe, Think, Act
• ReWOO and Plan-and-Execute: Decoupled Planning
• Reflexion: Verbal Reinforcement Learning
• Tree of Thoughts and LATS: Deliberate Search
• Self-Refine and CRITIC: Iterative Output Improvement
• Tool Use and Function Calling
• Agent Memory — Virtual Context and Memory Paging
• Memory Blocks and Sleep-Time Compute
• Hybrid Memory: Vector + Graph + KV
• Skill Libraries and Lifelong Learning (Voyager)
• Planning with HTN and Evolutionary Search
• Anthropic’s Workflow Patterns: Simple Over Complex
• Stateful Graph Orchestration — Durable Execution and Checkpoints
• The Actor Model for Agents — Async Messages and Typed Runtimes
• Role-Based Agent Teams — Roles, Tasks, Processes
• OpenAI Agents SDK: Handoffs, Guardrails, Tracing
• The Harness as a Library — Subagents and Session Store
• Production Agent Runtimes — Fast Instantiation and Typed Workflows
• Benchmarks: SWE-bench, GAIA, AgentBench
• Benchmarks: WebArena and OSWorld
• Computer Use: Claude, OpenAI CUA, Gemini
• Voice Agents: Pipecat and LiveKit
• OpenTelemetry GenAI Semantic Conventions
• Agent Observability: Langfuse, Phoenix, Opik
• Multi-Agent Debate and Collaboration
• Failure Modes: Why Agents Break
• Prompt Injection and the PVE Defense
• Orchestration Patterns: Supervisor, Swarm, Hierarchical
• Production Runtimes: Queue, Event, Cron
• Eval-Driven Agent Development
• Agent Workbench Engineering: Why Capable Models Still Fail
• The Minimal Agent Workbench
• Agent Instructions as Executable Constraints
• Repo Memory and Durable State
• Initialization Scripts for Agents
• Scope Contracts and Task Boundaries
• Runtime Feedback Loops
• Verification Gates
• Reviewer Agent: Separate Builder from Marker
• Multi-Session Handoff
• The Workbench on a Real Repo
• Capstone: Ship a Reusable Agent Workbench Pack
• Frame the Task Before the Agent Writes Code
• Build an Evidence-Backed Execution Plan
• Delegate Agent Work with Isolation and Merge Contracts
• Turn Every Agent Correction into a System Improvement
• Define the Outcome Before You Choose the Output
• Discover the Workflow People Actually Perform
• Map Assumptions and Resolve the Riskiest One First
• Choose the Smallest Slice That Can Change the Decision
• Write Specifications That Preserve Judgment
• Design Success Metrics Before the Result Exists
• Choose Prototype, Pilot, or Production Deliberately
• Build a Feedback Ratchet with Ownership and Retirement