Phase 9: Reinforcement Learning

Phase 9: Reinforcement Learning

> Agents that learn by doing. The foundation of RLHF.

Start this phase on GitHub

Prerequisites: Phase 1 probability and distributions, plus Phase 2 Lesson
01 for the ML taxonomy.

First lesson: MDPs, States, Actions and Rewards

Run this command from the repository root:

python3 phases/09-reinforcement-learning/01-mdps-states-actions-rewards/code/main.py

Keep the command, exit code, random and greedy returns, value grids, and one
sentence connecting policy quality to expected return.

Next action: Change the discount factor, predict the value shift, then
continue to Dynamic Programming.

Browse the full Phase 9 lesson list or the
cross-phase roadmap.

Lessons in this phase

• MDPs, States, Actions & Rewards
• Dynamic Programming — Policy Iteration & Value Iteration
• Monte Carlo Methods — Learning from Complete Episodes
• Temporal Difference — Q-Learning & SARSA
• Deep Q-Networks (DQN)
• Policy Gradient — REINFORCE from Scratch
• Actor-Critic — A2C and A3C
• Proximal Policy Optimization (PPO)
• Reward Modeling & RLHF
• Multi-Agent RL
• Sim-to-Real Transfer
• RL for Games — AlphaZero, MuZero, and the LLM-Reasoning Era