Phase 7: Transformers Deep Dive

Phase 7: Transformers Deep Dive

> The architecture that changed everything. Understand every layer.

Start this phase on GitHub

Prerequisites: Phase 3 Deep Learning Core, Phase 5 Lesson 09 on
sequence-to-sequence models, and Phase 5 Lesson 10 on attention.

First lesson: Why Transformers

Run this command from the repository root:

python3 phases/07-transformers-deep-dive/01-why-transformers/code/main.py

Keep the command, exit code, serial and parallel depth table, equivalence
check, and one sentence describing the speed-versus-memory tradeoff.

Next action: Explain why parallel depth changes the hardware story, then
continue to Self-Attention from Scratch.

Browse the full Phase 7 lesson list or the
cross-phase roadmap.

Lessons in this phase

• Why Transformers — The Problems with RNNs
• Self-Attention from Scratch
• Multi-Head Attention
• Positional Encoding — Sinusoidal, RoPE, ALiBi
• The Full Transformer — Encoder + Decoder
• BERT — Masked Language Modeling
• GPT — Causal Language Modeling
• T5, BART — Encoder-Decoder Models
• Vision Transformers (ViT)
• Audio Transformers — Whisper Architecture
• Mixture of Experts (MoE)
• KV Cache, Flash Attention & Inference Optimization
• Scaling Laws
• Build a Transformer from Scratch — The Capstone
• Attention Variants — Sliding Window, Sparse, Differential
• Speculative Decoding — Draft, Verify, Repeat