Course
From Atomic Ops to GPT
A first-principles walk from scalar operations to GPT. The order is dependency order — values build graphs, graphs build gradients, gradients build learning, learning builds representations, representations build attention. Read the chapters in sequence, or deep-link to any lesson if you already know the territory.
Chapters
Chapter 1
Why Atomic Ops Matter
2 lessons
Chapter 2
Computational Graphs and Backprop
5 lessons
Chapter 3
Optimization and Learning
Coming soon
Chapter 4
Vectors, Matrices, and Embeddings
Coming soon
Chapter 5
Projections, Attention, and the Transformer Block
Coming soon
Chapter 6
Language Modeling, Training Loop, and Inference Loop
Coming soon
Chapter 7
What Scales from Tiny GPT to Modern LLMs
Coming soon