Skip to content

Course

From Atomic Ops to GPT

A first-principles walk from scalar operations to GPT. The order is dependency order — values build graphs, graphs build gradients, gradients build learning, learning builds representations, representations build attention. Read the chapters in sequence, or deep-link to any lesson if you already know the territory.

Chapters

Chapter 1

Why Atomic Ops Matter

2 lessons

Chapter 2

Computational Graphs and Backprop

5 lessons

Chapter 3

Optimization and Learning

Coming soon

Chapter 4

Vectors, Matrices, and Embeddings

Coming soon

Chapter 5

Projections, Attention, and the Transformer Block

Coming soon

Chapter 6

Language Modeling, Training Loop, and Inference Loop

Coming soon

Chapter 7

What Scales from Tiny GPT to Modern LLMs

Coming soon