Let's build GPT: from scratch, in code, spelled out

Andrej Karpathy builds a GPT language model from scratch, explaining every concept along the way.

Up Next in AI Programming

Intro to Large Language Models

Let's build the GPT Tokenizer

How I Use AI Coding Agents (as a Staff Engineer)

Stanford CS229 — Machine Learning (Andrew Ng)

Attention is All You Need — Paper Explained

Transformers, the tech behind LLMs | Deep Learning Chapter 5

Attention in transformers, step-by-step | Deep Learning Chapter 6

Let's build the GPT Tokenizer

LangChain Crash Course for Beginners

LLMs from Scratch – Practical Engineering from Base Model to PPO RLHF

Build an LLM from Scratch 7: Instruction Finetuning

Code Your Own Llama 4 LLM from Scratch – Full Course

Intro to Machine Learning featuring Generative AI

Train Your Own LLM – Tutorial