Roadmap

Goal. Understand how any neural network learns, and be able to write a training loop in PyTorch from memory.

Study from.

• 3Blue1Brown, “Neural Networks” series (chapters 1–4)
• Andrej Karpathy, “Zero to Hero”: the micrograd video
• Raschka, Build a Large Language Model (From Scratch), Appendix A: Introduction to PyTorch

Notes.

1. What is a neural network? ready
2. Loss functions: measuring how wrong we are to be written
3. Gradient descent and backpropagation to be written
4. Optimizers: SGD, Adam, AdamW and the learning rate to be written
5. Logits, softmax and cross-entropy to be written
6. PyTorch essentials: tensors, autograd, the training loop to be written
7. Overfitting, regularization and data splits to be written

Goal. Explain and code every part of a GPT-style model: text in, next token out.

Build. Project 1: GPT from scratch

Study from.

• Vizuara, “Building LLMs from Scratch” (the full playlist, at 1.5–2x)
• Raschka, Build a Large Language Model (From Scratch), chapters 1–7
• Vaswani et al., “Attention Is All You Need” (2017)
• Jay Alammar, “The Illustrated Transformer” and “The Illustrated GPT-2”
• Karpathy, “Let’s build GPT: from scratch, in code, spelled out”

Notes.

1. What is an LLM? Pretraining, fine-tuning and the big picture to be written
2. Tokenization and byte-pair encoding (BPE) to be written
3. Token embeddings: turning IDs into vectors to be written
4. Positional encoding: telling the model about word order to be written
5. Preparing data: sliding windows and next-token prediction to be written
6. Self-attention, the simple version to be written
7. Scaled dot-product attention: queries, keys and values to be written
8. Causal masking and attention dropout to be written
9. Multi-head attention to be written
10. LayerNorm, GELU, feed-forward layers and residual connections to be written
11. The transformer block and the full GPT architecture to be written
12. Counting parameters and weight tying to be written
13. Pretraining: the training loop, cross-entropy and perplexity to be written
14. Decoding: greedy, temperature, top-k and top-p to be written
15. Loading pretrained GPT-2 weights to be written
16. Fine-tuning for classification to be written
17. Instruction fine-tuning to be written
18. Encoder, decoder, encoder–decoder: BERT vs GPT vs T5 to be written

Goal. Know what changed between GPT-2 and today’s models, and how models are adapted and shrunk cheaply.

Build. Project 2: QLoRA fine-tune, quantize, run locally

Study from.

• Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models” (2021)
• Dettmers et al., “QLoRA” (2023)
• Su et al., “RoFormer: Rotary Position Embedding” (2021)
• Raschka, Build a Large Language Model (From Scratch), Appendix E: LoRA
• Hugging Face PEFT and Transformers documentation

Notes.

1. From GPT-2 to modern LLMs: pre-norm, RMSNorm, SwiGLU, grouped-query attention to be written
2. Rotary position embeddings (RoPE) to be written
3. The KV cache: why generation is fast to be written
4. Quantization: 16-bit, 8-bit, 4-bit and GGUF to be written
5. LoRA, QLoRA and parameter-efficient fine-tuning to be written
6. Teaching preferences: RLHF and DPO to be written
7. Context windows and reasoning models to be written
8. Prompting vs RAG vs fine-tuning: how to choose to be written

Goal. Design, build and measure a RAG system, and explain every trade-off in it.

Build. Project 3: production RAG with evals

Study from.

• Chip Huyen, AI Engineering (2025), the RAG and agents chapter
• Jay Alammar and Maarten Grootendorst, Hands-On Large Language Models
• Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” (2020)
• Malkov and Yashunin, “HNSW” (2016)
• Anthropic, “Introducing Contextual Retrieval” (2024)

Notes.

1. Text embeddings and semantic similarity to be written
2. Vector databases and approximate nearest neighbour search (HNSW, IVF) to be written
3. Chunking strategies to be written
4. Retrieval: BM25, dense, hybrid search and rank fusion to be written
5. Reranking with cross-encoders to be written
6. The RAG pipeline end to end to be written
7. Advanced RAG: query rewriting, HyDE, multi-hop, GraphRAG, agentic RAG to be written
8. Evaluating RAG: recall@k, MRR, nDCG, faithfulness to be written
9. RAG failure modes and how to debug them to be written

Goal. Build reliable LLM features: good context, structured output, tool use and agent loops.

Build. Project 4: agent with your own MCP server

Study from.

• Anthropic, “Building Effective Agents” (2024)
• Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models” (2022)
• Model Context Protocol specification and documentation
• Anthropic and OpenAI prompt engineering guides
• Chip Huyen, AI Engineering (2025), the prompt engineering chapter

Notes.

1. Prompt engineering that survives production to be written
2. Context engineering: what goes in the window to be written
3. Structured output and function (tool) calling to be written
4. The agent loop and ReAct to be written
5. Workflows vs agents: routing, orchestrator–workers, evaluator–optimizer to be written
6. Model Context Protocol (MCP) to be written
7. Agent memory: short-term, long-term, compaction to be written
8. Multi-agent systems: when they help and when they hurt to be written
9. Prompt injection and agent security to be written

Goal. Prove an LLM feature works, then run it cheaply, quickly and safely.

Build. Project 4 continued: evals, guardrails, cost and latency report

Study from.

• Chip Huyen, AI Engineering (2025), the evaluation and inference optimization chapters
• Hamel Husain, “Your AI Product Needs Evals”
• vLLM documentation (continuous batching, PagedAttention)

Notes.

1. Evals: golden sets, metrics and regression testing to be written
2. LLM-as-judge: how to trust a model grading a model to be written
3. Hallucination: causes, detection, mitigation to be written
4. Guardrails and safety layers to be written
5. Cost and latency: caching, batching, streaming, model routing to be written
6. Serving LLMs: time to first token, throughput, continuous batching to be written
7. Observability and tracing to be written

Goal. Turn everything into spoken, structured answers under time pressure.

Build. Polish all four projects: READMEs, demos, resume bullets

Study from.

• Our own notes, phases 1–6
• Alex Xu, System Design Interview (for the classic distributed-systems parts)
• Chip Huyen, AI Engineering (2025), the architecture chapter

Notes.

1. A framework for LLM system design rounds to be written
2. Design: a customer support assistant to be written
3. Design: enterprise document search at scale to be written
4. Design: a coding assistant to be written
5. The ML coding round: attention, BPE and sampling from memory to be written
6. Question bank: fundamentals to be written
7. Question bank: applied AI engineering to be written
8. Talking about your projects to be written