← Goal templates
Neural networks from scratch
From a single neuron to backpropagation you could code yourself.
Learn how neural networks really work: neurons, layers, gradient descent and backpropagation, with 3Blue1Brown's visual series, Michael Nielsen's free book and Andrej Karpathy building a tiny autograd engine line by line.
beginnerabout 2 weeks at 45 min a day6 free sources
What you'll cover
Phase 1
What a network is
- Neurons, weights and biases
- Layers and activations
- Recognising handwritten digits
Phase 2
How it learns
- Cost functions
- Gradient descent
- Backpropagation, intuitively
- Backpropagation, the calculus
Phase 3
Build one
- Derivatives as code
- A tiny autograd engine
- Training a small network
An outline. Your own plan is made when you start, around your level and deadline.
Sources
Free, and fetched for you
- LectureBut what is a neural network? (3Blue1Brown)youtube.com ↗
- LectureGradient descent, how neural networks learn (3Blue1Brown)youtube.com ↗
- LectureBackpropagation, intuitively (3Blue1Brown)youtube.com ↗
- LectureBackpropagation calculus (3Blue1Brown)youtube.com ↗
- Web pageUsing neural nets to recognize handwritten digits (Michael Nielsen)neuralnetworksanddeeplearning.com ↗
- LectureBuilding micrograd: neural networks and backpropagation, spelled out (Andrej Karpathy)youtube.com ↗
All sources belong to their creators and are free to watch or read; Confu links to them and turns them into study PDFs for you.
More templates
- Transformers and large language modelsHow ChatGPT-style models work, down to attention and a GPT you build yourself.
- Reinforcement learning, from basics to PPOAgents, rewards and policies, up to the algorithm behind RLHF.
- Linear algebra, the visual wayVectors, matrices and eigenvectors you can actually picture.
- The essence of calculusDerivatives and integrals as ideas, not just rules.