Applied Deep Learning · Course book
Chapter 1: Deep Learning from the Top Down
Most technical subjects are taught from the bottom up: theory first, applications much later. The fast.ai approach turns this around. You train a working, state-of-the-art model in the first lesson, and only then peel back the layers to see how it works, the way people learn a sport by playing a game before studying its finer points. This chapter follows that top-down path. It starts with training an image classifier by fine-tuning a pre-trained network, then looks at how to weigh evidence, how to take a model into production, and how stochastic gradient descent works when written from scratch. It ends with the ethical questions every practitioner must face and with collaborative filtering, the technique behind recommendation systems.
Your first models
A first image classifier can be trained in a few lines with the fastai library, which is built on PyTorch. You point it at a folder of labelled photos, for example pictures of cats and dogs, choose a pre-trained architecture such as a ResNet that has already learned general visual features from more than a million images, and call fine_tune. In minutes on a GPU, the model typically identifies cats and dogs with an error rate of around one percent.
The secret is transfer learning. The early layers of a pre-trained network already detect edges, textures and shapes, so only the final layers need to learn the new task. Fine-tuning first trains just a new output layer for an epoch while the rest is frozen, then unfreezes everything and trains all layers a little more, with the early layers changing least. The same pattern applies far beyond photos: sounds can be turned into images of their frequencies, and mouse movements into pictures of their paths, and then classified in the same way.
Several terms recur from the first lesson onwards. The architecture is the template of the model; the parameters or weights are the numbers it learns; the loss measures how wrong it is for training purposes; a metric, such as error rate, measures quality for people. The validation set, often 20 percent of the data chosen at random, is never used for training and shows whether the model has learned general patterns or memorised the training images.
Judging evidence
Data science is as much about judging evidence as building models, and a common tool is the p-value. It answers one specific question: if there were really no effect at all, how likely is a result at least as extreme as the one observed? A small p-value says the data would be surprising if there were no effect. It does not say how large or important the effect is, and it is not the probability that a hypothesis is true.
The wider lesson is to treat p-values as one piece of evidence among many. A tiny effect measured on huge amounts of data can have a very small p-value yet matter little in practice, while an important effect in a small study may not reach an arbitrary threshold such as 0.05. Practitioners should ask how big an effect is, how it was measured, and whether the result holds up in other data, rather than relying on a single cut-off.
From model to production
A model is only useful once people can use it. The production lesson builds a classifier for three kinds of bear from images collected with a web image search, and walks through the steps from raw data to a working application. fastai's DataBlock API describes how to build the dataset: what the inputs and labels are, how to find the files, how to split training from validation data, how to get each label, and which transforms to apply. Resizing choices matter: squishing distorts the shape, padding adds empty borders, and random resized cropping takes a different part of each image every epoch, which doubles as data augmentation.
A neat trick is to use the model to clean the data. After a first training run, the images with the highest loss are those the model found most confusing, and they are often mislabelled or not bears at all. Reviewing and fixing those first, then retraining, improves results with little effort. The trained model is then exported to a file containing both the weights and the data processing steps, and loaded in an application that accepts an uploaded photo and returns a prediction. Real deployments must also plan for data that drifts away from what the model was trained on, rolling out gradually with people checking results before trusting the model fully.
Stochastic gradient descent from scratch
To see what fine_tune hides, the course builds a classifier for handwritten 3s and 7s from first principles. A simple baseline averages all the training 3s into an "ideal 3" and all the 7s into an "ideal 7", then labels a new image by whichever average it is closer to, measured by the mean absolute difference or the root mean squared difference between pixels. This baseline is surprisingly accurate, and having a baseline at all is a good habit: it shows whether a complex model is actually adding anything.
The learned approach gives each of the 784 pixels a weight, computes a weighted sum plus a bias, and passes it through a sigmoid to get a probability. Learning uses stochastic gradient descent in seven steps: initialise the weights randomly, predict, compute the loss, compute the gradients, step the weights against the gradients scaled by the learning rate, repeat, and stop when the model is good enough. Accuracy cannot be the loss because it changes in jumps and has no useful gradient, so a smooth loss based on how far each predicted probability is from the correct answer is used instead. Working on mini-batches of a few hundred images makes each step cheap while keeping the gradient estimate reasonable. Adding a hidden layer with a ReLU between two linear layers turns this into a neural network, and the result rivals the fastai model on this simple task.
Choosing the learning rate is the most important practical decision in this loop. fastai provides a learning rate finder, which trains briefly while increasing the rate step by step and plots the loss against it. The loss first falls, then flattens and finally shoots upwards as the rate becomes too large. A good choice is a rate somewhat below the point where the loss falls most steeply, well before it starts to climb.
Data ethics and collaborative filtering
Models affect people, so practitioners share responsibility for their effects. The ethics lesson discusses recurring problems. Feedback loops arise when a model's outputs shape the data it later learns from, as when a recommendation system keeps promoting what it already promoted. Bias can enter through unrepresentative data or through the choice of what to measure. Recourse and accountability matter because people harmed by an automated decision need a way to appeal it. Practical steps include asking who could be affected, testing performance across different groups and including diverse perspectives on a team.
The final lesson covers multi-label image classification and collaborative filtering, used to recommend films, products or songs. Each user and each item is given a short vector of learned numbers called latent factors. The predicted rating is the dot product of the two vectors plus a bias for the user and a bias for the item, often squeezed into the rating range with a scaled sigmoid. Gradient descent learns the factors from known ratings, so that users with similar tastes end up with similar vectors.
| Quantity | Factor 1 | Factor 2 | Factor 3 | Bias |
|---|---|---|---|---|
| User | 0.9 | −0.6 | 0.2 | 0.3 |
| Film | 0.8 | 0.1 | 0.5 | 0.2 |
| Products | 0.72 | −0.06 | 0.10 | – |
The dot product is 0.72 − 0.06 + 0.10 = 0.76. Adding both biases gives 1.26, and a sigmoid scaled to the range 0 to 5.5 gives 5.5 / (1 + e−1.26) = 4.28, a predicted rating of about 4.3 stars. With 1,000 users, 1,700 films and 50 factors each, the model learns (1,000 + 1,700) × 50 = 135,000 factors plus 2,700 biases.
Check your understanding
- Explain the two stages of
fine_tuneand why the early layers change least. - Compute the probability of 10 heads in 10 flips of a fair coin, and of 8 or more heads.
- Why might a model's highest-loss training images be worth reviewing by hand?
- List the seven steps of stochastic gradient descent in order.
- Give an example of a feedback loop in a recommendation system and one way to reduce it.
- A user's factors are (0.5, 0.5, −0.2) and a film's are (0.4, −0.2, 0.6), with no biases. Compute the dot product.
This course book is available to read online only at tech-academy.chrysolites.org.