Deep Learning with TensorFlow and Keras · Course book
Chapter 3: Keras in Practice: From Data to Fine-Tuning
Keras is the part of TensorFlow most people actually type: a high-level interface that turns a neural network into a short list of layers and a handful of method calls. This chapter covers the full path a Keras practitioner follows. It starts with processing raw data into the arrays a network expects, then builds, trains and validates a simple network, uses it for inference and saves it in several ways. The second half moves to images, building a convolutional network from scratch and then reaching much higher accuracy by fine-tuning large networks that were already trained on millions of photos, first VGG16 and then the lightweight MobileNet. It finishes with data augmentation, reproducible results and how to count a model's learnable parameters by hand.
Processing data and building a first network
A Keras model trained with fit expects its inputs as a NumPy array or a TensorFlow tensor, with one row per sample, and a matching array of labels. Raw data rarely arrives that way. A simple tabular example might store each sample as a single number, such as a measurement between 13 and 100, with a label of 0 or 1. Processing it involves three steps: collecting the samples and labels into arrays, shuffling them together so that any order in the original collection is removed, and scaling the inputs into the range 0 to 1 with a min-max scaler. A value of 56 in the range 13 to 100 becomes (56 − 13) / 87 = 0.494.
model = Sequential([
Dense(16, input_shape=(1,), activation="relu"),
Dense(32, activation="relu"),
Dense(2, activation="softmax"),
])
model.compile(optimizer=Adam(learning_rate=0.0001),
loss="sparse_categorical_crossentropy",
metrics=["accuracy"])
model.fit(x=scaled_train, y=train_labels, validation_split=0.1,
batch_size=10, epochs=30, shuffle=True, verbose=2)
Several arguments deserve attention. The first layer declares the input shape, here a single number per sample. The last layer has one neuron per class with softmax, so its two outputs are probabilities. batch_size=10 means the weights update after every ten samples, and epochs=30 means thirty passes over the data. shuffle=True reshuffles the training data before each epoch.
The validation set is the honest witness during training. With validation_split=0.1, Keras sets aside 10 percent of the training data and reports the loss and accuracy on it after every epoch, without ever training on it. One detail catches many people out: Keras takes the last 10 percent of the arrays as they are, before any shuffling it does itself, so if the data is sorted, for example all of one class at the end, the validation set will be lopsided. Shuffling the arrays yourself first avoids the problem. You can also pass a separate validation set explicitly with validation_data.
Inference, confusion matrices and saving
Inference means using a trained model on data it has never seen. The test set is processed in exactly the same way as the training set, with the same scaler, and passed to model.predict, which returns one row of class probabilities per sample. Taking the index of the largest value in each row gives the predicted class.
A confusion matrix summarises the results by counting each combination of true and predicted class. Suppose a cats-and-dogs classifier is tested on 50 cats and 50 dogs.
| True class | Predicted cat | Predicted dog | Correct for this class |
|---|---|---|---|
| Cat (50) | 44 | 6 | 44 / 50 = 88% |
| Dog (50) | 9 | 41 | 41 / 50 = 82% |
| Overall | 53 predicted | 47 predicted | 85 / 100 = 85% |
The overall accuracy of 85 percent hides a useful detail: the model mistakes dogs for cats more often than the reverse, which suggests looking at the dog images it gets wrong.
Keras offers three ways to save a model, depending on what you need to keep.
model.savestores everything: the architecture, the learned weights, the training configuration and the optimiser's state. Loading it lets you predict immediately or carry on training exactly where you stopped.model.to_jsonstores only the architecture as text. Rebuilding from it gives a fresh model with random weights, useful for sharing a design.model.save_weightsstores only the weights. They can be loaded into a model built with the same architecture.
Convolutional networks for images
Image data is usually stored as folders of files, one folder per class. Keras can read such folders in batches, resize every image to a common size such as 224 × 224 pixels, apply the preprocessing a particular network expects, and attach labels taken from the folder names. Splitting the images into separate training, validation and test folders before starting keeps the three sets cleanly apart.
A small convolutional network built from scratch alternates convolutional layers, which slide small learned filters across the image to detect patterns, with max-pooling layers, which shrink each feature map by keeping the largest value in each 2 × 2 block. A Flatten layer then turns the final maps into a vector for a dense output layer. Trained on a few thousand photos, such a network often learns the training images very well while scoring much worse on validation images, a clear case of overfitting: there are simply not enough examples to learn general visual features from nothing.
Fine-tuning VGG16 and MobileNet
Transfer learning starts from a network that has already learned general visual features from a huge dataset, the ImageNet collection of more than a million images in 1,000 classes. Fine-tuning adapts it to a new task. VGG16, a sixteen-layer network with about 138 million parameters, is a classic starting point. The recipe is to copy every layer except the final 1,000-class output, freeze the copied layers so their weights do not change, and add a new output layer with one neuron per new class.
The numbers show why this is so efficient. VGG16's last hidden layer has 4,096 neurons. Its original output layer has 4,096 × 1,000 + 1,000 = 4,097,000 parameters. A new cats-and-dogs output needs only 4,096 × 2 + 2 = 8,194 trainable parameters, while all the rest stay frozen. Because the frozen layers already recognise edges, textures and object parts, the fine-tuned network can reach far higher validation accuracy than the small network trained from scratch, after only a few epochs.
VGG16 is large and slow. MobileNet was designed for phones and browsers: it uses a cheaper kind of convolution and has only about 4.2 million parameters, roughly a thirtieth of VGG16, at a modest cost in accuracy. Fine-tuning MobileNet follows the same idea, but instead of freezing everything, you might keep the last few dozen layers trainable so they can adapt more to the new images. MobileNet works well even on a small subset of data, such as a few hundred images of hand signs for the digits 0 to 9. Its pooled output has 1,024 features, so a new ten-class output layer has 1,024 × 10 + 10 = 10,250 parameters.
Augmentation, reproducibility and learnable parameters
Data augmentation creates new training images by applying random changes to existing ones: small rotations, shifts left and right or up and down, zooms, changes of brightness and horizontal flips. A cat flipped left to right is still a cat, so the network sees more variety and overfits less. Some changes do not suit every task: flipping is harmless for animals but would turn some hand-written digits or letters into different symbols, so augmentations must be chosen with the data in mind.
Training involves randomness everywhere: the starting weights, the shuffling of data and dropout. To get the same results every run, which matters when comparing two changes fairly, you set the random seeds of Python, NumPy and TensorFlow at the start of the program. Some operations on graphics cards remain slightly non-deterministic, so exact reproducibility can also require running on the processor or enabling deterministic operations.
Learnable parameters are the weights and biases a model adjusts during training, and counting them by hand is a good test of understanding. For a dense layer, the count is inputs × outputs + outputs. For a convolutional layer, it is filter height × filter width × input channels × number of filters + number of filters, because each filter has one bias. Pooling and flatten layers have none.
model.summary() prints the same counts, so you can check your arithmetic.Practice tasks
- Scale the values 13, 40 and 100 to the range 0 to 1, using 13 and 100 as the minimum and maximum.
- Your data has 2,000 samples sorted by class, with all the class 1 samples last. What does
validation_split=0.1give you, and how do you fix it? - From the confusion matrix in this chapter, how many images were predicted as cats, and what share of those were really cats?
- Count the trainable parameters when a new five-class output layer replaces VGG16's original output.
- Count the parameters of a convolutional layer with 16 filters of 5 × 5 on a greyscale image.
- Name one augmentation that suits photos of animals and one that would be harmful for photos of written digits, and explain why.
This course book is available to read online only at tech-academy.chrysolites.org.