Chrysolites Tech Academy

Deep Learning with TensorFlow and Keras · Course book

Chapter 1: TensorFlow from Tensors to Q-Learning

Pairs with Lesson 1: TensorFlow 2 Complete Course

TensorFlow is Google's open-source library for machine learning, and version 2 made it far friendlier by building in Keras, a simple way to describe networks layer by layer, and by running code immediately rather than in a separate compiled graph. This chapter is a map of the territory a complete TensorFlow course covers. It begins with how artificial intelligence, machine learning and neural networks relate, and with tensors, the data structure the library is named after. It then works through the core learning algorithms, including a hidden Markov model you can compute by hand, before building neural networks for clothing images, convolutional networks for photos, recurrent networks for text and, finally, an agent that learns by trial and error with Q-learning.

Machine learning and tensors

Artificial intelligence is the broadest idea: any technique that makes a computer behave in a way we would call intelligent, including hand-written rules, such as a chess program that follows programmed strategies. Machine learning is the subset in which the rules are learned from data: you supply inputs and the expected outputs, and the algorithm works out the rules. Neural networks, and the deep learning built from them, are one family of machine learning methods that pass data through layers of transformations. Every model in this chapter belongs to machine learning, and the later ones are neural networks.

A tensor is a generalisation of numbers, vectors and matrices to any number of dimensions. Its rank, sometimes called its degree, is the number of dimensions, and its shape lists the size of each dimension.

ExampleRankShapeNumber of values
A single number, such as 70[]1
A list of 3 numbers1[3]3
A 2 × 3 table2[2, 3]6
A 28 × 28 greyscale image2[28, 28]784
A batch of 32 colour images, 32 × 32 pixels4[32, 32, 32, 3]98,304

Reshaping rearranges the same values into a new shape, so the number of values must stay the same. A tensor of shape [2, 3, 4] holds 24 values and can be reshaped to [6, 4] or [24], but not to [5, 5]. Passing −1 for one dimension asks TensorFlow to work it out: reshaping [2, 3, 4] to [3, −1] gives [3, 8]. Most bugs in early TensorFlow projects are shape mismatches, so printing shapes is a habit worth forming.

Core learning algorithms

Before neural networks, the course covers four classic approaches, each suited to a different kind of question. Linear regression predicts a number from a weighted sum of features; a classic exercise predicts whether passengers survived a historic shipwreck from their age, class and fare, using the probability output of a linear model. Categorical columns, such as the class of ticket, are converted to numbers first. Classification predicts which of several classes an example belongs to, such as the species of a flower from the size of its petals, with the model outputting a probability for each class. Clustering groups unlabelled data, most commonly with k-means, which repeatedly assigns points to their nearest centre and moves each centre to the average of its points.

The fourth, the hidden Markov model, is the least familiar and the easiest to compute by hand. It describes a system that moves between hidden states with fixed transition probabilities, where each state produces observations according to its own distribution. We never see the state directly, only the observations.

Worked example. Weather has two hidden states, cold and hot. On day 1 the chance of a cold day is 0.8 and of a hot day 0.2. A cold day is followed by another cold day with probability 0.7 and by a hot day with 0.3; a hot day is followed by a cold day with 0.2 and by another hot day with 0.8. Cold days average 0°C and hot days 15°C. On day 1 the expected temperature is 0.8 × 0 + 0.2 × 15 = 3°C. On day 2 the chance of cold is 0.8 × 0.7 + 0.2 × 0.2 = 0.6, so hot is 0.4 and the expected temperature is 6°C. Day 3 gives 0.5 and 0.5, so 7.5°C, and day 4 gives 0.45 and 0.55, so 8.25°C. Over time the chance of cold settles at 0.2 / (0.3 + 0.2) = 0.4, so the long-run expected temperature is 0.6 × 15 = 9°C.
long run: 9°C 12345678 0510 dayexpected temperature (°C)
Figure 1.1 — Expected temperature in the two-state weather model. Whatever the starting probabilities, the forecast settles towards the long-run value.

TensorFlow's companion probability library can build exactly this model from the starting, transition and observation distributions and compute the expected values for any number of days, giving the same numbers as the hand calculation.

Neural networks with Keras

A neural network learns by repeating forward passes, which make predictions, and backward passes, which use backpropagation to compute how each weight affects the loss, followed by a gradient descent step that adjusts every weight a little. Keras hides the calculus and lets you describe a model as a list of layers.

model = keras.Sequential([
    keras.layers.Flatten(input_shape=(28, 28)),
    keras.layers.Dense(128, activation="relu"),
    keras.layers.Dense(10, activation="softmax"),
])
model.compile(optimizer="adam",
              loss="sparse_categorical_crossentropy",
              metrics=["accuracy"])
model.fit(train_images, train_labels, epochs=10)

This network classifies the Fashion MNIST dataset: 70,000 greyscale images of clothing, 60,000 for training and 10,000 for testing, each 28 × 28 pixels and labelled with one of 10 classes such as trouser, sneaker and bag. The Flatten layer turns each image into 784 inputs. The hidden layer has 784 × 128 weights plus 128 biases, which is 100,480 parameters, and the output layer has 128 × 10 + 10 = 1,290, for 101,770 in total. Pixel values from 0 to 255 are divided by 255 first so that they lie between 0 and 1, which helps training. After fitting, model.evaluate scores the test images and model.predict returns ten probabilities per image, the largest of which is the prediction.

Convolutional networks for computer vision

Flattening an image throws away the fact that neighbouring pixels belong together. A convolutional neural network keeps the image's shape and slides small learned filters across it, each looking for one pattern, such as an edge or a patch of colour, wherever it occurs. A pooling layer then shrinks the result, usually keeping the largest value in each 2 × 2 block, so that later layers see larger regions at lower cost.

A common exercise uses the CIFAR-10 dataset of 60,000 colour images, 32 × 32 pixels, in 10 classes such as aeroplane, cat and lorry, split into 50,000 for training and 10,000 for testing. A first convolutional layer with 32 filters of size 3 × 3 over 3 colour channels has 3 × 3 × 3 × 32 = 864 weights plus 32 biases, 896 parameters, and turns each 32 × 32 image into 32 maps of 30 × 30, because a 3 × 3 filter fits 30 positions across. Pooling halves that to 15 × 15. Stacks of these layers end in dense layers that make the final decision.

When data is scarce, two techniques help. Data augmentation creates new training images by flipping, rotating, shifting and zooming existing ones. Transfer learning starts from a large network already trained on millions of images, freezes its convolutional base, which already knows general visual features, and trains only a new small classifier on top, often reaching high accuracy with a few thousand images.

Language with RNNs and learning by reward

Natural language processing needs text turned into numbers. Each word in the vocabulary gets an integer, reviews are padded or cut to a fixed length, and an embedding layer learns a vector for each word so that words used in similar ways end up close together. A recurrent neural network then reads the sequence one word at a time, carrying a hidden state forward, and an LSTM layer adds gates that help it remember across long sentences. Two classic projects follow: sentiment analysis, deciding whether a film review is positive or negative, and character generation, where a network trained on a play predicts the next character over and over to write new text in a similar style.

Reinforcement learning drops the dataset altogether. An agent acts in an environment, moves between states and receives rewards. Q-learning keeps a table of Q values, one for each state and action, estimating the total future reward of taking that action in that state. After each move it updates the value with Q(s, a) ← Q(s, a) + α [r + γ max Q(s′, a′) − Q(s, a)], where α is the learning rate and γ the discount on future rewards. For example, with Q(s, a) = 2, a reward of 1, a best next value of 4, α = 0.5 and γ = 0.9, the target is 1 + 0.9 × 4 = 4.6 and the new value is 2 + 0.5 × (4.6 − 2) = 3.3. To keep exploring, the agent picks a random action with a probability ε that shrinks over time, a balance between exploration and exploitation.

Check your understanding

  1. Give the rank, shape and number of values of a batch of 64 greyscale images of 28 × 28 pixels with a channel dimension of 1.
  2. Can a tensor of shape [4, 6] be reshaped to [3, 8]? To [5, 5]? What does [2, −1] give?
  3. Continue the weather example to day 5. What are the chances of cold and hot, and what is the expected temperature?
  4. Count the parameters in a Keras model with a 784-value input, a dense layer of 64 neurons and an output layer of 10.
  5. A convolutional layer has 64 filters of size 3 × 3 and receives 32 input maps. How many parameters does it have?
  6. Apply one Q-learning update with Q(s, a) = 5, reward 0, best next value 6, α = 0.1 and γ = 0.95.

Install Chrysolites Tech Academy

Android / Chrome / Edge: open the browser menu and choose Install app or Add to Home screen.

iPhone / iPad (Safari): tap the Share button, then Add to Home Screen.