Video by freeCodeCamp.org · taught by Aakash N S · streamed from YouTube. All rights remain with the creator.
Lesson 4: Training Deep Neural Networks on GPUs
About this lesson
This third session of the Deep Learning with PyTorch live course, taught by Aakash N S, takes the step from linear models to real neural networks and shows how to train them quickly on graphics processing units. It begins by explaining how to use free cloud platforms that provide GPUs, such as Kaggle and Google Colab, so that anyone can run the code without owning special hardware, and how to check from PyTorch whether a GPU is available. The session then builds a multilayer feedforward neural network for the handwritten digits dataset, defining it as a class that extends PyTorch's module base class, with a hidden layer between the inputs and the outputs. It explains the role of activation functions such as the rectified linear unit, why non-linearity is what allows a network to learn more than a linear model can, and what the universal approximation theorem says about the power of networks with hidden layers. The second half deals with performance. You learn how to write code that uses a GPU when one is present and falls back to the processor otherwise, how to move a model's parameters and each batch of data onto the GPU, and how a small wrapper around the data loader does this automatically. The network is then trained on the GPU, evaluated on validation data and compared with the logistic regression model from the previous session. Each step is shown running live in a notebook. Afterwards, read Chapter 4 of the course book, which solves an exclusive-or problem by hand.