Video by freeCodeCamp.org · streamed from YouTube. All rights remain with the creator.
Lesson 3: Reinforcement Learning Built From Nothing in C
About this lesson
Reinforcement learning is usually taught with ready-made libraries that hide how it works. This lesson removes every layer of help and builds a complete framework in standard C, with no outside libraries. The work falls into four parts. It begins with automatic differentiation and memory: you write the system that records calculations as a graph while they run, then walks back through that graph to work out gradients, along with the memory allocation and matrix structures that the forward and backward passes rely on. The second part adds the linear algebra and the core functions of a neural network, including the ReLU activation and a version of softmax written to stay numerically stable when its inputs are very large. The third part builds the environment, a Snake game coded in C, and covers how the state of the game is encoded as numbers for the network, how collisions are detected and how rewards are assigned. The final part brings everything together in a policy gradient training pipeline, with rollouts in which the agent plays, buffers that store its experience, estimation of the advantage of each action, and the updates that change the network's parameters. By the end the program learns to play the game through trial and error. You need to be comfortable reading C, particularly pointers and manual memory management, and to know the basics of neural networks. Afterwards, read Chapter 3 of the course book, which works a discounted return and a backward pass by hand and explains why each part of the framework is needed.