Chrysolites Tech Academy

Video by freeCodeCamp.org · taught by Robert Förster · streamed from YouTube. All rights remain with the creator.

Lesson 4: AlphaZero from Scratch

About this lesson

This course by Robert Förster shows how to build AlphaZero, the algorithm that learns to play board games at a superhuman level purely by playing against itself, from scratch in Python with PyTorch. It begins with an overview of how AlphaZero works, explaining Monte Carlo tree search, the planning method at the heart of the algorithm, and then AlphaMCTS, the version of the search guided by a neural network, before giving a second overview of how self-play and training fit together. The implementation is then built up in ten chapters. The first creates a simple game of Tic Tac Toe to serve as the environment, and the second implements Monte Carlo tree search to play it. The third builds the neural network model, which looks at a board and predicts both which moves are promising and who is likely to win. The fourth combines that model with the search to create AlphaMCTS. The fifth and sixth chapters add self-play, in which the model generates its own training data by playing games against itself, and the training step that learns from those games. The seventh chapter introduces tweaks that make learning faster and more reliable. The eighth extends everything to Connect Four, a much harder game, the ninth speeds up self-play by running many games in parallel, and the tenth evaluates the trained models, including by playing against them. Afterwards, read Chapter 4 of the course book, which works the search scores and the training loss by hand.

Install Chrysolites Tech Academy

Android / Chrome / Edge: open the browser menu and choose Install app or Add to Home screen.

iPhone / iPad (Safari): tap the Share button, then Add to Home Screen.