Video by freeCodeCamp.org · taught by Sakshay Mahna · streamed from YouTube. All rights remain with the creator.
Lesson 3: Perception for Self-Driving Cars
About this lesson
This course by Sakshay Mahna covers the computer vision tasks that the perception unit of a self-driving car must perform, and is aimed at learners who already know Python and the basics of deep learning. It is organised as a series of projects, each built in a notebook on real driving data, so that you see how popular deep learning techniques are applied to the problems a car faces on the road. The first project performs road segmentation with a fully convolutional network, labelling which pixels of a camera image belong to the road. The second uses YOLO for two-dimensional object detection, finding vehicles and other objects in images and drawing boxes around them. The third adds object tracking with Deep SORT, following each detected object from frame to frame, and explains the ideas it relies on, such as the Kalman filter and the matching of detections to tracks. The fourth project visualises three-dimensional data from the KITTI dataset, including laser scanner point clouds, and explains the homogeneous transformations that convert coordinates between sensors. The fifth introduces multi-task learning with a Multi-Task Attention Network, which predicts several outputs such as segmentation and depth from one image. The sixth detects objects in three dimensions with SFA3D, and the final project uses UNetXST to transform camera images into a bird's-eye view of the scene. Afterwards, read Chapter 3 of the course book, which works a Kalman filter by hand.