Chrysolites Tech Academy

Applied Deep Learning · Course book

Chapter 2: Image Processing with OpenCV

Pairs with Lesson 2: Computer Vision with OpenCV

Before deep learning, computer vision was built from a toolkit of image-processing operations: changing colours, blurring, thresholding, finding edges and outlines, and detecting objects with hand-designed features. That toolkit has not gone away. It is still the fastest way to prepare images, to solve simple problems without training anything, and to understand what neural networks are doing. OpenCV is the standard open-source library for it, and this chapter works through its main operations in Python with small calculations you can check by hand. It covers reading and transforming images, colour spaces and channels, blurring and masking, histograms and thresholds, edges and contours, face detection with Haar cascades, and finally how classic processing feeds into a deep learning classifier.

Images as arrays

To OpenCV, an image is a NumPy array. A colour image of 640 × 480 pixels has shape (480, 640, 3): 480 rows, 640 columns and 3 colour values per pixel, each an integer from 0 to 255. One detail catches everyone out: OpenCV stores colours in BGR order, blue first, rather than the RGB order most other libraries use, so images shown with another plotting library look strangely tinted until converted. cv.imread reads an image, cv.imshow displays it and cv.VideoCapture reads a video or a webcam frame by frame in a loop, which ends cleanly when a key is pressed.

Resizing is often the first step, because large frames are slow to process. A helper that rescales by a factor multiplies both dimensions: a 1,920 × 1,080 frame at a scale of 0.75 becomes 1,440 × 810. Shrinking is best done with area-based interpolation and enlarging with linear or cubic interpolation. OpenCV can also draw rectangles, circles, lines and text onto an image, which is how detection results are usually shown.

A few essential functions appear in nearly every project: converting to greyscale, blurring, detecting edges, dilating and eroding those edges, resizing and cropping, which is simply array slicing. Greyscale conversion combines the channels with weights that reflect how bright each colour looks to the eye: grey = 0.299R + 0.587G + 0.114B. A BGR pixel of (200, 100, 50), meaning blue 200, green 100 and red 50, becomes 0.299 × 50 + 0.587 × 100 + 0.114 × 200 = 14.95 + 58.7 + 22.8 = 96.45, stored as 96.

Dilation and erosion work on binary images such as edge maps. Dilation sets a pixel on if any pixel under the kernel is on, which thickens lines and closes small gaps; erosion keeps a pixel on only if every pixel under the kernel is on, which thins lines and removes isolated dots. Applying dilation and then erosion with the same kernel, called closing, fills small holes in shapes while keeping their overall size, and the reverse order, opening, removes small specks. These simple operations often turn a messy edge map into clean shapes that are easy to measure.

Transformations and contours

Geometric transformations move pixels to new places. A translation shifts an image by tx pixels across and ty pixels down using the matrix with rows (1, 0, tx) and (0, 1, ty); the pixel at (10, 20) moves to (10 + tx, 20 + ty). A rotation turns the image by an angle around a chosen point, usually the centre, and flipping mirrors it horizontally, vertically or both. These operations matter for data augmentation as well as for correcting the orientation of photos.

Contours are the curves that join continuous points along the boundary of a shape with the same intensity. They differ from edges: edges are local changes in brightness, while contours are complete outlines that OpenCV returns as lists of points, ready to count, measure or draw. The usual recipe is to convert to greyscale, blur slightly, then either detect edges or threshold the image, and pass the result to cv.findContours. Blurring first matters because noise creates thousands of tiny spurious contours.

Colour, blurring and masking

A colour space is a way of describing colours with numbers. BGR is how cameras and screens work, but it mixes colour and brightness together. HSV separates hue, the colour itself, from saturation and value, which makes it much easier to pick out, say, every red object regardless of lighting. LAB is designed so that equal numerical differences look roughly equally different to people. Each image can be split into its separate channels and merged back; displayed alone, each channel appears as a greyscale image that is bright where that colour is strong.

Blurring smooths noise by replacing each pixel with a combination of its neighbours, using a small square kernel such as 3 × 3. Different methods combine the neighbours differently.

MethodHow the new pixel is computedGood for
AverageMean of all pixels under the kernelSimple, fast smoothing
GaussianWeighted mean, nearer pixels count moreNatural-looking smoothing before edge detection
MedianMiddle value of the pixels under the kernelRemoving isolated specks of noise
BilateralWeights by both distance and similarity of valueSmoothing while keeping edges sharp
Worked example. A 3 × 3 patch has rows (10, 12, 11), (13, 250, 12) and (11, 10, 12), where 250 is a bright speck of noise. The average blur gives 341 / 9 = 37.9 for the centre, smearing the speck into a grey blotch. The median blur sorts the nine values, 10, 10, 11, 11, 12, 12, 12, 13, 250, and takes the middle one, 12, removing the speck entirely. That is why median blurring is the standard cure for "salt and pepper" noise.

Bitwise operations combine images pixel by pixel as if they were true-or-false values: AND keeps only pixels that are on in both, OR keeps pixels on in either, XOR keeps pixels on in exactly one, and NOT inverts. Their main use is masking: draw a white circle on a black image of the same size, AND it with a photo, and only the part of the photo inside the circle survives.

Histograms, thresholds and edges

A histogram counts how many pixels fall into each range of intensity. It shows at a glance whether an image is dark, bright or high in contrast, and it can be computed for a masked region only, or for each colour channel separately. Consider a 4 × 4 greyscale image with rows (12, 40, 200, 220), (30, 60, 210, 240), (25, 90, 180, 250) and (15, 70, 140, 230), counted in four ranges of 64 levels each.

6226 0–6364–127128–191192–255 threshold 128 036 pixel intensity
Figure 2.1 — A two-peaked histogram. A threshold placed in the valley between the peaks separates dark background from bright foreground cleanly.

Thresholding turns a greyscale image into a binary one: every pixel above the threshold becomes 255 and every other pixel becomes 0. With a threshold of 128, the example above becomes 8 white pixels and 8 black ones. A fixed threshold fails when lighting varies across the image, so adaptive thresholding computes a separate threshold for each neighbourhood, typically the local mean minus a small constant.

Edge detection finds places where intensity changes sharply. The Sobel operator computes the gradient in the horizontal and vertical directions separately; the Laplacian responds to changes in the gradient itself; and the Canny detector, the most widely used, combines smoothing, gradients, thinning to one-pixel lines and two thresholds that keep strong edges and only those weak edges connected to strong ones. Canny usually gives the cleanest result.

Faces and deep computer vision

Haar cascades detect objects such as faces using simple rectangular features, differences between the brightness of adjacent rectangles, like the darker band of the eyes above the brighter cheeks. A cascade applies a long series of such tests and rejects a window as soon as one fails, which makes it fast. OpenCV ships pre-trained cascades as XML files; detectMultiScale scans the greyscale image at several sizes, controlled by a scale factor such as 1.1 and a minimum number of overlapping detections, and returns rectangles. Cascades are quick but sensitive to lighting and angle, so they produce some false detections. OpenCV also includes a simple built-in face recogniser that can be trained on labelled face images. Any work with faces should respect privacy and consent: use images people have agreed to share, and be cautious about what such systems are used for.

The capstone moves to deep computer vision: training a convolutional neural network to recognise characters from a cartoon series. Classic processing still does much of the work: images are converted to greyscale, resized to a fixed size, scaled and split into training and validation sets, and augmentation adds variety, before a Keras model learns to classify them. The lesson is that classic image processing and deep learning are partners, not rivals.

Practice tasks

  1. Convert the BGR pixel (30, 180, 220) to greyscale using the standard weights.
  2. Rescale a 1,280 × 720 frame by a factor of 0.6. What are the new dimensions?
  3. Apply an average blur and a median blur to the centre of the patch with rows (5, 5, 5), (5, 0, 5) and (5, 5, 5).
  4. Threshold the 4 × 4 example image at 200. How many pixels become white?
  5. Explain the difference between edges and contours, and why blurring comes before finding contours.
  6. Describe how a mask and a bitwise AND can keep only the centre of an image.

Install Chrysolites Tech Academy

Android / Chrome / Edge: open the browser menu and choose Install app or Add to Home screen.

iPhone / iPad (Safari): tap the Share button, then Add to Home Screen.