Chrysolites Tech Academy

Containers with Docker and Kubernetes · Course book

Chapter 2: Underneath the Docker Commands

Pairs with Lesson 2: Docker in Depth: Building and Deploying Containerised Apps

Chapter 1 covered the everyday commands of Docker. This chapter goes underneath them. It explains why the industry moved from physical servers to virtual machines and then to containers, what Linux features make containers possible, how to write Dockerfiles that produce small, secure and fast images, how container networking and storage really work, and how Docker Compose and Docker Swarm manage applications made of many containers. The aim is to understand Docker well enough to reason about problems you have not seen before.

From physical servers to containers

In the early days of the web, each application usually had its own physical server. Buying a server took weeks, and to be safe it was sized for the busiest moment of the year, so most of the time it sat largely idle. Suppose a company runs 10 applications, each on its own server, and each uses on average 15 percent of its machine. Together they use the equivalent of 10 × 0.15 = 1.5 servers' worth of capacity, while paying for, powering and cooling 10.

Virtualisation changed that. A hypervisor lets one physical machine run many virtual machines, so the same 10 applications might fit on 2 or 3 physical servers, each now much busier. New VMs could be created in minutes rather than bought in weeks. But every VM still carried a whole operating system, consuming memory and disk and taking minutes to boot.

Containers took the next step by sharing one operating system kernel among many isolated applications. A container starts in about a second and can be only tens of megabytes, so one machine can run far more containers than VMs, and an application can be scaled from one copy to fifty almost instantly. Just as important, the container image became a standard package for software: developers, testers and operations all handle the same image.

EraUnit of deploymentTime to provisionTypical problem
Physical serversA whole machineWeeksExpensive machines sitting mostly idle
Virtual machinesA simulated machine with its own OSMinutesEvery VM carries a full operating system
ContainersAn application and its librariesSecondsMany small pieces to coordinate

What makes a container

A container is not a special kind of machine; it is an ordinary process on the host, wrapped by Linux features that limit what it can see and use.

  • Namespaces give the process its own view of the system. In its own process namespace it sees only its own processes, numbered from 1; in its own network namespace it has its own network interfaces and ports; in its own mount namespace it sees its own file system.
  • Control groups, cgroups, limit how much of the machine it can use. docker run --memory 512m --cpus 1.5 caps a container at 512 megabytes of memory and the equivalent of one and a half processor cores, so one misbehaving container cannot starve the others.
  • A layered file system stacks the image's read-only layers and adds a thin writable layer on top for the running container. Ten containers started from the same image share the same layers on disk, and each stores only its own changes.
layer 1: base OS files (FROM)layer 2: language runtimelayer 3: installed dependencies (RUN)layer 4: application code (COPY) container Awritable layercontainer Bwritable layer read-only,shared on disk Each container writes only to its own top layer; the image underneath never changes.
Figure 2.1 — Layers in practice. Containers from the same image share its read-only layers, so starting another copy costs almost no extra disk space.

The writable layer uses a technique called copy on write. When a container reads a file, it reads it straight from the shared image layers. Only when it changes a file is that file copied up into its own writable layer and modified there. Deleting a container deletes its writable layer and every change in it, which is exactly why lasting data must live in volumes instead.

Because containers share the host kernel, their isolation, while strong, is not as complete as a virtual machine's. That is why good practice is to run applications inside containers as an ordinary user, not as root, and to grant them no more access than they need.

Writing better Dockerfiles

A working Dockerfile is easy; a good one takes some care. Four habits make the biggest difference.

Choose a small base image. A full language image includes compilers and tools an application does not need at run time. "Slim" and "alpine" variants of the same image are often several times smaller. Smaller images download faster, start faster and contain fewer components that could carry security flaws.

Use multi-stage builds. A multi-stage Dockerfile builds the application in one stage, with all the tools needed, then copies only the finished result into a second, minimal stage. The build tools never reach the final image.

FROM node:20 AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM nginx:alpine
COPY --from=build /app/dist /usr/share/nginx/html

The first stage installs dependencies and builds a website; the second is a small web server holding nothing but the built files. The final image can be a small fraction of the size of the first stage.

Order instructions for caching, as Chapter 1 explained: things that change rarely first, your own code last.

Exclude what is not needed. A .dockerignore file lists files and folders, such as local dependency folders, logs and secrets, that COPY should skip. It keeps images small and stops private files from leaking into images that may be shared.

~1,100 MB~250 MB~50 MB full base imageslim base imagemulti-stage buildthe same application, built three ways (illustrative) 05001,000 final image size (megabytes)
Figure 2.2 — An illustration of how much base images and multi-stage builds matter. Exact sizes vary, but differences of ten times or more are common.

Two smaller habits round this off. Add a USER instruction so that the application runs as an unprivileged user, and pin base images to specific versions, such as node:20.11-slim instead of node:latest, so that a rebuild next month produces the same result as today. Images can also be scanned automatically for components with known security flaws, a step many pipelines now include before an image is pushed.

The savings compound. If a server pulls a 1,100-megabyte image over a link that delivers 50 megabytes a second, the download takes 22 seconds; a 50-megabyte image takes 1 second. Across dozens of servers and many releases a day, that difference decides how quickly a fix reaches users.

Networks and storage in depth

Every user-defined network has a built-in DNS service: when a container looks up the name of another container or Compose service on the same network, Docker answers with its current address. This is why services should always refer to each other by name; addresses change whenever containers are recreated. For containers spread across several hosts, an overlay network joins them into one virtual network, which is what Docker Swarm uses.

Storage comes in three forms. Named volumes are managed by Docker and are the right default for data such as databases. Bind mounts map a specific host folder, ideal for development or for providing configuration files. tmpfs mounts keep data in memory only, never written to disk, suitable for temporary or sensitive scratch data. A useful rule: if you would be upset to lose it, it belongs in a named volume, and the volume should be backed up like any other important data. Removing a container never removes its named volumes automatically; they must be deleted deliberately, which protects data but means unused volumes should be tidied up from time to time.

Compose and Swarm in practice

Docker Compose files grow with an application. A few settings matter in real projects. depends_on controls start-up order, so a web service waits for its database container to start; combined with a health check, a command that tests whether a service is really ready, it can wait until the database is accepting connections, not just running. restart policies such as unless-stopped bring a crashed container back automatically. Separate Compose files, or overrides, let one project use bind mounts and debug settings in development and fixed images and stricter settings in production.

Docker Swarm extends the same ideas across several machines. In Swarm you declare a service, an image and the number of copies, called replicas, you want running. Swarm places the replicas across the cluster's nodes, restarts any that fail and, when you update the image, performs a rolling update, replacing replicas a few at a time so the service never goes down. A Compose file can be deployed to a Swarm as a stack with little change, which makes Swarm a gentle first step into orchestration before the larger world of Kubernetes.

Check your understanding

  • Twelve servers each average 20 percent use. How many fully used servers' worth of work is that? (Answer: 12 × 0.2 = 2.4.)
  • Which Linux feature gives a container its own view of processes and networks, and which limits its memory?
  • Why do ten containers from the same image use little more disk space than one?
  • How does a multi-stage build keep build tools out of the final image?
  • A 600-megabyte image is pulled over a 40-megabyte-a-second link. How long does it take? (Answer: 15 seconds.)
  • Why should services refer to each other by name rather than by IP address?
  • What does a Swarm service do when one of its replicas crashes?

Install Chrysolites Tech Academy

Android / Chrome / Edge: open the browser menu and choose Install app or Add to Home screen.

iPhone / iPad (Safari): tap the Share button, then Add to Home Screen.