Part 2: Containerization with Docker

Introduction — Why Containers Matter

Every developer has encountered the infamous phrase: "It works on my machine." This seemingly innocuous statement has been the source of countless hours of debugging, delayed deployments, and frustrated operations teams. The root cause is simple — applications depend on their environment, and environments differ between development laptops, staging servers, and production infrastructure.

Containers solve this problem by packaging an application together with everything it needs to run — code, runtime, system libraries, and configuration — into a single, portable unit. When you ship a container, you ship the entire environment. The application runs identically whether it's on a developer's MacBook, a CI/CD pipeline runner, or a Kubernetes cluster in the cloud.

Docker, released in 2013, democratized containerization by making it accessible to every developer. Before Docker, container technologies like LXC existed but required deep Linux expertise. Docker provided a simple CLI, a declarative build format (Dockerfile), and a registry ecosystem (Docker Hub) that transformed how we build, ship, and run software.

In this article, we'll journey from container fundamentals through advanced Docker patterns. By the end, you'll be able to containerize any application, optimize images for production, compose multi-service architectures, and implement security best practices that protect your infrastructure.

Container Fundamentals

Containers vs. Virtual Machines

To understand containers, it helps to contrast them with virtual machines (VMs). A VM runs a complete operating system — its own kernel, system libraries, and userspace — on top of a hypervisor. Each VM might consume 1–2 GB of RAM just for the OS overhead, and take 30–60 seconds to boot.

Containers share the host operating system's kernel. They use Linux kernel features — namespaces for isolation and cgroups for resource limiting — to create lightweight, isolated environments. A container typically adds only 5–50 MB of overhead and starts in under a second.

flowchart TB
    subgraph VM["Virtual Machine Stack"]
        direction TB
        HW1[Physical Hardware]
        HV[Hypervisor]
        OS1[Guest OS 1
~1-2 GB]
        OS2[Guest OS 2
~1-2 GB]
        APP1[App A + Libs]
        APP2[App B + Libs]
        HW1 --> HV
        HV --> OS1
        HV --> OS2
        OS1 --> APP1
        OS2 --> APP2
    end

    subgraph CT["Container Stack"]
        direction TB
        HW2[Physical Hardware]
        HOS[Host OS + Kernel]
        CR[Container Runtime]
        C1[Container A
App + Libs ~50 MB]
        C2[Container B
App + Libs ~50 MB]
        C3[Container C
App + Libs ~50 MB]
        HW2 --> HOS
        HOS --> CR
        CR --> C1
        CR --> C2
        CR --> C3
    end
                            

The Linux Kernel Primitives

Containers are built on three fundamental Linux kernel features:

Namespaces provide isolation. Each container gets its own view of system resources: its own process tree (PID namespace), network stack (NET namespace), filesystem mounts (MNT namespace), hostname (UTS namespace), user IDs (USER namespace), and inter-process communication (IPC namespace). A process inside a container cannot see processes in other containers or on the host.

Control Groups (cgroups) enforce resource limits. They restrict how much CPU, memory, disk I/O, and network bandwidth a container can consume. Without cgroups, a runaway container could starve the entire host of resources.

Union Filesystems (like OverlayFS) enable the layered image model. Each layer in a Docker image is read-only; when a container runs, a thin writable layer is added on top. This means 100 containers running the same image share the base layers, saving enormous amounts of disk space.

Docker Architecture

Docker uses a client-server architecture. Understanding the components and how they interact is essential for effective container management.

flowchart LR
    CLI[Docker CLI
docker build/run/push]
    API[REST API]
    DAEMON[Docker Daemon
dockerd]
    CONTAINERD[containerd]
    RUNC[runc]
    REG[Registry
Docker Hub / ACR / GHCR]
    IMG[Images]
    CON[Containers]
    VOL[Volumes]
    NET[Networks]

    CLI -->|HTTP/Unix Socket| API
    API --> DAEMON
    DAEMON --> CONTAINERD
    CONTAINERD --> RUNC
    DAEMON --> IMG
    DAEMON --> CON
    DAEMON --> VOL
    DAEMON --> NET
    DAEMON <-->|pull/push| REG
                            

Docker Client (CLI) — The command-line interface that developers interact with. Commands like docker build, docker run, and docker push are sent to the Docker daemon via REST API calls over a Unix socket or TCP connection.

Docker Daemon (dockerd) — The background service that manages Docker objects: images, containers, networks, and volumes. It listens for API requests and orchestrates container lifecycle operations.

containerd — The industry-standard container runtime that manages the complete container lifecycle — image transfer, container execution, storage, and networking. Docker delegates actual container operations to containerd.

runc — The low-level OCI-compliant runtime that creates and runs containers. It interfaces directly with the Linux kernel to set up namespaces, cgroups, and execute the container's entrypoint process.

Registry — A storage and distribution service for Docker images. Docker Hub is the default public registry, but organizations typically use private registries like Azure Container Registry (ACR), GitHub Container Registry (GHCR), or Amazon ECR.

Writing Dockerfiles

A Dockerfile is a text document containing instructions for building a Docker image. Each instruction creates a layer in the image, and layers are cached for fast rebuilds. Understanding instruction ordering and best practices is crucial for efficient, secure images.

Core Dockerfile Instructions

# Base image — always start with a specific tag, never :latest
FROM python:3.12-slim

# Set metadata labels for image identification
LABEL maintainer="developer@example.com"
LABEL version="1.0"
LABEL description="Production Flask API"

# Set working directory inside the container
WORKDIR /app

# Copy dependency files first (for better layer caching)
COPY requirements.txt .

# Install dependencies
RUN pip install --no-cache-dir -r requirements.txt

# Copy application source code
COPY src/ ./src/

# Document which port the application uses
EXPOSE 8000

# Set environment variables
ENV FLASK_ENV=production
ENV PYTHONUNBUFFERED=1

# Health check for container orchestrators
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
    CMD curl -f http://localhost:8000/health || exit 1

# Define the command to run when the container starts
CMD ["gunicorn", "--bind", "0.0.0.0:8000", "--workers", "4", "src.app:app"]

CMD vs ENTRYPOINT

CMD provides default arguments that can be overridden at runtime. ENTRYPOINT defines the executable that always runs. When both are used together, CMD provides default arguments to ENTRYPOINT:

# ENTRYPOINT + CMD pattern
# Container always runs gunicorn, but workers count can be overridden
FROM python:3.12-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY src/ ./src/

ENTRYPOINT ["gunicorn", "--bind", "0.0.0.0:8000", "src.app:app"]
CMD ["--workers", "4"]

# Usage:
# docker run myapp                    → runs with 4 workers (default)
# docker run myapp --workers 8        → runs with 8 workers (overridden)

Multi-Stage Builds

Multi-stage builds are a game-changer for production images. They allow you to use one stage for building (with compilers, dev tools, test frameworks) and copy only the final artifacts into a minimal runtime image. This dramatically reduces image size and attack surface.

flowchart LR
    subgraph Stage1["Build Stage (~900 MB)"]
        S1[python:3.12
Full SDK]
        S2[Install build deps
gcc, libffi-dev]
        S3[pip install
all dependencies]
        S4[Run tests]
        S1 --> S2 --> S3 --> S4
    end

    subgraph Stage2["Production Stage (~120 MB)"]
        P1[python:3.12-slim
Minimal runtime]
        P2[COPY --from=build
installed packages]
        P3[COPY app source]
        P4[Non-root user
Final image]
        P1 --> P2 --> P3 --> P4
    end

    Stage1 -->|"COPY --from=build"| Stage2
                            

# ============================================
# Stage 1: Build stage with full toolchain
# ============================================
FROM python:3.12 AS build

WORKDIR /app

# Install build dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
    gcc \
    libffi-dev \
    && rm -rf /var/lib/apt/lists/*

# Install Python dependencies into a virtual environment
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy source and run tests
COPY src/ ./src/
COPY tests/ ./tests/
RUN python -m pytest tests/ --tb=short

# ============================================
# Stage 2: Production runtime (minimal image)
# ============================================
FROM python:3.12-slim AS production

WORKDIR /app

# Create non-root user
RUN groupadd -r appuser && useradd -r -g appuser appuser

# Copy only the virtual environment from build stage
COPY --from=build /opt/venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

# Copy application source
COPY src/ ./src/

# Switch to non-root user
USER appuser

EXPOSE 8000

HEALTHCHECK --interval=30s --timeout=10s --retries=3 \
    CMD curl -f http://localhost:8000/health || exit 1

CMD ["gunicorn", "--bind", "0.0.0.0:8000", "--workers", "4", "src.app:app"]

Building a Flask API Container

Let's build a complete, production-ready Flask API and containerize it step by step. This example demonstrates real-world patterns you'll use in every containerized application.

The Application Code

# src/app.py — A production-ready Flask API
from flask import Flask, jsonify, request
import os
import time

app = Flask(__name__)

# Configuration from environment variables
APP_VERSION = os.environ.get("APP_VERSION", "1.0.0")
ENVIRONMENT = os.environ.get("FLASK_ENV", "development")

@app.route("/health", methods=["GET"])
def health_check():
    """Health endpoint for container orchestrators."""
    return jsonify({
        "status": "healthy",
        "version": APP_VERSION,
        "environment": ENVIRONMENT,
        "timestamp": time.time()
    }), 200

@app.route("/api/items", methods=["GET"])
def get_items():
    """Return a list of items."""
    items = [
        {"id": 1, "name": "Docker Fundamentals", "category": "containers"},
        {"id": 2, "name": "Kubernetes Basics", "category": "orchestration"},
        {"id": 3, "name": "CI/CD Pipelines", "category": "automation"},
    ]
    return jsonify({"items": items, "count": len(items)}), 200

@app.route("/api/items", methods=["POST"])
def create_item():
    """Create a new item."""
    data = request.get_json()
    if not data or "name" not in data:
        return jsonify({"error": "name field is required"}), 400
    new_item = {
        "id": int(time.time()),
        "name": data["name"],
        "category": data.get("category", "general")
    }
    return jsonify(new_item), 201

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8000, debug=(ENVIRONMENT == "development"))

Requirements File

# requirements.txt
flask==3.1.0
gunicorn==22.0.0

The Dockerfile

# Dockerfile for Flask API
FROM python:3.12-slim

# Security: Create non-root user
RUN groupadd -r appuser && useradd -r -g appuser -d /app -s /sbin/nologin appuser

WORKDIR /app

# Install dependencies first (layer caching optimization)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY src/ ./src/

# Set ownership and switch to non-root user
RUN chown -R appuser:appuser /app
USER appuser

# Environment configuration
ENV FLASK_ENV=production
ENV APP_VERSION=1.0.0
ENV PYTHONUNBUFFERED=1

EXPOSE 8000

HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
    CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')" || exit 1

CMD ["gunicorn", "--bind", "0.0.0.0:8000", "--workers", "2", "--timeout", "120", "src.app:app"]

Building the Image

# Build the Docker image with a tag
docker build -t flask-api:1.0.0 .

# View the build output and image size
docker images flask-api
# REPOSITORY   TAG     IMAGE ID       CREATED         SIZE
# flask-api    1.0.0   a1b2c3d4e5f6   5 seconds ago   135MB

# Inspect image layers to understand size distribution
docker history flask-api:1.0.0 --human --no-trunc

Running Containers

Once you have an image, you run it as a container. Docker provides extensive options for port mapping, environment variables, volumes, resource limits, and networking.

# Basic: Run the container in detached mode with port mapping
docker run -d \
    --name flask-api \
    -p 8000:8000 \
    flask-api:1.0.0

# With environment variables and resource limits
docker run -d \
    --name flask-api-prod \
    -p 8000:8000 \
    -e FLASK_ENV=production \
    -e APP_VERSION=1.0.0 \
    --memory="256m" \
    --cpus="0.5" \
    --restart=unless-stopped \
    flask-api:1.0.0

# With a volume mount for persistent data
docker run -d \
    --name flask-api-dev \
    -p 8000:8000 \
    -v $(pwd)/src:/app/src:ro \
    -e FLASK_ENV=development \
    flask-api:1.0.0

# Test the running container
curl http://localhost:8000/health
# {"status":"healthy","version":"1.0.0","environment":"production","timestamp":1715692800.0}

# View container logs
docker logs flask-api --follow --tail 50

# Execute a command inside the running container
docker exec -it flask-api /bin/bash

# Stop and remove the container
docker stop flask-api && docker rm flask-api

Docker Registries

A registry is where you store and distribute Docker images. Think of it as "GitHub for container images." Understanding registry workflows is essential for CI/CD pipelines and team collaboration.

Registry Options

Docker Hub — The default public registry. Free tier allows unlimited public images and one private repository. Best for open-source projects and public base images.

GitHub Container Registry (GHCR) — Tightly integrated with GitHub Actions. Images are tied to repositories, inheriting access controls. Excellent for teams already using GitHub for CI/CD.

Azure Container Registry (ACR) — Enterprise-grade private registry with geo-replication, content trust, image scanning, and integration with Azure Kubernetes Service. Supports Helm charts and OCI artifacts.

# ============================================
# Docker Hub workflow
# ============================================
# Login to Docker Hub
docker login -u myusername

# Tag image for Docker Hub (format: username/repository:tag)
docker tag flask-api:1.0.0 myusername/flask-api:1.0.0
docker tag flask-api:1.0.0 myusername/flask-api:latest

# Push to Docker Hub
docker push myusername/flask-api:1.0.0
docker push myusername/flask-api:latest

# ============================================
# GitHub Container Registry workflow
# ============================================
# Login with a Personal Access Token (PAT)
echo $GITHUB_TOKEN | docker login ghcr.io -u USERNAME --password-stdin

# Tag for GHCR (format: ghcr.io/owner/image:tag)
docker tag flask-api:1.0.0 ghcr.io/myorg/flask-api:1.0.0

# Push to GHCR
docker push ghcr.io/myorg/flask-api:1.0.0

# ============================================
# Azure Container Registry workflow
# ============================================
# Login to ACR
az acr login --name myregistry

# Tag for ACR (format: registryname.azurecr.io/image:tag)
docker tag flask-api:1.0.0 myregistry.azurecr.io/flask-api:1.0.0

# Push to ACR
docker push myregistry.azurecr.io/flask-api:1.0.0

# Build directly in ACR (no local Docker needed!)
az acr build --registry myregistry --image flask-api:1.0.0 .

Docker Compose

Real applications rarely run as a single container. A typical web application needs an API server, a database, a cache, and possibly a message queue. Docker Compose lets you define and run multi-container applications using a declarative YAML file.

# docker-compose.yml — Flask API + Redis + PostgreSQL
version: "3.9"

services:
  # Flask API service
  api:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8000:8000"
    environment:
      - FLASK_ENV=development
      - REDIS_URL=redis://cache:6379/0
      - DATABASE_URL=postgresql://appuser:secretpass@db:5432/appdb
    depends_on:
      cache:
        condition: service_healthy
      db:
        condition: service_healthy
    volumes:
      - ./src:/app/src:ro
    networks:
      - app-network
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3

  # Redis cache service
  cache:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    volumes:
      - redis-data:/data
    networks:
      - app-network
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5
    command: redis-server --maxmemory 128mb --maxmemory-policy allkeys-lru

  # PostgreSQL database service
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: appuser
      POSTGRES_PASSWORD: secretpass
      POSTGRES_DB: appdb
    ports:
      - "5432:5432"
    volumes:
      - postgres-data:/var/lib/postgresql/data
    networks:
      - app-network
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U appuser -d appdb"]
      interval: 10s
      timeout: 5s
      retries: 5

volumes:
  redis-data:
    driver: local
  postgres-data:
    driver: local

networks:
  app-network:
    driver: bridge

# Start all services in detached mode
docker compose up -d

# View running services and their status
docker compose ps

# Stream logs from all services
docker compose logs -f

# Scale a specific service
docker compose up -d --scale api=3

# Stop and remove all containers, networks, and volumes
docker compose down -v

# Rebuild images and restart
docker compose up -d --build

Container Security

Containers provide isolation, but they are not inherently secure. A misconfigured container can expose your entire infrastructure. Security must be considered at every layer — from the base image you choose to how you run the container in production.

Security Best Practices Dockerfile

# Security-hardened Dockerfile
FROM python:3.12-slim AS production

# 1. Use specific base image digest for supply chain security
# FROM python:3.12-slim@sha256:abc123...

# 2. Create non-root user with no shell access
RUN groupadd -r appuser && \
    useradd -r -g appuser -d /app -s /sbin/nologin appuser

WORKDIR /app

# 3. Install dependencies as root, then lock down
COPY requirements.txt .
RUN pip install --no-cache-dir --no-compile -r requirements.txt && \
    rm -rf /root/.cache/pip

# 4. Copy app with correct ownership
COPY --chown=appuser:appuser src/ ./src/

# 5. Remove unnecessary system packages
RUN apt-get purge -y --auto-remove && \
    rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# 6. Make filesystem read-only where possible
RUN chmod -R 555 /app/src

# 7. Switch to non-root user
USER appuser

# 8. Drop all Linux capabilities
# (enforced at runtime: docker run --cap-drop=ALL)

# 9. Use COPY instead of ADD (ADD can fetch remote URLs)
# 10. No secrets in image layers — use runtime secrets

EXPOSE 8000

CMD ["gunicorn", "--bind", "0.0.0.0:8000", "--workers", "2", "src.app:app"]

Runtime Security Flags

# Run container with maximum security constraints
docker run -d \
    --name flask-api-secure \
    -p 8000:8000 \
    --read-only \
    --tmpfs /tmp:rw,noexec,nosuid,size=64m \
    --cap-drop=ALL \
    --security-opt=no-new-privileges:true \
    --memory="256m" \
    --cpus="0.5" \
    --pids-limit=100 \
    --user 1000:1000 \
    flask-api:1.0.0

# Scan image for vulnerabilities
docker scout cves flask-api:1.0.0

# Alternative: Trivy scanner
trivy image flask-api:1.0.0

# View image SBOM (Software Bill of Materials)
docker scout sbom flask-api:1.0.0

Image Scanning and Supply Chain

Always scan your images for known vulnerabilities before deploying to production. Integrate scanning into your CI/CD pipeline so vulnerable images never reach production registries.

Container Best Practices

Layer Caching Optimization

Docker caches each layer. If a layer hasn't changed, Docker reuses the cached version. Order your Dockerfile instructions from least-frequently-changed to most-frequently-changed:

# GOOD: Optimal layer ordering for cache efficiency
FROM python:3.12-slim

WORKDIR /app

# Layer 1: System packages (changes rarely)
RUN apt-get update && apt-get install -y --no-install-recommends \
    curl \
    && rm -rf /var/lib/apt/lists/*

# Layer 2: Python dependencies (changes occasionally)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Layer 3: Application code (changes frequently)
COPY src/ ./src/

# Only Layer 3 rebuilds when you change app code!
CMD ["gunicorn", "--bind", "0.0.0.0:8000", "src.app:app"]

.dockerignore

A .dockerignore file prevents unnecessary files from being sent to the build context, speeding up builds and preventing sensitive data leakage:

# .dockerignore
.git
.gitignore
.env
.env.*
__pycache__
*.pyc
*.pyo
.pytest_cache
.coverage
htmlcov/
node_modules/
.vscode/
.idea/
*.md
LICENSE
docker-compose*.yml
Dockerfile*
.dockerignore
tests/
docs/
*.log

Health Checks

Health checks tell container orchestrators whether your application is ready to receive traffic. Without health checks, a container is considered healthy as long as the process is running — even if the application is deadlocked or failing to serve requests.

# HTTP health check (most common for web services)
HEALTHCHECK --interval=30s --timeout=10s --start-period=10s --retries=3 \
    CMD curl -f http://localhost:8000/health || exit 1

# TCP health check (for non-HTTP services)
HEALTHCHECK --interval=15s --timeout=5s --retries=5 \
    CMD nc -z localhost 5432 || exit 1

# Script-based health check (for complex checks)
HEALTHCHECK --interval=30s --timeout=10s --retries=3 \
    CMD python /app/healthcheck.py || exit 1

Logging Best Practices

Containers should log to stdout/stderr, not to files. This allows Docker's logging drivers to collect, forward, and manage logs centrally. Configure your application to write structured JSON logs to stdout:

# src/logging_config.py — Structured JSON logging for containers
import logging
import json
import sys
from datetime import datetime, timezone

class JSONFormatter(logging.Formatter):
    """Format log records as JSON for container log aggregation."""
    def format(self, record):
        log_entry = {
            "timestamp": datetime.now(timezone.utc).isoformat(),
            "level": record.levelname,
            "logger": record.name,
            "message": record.getMessage(),
            "module": record.module,
            "function": record.funcName,
            "line": record.lineno
        }
        if record.exc_info:
            log_entry["exception"] = self.formatException(record.exc_info)
        return json.dumps(log_entry)

def setup_logging():
    """Configure structured logging for containerized applications."""
    handler = logging.StreamHandler(sys.stdout)
    handler.setFormatter(JSONFormatter())

    root_logger = logging.getLogger()
    root_logger.setLevel(logging.INFO)
    root_logger.addHandler(handler)

    return root_logger

# Usage
logger = setup_logging()
logger.info("Application started", extra={"port": 8000, "workers": 4})

Conclusion & What's Next

Docker has fundamentally transformed how we build, ship, and run software. In this article, we covered the entire Docker ecosystem — from understanding the kernel primitives that make containers possible, to writing production-ready Dockerfiles, composing multi-service architectures, and implementing security best practices that protect your infrastructure.

The key takeaways from this deep dive:

• Containers are not VMs — they share the host kernel and use namespaces/cgroups for lightweight isolation
• Multi-stage builds are essential for production — separate build-time dependencies from runtime artifacts
• Layer caching matters — order Dockerfile instructions from least to most frequently changed
• Docker Compose enables declarative multi-service applications for development and testing
• Security is not optional — run as non-root, drop capabilities, scan images, use read-only filesystems
• Structured logging to stdout enables centralized log management in container environments

But Docker alone doesn't solve the challenges of running containers at scale — scheduling across clusters, self-healing, rolling updates, service discovery, and load balancing. For that, we need an orchestration platform.