Part 9: BuildKit & Modern Build Techniques

What Is BuildKit?

BuildKit is a toolkit for converting source code to build artifacts in an efficient, expressive, and repeatable manner. Originally developed at Moby (Docker's open-source project), it replaced Docker's legacy builder as the default engine in Docker 23.0 (February 2023).

Legacy Builder vs BuildKit

Enabling BuildKit

If you're running Docker 23.0+, BuildKit is already your default builder. For older installations or explicit control, here are the activation methods:

Method 1: Environment Variable

# Enable for a single build
DOCKER_BUILDKIT=1 docker build -t myapp .

# Enable permanently in your shell profile (~/.bashrc or ~/.zshrc)
export DOCKER_BUILDKIT=1

Method 2: Docker Daemon Configuration

{
  "features": {
    "buildkit": true
  },
  "builder": {
    "gc": {
      "enabled": true,
      "defaultKeepStorage": "20GB"
    }
  }
}

# Apply daemon config (Linux)
sudo systemctl restart docker

# Verify BuildKit is active
docker build --help | grep buildkit
docker info | grep -i buildkit
# BuildKit: true

Method 3: Docker Buildx (Recommended)

# Buildx is the CLI plugin that wraps BuildKit
# Check if available
docker buildx version
# github.com/docker/buildx v0.14.0

# Create a new builder instance with more features
docker buildx create --name mybuilder --use --bootstrap

# Inspect the builder
docker buildx inspect mybuilder
# Name:   mybuilder
# Driver: docker-container
# Nodes:
# Name:      mybuilder0
# Endpoint:  unix:///var/run/docker.sock
# Status:    running
# Platforms: linux/amd64, linux/arm64, linux/arm/v7

# Build with buildx (equivalent to docker build but more options)
docker buildx build -t myapp:latest --load .

Parallel Stage Execution

BuildKit analyzes the dependency graph of your multi-stage Dockerfile and executes independent stages simultaneously. This can cut build times dramatically for complex Dockerfiles with multiple independent stages.

flowchart LR
    subgraph Legacy["Legacy Builder (Sequential)"]
        direction TB
        A1[Stage: deps
45s] --> B1[Stage: test
30s]
        B1 --> C1[Stage: build
20s]
        C1 --> D1[Stage: runtime
5s]
    end
    subgraph BK["BuildKit (Parallel)"]
        direction TB
        A2[Stage: deps
45s] --> C2[Stage: build
20s]
        A2 --> B2[Stage: test
30s]
        C2 --> D2[Stage: runtime
5s]
    end
    style Legacy fill:#fff3f3,stroke:#BF092F
    style BK fill:#f0fff7,stroke:#3B9797
                            

In this example, the test and build stages both depend on deps but not on each other. The legacy builder runs them sequentially (100s total). BuildKit runs them in parallel (70s total — a 30% improvement).

# syntax=docker/dockerfile:1
# Multi-stage Dockerfile with parallel-friendly stages

FROM node:20-alpine AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci

# These two stages run IN PARALLEL (both depend only on 'deps')
FROM deps AS test
COPY . .
RUN npm run lint
RUN npm run test

FROM deps AS build
COPY . .
RUN npm run build

# Final stage — depends on 'build' (test must also pass)
FROM node:20-alpine AS runtime
WORKDIR /app
ENV NODE_ENV=production
COPY --from=deps /app/node_modules ./node_modules
COPY --from=build /app/dist ./dist
COPY package.json .
USER node
EXPOSE 3000
CMD ["node", "dist/index.js"]

# Build with progress=plain to see parallel execution
docker build --progress=plain -t myapp .

# You'll see output like:
# #5 [test 1/3] COPY . .
# #6 [build 1/2] COPY . .     ← running simultaneously!
# #5 [test 2/3] RUN npm run lint
# #6 [build 2/2] RUN npm run build

Advanced Cache Mounts

Cache mounts are BuildKit's killer feature for build performance. They persist a directory across builds without including it in the image layers — perfect for package manager caches that would otherwise be re-downloaded on every build.

APT Package Cache

# syntax=docker/dockerfile:1

# ❌ Without cache mount: downloads packages every build (~30s)
RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/*

# ✅ With cache mount: downloads only new/updated packages (~3s)
RUN --mount=type=cache,target=/var/cache/apt \
    --mount=type=cache,target=/var/lib/apt \
    apt-get update && apt-get install -y --no-install-recommends curl

Python pip Cache

# syntax=docker/dockerfile:1

# ❌ Without cache: downloads all wheels every build
RUN pip install --no-cache-dir -r requirements.txt

# ✅ With cache mount: reuses previously downloaded wheels
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

Node.js npm/yarn Cache

# syntax=docker/dockerfile:1

# npm with cache mount
RUN --mount=type=cache,target=/root/.npm \
    npm ci

# yarn with cache mount
RUN --mount=type=cache,target=/usr/local/share/.cache/yarn \
    yarn install --frozen-lockfile

# pnpm with cache mount
RUN --mount=type=cache,target=/root/.local/share/pnpm/store \
    pnpm install --frozen-lockfile

Go Module & Build Cache

# syntax=docker/dockerfile:1

# Go benefits from TWO cache mounts: module downloads + compilation
RUN --mount=type=cache,target=/go/pkg/mod \
    --mount=type=cache,target=/root/.cache/go-build \
    go build -o /app/server .

Cache Mount Performance Impact

Secret Mounts

Secret mounts solve the fundamental problem of needing credentials during build without embedding them in image layers. Unlike build arguments (which appear in docker history), secrets are mounted as tmpfs and never persist in any layer.

# syntax=docker/dockerfile:1

FROM python:3.12-slim

WORKDIR /app
COPY requirements.txt .

# Mount secret at build time — accessible only during this RUN
RUN --mount=type=secret,id=pip_conf,target=/etc/pip.conf \
    pip install -r requirements.txt

# Alternative: read secret as environment variable
RUN --mount=type=secret,id=api_key \
    API_KEY=$(cat /run/secrets/api_key) && \
    curl -H "Authorization: Bearer $API_KEY" https://registry.example.com/config > config.json

Build commands pass secrets from the host:

# Pass a file as a secret
docker build --secret id=pip_conf,src=$HOME/.pip/pip.conf -t myapp .

# Pass a secret from an environment variable
docker build --secret id=api_key,env=MY_API_KEY -t myapp .

# Multiple secrets
docker build \
    --secret id=npm_token,src=.npmrc \
    --secret id=github_token,env=GITHUB_TOKEN \
    -t myapp .

# Verify: secret is NOT in image history
docker history myapp --no-trunc | grep -i secret
# (no output — secrets are invisible)

Private npm Registry Example

# syntax=docker/dockerfile:1
FROM node:20-alpine AS build

WORKDIR /app
COPY package.json package-lock.json ./

# Mount .npmrc with private registry token
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc \
    npm ci

COPY . .
RUN npm run build

FROM node:20-alpine
WORKDIR /app
COPY --from=build /app/dist ./dist
COPY --from=build /app/node_modules ./node_modules
CMD ["node", "dist/index.js"]

SSH Forwarding

SSH mounts forward your local SSH agent into the build container, allowing operations like cloning private Git repositories without copying SSH keys into the image. The SSH socket is available only during the RUN instruction and leaves no trace in layers.

# syntax=docker/dockerfile:1
FROM golang:1.22-alpine AS build

# Install git and SSH client
RUN apk add --no-cache git openssh-client

# Configure SSH to trust GitHub's host key
RUN mkdir -p -m 0700 ~/.ssh && \
    ssh-keyscan github.com >> ~/.ssh/known_hosts

WORKDIR /app
COPY go.mod go.sum ./

# Use SSH agent forwarding to download private modules
RUN --mount=type=ssh \
    git config --global url."git@github.com:".insteadOf "https://github.com/" && \
    go mod download

COPY . .
RUN CGO_ENABLED=0 go build -o /server .

FROM scratch
COPY --from=build /server /server
ENTRYPOINT ["/server"]

# Ensure SSH agent is running with your key loaded
eval $(ssh-agent)
ssh-add ~/.ssh/id_ed25519

# Build with SSH forwarding
docker build --ssh default -t myapp .

# Or specify a specific key file
docker build --ssh default=$HOME/.ssh/id_ed25519 -t myapp .

Build Arguments & Named Build Contexts

BuildKit extends build arguments with named contexts — the ability to pull files from multiple directories, images, or Git repositories during a single build.

Named Build Contexts

# syntax=docker/dockerfile:1

# Reference a named context (provided at build time)
FROM node:20-alpine AS build
WORKDIR /app
COPY . .
RUN npm run build

FROM nginx:alpine
# Copy from the build stage
COPY --from=build /app/dist /usr/share/nginx/html
# Copy from a named context "configs"
COPY --from=configs nginx.conf /etc/nginx/nginx.conf

# Provide named contexts at build time
docker buildx build \
    --build-context configs=./deploy/nginx \
    --build-context shared=../shared-libraries \
    -t myapp .

# Use a remote image as a context
docker buildx build \
    --build-context base=docker-image://ubuntu:24.04 \
    -t myapp .

# Use a Git repository as a context
docker buildx build \
    --build-context config=https://github.com/org/config.git#main \
    -t myapp .

Dynamic Base Images with ARG

# syntax=docker/dockerfile:1
ARG BASE_IMAGE=node:20-alpine
ARG NODE_VERSION=20

FROM ${BASE_IMAGE} AS runtime
# ARG must be re-declared after FROM to use in this stage
ARG NODE_VERSION
RUN echo "Running on Node ${NODE_VERSION}"

# Override at build time for different environments
docker build --build-arg BASE_IMAGE=node:20-slim -t myapp:debian .
docker build --build-arg BASE_IMAGE=node:20-alpine -t myapp:alpine .

Inline Cache Export for CI/CD

In CI/CD pipelines, every build starts fresh — no local cache. BuildKit solves this by exporting cache metadata to a registry, allowing subsequent builds to pull cache layers remotely instead of rebuilding from scratch.

# Export cache inline (stored in the image manifest)
docker buildx build \
    --cache-to type=inline \
    --push \
    -t registry.example.com/myapp:latest .

# Next build imports cache from the registry
docker buildx build \
    --cache-from type=registry,ref=registry.example.com/myapp:latest \
    -t registry.example.com/myapp:latest .

# Separate cache image (doesn't bloat your app image)
docker buildx build \
    --cache-to type=registry,ref=registry.example.com/myapp:cache,mode=max \
    --cache-from type=registry,ref=registry.example.com/myapp:cache \
    --push \
    -t registry.example.com/myapp:latest .

GitHub Actions with BuildKit Cache

# .github/workflows/build.yml
name: Build and Push
on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to Container Registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build and Push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: ghcr.io/${{ github.repository }}:${{ github.sha }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

Reproducible Builds

A reproducible build produces a bit-for-bit identical image regardless of when or where it's built. This matters for supply chain security — you can verify that a deployed image was built from specific source code without trusting the build system.

Why Reproducibility Matters

• Supply chain verification — Prove an image comes from specific source
• Audit trails — Identical hash = identical content
• Cache efficiency — Deterministic builds maximize cache hits
• Regulatory compliance — Some industries require build provenance

Achieving Reproducibility

# syntax=docker/dockerfile:1

# Pin base image by digest (immutable reference)
FROM golang:1.22.0@sha256:a84e0fbb0e0e... AS build

WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .

# Reproducible build flags
RUN CGO_ENABLED=0 \
    GOFLAGS="-trimpath" \
    go build -ldflags="-s -w -buildid=" -o /server .

FROM scratch
COPY --from=build /server /server
ENTRYPOINT ["/server"]

# Set SOURCE_DATE_EPOCH for reproducible timestamps
SOURCE_DATE_EPOCH=$(git log -1 --format=%ct) \
docker buildx build \
    --build-arg SOURCE_DATE_EPOCH=$SOURCE_DATE_EPOCH \
    --output type=oci,dest=image.tar,rewrite-timestamp=true \
    -t myapp:latest .

# Verify reproducibility: build twice and compare digests
docker buildx build --load -t myapp:test1 .
docker buildx build --load -t myapp:test2 .
docker inspect myapp:test1 --format='{{.Id}}'
docker inspect myapp:test2 --format='{{.Id}}'
# Should produce identical IDs

Buildx & Multi-Platform Builds

Multi-platform builds produce images for multiple CPU architectures (amd64, arm64, armv7) from a single Dockerfile and build command. This is essential for supporting both x86 servers and ARM-based devices (Raspberry Pi, Apple Silicon, AWS Graviton).

flowchart TD
    A["docker buildx build
--platform linux/amd64,linux/arm64"] --> B["BuildKit Engine"]
    B --> C["Build for amd64"]
    B --> D["Build for arm64"]
    C --> E["Image manifest
(amd64)"]
    D --> F["Image manifest
(arm64)"]
    E --> G["Multi-arch
Manifest List"]
    F --> G
    G --> H["Registry Push"]
    style B fill:#f0f7ff,stroke:#16476A
    style G fill:#f0fff7,stroke:#3B9797
                            

# Create a builder that supports multi-platform
docker buildx create --name multiarch --driver docker-container --use --bootstrap

# Build for multiple platforms simultaneously
docker buildx build \
    --platform linux/amd64,linux/arm64,linux/arm/v7 \
    --push \
    -t registry.example.com/myapp:latest .

# Build for a specific platform (useful for testing)
docker buildx build \
    --platform linux/arm64 \
    --load \
    -t myapp:arm64 .

# Inspect the multi-arch manifest
docker buildx imagetools inspect registry.example.com/myapp:latest
# Name:      registry.example.com/myapp:latest
# MediaType: application/vnd.oci.image.index.v1+json
# Manifests:
#   Name:      ...@sha256:abc123
#   Platform:  linux/amd64
#   Name:      ...@sha256:def456
#   Platform:  linux/arm64
#   Name:      ...@sha256:ghi789
#   Platform:  linux/arm/v7

Platform-Aware Dockerfiles

# syntax=docker/dockerfile:1

# BuildKit provides TARGETPLATFORM, TARGETOS, TARGETARCH automatically
FROM --platform=$BUILDPLATFORM golang:1.22-alpine AS build

ARG TARGETOS
ARG TARGETARCH

WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .

# Cross-compile for the target platform
RUN CGO_ENABLED=0 GOOS=${TARGETOS} GOARCH=${TARGETARCH} \
    go build -ldflags="-s -w" -o /server .

FROM scratch
COPY --from=build /server /server
ENTRYPOINT ["/server"]

Build Performance Tips

Diagnosing Slow Builds

# Show detailed timing for each step
docker build --progress=plain -t myapp . 2>&1 | tee build.log

# Find the slowest steps
grep "DONE\|CACHED" build.log | sort -t'=' -k2 -n

# Analyze build history layers
docker history myapp --format "{{.Size}}\t{{.CreatedBy}}" | sort -rh | head -10

# Check what's in the build context (should match .dockerignore)
docker build --no-cache --progress=plain . 2>&1 | grep "transferring context"
# => transferring context: 2.1MB (good) vs 800MB (bad — missing .dockerignore)

Exercises

Conclusion & Next Steps

BuildKit transforms Docker builds from a slow, sequential process into a fast, secure, and intelligent build system. The key capabilities to adopt immediately are:

• Cache mounts — Eliminate re-downloads of packages with --mount=type=cache for 5-30× faster rebuilds
• Secret mounts — Handle credentials without ever exposing them in image layers
• Parallel stages — Structure Dockerfiles so independent stages run simultaneously
• Registry cache — Export/import cache in CI/CD for fast builds on ephemeral runners
• Multi-platform — Build for amd64 and arm64 from a single command
• Reproducibility — Pin digests and set SOURCE_DATE_EPOCH for verifiable builds

Combined with the multi-stage patterns from Part 8, BuildKit gives you production-grade builds that are fast, secure, and reproducible — the three pillars of modern CI/CD infrastructure.