Docker Build Environments

Why Docker for C++ Builds

C++ projects have notoriously complex dependency chains — specific compiler versions, system libraries, build tools, and third-party packages that must align exactly. Docker solves this by encapsulating the entire build environment in a reproducible image. Every team member and CI runner builds with identical toolchains, eliminating "works on my machine" failures.

Multi-Stage Dockerfile

Multi-stage builds separate compilation from the final runtime image. The build stage contains GCC, CMake, and development headers (often 2–3GB). The runtime stage contains only the compiled binary and shared libraries (typically 50–200MB).

# Dockerfile — Multi-stage C++ build with CMake
# Stage 1: Build environment
FROM ubuntu:24.04 AS builder

# Install build tools
RUN apt-get update && apt-get install -y --no-install-recommends \
    cmake \
    ninja-build \
    g++-14 \
    pkg-config \
    libssl-dev \
    libcurl4-openssl-dev \
    libsqlite3-dev \
    git \
    && rm -rf /var/lib/apt/lists/*

# Set compiler
ENV CC=gcc-14 CXX=g++-14

# Copy source code
WORKDIR /src
COPY CMakeLists.txt CMakePresets.json ./
COPY src/ src/
COPY include/ include/
COPY cmake/ cmake/
COPY tests/ tests/

# Configure and build
RUN cmake --preset release -DBUILD_TESTING=OFF
RUN cmake --build build/release --parallel $(nproc)
RUN cmake --install build/release --prefix /install

# Stage 2: Minimal runtime image
FROM ubuntu:24.04 AS runtime

# Install only runtime dependencies (no -dev packages)
RUN apt-get update && apt-get install -y --no-install-recommends \
    libssl3 \
    libcurl4 \
    libsqlite3-0 \
    ca-certificates \
    && rm -rf /var/lib/apt/lists/*

# Copy built artifacts from builder
COPY --from=builder /install /usr/local

# Create non-root user
RUN useradd -m appuser
USER appuser

ENTRYPOINT ["/usr/local/bin/myapp"]

# Build the image
docker build -t myapp:latest .

# Run the application
docker run --rm myapp:latest --config /etc/app.conf

# Check image sizes
docker images myapp
# REPOSITORY   TAG     SIZE
# myapp        latest  85MB   (runtime only!)

# Compare with builder stage
docker build --target builder -t myapp:builder .
docker images myapp:builder
# REPOSITORY      TAG      SIZE
# myapp           builder  2.1GB  (full build environment)

Build Environment Images

Create reusable base images with all build tools pre-installed. Teams share these via a container registry, ensuring consistent environments without re-downloading GBs of packages on every build.

# Dockerfile.build-env — Reusable C++ build environment
FROM ubuntu:24.04

LABEL maintainer="team@example.com"
LABEL description="C++ build environment with GCC 14, Clang 18, CMake 3.29"

# Install multiple compiler versions for matrix testing
RUN apt-get update && apt-get install -y --no-install-recommends \
    # Build tools
    cmake ninja-build ccache pkg-config \
    # GCC
    gcc-14 g++-14 gcc-13 g++-13 \
    # Clang
    clang-18 clang++-18 lld-18 \
    # Testing & coverage
    lcov gcovr valgrind \
    # Common libraries
    libboost-all-dev libgtest-dev libfmt-dev \
    libspdlog-dev nlohmann-json3-dev \
    # Tools
    git curl wget ca-certificates \
    && rm -rf /var/lib/apt/lists/*

# Install latest CMake (if distro version is too old)
# RUN wget -qO- https://cmake.org/files/v3.29/cmake-3.29.0-linux-x86_64.tar.gz \
#     | tar xz --strip-components=1 -C /usr/local

# Configure ccache defaults
ENV CCACHE_DIR=/ccache
ENV CCACHE_MAXSIZE=2G
RUN mkdir -p /ccache

WORKDIR /workspace

# Build and push to registry
docker build -f Dockerfile.build-env -t registry.example.com/cpp-build:gcc14 .
docker push registry.example.com/cpp-build:gcc14

# Use in project Dockerfile
# FROM registry.example.com/cpp-build:gcc14 AS builder
# COPY . /workspace
# RUN cmake --preset release && cmake --build build/release

CMake Cache Persistence

Docker's layer cache helps, but CMake's own build cache (object files, configuration) is lost between builds unless persisted. Docker volumes and BuildKit cache mounts solve this.

# Method 1: BuildKit cache mount (best for CI)
# syntax=docker/dockerfile:1.4
FROM ubuntu:24.04 AS builder

RUN apt-get update && apt-get install -y cmake ninja-build g++-14

WORKDIR /src
COPY . .

# Mount CMake build directory as cache
# Persists between builds on the same host
RUN --mount=type=cache,target=/src/build \
    cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -S . -B build && \
    cmake --build build --parallel $(nproc) && \
    cmake --install build --prefix /install

# Method 2: Docker volume for local development
# Create a persistent volume for build artifacts
docker volume create myapp-build-cache

# Build with volume-mounted cache
docker run --rm \
    -v $(pwd):/src \
    -v myapp-build-cache:/src/build \
    registry.example.com/cpp-build:gcc14 \
    bash -c "cmake -G Ninja -S /src -B /src/build && cmake --build /src/build -j$(nproc)"

# Subsequent builds reuse cached objects (fast incremental builds)

# Method 3: ccache inside Docker (compiler cache)
docker run --rm \
    -v $(pwd):/src \
    -v ccache-vol:/ccache \
    -e CCACHE_DIR=/ccache \
    -e CMAKE_C_COMPILER_LAUNCHER=ccache \
    -e CMAKE_CXX_COMPILER_LAUNCHER=ccache \
    registry.example.com/cpp-build:gcc14 \
    bash -c "cmake -G Ninja \
        -DCMAKE_C_COMPILER_LAUNCHER=ccache \
        -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
        -S /src -B /src/build && \
    cmake --build /src/build -j$(nproc)"

CI/CD with Docker

Docker containers provide hermetic CI builds. GitHub Actions and GitLab CI both support container-based jobs that use your custom build image.

# .github/workflows/build.yml — GitHub Actions with Docker
name: Build & Test

on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest
    container:
      image: registry.example.com/cpp-build:gcc14
    
    steps:
      - uses: actions/checkout@v4

      - name: Configure
        run: cmake --preset ci-release

      - name: Build
        run: cmake --build build/release --parallel $(nproc)

      - name: Test
        run: ctest --test-dir build/release --output-on-failure

      - name: Package
        run: cmake --build build/release --target package

      - name: Upload artifacts
        uses: actions/upload-artifact@v4
        with:
          name: release-packages
          path: build/release/*.deb

  # Matrix: test with multiple compilers
  compiler-matrix:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        include:
          - compiler: gcc-13
            image: registry.example.com/cpp-build:gcc13
          - compiler: gcc-14
            image: registry.example.com/cpp-build:gcc14
          - compiler: clang-18
            image: registry.example.com/cpp-build:clang18

    container:
      image: ${{ matrix.image }}

    steps:
      - uses: actions/checkout@v4
      - run: cmake --preset ci-${{ matrix.compiler }}
      - run: cmake --build build/ci --parallel $(nproc)
      - run: ctest --test-dir build/ci --output-on-failure

Cross-Compilation in Containers

Docker containers can host cross-compilation toolchains, providing consistent cross-build environments without polluting the host system. Combined with QEMU user-mode emulation, you can even run the cross-compiled tests.

# Dockerfile.cross-arm64 — Cross-compile for ARM64 inside Docker
FROM ubuntu:24.04

RUN apt-get update && apt-get install -y --no-install-recommends \
    cmake ninja-build \
    gcc-aarch64-linux-gnu g++-aarch64-linux-gnu \
    qemu-user-static \
    && rm -rf /var/lib/apt/lists/*

# Register QEMU for transparent ARM64 binary execution
# (Allows running cross-compiled tests on x86 host)

WORKDIR /workspace
COPY cmake/aarch64-toolchain.cmake /toolchain.cmake

ENTRYPOINT ["cmake"]
CMD ["--help"]

# Build ARM64 binary from x86 host
docker run --rm \
    -v $(pwd):/workspace \
    cross-arm64 \
    -G Ninja \
    -DCMAKE_TOOLCHAIN_FILE=/toolchain.cmake \
    -DCMAKE_BUILD_TYPE=Release \
    -S /workspace -B /workspace/build-arm64

docker run --rm \
    -v $(pwd):/workspace \
    cross-arm64 \
    --build /workspace/build-arm64 --parallel

# Run tests via QEMU (transparent emulation)
docker run --rm --platform linux/arm64 \
    -v $(pwd)/build-arm64:/app \
    ubuntu:24.04 /app/tests/unit_tests

Dev Containers (VS Code)

VS Code Dev Containers provide a complete development environment inside Docker — with IntelliSense, debugging, and terminal access. The .devcontainer/ folder configures the environment declaratively.

# .devcontainer/devcontainer.json
{
    "name": "C++ Development",
    "build": {
        "dockerfile": "Dockerfile",
        "context": ".."
    },
    "features": {
        "ghcr.io/devcontainers/features/common-utils:2": {},
        "ghcr.io/devcontainers/features/github-cli:1": {}
    },
    "customizations": {
        "vscode": {
            "extensions": [
                "ms-vscode.cpptools",
                "ms-vscode.cmake-tools",
                "twxs.cmake",
                "ms-vscode.cpptools-extension-pack"
            ],
            "settings": {
                "cmake.configureOnOpen": true,
                "cmake.generator": "Ninja",
                "C_Cpp.default.compilerPath": "/usr/bin/g++-14"
            }
        }
    },
    "mounts": [
        "source=cpp-build-cache,target=/workspace/build,type=volume"
    ],
    "postCreateCommand": "cmake --preset dev",
    "remoteUser": "vscode"
}

# .devcontainer/Dockerfile
FROM registry.example.com/cpp-build:gcc14

# Install development-specific tools (not needed in CI)
RUN apt-get update && apt-get install -y --no-install-recommends \
    gdb \
    clang-format-18 \
    clang-tidy-18 \
    cppcheck \
    doxygen \
    && rm -rf /var/lib/apt/lists/*

# Create non-root user for VS Code
ARG USERNAME=vscode
ARG USER_UID=1000
ARG USER_GID=$USER_UID
RUN groupadd --gid $USER_GID $USERNAME \
    && useradd --uid $USER_UID --gid $USER_GID -m $USERNAME \
    && apt-get update && apt-get install -y sudo \
    && echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME

USER $USERNAME
WORKDIR /workspace

Compiler Version Pinning

Docker ensures every team member and CI system uses the exact same compiler version. This eliminates subtle bugs from compiler differences — particularly important for C++ where optimization behavior varies between releases.

# Pin exact compiler versions in your build image
FROM ubuntu:24.04

# Install SPECIFIC versions (not just gcc-14, but the exact patch)
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
        gcc-14=14.1.0-1ubuntu1 \
        g++-14=14.1.0-1ubuntu1 \
        libstdc++-14-dev=14.1.0-1ubuntu1 \
    && rm -rf /var/lib/apt/lists/*

# Or use LLVM's official apt repo for exact Clang versions
# RUN wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh
# RUN ./llvm.sh 18  # Installs clang-18 from LLVM's repo

Reproducibility Guarantees

True reproducible builds require controlling every variable — compiler, flags, timestamps, and system state. Docker combined with CMake best practices achieves bit-for-bit reproducibility.

cmake_minimum_required(VERSION 3.21)
project(ReproducibleBuild LANGUAGES CXX)

# Disable __DATE__ and __TIME__ macros (non-reproducible)
add_compile_options(-Werror=date-time)

# Use deterministic ar (no timestamps in static libraries)
set(CMAKE_C_ARCHIVE_CREATE " Dcr  ")
set(CMAKE_CXX_ARCHIVE_CREATE " Dcr  ")

# Sort source files for deterministic compilation order
file(GLOB_RECURSE SOURCES CONFIGURE_DEPENDS src/*.cpp)
list(SORT SOURCES)

add_executable(app ${SOURCES})

# Embed build info as compile definitions (not macros)
string(TIMESTAMP BUILD_TIMESTAMP "%Y-%m-%dT%H:%M:%SZ" UTC)
target_compile_definitions(app PRIVATE
    BUILD_VERSION="${PROJECT_VERSION}"
    BUILD_COMMIT="${GIT_SHA}"
)

# Reproducible Docker build command
docker build \
    --no-cache \
    --build-arg BUILDKIT_INLINE_CACHE=1 \
    --build-arg SOURCE_DATE_EPOCH=$(git log -1 --format=%ct) \
    -t myapp:$(git rev-parse --short HEAD) \
    .

# Verify reproducibility: build twice, compare hashes
docker build -t myapp:check1 .
docker build -t myapp:check2 .
# Extract and compare binaries
docker run --rm myapp:check1 sha256sum /usr/local/bin/myapp
docker run --rm myapp:check2 sha256sum /usr/local/bin/myapp
# Should produce identical hashes

Docker Compose for Testing

Integration tests often require external services (databases, message queues, APIs). Docker Compose spins up the entire test environment alongside your application.

# docker-compose.test.yml — Integration test environment
services:
  # Build and run tests
  test-runner:
    build:
      context: .
      target: builder
    command: >
      bash -c "
        cmake --preset ci-test &&
        cmake --build build/test --parallel $(nproc) &&
        ctest --test-dir build/test --output-on-failure
      "
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      - DATABASE_URL=postgresql://test:test@postgres:5432/testdb
      - REDIS_URL=redis://redis:6379

  # Test dependencies
  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: test
      POSTGRES_PASSWORD: test
      POSTGRES_DB: testdb
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U test"]
      interval: 2s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7-alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 2s
      timeout: 5s
      retries: 5

# Run full integration test suite
docker compose -f docker-compose.test.yml up --build --abort-on-container-exit

# Clean up
docker compose -f docker-compose.test.yml down -v