Part 22: Enterprise Container Platforms

Enterprise Container Challenges

An enterprise container environment isn't just "more containers." It introduces qualitatively different challenges that require dedicated tooling and processes:

• Multi-team governance — 50+ teams sharing infrastructure. Who can push images? Who can deploy to production? How do you prevent team A's broken image from crashing team B's service?
• Compliance requirements — SOC 2, HIPAA, PCI-DSS, FedRAMP all have specific requirements around image provenance, vulnerability management, and audit logging
• Heterogeneous infrastructure — On-premises servers, 3 cloud providers, edge locations, developer laptops. Images must work across amd64, arm64, and sometimes s390x
• Supply chain security — Verifying that every image in production was built from approved source code, scanned for vulnerabilities, and signed by authorized builders
• Air-gapped environments — Government, military, and highly regulated environments with no internet access. Every image must be mirrored internally

Why Docker Alone Isn't Enough

Container Registries at Scale

An enterprise registry is more than image storage. It's a security boundary, a compliance tool, and a distribution network. Requirements for production registries:

flowchart TD
    subgraph Producers["Image Producers"]
        CI["CI/CD Pipelines"]
        DEV["Developer Builds"]
    end
    subgraph Registry["Enterprise Registry (Harbor)"]
        SCAN["Vulnerability Scanner
(Trivy)"]
        SIGN["Image Signing
(Cosign/Notary)"]
        RBAC["RBAC Engine"]
        REP["Replication Controller"]
        STORE["Blob Storage
(S3/GCS/Minio)"]
    end
    subgraph Consumers["Image Consumers"]
        PROD["Production Clusters"]
        STAGE["Staging Environments"]
        EDGE["Edge Locations"]
    end

    CI -->|push| RBAC
    DEV -->|push| RBAC
    RBAC --> SCAN
    SCAN -->|"pass"| SIGN
    SIGN --> STORE
    STORE --> REP
    REP -->|replicate| PROD
    REP -->|replicate| STAGE
    REP -->|replicate| EDGE

    style Registry fill:#f0f9f9,stroke:#3B9797
    style Producers fill:#f8f9fa,stroke:#132440
    style Consumers fill:#f8f9fa,stroke:#16476A
                            

Harbor

Harbor is the CNCF's graduated container registry — the open-source standard for enterprise image management. It wraps the Docker Distribution registry with enterprise features: RBAC, vulnerability scanning, replication, content trust, and audit logging.

# Harbor deployment overview (production)
# Harbor consists of 8+ microservices:
# - Core (API + UI)
# - Registry (Docker Distribution v2)
# - JobService (async tasks: scanning, replication, GC)
# - Trivy adapter (vulnerability scanning)
# - Notary (content trust / image signing)
# - Database (PostgreSQL)
# - Redis (caching + job queue)
# - Proxy (nginx reverse proxy)

# Install Harbor via Helm (Kubernetes)
helm repo add harbor https://helm.goharbor.io
helm repo update

helm install harbor harbor/harbor \
    --namespace harbor --create-namespace \
    --set expose.type=ingress \
    --set expose.ingress.hosts.core=registry.company.com \
    --set externalURL=https://registry.company.com \
    --set persistence.persistentVolumeClaim.registry.size=500Gi \
    --set trivy.enabled=true \
    --set notary.enabled=true

# Login to Harbor
docker login registry.company.com
# Username: admin
# Password: ********

# Push an image to a Harbor project
docker tag myapp:1.0 registry.company.com/team-backend/myapp:1.0
docker push registry.company.com/team-backend/myapp:1.0

# Harbor automatically:
# 1. Checks RBAC (does this user have push rights to team-backend?)
# 2. Runs Trivy vulnerability scan
# 3. Blocks push if critical CVEs found (if policy enabled)
# 4. Replicates to configured remote registries
# 5. Logs the event for audit

# Configure vulnerability scanning policy via Harbor API
curl -X PUT "https://registry.company.com/api/v2.0/projects/team-backend" \
    -H "Content-Type: application/json" \
    -d '{
        "metadata": {
            "auto_scan": "true",
            "prevent_vul": "true",
            "severity": "critical"
        }
    }'
# Now: images with critical vulnerabilities cannot be pulled

Cloud Registries

# AWS ECR — Push workflow
aws ecr get-login-password --region us-east-1 | \
    docker login --username AWS --password-stdin 123456789.dkr.ecr.us-east-1.amazonaws.com

docker tag myapp:1.0 123456789.dkr.ecr.us-east-1.amazonaws.com/myapp:1.0
docker push 123456789.dkr.ecr.us-east-1.amazonaws.com/myapp:1.0

# Azure ACR — Push workflow
az acr login --name myregistry
docker tag myapp:1.0 myregistry.azurecr.io/myapp:1.0
docker push myregistry.azurecr.io/myapp:1.0

# Google Artifact Registry — Push workflow
gcloud auth configure-docker us-central1-docker.pkg.dev
docker tag myapp:1.0 us-central1-docker.pkg.dev/my-project/my-repo/myapp:1.0
docker push us-central1-docker.pkg.dev/my-project/my-repo/myapp:1.0

Air-Gapped Registries

Air-gapped environments (government, military, financial trading floors, submarine systems) have no internet connectivity whatsoever. Every container image must be physically transported into the network. This creates unique engineering challenges:

# Strategy 1: docker save/load (simple, small scale)
# On connected machine:
docker pull nginx:1.25-alpine
docker pull redis:7-alpine
docker save nginx:1.25-alpine redis:7-alpine | gzip > images-bundle.tar.gz
# Transfer images-bundle.tar.gz via USB drive, optical media, or data diode

# On air-gapped machine:
docker load < images-bundle.tar.gz
# Loaded image: nginx:1.25-alpine
# Loaded image: redis:7-alpine

# Strategy 2: Registry mirror (medium scale)
# Set up an internal Harbor instance in the air-gapped network
# On connected side: export entire projects from source registry
# Harbor supports "export/import" of entire repositories with metadata

# Strategy 3: skopeo for efficient registry-to-registry copy
# On connected machine with access to both networks (via data diode):
skopeo copy --all \
    docker://docker.io/library/nginx:1.25 \
    docker://internal-registry.airgap.local/library/nginx:1.25

# --all copies all architectures (manifest list)
# Copies only changed layers (efficient delta sync)

# Strategy 4: Crane (Google's registry tool) for bulk operations
crane copy docker.io/library/nginx:1.25 internal-registry:5000/nginx:1.25

# Strategy 5: OCI layout on disk (portable, standards-based)
skopeo copy docker://nginx:1.25 oci:///media/usb/images/nginx:1.25
# Creates OCI layout directory on USB drive
# On air-gapped side:
skopeo copy oci:///media/usb/images/nginx:1.25 docker://internal-registry:5000/nginx:1.25

# Configure Docker daemon to use internal registry as mirror
cat /etc/docker/daemon.json
# {
#   "registry-mirrors": ["https://internal-registry.airgap.local"],
#   "insecure-registries": []
# }

Access Control & RBAC

Enterprise registries implement Role-Based Access Control at multiple granularity levels — from system-wide admin down to per-tag permissions:

# Harbor RBAC Model (most comprehensive open-source implementation)
# System Level:
#   - System Admin: full control over all projects and configuration
#
# Project Level:
#   - Project Admin: manage members, policies, replication rules
#   - Maintainer: push/pull images, scan, sign, delete tags
#   - Developer: push/pull images only
#   - Guest: pull images only (read-only access)
#   - Limited Guest: pull specific images only

# Create a project with restricted access
curl -X POST "https://registry.company.com/api/v2.0/projects" \
    -H "Content-Type: application/json" \
    -d '{
        "project_name": "team-payments",
        "public": false,
        "metadata": {
            "auto_scan": "true",
            "prevent_vul": "true",
            "severity": "high"
        }
    }'

# Add a team member with Developer role
curl -X POST "https://registry.company.com/api/v2.0/projects/team-payments/members" \
    -H "Content-Type: application/json" \
    -d '{
        "role_id": 2,
        "member_user": {"username": "developer1"}
    }'
# role_id: 1=Admin, 2=Developer, 3=Guest, 4=Maintainer

# Robot accounts for CI/CD (no human identity)
curl -X POST "https://registry.company.com/api/v2.0/projects/team-payments/robots" \
    -H "Content-Type: application/json" \
    -d '{
        "name": "ci-pipeline",
        "duration": 365,
        "permissions": [
            {"kind": "project", "namespace": "team-payments",
             "access": [{"resource": "repository", "action": "push"},
                        {"resource": "repository", "action": "pull"}]}
        ]
    }'
# Returns: {"name": "robot$team-payments+ci-pipeline", "secret": "..."}

Policy Enforcement

Policy enforcement ensures that only approved, secure, compliant images run in production — regardless of what developers push. Policy engines intercept deployment requests and block violations:

# Kyverno Policy: Block images using :latest tag
# (Kyverno is a Kubernetes-native policy engine)
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: disallow-latest-tag
spec:
  validationFailureAction: Enforce    # Block (not just warn)
  rules:
    - name: require-image-tag
      match:
        any:
          - resources:
              kinds: ["Pod"]
      validate:
        message: "Images must use a specific tag, not :latest"
        pattern:
          spec:
            containers:
              - image: "!*:latest"

---
# Kyverno Policy: Only allow images from approved registries
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: restrict-image-registries
spec:
  validationFailureAction: Enforce
  rules:
    - name: validate-registries
      match:
        any:
          - resources:
              kinds: ["Pod"]
      validate:
        message: "Images must come from approved registries"
        pattern:
          spec:
            containers:
              - image: "registry.company.com/* | gcr.io/company-project/*"

---
# Kyverno Policy: Require image signatures (Cosign/Sigstore)
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: verify-image-signatures
spec:
  validationFailureAction: Enforce
  rules:
    - name: check-signature
      match:
        any:
          - resources:
              kinds: ["Pod"]
      verifyImages:
        - imageReferences: ["registry.company.com/*"]
          attestors:
            - entries:
                - keys:
                    publicKeys: |
                      -----BEGIN PUBLIC KEY-----
                      MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE...
                      -----END PUBLIC KEY-----

# OPA/Gatekeeper alternative (Open Policy Agent)
# Constraint Template: Block images without vulnerability scan
cat <<EOF | kubectl apply -f -
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: requiredlabels
spec:
  crd:
    spec:
      names:
        kind: RequiredImageScan
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package requiredimagescan
        violation[{"msg": msg}] {
          container := input.review.object.spec.containers[_]
          not has_scan_annotation(container.image)
          msg := sprintf("Image %v has not been scanned", [container.image])
        }
EOF

# Connaisseur: Admission controller for image signature verification
# Blocks any unsigned image from being deployed
# Supports Cosign, Notary v1/v2, and custom validators

Compliance & Audit

Container platforms must satisfy regulatory frameworks. Here's how container-specific controls map to compliance requirements:

Multi-Architecture Containers

Enterprise workloads increasingly span multiple CPU architectures: amd64 (traditional servers), arm64 (AWS Graviton, Apple Silicon, edge/IoT), and sometimes s390x (IBM mainframes) or ppc64le (IBM Power). Multi-arch images let you deploy one tag that works everywhere:

# A multi-arch image uses a "manifest list" (OCI image index)
# that maps each architecture to a specific image digest

# Inspect a multi-arch image to see supported platforms
docker manifest inspect nginx:1.25-alpine
# {
#   "schemaVersion": 2,
#   "mediaType": "application/vnd.oci.image.index.v1+json",
#   "manifests": [
#     {
#       "digest": "sha256:aaa...",
#       "platform": {"architecture": "amd64", "os": "linux"}
#     },
#     {
#       "digest": "sha256:bbb...",
#       "platform": {"architecture": "arm64", "os": "linux"}
#     },
#     {
#       "digest": "sha256:ccc...",
#       "platform": {"architecture": "s390x", "os": "linux"}
#     }
#   ]
# }

# When you `docker pull nginx:1.25-alpine`:
# - On amd64 machine → pulls sha256:aaa...
# - On arm64 machine → pulls sha256:bbb...
# - Completely transparent to the user

# Check what architecture your current images support
docker buildx imagetools inspect myapp:1.0
# Name: registry.company.com/myapp:1.0
# MediaType: application/vnd.oci.image.index.v1+json
# Manifests:
#   Name: registry.company.com/myapp:1.0@sha256:111...
#   Platform: linux/amd64
#
#   Name: registry.company.com/myapp:1.0@sha256:222...
#   Platform: linux/arm64

Buildx for Multi-Platform Builds

Docker Buildx extends the Docker build command with multi-platform build capabilities. It can produce images for architectures different from your host machine using QEMU emulation or remote native builders:

# Create a multi-platform builder instance
docker buildx create --name multiarch-builder \
    --driver docker-container \
    --bootstrap \
    --use

# Verify builder supports multiple platforms
docker buildx inspect multiarch-builder
# Platforms: linux/amd64, linux/amd64/v2, linux/amd64/v3,
#            linux/arm64, linux/riscv64, linux/ppc64le,
#            linux/s390x, linux/386, linux/arm/v7, linux/arm/v6

# Build for multiple platforms simultaneously
docker buildx build \
    --platform linux/amd64,linux/arm64 \
    --tag registry.company.com/myapp:1.0 \
    --push \
    .
# This builds the Dockerfile TWICE:
# 1. Once targeting linux/amd64 (native or QEMU)
# 2. Once targeting linux/arm64 (native or QEMU)
# Then creates a manifest list pointing to both and pushes everything

# Build with platform-specific optimizations in Dockerfile
# Using build arguments to select architecture-specific dependencies

# Multi-platform Dockerfile with conditional logic
FROM --platform=$BUILDPLATFORM golang:1.22-alpine AS builder

# BUILDPLATFORM = host architecture (where build runs)
# TARGETPLATFORM = target architecture (what we're building for)
# TARGETOS, TARGETARCH = components (e.g., "linux", "arm64")

ARG TARGETOS TARGETARCH

WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download

COPY . .

# Cross-compile: build binary for target architecture
RUN CGO_ENABLED=0 GOOS=${TARGETOS} GOARCH=${TARGETARCH} \
    go build -ldflags="-s -w" -o /app/server ./cmd/server

# Runtime stage — uses target platform's base image automatically
FROM alpine:3.19
COPY --from=builder /app/server /usr/local/bin/server
ENTRYPOINT ["/usr/local/bin/server"]

# Strategy comparison for multi-platform builds:

# 1. QEMU emulation (simplest, slowest)
# Builds run on host CPU, instructions translated via QEMU
# 10-30x slower than native builds
docker buildx build --platform linux/arm64 .  # On amd64 host

# 2. Cross-compilation (fastest for compiled languages)
# Compiler generates target-architecture binary on host
# Near-native speed (Go, Rust, C with cross-compiler)
# Requires language support (CGO_ENABLED=0 for Go)

# 3. Native remote builders (fastest, most complex)
# Build dispatched to actual arm64 machine via BuildKit
docker buildx create --name arm-builder \
    --node arm-node \
    --platform linux/arm64 \
    ssh://user@arm64-build-server

docker buildx create --name arm-builder \
    --append \
    --node amd-node \
    --platform linux/amd64 \
    ssh://user@amd64-build-server

# Now builds run on native hardware for each architecture

Enterprise Build Pipelines

In enterprise environments, images flow through a governed pipeline: build → scan → sign → test → promote. No image reaches production without passing every gate:

flowchart LR
    subgraph Build["Build Stage"]
        SRC["Source Code
(Git)"] --> CI["CI Build
(BuildKit)"]
        CI --> IMG["Image
:git-sha"]
    end
    subgraph Security["Security Gate"]
        IMG --> SCAN["Vuln Scan
(Trivy)"]
        SCAN -->|"No Critical"| SIGN["Sign
(Cosign)"]
        SCAN -->|"Critical CVE"| BLOCK["❌ Blocked"]
    end
    subgraph Promotion["Promotion Pipeline"]
        SIGN --> DEV["Dev Registry
:sha-abc123"]
        DEV -->|"Tests pass"| STAGE["Staging Registry
:rc-1.0.0"]
        STAGE -->|"Approval"| PROD["Prod Registry
:1.0.0"]
    end

    style Build fill:#f8f9fa,stroke:#132440
    style Security fill:#fff5f5,stroke:#BF092F
    style Promotion fill:#f0f9f9,stroke:#3B9797
    style BLOCK fill:#BF092F,stroke:#BF092F,color:#fff
                            

# GitHub Actions: Complete enterprise container pipeline
name: Container Pipeline
on:
  push:
    branches: [main]

env:
  REGISTRY: registry.company.com
  IMAGE_NAME: team-backend/myapp

jobs:
  build-scan-push:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write    # For Cosign keyless signing

    steps:
      - uses: actions/checkout@v4

      # Multi-platform build
      - uses: docker/setup-buildx-action@v3
      - uses: docker/setup-qemu-action@v3

      - uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ secrets.REGISTRY_USER }}
          password: ${{ secrets.REGISTRY_PASSWORD }}

      # Build and push multi-arch image
      - uses: docker/build-push-action@v5
        id: build
        with:
          push: true
          platforms: linux/amd64,linux/arm64
          tags: |
            ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
            ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max

      # Vulnerability scan (fail on critical)
      - uses: aquasecurity/trivy-action@master
        with:
          image-ref: "${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}"
          severity: "CRITICAL,HIGH"
          exit-code: "1"    # Fail pipeline on findings

      # Sign image with Cosign (keyless, OIDC-based)
      - uses: sigstore/cosign-installer@v3
      - run: |
          cosign sign --yes \
            ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${{ steps.build.outputs.digest }}

      # Generate and attach SBOM
      - run: |
          syft ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }} -o spdx-json > sbom.json
          cosign attach sbom --sbom sbom.json \
            ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${{ steps.build.outputs.digest }}

Container Platform Comparison

Exercises

Series Conclusion: The Complete Container Mental Model

Over 22 articles, we've journeyed from "what is a container?" to enterprise-scale container platforms. Let's consolidate the mental model that ties everything together:

What to Learn Next

With container fundamentals mastered, your natural next step is Kubernetes — the industry-standard container orchestrator. The knowledge from this series transfers directly:

• Namespaces and cgroups → Kubernetes resource requests and limits
• Docker networking → Kubernetes Services, Ingress, NetworkPolicy
• Docker volumes → Kubernetes PersistentVolumes and StorageClasses
• Dockerfile best practices → Pod Security Standards
• Docker Compose → Kubernetes manifests (Deployments, Services, ConfigMaps)
• Container monitoring → Kubernetes observability (Prometheus Operator, Grafana, OpenTelemetry)
• Image security → Kubernetes admission controllers and policy engines

You don't need to forget Docker to learn Kubernetes — Kubernetes uses everything we covered. containerd runs inside every Kubernetes node. OCI images are what Kubernetes deploys. The networking concepts, the security model, the troubleshooting techniques — all carry forward.