Kustomize Track Part 1: Bases & Overlays

Kustomize vs Helm

Design Philosophy

Helm uses a templating language (Go templates inside YAML files) to produce Kubernetes manifests from parameterised charts. Kustomize takes a different approach: your base manifests are valid Kubernetes YAML with no placeholders. Kustomize transforms them by applying patches, label additions, name prefixes, and image tag overrides — composable transformations rather than substitutions.

• Helm: Chart → values.yaml → rendered YAML → applied to cluster. Powerful packaging and release management, but {{ everywhere in templates makes diffs hard to read.
• Kustomize: base YAML → overlay patches → merged YAML → applied to cluster. Every file is valid YAML, diffs are easy to read, and you need zero tooling besides kubectl.

When to Use Each

• Use Kustomize when: You maintain your own application manifests and want simple per-environment differences (namespace, replicas, resource limits). Works great with GitOps tools (Flux, Argo CD) that have native Kustomize support.
• Use Helm when: You're packaging software for distribution or reuse, need complex conditional logic, or are consuming off-the-shelf charts (Prometheus, cert-manager, nginx-ingress) from public chart repositories.
• Use both: Helm for third-party dependencies; Kustomize for your own application manifests. Flux's HelmRelease handles Helm; Flux's Kustomization handles your app overlays.

kustomization.yaml Anatomy

# kustomization.yaml — the entry point for every Kustomize directory
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

# ── RESOURCES: what to include ──────────────────────────────────
resources:
  - deployment.yaml             # local file
  - service.yaml
  - ingress.yaml
  - ../other-dir/configmap.yaml # relative path to another directory
  # Can also reference other kustomization directories (used in overlays):
  # - ../../base

# ── NAMESPACE: set namespace on all resources ────────────────────
namespace: grade-api-dev

# ── NAME PREFIX / SUFFIX ─────────────────────────────────────────
namePrefix: dev-     # adds 'dev-' prefix to all resource names
# nameSuffix: -v1

# ── COMMON LABELS: add to all resources ──────────────────────────
commonLabels:
  app.kubernetes.io/name: grade-api
  app.kubernetes.io/managed-by: kustomize
  environment: dev

# ── COMMON ANNOTATIONS ───────────────────────────────────────────
commonAnnotations:
  config.kubernetes.io/origin: |
    path: k8s/overlays/dev/kustomization.yaml

# ── IMAGES: override image tags without editing manifests ─────────
images:
  - name: myapp                 # matches image: myapp in any manifest
    newTag: v1.2.3              # override the tag
    # newName: gcr.io/myproject/myapp  # also rename the image

# ── REPLICAS (Kustomize 5.0+) ─────────────────────────────────────
replicas:
  - name: grade-api             # matches Deployment/StatefulSet name
    count: 2

# ── GENERATOR OPTIONS ─────────────────────────────────────────────
generatorOptions:
  disableNameSuffixHash: false  # true disables hash suffix on generated CM/Secrets
  labels:
    generator: kustomize

resources

The resources list tells Kustomize which Kubernetes manifests to include. Files can be in the same directory, relative paths to other directories, or references to other kustomization.yaml directories (which Kustomize processes recursively). Overlays reference their base this way.

namespace & namePrefix

# Without namePrefix, the base Deployment is named 'grade-api'.
# With namePrefix: dev-, it becomes 'dev-grade-api'.
# This allows multiple overlays to coexist in the same namespace.

# Common pattern: one cluster, multiple teams, one namespace per overlay
# base:         name: grade-api
# overlays/dev: namePrefix: dev-    → dev-grade-api
# overlays/stg: namePrefix: stg-    → stg-grade-api

# Or use distinct namespaces per overlay without a prefix:
# overlays/dev: namespace: grade-api-dev
# overlays/stg: namespace: grade-api-staging
# overlays/prd: namespace: grade-api-production

images: Overriding Tags

# In the base deployment.yaml:
# image: myapp   (no tag — Kustomize will supply it)
# OR
# image: myapp:latest  (tag will be overridden)

# In the overlay kustomization.yaml:
images:
  - name: myapp
    newTag: "{{ .Values.imageTag }}"   # For Helm-wrapped Kustomize (unusual)

# More commonly, set the actual tag:
images:
  - name: myapp
    newTag: 1.2.3-alpine

  # Also rename + re-tag (for pushing to a private registry):
  - name: myapp
    newName: gcr.io/myproject/myapp
    newTag: sha256-abc123

  # Digest pinning (most secure — immutable):
  - name: myapp
    newName: gcr.io/myproject/myapp
    digest: sha256:abc123def456...

Bases & Overlays

The canonical Kustomize directory structure separates shared base manifests from environment-specific overlays. The base contains your standard Kubernetes YAML. Each overlay references the base and applies only the differences needed for that environment.

# Directory structure
k8s/
├── base/
│   ├── kustomization.yaml       # references deployment.yaml, service.yaml, ingress.yaml
│   ├── deployment.yaml
│   ├── service.yaml
│   └── ingress.yaml
└── overlays/
    ├── dev/
    │   └── kustomization.yaml   # references ../../base, sets namespace/replicas/tag
    ├── staging/
    │   ├── kustomization.yaml
    │   └── resource-limits.yaml # env-specific patches
    └── prod/
        ├── kustomization.yaml
        ├── hpa.yaml              # only prod has HPA
        └── resource-limits.yaml

Base Directory

# k8s/base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - deployment.yaml
  - service.yaml
  - ingress.yaml

# Minimal labels applied to all environments
commonLabels:
  app.kubernetes.io/name: grade-api
  app.kubernetes.io/part-of: grade-platform

# k8s/base/deployment.yaml — no environment-specific values
apiVersion: apps/v1
kind: Deployment
metadata:
  name: grade-api              # will be prefixed/namespaced by overlays
  labels:
    app: grade-api
spec:
  replicas: 1                  # will be overridden by overlays
  selector:
    matchLabels:
      app: grade-api
  template:
    metadata:
      labels:
        app: grade-api
    spec:
      containers:
        - name: grade-api
          image: myapp          # tag will be set by overlays
          ports:
            - containerPort: 8080
          # No resource limits in base — each overlay defines what's appropriate

Overlays

# k8s/overlays/dev/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

# Reference the base
resources:
  - ../../base

# Dev-specific settings
namespace: grade-api-dev
namePrefix: dev-

images:
  - name: myapp
    newTag: dev-latest

# Override replica count
replicas:
  - name: grade-api
    count: 1

commonLabels:
  environment: dev

# k8s/overlays/staging/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - ../../base
  - resource-limits.yaml       # additional manifest only for staging

namespace: grade-api-staging

images:
  - name: myapp
    newTag: "1.2.0"

replicas:
  - name: grade-api
    count: 2

commonLabels:
  environment: staging

# k8s/overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - ../../base
  - resource-limits.yaml
  - hpa.yaml                   # HPA only in production

namespace: grade-api-production

images:
  - name: myapp
    newName: gcr.io/myproject/myapp
    newTag: "1.2.0"

replicas:
  - name: grade-api
    count: 5

commonLabels:
  environment: production

Building & Applying

# Preview the rendered output for an overlay (does NOT apply to cluster)
kubectl kustomize k8s/overlays/dev

# Build using the standalone kustomize CLI (same as kubectl kustomize)
kustomize build k8s/overlays/dev

# Build and pipe to kubectl apply
kustomize build k8s/overlays/prod | kubectl apply -f -

# Apply directly (kubectl apply -k is equivalent to kustomize build | kubectl apply)
kubectl apply -k k8s/overlays/dev

# Dry-run to see what would change without applying
kubectl apply -k k8s/overlays/dev --dry-run=client

# Server-side dry run (validates against admission webhooks)
kubectl apply -k k8s/overlays/prod --dry-run=server

# Diff: shows what would change in the cluster
kubectl diff -k k8s/overlays/staging
# --- current in cluster
# +++ proposed from kustomize build

# Delete all resources in an overlay
kubectl delete -k k8s/overlays/dev

# Apply with timeout and wait for rollout
kubectl apply -k k8s/overlays/prod
kubectl rollout status deployment/grade-api -n grade-api-production --timeout=120s

# Get which Kustomize version is bundled in kubectl
kubectl version --client -o json | jq '.kustomizeVersion'
# "v5.0.3"

# Install standalone kustomize CLI for advanced use
brew install kustomize
kustomize version
# {Version:kustomize/v5.4.1 ...}

CommonLabels & Annotations

# commonLabels are added to:
# - metadata.labels on all resources
# - spec.selector.matchLabels on Deployments/StatefulSets (if not already set)
# - spec.template.metadata.labels on pod templates
# WARNING: Changing commonLabels on an existing Deployment causes a selector
# conflict. Prefer commonAnnotations for metadata you might change later.

# Best practice: set selectors explicitly in base, use commonLabels only for
# non-selector labels
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - ../../base

# Safe labels (not used as selectors)
commonLabels:
  environment: staging
  team: platform
  cost-center: "1234"

# Annotations (never used as selectors, always safe to add/change)
commonAnnotations:
  deployment.kubernetes.io/revision: "42"
  alerting.company.com/contact: "platform-team@company.com"

Image Tags

# Update image tags per overlay without editing base manifests.
# This is the canonical approach for CD pipelines:
# 1. CI builds image and pushes with the git SHA tag
# 2. CD updates the overlay kustomization.yaml with the new tag
# 3. GitOps tool (Flux/Argo CD) detects the change and reconciles

# ── In overlay kustomization.yaml ───────────────────────────────
images:
  - name: myapp
    newTag: "abc123de"        # git SHA from CI pipeline

  # Multiple images in one app
  - name: sidecar
    newTag: "1.5.0"

  # Rename AND retag (for private registries)
  - name: myapp
    newName: private-registry.company.com/myapp
    newTag: "abc123de"

# ── Updating from CI pipeline ────────────────────────────────────
# Using kustomize CLI to update the tag programmatically (no YAML editing):
kustomize edit set image myapp:$NEW_TAG
# This modifies the kustomization.yaml in-place, then you git commit + push

# Automate tag updates in CI (script approach)
NEW_TAG=$(git rev-parse --short HEAD)

cd k8s/overlays/staging
kustomize edit set image myapp=myapp:$NEW_TAG

# Commit the updated kustomization.yaml back to git
git add kustomization.yaml
git commit -m "ci: update grade-api image to $NEW_TAG"
git push

# Flux or Argo CD detects the commit and applies the new tag to the cluster

Exercises

Key Takeaways