Helm Track Part 4: Production Patterns

Chart Unit Testing

helm-unittest Plugin

Unit tests for Helm charts test the template rendering output without deploying to a cluster. The helm-unittest plugin lets you write YAML-based assertions against generated manifests:

# Install the plugin
helm plugin install https://github.com/helm-unittest/helm-unittest

# Run all tests in a chart
helm unittest ./grade-api

# Run with verbose output
helm unittest ./grade-api --strict --debug

# Watch mode (auto-reruns on file change)
helm unittest ./grade-api --update-snapshot

Writing Tests

Test files go in grade-api/tests/ and follow a YAML-based assertion format:

# grade-api/tests/deployment_test.yaml
suite: "Deployment template tests"
templates:
  - templates/deployment.yaml
  - templates/configmap.yaml

tests:
  - it: "should render correct replica count from values"
    set:
      replicaCount: 3
    asserts:
      - equal:
          path: spec.replicas
          value: 3

  - it: "should use chart appVersion as default image tag"
    asserts:
      - matchRegex:
          path: spec.template.spec.containers[0].image
          pattern: "ghcr.io/example/grade-api:.*"

  - it: "should mount the configmap env var for DB_HOST"
    asserts:
      - contains:
          path: spec.template.spec.containers[0].env
          content:
            name: DB_HOST
            valueFrom:
              configMapKeyRef:
                name: RELEASE-NAME-grade-api-config
                key: db.host

  - it: "should NOT create HPA when autoscaling disabled"
    templates:
      - templates/hpa.yaml
    set:
      autoscaling.enabled: false
    asserts:
      - hasDocuments:
          count: 0

  - it: "should create HPA with correct maxReplicas when enabled"
    templates:
      - templates/hpa.yaml
    set:
      autoscaling.enabled: true
      autoscaling.maxReplicas: 15
    asserts:
      - equal:
          path: spec.maxReplicas
          value: 15

# grade-api/tests/ingress_test.yaml
suite: "Ingress template tests"
templates:
  - templates/ingress.yaml

tests:
  - it: "should not render ingress when disabled"
    set:
      ingress.enabled: false
    asserts:
      - hasDocuments:
          count: 0

  - it: "should render ingress with correct host when enabled"
    set:
      ingress.enabled: true
      ingress.host: "grades.example.com"
      ingress.className: nginx
    asserts:
      - equal:
          path: spec.rules[0].host
          value: "grades.example.com"
      - equal:
          path: spec.ingressClassName
          value: nginx

Running in CI

# Full quality gate — run in CI before every merge
# 1. Lint
helm lint ./grade-api

# 2. Unit tests
helm unittest ./grade-api --strict

# 3. Template render check (no cluster needed)
helm template grade-api ./grade-api \
  --values ./grade-api/values.yaml \
  > /dev/null

# 4. All three in one job (fail fast)
helm lint ./grade-api && \
helm unittest ./grade-api --strict && \
echo "All chart quality checks passed"

OCI Registry Publishing

OCI (Open Container Initiative) registries store Helm charts alongside container images. This is now the preferred distribution method over HTTP chart repos — charts are versioned, content-addressable, and accessible via standard container tooling:

helm push to OCI

# Package the chart into a .tgz
helm package ./grade-api
# Successfully packaged chart and saved it to: grade-api-0.1.0.tgz

# Login to OCI registry (GitHub Container Registry example)
echo $CR_PAT | helm registry login ghcr.io -u USERNAME --password-stdin

# Push to OCI registry
# Format: oci://REGISTRY/NAMESPACE
helm push grade-api-0.1.0.tgz oci://ghcr.io/your-org

# Result:
# Pushed: ghcr.io/your-org/grade-api:0.1.0
# Digest: sha256:abc123...

# Show what's in the registry
helm show chart oci://ghcr.io/your-org/grade-api --version 0.1.0

Installing from OCI

# Install directly from OCI — no 'helm repo add' needed
helm install grade-api oci://ghcr.io/your-org/grade-api \
  --version 0.1.0 \
  --namespace production \
  --values values-prod.yaml \
  --wait

# Upgrade from OCI
helm upgrade grade-api oci://ghcr.io/your-org/grade-api \
  --version 0.2.0 \
  --namespace production \
  --values values-prod.yaml \
  --atomic

# Pull without installing (inspect first)
helm pull oci://ghcr.io/your-org/grade-api --version 0.1.0

Cloud Registry Authentication

# AWS ECR
aws ecr get-login-password --region us-east-1 | \
  helm registry login \
  --username AWS \
  --password-stdin \
  123456789.dkr.ecr.us-east-1.amazonaws.com

helm push grade-api-0.1.0.tgz \
  oci://123456789.dkr.ecr.us-east-1.amazonaws.com/charts

# Azure Container Registry
az acr login --name myregistry
helm push grade-api-0.1.0.tgz oci://myregistry.azurecr.io/charts

# Google Artifact Registry
gcloud auth print-access-token | \
  helm registry login \
  --username oauth2accesstoken \
  --password-stdin \
  us-docker.pkg.dev

helm push grade-api-0.1.0.tgz \
  oci://us-docker.pkg.dev/my-project/helm-charts

Library Charts

A library chart (type: library in Chart.yaml) contains only named templates — it produces no Kubernetes manifests itself. Platform teams use library charts to share common template helpers across all their application charts:

Creating a Library Chart

# platform-helpers/Chart.yaml
apiVersion: v2
name: platform-helpers
description: Shared template helpers for all platform charts
type: library    # CRITICAL — makes this a library, not an application
version: 1.0.0

# platform-helpers/templates/_standard-labels.tpl

{{/*
Standard labels applied to all resources — enforced by platform team.
Adds team, cost-center, and compliance labels from global values.
*/}}
{{- define "platform.standardLabels" -}}
helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version }}
app.kubernetes.io/name: {{ .Chart.Name }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
app.kubernetes.io/managed-by: Helm
team: {{ required "global.team is required" .Values.global.team | quote }}
cost-center: {{ .Values.global.costCenter | default "unassigned" | quote }}
{{- end }}

{{/*
Standard resource limits validation — fails if limits > platform maximums
*/}}
{{- define "platform.validateResources" -}}
{{- $maxCPU := 4.0 -}}
{{- $maxMem := "4Gi" -}}
{{- if .Values.resources.limits.cpu -}}
{{- /* In a real scenario you'd parse and compare these */ -}}
{{- end -}}
{{- end }}

Consuming a Library Chart

# grade-api/Chart.yaml — declare library as dependency
dependencies:
  - name: platform-helpers
    version: "1.0.0"
    repository: oci://ghcr.io/your-org

# grade-api/values.yaml — library chart requires global values
global:
  team: "platform"
  costCenter: "engineering"

# grade-api/templates/deployment.yaml — use library template
metadata:
  labels:
    {{- include "platform.standardLabels" . | nindent 4 }}

Chart Signing & Provenance

Helm supports GPG-based chart signing for supply chain security — verifying that a chart hasn't been tampered with and came from a trusted source:

# Generate a GPG key (if you don't have one)
gpg --gen-key

# Export your key ID
gpg --list-secret-keys --keyid-format=long

# Package and sign simultaneously
helm package ./grade-api --sign \
  --key "Platform Team" \
  --keyring ~/.gnupg/secring.gpg

# Creates:
# grade-api-0.1.0.tgz
# grade-api-0.1.0.tgz.prov   ← provenance file

# Verify before installing
helm verify grade-api-0.1.0.tgz

# Install with verification
helm install grade-api grade-api-0.1.0.tgz \
  --verify \
  --keyring ./platform-team-pubkey.gpg

# In GitOps workflows, Flux and Argo CD can verify chart signatures
# automatically using their own signature verification mechanisms

CI/CD Pipeline Integration

GitHub Actions Workflow

A complete chart CI/CD pipeline: test on every PR, build and publish on every main branch merge:

# .github/workflows/helm-chart-ci.yaml
name: Helm Chart CI/CD

on:
  push:
    paths:
      - 'charts/**'
    branches: [main]
  pull_request:
    paths:
      - 'charts/**'

jobs:
  lint-and-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install Helm
        uses: azure/setup-helm@v4
        with:
          version: v3.15.3

      - name: Install helm-unittest plugin
        run: helm plugin install https://github.com/helm-unittest/helm-unittest

      - name: Add chart repositories
        run: |
          helm repo add bitnami https://charts.bitnami.com/bitnami
          helm repo update

      - name: Resolve dependencies
        run: helm dependency build ./charts/grade-api

      - name: Lint charts
        run: helm lint ./charts/grade-api --strict

      - name: Run unit tests
        run: helm unittest ./charts/grade-api --strict

      - name: Render templates (dry-run check)
        run: |
          helm template grade-api ./charts/grade-api \
            --values ./charts/grade-api/values.yaml \
            > /dev/null

  publish:
    needs: lint-and-test
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    permissions:
      contents: read
      packages: write
    steps:
      - uses: actions/checkout@v4

      - name: Install Helm
        uses: azure/setup-helm@v4
        with:
          version: v3.15.3

      - name: Log in to GHCR
        run: |
          echo "${{ secrets.GITHUB_TOKEN }}" | \
            helm registry login ghcr.io \
            --username ${{ github.actor }} \
            --password-stdin

      - name: Package chart
        run: helm package ./charts/grade-api

      - name: Push to OCI registry
        run: |
          helm push grade-api-*.tgz \
            oci://ghcr.io/${{ github.repository_owner }}/charts

chart-releaser-action (GitHub Pages repo)

# Alternative: publish to GitHub Pages as HTTP chart repo
# Uses helm/chart-releaser-action

name: Release Charts

on:
  push:
    branches: [main]

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Configure Git
        run: |
          git config user.name "$GITHUB_ACTOR"
          git config user.email "$GITHUB_ACTOR@users.noreply.github.com"

      - uses: azure/setup-helm@v4
        with:
          version: v3.15.3

      - uses: helm/chart-releaser-action@v1.6.0
        env:
          CR_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
        with:
          charts_dir: charts
          # Creates GitHub Releases for changed chart versions
          # Pushes tarballs to gh-pages branch
          # Users can then: helm repo add your-charts https://org.github.io/repo

Monorepo Chart Organisation

Platform teams typically manage dozens of charts in a single Git repository. A standard layout:

helm-charts/                    # Git repository root
├── charts/                     # Application charts
│   ├── grade-api/
│   │   ├── Chart.yaml
│   │   ├── values.yaml
│   │   ├── templates/
│   │   └── tests/
│   ├── notification-service/
│   └── report-generator/
├── library/                    # Library charts (shared helpers)
│   └── platform-helpers/
│       ├── Chart.yaml          # type: library
│       └── templates/
│           └── _helpers.tpl
├── environments/               # Environment-specific values
│   ├── dev/
│   │   ├── grade-api.yaml      # dev overrides
│   │   └── notification-service.yaml
│   ├── staging/
│   └── production/
├── .github/
│   └── workflows/
│       ├── lint-test.yaml      # PR checks
│       └── release.yaml        # publish on merge
└── README.md

# CI: detect which charts changed (only test affected charts)
CHANGED_CHARTS=$(git diff --name-only HEAD~1 HEAD \
  | grep '^charts/' \
  | cut -d'/' -f1-2 \
  | sort -u)

for CHART in $CHANGED_CHARTS; do
  echo "Testing $CHART..."
  helm lint "./$CHART"
  helm unittest "./$CHART" --strict
done

helm diff & Safe Upgrades

The helm diff plugin is the single most important safety tool for Helm upgrades — it shows exactly what will change before you apply it:

# Install helm diff plugin
helm plugin install https://github.com/databus23/helm-diff

# See what would change in an upgrade (without applying)
helm diff upgrade grade-api ./grade-api \
  --namespace production \
  --values values-prod.yaml \
  --values values-prod-v2.yaml

# Output (color-coded diff):
# REVISION: 3 → 4
# --- Deployment/grade-api
# @@ -15,7 +15,7 @@
#          image: "ghcr.io/example/grade-api:1.1.0"
# -        image: "ghcr.io/example/grade-api:1.0.0"

# Compare two revisions already deployed
helm diff revision grade-api 2 3 -n production

# Diff against a specific values file
helm diff upgrade grade-api oci://ghcr.io/your-org/grade-api \
  --version 0.2.0 \
  --namespace production \
  --values values-prod.yaml

Exercises

Track Summary & What's Next