Back to Software Engineering & Delivery Mastery Series GitHub Actions Bootcamp

Module 6: Advanced Workflow Patterns

June 2, 2026 Wasil Zafar 30 min read

Master the patterns that separate junior from senior CI/CD engineers — reusable workflows with typed inputs/outputs, nested workflow composition, concurrency control for safe deployments, and dynamic matrices generated from step outputs.

Table of Contents

  1. Reusable Workflows
  2. Inputs and Outputs
  3. Nesting Reusable Workflows
  4. Workflow Call vs Dispatch
  5. Concurrency Management
  6. Dynamic Matrices from Outputs
  7. Exercises

Reusable Workflows — DRY Principles for CI/CD

Reusable workflows solve the copy-paste problem across repositories. Instead of duplicating 200-line deployment workflows in 50 repos, you define the workflow once and call it from any other workflow using the workflow_call trigger. This is GitHub's native answer to shared pipeline libraries (like Jenkins Shared Libraries or Azure DevOps Templates).

A reusable workflow is simply a standard workflow file that includes on: workflow_call as one of its triggers. It lives in .github/workflows/ in any repository (the same repo or a different one) and can define inputs, outputs, and secrets that callers must provide.

Reusable Workflow Call Hierarchy
flowchart TD
    A[Caller Workflow
repo-a/.github/workflows/ci.yml] -->|uses| B[Reusable: Build & Test
shared-org/.github/workflows/build.yml] A -->|uses| C[Reusable: Deploy
shared-org/.github/workflows/deploy.yml] B -->|uses| D[Reusable: Lint
shared-org/.github/workflows/lint.yml] C -->|uses| E[Reusable: Notify
shared-org/.github/workflows/notify.yml] style A fill:#132440,color:#fff style B fill:#3B9797,color:#fff style C fill:#3B9797,color:#fff style D fill:#16476A,color:#fff style E fill:#16476A,color:#fff

Creating and Calling Reusable Workflows

The callee (reusable workflow) declares on: workflow_call and optionally defines inputs, outputs, and secrets. The caller uses the uses: keyword at the job level (not step level) to invoke it.

# CALLEE: .github/workflows/reusable-build.yml
# This workflow can be called by other workflows
name: Reusable Build Pipeline

on:
  workflow_call:
    inputs:
      node-version:
        description: 'Node.js version to use'
        required: true
        type: string
      run-tests:
        description: 'Whether to run tests after build'
        required: false
        type: boolean
        default: true
      environment:
        description: 'Target environment'
        required: false
        type: string
        default: 'staging'

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: ${{ inputs.node-version }}
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Build for ${{ inputs.environment }}
        run: npm run build
        env:
          NODE_ENV: ${{ inputs.environment == 'production' && 'production' || 'development' }}

      - name: Run tests
        if: inputs.run-tests
        run: npm test

      - name: Upload build output
        uses: actions/upload-artifact@v4
        with:
          name: build-${{ inputs.environment }}
          path: dist/
          retention-days: 5
# CALLER: .github/workflows/ci.yml
# This workflow calls the reusable build workflow
name: CI Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  build-staging:
    uses: my-org/shared-workflows/.github/workflows/reusable-build.yml@main
    with:
      node-version: '20'
      run-tests: true
      environment: 'staging'

  build-production:
    needs: build-staging
    if: github.ref == 'refs/heads/main'
    uses: my-org/shared-workflows/.github/workflows/reusable-build.yml@main
    with:
      node-version: '20'
      run-tests: false
      environment: 'production'

Key rules for calling reusable workflows:

  • uses: at job level — Not inside steps:. The entire job is replaced by the reusable workflow's jobs.
  • Reference format{owner}/{repo}/.github/workflows/{filename}@{ref} for cross-repo, or ./.github/workflows/{filename} for same-repo.
  • Ref pinning — Always pin to a SHA, tag, or branch. Use SHA for production: @a1b2c3d4
  • Cannot add steps — A calling job that uses uses: cannot also define steps:, runs-on:, or container:.

When to Use Reusable Workflows vs Composite Actions

Reusable Workflows vs Composite Actions — Decision Guide:

Use reusable workflows when you need to share entire job definitions (including runner selection, services, environment, concurrency), when you need multiple jobs with dependencies, or when the shared logic requires secrets access with explicit declarations.

Use composite actions when you need to share a sequence of steps within a single job, when the caller needs to add more steps before/after, or when you want maximum flexibility in how the shared logic integrates into existing jobs.

Rule of thumb: If you're sharing a complete pipeline stage (build, test, deploy), use a reusable workflow. If you're sharing a task (setup Node, run linting, send notification), use a composite action.

Reusable Workflow Inputs and Outputs

Inputs provide parameterization — making workflows configurable without modification. Outputs allow workflows to return data to the caller for use in subsequent jobs. Secrets provide secure credential passing with two models: explicit declaration or blanket inheritance.

Typed Input Definitions

Reusable workflow inputs support four types: string, number, boolean, and choice (with options:). Type validation happens at call time — passing a string where a boolean is expected will fail the workflow.

# Callee with comprehensive input types
name: Reusable Deploy

on:
  workflow_call:
    inputs:
      environment:
        description: 'Deployment target'
        required: true
        type: string
      replicas:
        description: 'Number of replicas to deploy'
        required: false
        type: number
        default: 2
      dry-run:
        description: 'Simulate deployment without applying changes'
        required: false
        type: boolean
        default: false
      log-level:
        description: 'Logging verbosity'
        required: false
        type: string
        default: 'info'

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Deploy application
        run: |
          echo "Environment: ${{ inputs.environment }}"
          echo "Replicas: ${{ inputs.replicas }}"
          echo "Dry run: ${{ inputs.dry-run }}"
          echo "Log level: ${{ inputs.log-level }}"

          if [ "${{ inputs.dry-run }}" = "true" ]; then
            echo "DRY RUN — no changes applied"
            kubectl apply -f k8s/ --dry-run=client
          else
            kubectl apply -f k8s/
            kubectl scale deployment/app --replicas=${{ inputs.replicas }}
          fi

Workflow Outputs

Outputs let a reusable workflow return values to the caller. This enables chaining — a build workflow can output the image tag, which a deploy workflow then consumes. Outputs are defined at the workflow level and mapped from job outputs.

# Callee: Returns build metadata to the caller
name: Reusable Build with Outputs

on:
  workflow_call:
    inputs:
      image-name:
        required: true
        type: string
    outputs:
      image-tag:
        description: 'The Docker image tag that was built'
        value: ${{ jobs.build.outputs.tag }}
      image-digest:
        description: 'The image digest for immutable reference'
        value: ${{ jobs.build.outputs.digest }}
      build-timestamp:
        description: 'When the build completed'
        value: ${{ jobs.build.outputs.timestamp }}

jobs:
  build:
    runs-on: ubuntu-latest
    outputs:
      tag: ${{ steps.meta.outputs.tag }}
      digest: ${{ steps.push.outputs.digest }}
      timestamp: ${{ steps.meta.outputs.timestamp }}
    steps:
      - uses: actions/checkout@v4

      - name: Generate metadata
        id: meta
        run: |
          TAG="${{ inputs.image-name }}:${{ github.sha }}"
          echo "tag=${TAG}" >> $GITHUB_OUTPUT
          echo "timestamp=$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> $GITHUB_OUTPUT

      - name: Build and push
        id: push
        run: |
          docker build -t ${{ steps.meta.outputs.tag }} .
          docker push ${{ steps.meta.outputs.tag }}
          DIGEST=$(docker inspect --format='{{index .RepoDigests 0}}' ${{ steps.meta.outputs.tag }} | cut -d@ -f2)
          echo "digest=${DIGEST}" >> $GITHUB_OUTPUT
# Caller: Consumes outputs from the reusable workflow
name: Build and Deploy Pipeline

on:
  push:
    branches: [main]

jobs:
  build:
    uses: my-org/shared-workflows/.github/workflows/reusable-build.yml@v2
    with:
      image-name: ghcr.io/my-org/my-app

  deploy-staging:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Deploy image to staging
        run: |
          echo "Deploying ${{ needs.build.outputs.image-tag }}"
          echo "Digest: ${{ needs.build.outputs.image-digest }}"
          echo "Built at: ${{ needs.build.outputs.build-timestamp }}"
          helm upgrade my-app ./chart \
            --set image.tag=${{ needs.build.outputs.image-tag }} \
            --namespace staging

Secrets: Inherit vs Explicit

Reusable workflows cannot access the caller's secrets by default — they must be explicitly passed or inherited. Two approaches exist:

# Approach 1: Explicit secret declarations (recommended for shared repos)
# Callee declares required secrets
on:
  workflow_call:
    secrets:
      DEPLOY_TOKEN:
        description: 'Token for deploying to production'
        required: true
      SLACK_WEBHOOK:
        description: 'Slack notification URL'
        required: false

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Deploy
        run: ./deploy.sh
        env:
          TOKEN: ${{ secrets.DEPLOY_TOKEN }}

      - name: Notify
        if: secrets.SLACK_WEBHOOK != ''
        run: |
          curl -X POST ${{ secrets.SLACK_WEBHOOK }} \
            -d '{"text": "Deployment complete"}'
# Caller: Passes secrets explicitly
jobs:
  deploy:
    uses: my-org/shared-workflows/.github/workflows/deploy.yml@main
    with:
      environment: production
    secrets:
      DEPLOY_TOKEN: ${{ secrets.PROD_DEPLOY_TOKEN }}
      SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_URL }}
# Approach 2: Inherit all secrets (convenient for same-org workflows)
jobs:
  deploy:
    uses: my-org/shared-workflows/.github/workflows/deploy.yml@main
    with:
      environment: production
    secrets: inherit  # Passes ALL caller secrets to the callee
Security Warning — secrets: inherit: Using secrets: inherit passes all secrets from the caller to the callee. This is convenient but dangerous for cross-organization reusable workflows. If you call a third-party reusable workflow with secrets: inherit, that workflow has access to every secret in your repository. Always use explicit secret passing for workflows you don't fully control.

Nesting Reusable Workflows

Reusable workflows can call other reusable workflows, enabling composable pipeline architectures. This lets you build a library of small, focused workflows that compose into complex pipelines — similar to function composition in programming.

Nested Workflow Composition (Max 4 Levels)
flowchart TD
    L1[Level 1: CI Pipeline
Caller Workflow] -->|calls| L2A[Level 2: Build & Test
Reusable Workflow] L1 -->|calls| L2B[Level 2: Deploy
Reusable Workflow] L2A -->|calls| L3A[Level 3: Lint
Reusable Workflow] L2A -->|calls| L3B[Level 3: Unit Tests
Reusable Workflow] L2B -->|calls| L3C[Level 3: Provision Infra
Reusable Workflow] L3C -->|calls| L4[Level 4: Terraform Apply
Reusable Workflow] L4 -.-x|BLOCKED| L5[Level 5: ❌ Exceeds max depth] style L1 fill:#132440,color:#fff style L2A fill:#3B9797,color:#fff style L2B fill:#3B9797,color:#fff style L3A fill:#16476A,color:#fff style L3B fill:#16476A,color:#fff style L3C fill:#16476A,color:#fff style L4 fill:#BF092F,color:#fff style L5 fill:#666,color:#fff
# Level 2: Reusable workflow that calls another reusable workflow
name: Reusable Build and Test

on:
  workflow_call:
    inputs:
      node-version:
        required: true
        type: string
    outputs:
      coverage-percent:
        description: 'Test coverage percentage'
        value: ${{ jobs.test.outputs.coverage }}

jobs:
  lint:
    # Level 3: Calling another reusable workflow
    uses: my-org/shared-workflows/.github/workflows/lint.yml@main
    with:
      node-version: ${{ inputs.node-version }}

  test:
    needs: lint
    runs-on: ubuntu-latest
    outputs:
      coverage: ${{ steps.cov.outputs.percent }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ inputs.node-version }}
      - run: npm ci && npm test -- --coverage
      - name: Extract coverage
        id: cov
        run: |
          PERCENT=$(cat coverage/coverage-summary.json | jq '.total.lines.pct')
          echo "percent=${PERCENT}" >> $GITHUB_OUTPUT

Limitations and Workarounds

Nesting Limitations (Critical):
  • Maximum 4 levels deep — The top-level caller is level 1. You can nest up to 3 additional levels of reusable workflows. Exceeding this fails with a clear error.
  • Maximum 20 unique reusable workflows per workflow file — Hitting this limit requires refactoring into fewer, more capable workflows.
  • No circular references — Workflow A cannot call workflow B if B (directly or indirectly) calls A.
  • env context not inherited — Environment variables set in the caller's env: map are NOT available in the callee. Pass them as inputs instead.
  • Permissions not inherited — Each reusable workflow uses the permissions of the caller by default, but permissions: in the callee can only restrict (not expand) what the caller granted.
# Workaround: Flattening deep nesting with a facade workflow
# Instead of A -> B -> C -> D -> E (5 levels, BLOCKED),
# create a facade that orchestrates B, C, D, E as sibling jobs

name: Reusable Full Pipeline (Facade)

on:
  workflow_call:
    inputs:
      ref:
        required: true
        type: string
      environment:
        required: true
        type: string
    secrets:
      DEPLOY_TOKEN:
        required: true

jobs:
  lint:
    uses: my-org/shared-workflows/.github/workflows/lint.yml@main
    with:
      ref: ${{ inputs.ref }}

  build:
    needs: lint
    uses: my-org/shared-workflows/.github/workflows/build.yml@main
    with:
      ref: ${{ inputs.ref }}

  test:
    needs: build
    uses: my-org/shared-workflows/.github/workflows/test.yml@main
    with:
      ref: ${{ inputs.ref }}

  deploy:
    needs: test
    uses: my-org/shared-workflows/.github/workflows/deploy.yml@main
    with:
      ref: ${{ inputs.ref }}
      environment: ${{ inputs.environment }}
    secrets:
      DEPLOY_TOKEN: ${{ secrets.DEPLOY_TOKEN }}

Workflow Call vs Workflow Dispatch Events

Both workflow_call and workflow_dispatch accept inputs, but they serve fundamentally different purposes. Understanding when to use each is critical for designing maintainable CI/CD systems.

Feature workflow_call workflow_dispatch
Trigger source Another workflow (programmatic) GitHub UI, API, or CLI (manual/external)
Execution context Runs as part of the caller's workflow run Creates a new, independent workflow run
Appears in UI Nested inside the caller's run As its own separate run
Input types string, number, boolean string, boolean, choice, environment
Can return outputs Yes (via outputs:) No
Secrets Passed from caller or inherited Access own repo secrets directly
GITHUB_TOKEN scope Caller's token permissions Own repo's token permissions
Primary use case Code reuse, shared pipeline libraries On-demand ops, manual deploys, ad-hoc tasks
# A workflow can support BOTH triggers simultaneously
name: Deploy Application

on:
  # Can be called by other workflows (reusable)
  workflow_call:
    inputs:
      environment:
        required: true
        type: string
      version:
        required: true
        type: string
    secrets:
      DEPLOY_KEY:
        required: true

  # Can also be triggered manually via UI/API
  workflow_dispatch:
    inputs:
      environment:
        description: 'Target environment'
        required: true
        type: choice
        options:
          - staging
          - production
      version:
        description: 'Version to deploy (e.g., v1.2.3)'
        required: true
        type: string

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: ${{ inputs.environment }}
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ inputs.version }}

      - name: Deploy to ${{ inputs.environment }}
        run: |
          echo "Deploying version ${{ inputs.version }}"
          ./scripts/deploy.sh --env=${{ inputs.environment }}
        env:
          # secrets.DEPLOY_KEY works for workflow_call (passed by caller)
          # For workflow_dispatch, it reads from repo secrets directly
          DEPLOY_KEY: ${{ secrets.DEPLOY_KEY }}

Repository Dispatch for Cross-Repo Triggers

When you need to trigger workflows across repositories without the tight coupling of workflow_call, use repository_dispatch. This is an event-driven pattern where one repo sends an event and another repo reacts to it.

# Sender: Trigger deployment in another repo after release
name: Trigger Downstream Deploy

on:
  release:
    types: [published]

jobs:
  notify-deploy-repo:
    runs-on: ubuntu-latest
    steps:
      - name: Trigger deployment workflow
        uses: peter-evans/repository-dispatch@v3
        with:
          token: ${{ secrets.CROSS_REPO_PAT }}
          repository: my-org/deploy-infra
          event-type: app-release
          client-payload: |
            {
              "app": "${{ github.repository }}",
              "version": "${{ github.event.release.tag_name }}",
              "sha": "${{ github.sha }}",
              "actor": "${{ github.actor }}"
            }
# Receiver: React to the repository_dispatch event
name: Deploy on App Release

on:
  repository_dispatch:
    types: [app-release]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Deploy new version
        run: |
          echo "App: ${{ github.event.client_payload.app }}"
          echo "Version: ${{ github.event.client_payload.version }}"
          echo "Triggered by: ${{ github.event.client_payload.actor }}"
          ./deploy.sh \
            --app=${{ github.event.client_payload.app }} \
            --version=${{ github.event.client_payload.version }}

Managing Concurrency at Workflow and Job Level

The concurrency key controls how GitHub Actions handles simultaneous runs of the same workflow. Without concurrency control, rapid pushes to a branch can trigger multiple overlapping deployments, leading to race conditions, wasted resources, and unpredictable states.

Concurrency operates on groups — a string identifier. Only one workflow run (or job) with the same concurrency group can execute at a time. Additional runs are either queued (waiting) or cancelled (superseded by newer runs).

Concurrency Queue Behavior
sequenceDiagram
    participant Dev as Developer
    participant GH as GitHub Actions
    participant Env as Production

    Dev->>GH: Push commit A (run #1)
    GH->>Env: Deploy A (running)
    Dev->>GH: Push commit B (run #2)
    Note over GH: Same concurrency group
cancel-in-progress: true GH--xGH: Cancel run #1 GH->>Env: Deploy B (running) Dev->>GH: Push commit C (run #3) GH--xGH: Cancel run #2 GH->>Env: Deploy C (running) GH->>Dev: ✅ Only latest deployed
# Workflow-level concurrency: Only one deployment per environment
name: Deploy Application

on:
  push:
    branches: [main]

concurrency:
  # Dynamic group: allows parallel deploys to different environments
  # but serializes deploys to the same environment
  group: deploy-${{ github.ref_name }}
  cancel-in-progress: false  # Queue, don't cancel (safe for deploys)

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: actions/checkout@v4
      - name: Deploy
        run: ./deploy.sh

Job-Level Concurrency

Concurrency can also be applied at the job level, allowing finer control. A workflow with multiple jobs can have different concurrency rules for each.

# Mix of workflow and job-level concurrency
name: CI/CD Pipeline

on:
  push:
    branches: [main, 'release/**']

# Workflow-level: cancel redundant CI runs on same branch
concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true  # Safe for CI — just re-runs tests

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npm test

  deploy-staging:
    needs: test
    runs-on: ubuntu-latest
    # Job-level: only one staging deploy at a time, queue others
    concurrency:
      group: deploy-staging
      cancel-in-progress: false
    environment: staging
    steps:
      - uses: actions/checkout@v4
      - run: ./deploy.sh staging

  deploy-production:
    needs: deploy-staging
    if: startsWith(github.ref, 'refs/heads/release/')
    runs-on: ubuntu-latest
    # Job-level: only one production deploy, never cancel in progress
    concurrency:
      group: deploy-production
      cancel-in-progress: false
    environment: production
    steps:
      - uses: actions/checkout@v4
      - run: ./deploy.sh production

Advanced Concurrency Patterns

Environment-Based Concurrency Groups

# Dynamic concurrency groups per PR and environment
name: Preview Environments

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  deploy-preview:
    runs-on: ubuntu-latest
    concurrency:
      # Each PR gets its own concurrency group
      # New pushes to same PR cancel the previous preview deploy
      group: preview-pr-${{ github.event.pull_request.number }}
      cancel-in-progress: true
    environment:
      name: preview-pr-${{ github.event.pull_request.number }}
      url: https://pr-${{ github.event.pull_request.number }}.preview.example.com
    steps:
      - uses: actions/checkout@v4
      - name: Deploy preview
        run: |
          ./deploy-preview.sh \
            --pr=${{ github.event.pull_request.number }} \
            --sha=${{ github.sha }}

Serialized Deployment Queue

# Ensure deployments happen in order — never skip, never overlap
name: Ordered Deployment

on:
  push:
    branches: [main]

concurrency:
  # Single queue for production deployments
  group: production-deploy
  # CRITICAL: false means "wait in queue" not "cancel previous"
  cancel-in-progress: false

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: actions/checkout@v4

      - name: Acquire deployment lock
        run: echo "This job waited for all previous runs to complete"

      - name: Deploy
        run: ./deploy.sh --ordered

      - name: Verify deployment
        run: ./smoke-test.sh
Concurrency Best Practices:
  • CI (tests/lint): cancel-in-progress: true — supersede stale runs, only the latest matters
  • Staging deploys: cancel-in-progress: true — safe to cancel, always want latest
  • Production deploys: cancel-in-progress: false — NEVER cancel a running deploy, queue instead
  • PR previews: cancel-in-progress: true with PR number in group — isolate per PR
  • Group naming: Include github.ref for branch isolation, literal strings for global locks

Dynamic Matrices and Step Outputs

While Module 5 introduced fromJSON() for dynamic matrices, this section covers advanced patterns where the matrix is computed from complex logic — reading config files, querying APIs, analyzing git history, or combining multiple data sources into a single matrix definition.

fromJSON() with Computed Step Outputs

# Generate a test matrix from a configuration file
name: Config-Driven Matrix

on:
  push:
    branches: [main]
  pull_request:

jobs:
  setup:
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.generate.outputs.matrix }}
    steps:
      - uses: actions/checkout@v4

      - name: Generate matrix from config
        id: generate
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const config = JSON.parse(fs.readFileSync('ci-config.json', 'utf8'));

            // Build matrix from configuration file
            const matrix = {
              include: config.services.map(svc => ({
                service: svc.name,
                directory: svc.path,
                dockerfile: svc.dockerfile || 'Dockerfile',
                'node-version': svc.nodeVersion || '20',
                'run-e2e': svc.hasE2E || false
              }))
            };

            core.setOutput('matrix', JSON.stringify(matrix));
            core.info(`Generated matrix with ${matrix.include.length} services`);

  test:
    needs: setup
    strategy:
      fail-fast: false
      matrix: ${{ fromJSON(needs.setup.outputs.matrix) }}
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: ${{ matrix.directory }}
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}

      - name: Test ${{ matrix.service }}
        run: npm ci && npm test

      - name: E2E tests
        if: matrix.run-e2e
        run: npm run test:e2e

Generating Matrices from File Lists

# Build matrix from Terraform workspace directories
name: Terraform Multi-Environment

on:
  push:
    branches: [main]
    paths:
      - 'terraform/**'

jobs:
  detect-environments:
    runs-on: ubuntu-latest
    outputs:
      environments: ${{ steps.find.outputs.environments }}
      has-changes: ${{ steps.find.outputs.has-changes }}
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Find changed Terraform environments
        id: find
        run: |
          # Get directories under terraform/ that have changes
          CHANGED_DIRS=$(git diff --name-only origin/main...HEAD \
            | grep '^terraform/' \
            | cut -d/ -f2 \
            | sort -u \
            | jq -R -s -c 'split("\n") | map(select(. != ""))')

          echo "environments=${CHANGED_DIRS}" >> $GITHUB_OUTPUT

          if [ "$CHANGED_DIRS" = "[]" ]; then
            echo "has-changes=false" >> $GITHUB_OUTPUT
          else
            echo "has-changes=true" >> $GITHUB_OUTPUT
          fi

          echo "Changed environments: ${CHANGED_DIRS}"

  plan:
    needs: detect-environments
    if: needs.detect-environments.outputs.has-changes == 'true'
    strategy:
      fail-fast: false
      max-parallel: 3
      matrix:
        environment: ${{ fromJSON(needs.detect-environments.outputs.environments) }}
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: terraform/${{ matrix.environment }}
    steps:
      - uses: actions/checkout@v4

      - uses: hashicorp/setup-terraform@v3

      - name: Terraform Init (${{ matrix.environment }})
        run: terraform init

      - name: Terraform Plan (${{ matrix.environment }})
        run: terraform plan -out=plan.tfplan

      - name: Upload plan
        uses: actions/upload-artifact@v4
        with:
          name: tfplan-${{ matrix.environment }}
          path: terraform/${{ matrix.environment }}/plan.tfplan

Conditional Matrix Expansion

Sometimes you want a minimal matrix for PRs (fast feedback) but a full matrix for main (comprehensive coverage). Conditional expansion dynamically adjusts the matrix based on context.

# Expand matrix based on trigger context
name: Adaptive CI Matrix

on:
  push:
    branches: [main]
  pull_request:

jobs:
  setup:
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.build.outputs.matrix }}
    steps:
      - name: Build adaptive matrix
        id: build
        uses: actions/github-script@v7
        with:
          script: |
            const isPR = context.eventName === 'pull_request';
            const isMain = context.ref === 'refs/heads/main';

            let matrix;

            if (isPR) {
              // Minimal matrix for fast PR feedback
              matrix = {
                os: ['ubuntu-latest'],
                node: ['20'],
                include: [
                  { os: 'ubuntu-latest', node: '20', coverage: true }
                ]
              };
              core.info('PR detected — using minimal matrix (1 job)');
            } else {
              // Full matrix for main branch
              matrix = {
                os: ['ubuntu-latest', 'windows-latest', 'macos-latest'],
                node: ['18', '20', '22'],
                exclude: [
                  { os: 'macos-latest', node: '18' }
                ],
                include: [
                  { os: 'ubuntu-latest', node: '20', coverage: true },
                  { os: 'ubuntu-latest', node: '23', experimental: true }
                ]
              };
              core.info('Main branch — using full matrix (9 jobs)');
            }

            core.setOutput('matrix', JSON.stringify(matrix));

  test:
    needs: setup
    strategy:
      fail-fast: ${{ github.event_name == 'pull_request' }}
      matrix: ${{ fromJSON(needs.setup.outputs.matrix) }}
    runs-on: ${{ matrix.os }}
    continue-on-error: ${{ matrix.experimental || false }}
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}
          cache: 'npm'

      - run: npm ci
      - run: npm test

      - name: Upload coverage
        if: matrix.coverage
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.os }}-node${{ matrix.node }}
          path: coverage/

Matrix from API Response

# Generate matrix from an external API (e.g., supported regions)
name: Multi-Region Deployment

on:
  workflow_dispatch:
    inputs:
      version:
        description: 'Version to deploy'
        required: true
        type: string

jobs:
  get-regions:
    runs-on: ubuntu-latest
    outputs:
      regions: ${{ steps.fetch.outputs.regions }}
    steps:
      - name: Fetch active regions from API
        id: fetch
        run: |
          # Query your infrastructure API for active deployment targets
          REGIONS=$(curl -s https://api.internal.example.com/regions \
            -H "Authorization: Bearer ${{ secrets.INFRA_TOKEN }}" \
            | jq -c '[.[] | select(.active == true) | {region: .name, cluster: .cluster_id, priority: .priority}]')

          echo "regions=${REGIONS}" >> $GITHUB_OUTPUT
          echo "Active regions: ${REGIONS}"

  deploy:
    needs: get-regions
    strategy:
      fail-fast: false
      max-parallel: 2  # Roll out 2 regions at a time
      matrix:
        target: ${{ fromJSON(needs.get-regions.outputs.regions) }}
    runs-on: ubuntu-latest
    environment: production-${{ matrix.target.region }}
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ inputs.version }}

      - name: Deploy to ${{ matrix.target.region }}
        run: |
          echo "Region: ${{ matrix.target.region }}"
          echo "Cluster: ${{ matrix.target.cluster }}"
          echo "Priority: ${{ matrix.target.priority }}"
          kubectl config use-context ${{ matrix.target.cluster }}
          helm upgrade app ./chart \
            --set image.tag=${{ inputs.version }} \
            --set region=${{ matrix.target.region }}
Dynamic Matrix Output Size Limit: Step outputs are limited to 1 MB. If your matrix JSON exceeds this (unlikely but possible with hundreds of entries), split it into multiple outputs or write to a file and upload as an artifact. For most real-world scenarios (even 100+ matrix entries), this limit is never hit.

Exercises

Exercise 1 Difficulty: Intermediate

Reusable CI Workflow Library

Create a reusable workflow library with the following components:

  • A reusable lint workflow accepting node-version (string) and eslint-config (string, default '@company/standard') as inputs
  • A reusable test workflow accepting node-version, coverage-threshold (number, default 80), and upload-coverage (boolean)
  • A caller workflow that chains lint → test, passes the same node-version to both, and only runs the test if lint succeeds
  • The test workflow should output the actual coverage percentage
  • The caller should fail if coverage output is below 70%
workflow_call inputs outputs chaining
Exercise 2 Difficulty: Advanced

Concurrency-Controlled Deployment Pipeline

Design a deployment workflow with sophisticated concurrency management:

  • CI jobs (lint, test) use cancel-in-progress: true grouped by branch
  • Staging deploys use cancel-in-progress: true with a deploy-staging group
  • Production deploys use cancel-in-progress: false (queue, never cancel)
  • PR preview environments use per-PR concurrency groups
  • Include a "deployment lock" mechanism that checks if a hotfix is in progress before queuing
  • Add Slack notification when a deployment is waiting in queue
concurrency cancel-in-progress deployment queue environment protection
Exercise 3 Difficulty: Advanced

Adaptive Matrix with Config File

Build a CI system that reads a ci-matrix.json configuration file from the repository root to determine what to test:

  • The config file defines services, each with: name, path, language, test-command, needs-docker
  • A setup job reads the file and generates the matrix JSON
  • For PRs, filter the matrix to only services with changed files (using git diff)
  • For main branch pushes, run the full matrix
  • Services with needs-docker: true should get services: containers in their matrix entry
  • Output a summary of which services were tested and their pass/fail status
dynamic matrix fromJSON config-driven CI conditional expansion
Exercise 4 Difficulty: Expert

Nested Reusable Workflow Platform

Design a 3-level reusable workflow platform for a microservices organization:

  • Level 1 (Caller): Each microservice repo has a thin ci.yml that calls the platform workflow
  • Level 2 (Platform): A "facade" reusable workflow that orchestrates build → test → security-scan → deploy
  • Level 3 (Primitives): Individual reusable workflows for each stage (build.yml, test.yml, scan.yml, deploy.yml)
  • The platform workflow accepts inputs for: language, framework, deploy-target, and feature-flags (as JSON string)
  • Outputs from build (image-tag) flow through to deploy
  • Use secrets: inherit at level 1→2 but explicit secrets at level 2→3
  • Document why you can't add a level 4 and show the workaround
nested workflows workflow composition platform engineering secrets inherit

Next in the Series

In Module 7: Containers and Docker, we'll explore service containers, custom Docker actions, building and publishing container images, multi-stage builds in CI, and container-based runners for reproducible environments.