Reusable Workflows — DRY Principles for CI/CD
Reusable workflows solve the copy-paste problem across repositories. Instead of duplicating 200-line deployment workflows in 50 repos, you define the workflow once and call it from any other workflow using the workflow_call trigger. This is GitHub's native answer to shared pipeline libraries (like Jenkins Shared Libraries or Azure DevOps Templates).
A reusable workflow is simply a standard workflow file that includes on: workflow_call as one of its triggers. It lives in .github/workflows/ in any repository (the same repo or a different one) and can define inputs, outputs, and secrets that callers must provide.
flowchart TD
A[Caller Workflow
repo-a/.github/workflows/ci.yml] -->|uses| B[Reusable: Build & Test
shared-org/.github/workflows/build.yml]
A -->|uses| C[Reusable: Deploy
shared-org/.github/workflows/deploy.yml]
B -->|uses| D[Reusable: Lint
shared-org/.github/workflows/lint.yml]
C -->|uses| E[Reusable: Notify
shared-org/.github/workflows/notify.yml]
style A fill:#132440,color:#fff
style B fill:#3B9797,color:#fff
style C fill:#3B9797,color:#fff
style D fill:#16476A,color:#fff
style E fill:#16476A,color:#fff
Creating and Calling Reusable Workflows
The callee (reusable workflow) declares on: workflow_call and optionally defines inputs, outputs, and secrets. The caller uses the uses: keyword at the job level (not step level) to invoke it.
# CALLEE: .github/workflows/reusable-build.yml
# This workflow can be called by other workflows
name: Reusable Build Pipeline
on:
workflow_call:
inputs:
node-version:
description: 'Node.js version to use'
required: true
type: string
run-tests:
description: 'Whether to run tests after build'
required: false
type: boolean
default: true
environment:
description: 'Target environment'
required: false
type: string
default: 'staging'
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node-version }}
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Build for ${{ inputs.environment }}
run: npm run build
env:
NODE_ENV: ${{ inputs.environment == 'production' && 'production' || 'development' }}
- name: Run tests
if: inputs.run-tests
run: npm test
- name: Upload build output
uses: actions/upload-artifact@v4
with:
name: build-${{ inputs.environment }}
path: dist/
retention-days: 5
# CALLER: .github/workflows/ci.yml
# This workflow calls the reusable build workflow
name: CI Pipeline
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
jobs:
build-staging:
uses: my-org/shared-workflows/.github/workflows/reusable-build.yml@main
with:
node-version: '20'
run-tests: true
environment: 'staging'
build-production:
needs: build-staging
if: github.ref == 'refs/heads/main'
uses: my-org/shared-workflows/.github/workflows/reusable-build.yml@main
with:
node-version: '20'
run-tests: false
environment: 'production'
Key rules for calling reusable workflows:
uses:at job level — Not insidesteps:. The entire job is replaced by the reusable workflow's jobs.- Reference format —
{owner}/{repo}/.github/workflows/{filename}@{ref}for cross-repo, or./.github/workflows/{filename}for same-repo. - Ref pinning — Always pin to a SHA, tag, or branch. Use SHA for production:
@a1b2c3d4 - Cannot add steps — A calling job that uses
uses:cannot also definesteps:,runs-on:, orcontainer:.
When to Use Reusable Workflows vs Composite Actions
Use reusable workflows when you need to share entire job definitions (including runner selection, services, environment, concurrency), when you need multiple jobs with dependencies, or when the shared logic requires secrets access with explicit declarations.
Use composite actions when you need to share a sequence of steps within a single job, when the caller needs to add more steps before/after, or when you want maximum flexibility in how the shared logic integrates into existing jobs.
Rule of thumb: If you're sharing a complete pipeline stage (build, test, deploy), use a reusable workflow. If you're sharing a task (setup Node, run linting, send notification), use a composite action.
Reusable Workflow Inputs and Outputs
Inputs provide parameterization — making workflows configurable without modification. Outputs allow workflows to return data to the caller for use in subsequent jobs. Secrets provide secure credential passing with two models: explicit declaration or blanket inheritance.
Typed Input Definitions
Reusable workflow inputs support four types: string, number, boolean, and choice (with options:). Type validation happens at call time — passing a string where a boolean is expected will fail the workflow.
# Callee with comprehensive input types
name: Reusable Deploy
on:
workflow_call:
inputs:
environment:
description: 'Deployment target'
required: true
type: string
replicas:
description: 'Number of replicas to deploy'
required: false
type: number
default: 2
dry-run:
description: 'Simulate deployment without applying changes'
required: false
type: boolean
default: false
log-level:
description: 'Logging verbosity'
required: false
type: string
default: 'info'
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Deploy application
run: |
echo "Environment: ${{ inputs.environment }}"
echo "Replicas: ${{ inputs.replicas }}"
echo "Dry run: ${{ inputs.dry-run }}"
echo "Log level: ${{ inputs.log-level }}"
if [ "${{ inputs.dry-run }}" = "true" ]; then
echo "DRY RUN — no changes applied"
kubectl apply -f k8s/ --dry-run=client
else
kubectl apply -f k8s/
kubectl scale deployment/app --replicas=${{ inputs.replicas }}
fi
Workflow Outputs
Outputs let a reusable workflow return values to the caller. This enables chaining — a build workflow can output the image tag, which a deploy workflow then consumes. Outputs are defined at the workflow level and mapped from job outputs.
# Callee: Returns build metadata to the caller
name: Reusable Build with Outputs
on:
workflow_call:
inputs:
image-name:
required: true
type: string
outputs:
image-tag:
description: 'The Docker image tag that was built'
value: ${{ jobs.build.outputs.tag }}
image-digest:
description: 'The image digest for immutable reference'
value: ${{ jobs.build.outputs.digest }}
build-timestamp:
description: 'When the build completed'
value: ${{ jobs.build.outputs.timestamp }}
jobs:
build:
runs-on: ubuntu-latest
outputs:
tag: ${{ steps.meta.outputs.tag }}
digest: ${{ steps.push.outputs.digest }}
timestamp: ${{ steps.meta.outputs.timestamp }}
steps:
- uses: actions/checkout@v4
- name: Generate metadata
id: meta
run: |
TAG="${{ inputs.image-name }}:${{ github.sha }}"
echo "tag=${TAG}" >> $GITHUB_OUTPUT
echo "timestamp=$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> $GITHUB_OUTPUT
- name: Build and push
id: push
run: |
docker build -t ${{ steps.meta.outputs.tag }} .
docker push ${{ steps.meta.outputs.tag }}
DIGEST=$(docker inspect --format='{{index .RepoDigests 0}}' ${{ steps.meta.outputs.tag }} | cut -d@ -f2)
echo "digest=${DIGEST}" >> $GITHUB_OUTPUT
# Caller: Consumes outputs from the reusable workflow
name: Build and Deploy Pipeline
on:
push:
branches: [main]
jobs:
build:
uses: my-org/shared-workflows/.github/workflows/reusable-build.yml@v2
with:
image-name: ghcr.io/my-org/my-app
deploy-staging:
needs: build
runs-on: ubuntu-latest
steps:
- name: Deploy image to staging
run: |
echo "Deploying ${{ needs.build.outputs.image-tag }}"
echo "Digest: ${{ needs.build.outputs.image-digest }}"
echo "Built at: ${{ needs.build.outputs.build-timestamp }}"
helm upgrade my-app ./chart \
--set image.tag=${{ needs.build.outputs.image-tag }} \
--namespace staging
Secrets: Inherit vs Explicit
Reusable workflows cannot access the caller's secrets by default — they must be explicitly passed or inherited. Two approaches exist:
# Approach 1: Explicit secret declarations (recommended for shared repos)
# Callee declares required secrets
on:
workflow_call:
secrets:
DEPLOY_TOKEN:
description: 'Token for deploying to production'
required: true
SLACK_WEBHOOK:
description: 'Slack notification URL'
required: false
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Deploy
run: ./deploy.sh
env:
TOKEN: ${{ secrets.DEPLOY_TOKEN }}
- name: Notify
if: secrets.SLACK_WEBHOOK != ''
run: |
curl -X POST ${{ secrets.SLACK_WEBHOOK }} \
-d '{"text": "Deployment complete"}'
# Caller: Passes secrets explicitly
jobs:
deploy:
uses: my-org/shared-workflows/.github/workflows/deploy.yml@main
with:
environment: production
secrets:
DEPLOY_TOKEN: ${{ secrets.PROD_DEPLOY_TOKEN }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_URL }}
# Approach 2: Inherit all secrets (convenient for same-org workflows)
jobs:
deploy:
uses: my-org/shared-workflows/.github/workflows/deploy.yml@main
with:
environment: production
secrets: inherit # Passes ALL caller secrets to the callee
secrets: inherit: Using secrets: inherit passes all secrets from the caller to the callee. This is convenient but dangerous for cross-organization reusable workflows. If you call a third-party reusable workflow with secrets: inherit, that workflow has access to every secret in your repository. Always use explicit secret passing for workflows you don't fully control.
Nesting Reusable Workflows
Reusable workflows can call other reusable workflows, enabling composable pipeline architectures. This lets you build a library of small, focused workflows that compose into complex pipelines — similar to function composition in programming.
flowchart TD
L1[Level 1: CI Pipeline
Caller Workflow] -->|calls| L2A[Level 2: Build & Test
Reusable Workflow]
L1 -->|calls| L2B[Level 2: Deploy
Reusable Workflow]
L2A -->|calls| L3A[Level 3: Lint
Reusable Workflow]
L2A -->|calls| L3B[Level 3: Unit Tests
Reusable Workflow]
L2B -->|calls| L3C[Level 3: Provision Infra
Reusable Workflow]
L3C -->|calls| L4[Level 4: Terraform Apply
Reusable Workflow]
L4 -.-x|BLOCKED| L5[Level 5: ❌ Exceeds max depth]
style L1 fill:#132440,color:#fff
style L2A fill:#3B9797,color:#fff
style L2B fill:#3B9797,color:#fff
style L3A fill:#16476A,color:#fff
style L3B fill:#16476A,color:#fff
style L3C fill:#16476A,color:#fff
style L4 fill:#BF092F,color:#fff
style L5 fill:#666,color:#fff
# Level 2: Reusable workflow that calls another reusable workflow
name: Reusable Build and Test
on:
workflow_call:
inputs:
node-version:
required: true
type: string
outputs:
coverage-percent:
description: 'Test coverage percentage'
value: ${{ jobs.test.outputs.coverage }}
jobs:
lint:
# Level 3: Calling another reusable workflow
uses: my-org/shared-workflows/.github/workflows/lint.yml@main
with:
node-version: ${{ inputs.node-version }}
test:
needs: lint
runs-on: ubuntu-latest
outputs:
coverage: ${{ steps.cov.outputs.percent }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node-version }}
- run: npm ci && npm test -- --coverage
- name: Extract coverage
id: cov
run: |
PERCENT=$(cat coverage/coverage-summary.json | jq '.total.lines.pct')
echo "percent=${PERCENT}" >> $GITHUB_OUTPUT
Limitations and Workarounds
- Maximum 4 levels deep — The top-level caller is level 1. You can nest up to 3 additional levels of reusable workflows. Exceeding this fails with a clear error.
- Maximum 20 unique reusable workflows per workflow file — Hitting this limit requires refactoring into fewer, more capable workflows.
- No circular references — Workflow A cannot call workflow B if B (directly or indirectly) calls A.
envcontext not inherited — Environment variables set in the caller'senv:map are NOT available in the callee. Pass them as inputs instead.- Permissions not inherited — Each reusable workflow uses the permissions of the caller by default, but
permissions:in the callee can only restrict (not expand) what the caller granted.
# Workaround: Flattening deep nesting with a facade workflow
# Instead of A -> B -> C -> D -> E (5 levels, BLOCKED),
# create a facade that orchestrates B, C, D, E as sibling jobs
name: Reusable Full Pipeline (Facade)
on:
workflow_call:
inputs:
ref:
required: true
type: string
environment:
required: true
type: string
secrets:
DEPLOY_TOKEN:
required: true
jobs:
lint:
uses: my-org/shared-workflows/.github/workflows/lint.yml@main
with:
ref: ${{ inputs.ref }}
build:
needs: lint
uses: my-org/shared-workflows/.github/workflows/build.yml@main
with:
ref: ${{ inputs.ref }}
test:
needs: build
uses: my-org/shared-workflows/.github/workflows/test.yml@main
with:
ref: ${{ inputs.ref }}
deploy:
needs: test
uses: my-org/shared-workflows/.github/workflows/deploy.yml@main
with:
ref: ${{ inputs.ref }}
environment: ${{ inputs.environment }}
secrets:
DEPLOY_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
Workflow Call vs Workflow Dispatch Events
Both workflow_call and workflow_dispatch accept inputs, but they serve fundamentally different purposes. Understanding when to use each is critical for designing maintainable CI/CD systems.
| Feature | workflow_call |
workflow_dispatch |
|---|---|---|
| Trigger source | Another workflow (programmatic) | GitHub UI, API, or CLI (manual/external) |
| Execution context | Runs as part of the caller's workflow run | Creates a new, independent workflow run |
| Appears in UI | Nested inside the caller's run | As its own separate run |
| Input types | string, number, boolean |
string, boolean, choice, environment |
| Can return outputs | Yes (via outputs:) |
No |
| Secrets | Passed from caller or inherited | Access own repo secrets directly |
| GITHUB_TOKEN scope | Caller's token permissions | Own repo's token permissions |
| Primary use case | Code reuse, shared pipeline libraries | On-demand ops, manual deploys, ad-hoc tasks |
# A workflow can support BOTH triggers simultaneously
name: Deploy Application
on:
# Can be called by other workflows (reusable)
workflow_call:
inputs:
environment:
required: true
type: string
version:
required: true
type: string
secrets:
DEPLOY_KEY:
required: true
# Can also be triggered manually via UI/API
workflow_dispatch:
inputs:
environment:
description: 'Target environment'
required: true
type: choice
options:
- staging
- production
version:
description: 'Version to deploy (e.g., v1.2.3)'
required: true
type: string
jobs:
deploy:
runs-on: ubuntu-latest
environment: ${{ inputs.environment }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.version }}
- name: Deploy to ${{ inputs.environment }}
run: |
echo "Deploying version ${{ inputs.version }}"
./scripts/deploy.sh --env=${{ inputs.environment }}
env:
# secrets.DEPLOY_KEY works for workflow_call (passed by caller)
# For workflow_dispatch, it reads from repo secrets directly
DEPLOY_KEY: ${{ secrets.DEPLOY_KEY }}
Repository Dispatch for Cross-Repo Triggers
When you need to trigger workflows across repositories without the tight coupling of workflow_call, use repository_dispatch. This is an event-driven pattern where one repo sends an event and another repo reacts to it.
# Sender: Trigger deployment in another repo after release
name: Trigger Downstream Deploy
on:
release:
types: [published]
jobs:
notify-deploy-repo:
runs-on: ubuntu-latest
steps:
- name: Trigger deployment workflow
uses: peter-evans/repository-dispatch@v3
with:
token: ${{ secrets.CROSS_REPO_PAT }}
repository: my-org/deploy-infra
event-type: app-release
client-payload: |
{
"app": "${{ github.repository }}",
"version": "${{ github.event.release.tag_name }}",
"sha": "${{ github.sha }}",
"actor": "${{ github.actor }}"
}
# Receiver: React to the repository_dispatch event
name: Deploy on App Release
on:
repository_dispatch:
types: [app-release]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Deploy new version
run: |
echo "App: ${{ github.event.client_payload.app }}"
echo "Version: ${{ github.event.client_payload.version }}"
echo "Triggered by: ${{ github.event.client_payload.actor }}"
./deploy.sh \
--app=${{ github.event.client_payload.app }} \
--version=${{ github.event.client_payload.version }}
Managing Concurrency at Workflow and Job Level
The concurrency key controls how GitHub Actions handles simultaneous runs of the same workflow. Without concurrency control, rapid pushes to a branch can trigger multiple overlapping deployments, leading to race conditions, wasted resources, and unpredictable states.
Concurrency operates on groups — a string identifier. Only one workflow run (or job) with the same concurrency group can execute at a time. Additional runs are either queued (waiting) or cancelled (superseded by newer runs).
sequenceDiagram
participant Dev as Developer
participant GH as GitHub Actions
participant Env as Production
Dev->>GH: Push commit A (run #1)
GH->>Env: Deploy A (running)
Dev->>GH: Push commit B (run #2)
Note over GH: Same concurrency group
cancel-in-progress: true
GH--xGH: Cancel run #1
GH->>Env: Deploy B (running)
Dev->>GH: Push commit C (run #3)
GH--xGH: Cancel run #2
GH->>Env: Deploy C (running)
GH->>Dev: ✅ Only latest deployed
# Workflow-level concurrency: Only one deployment per environment
name: Deploy Application
on:
push:
branches: [main]
concurrency:
# Dynamic group: allows parallel deploys to different environments
# but serializes deploys to the same environment
group: deploy-${{ github.ref_name }}
cancel-in-progress: false # Queue, don't cancel (safe for deploys)
jobs:
deploy:
runs-on: ubuntu-latest
environment: production
steps:
- uses: actions/checkout@v4
- name: Deploy
run: ./deploy.sh
Job-Level Concurrency
Concurrency can also be applied at the job level, allowing finer control. A workflow with multiple jobs can have different concurrency rules for each.
# Mix of workflow and job-level concurrency
name: CI/CD Pipeline
on:
push:
branches: [main, 'release/**']
# Workflow-level: cancel redundant CI runs on same branch
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true # Safe for CI — just re-runs tests
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm ci && npm test
deploy-staging:
needs: test
runs-on: ubuntu-latest
# Job-level: only one staging deploy at a time, queue others
concurrency:
group: deploy-staging
cancel-in-progress: false
environment: staging
steps:
- uses: actions/checkout@v4
- run: ./deploy.sh staging
deploy-production:
needs: deploy-staging
if: startsWith(github.ref, 'refs/heads/release/')
runs-on: ubuntu-latest
# Job-level: only one production deploy, never cancel in progress
concurrency:
group: deploy-production
cancel-in-progress: false
environment: production
steps:
- uses: actions/checkout@v4
- run: ./deploy.sh production
Advanced Concurrency Patterns
Environment-Based Concurrency Groups
# Dynamic concurrency groups per PR and environment
name: Preview Environments
on:
pull_request:
types: [opened, synchronize, reopened]
jobs:
deploy-preview:
runs-on: ubuntu-latest
concurrency:
# Each PR gets its own concurrency group
# New pushes to same PR cancel the previous preview deploy
group: preview-pr-${{ github.event.pull_request.number }}
cancel-in-progress: true
environment:
name: preview-pr-${{ github.event.pull_request.number }}
url: https://pr-${{ github.event.pull_request.number }}.preview.example.com
steps:
- uses: actions/checkout@v4
- name: Deploy preview
run: |
./deploy-preview.sh \
--pr=${{ github.event.pull_request.number }} \
--sha=${{ github.sha }}
Serialized Deployment Queue
# Ensure deployments happen in order — never skip, never overlap
name: Ordered Deployment
on:
push:
branches: [main]
concurrency:
# Single queue for production deployments
group: production-deploy
# CRITICAL: false means "wait in queue" not "cancel previous"
cancel-in-progress: false
jobs:
deploy:
runs-on: ubuntu-latest
environment: production
steps:
- uses: actions/checkout@v4
- name: Acquire deployment lock
run: echo "This job waited for all previous runs to complete"
- name: Deploy
run: ./deploy.sh --ordered
- name: Verify deployment
run: ./smoke-test.sh
- CI (tests/lint):
cancel-in-progress: true— supersede stale runs, only the latest matters - Staging deploys:
cancel-in-progress: true— safe to cancel, always want latest - Production deploys:
cancel-in-progress: false— NEVER cancel a running deploy, queue instead - PR previews:
cancel-in-progress: truewith PR number in group — isolate per PR - Group naming: Include
github.reffor branch isolation, literal strings for global locks
Dynamic Matrices and Step Outputs
While Module 5 introduced fromJSON() for dynamic matrices, this section covers advanced patterns where the matrix is computed from complex logic — reading config files, querying APIs, analyzing git history, or combining multiple data sources into a single matrix definition.
fromJSON() with Computed Step Outputs
# Generate a test matrix from a configuration file
name: Config-Driven Matrix
on:
push:
branches: [main]
pull_request:
jobs:
setup:
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.generate.outputs.matrix }}
steps:
- uses: actions/checkout@v4
- name: Generate matrix from config
id: generate
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
const config = JSON.parse(fs.readFileSync('ci-config.json', 'utf8'));
// Build matrix from configuration file
const matrix = {
include: config.services.map(svc => ({
service: svc.name,
directory: svc.path,
dockerfile: svc.dockerfile || 'Dockerfile',
'node-version': svc.nodeVersion || '20',
'run-e2e': svc.hasE2E || false
}))
};
core.setOutput('matrix', JSON.stringify(matrix));
core.info(`Generated matrix with ${matrix.include.length} services`);
test:
needs: setup
strategy:
fail-fast: false
matrix: ${{ fromJSON(needs.setup.outputs.matrix) }}
runs-on: ubuntu-latest
defaults:
run:
working-directory: ${{ matrix.directory }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
- name: Test ${{ matrix.service }}
run: npm ci && npm test
- name: E2E tests
if: matrix.run-e2e
run: npm run test:e2e
Generating Matrices from File Lists
# Build matrix from Terraform workspace directories
name: Terraform Multi-Environment
on:
push:
branches: [main]
paths:
- 'terraform/**'
jobs:
detect-environments:
runs-on: ubuntu-latest
outputs:
environments: ${{ steps.find.outputs.environments }}
has-changes: ${{ steps.find.outputs.has-changes }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Find changed Terraform environments
id: find
run: |
# Get directories under terraform/ that have changes
CHANGED_DIRS=$(git diff --name-only origin/main...HEAD \
| grep '^terraform/' \
| cut -d/ -f2 \
| sort -u \
| jq -R -s -c 'split("\n") | map(select(. != ""))')
echo "environments=${CHANGED_DIRS}" >> $GITHUB_OUTPUT
if [ "$CHANGED_DIRS" = "[]" ]; then
echo "has-changes=false" >> $GITHUB_OUTPUT
else
echo "has-changes=true" >> $GITHUB_OUTPUT
fi
echo "Changed environments: ${CHANGED_DIRS}"
plan:
needs: detect-environments
if: needs.detect-environments.outputs.has-changes == 'true'
strategy:
fail-fast: false
max-parallel: 3
matrix:
environment: ${{ fromJSON(needs.detect-environments.outputs.environments) }}
runs-on: ubuntu-latest
defaults:
run:
working-directory: terraform/${{ matrix.environment }}
steps:
- uses: actions/checkout@v4
- uses: hashicorp/setup-terraform@v3
- name: Terraform Init (${{ matrix.environment }})
run: terraform init
- name: Terraform Plan (${{ matrix.environment }})
run: terraform plan -out=plan.tfplan
- name: Upload plan
uses: actions/upload-artifact@v4
with:
name: tfplan-${{ matrix.environment }}
path: terraform/${{ matrix.environment }}/plan.tfplan
Conditional Matrix Expansion
Sometimes you want a minimal matrix for PRs (fast feedback) but a full matrix for main (comprehensive coverage). Conditional expansion dynamically adjusts the matrix based on context.
# Expand matrix based on trigger context
name: Adaptive CI Matrix
on:
push:
branches: [main]
pull_request:
jobs:
setup:
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.build.outputs.matrix }}
steps:
- name: Build adaptive matrix
id: build
uses: actions/github-script@v7
with:
script: |
const isPR = context.eventName === 'pull_request';
const isMain = context.ref === 'refs/heads/main';
let matrix;
if (isPR) {
// Minimal matrix for fast PR feedback
matrix = {
os: ['ubuntu-latest'],
node: ['20'],
include: [
{ os: 'ubuntu-latest', node: '20', coverage: true }
]
};
core.info('PR detected — using minimal matrix (1 job)');
} else {
// Full matrix for main branch
matrix = {
os: ['ubuntu-latest', 'windows-latest', 'macos-latest'],
node: ['18', '20', '22'],
exclude: [
{ os: 'macos-latest', node: '18' }
],
include: [
{ os: 'ubuntu-latest', node: '20', coverage: true },
{ os: 'ubuntu-latest', node: '23', experimental: true }
]
};
core.info('Main branch — using full matrix (9 jobs)');
}
core.setOutput('matrix', JSON.stringify(matrix));
test:
needs: setup
strategy:
fail-fast: ${{ github.event_name == 'pull_request' }}
matrix: ${{ fromJSON(needs.setup.outputs.matrix) }}
runs-on: ${{ matrix.os }}
continue-on-error: ${{ matrix.experimental || false }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
cache: 'npm'
- run: npm ci
- run: npm test
- name: Upload coverage
if: matrix.coverage
uses: actions/upload-artifact@v4
with:
name: coverage-${{ matrix.os }}-node${{ matrix.node }}
path: coverage/
Matrix from API Response
# Generate matrix from an external API (e.g., supported regions)
name: Multi-Region Deployment
on:
workflow_dispatch:
inputs:
version:
description: 'Version to deploy'
required: true
type: string
jobs:
get-regions:
runs-on: ubuntu-latest
outputs:
regions: ${{ steps.fetch.outputs.regions }}
steps:
- name: Fetch active regions from API
id: fetch
run: |
# Query your infrastructure API for active deployment targets
REGIONS=$(curl -s https://api.internal.example.com/regions \
-H "Authorization: Bearer ${{ secrets.INFRA_TOKEN }}" \
| jq -c '[.[] | select(.active == true) | {region: .name, cluster: .cluster_id, priority: .priority}]')
echo "regions=${REGIONS}" >> $GITHUB_OUTPUT
echo "Active regions: ${REGIONS}"
deploy:
needs: get-regions
strategy:
fail-fast: false
max-parallel: 2 # Roll out 2 regions at a time
matrix:
target: ${{ fromJSON(needs.get-regions.outputs.regions) }}
runs-on: ubuntu-latest
environment: production-${{ matrix.target.region }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ inputs.version }}
- name: Deploy to ${{ matrix.target.region }}
run: |
echo "Region: ${{ matrix.target.region }}"
echo "Cluster: ${{ matrix.target.cluster }}"
echo "Priority: ${{ matrix.target.priority }}"
kubectl config use-context ${{ matrix.target.cluster }}
helm upgrade app ./chart \
--set image.tag=${{ inputs.version }} \
--set region=${{ matrix.target.region }}
Exercises
Reusable CI Workflow Library
Create a reusable workflow library with the following components:
- A reusable lint workflow accepting
node-version(string) andeslint-config(string, default'@company/standard') as inputs - A reusable test workflow accepting
node-version,coverage-threshold(number, default 80), andupload-coverage(boolean) - A caller workflow that chains lint → test, passes the same
node-versionto both, and only runs the test if lint succeeds - The test workflow should output the actual coverage percentage
- The caller should fail if coverage output is below 70%
Concurrency-Controlled Deployment Pipeline
Design a deployment workflow with sophisticated concurrency management:
- CI jobs (lint, test) use
cancel-in-progress: truegrouped by branch - Staging deploys use
cancel-in-progress: truewith adeploy-staginggroup - Production deploys use
cancel-in-progress: false(queue, never cancel) - PR preview environments use per-PR concurrency groups
- Include a "deployment lock" mechanism that checks if a hotfix is in progress before queuing
- Add Slack notification when a deployment is waiting in queue
Adaptive Matrix with Config File
Build a CI system that reads a ci-matrix.json configuration file from the repository root to determine what to test:
- The config file defines services, each with:
name,path,language,test-command,needs-docker - A setup job reads the file and generates the matrix JSON
- For PRs, filter the matrix to only services with changed files (using
git diff) - For main branch pushes, run the full matrix
- Services with
needs-docker: trueshould getservices:containers in their matrix entry - Output a summary of which services were tested and their pass/fail status
Nested Reusable Workflow Platform
Design a 3-level reusable workflow platform for a microservices organization:
- Level 1 (Caller): Each microservice repo has a thin
ci.ymlthat calls the platform workflow - Level 2 (Platform): A "facade" reusable workflow that orchestrates build → test → security-scan → deploy
- Level 3 (Primitives): Individual reusable workflows for each stage (build.yml, test.yml, scan.yml, deploy.yml)
- The platform workflow accepts inputs for: language, framework, deploy-target, and feature-flags (as JSON string)
- Outputs from build (image-tag) flow through to deploy
- Use
secrets: inheritat level 1→2 but explicit secrets at level 2→3 - Document why you can't add a level 4 and show the workaround
Next in the Series
In Module 7: Containers and Docker, we'll explore service containers, custom Docker actions, building and publishing container images, multi-stage builds in CI, and container-based runners for reproducible environments.