Module 5: Agents, Pools & CI Builds

How Pipelines Execute

In Module 4, we wrote pipeline YAML — triggers, variables, expressions, and conditions. But where does that YAML actually execute? The answer is agents — software processes that run your pipeline jobs on physical or virtual machines.

Think of it this way: agents are like construction workers. The pipeline YAML is the blueprint, agent pools are work crews, and each agent is a worker who picks up a job, completes it, and reports back. Microsoft-hosted agents are contractors — fresh each time, no memory of previous projects. Self-hosted agents are full-time employees — persistent, familiar with your codebase, and equipped with specialized tools.

The Execution Model

When you push code and a pipeline triggers, here's what happens behind the scenes:

1. Pipeline service queues the job — Azure DevOps evaluates triggers, parses YAML, and creates a job request
2. Job lands in an agent pool — the request is routed to the specified pool (e.g., vmImage: 'ubuntu-latest')
3. Agent picks up the job — an available agent with matching capabilities claims the work
4. Agent executes steps sequentially — checkout, restore, build, test, publish — reporting progress in real-time
5. Agent reports completion — success/failure status, logs, and artifacts are sent back to the service

flowchart LR
    A[Code Push] --> B[Pipeline Service]
    B --> C{Agent Pool}
    C --> D[Agent 1]
    C --> E[Agent 2]
    C --> F[Agent N]
    D --> G[Execute Job]
    G --> H[Report Results]
    H --> B
                            

Agent Types at a Glance

Microsoft-Hosted Agents

Microsoft-hosted agents are the easiest way to get started — specify an image name and Azure DevOps provisions a fresh VM for your job. No setup, no maintenance, no infrastructure to manage.

Available Images

Limitations & Cost Model

• Job timeout: 60 minutes (free tier) / 360 minutes (paid tier, 6 hours max)
• No persistent state: VM is destroyed after each job — no caching between jobs without Cache@2 task
• No private network: Cannot access resources behind firewalls or VNETs
• Disk space: ~14 GB free on Ubuntu, ~14 GB on Windows
• Free tier: 1,800 minutes/month for private repos (public repos get unlimited)
• Paid: $40/month per additional parallel job (run multiple jobs simultaneously)

Specifying Different Pools per Job

# Use different agent images for different jobs.
# This is common when you need OS-specific testing.
trigger:
  - main

stages:
  - stage: Build
    jobs:
      # ─── Job 1: Build on Linux (fast, cheap) ─────────────────
      - job: BuildLinux
        pool:
          vmImage: 'ubuntu-latest'      # Most common choice
        steps:
          - script: dotnet build
            displayName: 'Build on Linux'

      # ─── Job 2: Build on Windows (for .NET Framework) ────────
      - job: BuildWindows
        pool:
          vmImage: 'windows-latest'     # Needed for .NET Framework
        steps:
          - script: dotnet build
            displayName: 'Build on Windows'

      # ─── Job 3: Build on macOS (for iOS/Xcode) ──────────────
      - job: BuildMac
        pool:
          vmImage: 'macos-latest'       # Needed for Xcode/iOS
        steps:
          - script: xcodebuild -version
            displayName: 'Verify Xcode version'

Self-Hosted Agents

When Microsoft-hosted agents don't meet your needs — private network access, custom hardware (GPUs), compliance requirements, or cost optimization at scale — you run your own agents.

When You Need Self-Hosted

• Private network access: Deploy to databases, VMs, or services behind a firewall/VNET
• Custom hardware: GPU builds (ML model training), ARM64, specialized test devices
• Cost at scale: If you run >10 parallel jobs, self-hosted can be cheaper than $40/job/month
• Compliance: Data residency requirements — code never leaves your network
• Build speed: Persistent caches, pre-pulled Docker images, warm NuGet/npm caches

Installing the Agent (Linux)

#!/bin/bash
# ═══════════════════════════════════════════════════════════════
# Azure DevOps Self-Hosted Agent Installation — Ubuntu/Debian
# Run this on the machine that will become your build agent.
# Prerequisites: curl, systemd, a PAT token with "Agent Pools (Read & Manage)" scope
# ═══════════════════════════════════════════════════════════════

# ─── Step 1: Create a dedicated agent user (don't run as root!) ───
sudo useradd -m -s /bin/bash azagent
sudo usermod -aG docker azagent    # Allow Docker access (if needed)

# ─── Step 2: Create agent directory ──────────────────────────────
sudo mkdir -p /opt/azure-agent
sudo chown azagent:azagent /opt/azure-agent
cd /opt/azure-agent

# ─── Step 3: Download the latest agent package ───────────────────
# Check https://github.com/microsoft/azure-pipelines-agent/releases
# for the latest version
AGENT_VERSION="3.240.1"
curl -LO "https://vstsagentpackage.azureedge.net/agent/${AGENT_VERSION}/vsts-agent-linux-x64-${AGENT_VERSION}.tar.gz"
tar xzf "vsts-agent-linux-x64-${AGENT_VERSION}.tar.gz"

# ─── Step 4: Configure the agent ────────────────────────────────
# This prompts for: server URL, PAT token, agent pool, agent name
sudo -u azagent ./config.sh \
  --unattended \
  --url https://dev.azure.com/YOUR_ORG \
  --auth pat \
  --token YOUR_PAT_TOKEN \
  --pool "Default" \
  --agent "$(hostname)" \
  --acceptTeeEula \
  --replace

# ─── Step 5: Install and start as a systemd service ─────────────
sudo ./svc.sh install azagent      # Install service (runs as azagent user)
sudo ./svc.sh start                # Start the agent service
sudo ./svc.sh status               # Verify it's running

# ─── Step 6: Verify in Azure DevOps ─────────────────────────────
# Go to: Project Settings → Agent Pools → Default → Agents tab
# Your agent should appear as "Online" with a green icon.
echo "Agent installed! Check Azure DevOps portal for status."

Agent Lifecycle Commands

Security Best Practices

• Never run as root: Create a dedicated azagent user with minimal permissions
• Network isolation: Place agents in a dedicated subnet with outbound-only access to Azure DevOps
• Rotate PAT tokens: Use short-lived tokens and rotate quarterly
• Restrict pool access: Use pipeline permissions to control which pipelines can use which pools
• Ephemeral agents: For maximum security, use VMSS-based auto-scaling agents that are destroyed after each job

Agent Pools & Capabilities

Agent pools are logical groups of agents that share the same configuration and purpose. They let you route specific types of work to specific sets of machines.

Default Pools

• Azure Pipelines: The built-in pool for Microsoft-hosted agents (ubuntu-latest, windows-latest, etc.)
• Default: The default pool for self-hosted agents — all self-hosted agents go here unless you create custom pools

Creating Custom Pools

Custom pools let you organize agents by capability, environment, or purpose:

• GPU-Agents — machines with NVIDIA GPUs for ML training
• Production-Deploy — agents in production VNET for deployments
• Linux-ARM64 — ARM-based agents for cross-compilation
• High-Memory — 64GB+ RAM agents for large .NET solutions

Capabilities & Demands

Every agent advertises its capabilities (installed software, hardware specs). Pipelines specify demands to match jobs to agents that have what they need.

# ═══════════════════════════════════════════════════════════════
# Capabilities and Demands — matching jobs to the right agents
# ═══════════════════════════════════════════════════════════════

# System capabilities are auto-detected:
#   Agent.OS = Linux
#   Agent.OSArchitecture = X64
#   dotnet = /usr/bin/dotnet
#   docker = /usr/bin/docker
#   node = /usr/local/bin/node

# User capabilities are manually added in Azure DevOps:
#   gpu = nvidia-a100
#   environment = production
#   disk-type = ssd

# ─── Pipeline demands: "I need an agent with Docker AND GPU" ──
pool:
  name: 'GPU-Agents'               # Use a specific self-hosted pool
  demands:
    - docker                        # Agent must have Docker installed
    - gpu -equals nvidia-a100       # Agent must have nvidia-a100 GPU
    - Agent.OS -equals Linux        # Must be Linux

jobs:
  - job: TrainModel
    steps:
      - script: |
          nvidia-smi                # Verify GPU is available
          python train.py --epochs 100
        displayName: 'Train ML model on GPU'

flowchart TD
    A[Pipeline Job] --> B{Pool Selection}
    B -->|"pool: vmImage: ubuntu-latest"| C[Azure Pipelines Pool]
    B -->|"pool: name: Default"| D[Default Pool]
    B -->|"pool: name: GPU-Agents"| E[GPU-Agents Pool]
    C --> F[MS-Hosted VM]
    D --> G{Demands Check}
    E --> H{Demands Check}
    G -->|Match| I[Agent picks job]
    G -->|No match| J[Job waits]
    H -->|Match| K[GPU Agent picks job]
    H -->|No match| L[Job waits]
                            

Pool Maintenance Tips

• Monitor offline agents: Set alerts for agents going offline — a dead agent delays all jobs in that pool
• Parallel job limits: Each pool has a concurrency limit — if all agents are busy, new jobs queue
• Agent version: Azure DevOps auto-updates agents, but verify all agents in a pool run the same version
• Maintenance jobs: Schedule weekly pipelines that clean up Docker images, old workspaces, and temp files on self-hosted agents

Build Task Ecosystem

Tasks are pre-built, versioned automation blocks that handle common CI/CD actions. Instead of writing shell scripts for everything, you use tasks — they handle cross-platform differences, error reporting, and integration with Azure DevOps features (test results, code coverage, artifacts).

Task Categories

Key Build Tasks

# ═══════════════════════════════════════════════════════════════
# Essential Build Tasks — the most commonly used in CI pipelines
# ═══════════════════════════════════════════════════════════════

steps:
  # ─── .NET Build Task ──────────────────────────────────────────
  - task: DotNetCoreCLI@2
    displayName: 'Restore NuGet packages'
    inputs:
      command: 'restore'              # restore, build, test, publish, pack, push
      projects: '**/*.csproj'         # Glob pattern for project files
      feedsToUse: 'select'            # Use feeds configured in NuGet.config

  # ─── Node.js Version Task ─────────────────────────────────────
  - task: NodeTool@0
    displayName: 'Install Node.js 20'
    inputs:
      versionSpec: '20.x'            # Semantic version range
      checkLatest: true              # Check for latest patch version

  # ─── Python Version Task ──────────────────────────────────────
  - task: UsePythonVersion@0
    displayName: 'Use Python 3.12'
    inputs:
      versionSpec: '3.12'
      addToPath: true                # Add to PATH for subsequent steps

  # ─── Go Version Task ──────────────────────────────────────────
  - task: GoTool@0
    displayName: 'Install Go 1.22'
    inputs:
      version: '1.22'

  # ─── Docker Build & Push ──────────────────────────────────────
  - task: Docker@2
    displayName: 'Build and push Docker image'
    inputs:
      containerRegistry: 'myACR'     # Service connection name
      repository: 'myapp'            # Image repository name
      command: 'buildAndPush'
      Dockerfile: '**/Dockerfile'
      tags: |
        $(Build.BuildId)
        latest

Task Control Options

# Every task supports these control options.
steps:
  - task: DotNetCoreCLI@2
    displayName: 'Run integration tests'
    inputs:
      command: 'test'
      projects: '**/*IntegrationTests.csproj'
    continueOnError: true            # Don't fail the job if this step fails
    timeoutInMinutes: 15             # Kill the step after 15 minutes
    retryCountOnTaskFailure: 2       # Retry up to 2 times on failure
    condition: eq(variables['runIntegrationTests'], 'true')
    env:
      CONNECTION_STRING: $(dbConnectionString)  # Map secret to env var

Script Tasks — When to Use Each

Caching & Performance Optimization

Microsoft-hosted agents are ephemeral — every job starts on a fresh VM. Without caching, you re-download hundreds of megabytes of dependencies every single build. The Cache@2 task solves this by persisting files between pipeline runs.

How Pipeline Caching Works

1. Cache key: A fingerprint (usually hash of lock files) that uniquely identifies the cache
2. Cache hit: If the key matches a stored cache, files are restored (~5 seconds)
3. Cache miss: If no match, the step runs normally — then saves the result for next time
4. Cache scope: Caches are scoped to the branch (with fallback to the default branch)

# ═══════════════════════════════════════════════════════════════
# Caching Examples — Node.js, .NET, Python, Go
# ═══════════════════════════════════════════════════════════════

steps:
  # ─── Node.js: Cache node_modules ──────────────────────────────
  - task: Cache@2
    displayName: 'Cache node_modules'
    inputs:
      key: 'npm | "$(Agent.OS)" | package-lock.json'
      path: '$(System.DefaultWorkingDirectory)/node_modules'
      # Key = "npm" + OS + hash of package-lock.json
      # If package-lock.json changes → cache invalidated → fresh install
      restoreKeys: |
        npm | "$(Agent.OS)"

  - script: npm ci                   # Skipped if cache hit is perfect
    displayName: 'Install Node dependencies'
    condition: ne(variables['CacheRestored'], 'true')

  # ─── .NET: Cache NuGet packages ───────────────────────────────
  - task: Cache@2
    displayName: 'Cache NuGet packages'
    inputs:
      key: 'nuget | "$(Agent.OS)" | **/packages.lock.json'
      path: '$(UserProfile)/.nuget/packages'
      restoreKeys: |
        nuget | "$(Agent.OS)"

  # ─── Python: Cache pip packages ───────────────────────────────
  - task: Cache@2
    displayName: 'Cache pip packages'
    inputs:
      key: 'pip | "$(Agent.OS)" | requirements.txt'
      path: '$(PIP_CACHE_DIR)'
      restoreKeys: |
        pip | "$(Agent.OS)"

  # ─── Go: Cache Go modules ────────────────────────────────────
  - task: Cache@2
    displayName: 'Cache Go modules'
    inputs:
      key: 'go | "$(Agent.OS)" | go.sum'
      path: '$(GOPATH)/pkg/mod'
      restoreKeys: |
        go | "$(Agent.OS)"

Cache Key Best Practices

• Always include Agent.OS — Linux and Windows packages aren't interchangeable
• Use lock file hashes — package-lock.json, packages.lock.json, requirements.txt, go.sum
• Use restoreKeys for partial matches — if the exact key misses, fall back to a broader match (slower restore but better than nothing)
• Don't cache build outputs — only cache dependencies. Build outputs change every commit and waste cache space

CI Pipeline: .NET Applications

A complete CI pipeline for .NET 8 applications — restore, build, test with code coverage, and publish deployable artifacts.

# ═══════════════════════════════════════════════════════════════
# .NET 8 CI Pipeline — Complete Production-Ready Example
# File: azure-pipelines.yml
# ═══════════════════════════════════════════════════════════════

trigger:
  branches:
    include: [main, release/*]
  paths:
    exclude: ['docs/**', '*.md']

pr:
  branches:
    include: [main]
  drafts: false

variables:
  buildConfiguration: 'Release'
  dotnetVersion: '8.0.x'

pool:
  vmImage: 'ubuntu-latest'

steps:
  # ─── Install .NET SDK ──────────────────────────────────────────
  - task: UseDotNet@2
    displayName: 'Install .NET $(dotnetVersion)'
    inputs:
      packageType: 'sdk'
      version: '$(dotnetVersion)'

  # ─── Cache NuGet packages ──────────────────────────────────────
  - task: Cache@2
    displayName: 'Cache NuGet'
    inputs:
      key: 'nuget | "$(Agent.OS)" | **/packages.lock.json'
      path: '$(UserProfile)/.nuget/packages'
      restoreKeys: nuget | "$(Agent.OS)"

  # ─── Restore dependencies ─────────────────────────────────────
  - task: DotNetCoreCLI@2
    displayName: 'Restore packages'
    inputs:
      command: 'restore'
      projects: '**/*.csproj'

  # ─── Build the solution ────────────────────────────────────────
  - task: DotNetCoreCLI@2
    displayName: 'Build solution'
    inputs:
      command: 'build'
      projects: '**/*.csproj'
      arguments: '--configuration $(buildConfiguration) --no-restore'

  # ─── Run tests with code coverage ─────────────────────────────
  - task: DotNetCoreCLI@2
    displayName: 'Run tests + coverage'
    inputs:
      command: 'test'
      projects: '**/*Tests.csproj'
      arguments: >-
        --configuration $(buildConfiguration)
        --no-build
        --collect:"XPlat Code Coverage"
        --logger trx
        --results-directory $(Agent.TempDirectory)/TestResults

  # ─── Publish test results to Azure DevOps ──────────────────────
  - task: PublishTestResults@2
    displayName: 'Publish test results'
    inputs:
      testResultsFormat: 'VSTest'
      testResultsFiles: '$(Agent.TempDirectory)/TestResults/**/*.trx'
    condition: succeededOrFailed()    # Publish even if tests fail

  # ─── Publish code coverage ─────────────────────────────────────
  - task: PublishCodeCoverageResults@2
    displayName: 'Publish code coverage'
    inputs:
      summaryFileLocation: '$(Agent.TempDirectory)/TestResults/**/coverage.cobertura.xml'

  # ─── Publish deployable artifact (main branch only) ────────────
  - task: DotNetCoreCLI@2
    displayName: 'Publish app'
    inputs:
      command: 'publish'
      projects: 'src/MyApp.Api/MyApp.Api.csproj'
      arguments: '--configuration $(buildConfiguration) --no-build --output $(Build.ArtifactStagingDirectory)'
    condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')

  - task: PublishPipelineArtifact@1
    displayName: 'Upload artifact'
    inputs:
      targetPath: '$(Build.ArtifactStagingDirectory)'
      artifactName: 'webapp'
    condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')

CI Pipeline: Node.js Applications

A Node.js CI pipeline with matrix strategy for testing across multiple Node versions simultaneously.

# ═══════════════════════════════════════════════════════════════
# Node.js CI Pipeline — Matrix Strategy (multiple versions)
# File: azure-pipelines.yml
# ═══════════════════════════════════════════════════════════════

trigger:
  branches:
    include: [main, develop]
  paths:
    exclude: ['docs/**', '*.md', '.vscode/**']

pr:
  branches:
    include: [main]

pool:
  vmImage: 'ubuntu-latest'

# ─── Matrix: Test on Node 18 AND Node 20 simultaneously ──────
strategy:
  matrix:
    Node18:
      nodeVersion: '18.x'
    Node20:
      nodeVersion: '20.x'

steps:
  # ─── Install specified Node.js version ─────────────────────────
  - task: NodeTool@0
    displayName: 'Install Node.js $(nodeVersion)'
    inputs:
      versionSpec: '$(nodeVersion)'

  # ─── Cache node_modules ────────────────────────────────────────
  - task: Cache@2
    displayName: 'Cache node_modules'
    inputs:
      key: 'npm | "$(Agent.OS)" | "$(nodeVersion)" | package-lock.json'
      path: '$(System.DefaultWorkingDirectory)/node_modules'
      restoreKeys: |
        npm | "$(Agent.OS)" | "$(nodeVersion)"

  # ─── Install dependencies (clean install) ──────────────────────
  - script: npm ci
    displayName: 'Install dependencies'

  # ─── Run linter ────────────────────────────────────────────────
  - script: npm run lint
    displayName: 'Lint code (ESLint)'

  # ─── Run tests with coverage ───────────────────────────────────
  - script: npm test -- --coverage --ci --reporters=default --reporters=jest-junit
    displayName: 'Run tests'
    env:
      JEST_JUNIT_OUTPUT_DIR: '$(System.DefaultWorkingDirectory)/test-results'

  # ─── Publish test results ──────────────────────────────────────
  - task: PublishTestResults@2
    displayName: 'Publish test results'
    inputs:
      testResultsFormat: 'JUnit'
      testResultsFiles: 'test-results/junit.xml'
    condition: succeededOrFailed()

  # ─── Build production bundle ───────────────────────────────────
  - script: npm run build
    displayName: 'Build production bundle'

  # ─── Publish artifact (main branch + Node 20 only) ─────────────
  - task: PublishPipelineArtifact@1
    displayName: 'Upload build artifact'
    inputs:
      targetPath: '$(System.DefaultWorkingDirectory)/dist'
      artifactName: 'webapp-$(nodeVersion)'
    condition: |
      and(
        succeeded(),
        eq(variables['Build.SourceBranch'], 'refs/heads/main'),
        eq(variables['nodeVersion'], '20.x')
      )

CI Pipeline: Python Applications

A Python CI pipeline with virtual environments, type checking (mypy), linting (ruff), and test coverage (pytest).

# ═══════════════════════════════════════════════════════════════
# Python CI Pipeline — Complete with linting, typing, testing
# File: azure-pipelines.yml
# ═══════════════════════════════════════════════════════════════

trigger:
  branches:
    include: [main, develop]
  paths:
    include: ['src/**', 'tests/**', 'requirements*.txt', 'pyproject.toml']

pr:
  branches:
    include: [main]

pool:
  vmImage: 'ubuntu-latest'

variables:
  pythonVersion: '3.12'
  PIP_CACHE_DIR: '$(Pipeline.Workspace)/.pip'

steps:
  # ─── Install Python ────────────────────────────────────────────
  - task: UsePythonVersion@0
    displayName: 'Use Python $(pythonVersion)'
    inputs:
      versionSpec: '$(pythonVersion)'
      addToPath: true

  # ─── Cache pip packages ────────────────────────────────────────
  - task: Cache@2
    displayName: 'Cache pip'
    inputs:
      key: 'pip | "$(Agent.OS)" | requirements.txt | requirements-dev.txt'
      path: '$(PIP_CACHE_DIR)'
      restoreKeys: pip | "$(Agent.OS)"

  # ─── Install dependencies ──────────────────────────────────────
  - script: |
      python -m pip install --upgrade pip
      pip install -r requirements.txt
      pip install -r requirements-dev.txt
    displayName: 'Install dependencies'

  # ─── Lint with Ruff (fast Python linter) ───────────────────────
  - script: |
      ruff check src/ tests/
      ruff format --check src/ tests/
    displayName: 'Lint & format check (Ruff)'

  # ─── Type check with mypy ─────────────────────────────────────
  - script: mypy src/ --ignore-missing-imports
    displayName: 'Type check (mypy)'

  # ─── Run tests with coverage ───────────────────────────────────
  - script: |
      pytest tests/ \
        --cov=src \
        --cov-report=xml:coverage.xml \
        --cov-report=html:htmlcov \
        --junitxml=test-results/results.xml \
        -v
    displayName: 'Run tests (pytest)'

  # ─── Publish test results ──────────────────────────────────────
  - task: PublishTestResults@2
    displayName: 'Publish test results'
    inputs:
      testResultsFormat: 'JUnit'
      testResultsFiles: 'test-results/results.xml'
    condition: succeededOrFailed()

  # ─── Publish code coverage ─────────────────────────────────────
  - task: PublishCodeCoverageResults@2
    displayName: 'Publish coverage'
    inputs:
      summaryFileLocation: 'coverage.xml'

  # ─── Build wheel for distribution ─────────────────────────────
  - script: |
      pip install build
      python -m build --wheel --outdir dist/
    displayName: 'Build wheel'
    condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')

  - task: PublishPipelineArtifact@1
    displayName: 'Upload wheel'
    inputs:
      targetPath: 'dist'
      artifactName: 'python-package'
    condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')

CI Pipeline: Container Builds

Building and pushing Docker images to Azure Container Registry (ACR) — the foundation for deploying to AKS, App Service, or Container Apps.

# ═══════════════════════════════════════════════════════════════
# Container CI Pipeline — Build, Scan, Push to ACR
# File: azure-pipelines.yml
# ═══════════════════════════════════════════════════════════════

trigger:
  branches:
    include: [main]
  paths:
    include: ['src/**', 'Dockerfile', 'docker-compose*.yml']

pr:
  branches:
    include: [main]

variables:
  acrName: 'mycompanyacr'
  imageName: 'myapp-api'
  # Tag with build ID for traceability + "latest" for convenience
  imageTag: '$(Build.BuildId)'

pool:
  vmImage: 'ubuntu-latest'

steps:
  # ─── Login to Azure Container Registry ────────────────────────
  - task: Docker@2
    displayName: 'Login to ACR'
    inputs:
      containerRegistry: 'acr-service-connection'  # Service connection name
      command: 'login'

  # ─── Build Docker image (multi-stage Dockerfile) ───────────────
  - task: Docker@2
    displayName: 'Build image'
    inputs:
      containerRegistry: 'acr-service-connection'
      repository: '$(imageName)'
      command: 'build'
      Dockerfile: 'Dockerfile'
      buildContext: '.'
      tags: |
        $(imageTag)
        latest
      arguments: '--build-arg BUILD_VERSION=$(Build.BuildNumber)'

  # ─── Scan image for vulnerabilities (Trivy) ───────────────────
  - script: |
      # Install Trivy scanner
      curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin

      # Scan the built image — fail on HIGH/CRITICAL vulnerabilities
      trivy image \
        --severity HIGH,CRITICAL \
        --exit-code 1 \
        --no-progress \
        $(acrName).azurecr.io/$(imageName):$(imageTag)
    displayName: 'Scan image (Trivy)'
    continueOnError: false           # Block push if vulnerabilities found

  # ─── Push image to ACR (only on main, after scan passes) ──────
  - task: Docker@2
    displayName: 'Push to ACR'
    inputs:
      containerRegistry: 'acr-service-connection'
      repository: '$(imageName)'
      command: 'push'
      tags: |
        $(imageTag)
        latest
    condition: |
      and(
        succeeded(),
        eq(variables['Build.SourceBranch'], 'refs/heads/main')
      )

  # ─── Output the deployed image reference ──────────────────────
  - script: |
      echo "##vso[task.setvariable variable=deployedImage;isOutput=true]$(acrName).azurecr.io/$(imageName):$(imageTag)"
      echo "Image pushed: $(acrName).azurecr.io/$(imageName):$(imageTag)"
    displayName: 'Set output variable'
    name: 'imageRef'
    condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')

Publishing Build Artifacts

Artifacts are the outputs of your build — compiled binaries, Docker images, test reports, or deployment packages. Publishing them makes outputs available to later stages (deployment) or for manual download.

Pipeline Artifacts vs Build Artifacts

Publishing & Downloading Artifacts

# ═══════════════════════════════════════════════════════════════
# Artifact Publishing & Consumption Across Stages
# ═══════════════════════════════════════════════════════════════

stages:
  # ─── Stage 1: Build & publish artifacts ────────────────────────
  - stage: Build
    jobs:
      - job: BuildApp
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - script: dotnet publish -o $(Build.ArtifactStagingDirectory)/app
            displayName: 'Build app'

          # Publish using the task (full control)
          - task: PublishPipelineArtifact@1
            displayName: 'Publish app artifact'
            inputs:
              targetPath: '$(Build.ArtifactStagingDirectory)/app'
              artifactName: 'webapp'          # Name for download reference
              publishLocation: 'pipeline'

          # Also publish infrastructure scripts
          - task: PublishPipelineArtifact@1
            displayName: 'Publish infra scripts'
            inputs:
              targetPath: '$(System.DefaultWorkingDirectory)/infra'
              artifactName: 'infrastructure'

  # ─── Stage 2: Deploy (downloads artifacts from Build stage) ────
  - stage: Deploy
    dependsOn: Build
    jobs:
      - job: DeployApp
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          # Download the artifact published in the Build stage
          - task: DownloadPipelineArtifact@2
            displayName: 'Download app artifact'
            inputs:
              artifactName: 'webapp'
              targetPath: '$(Pipeline.Workspace)/webapp'

          - task: DownloadPipelineArtifact@2
            displayName: 'Download infra scripts'
            inputs:
              artifactName: 'infrastructure'
              targetPath: '$(Pipeline.Workspace)/infra'

          # Use the downloaded files
          - script: |
              ls -la $(Pipeline.Workspace)/webapp/
              echo "Deploying from artifact..."
            displayName: 'Deploy application'

Artifact Naming Conventions

• webapp — main application binary/package
• webapp-$(Build.BuildId) — versioned artifact for traceability
• test-results — test reports and coverage files
• infrastructure — Terraform/Bicep/ARM templates
• docker-compose — container orchestration files

Exercises