Argo CD Notifications
Installation
# Install notifications controller (included in recent Argo CD versions)
# Check if already installed:
kubectl get deployment argocd-notifications-controller -n argocd
# If not present, install separately:
kubectl apply -n argocd -f \
https://raw.githubusercontent.com/argoproj/argo-cd/stable/notifications_catalog/install.yaml
# Verify pods:
kubectl get pods -n argocd -l app.kubernetes.io/name=argocd-notifications-controller
Triggers & Templates
# argocd-notifications-cm — configure triggers, templates, and services
apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-notifications-cm
namespace: argocd
data:
# ── TEMPLATES ────────────────────────────────────────────────────
# What message to send (supports Sprig templating)
template.app-deployed: |
message: |
Application {{.app.metadata.name}} is now running version {{.app.spec.source.targetRevision}}.
slack:
attachments: |
[{
"color": "#18be52",
"title": "{{.app.metadata.name}}",
"title_link": "{{.context.argocdUrl}}/applications/{{.app.metadata.name}}",
"fields": [{
"title": "Status",
"value": "Synced",
"short": true
}, {
"title": "Repo",
"value": "{{.app.spec.source.repoURL}}",
"short": true
}, {
"title": "Revision",
"value": "{{.app.spec.source.targetRevision}}",
"short": true
}]
}]
template.app-sync-failed: |
message: |
FAILED: Application {{.app.metadata.name}} sync failed.
Error: {{.app.status.operationState.message}}
slack:
attachments: |
[{
"color": "#ff0000",
"title": "Sync Failed: {{.app.metadata.name}}",
"title_link": "{{.context.argocdUrl}}/applications/{{.app.metadata.name}}",
"fields": [{
"title": "Error",
"value": "{{.app.status.operationState.message}}",
"short": false
}]
}]
template.app-health-degraded: |
message: |
WARNING: Application {{.app.metadata.name}} health status is {{.app.status.health.status}}.
slack:
attachments: |
[{
"color": "#f4c030",
"title": "Health Degraded: {{.app.metadata.name}}",
"fields": [{
"title": "Health Status",
"value": "{{.app.status.health.status}}",
"short": true
}]
}]
# ── TRIGGERS ─────────────────────────────────────────────────────
# When to send a notification
trigger.on-deployed: |
- when: app.status.operationState.phase in ['Succeeded'] and app.status.health.status == 'Healthy'
send: [app-deployed]
trigger.on-sync-failed: |
- when: app.status.operationState.phase in ['Error', 'Failed']
send: [app-sync-failed]
trigger.on-health-degraded: |
- when: app.status.health.status == 'Degraded'
send: [app-health-degraded]
# ── SERVICES ─────────────────────────────────────────────────────
# Configured per integration type (Slack, Teams, email, webhook)
service.slack: |
token: $slack-token
username: ArgoCD
icon: ":argo:"
Slack Setup
# 1. Create Slack App at api.slack.com/apps
# - Add OAuth scopes: chat:write, chat:write.public
# - Install to workspace
# - Copy Bot User OAuth Token (xoxb-...)
# 2. Store token in Kubernetes secret
kubectl create secret generic argocd-notifications-secret \
--from-literal=slack-token=xoxb-YOUR-TOKEN-HERE \
-n argocd
# OR: add to existing argocd-notifications-secret
kubectl patch secret argocd-notifications-secret \
-n argocd \
--type='json' \
-p='[{"op":"add","path":"/data/slack-token","value":"'$(echo -n 'xoxb-...' | base64)'"}]'
# Annotate Applications to subscribe to notifications
# Option 1: On the Application itself
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: grade-api-prod
namespace: argocd
annotations:
# Subscribe to triggers, send to slack channel
notifications.argoproj.io/subscribe.on-deployed.slack: "deployments"
notifications.argoproj.io/subscribe.on-sync-failed.slack: "deployments"
notifications.argoproj.io/subscribe.on-health-degraded.slack: "alerts-critical"
spec:
# ... rest of Application spec
# Option 2: Subscribe via CLI
argocd app set grade-api-prod \
--annotations "notifications.argoproj.io/subscribe.on-deployed.slack=deployments"
# Test trigger manually
argocd app notify grade-api-prod --trigger on-deployed
Webhook Notifications
# argocd-notifications-cm — add webhook service for GitHub PR status, Jira, etc.
data:
service.webhook.github: |
url: https://api.github.com
headers:
- name: Authorization
value: token $github-token
- name: Content-Type
value: application/json
template.github-commit-status: |
webhook:
github:
method: POST
path: /repos/{{call .repo.FullNameByRepoURL .app.spec.source.repoURL}}/statuses/{{.app.status.sync.revision}}
body: |
{
"state": "{{if eq .app.status.health.status "Healthy"}}success{{else}}failure{{end}}",
"description": "ArgoCD",
"target_url": "{{.context.argocdUrl}}/applications/{{.app.metadata.name}}",
"context": "continuous-delivery/argocd"
}
trigger.sync-operation-change: |
- when: app.status.operationState.phase in ['Succeeded'] and app.status.health.status == 'Healthy'
send: [github-commit-status]
Argo CD Image Updater
Argo CD Image Updater watches container registries for new image tags. When a new tag matching your policy appears, it either updates the Application directly or commits the new tag back to Git — completing the loop between CI and CD.
Full GitOps Loop with Image Updater
flowchart LR
DEV[Developer
git push] -->|triggers| CI[CI Pipeline
GitHub Actions]
CI -->|builds & pushes| REG[Container Registry
ghcr.io]
REG -->|new tag detected| IU[Image Updater]
IU -->|commits tag to| GIT[Git Repo
kustomization.yaml]
GIT -->|Argo CD detects diff| ACD[Argo CD]
ACD -->|syncs| K8S[Kubernetes Cluster]
style DEV fill:#132440,color:#fff
style CI fill:#3B9797,color:#fff
style REG fill:#16476A,color:#fff
style IU fill:#BF092F,color:#fff
style GIT fill:#3B9797,color:#fff
style ACD fill:#132440,color:#fff
style K8S fill:#16476A,color:#fff
Installation & Config
# Install Image Updater
kubectl apply -n argocd -f \
https://raw.githubusercontent.com/argoproj-labs/argocd-image-updater/stable/manifests/install.yaml
# Verify:
kubectl get pods -n argocd -l app.kubernetes.io/name=argocd-image-updater
# Configure registry access (for private registries)
# argocd-image-updater-config ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-image-updater-config
namespace: argocd
data:
# Log level: trace, debug, info, warn, error
log.level: info
# Registry configuration
registries.conf: |
registries:
- name: GitHub Container Registry
api_url: https://ghcr.io
prefix: ghcr.io
credentials: secret:argocd/ghcr-credentials#.dockerconfigjson
credsexpire: 10h
- name: AWS ECR
api_url: https://012345678901.dkr.ecr.us-east-1.amazonaws.com
prefix: 012345678901.dkr.ecr.us-east-1.amazonaws.com
credentials: ext:/path/to/ecr-auth-script.sh
credsexpire: 12h
SemVer Update Strategy
# Annotate Application to use Image Updater
# SemVer strategy: update to highest semver matching constraint
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: grade-api-dev
namespace: argocd
annotations:
# Image list: alias=registry/repo
argocd-image-updater.argoproj.io/image-list: >
grade-api=ghcr.io/your-org/grade-api
# Update strategy: semver constraint
argocd-image-updater.argoproj.io/grade-api.update-strategy: semver
argocd-image-updater.argoproj.io/grade-api.allow-tags: "regexp:^v[0-9]+\\.[0-9]+\\.[0-9]+$"
# Write back to Git (recommended)
argocd-image-updater.argoproj.io/write-back-method: git
argocd-image-updater.argoproj.io/git-branch: main
# For Kustomize apps: update kustomization.yaml
argocd-image-updater.argoproj.io/grade-api.kustomize.image-name: ghcr.io/your-org/grade-api
# For Helm apps: update values.yaml key
# argocd-image-updater.argoproj.io/grade-api.helm.image-name: image.repository
# argocd-image-updater.argoproj.io/grade-api.helm.image-tag: image.tag
spec:
source:
repoURL: https://github.com/your-org/grade-api-gitops
targetRevision: main
path: kustomize/overlays/dev
Digest Strategy
# Digest strategy: always pull and track the latest digest for a fixed tag
# Useful for 'latest', 'stable', 'edge' tags where you want immutable deploys
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: grade-api-dev
namespace: argocd
annotations:
argocd-image-updater.argoproj.io/image-list: >
grade-api=ghcr.io/your-org/grade-api:latest
# Digest strategy: track SHA digest of the 'latest' tag
argocd-image-updater.argoproj.io/grade-api.update-strategy: digest
argocd-image-updater.argoproj.io/write-back-method: git
Write-Back Methods
# Write-back method 1: 'argocd' — updates Application spec directly
# Pros: instant, no Git commit
# Cons: drift between Git and cluster, not truly GitOps
# Write-back method 2: 'git' — commits updated tag to Git repo
# Pros: fully GitOps, auditable, rollback via git revert
# Cons: requires Git credentials
# For git write-back, create deploy key:
ssh-keygen -t ed25519 -C "argocd-image-updater" -f ./image-updater-key
# Add public key as deploy key to your Git repo (with write access)
# Create secret in Kubernetes:
kubectl create secret generic git-creds-grade-api \
--from-file=sshPrivateKey=./image-updater-key \
-n argocd
# Reference in Application annotation:
# argocd-image-updater.argoproj.io/write-back-method: git:secret:argocd/git-creds-grade-api
# When Image Updater commits to Git, it creates a file in the app path:
# .argocd-source-grade-api-dev.yaml
# This file overrides the image tag in the Application's source config
# Argo CD detects this change and syncs automatically
# Example committed file content:
kustomize:
images:
- ghcr.io/your-org/grade-api:v2.3.1 # Updated from v2.3.0
# For Helm:
helm:
parameters:
- name: image.tag
value: v2.3.1
The Complete GitOps Loop
# The automation loop we've built across this 5-part track:
# 1. Developer pushes code to application repo
git add . ; git commit -m "feat: add grade calculation endpoint"
git push origin main
# 2. CI pipeline triggers (GitHub Actions, GitLab CI, etc.)
# - Build Docker image
# - Run tests
# - Push image: ghcr.io/your-org/grade-api:v2.4.0
# 3. Image Updater detects new tag (within ~2min polling interval)
# - Matches against semver constraint ^v2
# - Commits new tag to GitOps repo
# git commit: "build: automatic update of grade-api to v2.4.0"
# 4. Argo CD detects Git change in GitOps repo
# - Compares desired state (new tag) with live state (old tag)
# - Reports: OutOfSync
# 5a. Auto-sync: Argo CD syncs automatically (if enabled)
# 5b. Manual sync: Developer/operator runs: argocd app sync grade-api-dev
# 6. Notifications fire:
# - Slack #deployments: "grade-api-dev deployed v2.4.0 ✓"
# - GitHub commit status: green checkmark on the CI commit
# 7. Health check: Argo CD monitors rollout
# - If Degraded: fires on-health-degraded trigger to #alerts-critical
# - If Healthy: sync window closes, everything quiet until next push
Exercises
Exercise 1 — Notifications: Install the notifications controller. Create a template that sends a minimal Slack message (application name + sync status) using the in-cluster webhook service type instead of Slack (use a free webhook.site URL so you can see the payload). Annotate an Application to subscribe and trigger a manual notification via CLI.
Exercise 2 — Image Updater (argocd write-back): Install Image Updater. Annotate a Kustomize-based Application with the
semver strategy pointing to a public image (e.g., nginx). Use the argocd write-back method to avoid needing Git credentials. Force a check and observe what Image Updater does in its logs.
Exercise 3 — Full Loop Simulation: Push a new image tag to a registry (e.g., GitHub Packages) using any small change. Confirm Image Updater detects it, commits to GitOps repo, and Argo CD auto-syncs. Observe the notification arrive. Time the entire loop from push to healthy deployment.
Track Completion & Next Steps
Argo CD Track Complete! You've now covered:
- Part 1: Core GitOps concepts, Argo CD architecture, installation with HA considerations
- Part 2: Application CRD anatomy, sync strategies, sync windows, health checks
- Part 3: App-of-Apps pattern, ApplicationSets for fleet management, sync waves for ordered rollouts
- Part 4: AppProjects for multi-tenancy, RBAC with Casbin policies, SSO with GitHub/Okta
- Part 5: Notifications for observability, Image Updater to close the GitOps loop
What to explore next:
- Combine Argo CD with Helm Track — manage Argo CD-deployed apps using Helm charts with multi-environment value files
- Explore Flux Track for an alternative pull-based GitOps model built on controllers rather than a central UI
- Use Argo Rollouts (separate CRD, extends Argo CD) for progressive delivery: canary, blue-green, analysis templates
- Integrate with the Vault Track (Part 14) to inject secrets into Argo CD-managed apps via ESO