BlogMonitoring as Code Guide
🏗️ DevOps Guide · 11 min read

Monitoring as Code: Complete Guide 2026

Monitoring as code (MaC) means managing your monitors, alerts, dashboards, and on-call schedules as version-controlled configuration files — just like you manage your infrastructure. Here's how to implement it.

Last updated: April 2026·By API Status Check Team
Staff Pick

📡 Monitor your APIs — know when they go down before your users do

Better Stack checks uptime every 30 seconds with instant Slack, email & SMS alerts. Free tier available.

Start Free →

Affiliate link — we may earn a commission at no extra cost to you

🚀 The Problem Monitoring as Code Solves

Most teams manage monitoring through a web UI — clicking around to create monitors, setting thresholds manually, building dashboards one panel at a time. The result:

Monitoring as code solves all of these by treating observability configuration as a first-class engineering artifact.

What Is Monitoring as Code?

Monitoring as code (MaC) is the practice of defining your entire monitoring configuration in machine-readable files that live in a Git repository. This includes:

Changes to monitoring are made through pull requests, reviewed by peers, tested in CI, and deployed automatically — the same workflow used for application code and infrastructure.

6 Benefits of Monitoring as Code

🔀

Version control for every change

Every alert threshold change, new monitor, and dashboard edit is captured in Git. Review who changed what, when, and why — and roll back in 30 seconds if a change causes problems.

🔍

Peer review prevents mistakes

Alert changes go through pull request review before reaching production. Catch misconfigured thresholds, missing runbook links, and wrong severity levels before they cause missed incidents.

♻️

Reproducible across environments

Deploy identical monitoring to dev, staging, and production. No more "production has alerts that staging doesn't" or "we set that up manually 2 years ago."

🚀

Deploy monitoring with your application

Add new monitors automatically when you deploy a new service. Include monitoring configuration in your service template so every new service starts with baseline coverage.

📖

Code IS the documentation

No more mystery alerts that nobody understands. The code explains the threshold, the annotations explain the context, and Git history explains why it was set that way.

💥

Disaster recovery in minutes

If your monitoring platform has an incident or you need to migrate providers, recreate all your monitoring from scratch from the code in your repository.

📡
Recommended

Monitor your services before your users notice

Try Better Stack Free →

Prometheus Alert Rules as Code

Prometheus alert rules are natively YAML files, making them the simplest entry point into monitoring as code. Store them in Git, validate in CI, and deploy via Kubernetes ConfigMaps or Helm charts.

# alerting-rules.yaml — committed to your repo
groups:
- name: api-availability
interval: 30s
rules:
- alert: HighAPIErrorRate
expr: sum(rate(http_requests_total{status=~"5..",service="api"}[5m]))
/ sum(rate(http_requests_total{service="api"}[5m])) > 0.01
for: 5m
labels:
severity: critical
team: platform
annotations:
summary: "API error rate above 1% for 5 minutes"
description: "Error rate is {{ $value | humanizePercentage }}"
runbook_url: "https://runbooks.company.com/high-api-errors"
# Validate in CI before merging
promtool check rules alerting-rules.yaml
# Unit test alert rules
promtool test rules tests/alerting-rules-test.yaml

Terraform for Datadog Monitoring

The Datadog Terraform provider lets you manage monitors, dashboards, SLOs, and synthetics as Terraform resources. This is one of the most popular monitoring-as-code patterns for teams using Datadog.

# main.tf — Datadog monitor as Terraform
resource
"datadog_monitor" "api_error_rate" {
name = "API Error Rate Above 1%"
type = "metric alert"
message = "API error rate above 1%. @pagerduty-platform-team"
query = "sum(last_5m):sum:trace.http.request.errors{service:api}.as_rate() > 0.01"
thresholds {
critical = 0.01
warning = 0.005
}
tags = ["service:api", "team:platform", "env:production"]
require_full_window = true
}
# SLO as code
resource
"datadog_service_level_objective" "api_availability" {
name = "API Availability SLO"
type = "monitor"
monitor_ids = [datadog_monitor.api_error_rate.id]
thresholds {
timeframe = "30d"
target = 99.9
}
}

PagerDuty On-Call Configuration as Code

The PagerDuty Terraform provider manages your escalation policies, schedules, and service integrations in code. This is especially valuable when you need to onboard new teams consistently or replicate on-call structures across multiple regions.

# pagerduty.tf — escalation policy as code
resource
"pagerduty_escalation_policy" "platform_team" {
name = "Platform Team Escalation"
num_loops = 2
teams = [pagerduty_team.platform.id]
rule {
escalation_delay_in_minutes = 10
target {
type = "schedule_reference"
id = pagerduty_schedule.primary.id
}
}
rule {
escalation_delay_in_minutes = 15
target {
type = "user_reference"
id = pagerduty_user.oncall_manager.id
}
}
}

Grafana Dashboards as Code

Grafana dashboards are JSON at their core, but raw JSON is hard to maintain. Two better approaches:

Option 1: Grafana Terraform Provider

Use the grafana/grafana Terraform provider to manage dashboards, data sources, alert rules, and folders. Best if you're already using Terraform for infrastructure.

resource "grafana_dashboard" "api_overview" {
config_json = file("./dashboards/api-overview.json")
folder = grafana_folder.platform.id
}

Option 2: Grafonnet (Jsonnet)

Grafonnet is a Jsonnet library that lets you generate Grafana dashboard JSON programmatically with full type safety and reusable components. Better for complex dashboards with many panels that share common patterns.

CI/CD Pipeline for Monitoring Changes

A complete monitoring-as-code CI/CD pipeline validates changes before they reach production:

# .github/workflows/monitoring.yaml
on: [pull_request, push]
jobs:
validate-monitoring:
steps:
- name: Validate Prometheus rules
run: promtool check rules monitoring/rules/*.yaml
- name: Unit test alert rules
run: promtool test rules monitoring/tests/*.yaml
- name: Validate Alertmanager config
run: amtool check-config monitoring/alertmanager.yaml
- name: Terraform plan (dry run)
run: terraform plan -out=tfplan
- name: Terraform apply (on merge to main)
if: github.ref == 'refs/heads/main'
run: terraform apply tfplan

Monitoring Platforms with Best Code Support

Prometheus + Alertmanager

⭐⭐⭐⭐⭐

Natively code-first. Rules are YAML files. Config is YAML. First-class IaC citizen.

N/A — config files directly

Grafana

⭐⭐⭐⭐⭐

Dashboards are JSON. Terraform provider, Grafonnet (Jsonnet), and Grafana CLI. Best-in-class MaC support.

grafana/grafana provider

Datadog

⭐⭐⭐⭐⭐

Comprehensive Terraform provider covers monitors, SLOs, synthetics, dashboards, and alert policies.

hashicorp/datadog provider

Better Stack

⭐⭐⭐⭐

Full REST API for managing monitors and on-call. Terraform provider in beta. Great for uptime check automation.

API-driven (Terraform provider beta)

PagerDuty

⭐⭐⭐⭐

Mature Terraform provider for services, schedules, escalation policies, and integrations.

PagerDuty/pagerduty provider

New Relic

⭐⭐⭐⭐

Good Terraform provider covers alert policies, conditions, dashboards, and synthetic monitors.

newrelic/newrelic provider

OpsGenie

⭐⭐⭐

Terraform provider available. Covers teams, routing, integrations. Less comprehensive than PagerDuty.

opsgenie/opsgenie provider

🛠 Tools We Use & Recommend

Tested across our own infrastructure monitoring 200+ APIs daily

Better StackBest for API Teams

Uptime Monitoring & Incident Management

Used by 100,000+ websites

Monitors your APIs every 30 seconds. Instant alerts via Slack, email, SMS, and phone calls when something goes down.

We use Better Stack to monitor every API on this site. It caught 23 outages last month before users reported them.

Free tier · Paid from $24/moStart Free Monitoring
1PasswordBest for Credential Security

Secrets Management & Developer Security

Trusted by 150,000+ businesses

Manage API keys, database passwords, and service tokens with CLI integration and automatic rotation.

After covering dozens of outages caused by leaked credentials, we recommend every team use a secrets manager.

OpteryBest for Privacy

Automated Personal Data Removal

Removes data from 350+ brokers

Removes your personal data from 350+ data broker sites. Protects against phishing and social engineering attacks.

Service outages sometimes involve data breaches. Optery keeps your personal info off the sites attackers use first.

From $9.99/moFree Privacy Scan
ElevenLabsBest for AI Voice

AI Voice & Audio Generation

Used by 1M+ developers

Text-to-speech, voice cloning, and audio AI for developers. Build voice features into your apps with a simple API.

The best AI voice API we've tested — natural-sounding speech with low latency. Essential for any app adding voice features.

Free tier · Paid from $5/moTry ElevenLabs Free
SEMrushBest for SEO

SEO & Site Performance Monitoring

Used by 10M+ marketers

Track your site health, uptime, search rankings, and competitor movements from one dashboard.

We use SEMrush to track how our API status pages rank and catch site health issues early.

From $129.95/moTry SEMrush Free
View full comparison & more tools →Affiliate links — we earn a commission at no extra cost to you

Getting Started: Practical Steps

1
Export existing configuration
Use your monitoring platform's export feature or API to export current monitors and dashboards as JSON/YAML. This is your baseline.
2
Create a monitoring repo
Create a dedicated repo (or a monitoring/ directory in your infra repo) for monitoring configuration. Add .gitignore for secrets and Terraform state.
3
Start with critical alerts only
Don't try to codify everything at once. Start with your most important production alerts. Get the workflow working before expanding.
4
Add CI validation
Add promtool, terraform validate, and amtool checks to your CI pipeline. Any monitoring PR that fails validation should be blocked from merging.
5
Migrate incrementally
Gradually move existing monitors from the UI to code. Delete the UI-managed version only after the code-managed version is confirmed working.
6
Enforce code-only going forward
Once you have the workflow working, make a team agreement: no new monitors via UI. All changes via code. Gradually sunset legacy UI-managed monitors.

Alert Pro

14-day free trial

Stop checking — get alerted instantly

Next time your monitoring service goes down, you'll know in under 60 seconds — not when your users start complaining.

  • Email alerts for your monitoring service + 9 more APIs
  • $0 due today for trial
  • Cancel anytime — $9/mo after trial

Frequently Asked Questions

What is monitoring as code?

Monitoring as code (MaC) is the practice of defining and managing your monitoring configuration — including uptime checks, alert rules, dashboards, and on-call schedules — as version-controlled code files rather than through a web UI. The same principles that drive infrastructure as code (IaC) apply: configurations stored in Git, reviewed via pull requests, deployed via CI/CD, and rolled back when something goes wrong. Popular tools include Terraform providers for Datadog, PagerDuty, and Grafana; Prometheus alert rules in YAML; and Pulumi for platforms with SDKs.

What are the benefits of monitoring as code?

Benefits of monitoring as code: (1) Version control — every alert change is reviewed, attributed, and reversible. Know exactly who changed a threshold and why. (2) Consistency — deploy identical monitoring across staging, production, and multiple regions. (3) Peer review — alert changes go through PR review, catching mistakes before they cause missed incidents. (4) Disaster recovery — recreate all your monitoring from scratch in minutes after a platform migration or incident. (5) Documentation — the code IS the documentation. No more undocumented alerts that nobody understands. (6) Testing — validate alert configurations in CI before deploying.

How do I manage Prometheus alerts as code?

Prometheus alert rules are already YAML files, making them natively code-friendly. Store your alerting rules in a Git repository, validate them with `promtool check rules`, and deploy them via Helm or ConfigMaps in Kubernetes. Use a rules structure like: groups > name > rules > alert/expr/for/labels/annotations. Add a runbook_url annotation to every alert linking to your incident runbook. Automate validation in CI: run promtool check rules on every PR to catch syntax errors before they reach production.

What Terraform providers exist for monitoring?

The main Terraform providers for monitoring as code are: (1) hashicorp/datadog — manages Datadog monitors, dashboards, SLOs, and synthetics. Most widely used. (2) PagerDuty/pagerduty — manages services, escalation policies, schedules, and alert routing. (3) grafana/grafana — manages Grafana dashboards, alert rules, data sources, and folders. (4) newrelic/newrelic — manages New Relic alert policies, conditions, and dashboards. (5) Better Stack — manages uptime monitors and on-call configuration via API (Terraform provider in development). (6) Prometheus Operator — manages Prometheus rules and Alertmanager configs as Kubernetes CRDs.

Can I test monitoring configurations before deploying?

Yes, and you should. Testing approaches for monitoring as code: (1) Prometheus: use promtool check rules to validate YAML syntax and expression correctness. Unit test alert rules with promtool test rules and a test fixture file. (2) Terraform: use terraform validate and terraform plan to preview changes before applying. (3) Alertmanager: use amtool check-config to validate routing configuration. (4) Grafana: Grafonnet (Jsonnet library) lets you generate and validate dashboard JSON programmatically. (5) Integration testing: deploy to a staging environment and fire synthetic alerts to verify routing before promoting to production.

Related Guides