Rust's performance characteristics make it an increasingly popular choice for high-throughput services, APIs, and system-level applications. But high performance without observability is a liability — you need to know what's happening inside your Rust services in production. This guide covers the complete Rust observability stack: metrics, distributed tracing, structured logging, error tracking, and uptime monitoring.
The Rust Observability Stack
Rust's observability ecosystem has matured significantly. The three pillars of observability — metrics, traces, and logs — are well-supported through a combination of first-party crates and the OpenTelemetry standard:
| Pillar | Primary Crate | Backend |
|---|---|---|
| Metrics | metrics / metrics-exporter-prometheus | Prometheus + Grafana |
| Distributed Tracing | tracing + opentelemetry-otlp | Jaeger / Grafana Tempo / Honeycomb |
| Structured Logging | tracing-subscriber (JSON) | Loki / CloudWatch / Datadog |
| Error Tracking | sentry (sentry-rust) | Sentry.io |
| Uptime / API Health | reqwest + tokio (health check endpoint) | Better Stack / APIStatusCheck |
1. Metrics with Prometheus
The metrics crate provides a facade that decouples your application from the specific metrics backend. This means you instrument once and switch backends without changing application code.
Setup: Cargo.toml
[dependencies]
metrics = "0.23"
metrics-exporter-prometheus = "0.15"
axum = "0.7"
tokio = { version = "1", features = ["full"] }Initializing the Prometheus Exporter
use metrics_exporter_prometheus::PrometheusBuilder;
use std::net::SocketAddr;
pub fn init_metrics() -> PrometheusHandle {
PrometheusBuilder::new()
.with_http_listener("0.0.0.0:9000".parse::<SocketAddr>().unwrap())
.install()
.expect("failed to install Prometheus exporter")
}Recording Metrics in Application Code
use metrics::{counter, gauge, histogram};
async fn handle_request(req: Request) -> Response {
let start = std::time::Instant::now();
// Increment request counter with labels
counter!("http_requests_total", "method" => req.method().as_str().to_owned(),
"path" => req.uri().path().to_owned()).increment(1);
let response = process_request(req).await;
// Record latency histogram
let duration = start.elapsed().as_secs_f64();
histogram!("http_request_duration_seconds",
"status" => response.status().as_str().to_owned())
.record(duration);
response
}
// Track active connections
fn update_active_connections(count: f64) {
gauge!("active_connections").set(count);
}Monitor your Rust service health endpoints
Better Stack provides uptime monitoring for any HTTP endpoint with 30-second check intervals. Ideal for Rust services exposing /health or /metrics endpoints.
Try Better Stack Free →2. Distributed Tracing with OpenTelemetry
The tracing crate is Rust's standard instrumentation library, and tracing-opentelemetry bridges it to the OpenTelemetry ecosystem.
Setup: Cargo.toml
[dependencies]
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
tracing-opentelemetry = "0.27"
opentelemetry = "0.26"
opentelemetry-otlp = { version = "0.26", features = ["http-proto", "reqwest-client"] }
opentelemetry_sdk = { version = "0.26", features = ["rt-tokio"] }Initializing the Tracing Pipeline
use opentelemetry::global;
use opentelemetry_otlp::WithExportConfig;
use opentelemetry_sdk::runtime;
use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt};
pub fn init_tracing(service_name: &str, otlp_endpoint: &str) {
let tracer = opentelemetry_otlp::new_pipeline()
.tracing()
.with_exporter(
opentelemetry_otlp::new_exporter()
.http()
.with_endpoint(otlp_endpoint),
)
.with_trace_config(
opentelemetry_sdk::trace::Config::default()
.with_resource(opentelemetry_sdk::Resource::new(vec![
opentelemetry::KeyValue::new("service.name", service_name.to_string()),
])),
)
.install_batch(runtime::Tokio)
.expect("failed to initialize tracer");
let otel_layer = tracing_opentelemetry::layer().with_tracer(tracer);
tracing_subscriber::registry()
.with(tracing_subscriber::EnvFilter::from_default_env())
.with(tracing_subscriber::fmt::layer().json())
.with(otel_layer)
.init();
}Instrumenting Functions
use tracing::{info, instrument, Span};
// Auto-creates a span for the function
#[instrument(fields(user_id = %user_id, email = tracing::field::Empty))]
async fn process_order(user_id: u64, order: Order) -> Result<OrderResult, Error> {
// Add dynamic fields to the current span
Span::current().record("email", tracing::field::display(&order.email));
info!(order_id = %order.id, "processing order");
let result = database.save_order(&order).await?;
info!(order_result = ?result, "order saved");
Ok(result)
}
// Manual span creation
async fn external_api_call(url: &str) -> Result<Response, Error> {
let span = tracing::info_span!("external_api_call", url = %url);
let _guard = span.enter();
// All code here is within the span
reqwest::get(url).await
}3. Structured Logging
The tracing-subscriber crate's JSON formatter outputs machine-parseable logs ideal for log aggregation systems like Grafana Loki, Datadog, or AWS CloudWatch.
use tracing::{debug, error, info, warn};
// Structured fields — machine-parseable by log aggregators
info!(
user_id = 12345,
action = "login",
ip = "192.168.1.1",
duration_ms = 45,
"user authenticated"
);
// Error with context — captured automatically when using #[instrument]
error!(
error = %e,
request_id = %req_id,
"database connection failed"
);JSON output from tracing-subscriber looks like:
{
"timestamp": "2026-05-04T09:00:00Z",
"level": "INFO",
"target": "my_service::auth",
"fields": {
"message": "user authenticated",
"user_id": 12345,
"action": "login",
"ip": "192.168.1.1",
"duration_ms": 45
},
"span": {
"name": "handle_request",
"request_id": "abc-123"
}
}📡 Monitor your Rust service uptime every 30 seconds — get alerted in under a minute
Trusted by 100,000+ websites · Free tier available
4. Health Check Endpoints
Every production Rust service should expose a health check endpoint for uptime monitoring. Here's a standard implementation using Axum:
use axum::{extract::State, http::StatusCode, response::Json, routing::get, Router};
use serde::Serialize;
#[derive(Serialize)]
struct HealthResponse {
status: &'static str,
version: &'static str,
db: bool,
uptime_seconds: u64,
}
async fn health_check(State(state): State<AppState>) -> (StatusCode, Json<HealthResponse>) {
let db_ok = state.db.ping().await.is_ok();
let body = HealthResponse {
status: if db_ok { "ok" } else { "degraded" },
version: env!("CARGO_PKG_VERSION"),
db: db_ok,
uptime_seconds: state.started_at.elapsed().as_secs(),
};
let status = if db_ok { StatusCode::OK } else { StatusCode::SERVICE_UNAVAILABLE };
(status, Json(body))
}
pub fn app(state: AppState) -> Router {
Router::new()
.route("/health", get(health_check))
.route("/metrics", get(metrics_handler)) // Prometheus scrape endpoint
.with_state(state)
}5. Error Tracking with Sentry
The official sentry crate integrates with Rust services to capture panics, errors, and custom events:
// Cargo.toml
// sentry = { version = "0.34", features = ["tracing"] }
fn main() {
let _guard = sentry::init(("https://your-dsn@sentry.io/123", sentry::ClientOptions {
release: sentry::release_name!(),
environment: Some("production".into()),
traces_sample_rate: 0.1, // 10% of transactions
..Default::default()
}));
// Panics are automatically captured
// Use capture_anyhow for explicit error reporting:
if let Err(e) = run_application() {
sentry::capture_error(&e);
}
}6. Rust-Specific Metrics to Monitor
Beyond standard application metrics, monitor Rust-specific runtime characteristics:
Tokio task count
Why it matters: Task accumulation indicates backpressure or missing .await calls
How to measure: tokio-metrics crate or tokio console
Thread pool utilization
Why it matters: Blocking calls on async threads cause latency spikes
How to measure: tokio::task::spawn_blocking calls; monitor separately
Memory allocations
Why it matters: Rust is efficient but custom allocators have overhead
How to measure: jemalloc + malloc_stats; or dhat for heap profiling
Channel queue depth
Why it matters: Full channels cause backpressure that blocks callers
How to measure: Expose gauge metrics from mpsc channel receivers
Connection pool utilization
Why it matters: Pool exhaustion causes request queuing and timeout errors
How to measure: sqlx / deadpool expose pool size and wait metrics
Recommended Alerting Rules for Rust Services
| Alert | Threshold | Severity |
|---|---|---|
| HTTP 5xx error rate | > 1% of requests for 5 minutes | Critical |
| P99 request latency | > 500ms for 10 minutes | Warning |
| Health check failure | Endpoint returns non-200 | Critical |
| Tokio runtime blocked | tokio_blocking_threads > 90% of pool | Warning |
| DB connection pool exhausted | Wait queue > 10 for 2 minutes | Critical |
| Panic count | Any panic in 5 minutes | Critical |
Set up uptime monitoring for your Rust services
Better Stack monitors your /health endpoints every 30 seconds and alerts you the moment your Rust service goes down. Free tier available.
Try Better Stack Free →Frequently Asked Questions
What is the best crate for Rust metrics?
The metrics crate (facade) + metrics-exporter-prometheus is the most popular combination. It decouples your app from the backend — you can switch from Prometheus to StatsD without changing application code. For direct Prometheus integration, the prometheus crate is the official Prometheus Rust client.
How do I add OpenTelemetry tracing to a Rust application?
Add tracing, tracing-opentelemetry, opentelemetry-otlp, and opentelemetry_sdk to Cargo.toml. Initialize an OTLP exporter pointing at your collector (Jaeger, Grafana Tempo, or Honeycomb). Use #[instrument] on async functions and the span! macro for manual spans.
How do I do structured logging in Rust?
Use tracing-subscriber with JSON output: tracing_subscriber::fmt::layer().json(). This outputs machine-parseable JSON logs with all structured fields. Route stdout to your log aggregation system (Loki, Datadog, CloudWatch). Use info!(), warn!(), error!() macros with key=value fields.
Should I use log or tracing in Rust?
Use tracing for new projects. It's a superset of log with structured data, async-aware context propagation, and OpenTelemetry integration. The tracing-log crate bridges old log-based libraries into the tracing ecosystem, so the two are compatible.
Related Monitoring Guides
Alert Pro
14-day free trialStop checking — get alerted instantly
Alert Pro checks the 66 services we monitor every hour and emails you within the hour of a detected change.
- Email alerts for up to 10 of the APIs we monitor
- $0 charged today — card required to start
- Cancel anytime — $9/mo after trial
The 66 services Alert Pro covers
Airtable · Amplitude · Anthropic (Claude) · Asana · Atlassian · AWS · Azure · Bitbucket · Canva · Canvas LMS · Cash App · ChatGPT · Claude · Clerk · Cloudflare · CockroachDB · Cohere · Coinbase · Confluence · Cursor · DigitalOcean · Discord · Dropbox · Elasticsearch · Epic Games · Figma · Fly.io · Fortnite · GitHub · Google Cloud · Google Cloud Platform · Grafana · Grammarly · Groq · HubSpot · Jira · LaunchDarkly · Linear · Mailgun · Mixpanel · Monday.com · MongoDB Atlas · Netlify · New Relic · npm · OpenAI · PayPal · Pinterest · Plaid · PlanetScale · Reddit · Render · Robinhood · Runway · Segment · SendGrid · Sentry · Shopify · Square · Stability AI · Supabase · Trello · Twilio · Twitch · Vercel · Zoom