Webhook Testing: Complete Guide to Testing, Debugging & Monitoring Webhooks (2026)

A webhook fires. Your endpoint returns 200. The integration looks healthy — until a customer reports they never received their order confirmation, a payment went unrecorded, or a deployment never triggered. Webhook failures are invisible by default. Unlike API calls where you control the request, webhooks arrive on someone else's schedule, with someone else's payload, at whatever frequency they choose.

Testing webhooks is fundamentally different from testing APIs. You can't just fire up Postman and hit an endpoint — you need to receive events, validate signatures, handle retries, and verify that your processing logic works under real-world conditions like duplicate deliveries, out-of-order events, and malformed payloads.

This guide covers everything: local development testing, automated test suites, production monitoring, and the debugging workflows that save you when webhooks silently fail at 2 AM.

Why Webhook Testing Is Different From API Testing

When you test a REST API, you control the conversation. You craft a request, send it, and inspect the response. Webhooks flip that model — you're the server, and the external service decides when, what, and how often to send.

This creates testing challenges that don't exist with traditional API testing:

You can't trigger events on demand. To test a Stripe payment_intent.succeeded webhook, you need an actual payment to succeed. In development, that means test mode transactions. In production, you need synthetic events or replay capabilities.

Payloads change without warning. Webhook providers add fields, deprecate formats, and modify structures. Your code needs to handle payloads it's never seen before without crashing.

Delivery is unreliable by design. Webhooks use at-least-once delivery with retries. Your handler must be idempotent — processing the same event twice should produce the same result, not duplicate a charge or send two emails.

Timing is unpredictable. Events can arrive out of order. A subscription.updated event might arrive before subscription.created if the provider's event queue is backed up.

Signatures add complexity. Most webhook providers sign payloads with HMAC-SHA256 or similar. Your test environment needs to either replicate signing or bypass it safely — and you need to test both valid and invalid signatures.

Setting Up Local Webhook Testing

The first hurdle: webhook providers need a publicly reachable URL to deliver events, but your development server runs on localhost:3000. Here are the proven approaches, from simplest to most robust.

Tunnel-Based Testing (ngrok, Cloudflare Tunnel)

Tunneling tools create a public URL that forwards traffic to your local server. This is the most common approach for webhook development.

ngrok is the industry standard:

# Install
brew install ngrok  # macOS
# or download from https://ngrok.com

# Expose your local webhook endpoint
ngrok http 3000

# Output:
# Forwarding https://a1b2c3d4.ngrok-free.app -> http://localhost:3000

Copy the HTTPS URL and register it as your webhook endpoint with the provider. Every request to that URL reaches your local server.

ngrok's inspection UI (http://127.0.0.1:4040) is incredibly useful — it shows every request with full headers, body, and timing. You can replay requests with one click, which is invaluable for debugging.

Cloudflare Tunnel is the free alternative:

# Install
brew install cloudflared

# Quick tunnel (no account needed)
cloudflared tunnel --url http://localhost:3000

# Output:
# https://random-words.trycloudflare.com -> http://localhost:3000

Cloudflare Tunnels don't have the inspection UI, but they're completely free with no request limits.

When to use tunnels: Active development, manual testing, debugging specific webhook flows. Not suitable for CI/CD pipelines (URL changes each restart unless you pay for a fixed subdomain).

Webhook Inspection Tools

Sometimes you just need to see what a webhook provider is sending — before writing any handler code.

webhook.site — Generates a unique URL that captures all incoming requests. Shows headers, body, query params, and timing. Free tier handles most testing needs. Great for initial integration exploration.

RequestBin — Similar concept, captures and displays webhook payloads. Useful for sharing captured payloads with team members.

Hookdeck Console — Purpose-built for webhook testing with filtering, transformation preview, and retry simulation. More powerful than webhook.site for complex workflows.

When to use inspection tools: Understanding a provider's webhook format before writing code, documenting payload structures, debugging production issues by comparing expected vs. actual payloads.

CLI-Based Testing Tools

For developers who prefer the terminal:

# Stripe CLI — best-in-class webhook testing
stripe listen --forward-to localhost:3000/api/webhooks/stripe

# Triggers a specific event
stripe trigger payment_intent.succeeded

# GitHub CLI webhook forwarding
gh webhook forward --repo=owner/repo --events=push \
  --url=http://localhost:3000/api/webhooks/github

Provider-specific CLIs are the gold standard for testing because they generate properly signed events with realistic payloads. If your webhook provider offers a CLI, use it.

Writing Automated Webhook Tests

Manual testing catches obvious issues. Automated tests catch regressions. Here's how to build a comprehensive webhook test suite.

Unit Testing Webhook Handlers

Start by testing your handler function in isolation, mocking the HTTP layer:

// tests/webhooks/stripe.test.ts
import { describe, it, expect, vi } from 'vitest';
import { handleStripeWebhook } from '@/lib/webhooks/stripe';
import { createHmac } from 'crypto';

const WEBHOOK_SECRET = 'whsec_test_secret';

function signPayload(payload: string, secret: string): string {
  const timestamp = Math.floor(Date.now() / 1000);
  const signedPayload = `${timestamp}.${payload}`;
  const signature = createHmac('sha256', secret)
    .update(signedPayload)
    .digest('hex');
  return `t=${timestamp},v1=${signature}`;
}

describe('Stripe Webhook Handler', () => {
  it('processes payment_intent.succeeded correctly', async () => {
    const payload = JSON.stringify({
      id: 'evt_test_123',
      type: 'payment_intent.succeeded',
      data: {
        object: {
          id: 'pi_test_456',
          amount: 2000,
          currency: 'usd',
          customer: 'cus_test_789',
        },
      },
    });

    const signature = signPayload(payload, WEBHOOK_SECRET);

    const result = await handleStripeWebhook({
      body: payload,
      headers: { 'stripe-signature': signature },
      secret: WEBHOOK_SECRET,
    });

    expect(result.status).toBe('processed');
    expect(result.eventId).toBe('evt_test_123');
  });

  it('rejects invalid signatures', async () => {
    const payload = JSON.stringify({
      id: 'evt_test_123',
      type: 'payment_intent.succeeded',
      data: { object: {} },
    });

    await expect(
      handleStripeWebhook({
        body: payload,
        headers: { 'stripe-signature': 'invalid_signature' },
        secret: WEBHOOK_SECRET,
      })
    ).rejects.toThrow('Invalid signature');
  });

  it('handles duplicate events idempotently', async () => {
    const payload = JSON.stringify({
      id: 'evt_test_duplicate',
      type: 'payment_intent.succeeded',
      data: { object: { id: 'pi_test_456', amount: 2000 } },
    });
    const signature = signPayload(payload, WEBHOOK_SECRET);
    const args = {
      body: payload,
      headers: { 'stripe-signature': signature },
      secret: WEBHOOK_SECRET,
    };

    // Process first time
    const first = await handleStripeWebhook(args);
    expect(first.status).toBe('processed');

    // Process same event again — should be idempotent
    const second = await handleStripeWebhook(args);
    expect(second.status).toBe('already_processed');
  });
});

Testing Signature Verification

Signature verification is the most security-critical part of webhook handling. Test it thoroughly:

describe('Webhook Signature Verification', () => {
  const secret = 'whsec_test_secret_key';

  it('accepts valid HMAC-SHA256 signatures', () => {
    const body = '{"event":"test"}';
    const timestamp = Math.floor(Date.now() / 1000);
    const sig = createHmac('sha256', secret)
      .update(`${timestamp}.${body}`)
      .digest('hex');

    expect(
      verifySignature(body, `t=${timestamp},v1=${sig}`, secret)
    ).toBe(true);
  });

  it('rejects tampered payloads', () => {
    const body = '{"event":"test"}';
    const tamperedBody = '{"event":"test","amount":99999}';
    const timestamp = Math.floor(Date.now() / 1000);
    const sig = createHmac('sha256', secret)
      .update(`${timestamp}.${body}`)
      .digest('hex');

    expect(
      verifySignature(tamperedBody, `t=${timestamp},v1=${sig}`, secret)
    ).toBe(false);
  });

  it('rejects expired timestamps (replay attacks)', () => {
    const body = '{"event":"test"}';
    const oldTimestamp = Math.floor(Date.now() / 1000) - 600; // 10 min ago
    const sig = createHmac('sha256', secret)
      .update(`${oldTimestamp}.${body}`)
      .digest('hex');

    expect(
      verifySignature(body, `t=${oldTimestamp},v1=${sig}`, secret, {
        tolerance: 300, // 5 min tolerance
      })
    ).toBe(false);
  });

  it('rejects missing signature header', () => {
    expect(() => verifySignature('{}', '', secret)).toThrow();
  });
});

Integration Testing With Real HTTP

Test the full request-response cycle:

// tests/integration/webhook-endpoint.test.ts
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { createServer } from 'http';
import { app } from '@/app';

let server: ReturnType<typeof createServer>;
let baseUrl: string;

beforeAll(async () => {
  server = createServer(app);
  await new Promise<void>((resolve) => {
    server.listen(0, () => {
      const addr = server.address();
      baseUrl = `http://127.0.0.1:${(addr as any).port}`;
      resolve();
    });
  });
});

afterAll(() => server.close());

describe('Webhook Endpoint Integration', () => {
  it('returns 200 for valid webhook', async () => {
    const payload = JSON.stringify({
      id: 'evt_integration_test',
      type: 'customer.created',
      data: { object: { id: 'cus_test', email: 'test@example.com' } },
    });

    const response = await fetch(`${baseUrl}/api/webhooks/stripe`, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'stripe-signature': signPayload(payload, process.env.WEBHOOK_SECRET!),
      },
      body: payload,
    });

    expect(response.status).toBe(200);
  });

  it('returns 401 for invalid signature', async () => {
    const response = await fetch(`${baseUrl}/api/webhooks/stripe`, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'stripe-signature': 'invalid',
      },
      body: '{}',
    });

    expect(response.status).toBe(401);
  });

  it('returns 200 quickly (under 5s)', async () => {
    const start = Date.now();
    const payload = JSON.stringify({
      id: 'evt_timing_test',
      type: 'invoice.paid',
      data: { object: { id: 'inv_test' } },
    });

    await fetch(`${baseUrl}/api/webhooks/stripe`, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'stripe-signature': signPayload(payload, process.env.WEBHOOK_SECRET!),
      },
      body: payload,
    });

    const elapsed = Date.now() - start;
    expect(elapsed).toBeLessThan(5000);
  });
});

Testing Edge Cases That Break Production

These are the scenarios that work fine in development but fail in production:

describe('Webhook Edge Cases', () => {
  it('handles events arriving out of order', async () => {
    // Send "updated" before "created"
    await processWebhook({
      type: 'subscription.updated',
      data: { object: { id: 'sub_123', status: 'active' } },
    });

    await processWebhook({
      type: 'subscription.created',
      data: { object: { id: 'sub_123', status: 'trialing' } },
    });

    // Status should reflect the latest event, not the last-received
    const sub = await getSubscription('sub_123');
    expect(sub.status).toBe('active'); // Not "trialing"
  });

  it('survives unexpected payload fields', async () => {
    const result = await processWebhook({
      type: 'payment_intent.succeeded',
      data: {
        object: {
          id: 'pi_test',
          amount: 2000,
          // New field the provider added yesterday
          risk_assessment: { score: 0.12, factors: ['geographic'] },
        },
      },
    });

    expect(result.status).toBe('processed');
  });

  it('handles extremely large payloads gracefully', async () => {
    const largePayload = {
      type: 'batch.completed',
      data: {
        object: {
          id: 'batch_123',
          items: Array.from({ length: 10000 }, (_, i) => ({
            id: `item_${i}`,
            status: 'processed',
          })),
        },
      },
    };

    const result = await processWebhook(largePayload);
    expect(result.status).toBe('processed');
  });

  it('returns 200 even when processing fails (to prevent retries)', async () => {
    // Simulate a processing error
    vi.spyOn(db, 'insert').mockRejectedValueOnce(new Error('DB down'));

    const response = await sendWebhook({
      type: 'payment_intent.succeeded',
      data: { object: { id: 'pi_fail_test' } },
    });

    // Return 200 to acknowledge receipt
    // Queue for retry internally rather than relying on provider retries
    expect(response.status).toBe(200);
    expect(await getDeadLetterQueue()).toContainEqual(
      expect.objectContaining({ eventId: 'pi_fail_test' })
    );
  });
});

Debugging Failed Webhooks in Production

When webhooks fail in production, you're often working backwards from a symptom ("customer didn't get their confirmation email") to a root cause. Here's a systematic debugging workflow.

Step 1: Check the Provider's Dashboard

Almost every webhook provider has a delivery log. Start there:

• Stripe: Dashboard → Developers → Webhooks → select endpoint → Recent deliveries
• GitHub: Settings → Webhooks → Recent deliveries (with full request/response)
• Shopify: Settings → Notifications → Webhooks → show delivery attempts
• Twilio: Console → Monitor → Logs

Look for: HTTP status codes (did your server return 200?), response time (did you time out?), and retry attempts (how many times did they try?).

Step 2: Reproduce With the Exact Payload

Most provider dashboards let you copy the exact payload that failed. Use it:

# Copy the failing payload from the dashboard, then replay it locally
curl -X POST http://localhost:3000/api/webhooks/stripe \
  -H "Content-Type: application/json" \
  -H "stripe-signature: $(stripe webhook sign --payload-file failing-event.json)" \
  -d @failing-event.json

With ngrok's inspection UI, you can also replay any captured request directly.

Step 3: Check Your Logs

If you're not logging webhook events, start now. At minimum, log:

// Webhook logging — the minimum viable approach
async function handleWebhook(req: Request) {
  const eventId = req.headers.get('x-webhook-id');
  const eventType = body.type;

  console.log(`[WEBHOOK] Received: ${eventType} (${eventId})`);

  try {
    const result = await processEvent(body);
    console.log(`[WEBHOOK] Processed: ${eventType} (${eventId}) → ${result.status}`);
    return new Response('OK', { status: 200 });
  } catch (error) {
    console.error(`[WEBHOOK] Failed: ${eventType} (${eventId})`, error);
    // Still return 200 — handle retry internally
    return new Response('OK', { status: 200 });
  }
}

Step 4: Common Failure Patterns

Timeout (provider sees 5xx or no response) Your handler is doing too much work synchronously. Webhook providers typically wait 5-30 seconds for a response. Fix: acknowledge the webhook immediately, process asynchronously.

// Bad — processing before responding
app.post('/webhook', async (req, res) => {
  await sendEmail(req.body);        // 3 seconds
  await updateDatabase(req.body);    // 2 seconds
  await notifySlack(req.body);       // 1 second
  res.status(200).send('OK');        // 6 seconds later — might timeout
});

// Good — respond immediately, process async
app.post('/webhook', async (req, res) => {
  res.status(200).send('OK');        // Immediate response
  
  // Process asynchronously (or use a queue)
  queueWebhookProcessing(req.body).catch(console.error);
});

Signature mismatch Usually means: wrong secret configured, body was parsed/modified before verification (Express json() middleware running before raw body access), or clock skew between servers.

Duplicate processing The same event was processed multiple times because your handler isn't idempotent. Fix: store processed event IDs and check before processing.

const processedEvents = new Set<string>(); // Use Redis in production

async function handleEvent(event: WebhookEvent) {
  if (processedEvents.has(event.id)) {
    return { status: 'already_processed' };
  }
  
  processedEvents.add(event.id);
  // Process the event...
}

Monitoring Webhooks in Production

Testing catches bugs before deployment. Monitoring catches failures after. You need both.

What to Monitor

Delivery success rate. Track the percentage of webhooks your endpoint successfully processes. Alert if it drops below 99%.

Response time. Your webhook endpoint should respond in under 3 seconds. Slower responses increase timeout risk and provider retry pressure.

Event processing lag. Time between event creation (provider timestamp) and successful processing (your timestamp). Growing lag means your queue is backing up.

Retry rate. If the provider is retrying frequently, your endpoint is unreliable. Track retries as a leading indicator of failures.

Dead letter queue depth. Events that failed processing after all retries. This queue should trend toward zero — if it's growing, you have an unresolved bug.

Setting Up Webhook Monitoring

For teams that run their own API infrastructure, monitoring webhook delivery is as important as monitoring API uptime. Tools like API Status Check monitor your API endpoints, and the same principles apply to webhook endpoints — if your receiver goes down, you'll miss critical events.

A practical monitoring setup:

// Webhook health metrics
const metrics = {
  received: new Counter('webhooks_received_total', 'Total webhooks received', ['provider', 'event_type']),
  processed: new Counter('webhooks_processed_total', 'Total webhooks processed', ['provider', 'event_type', 'status']),
  processingDuration: new Histogram('webhook_processing_seconds', 'Webhook processing duration', ['provider']),
  queueDepth: new Gauge('webhook_queue_depth', 'Pending webhook events in queue'),
};

async function monitoredWebhookHandler(req: Request) {
  const provider = detectProvider(req);
  const eventType = extractEventType(req);

  metrics.received.inc({ provider, event_type: eventType });
  const timer = metrics.processingDuration.startTimer({ provider });

  try {
    await processWebhook(req);
    metrics.processed.inc({ provider, event_type: eventType, status: 'success' });
  } catch (error) {
    metrics.processed.inc({ provider, event_type: eventType, status: 'error' });
    throw error;
  } finally {
    timer();
  }
}

Alert Rules

Set up alerts for:

• Webhook endpoint returns non-200 for {'>'} 1 minute — your handler is down
• No webhooks received in 1 hour (for high-frequency integrations) — delivery may be broken
• Processing time {'>'} 10 seconds — handler is too slow, timeouts imminent
• Dead letter queue {'>'} 10 events — unprocessed failures accumulating

Webhook Testing Checklist

Before going live with any webhook integration, verify each of these:

Security

• Signature verification is enabled and tested
• Replay attack protection (timestamp validation) is in place
• Raw request body is used for signature verification (not parsed JSON)
• Webhook secret is stored in environment variables, not code
• HTTPS endpoint only (never accept webhooks over HTTP)

Reliability

• Handler is idempotent (duplicate events don't cause duplicate side effects)
• Handler responds within 5 seconds (heavy processing is async)
• Out-of-order events are handled correctly
• Unknown event types are logged but don't cause errors
• Unknown fields in payloads don't break deserialization

Observability

• All received webhooks are logged (event ID, type, timestamp)
• Processing failures are logged with full context
• Failed events are sent to a dead letter queue for manual review
• Metrics track success rate, latency, and queue depth
• Alerts fire when delivery or processing degrades

Testing Coverage

• Valid payload processing (happy path)
• Invalid/missing signature rejection
• Duplicate event handling
• Unknown event type handling
• Malformed payload handling
• Timeout behavior (handler too slow)
• Large payload handling

Webhook Testing Tools Compared

Choosing the right tool depends on your workflow:

For initial exploration — use webhook.site. It's free, instant, and requires zero setup. Just copy the URL and register it with your provider.

For local development — use ngrok or Cloudflare Tunnel. ngrok's inspection UI makes debugging effortless. Pay for a fixed subdomain if you're tired of re-registering URLs.

For provider-specific testing — use the provider's CLI. Stripe CLI, GitHub CLI, and Shopify CLI all support webhook forwarding with properly signed events. These are the most realistic local testing tools available.

For automated testing — write custom test fixtures using the patterns in this guide. No external tool needed — just craft payloads, sign them, and send them to your handler.

For production monitoring — use your existing observability stack (Datadog, Grafana, etc.) with the metrics described above. Add API Status Check to monitor your webhook endpoint's availability alongside your APIs. If your receiving server goes down, you need to know before your webhook provider exhausts its retry budget.

For debugging production issues — combine provider dashboard logs with your application logs. Most issues are diagnosable from the provider's delivery log (HTTP status + response time) paired with your error logs.

Common Mistakes That Break Webhook Integrations

Parsing the body before verifying the signature. Express middleware like express.json() parses the request body, modifying it. Signature verification requires the raw bytes. Use express.raw() for your webhook route.

// Wrong — body is parsed before signature check
app.use(express.json());
app.post('/webhook', verifyAndProcess);

// Right — raw body for signature, then parse
app.post('/webhook', express.raw({ type: 'application/json' }), (req, res) => {
  const isValid = verifySignature(req.body, req.headers['stripe-signature']);
  if (!isValid) return res.status(401).send('Invalid signature');

  const event = JSON.parse(req.body.toString());
  // Process event...
});

Returning errors to the provider. If your handler fails, return 200 anyway and handle the retry internally. Returning 500 triggers the provider's exponential backoff, and you lose control of retry timing.

Not testing with the provider's test mode. Most payment and SaaS platforms have test/sandbox environments that generate realistic webhook events. Use them instead of manually crafting payloads — the structure is guaranteed to match production.

Hardcoding event types. New event types appear when providers add features. If your handler crashes on unknown types, a provider update can take you down. Always have a default case:

switch (event.type) {
  case 'payment_intent.succeeded':
    await handlePaymentSuccess(event);
    break;
  case 'customer.subscription.deleted':
    await handleCancellation(event);
    break;
  default:
    console.log(`Unhandled webhook event: ${event.type}`);
    // Don't throw — just log and acknowledge
}

Ignoring retry behavior. Each provider has different retry schedules. Stripe retries up to 3 days with exponential backoff. GitHub retries for 3 days. Shopify gives 48 hours. Know your provider's retry window — that's how long you have to fix a broken handler before events are permanently lost.

FAQ

How do I test webhooks without deploying to a server?

Use a tunneling tool like ngrok or Cloudflare Tunnel to expose your local development server to the internet. Run ngrok http 3000 to get a public HTTPS URL that forwards to localhost, then register that URL as your webhook endpoint with the provider.

What's the best free webhook testing tool?

For quick payload inspection, webhook.site is free and requires zero setup. For local development, Cloudflare Tunnel (cloudflared tunnel --url http://localhost:3000) is completely free with no request limits. For provider-specific testing, most CLIs (Stripe CLI, GitHub CLI) are free.

How do I verify webhook signatures in my tests?

Generate valid signatures using the same HMAC algorithm the provider uses — typically HMAC-SHA256. Create a helper function that takes a payload and secret, generates the signature, and returns it in the format the provider expects. Test both valid signatures and tampered payloads.

Should my webhook endpoint return 200 even if processing fails?

Generally yes. Return 200 to acknowledge receipt, then handle failures internally with a dead letter queue. If you return 500, the provider will retry — potentially causing duplicate processing when your handler recovers. The exception: return 401 for invalid signatures to signal a misconfiguration.

How do I handle duplicate webhook events?

Implement idempotency by storing processed event IDs. Before processing an event, check if you've already handled it. Use the event's unique ID (provided by the webhook sender) as the key. In production, use Redis or your database for the idempotency store — not in-memory sets.

How do I test webhooks in a CI/CD pipeline?

Don't rely on external tunnels in CI. Instead, write integration tests that send HTTP requests directly to your webhook handler. Craft payloads that match the provider's format, sign them with a test secret, and verify your handler processes them correctly. Mock external dependencies.

What causes webhook signature verification to fail?

The most common cause is middleware modifying the request body before verification. JSON parsing, body size limits, or character encoding changes can alter the raw bytes. Always verify signatures against the raw, unmodified request body. Also check for clock skew if the provider uses timestamp-based verification.

How often should I test my webhook integrations?

Run automated tests in every CI build. Additionally, do a manual end-to-end test (trigger a real event in the provider's test mode) before every major release. Set up production monitoring to continuously verify delivery health. Review the provider's changelog monthly for payload changes that might affect your handler.