Skip to content
Webhooks

Outgoing webhooks

Send diagnosis events to any HTTP endpoint. Denpex POSTs a JSON payload with the root cause, classification, fix, and originating rank whenever a failure is diagnosed.

Add a webhook

  1. Go to your Denpex dashboard and navigate to Alerts.
  2. Click Add Channel and select Webhook.
  3. Enter a name for the webhook and the endpoint URL that will receive POST requests.
  4. Optionally set a secret for HMAC-SHA256 signature verification. The signature is sent in the X-Denpex-Signature header.
  5. Set the minimum severity and quiet hours, then save.

Payload format

Every diagnosis triggers a POST to your webhook URL with this JSON body:

{ "event": "diagnosis.completed", "id": "dgn_abc123", "timestamp": "2026-07-05T12:00:00Z", "failure": { "type": "nccl_timeout", "classification": "environment", "confidence": 0.97, "severity": "critical" }, "fix": { "summary": "NCCL watch dog timeout on rank 12", "action": "Check the InfiniBand link on node gpu-12. Run: ibstatus and look for 'Link层的层: Active' on port 1. If the link is down, reseat the cable or replace the HCA.", "resumeFrom": "last_checkpoint" }, "meta": { "originatingRank": 12, "coalesced": false, "crashCount": 1, "jobName": "llama3-finetune", "hostname": "gpu-12" } }

Signature verification

If you configure a webhook secret, every request includes a X-Denpex-Signature header containing an HMAC-SHA256 signature of the request body. Verify it on your end:

# Python example import hmac, hashlib secret = b"your-webhook-secret" body = request.get_data() sig = request.headers.get("X-Denpex-Signature") expected = hmac.new(secret, body, hashlib.sha256).hexdigest() if hmac.compare_digest(sig, expected): # request is authentic

Testing your webhook

Use the Denpex dashboard to send a test event to your endpoint. The test payload matches the production format so you can validate your integration before real failures hit.

Best practices

  • Respond with HTTP 200 as quickly as possible. Denpex retries on 5xx responses with exponential backoff.
  • Use a webhook secret to verify payload authenticity.
  • Keep your endpoint idempotent. The same event may be delivered more than once.
  • Set up quiet hours to avoid alert fatigue during off-hours.