PagerDuty setup
Route diagnoses straight to your on-call schedule with severity mapping. Hardware failures trigger high-urgency incidents; software failures map to low urgency.
Setup steps
- In PagerDuty, go to Integrations and add a new Generic API (Events API v2) integration to the service you want to alert.
- Copy the Integration Key from PagerDuty.
- Go to your Denpex dashboard, navigate to Alerts, click Add Channel, select PagerDuty, and paste the integration key.
- That is it. Denpex maps failure types to PagerDuty severities automatically.
Severity mapping
Failure type
PagerDuty severity
Examples
Hardware
Critical
Xid errors, ECC uncorrectable, row-remap, thermal throttle
Environment
Warning
NCCL timeout, network partition, out of memory
Software
Warning
CUDA error, NaN loss, checkpoint corruption
ML / Data
Info
Gradient spike, data pipeline stall, label error
Configuration
- Minimum severity: suppress info or warning-level alerts during quiet hours
- Quiet hours: route only critical hardware failures to PagerDuty outside business hours
- Deduplication: cascade correlation collapses a fabric-wide event into a single PagerDuty incident