Denpex documentation
Everything you need to install the agent, configure alerts, and call the Denpex API. If you can't find what you need, email support@denpex.com.
Quick links
Get from zero to your first diagnosis in under 5 minutes.
Single-file, stdlib-only Python agent that wraps your training command.
Push diagnoses to Slack, PagerDuty, SMS, or webhooks.
REST endpoints for diagnosis, history, alerts, keys, jobs, and billing.
87 docs spanning architecture, security, deployment, SDKs, and the full API.
What is Denpex?
Denpex diagnoses ML training failures. CUDA OOM, NCCL timeouts, silent data corruption, gradient explosion, NaN loss, checkpoint corruption, and more. The diagnosis engine is deterministic. 10,800+ deterministic regex signatures plus retrieval over the 700+-class Failure Encyclopedia (IDF-weighted, with error-signature anchoring). With an AI fallback for novel errors. Outputs include ranked candidates with an evidence trail, the originating rank (in distributed training), hardware/ML classification, and a prescriptive fix (an exact env var, config change, or resume command). Known signatures usually resolve quickly; novel or ambiguous failures can take several minutes while the deep path checks competing hypotheses and gathers supporting context.
The agent is a single Python file. It heartbeats every 2 min, ships the last 500 log lines on a non-zero exit, and never modifies the training process. On Scale and Data Center, an in-VPC agent ships anonymized signatures only, no log egress.
Browse the encyclopedia
700+ failure classes with root cause, fix, and prevention on the Failure Encyclopedia.