Skip to content

Denpex documentation

Everything you need to install the agent, configure alerts, and call the Denpex API. If you can't find what you need, email support@denpex.com.

Quick links

What is Denpex?

Denpex diagnoses ML training failures — CUDA OOM, NCCL timeouts, silent data corruption, gradient explosion, NaN loss, checkpoint corruption, and more — in under 12 seconds. The diagnosis engine is deterministic — 1,400+ hand-tuned regex signatures (1,423 failure types) plus retrieval over 3500+ documented failure classes (IDF-weighted, with error-signature anchoring) — with an AI fallback for novel errors. Outputs include ranked candidates with an evidence trail, the originating rank (in distributed training), hardware/ML classification, and a prescriptive fix (an exact env var, config change, or resume command).

The agent is a single Python file. It heartbeats every 2 min, ships the last 500 log lines on a non-zero exit, and never modifies the training process. On Scale and Data Center, an in-VPC agent ships anonymized signatures only — no log egress.

Browse the encyclopedia

3500+ failure classes with root cause, fix, and prevention on the Failure Encyclopedia.