Denpex vs Ray
Ray supervises and restarts the work. Denpex explains why the work died. Supervision is not diagnosis.
What is Ray?
Ray is a distributed execution framework for Python. It schedules tasks and actors across a cluster, moves data through a shared object store, supervises workers and restarts them under a configurable retry policy, and autoscales nodes to match demand. Ray Train and Ray Serve build distributed training and serving on top of that core, and the Ray Dashboard gives you a live view of the task graph, actor states, resource usage and worker logs. It is the substrate a growing share of ML workloads runs on.
What is Denpex?
Denpex is a failure diagnosis engine for distributed training and inference. It reads the logs a failed run produced, correlates them across workers onto one timeline, classifies the failure against 16,400+ known signatures, and returns a ranked root cause with the evidence behind it and a concrete next action. It is runtime agnostic and works the same on Ray, Slurm, Kubernetes or a bare torchrun.
The core difference
Ray's job is to keep the workload running, and it is good at that in a way that can hide the problem. A retry policy that quietly restarts a failing actor five times converts a crisp failure into a slow degradation, and by the time anyone looks, the useful first error has scrolled past several rounds of restarts. Ray reports that an actor died and with what exception. Denpex reads across the workers and says which death came first, which of the others were caused by it, and what class of failure the original actually was.
Feature comparison
| Capability | Denpex | Ray |
|---|---|---|
| Distributed task and actor scheduling | No | Yes |
| Automatic actor restart and retry policy | No | Yes |
| Object store and distributed memory management | No | Yes |
| Live task graph and cluster view (Ray Dashboard) | No | Yes |
| Autoscaling worker nodes | No | Yes |
| Explains why an actor died rather than that it died | Yes | No |
| Separates the initiating fault from cascaded actor deaths | Yes | No |
| Interprets NCCL, CUDA and framework errors inside the worker | Yes | No |
| 11,700+ failure patterns with prescriptive fixes | Yes | No |
| Tells an OOM-killed worker apart from a CUDA OOM | Yes | Reports the death, not the class |
| Same diagnosis on Slurm, Kubernetes and bare torchrun | Yes | No |
Verdict
Ray stays your distributed runtime. It schedules tasks and actors, manages the object store, restarts what dies and autoscales the cluster, and the dashboard is the right place to watch all of it. Denpex is for the question Ray deliberately does not answer: not which actor died, but why it died, whether the other deaths were caused by it, and what to change so the retry is not just a slower way to fail again.
Key differences explained
Retries turn one failure into a cascade of logs
The first exception is the informative one and it is the one hardest to find, because automatic restarts bury it under later, less specific failures from workers that were merely downstream of it. Denpex orders the evidence causally rather than by recency, so the diagnosis points at the original fault instead of the last thing to break.
The exception Ray surfaces is often not the cause
A RayActorError or a worker dying unexpectedly is a true statement about the supervision layer and says nothing about the CUDA, NCCL or memory event inside the worker that produced it. Denpex reads what the worker itself printed before it went away, which is where the Xid, the OOM, the collective timeout or the assertion actually is.
Collective failures are not actor failures
Ray Train jobs still use NCCL underneath for gradient synchronization, and a NCCL failure is relational: it is about what one rank was waiting for from another. Ray's model is per-actor, so it can tell you several actors became unhealthy without being able to say they were all blocked on the same collective waiting for one of them. That reconstruction needs the rank logs on a common clock.
One answer across every runtime you use
Teams that adopt Ray rarely run only Ray. There is usually a Slurm cluster, some Kubernetes, and researchers on bare torchrun. Because Denpex works from logs, the same diagnosis applies everywhere and the answer does not change shape with the runtime, which is not something a runtime-specific dashboard can offer.
When to use each
Use Ray when…
You need distributed scheduling, actor supervision, autoscaling, the object store, or the live cluster and task-graph view. You are building the pipeline, tuning parallelism, or watching a job that is currently running.
Use Denpex when…
A Ray Train job failed and the logs are a pile of actor errors with no obvious first cause. You need to know whether the original failure was hardware, memory, collective or application, before the next retry. You want the worker's own error interpreted rather than the supervision-layer wrapper around it.
Use both when…
The usual arrangement. Ray runs and supervises the workload and the dashboard shows you its live state. When a run fails, Denpex reads the worker logs from that run and returns the causal ordering and the failure class, so the retry policy is not doing the diagnosis by trial and error.
Integration path
Nothing changes in your Ray cluster, retry policy or dashboard. Denpex reads logs, so the first useful step is pasting a failed job's worker output into the free diagnosis tool. If you install the agent it wraps the training entrypoint and does not modify Ray's scheduling, the object store or the autoscaler.
Frequently asked questions
Does Denpex replace Ray?
No. Ray is a distributed runtime and Denpex has no execution, scheduling or supervision functionality whatsoever. They sit at different layers and there is nothing to swap.
The Ray Dashboard already shows worker logs. What does Denpex add?
The dashboard shows you the logs; it does not read them for you. On a large job the useful work is correlating hundreds of worker log streams to find which failed first and which failures were consequences. That ordering, plus the classification of the original error, is what Denpex produces.
What is a RayActorError actually telling me?
That an actor became unreachable, which is a fact about supervision rather than about the cause. The worker usually printed something more specific just before it went away: a CUDA error, an OOM, a NCCL timeout or a Python exception. That earlier output is what identifies the failure, and it is what Denpex looks for.
Does Denpex integrate with the Ray API?
There is no native Ray integration or dashboard plugin today. Denpex works from the logs and artifacts you submit, and Ray output included in them is used as evidence alongside everything else.
Our Ray jobs retry and eventually succeed. Is there still a problem?
Frequently yes, and it is the case most worth investigating. Silent retries hide flaky hardware, a marginal NIC, a node that OOMs under a particular batch shape, or a straggler that is costing throughput continuously. The run completing says the retry policy worked, not that the cluster is healthy.