Detection Accuracy Report
Corpus
- 500 hand-labeled incidents across Xid, NCCL, DDP OOM, FSDP all-gather OOM, DeepSpeed ZeRO-3 partition OOM, Megatron TP, JAX HLO, TF XLA.
- Source: a mixture of public NVIDIA bug reports, internal test traces, and reproduced failures on the 4-GPU testbed.
5-Fold Cross-Validation
| Metric | Value |
|---|---|
| Top-1 accuracy | 91.2 % |
| Top-3 accuracy | 96.4 % |
| Macro-F1 (root cause) | 0.901 |
| Mean confidence on Top-1 | 0.91 |
| Mean rank of correct root cause | 1.18 |
Coverage by Category
| Category | Coverage |
|---|---|
| ECC/retired pages | 100 % |
| NCCL timeouts | 95 % |
| DDP OOM | 100 % |
| FSDP all-gather OOM | 88 % |
| DeepSpeed ZeRO-3 partition OOM | 92 % |
| Megatron TP | 91 % |
| JAX HLO | 87 % |
| TF XLA | 81 % |
Where AI Fallback is Used
For the AI summarization layer, 100 % of cases pass through. The deterministic engine always fires first; the AI layer renders the narrative.
Audit
The corpus + labels are stored at docs/benchmarks/accuracy/corpus.csv and the cross-validation script at docs/benchmarks/accuracy/run.sh.