Silent Corruption Detection Limits
What We Can Detect
The denpex SDK ships a canary SDC runner that:
- Runs a small fixed-input microbatch on a sampled rank in parallel with the real microbatch.
- Compares outputs bit-for-bit using
torch.allclosewith a 1-ULP tolerance. - Detects layer-level checksums via XXH3.
- Validates Adam moments across ranks at the same step.
What We CANNOT Detect
- Bit flips masked by FP32 rounding. A single-bit flip that happens to round to the same value passes the canary.
- Silent drift in long-tailed tensors. A 1 % shift in a rare-occurrence slice may not exceed the 1-ULP tolerance.
- Correlated hardware faults. A clock that runs 0.1 % slow produces identical outputs.
- Driver-level reordering. If the kernel reorders reductions, our hooks may not see the original ordering.
Empirical Coverage
From the synthetic corpus:
- Single-bit ECC failure in HBM: detected 91 % of the time at 2 GB/s DRAM.
- Cumulative drift (10 + ULP): detected 99 % of the time over a 1000-step window.
- Mid-stream compiler reordering: not detected; we recommend the user turn on strict determinism via PyTorch's
--deterministic-algorithms.
Recommendations to Customers
- Enable
--deterministic-algorithmsfor any reproducibility- critical run. - Enable the canary with a 1-in-10 sampling rate.
- Run the SDC-test corpus in CI before any infra change.
- Carry manifest plus smoke-inference as part of every checkpoint.
Limits Are Spelled Out Here
Not in marketing. Reproduce from docs/benchmarks/accuracy/corpus.csv and the script docs/benchmarks/silent-corruption/run.sh.