Free tools for GPU cluster engineers
Diagnostic references, local parsers and planning calculators that run without an account. Each tool states what it can establish and where a complete incident still needs more evidence.
NVIDIA Compute Sanitizer commands and troubleshooting covers memcheck, worker subprocesses, and CI exit codes. Run the instrumentation in your isolated GPU environment.
- Zero-upload browser log diagnosis
Offline GPU Log Analyzer
Diagnose multi-gigabyte PyTorch, NCCL, vLLM and Slurm logs in your browser with zero network egress. Evaluates rank cascades, OOM and hardware drops locally.
Open Offline GPU Log Analyzer - Resolve driver, CUDA, torch and FlashAttention builds
CUDA, Driver & PyTorch Compatibility Resolver
Find matching NVIDIA driver, CUDA toolkit, PyTorch and FlashAttention versions. Emits exact pip, uv and Docker commands to avoid broken ABI builds.
Open CUDA, Driver & PyTorch Compatibility Resolver - Hardware, software or more evidence
NVIDIA Xid Decoder
Paste an NVRM Xid line to extract the code, fault class, RMA relevance and first evidence-preserving action, then open the complete investigation.
Open NVIDIA Xid Decoder - Find the first evidence, not the last timeout
NCCL Log Decoder
Classify covered CUDA, bootstrap, transport, duplicate-GPU and timeout signatures across rank logs, with wrappers marked as secondary.
Open NCCL Log Decoder - Turn a GPU report into incident evidence
nvidia-smi Parser
Parse GPU identity, memory, thermal, power, utilization, ECC and row-remapper fields locally in your browser. Flags stay evidence, not automatic hardware verdicts.
Open nvidia-smi Parser - Differentiate fragmentation from true capacity exhaustion
PyTorch CUDA OOM Triage
Inspect PyTorch memory allocator dumps, reserved vs allocated pools, and get exact garbage collection, allocator config and batch sizing fixes.
Open PyTorch CUDA OOM Triage - Pinpoint illegal memory access and race conditions
Compute Sanitizer & Memcheck Guide
Minimal reproduction recipes for cuda-memcheck, racecheck and synccheck to isolate illegal memory accesses before collective timeout cascades.
Open Compute Sanitizer & Memcheck Guide - Pretraining completion time, MFU and cloud GPU costs
LLM Training Time & Cluster Burn Calculator
Calculate Chinchilla token FLOPs, Model FLOPs Utilization (MFU), cluster throughput, and cloud rental burn across H100, H200 and B200 GPU clusters.
Open LLM Training Time & Cluster Burn Calculator - Model weights, KV cache, activations and tensor parallelism
LLM Multi-Node VRAM Calculator
Size model weights, KV cache at context limits, optimizer states and pipeline/tensor parallelism across H100, H200, B200 and A100 GPU clusters.
Open LLM Multi-Node VRAM Calculator - Actor, reference policy & colocated vLLM rollout HBM sizing
GRPO & RL Post-Training VRAM Calculator
Model FSDP actor states, frozen KL reference footprints, critic overhead, and colocated vLLM KV-cache headroom for GRPO, DAPO, and PPO clusters.
Open GRPO & RL Post-Training VRAM Calculator - Estimate GPU memory, KV cache capacity and token throughput
LLM Inference Throughput & KV Cache Sizer
Estimate generation token latency, prompt processing throughput, and concurrency ceilings for production serving engines.
Open LLM Inference Throughput & KV Cache Sizer - Resolve GID indices, RoCE v2 parameters and InfiniBand fabric interfaces
RoCEv2 & InfiniBand NCCL GID Index Calculator
Inspect GID table output and estimate bandwidth and buffer requirements for RoCEv2 and InfiniBand networks.
Open RoCEv2 & InfiniBand NCCL GID Index Calculator - Calculate partitioned memory overheads across ZeRO stages 1, 2, and 3
DeepSpeed ZeRO & LoRA Memory Calculator
Estimate parameter, gradient, and optimizer state memory for distributed fine-tuning and pretraining.
Open DeepSpeed ZeRO & LoRA Memory Calculator - Generate Kubernetes GPU job manifests and check common configuration errors
Kubernetes GPU Job Generator & Manifest Linter
Configure resource requests, node selectors, tolerations, and shared memory volumes for reliable cluster orchestration.
Open Kubernetes GPU Job Generator & Manifest Linter - Decode Datacenter GPU Manager health violations and PCIe bus errors
DCGM & PCIe AER Diagnostic Error Decoder
Identify reported hardware, PCIe link and thermal signals in diagnostic logs and see what evidence to collect next.
Open DCGM & PCIe AER Diagnostic Error Decoder - Model direct capacity and engineer time
GPU Downtime Calculator
Calculate GPU hours, infrastructure cost, labor, monthly exposure and a scenario-based recoverable share using your own inputs.
Open GPU Downtime Calculator - Generate production sbatch scripts for distributed training
Slurm Multi-Node GPU Batch Script Generator
Configure PyTorch FSDP, DeepSpeed and Megatron distributed jobs with correct GPUs per node, cpus-per-task, master port and NCCL fabric flags.
Open Slurm Multi-Node GPU Batch Script Generator - Catch fabric misconfigs before step 0
NCCL Topology Linter
Paste nvidia-smi topo -m and your NCCL_* env. It checks GPU-to-NIC affinity, PCIe ACS, memlock limits and NCCL settings against the fabric you actually have, and names the ones that will cost you throughput or hang the job.
Open NCCL Topology Linter - Find what changed since the last good run
Cross-Run Environment Diff
Paste artifacts from the last good run and the failing one. Driver, CUDA, NCCL, PyTorch/JAX, pip packages and NCCL_* env are diffed, and every change is ranked against the observed failure class so you know where to bisect first.
Open Cross-Run Environment Diff - Stop losing hours to env vars
NCCL Environment Generator
Describe your interconnect and job shape and get a defensible NCCL and PyTorch environment, with each variable explained so you can justify it in review instead of copying a Stack Overflow block.
Open NCCL Environment Generator - Optimize NCCL flags for InfiniBand, RoCE v2 and AWS EFA
NCCL Tuning Guide & Flag Optimizer
Tune NCCL_NET_GDR_LEVEL, NCCL_BUFFSIZE, NCCL_CROSS_NIC and tree versus ring collective algorithms to maximize distributed training bandwidth.
Open NCCL Tuning Guide & Flag Optimizer - The failure data procurement never gets
GPU Reliability Index
H100 vs A100 vs H200 failure modes and rates drawn from the Denpex incident corpus, so a hardware decision can cite something other than a vendor datasheet.
Open GPU Reliability Index - Fix DataLoader worker killed by signal: Bus error
PyTorch DataLoader /dev/shm & Bus Error Sizer
Size Linux shared memory (/dev/shm) for multi-worker PyTorch DataLoaders. Get exact Docker --shm-size and Kubernetes emptyDir RAM configs to eliminate bus errors.
Open PyTorch DataLoader /dev/shm & Bus Error Sizer - Fix could not select device driver with capabilities: [[gpu]]
NVIDIA Container Toolkit & Driver Fix
Step-by-step resolution for Docker CDI, nvidia-container-cli and driver version mismatch errors in containerized workloads.
Open NVIDIA Container Toolkit & Driver Fix - Resolve libcudart.so and compute capability loading errors
bitsandbytes CUDA Setup Failed Fix
Fix CUDA Setup failed despite GPU being detected, missing shared libraries, and unsupported compute capability across 4-bit and 8-bit quantization.
Open bitsandbytes CUDA Setup Failed Fix - Fix indices should be either on cpu or on the same device as the indexed tensor
PyTorch Indices Device Mismatch Fix
Identify embedding lookups, masking operations and tensor indexing where tensors reside across CPU and distinct CUDA ranks.
Open PyTorch Indices Device Mismatch Fix - Pipeline bubble, NCCL comm overlap & HTA trace triage
PyTorch Profiler & Bubble Analyzer
Analyze PyTorch Kineto and Holistic Trace Analysis (HTA) traces locally in your browser to quantify pipeline bubbles and straggler waits.
Open PyTorch Profiler & Bubble Analyzer - 14 diagnostics for GPU logs, benchmarks, profiles, and topology
GPU Fleet Command Suite
Analyze outputs from gpud, HTA, SuperBench, NHC, nccl-tests, and other fleet tools. Run any sample without an account, or use the CLI to keep input on your machine.
Open GPU Fleet Command Suite