Skip to content
vLLM error triage

CUDA out of memory during vLLM weight loading

CUDA out of memory

First failing GPU allocation occurs during model loading or checkpoint materialization before serving starts. Confirm the stack trace points to weights.

Source checked against vLLM v0.32.0. Log wording and available flags may vary by release.

First action

Compare peak per-GPU allocation during loading against the available memory on each rank. Check the actual checkpoint and dtype, then try a supported lower-footprint model or a larger supported tensor-parallel deployment.

Verify the change. Confirm all ranks finish loading the same checkpoint and that the engine still has enough cache for a representative request.

Diagnose your vLLM log offline

Synthetic demonstration samples

Synthetic demonstration samples. Not customer telemetry.

Upload your log file. Large files are processed in your browser. Only selected evidence is submitted when you run diagnosis.

Your logs are processed entirely in your browser and are never uploaded by this tool.

Add host kernel / dmesg or nvidia-smi context (optional)

100% local browser engine (same deterministic core as the Open VSX & VS Code extension). 0 bytes of log data uploaded.

Handling a production incident? Replay it with your team.