CUDA out of memory during vLLM serving
CUDA out of memory
CUDA allocation fails mid-serving after earlier requests succeeded. Capture the first traceback and request shape to isolate cache or concurrency pressure.
Source checked against vLLM v0.32.0. Log wording and available flags may vary by release.
First action
Capture the first CUDA traceback, per-GPU memory, request lengths, and active sequences. Reproduce on a canary while reducing one concurrency or batching limit, such as --max-num-seqs, and keep the request shape fixed.
Verify the change. Repeat the failing shape at representative concurrency and watch allocation errors, preemption, and request latency before widening traffic.
Diagnose your vLLM log offline
Synthetic demonstration samples
Synthetic demonstration samples. Not customer telemetry.
Upload your log file. Large files are processed in your browser. Only selected evidence is submitted when you run diagnosis.
Your logs are processed entirely in your browser and are never uploaded by this tool.
Add host kernel / dmesg or nvidia-smi context (optional)
100% local browser engine (same deterministic core as the Open VSX & VS Code extension). 0 bytes of log data uploaded.
Handling a production incident? Replay it with your team.