vLLM failures usually fall into four groups: the model and KV cache do not fit the memory budget, a worker process died before the API server noticed, a tensor-parallel worker could not communicate, or a selected kernel does not support the request and hardware combination.
Start with the first worker or EngineCore traceback, not the final HTTP 500 or EngineDeadError. Parent processes report that the engine disappeared after the useful evidence was already printed, and an operating-system OOM kill may leave no Python traceback at all.
For capacity errors, separate total free GPU memory from the fraction vLLM is allowed to use. Then compare the served context length, concurrent sequence count, model weights, activation peak and KV cache budget. Changing all five at once hides which constraint actually failed.
ValueError: No available memory for the cache blocks
vLLM could not reserve any KV cache blocks after accounting for model weights and its profiled activation peak. This is a startup capacity calculation, not proof that another process exhausted the whole GPU.
First action: Run nvidia-smi to confirm the GPU is otherwise idle, then compare --gpu-memory-utilization, --max-model-len and --max-num-seqs. Change one limit and repeat the same launch.
The model's max seq len is larger than the maximum number of tokens that can be stored in KV cache
The context window inferred from the model configuration is larger than the token capacity vLLM can back with the available KV cache. The model may support the length in theory while this deployment cannot serve it.
First action: Set --max-model-len explicitly to a production requirement that fits the reported cache capacity, then canary a prompt plus completion at that exact combined length.
Engine core proc EngineCore_DP0 died unexpectedly, shutting down client
The API process discovered that EngineCore was already gone. This line is a wrapper, not the root cause. The cause is normally an earlier worker traceback, a CUDA failure, or an OOM kill recorded by the host kernel.
First action: Search upward for the first worker traceback and run dmesg -T | grep -i -E "killed process|out of memory|xid" on the host before restarting the service.
NCCL WARN Error while creating shared memory segment /dev/shm/nccl-*
A tensor-parallel worker could not create the shared-memory segment used for local communication. Container runtimes often expose only 64 MB at /dev/shm even when the host has ample RAM.
First action: Run df -h /dev/shm inside the container. If it shows 64 MB, relaunch with a measured --shm-size or host IPC, then repeat the same tensor-parallel startup.
A worker did not read from the shared-memory broadcast ring before its deadline. The worker may have died, but a long execute-model step can produce the same outer timeout while every process remains alive.
First action: Check the worker PIDs first. If they are alive, lower --max-num-batched-tokens and --max-num-seqs for one control run instead of masking the cause with a longer timeout.
CUDA error: an illegal memory access was encountered paged_attention
A paged-attention kernel accessed an invalid GPU address. The discriminators are the request shape, block-table state, attention backend and binary compatibility with the target GPU architecture.
First action: Restart the poisoned worker and reproduce one preserved request with CUDA_LAUNCH_BLOCKING=1, then compare the same request with a supported fallback attention backend.
Active LoRA adapter rank exceeds engine pre-allocated max_lora_rank
The requested adapter has a rank larger than the LoRA slots vLLM reserved when the engine started. The reservation is fixed for that engine process and cannot grow when the adapter is loaded.
First action: Inspect the r value in each adapter_config.json, then restart vLLM with --max-lora-rank set to at least the largest adapter rank you intend to serve.
PlacementGroupCreationError: placement group creation timed out
Ray could not find a simultaneous placement for all requested bundles. A group that needs eight GPUs on one node cannot use eight GPUs that are free but scattered across several nodes.
First action: Run ray status and compare the free resources per node with every bundle and its PACK, STRICT_PACK or SPREAD strategy before changing the placement timeout.
Sequence group is preempted by PreemptionMode.RECOMPUTE because there is not enough KV cache space
The scheduler admitted more active sequence state than the KV cache can retain, so it repeatedly evicts and recomputes requests. Throughput collapses even though the service may never throw an error.
First action: Lower --max-num-seqs for one load test and watch the preemption counter. If it falls while useful token throughput rises, admission pressure was the cause.
Most of these are diagnostics rather than fixes. If one makes a failure disappear, it has told you where the fault is, not removed it.
Environment variables and what each one does.
Variable
What it does
VLLM_LOGGING_LEVEL=DEBUG
Raises vLLM log detail so worker setup, scheduler decisions and the first engine failure remain visible. Use it for a bounded reproduction because output volume is high.
NCCL_DEBUG=TRACE
Adds communicator and transport evidence for tensor-parallel startup failures. It is diagnostic and should not remain enabled in normal serving.
VLLM_HOST_IP
Pins the address vLLM workers advertise when interface auto-detection selects an unreachable container or management address.
--enforce-eager
Disables CUDA graphs for a control run. If startup or a kernel failure disappears, graph capture or its memory overhead is part of the failing condition.
--gpu-memory-utilization
Sets the fraction of total device memory vLLM may use. Higher values enlarge the cache budget but leave less headroom for peaks and co-resident work.
--max-model-len
Caps the served context window explicitly. Prompt tokens plus requested completion tokens must fit this value and the available KV cache.
Frequently asked questions
Why does vLLM say there is no cache memory when nvidia-smi shows free memory?
vLLM budgets a configured fraction of total device memory, subtracts model weights and its profiled activation peak, then uses what remains for KV cache blocks. Memory outside that budget can appear free in nvidia-smi while the calculated cache allowance is still zero.
Is EngineDeadError the root cause of a vLLM outage?
No. EngineDeadError is the API-side process reporting that EngineCore has already exited. Preserve the first worker or engine traceback above it and check the host kernel log for an OOM kill or NVIDIA Xid before restarting, because that earlier evidence identifies the owner.
Should I raise gpu_memory_utilization for every vLLM memory error?
No. Raising it can create more KV cache space on an otherwise idle GPU, but it can also remove the headroom needed for activation peaks or co-resident processes. Confirm the reported memory profile and change one of context length, concurrency or utilization at a time.
How do I separate a bad request from a bad vLLM worker?
Replay the exact request against one known-good worker with the same model and pinned software stack. If the failure follows the request, inspect shape, length, dtype and adapter state. If it follows the node, inspect the device, driver, loaded kernels and host logs.
What evidence should I collect before restarting vLLM?
Keep the first traceback from every worker, the full launch command, model and quantization identifiers, vLLM and PyTorch versions, GPU model, driver version, the failing request shape, nvidia-smi output and relevant host kernel lines. A final HTTP 500 alone is not diagnostic.
Knowing the error is not knowing the cause
Which rank failed first, whether the checkpoint is safe to resume from, and whether this is your code or the hardware. Paste the log and Denpex answers all three, free and without an account.