vLLM returns 500 on every request after the EngineCore process dies

The vLLM V1 API server survives while the separate EngineCore subprocess that owns the model does not. Once that child is gone the server keeps accepting connections and answers every one of them with an internal error until it is restarted, so an outage presents as a healthy-looking process serving nothing.

An outage mode specific to vLLM's split-process design, in which the model-owning EngineCore subprocess terminates while the HTTP server that fronts it continues running and answering requests with internal errors.

vLLM V1 puts the model in its own process so the HTTP layer stays responsive. The consequence is that the two can fail independently. When the child dies, the parent finds out only when it next awaits output, and all it can honestly say is that the engine is gone. Nothing in that message distinguishes a missing CUDA kernel from a stalled worker from a subprocess reaped by the kernel, because the parent never saw any of them happen.

• Every completion request returns HTTP 500 while the container stays up and the port stays open
• A liveness probe that only checks the port keeps the dead instance in the load balancer
• The error the server reports names no cause, so there is nothing obvious to act on
• Restarting clears it for a while and it returns under the same conditions

• vLLM V1, where the API server and EngineCore are separate processes
• Containers whose health check tests only TCP reachability rather than a real completion
• Deployments using VLLM_WORKER_MULTIPROC_METHOD=spawn
• Any autoscaled fleet, where one dead replica silently absorbs its share of traffic

Apply the following checklist to a small reproduction: each box below is a positive signal that you are looking at this exact failure rather than a sibling in the same taxonomy.

• ✓Search the log backwards from the EngineDeadError for the first traceback attributed to a worker or engine-core process; that is the cause and it will name a device, a memory limit or a timeout.
• ✓Run dmesg -T | tail and look for an out-of-memory kill naming a Python process at the time of the failure. A kill leaves no application traceback.
• ✓Issue a one-token completion against the instance. If it returns 500 while the port accepts connections, the engine is dead and the process needs restarting rather than investigating live.

ERROR: Engine core proc EngineCore_DP0 died unexpectedly, shutting down client.
vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
INFO:     10.0.4.19:52344 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error

Timestamps and exact values vary across runs, but the pattern. An info-level start, an early WARN, an ERROR carrying the symptom. Is the actual fingerprint you should alert on. The Denpex platform flags this combination automatically.

Reading upward finds the exception raised where the failure actually occurred, which names a device, a limit or a timeout and therefore points at a specific change. Replacing the liveness probe fixes the second half of the problem: it converts a permanent silent outage into a restart, so a cause you have not diagnosed yet still stops costing traffic while you diagnose it.

// Typical pattern:
import torch.optim as optim
optimizer = optim.AdamW(model.parameters(), lr=3e-4)
scheduler = optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=num_training_steps)
for step in range(num_training_steps):
    optimizer.zero_grad()
    loss = model(batch)
    loss.backward()
    optimizer.step()
    scheduler.step()

Adapt the snippet to your framework. The same pattern holds for PyTorch Lightning, Hugging Face Trainer, DeepSpeed, Megatron-LM, and vLLM training wrappers. Where the wrapper exposes a config flag (for examplelr_scheduler_type in Trainer), prefer the flag over the imperative API to keep the schedule declarative and reproducible.