vLLM tensor parallelism fails in a container because shared memory is limited to 64 MB

A container gets 64 MB of shared memory by default. PyTorch and NCCL use that space to move data between the worker processes of a tensor-parallel deployment, so a model that runs on one GPU fails to initialise across several, with an error naming shared memory rather than the model.

Symptom

NCCL WARN Error while creating shared memory segment /dev/shm/nccl-vzIpS6 (size 9637888)

Root cause

Container runtimes mount a small tmpfs at /dev/shm by default, historically 64 MB. That default was chosen for ordinary applications and is unrelated to what collective communication libraries need. Tensor-parallel vLLM moves tensors between worker processes through that space, and NCCL creates segments there during communicator setup.

Recommended fix

NCCL_DEBUG=INFO

How Denpex helps

Denpex matches vLLM tensor parallelism fails in a container because shared memory is limited to 64 MB across every rank in a distributed run and reports which rank failed first, so you act on the initiating node instead of the loudest one.

A container configuration failure in which the default 64 MB shared memory mount is too small for the inter-process transfers that tensor-parallel vLLM and NCCL perform, preventing the engine from initialising across multiple GPUs.

The shared memory default is a property of the container runtime, chosen without reference to collective communication. vLLM's workers exchange tensors through that mount and NCCL creates segments in it during setup, so a limit that is generous for a web service is far too small here. Because a single-GPU deployment never touches the segment, the constraint stays hidden until the day someone scales across devices.

• The same image and model work with one GPU and fail with two or more
• The failure occurs during initialisation, before any request is served
• The error names NCCL or a shared memory segment, which does not obviously implicate the container runtime
• The deployment works locally and fails on the orchestrator, or the reverse

• Docker containers started without --shm-size or --ipc=host, which receive the 64 MB default
• Kubernetes pods, where the default shared memory size is frequently too small for these containers
• Docker Swarm, which does not support the host IPC option at all
• Any tensor-parallel or pipeline-parallel vLLM deployment, since single-GPU serving does not exercise this path

Use this checklist to test the hypothesis against a small reproduction. No single line proves the root cause, so preserve the preceding events and compare one variable at a time.

• ✓Run df -h /dev/shm inside the container. A total of 64 MB confirms the default mount is in place and is the limit being hit.
• ✓Start the identical image with tensor parallel size 1. Success on one GPU and failure on more localises the problem to inter-worker communication rather than to the model or weights.
• ✓Re-run with NCCL_DEBUG=INFO and read the first warning. A message naming a shared memory segment points here; one naming peer access points at the transport instead.

• Container runtimes mount a small tmpfs at /dev/shm by default, historically 64 MB. That default was chosen for ordinary applications and is unrelated to what collective communication libraries need.
• Tensor-parallel vLLM moves tensors between worker processes through that space, and NCCL creates segments there during communicator setup. When the mount is too small the segment cannot be created and initialisation fails with a system error that names the segment rather than the limit, which is why the cause is not obvious from the message.
• The failure is invisible on a single GPU because nothing needs the shared segment. It appears the first time the deployment is scaled across devices, which is often long after the image was validated.

NCCL WARN Error while creating shared memory segment /dev/shm/nccl-vzIpS6 (size 9637888)
torch.distributed.DistBackendError: NCCL error: unhandled system error (run with NCCL_DEBUG=INFO for details)
WARNING: /dev/shm has only 67108864 bytes available. This will harm performance! You may be able to free up space by deleting files in /dev/shm.

Use this text as a lookup key in logs and upstream issue trackers. It is not presented as a captured customer log. Confirm the cause from your own preceding events, versions, configuration and the cited references.

Enlarging the mount, or sharing the host's, gives the segment room to be created, so communicator setup completes and the workers can exchange tensors. It is a configuration change to the container rather than to vLLM, which is why no vLLM flag resolves it and why the same image behaves differently across environments.

Select the traceback or open the failed terminal, then run Denpex locally to see the initiating rank, collateral failures, exact fix, and verification command without uploading the log.

vLLM failures cross request, model, KV cache, worker, kernel and host boundaries. The hub compares common literal signatures and the first control that separates their causal owners.