Tensor on device meta is not on the expected device: a device mismatch from a parameter never materialised

A model built with automatic device placement runs until one submodule executes with a parameter still on the placeholder device. Meta tensors carry shape and dtype but no data, so the operation refuses rather than computing on nothing, and the error names a device that is not a device at all.

A forward-pass failure in which a parameter left on the meta placeholder device meets a real tensor in an operation, because automatic placement never attached a hook to materialise the submodule that owns it.

Large-model loading deliberately separates structure from data so a model larger than one device can be described before it is populated. That separation is safe only while every parameter is subsequently materialised, and the mechanism responsible knows only about the modules it enumerated. Anything outside that enumeration keeps its placeholder, and a placeholder is indistinguishable from a real parameter until an operation needs its values.

• Meta is not a real device, so the message reads as nonsense on first encounter
• Most of the model works, which points suspicion at the input rather than the weights
• The failing submodule is often a secondary encoder that placement treated differently from the main network
• The traceback ends in shape-inference internals, far from the placement decision that caused it

• Automatic device placement, which loads a skeleton on the placeholder device and fills it in per submodule
• Offload hooks that move weights onto the accelerator immediately before a submodule runs and off it afterwards
• Multi-component pipelines where a vision or text encoder is placed separately from the main network
• Modules constructed after placement was computed, which no hook was ever attached to

Apply the following checklist to a small reproduction: each box below is a positive signal that you are looking at this exact failure rather than a sibling in the same taxonomy.

• ✓Walk the model's parameters and buffers after loading and report any whose device is the placeholder. The list names exactly what was missed.
• ✓Compare the failing submodule's name against the placement map. A submodule with no entry was never scheduled to be materialised.
• ✓Load the same model onto a single device without automatic placement. Running correctly there confirms placement rather than the model or the input.

RuntimeError: Tensor on device meta is not on the expected device cuda:0!
  File ".../torch/_prims/__init__.py", in _prim_elementwise_meta
  File ".../torch/_library/fake_impl.py", in meta_kernel
  File ".../accelerate/hooks.py", in new_forward

Timestamps and exact values vary across runs, but the pattern. An info-level start, an early WARN, an ERROR carrying the symptom. Is the actual fingerprint you should alert on. The Denpex platform flags this combination automatically.

Placing the component explicitly gives its parameters real storage on a real device, so the operation has values on both sides and proceeds. Materialising the whole model achieves the same by leaving no placeholders anywhere. Neither changes how the model computes; they change whether the numbers exist at the moment they are required.

// Typical pattern:
import torch.optim as optim
optimizer = optim.AdamW(model.parameters(), lr=3e-4)
scheduler = optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=num_training_steps)
for step in range(num_training_steps):
    optimizer.zero_grad()
    loss = model(batch)
    loss.backward()
    optimizer.step()
    scheduler.step()

Adapt the snippet to your framework. The same pattern holds for PyTorch Lightning, Hugging Face Trainer, DeepSpeed, Megatron-LM, and vLLM training wrappers. Where the wrapper exposes a config flag (for examplelr_scheduler_type in Trainer), prefer the flag over the imperative API to keep the schedule declarative and reproducible.