DeepGEMM rejects FP8 block-scaled weights with Unknown SF transformation on an unsupported architecture

A block-scaled FP8 checkpoint fails while its weights are being prepared, before the server ever accepts a request. The runtime selected a specialised matrix-multiply library for the detected architecture, and that library does not implement the scale-factor layout this class of device requires, so it refuses the transformation outright.

A load-time failure in which a serving runtime selects a specialised FP8 matrix-multiply library for a device inside an accepted compute-capability family, and the library has no scale-factor layout transformation implemented for that architecture.

Kernel selection has to decide from something, and compute capability is the number available at runtime. Grouping it into families is a reasonable approximation right up until a family contains parts with genuinely different capabilities, which is what happens when a generation spans consumer and data-centre silicon. The runtime then admits a device to a fast path whose library never claimed it, and the refusal comes from the library rather than from the check that should have made it.

• The failure is at weight preparation, so nothing serves and there is no partial capability to fall back to
• The message is from a kernel library's internals and names a layout concept, not a device or a model
• The same checkpoint loads correctly on data-centre parts of the same generation
• Both workers fail identically, which makes it look like a model problem rather than a hardware-support one

• Block-scaled FP8 checkpoints, where each block of weights carries its own scale rather than one scale per tensor
• Consumer Blackwell parts reporting compute capability 12.0, which the runtime groups with data-centre Blackwell for kernel selection
• Serving runtimes that choose a specialised GEMM library from a compute-capability family rather than from a tested device list
• Tensor-parallel deployments, where the identical preparation runs on every worker and fails on all of them

Apply the following checklist to a small reproduction: each box below is a positive signal that you are looking at this exact failure rather than a sibling in the same taxonomy.

• ✓Check the reported compute capability of the device against the parts the kernel library documents. A capability inside an accepted family but outside the implemented list is the whole condition.
• ✓Load a per-tensor quantised version of the same model. Succeeding where the block-scaled one failed confirms the scale layout rather than the checkpoint.
• ✓Force a general-purpose quantised path and reload. Starting successfully identifies kernel selection as the decision that failed.

RuntimeError: Assertion error (/workspace/.deps/deepgemm-src/csrc/apis/layout.hpp:60): Unknown SF transformation
compressed_tensors_w8a8_fp8.py:169  process_weights_after_loading
  -> kernels/linear/scaled_mm/deep_gemm.py:96  process_weights_after_loading
    -> quantization/utils/fp8_utils.py:1077  deepgemm_post_process_weight_scale_block
      -> utils/deep_gemm.py:494  transform_sf_into_required_layout

Timestamps and exact values vary across runs, but the pattern. An info-level start, an early WARN, an ERROR carrying the symptom. Is the actual fingerprint you should alert on. The Denpex platform flags this combination automatically.

A general-purpose quantised kernel is written against the baseline features of the architecture rather than against a specific product's tensor cores, so it covers the device at a lower throughput. Moving to per-tensor scaling removes the requirement entirely, because a single scale per tensor needs no layout transformation. Both replace an unimplemented routine with one that exists, which is the only thing standing between the checkpoint and a running server.

// Typical pattern:
import torch.optim as optim
optimizer = optim.AdamW(model.parameters(), lr=3e-4)
scheduler = optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=num_training_steps)
for step in range(num_training_steps):
    optimizer.zero_grad()
    loss = model(batch)
    loss.backward()
    optimizer.step()
    scheduler.step()

Adapt the snippet to your framework. The same pattern holds for PyTorch Lightning, Hugging Face Trainer, DeepSpeed, Megatron-LM, and vLLM training wrappers. Where the wrapper exposes a config flag (for examplelr_scheduler_type in Trainer), prefer the flag over the imperative API to keep the schedule declarative and reproducible.