TensorRT-LLM serving fails with an AttributeError from the FlashInfer attention backend

The FlashInfer attention backend inside TensorRT-LLM reads fields from an attention metadata object that the release bundled beside it does not define. Serving fails with a plain Python AttributeError naming the missing field, so a version-skew problem between two components arrives looking like ordinary application code being wrong.

Symptom

AttributeError: 'TrtllmAttentionMetadata' object has no attribute 'kv_layout'

Root cause

The attention backend and the metadata object it consumes are written in two separately versioned components. The backend reads a field the metadata class in this build never declares, so Python raises at the moment of access. Nothing validates that the two agree before serving begins.

Recommended fix

Read the line reporting which attention backend the applied model defaults selected. It names the component that failed, which the command itself does not.

How Denpex helps

Denpex matches TensorRT-LLM serving fails with an AttributeError from the FlashInfer attention backend across every rank in a distributed run and reports which rank failed first, so you act on the initiating node instead of the loudest one.

A serving failure in which TensorRT-LLM's FlashInfer attention backend accesses a metadata field the bundled build does not define, raising an AttributeError at the first fused-attention call rather than at load time.

Python resolves attributes when they are reached, not when the module is imported. A backend compiled against one metadata contract can therefore load cleanly beside a build that implements another, and the mismatch waits until fused attention runs. Because model defaults choose the backend automatically, the component that breaks is one the operator never selected and never sees until the traceback.

• The failure names an attribute, which reads as a coding error rather than a packaging one
• The model, the flags and the hardware are all supported, and the command is the documented one
• It appears in a release-candidate container that is newer than the last one that worked
• Changing model or batch settings has no effect, because none of them are involved

• trtllm-serve running any model whose defaults select the FlashInfer attention backend
• Release-candidate TensorRT-LLM containers, where the two components move independently between builds
• Environments where FlashInfer, TensorRT-LLM or torch has been upgraded in place inside the image
• Model families such as Gemma whose applied defaults choose FlashInfer without the operator asking for it

Use this checklist to test the hypothesis against a small reproduction. No single line proves the root cause, so preserve the preceding events and compare one variable at a time.

• ✓Find the applied-model-defaults line in the startup log and confirm it selected FlashInfer, which identifies the component the traceback belongs to.
• ✓Compare the installed TensorRT-LLM and FlashInfer versions against the pairing the published image shipped with; a difference in either shows the image was modified.
• ✓Re-run with a different attention backend. Serving successfully confirms interface skew rather than a fault in the model or the request.

• The attention backend and the metadata object it consumes are written in two separately versioned components. The backend reads a field the metadata class in this build never declares, so Python raises at the moment of access. Nothing validates that the two agree before serving begins.
• Attribute access is late-bound, so the disagreement cannot be detected at import, at model load, or by any flag check. The first request to reach fused attention is where it surfaces, which is why the server starts, reports its defaults, and only then fails.
• The backend was usually not chosen by the operator. Model defaults applied at load time select it, so the failing component is one the command never mentions and the traceback is the first place it appears.

AttributeError: 'TrtllmAttentionMetadata' object has no attribute 'kv_layout'
  File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/_torch/attention_backend/flashinfer.py", line 1518, in forward_impl
    self.layer_idx, kv_layout=metadata.kv_layout)
[TRT-LLM] [I] [_torch] Applied model defaults for Gemma4ForConditionalGeneration: {'attn_backend': 'FLASHINFER'}

Use this text as a lookup key in logs and upstream issue trackers. It is not presented as a captured customer log. Confirm the cause from your own preceding events, versions, configuration and the cited references.

Using a container whose components were released together restores the contract the backend was written against, so the field it reads exists. Overriding the backend achieves the same outcome from the other direction, by routing attention through an implementation this build does satisfy. Neither is a tuning change, because no setting of the model or the request was ever involved.

Select the traceback or open the failed terminal, then run Denpex locally to see the initiating rank, collateral failures, exact fix, and verification command without uploading the log.