Skip to content

Operator task guide

CUDA 13.0 & 13.5 compatibility for PyTorch, vLLM and SGLang

CUDA 13.0 (cu130) and CUDA 13.5 toolkits require a Linux NVIDIA driver on branch 580 or newer on the host unless container forward-compatibility libraries match the compute capability and kernel interface.

Never mix cu128 PyTorch or FlashAttention extension wheels inside a CUDA 13.x runtime image without checking ABI symbols and bundled libcudart versions first.

Reviewed . Reference guidance is not a diagnosis of your workload.

Step-by-step method

  1. 1

    Compare host driver branch against CUDA 13.x container toolkit

    Inspect the host kernel driver version and confirm the active libcuda.so interface supports CUDA 13.0 or 13.5 runtime calls.

    nvidia-smi --query-gpu=driver_version,name,pci.bus_id --format=csv
    nvcc --version
  2. 2

    Inspect PyTorch, vLLM, SGLang, and FlashAttention compiled CUDA tags

    Print torch.version.cuda, _GLIBCXX_USE_CXX11_ABI, and loaded shared objects before rebuilding custom CUDA extensions from source.

    python -c "import torch; print(torch.__version__, torch.version.cuda, torch._C._GLIBCXX_USE_CXX11_ABI, torch.cuda.get_arch_list())"
    pip list | rg -i 'torch|vllm|sglang|flash|triton'
  3. 3

    Verify Blackwell sm_100 and Hopper sm_90a wheel coverage

    Confirm installed binaries include native SASS cubins for your GPU compute capability so inference engines do not stall on runtime PTX JIT compilation or abort with no kernel image.

    python -c "import torch; print(torch.cuda.get_device_capability(0))"

CUDA 13.0 / 13.5 upgrade compatibility checks

Signals, meanings and actions for cuda 13.0 & 13.5 compatibility for pytorch, vllm and sglang.
SignalWhat it meansNext action
CUDA driver version is insufficient for CUDA runtime versionHost driver is below 580.x for a native CUDA 13.x container without valid compat librariesUpgrade the host driver to branch 580+ or pin the container image to cu128 / cu126 matching the host.
undefined symbol: _ZN3c10... after cu130 upgradeExtension wheel (flash-attn, sgl-kernel, vllm C++ ops) linked against a mismatched PyTorch or CUDA minor versionUse /tools/cuda-pytorch-compatibility to select matched PyTorch, CUDA index, and ABI builds.
no kernel image is available for execution on the deviceBinary wheel omitted sm_100 (Blackwell B200) or sm_90 (Hopper H100) code generation flagsInstall a wheel built with TORCH_CUDA_ARCH_LIST including your device capability.

Evidence checklist

  • Host nvidia-smi driver version and container CUDA toolkit output
  • torch.__version__, torch.version.cuda, and C++11 ABI boolean
  • Exact pip or uv lockfile for PyTorch, vLLM, SGLang, and FlashAttention
  • First ImportError or CUDA runtime traceback before falling back to PTX JIT

Common mistakes

Upgrading only PyTorch without matching vLLM or SGLang custom kernels

Serving frameworks compile custom fused attention and quantization kernels against exact PyTorch and CUDA headers.

Assuming CUDA 12.x host drivers can always run CUDA 13.5 binaries

Forward-compatibility packages only bridge documented datacenter hardware and library boundaries; mismatch triggers immediate initialization failures.

Frequently asked questions

Which NVIDIA driver version does CUDA 13.0 and 13.5 require?

Native CUDA 13.0 and 13.5 toolkits expect NVIDIA Linux driver branch 580 or newer on bare-metal hosts unless an officially supported CUDA forward-compatibility package is mounted cleanly in the container.

Can I install CUDA 12.8 vLLM or SGLang wheels inside a CUDA 13.0 PyTorch image?

Mixing major CUDA toolkit ABIs across compiled extensions such as flash-attn, vllm, or sgl-kernel frequently produces undefined C++ symbol errors or segmented memory faults. Match the exact CUDA index across every native wheel.

Apply the method to your incident

Use the three free diagnoses to review your error and relevant evidence. Keep reference guidance separate from the cause and recovery status of your own workload.

Diagnose your incident