Fix: bitsandbytes Compiled Without GPU Support Error
Resolve the common runtime warning and quantization failures when bitsandbytes fails to detect your NVIDIA GPU driver or loads the default CPU stub wheel.
Bottom Line Up Front (BLUF)
When running model training, QLoRA fine-tuning, or inference with bitsandbytes quantization, you may encounter this exact warning or traceback:
UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and quantization features will be unavailable.Fix this immediately with the appropriate pip command for your environment:
pip uninstall -y bitsandbytes && pip install bitsandbytesTry Denpex for 30 days. FREE.
No credit card. No automatic subscription. See whether Denpex helps with an incident you already understand.
Start your free 30-day Team evaluation
Up to 50 diagnoses a day for 30 days. Verify your work email to activate an eligible company evaluation, then replay a resolved incident and compare the evidence and next steps with your team's confirmed cause.
Root Causes Explained
1. Generic CPU Wheel Default
Standard pip installation without explicit wheel selection often downloads a generic stub package built exclusively for CPU execution, lacking CUDA runtime bindings.
2. Missing Library Paths
The dynamic linker cannot locate libcudart.so or associated CUDA shared libraries because they are absent from LD_LIBRARY_PATH.
3. PyTorch and Toolkit Mismatch
A mismatch between your active PyTorch CUDA build (such as CUDA 11.8) and your system-installed NVIDIA driver toolkit (such as CUDA 12.4) breaks dynamic symbol resolution.
4. Windows vs WSL2 Discrepancies
Windows native Python installations require different DLL paths and prebuilt Windows wheels compared to Linux or Windows Subsystem for Linux (WSL2) environments.
Comprehensive Compatibility Matrix
| Environment | PyTorch Version | CUDA Toolkit | Recommended Install Command |
|---|---|---|---|
| Linux / WSL2 | 2.4.x or higher | CUDA 12.4 / 12.6 | pip install bitsandbytes |
| Linux / WSL2 | 2.1.x to 2.3.x | CUDA 12.1 | pip install bitsandbytes==0.43.1 |
| Legacy Cluster | 2.0.x or older | CUDA 11.8 | pip install bitsandbytes-cuda118 |
| Windows Native | 2.2.0+ | CUDA 12.1 / 12.4 | pip install bitsandbytes |
Diagnostic Command Checklist
Run these diagnostics in your active Python environment to verify whether bitsandbytes successfully detects your GPU hardware and CUDA libraries:
1. Inspect built-in module status
python -m bitsandbytes2. Locate compiled shared libraries on disk
find / -name libbitsandbytes_cuda*.so 2>/dev/null3. Verify PyTorch CUDA availability
python -c "import torch; print(torch.cuda.is_available(), torch.version.cuda)"Related Diagnostic Tools
Explore our suite of utilities for diagnosing GPU crashes, calculating memory overhead, and inspecting hardware errors: