Treating successful import as a GPU test
Import can succeed before the first 4-bit operation exercises the selected kernel.
Operator task guide
Start with the first exception from the same Python environment and GPU as the failing trainer. A CUDA visibility failure, a missing bitsandbytes library, an unsupported 4-bit kernel and an adapter configuration error require different checks.
Verify four stages independently: CUDA visibility, 4-bit model loading, the first forward and backward pass, and an adapter optimizer step. A model that loads successfully can still fail at the next stage.
Reviewed . Reference guidance is not a diagnosis of your workload.
Run these commands inside the failing container. If PyTorch cannot see the intended GPU, investigate assignment, driver visibility and the installed PyTorch build before changing quantization.
python -m pip show torch bitsandbytes transformers peft
python -c "import torch; print(torch.cuda.is_available(), torch.version.cuda)"
python -m bitsandbytesA missing shared library identifies a loader problem. A no-kernel-image error during a 4-bit operation points to a different check: compare the selected wheel, GPU compute capability and documented support for the package release.
Load a known supported model with the intended 4-bit configuration, prepare it for k-bit training and confirm PEFT selected the expected trainable adapter parameters. Run a small nonprivate batch through forward, backward and one optimizer step. Check finite loss and gradients and confirm an intended adapter parameter changed.
Add the original model, sequence length, batch size and distributed launch one at a time. Keep the first error and the failing stage, especially if the reduced example passed.
| Signal | What it means | Next action |
|---|---|---|
| CUDA is unavailable in PyTorch | The trainer cannot use the intended NVIDIA GPU. | Check GPU assignment and the installed PyTorch build in the failing environment. |
| bitsandbytes cannot load a native library | The selected backend is unavailable or one of its dependencies failed. | Read the first loader error and compare the installed release with its support matrix. |
| No kernel image during matmul_4bit | The selected binary may lack a compatible kernel for that GPU. | Compare compute capability and wheel support before changing model settings. |
| Load passes but backward or update fails | The failure is later than model loading. | Check the PEFT configuration, trainable parameters, first traceback and the exact batch. |
Import can succeed before the first 4-bit operation exercises the selected kernel.
Keep the baseline and change one compatibility variable so the next result is explainable.
Capture the failing interpreter, package versions and first loader error.
Check the device and selected binary at the first matmul failure.
Catch runner and environment differences before the training job starts.
No. Exercise forward, backward and an adapter optimizer step. Check finite values and verify an intended adapter parameter updates.
Use the three free diagnoses to review your error and relevant evidence. Keep reference guidance separate from the cause and recovery status of your own workload.
Diagnose your incident