Fix Docker Could Not Select Device Driver "" with Capabilities: [[gpu]]

Root Causes Explained

When you pass --gpus all to the Docker CLI, Docker sends a device request to dockerd asking for a driver that advertises the [[gpu]] capability. The empty quotes "" in the error message indicate that Docker received a default GPU request without an explicit driver name and could not find any registered handler in its runtime table.

OS-Specific NVIDIA Container Toolkit Installation

Before installing the container toolkit, verify that the host kernel driver is loaded by running nvidia-smi on the host machine. Once confirmed, run the repository setup and package installation commands for your Linux distribution.

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg

curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
  sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo

sudo dnf clean expire-cache
sudo dnf install -y nvidia-container-toolkit

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
  sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo

sudo dnf install -y nvidia-container-toolkit

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Complete /etc/docker/daemon.json Configuration

Running sudo nvidia-ctk runtime configure --runtime=docker updates /etc/docker/daemon.json. If you manage configuration via Ansible, Terraform, cloud-init, or manual editing, inspect the file to verify it matches the structure below.

{
  "runtimes": {
    "nvidia": {
      "args": [],
      "path": "nvidia-container-runtime"
    }
  }
}

{
  "default-runtime": "nvidia",
  "exec-opts": [
    "native.cgroupdriver=systemd"
  ],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m",
    "max-file": "3"
  },
  "runtimes": {
    "nvidia": {
      "args": [],
      "path": "nvidia-container-runtime"
    }
  }
}

$ docker info --format '{{json .Runtimes}}'
{"io.containerd.runc.v2":{"path":"runc"},"nvidia":{"path":"nvidia-container-runtime"},"runc":{"path":"runc"}}

Podman and Rootless Docker CDI Configuration

If you run Rootless Docker or Podman, modifying /etc/docker/daemon.json alone will not resolve the device driver failure. Use Container Device Interface (CDI) or user-scoped daemon configs as shown below.

nvidia-ctk runtime configure --runtime=docker \
  --config=$HOME/.config/docker/daemon.json

sudo nvidia-ctk config --set nvidia-container-cli.no-cgroups --in-place

systemctl --user restart docker

sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml

# Verify detected CDI devices
nvidia-ctk cdi list

# Run container with CDI device flag
podman run --rm --device nvidia.com/gpu=all \
  nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

Verification Using an Official CUDA Base Image

Test GPU device selection and library injection with a lightweight official NVIDIA CUDA base container. This image is under 100 MB and verifies that /dev/nvidia* nodes and libnvidia-ml.so are mounted inside the container namespace.

sudo docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

Expected healthy output:

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.90.07              Driver Version: 550.90.07      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|=========================================+========================+======================|
|   0  NVIDIA A100-SXM4-80GB          On  |   00000000:07:00.0 Off |                    0 |
| N/A   34C    P0             52W /  400W |       1MiB /  81920MiB |      0%      Default |
+-----------------------------------------+------------------------+----------------------+

services:
  inference-worker:
    image: nvidia/cuda:12.4.1-base-ubuntu22.04
    command: nvidia-smi
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

Frequently Asked Questions