Skip to content

NVIDIA Xid error codes

An Xid is the NVIDIA kernel driver reporting that something went wrong on the GPU. The number identifies an event family. The full message, GPU identity, application traceback, driver and peer evidence determine what to investigate next.

This reference covers 23 codes. Classifications and replacement candidates are triage guidance, not proof that a component is defective. Preserve the evidence and verify the affected workload after an approved recovery.

Compare Xid 43, 62, 74, 79 and 149 evidence and recovery checks.

What should I collect after an NVIDIA Xid error?

Keep the complete event, host, timestamp and timezone, GPU UUID, PCI address, workload and rank mapping. Compare nearby host and peer events before naming the cause. A container GPU index alone does not identify the physical device.

# Read-only inventory where host permissions allow it.
nvidia-smi --query-gpu=index,uuid,pci.bus_id --format=csv
# Ask the operator for a bounded kernel-log window if unavailable.
journalctl -k --since "2026-11-04 02:10:00 UTC" --until "2026-11-04 02:20:00 UTC"

Replace the example time window with the incident time. Inventory may fail after a device becomes inaccessible; use a retained pre-incident mapping rather than guessing.

Example: Xid 79 followed by an NCCL timeout

In an illustrative incident, a host logs Xid 79 at 02:14 and surviving ranks time out at 02:15. This establishes a device-access event before the timeout. It does not establish a defective card, a network cause or an exact workload owner without the device mapping.

Investigate Xid 79 and prepare the provider handoff, or follow the simulated monitoring-to-investigation walkthrough.

NVIDIA Xid decoder

Paste the kernel line to extract the code, fault class, RMA relevance and first evidence-preserving action.

Upload your log file. Large files are processed in your browser. Only selected evidence is submitted when you run diagnosis.

Example: NVRM: Xid (PCI:0000:81:00): 79. This decoder parses your log locally without uploading its contents.

Live decode

Every common Xid code

Sorted by code. Classifications guide investigation; they do not establish which component caused this incident.

NVIDIA Xid error codes with fault class, RMA guidance and first action.
XidNameClassWhat it meansFirst action
Xid 13Graphics Engine ExceptionApplication / softwareA kernel executed an illegal operation, most often an out-of-bounds memory access or a bad pointer in application or library code. The GPU raised an exception and killed the offending context.Run the workload under compute-sanitizer. Xid 13 is an application bug far more often than a hardware fault, and the sanitizer names the kernel and the access.
Xid 31GPU memory page fault (MMU fault)Application / softwareThe GPU MMU rejected a memory access: the address was unmapped, or the access type was not permitted for that mapping. Usually a bad pointer, a use-after-free, or a race on a freed allocation.compute-sanitizer --tool memcheck. The fault message names the engine and access type, which narrows it to a kernel quickly.
Xid 32Invalid or corrupted push buffer streamAmbiguous, needs evidenceThe command stream the driver submitted was rejected as malformed. Can be a driver/user-mode bug, and can also be memory corruption on the path that built the buffer.Check for concurrent Xid 48/94/95 (ECC) on the same GPU, corruption upstream explains a malformed stream. Absent that, treat it as a driver/library bug and check versions.
Xid 38Driver firmware errorDriver or firmwareThe driver firmware hit an internal error. Not caused by application code.Compare driver and VBIOS against the fleet baseline; a mismatched pair is the usual cause. Reset the GPU, then update if it recurs.
Xid 43GPU stopped processingApplication / softwareA GPU channel stopped processing after an error. This event alone does not establish whether the device is healthy or identify the initiating fault.Preserve the full event, surrounding kernel messages and application traceback. Compare earlier errors and the installed driver guidance before recovery.
Xid 45Preemptive cleanup, due to previous errorsApplication / softwareThe driver tore down a channel because something else already failed. This is TEARDOWN, not a root cause, it is also what appears when a user kills a job with Ctrl-C.Do not diagnose Xid 45. Run `dmesg -T | grep -i xid` and read the Xid immediately BEFORE it; that one is the incident.
Xid 48Double Bit ECC Error (DBE)HardwareRMA candidateAn uncorrectable memory error. ECC caught two flipped bits, which it can detect but not repair, so the data is lost and every context using that memory is suspect.Drain the GPU. Resume from a checkpoint written BEFORE the Xid timestamp, anything computed after it may be silently wrong.
Xid 61Internal micro-controller breakpoint/warningDriver or firmwareRMA candidateThe GPU management processor (PMU/falcon) hit an internal condition. The GPU is typically unusable until reset, and a repeat on one serial points at silicon or firmware.Reset the GPU (nvidia-smi --gpu-reset -i <id>) or reboot. Check the driver/VBIOS pairing against the fleet baseline.
Xid 62Internal micro-controller haltDriver or firmwareThe internal microcontroller halted. This event alone does not distinguish a driver or firmware issue from defective hardware.Preserve the bug report, drain GPU clients, and attempt a supported GPU reset. If recovery fails, reboot or power-cycle per the platform procedure. Escalate recurrence or failed diagnostics to NVIDIA/OEM support.
Xid 63ECC page retirement or row remapping recording eventHardwareThe GPU retired a bad memory page (or remapped a row on A100 and later) and recorded it. This is ECC working as designed, a single event is not a failure.Track the RATE, not the event. Check remap pool headroom: nvidia-smi -q -d ROW_REMAPPER. Retirements accelerating over days predict Xid 64.
Xid 64ECC page retirement or row remapper recording failureHardwareRMA candidateThe GPU tried to retire a failing page or row and could not record it. The protective mechanism itself has failed, so further errors cannot be contained.RMA. Unlike Xid 63 this is terminal, the spare-row reserve or its bookkeeping is exhausted.
Xid 65Video processor exceptionAmbiguous, needs evidenceThe video engine raised an exception, usually during encode or decode work.Confirm a codec workload was running: `nvidia-smi --query-gpu=utilization.encoder,utilization.decoder --format=csv`. Isolate by disabling the hardware codec path.
Xid 69Graphics Engine class errorApplication / softwareAn invalid class or method was submitted to the graphics engine, a driver or application-level programming error.Run `nvidia-smi` and compare the driver/CUDA pair against the application build. This is not a hardware signal, do not drain the node for it.
Xid 74NVLink errorHardwareRMA candidateAn error on an NVLink connection between GPUs, or between a GPU and an NVSwitch. Collectives crossing the affected link either fail or fall back to a much slower path.nvidia-smi nvlink -e for CRC/replay/recovery counters on BOTH endpoints. A counter ramp usually precedes the Xid by hours.
Xid 79GPU has fallen off the busHardwareRMA candidateThe driver lost PCIe communication with the GPU. The endpoint stopped responding due to a PCIe link degradation, power drop, thermal trip, or board fault. Recovery depends entirely on platform architecture and evidence collection.Capture host kernel logs, device identity, and an NVIDIA bug report before attempting any reboot. Coordinate recovery with your cloud provider or infrastructure team.
Xid 92High single-bit ECC error rateHardwareCorrectable errors are being corrected, but at an elevated rate. Nothing has failed yet; this is the early-warning signal that precedes uncorrectable errors.Schedule a drain rather than reacting immediately. Alert on the trend, a rising SBE rate is the cheapest predictor of an upcoming Xid 48/94/95.
Xid 94Contained ECC errorHardwareRMA candidateAn uncorrectable error that the GPU CONTAINED to the offending context. Other work on the GPU is unaffected; the faulting process is killed.Kill and restart the affected job. Check row-remap headroom, containment relies on spare rows being available.
Xid 95Uncontained ECC errorHardwareRMA candidateAn uncorrectable error the GPU could NOT contain. Corruption reached wider GPU state, so all work on the device is suspect, not just the faulting process.Drain the node. Resume from a checkpoint predating the Xid; do not trust the most recent optimizer state.
Xid 109Context switch timeout errorAmbiguous, needs evidenceThe GPU did not complete a context switch within its timeout. Can be an extremely long-running kernel, and can be a driver or hardware fault.Check for a kernel with an unbounded loop first. If the workload is normal, treat it as a driver-level fault and check the version.
Xid 119GSP RPC timeoutDriver or firmwareThe driver waited for a reply from the GPU System Processor firmware and did not get one. A firmware-side stall rather than an application fault.Reset the GPU. GSP firmware issues are strongly driver-version dependent, check NVIDIA release notes for your exact version before RMA.
Xid 120GSP errorDriver or firmwareThe GPU System Processor reported an internal error. GSP runs much of the driver logic on the GPU itself from R515 onward, so a GSP fault takes the device out even though no application did anything wrong.Reset with `nvidia-smi --gpu-reset -i <id>`, then match the driver against NVIDIA release notes. GSP can be disabled via the NVreg_EnableGpuFirmware kernel module parameter on known-affected versions.
Xid 140ECC unrecovered errorHardwareRMA candidateAn ECC error the GPU could not recover from. Treat as uncontained corruption.Drain, resume from a checkpoint predating the event, and collect `nvidia-smi -q -d ECC,ROW_REMAPPER` output for the RMA.
Xid 143GPU initialization failureHardwareRMA candidateThe GPU failed to initialize. It will not be usable until reset, and often not after.Full power cycle, not a warm reboot. Then `nvidia-smi -L` to confirm enumeration. If it recurs on the same slot, test whether the fault follows the card.

Codes not listed here exist. Rather than guess at their meaning, look them up in NVIDIA’s official Xid error reference, which is the authority for the code numbers and names above.

The three mistakes that cost the most

Diagnosing Xid 45

Xid 45 is preemptive cleanup, it fires because something else already failed, and it also fires on an ordinary Ctrl-C. The incident is whatever Xid came before it in dmesg.

RMA-ing on Xid 79 alone

GPU reachability loss alone does not identify the failed component. Preserve GPU identity, PCIe/AER messages and platform events before vendor-directed maintenance or a replacement decision.

Ignoring Xid 63 and 92

Retirement and corrected-error events deserve context, not a guaranteed failure forecast. Preserve counter history, event timing and the installed driver guidance before deciding whether maintenance is needed.

Frequently asked questions

Is NVIDIA Xid 43 the same as Windows Device Manager Code 43?
No. An NVRM Xid is a driver event in the host log. Windows Device Manager Code 43 is a different device-status report. Use the exact message and operating system to choose the right investigation.
Is an Xid error always a hardware failure?
No. An Xid identifies an event family, not a complete cause. Application, driver, device and peer evidence can lead to different next actions. Preserve the complete message and context before deciding whether the reporting GPU is defective.
Which Xid codes justify an RMA?
Memory, link and reachability faults can justify a support investigation, but a code alone does not establish replacement eligibility. Preserve device identity, full events, recurrence and the result of vendor-directed controls. The vendor decides whether replacement is appropriate.
Where do I find Xid messages?
In the host kernel log, usually prefixed NVRM. Run dmesg -T | grep -i xid or ask the administrator for the relevant journal window. Container workload logs may not include host kernel events, depending on permissions and collection setup.
Why did several GPUs report an Xid at the same instant?
Correlated events can concern shared power, fabric, driver behavior or a failing peer. Compare full events, endpoint identity and corrected timestamps before assigning the initiating component. Several reporters do not establish several defective GPUs.
Does a reboot clear an Xid?
A restart may restore service without removing the cause. Follow the NVIDIA and platform recovery workflow for the event and installed release. Xid 79 does not by itself prove a warm reboot cannot work. Capture evidence before any disruptive action and check workload recovery and recurrence afterward.

Reading an Xid is the easy part

Include the device messages and the failing workload log. Denpex investigates the supported causes and asks for missing evidence before attributing a bad node or recommending recovery. Checkpoint safety requires its own evidence.

Diagnose your logs free