Calling every timeout a GPU crash
A client response can include queueing, initialization and transport. Identify the failed phase before recommending a runtime fix.
Operator task guide
Is the inference worker starting, waiting or actually crashing?
An API timeout does not establish a GPU crash. First record whether the replica is starting, loading a model, ready but queueing, or exiting. Those states require different evidence and different owners.
Align request timing with the worker lifecycle and the first runtime exception. Keep a failed initialization distinct from a worker that becomes ready and later fails. Denpex can review submitted runtime evidence; it cannot reconstruct uncollected provider state.
Step-by-step method
- 1
Record the deployment and replica state
Retain deployment revision, replica identity and timestamps for creation, startup, loading and readiness. Check whether the event followed scale-to-zero, a new image or a rollout. Avoid comparing cold and already-warm replicas as equivalent conditions.
- 2
Separate queue time from worker execution
Collect request arrival, queue and worker timing when available. A request can expire while capacity starts. If no worker accepted the request, a client timeout is not evidence of a failed CUDA operation.
- 3
Read the first initialization or execution error
For a process that exits, preserve the first exception and the preceding loading output. Record storage access errors, runtime versions and available device memory. Cache-specific remedies apply only when the engine and its actual error support them.
- 4
Verify the implicated stage with a bounded replay
Compare one resolved event with the proposed cause and next check. A workload owner should confirm whether the failure was in startup, storage, queueing or execution before changing scaling or memory settings. Use an approved canary to verify readiness and request completion after a change.
Classify the state before choosing a remedy
| Signal | What it means | Next action |
|---|---|---|
| Replica starting, no runtime exception | Capacity is not ready; a crash is unproven. | Review startup stages, queue time and scaling history. |
| Model load fails before readiness | Initialization failed before serving could begin. | Preserve the first loading error and storage/device context. |
| Ready worker exits during execution | There is a runtime failure to investigate. | Collect its first exception and host events before retrying. |
Evidence checklist
- Deployment and replica identity with lifecycle timestamps
- Request, queue and execution timing without payloads
- First loading or runtime error and image revision
- Device context and the confirmed incident outcome
Common mistakes
Applying one provider's scaling defaults everywhere
Scaling windows, readiness behavior and warm-capacity controls vary. Use the platform's documentation and workload measurements.
Frequently asked questions
Does scaling to zero mean the next request will crash?
No. Starting capacity and a failed process are different states. Inspect lifecycle, readiness and request timing.
Can Denpex collect private provider state automatically from my upload?
No. Include approved lifecycle and runtime evidence that you have retained. Uncollected state remains a limitation.
Apply the method to your incident
Use the three free diagnoses to review your error and relevant evidence. Keep reference guidance separate from the cause and recovery status of your own workload.
Diagnose your incident