SGLang errors

SGLang optimizes inference through RadixAttention and aggressive prefix caching. When failures occur, they are typically tied to the cache memory pool allocation or specialized FlashInfer kernel constraints.

Because SGLang handles highly concurrent requests, a single illegal memory access in a custom attention kernel can bring down the entire server. Trace the failure back to the specific prompt length and system memory profile.

Diagnose your log with evidence Full failure encyclopedia

Working in VS Code or Remote-SSH? Diagnose your log locally with the free VS Code extension. No account or upload required for local diagnosis.

Every common sglang error

The literal string is what you paste into a search bar, so it is the heading. “Class” is a triage category. It is not proof of ownership in your workload.

Radix cache OOM

Configuration

RadixAttention: no available memory for cache blocks

The RadixAttention tree consumed all reserved KV cache blocks. Unlike standard eviction, if the active concurrent requests exceed the physical block capacity, the server cannot process the batch.

First action: Reduce the --max-concurrency limit or adjust the GPU memory utilization fraction used by the SGLang server to allocate more blocks.

FlashInfer kernel fault

Application bug

FlashInfer: illegal memory access in attention kernel

The FlashInfer attention backend attempted to read outside the allocated tensor bounds. This is often triggered by an edge-case sequence length or incompatible GPU architecture.

First action: Run the server with CUDA_LAUNCH_BLOCKING=1 to pinpoint the exact sequence length causing the crash, then disable FlashInfer to test the fallback kernel.

Worker timeout

Environment / runtime

TimeoutError: API server failed to receive response from worker

The frontend API router timed out waiting for the backend GPU worker to return tokens. The worker process may have died or is stuck in an infinite generation loop.

First action: Check dmesg for host OOM kills affecting the worker process, and inspect the worker logs for silent CUDA errors or infinite generation parameters.

Errors not listed here exist. Rather than guess at their meaning, check SGLang Documentation, which is the authority for the strings above.

Frequently asked questions

Why is RadixAttention running out of memory when the GPU has free space?

SGLang pre-allocates a fixed memory pool for the RadixAttention cache at startup. Any memory left free on the GPU is outside this pool. You must increase the memory fraction configuration if you want the cache to use more VRAM.

How do I know if FlashInfer is the cause of my SGLang crash?

If the traceback points to a specific CUDA kernel execution inside the attention block, try switching the attention backend. If the fallback backend works without crashing, the issue is isolated to the FlashInfer kernel.

Does SGLang support running across multiple nodes?

Yes, SGLang supports multi-node deployment via distributed execution engines. However, network latency and NCCL configuration become critical, and you must ensure the network interface is correctly specified in the environment.

What is the fastest way to clear a stuck SGLang server state?

A full process restart is required. Because a crashed CUDA kernel poisons the GPU context, you cannot simply flush the Radix cache or drop the HTTP requests. Restart the backend worker to reinitialize the hardware context.

Knowing the error is not knowing the cause

Which rank failed first, whether the checkpoint is safe to resume from, and whether this is your code or the hardware. Paste the log and Denpex answers all three, free and without an account.

Diagnose your logs free