Agent Overhead Benchmarks
Testbed
- Single node: 8x H100 (PCIe Gen5), 96 vCPUs, 1 TB RAM, NVMe storage.
- 64-rank PyTorch DDP job, all-reduce batch size 16, sequence length 1024, GPT-2 size.
- Reproduction:
benchmarks/agent-overhead/run.sh.
Measurements
Each measurement is the median of 5 runs. Reported on a per-rank basis.
| Workload | Throughput (delta) | CPU agent (delta) | Memory agent (delta) |
|---|---|---|---|
| GPT-2 1.5B | -0.8 % | +0.6 % | +18 MB |
| LLaMA-3 8B | -1.0 % | +0.7 % | +24 MB |
| Mixtral 8x7B | -1.2 % | +0.8 % | +36 MB |
The numbers come from running the benchmark suite for 5 trials each. The agent overhead remains bounded because:
- NVML/DCGM polling runs at 1 Hz and is throttled to one OS syscall per device per cycle.
- eBPF programs are pinned and only emit on overflow.
- Per-layer metrics are sampled 1-in-10 by default.
See benchmarks/agent-overhead/results.csv for raw data.
CI Enforcement
benchmarks/agent-overhead/budget.yaml:
max_throughput_delta_pct: 1.5 # CI fails if drift exceeds
max_cpu_overhead_pct: 1.0
max_memory_overhead_mb: 64CI runs the benchmark in a 4-GPU reduced testbed and fails if any budget is breached.