← Back to reference

Python Sdk Reference

Install

pip install denpex

Modules

  • denpex — top-level wrapper for training scripts.
  • denpex.sdc — SDC canary + runs.
  • denpex.instrument — per-layer metric hooks.
  • denpex.ddp — DDP context.
  • denpex.fsdp — FSDP integration.
  • denpex.deepspeed — DeepSpeed integration.
  • denpex.megatron — Megatron-LM integration.
  • denpex.jax — JAX integration.
  • denpex.tensorflow — TensorFlow integration.
  • denpex.checkpoint — checkpoint integrity.
  • denpex.nccl — NCCL timing.

CLI

denpex annotate
denpex incident list
denpex incident show <id>
denpex incident approve <id>
denpex checkpoint validate <path>
denpex rank map

Decorator

import denpex

@denpex.wrap(name="my-gpt-run", tenant="acme")
def main(cfg):
    train(cfg)

Trace Backing

The wrapper is a context manager that:

  1. Spawns a daemon that re-execs python with PTY.
  2. Captures stdout/stderr (last 10 MB truncated).
  3. Records per-step metrics.
  4. Emits a TrainingEvent at every Python error.
  5. Emits a final TrainingEvent::Done on clean exit.

Configuration

Env varDefaultPurpose
DENPEX_TOKEN(none)Auth token
DENPEX_TENANT(none)Tenant ID
DENPEX_SAMPLE_EVERY_N_STEPS10Layer sampling
DENPEX_SDC_EVERY_N_STEPS20SDC canary cadence
DENPEX_PROFILE_TRACE0Set 1 to capture torch profiler

API Stability

  • All public modules follow SemVer.
  • denpex._internal is unsupported.

Tests

Each module ships unit tests:

denpex_sdk/tests/test_*.py

Integration tests run against a mock OTLP receiver.