Agent configuration
All configuration is done through environment variables. No config file, no YAML, no TOML. Set them in your training script, shell profile, or orchestration tool.
Environment variables
DENPEX_API_KEY*
Your API key from the Denpex dashboard. Required for cloud mode.
DENPEX_PRIVACY
Privacy mode: mask (default, redact PII/PHI before egress), strict (anonymized failure signatures only), off (send raw logs unchanged).
DENPEX_DRY_RUN
Set to 1 to print the exact masked payload that would be sent and send nothing. Use this to audit egress before trusting the agent.
DENPEX_HEARTBEAT_SEC
Liveness heartbeat interval in seconds. Default 120, clamped 5-3600.
DENPEX_TAIL_LINES
Number of log lines to ship on crash. Default 500, max 5000.
DENPEX_LOCAL
Set to 1 for air-gapped mode. Diagnosis runs entirely on-host; nothing leaves the cluster. No API key required.
DENPEX_METRICS_PORT
Expose Prometheus /metrics on this port. Auto-enabled at 9836 in air-gapped mode.
DENPEX_INCIDENT_DIR
Directory for local incident JSON logs in air-gapped mode. Default ./denpex-incidents.
DENPEX_API_URL
Override the API base URL. Default https://api.denpex.com.
DENPEX_JOB_NAME
Explicit job name override. Auto-detected from scheduler env vars if not set.
DENPEX_TAGS
Comma-separated key=value tags attached to every diagnosis from this agent.
Privacy mode examples
# Default: mask PII/PHI before any byte leaves
export DENPEX_PRIVACY=mask
# Strict: only anonymized failure signatures, no raw logs
export DENPEX_PRIVACY=strict
# Off: send raw logs unchanged (opt-in)
export DENPEX_PRIVACY=off
Air-gapped mode
For clusters that cannot let any telemetry leave the network, set DENPEX_LOCAL=1. The agent runs the full deterministic engine on-host. You will need the offline engine file and pattern database alongside the agent:
curl -O https://denpex.com/agent/denpex.py
curl -O https://denpex.com/agent/denpex_local.py
curl -O https://denpex.com/agent/denpex_patterns.json
DENPEX_LOCAL=1 python denpex.py run --job llama3 -- torchrun --nproc_per_node=8 train.py
Using with SLURM
Set environment variables in your sbatch script or job submission command:
#SBATCH --export=DENPEX_API_KEY,DENPEX_PRIVACY
export DENPEX_API_KEY=dpx_...
export DENPEX_PRIVACY=mask
srun python denpex.py run --job $SLURM_JOB_NAME -- torchrun ...