Troubleshooting

Symptom

Running denpex.py produces a 'command not found' error even after installation.

Root cause

The file is not executable or the directory containing it is not on your system PATH.

Resolution steps

1. 1Run chmod +x denpex.py to make the file executable.
2. 2Alternatively, invoke it directly with Python: python denpex.py run -- your_training_command.
3. 3If you installed via pip, verify the installation path is on PATH: python -m denpex.py run -- your_training_command.
4. 4On SLURM clusters, add the denpex installation directory to PATH in your sbatch script before calling denpex.py.

Symptom

The agent starts but immediately fails with an authentication error.

Root cause

The DENPEX_API_KEY environment variable is not set, is set incorrectly, or the key has been deleted or rotated.

Resolution steps

1. 1Go to Dashboard → Monitoring → API Keys and verify your key exists and is active.
2. 2Copy the key exactly (no trailing spaces or newlines).
3. 3Set it in your environment: export DENPEX_API_KEY=dpx_your_key_here.
4. 4Verify it is set: echo $DENPEX_API_KEY.
5. 5If running on SLURM, add it to your sbatch script with #SBATCH --export=DENPEX_API_KEY=dpx_... or use srun --export.
6. 6If using Docker or Kubernetes, pass it as an environment variable in your container spec.

Symptom

The agent is running and sending heartbeats (visible in Dashboard → Monitoring), but after a training crash no diagnosis appears.

Root cause

The agent process is not the parent of the training command, so it never receives the exit signal when training crashes.

Resolution steps

1. 1Ensure the agent wraps your training command using python denpex.py run -- your_training_command.
2. 2Do not launch the training command as a background process from within your script. The agent must be the direct parent process.
3. 3If your scheduler (SLURM, Kubernetes) launches the training command independently, use the attach mode: python denpex.py attach --pid $(pgrep -f your_training_script).
4. 4Check that the agent has network access to api.denpex.com on port 443.
5. 5Review agent logs at /tmp/denpex-agent.log for any error output.

Symptom

The deterministic diagnosis returns no match and falls back to the LLM path.

Root cause

The failure did not meet the confidence threshold against any of the 3500+ documented failure classes. This can indicate a novel failure, a cluster-specific variant, or log output that was truncated before the meaningful signal.

Resolution steps

1. 1Wait for the LLM fallback result. It typically arrives within 60 seconds on Team plans.
2. 2If LLM fallback is disabled (UNKNOWN response with no fix), enable it in Dashboard → Settings → Diagnosis → LLM Fallback.
3. 3Review whether log collection is complete. If the training process was killed before writing its full stderr, the failure signal may be incomplete.
4. 4If this failure class recurs, file it from the diagnosis view in the dashboard. The team will add a deterministic pattern for it.
5. 5On the Team plan, add a custom knowledge base entry for cluster-specific variants of known failures.

Symptom

Slack alerts are not arriving, or the Denpex webhook test returns an 'invalid_auth' error.

Root cause

The Slack webhook URL is expired, the workspace revoked the webhook, or the webhook was created for a channel that no longer exists.

Resolution steps

1. 1In your Slack workspace, go to Apps → Incoming Webhooks and check whether the Denpex webhook is listed as active.
2. 2If it shows as revoked or missing, create a new webhook for your target channel.
3. 3Copy the new webhook URL (starts with https://hooks.slack.com/services/) and update it in Dashboard → Alerts → Slack.
4. 4Click Send test alert on the Slack channel row in Dashboard → Alerts. It posts a synthetic diagnosis to that channel so you can confirm delivery end to end.
5. 5Ensure the Denpex app has not been removed from your Slack workspace. If it was removed, the webhook is permanently invalid.

Symptom

Clicking 'Sign in with Google / GitHub / Discord' redirects back to the login page with a CSRF or state mismatch error.

Root cause

OAuth state mismatch, typically caused by a stale session cookie, a browser extension that modifies redirect URLs, or navigating to the OAuth callback from a different browser tab.

Resolution steps

1. 1Clear cookies for denpex.com: go to your browser settings, find cookies for denpex.com, and delete them all.
2. 2Try the OAuth flow in an incognito or private browsing window to rule out extension interference.
3. 3Disable any browser extensions that modify headers or redirect URLs (ad blockers, privacy extensions) and retry.
4. 4Ensure you are completing the OAuth flow in the same browser tab that initiated it. Do not open the callback URL in a new tab.
5. 5If the error persists, try a different browser.

Symptom

Diagnoses are taking significantly longer than the typical 12-second target.

Root cause

The diagnosis engine runs in tiers: deterministic pattern matching (fastest) → team knowledge base → community research (GitHub/Stack Overflow) → LLM fallback (slowest). Latency increases for failures that require later tiers. Network latency to api.denpex.com can also contribute.

Resolution steps

1. 1Check the diagnosis tier in the response. If it returned via LLM fallback, 30–60 seconds is expected.
2. 2If community research (GitHub/SO) is being triggered frequently, add a team knowledge base entry for your cluster's common failure variants to short-circuit to the deterministic tier.
3. 3For latency-critical deployments, enable the in-VPC agent (Scale+ plan) which runs the deterministic pattern engine locally with no network round-trip for the 3500+ covered failure classes.
4. 4Check your network path to api.denpex.com. High latency from your cluster to the API adds directly to diagnosis time.

Symptom

Clicking 'Upgrade' or 'Subscribe' produces a billing error or the Stripe checkout page does not load.

Root cause

A transient issue with our Stripe configuration. This is a server-side error, not a problem with your payment method.

Resolution steps

1. 1Refresh the page and try again. Transient Stripe errors often resolve on retry.
2. 2If the error persists for more than a few minutes, email support@denpex.com with the error message and the time you saw it.
3. 3Do not enter payment information into a page showing an error. Wait for a clean checkout page.

Symptom

Logs appear to be leaving your infrastructure even though you expected data to stay local.

Root cause

On Free and Team plans, logs are processed in the Denpex cloud API. Raw log lines are deleted immediately after diagnosis; only anonymized diagnostic metadata is retained for cross-run comparison. To prevent any egress, you need the in-VPC agent.

Resolution steps

1. 1To prevent all log egress: upgrade to Scale+ and enable the in-VPC agent (docs at /docs/agent/self-hosted).
2. 2The in-VPC agent runs the full pattern engine locally. Only diagnosis metadata (failure class, severity, rank ID) is sent to the API, no raw log content.
3. 3Set DENPEX_LOCAL=1 in your environment to enable zero-egress mode on the Scale+ plan.
4. 4To verify no raw logs are leaving: run Wireshark or tcpdump on the agent host and confirm outbound payloads contain only structured JSON metadata, not raw log lines.