Skip to content
ChangelogRSS

What shipped.

New diagnoses, performance, fixes, and security updates. Subscribe via RSS or follow us on Twitter.
Looking for a public roadmap? denpex.com/roadmap.

July 2026

Shipped2026-07-01

16,384-GPU fleets: mass-crash coalescing

Denpex now supports fleets up to 16,384 GPUs end-to-end. The new mass-crash coalescing engine handles the worst day at that scale: when one fabric event (a bad IB switch, a PDU trip) crashes ~2,000 nodes at once, all their crash reports are grouped by cluster and failure signature, ONE diagnosis runs, and every affected job links to it — one alert with the fix instead of thousands of pages, zero dropped diagnoses, and the whole event bills as a single diagnosis. Crash ingest also moved to a dedicated budget that counts distinct root causes, never raw crash reports. Fleet heartbeats already shard per-tenant with headroom at 2,000 nodes, and cascade analysis is tested to 65,536 ranks. Data Center plan copy updated to match.

June 2026

Security2026-06-25

Diagnosis history auto-purged at 90 days

The daily retention cron now deletes diagnoses, follow-up chat, user feedback, and async-diagnosis rows older than 90 days; login_attempts and stripe_events older than 30 days. Combined with the existing crash_logs (30d), alert_log (90d), rate_limit_hits (2d), and consent_receipts (3y) purges, this makes §5 of the privacy policy 100% code-true. Previously, the policy said 90 days for signatures but only crash_logs and alert_log were actually auto-deleted — diagnosis history was kept forever. No customer action required.

Security2026-06-25

Privacy policy: retention values now match the code

Rewrote §4 and §5 to reflect the actual retention cron (raw crash logs: 30 days, diagnosis signatures: 90 days, alert log: 90 days, rate-limit hits: 2 days, consent receipts: 3 years, audit log: 2 years, billing: 7 years). Account deletion is immediate hard-delete, not 30-day grace. Tightened §2 to say pattern-matching improvements use aggregate signatures, never raw logs. No code changes — these are policy corrections to match the implementation that already runs in production.

Shipped2026-06-20

Diagnosis engine: ranked matches, evidence trail, and an accuracy benchmark

The encyclopedia matcher (312 classes) moved from naive token-overlap to IDF-weighted retrieval with error-signature anchoring (Xid codes, CUDA error numbers, exception classes). It now returns ranked candidates with the evidence that fired the match and the full multi-step fix. Accuracy is benchmarked against 171 real GitHub failure logs with a CI regression gate; top-1 on real error traces rose from 30.6% to 42.4%. Added a dedicated CUDA device-side-assert class and a straggler / slow-rank detection class.

Shipped2026-06-15

Trust Center launched

SOC 2 status, sub-processor list, data flow, shared-responsibility matrix, and a one-click request form for DPA, BAA, SOC 2, pen-test, CAIQ, and MSA documents. /security/trust-center.

Shipped2026-06-15

Stripe billing fulfillment is live

Paid checkouts now upgrade the user's plan in the database via webhook. Idempotent on event id. /api/billing/webhook.

Shipped2026-06-15

Cookie consent banner (GDPR)

Accept / reject / customize; analytics and marketing toggles; persisted to localStorage and broadcast as a `denpex:consent` event.

Security2026-06-15

HSTS, CSP, X-Frame-Options, Referrer-Policy

Site-wide security headers via Cloudflare _headers. CSP starts in report-friendly mode and tightens over time.

Shipped2026-06-14

Architecture brief request flow

A 6-page PDF + a follow-up from sales, with an NDA option. /architecture-brief.

Fixed2026-06-12

Dashboard: brief 500 spike during Stripe webhook burst

Resolved in 14 minutes. Idempotency tightened on the subscription.updated path.

Shipped2026-06-10

GitHub OAuth login

Continue with GitHub on the login page. Most ML platform teams run on GitHub Enterprise, so this was overdue.

Shipped2026-06-09

Failure Encyclopedia: first entries live

The first failure-class entries have root cause, symptoms, fix, and prevention. New entries ship weekly (312 classes as of this entry; the encyclopedia has since grown past 360).

Shipped2026-06-05

Annual pricing toggle

Save 2 months on annual plans. Available on /pricing — Team, Scale, and Data Center.

Security2026-06-01

Sub-processor list published

Live list on the Trust Center. 30 days' notice of changes via email and the changelog RSS.

May 2026

Fixed2026-05-28

Console: diagnosis latency above SLO for ~22 minutes

Resolved in 22 minutes. Upstream rate limit on the GitHub tier triggered premature escalation; tier escalation is now rate-limit-aware.

Shipped2026-05-20

Customer responsibility matrix

Plain-English breakdown of who does what, called out from the Trust Center.

Shipped2026-05-15

Data flow diagram

End-to-end: agent → local masking → TLS 1.3 → engine → artifact store → alert fan-out. On the Trust Center.

Fixed2026-05-09

Auth: OAuth callback returned 502 for Google sign-ins

Resolved in 9 minutes. Cloudflare worker pool warm-up. No customer action required.

Shipped2026-05-01

In-VPC agent (Scale and Data Center)

Logs never leave your cluster. Configurable PII/PHI masking on the agent. Available to all Scale and Data Center customers.