← Back to reference

Operator Guide

Overview

The Denpex Operator is a Kubernetes operator built with kube-rs that manages:

  • DenpexNodeRemediation — CRD for per-node remediation.
  • DenpexCluster — top-level CR for cluster enrollment.
  • DenpexPolicy — tiered automation policy.
  • DenpexRBACBinding — Kubernetes-equivalent RBAC binding.

Install

helm install denpex-operator denpex/denpex-operator \
    --namespace denpex-system --create-namespace

Common Workflows

Trigger a remediation

apiVersion: denpex.ai/v1
kind: DenpexNodeRemediation
metadata:
  name: remediation-node-3
  namespace: denpex-system
spec:
  host: node-3
  rootCause: "DCGM_FI_DEV_XID_ERRORS=79"
  detectedAtUnixNano: 1700000000000000000
  policy:
    gracePeriodSeconds: 600
    runDcgmDiagnostics: true
    rerouteToWarmSpare: true
    openRmaTicket: true

The operator:

  1. Reads the CR.
  2. Calls the remediator binary.
  3. Updates the CR's status field as steps complete.
  4. Emits events (kubectl describe shows them).

Enroll a cluster

apiVersion: denpex.ai/v1
kind: DenpexCluster
metadata:
  name: prod
  namespace: denpex-system
spec:
  ptpServer: ptp.example.com
  fabricManager:
    url: https://fm.example.com
    tokenSecret: fabric-manager-token
  ufm:
    url: https://ufm.example.com
    caBundle: <base64>
  rmaWebhook:
    url: https://rma.example.com
    tokenSecret: rma-token

Apply an RBAC binding

apiVersion: denpex.ai/v1
kind: DenpexRBACBinding
metadata:
  name: tenant-acme-sre
  namespace: denpex-system
spec:
  tenant: acme
  role: sre
  principals:
    - subjectKind: User
      name: alice@example.com
    - subjectKind: Group
      name: sre-team

Reconciliation Loop

The operator polls every 30s by default. Webhook-driven reconciliation is supported via --webhook-port.

Observability

  • Operator exposes Prometheus metrics on :9091/metrics.
  • Audit log in /var/log/denpex/operator/audit.log.
  • Status conditions on every CR.

Troubleshooting

kubectl -n denpex-system logs -l app=denpex-operator
kubectl -n denpex-system get events --sort-by=.lastTimestamp

References

  • docs/upgrade-guide.md — operator upgrade.
  • docs/high-availability-guide.md — operator HA setup.