# Troubleshooting (/docs/enterprise/helm/guides/troubleshooting) Diagnostic commands and solutions for common issues with NetBox Enterprise on Kubernetes. ## Quick Diagnostics [#quick-diagnostics] ### Check Overall Status [#check-overall-status] ```bash # NetBoxEnterprise health kubectl -n netbox get netboxenterprises -o wide # All pods kubectl -n netbox get pods # Recent events kubectl -n netbox get events --sort-by='.lastTimestamp' | tail -20 ``` ### Check Operator Logs [#check-operator-logs] ```bash kubectl -n netbox logs -l app.kubernetes.io/name=nbe-operator --tail=100 ``` Increase verbosity for debugging: ```yaml operator: logging: level: "debug" format: "pretty" ``` ### Inspect NetBoxEnterprise Status [#inspect-netboxenterprise-status] ```bash kubectl -n netbox describe netboxenterprise netbox ``` Look at the `Status` section for: * **Conditions**: `Ready`, `Progressing`, `Degraded` * **Components**: Individual component health * **Message**: Human-readable status summary ## Common Issues [#common-issues] ### CRDs Not Installed [#crds-not-installed] **Symptom**: `error: the server doesn't have a resource type "netboxenterprises"` **Solution**: Install the CRDs before deploying: ```bash kubectl apply -f https://netboxlabs.com/docs/files/helm/netboxenterprises.netboxlabs.com-v2.2.1.yaml ``` Or install the CRD chart: ```bash helm install nbe-crds \ oci://registry.enterprise.netboxlabs.com/library/netbox-enterprise-crds \ --namespace netbox ``` ### Registry Authentication Failures [#registry-authentication-failures] **Symptom**: Pods stuck in `ImagePullBackOff` or `ErrImagePull` **Diagnosis**: ```bash kubectl -n netbox describe pod | grep -A5 "Events" ``` **Solution**: Verify your registry credentials: ```bash # Check the secret exists kubectl -n netbox get secret netbox-enterprise-helm-registry -o yaml # Test registry access read -rsp "License ID: " NBE_LICENSE_ID && echo echo "$NBE_LICENSE_ID" | docker login proxy.enterprise.netboxlabs.com \ --username your-email@example.com \ --password-stdin unset NBE_LICENSE_ID ``` ### PostgreSQL Not Ready [#postgresql-not-ready] **Symptom**: NetBox pods in `CrashLoopBackOff`, operator logs show database connection errors **Diagnosis**: ```bash # Check PGO pods kubectl -n netbox get pods -l postgres-operator.crunchydata.com/cluster # Check PGO logs kubectl -n netbox logs -l postgres-operator.crunchydata.com/role=master --tail=50 # Check if the PostgresCluster is ready kubectl -n netbox get postgrescluster ``` **Solution**: * Ensure PGO is enabled (`pgo.enabled: true`) and the operator is installed * Check storage: PGO needs a writable PVC - verify your `StorageClass` supports dynamic provisioning * For external PostgreSQL: verify hostname, port, and credentials ### Redis Not Ready [#redis-not-ready] **Symptom**: NetBox pods starting but not fully functional, task queue errors **Diagnosis**: ```bash kubectl -n netbox get pods -l app=redis kubectl -n netbox logs -l app=redis --tail=50 ``` ### Resource Constraints [#resource-constraints] **Symptom**: Pods stuck in `Pending` state **Diagnosis**: ```bash kubectl -n netbox describe pod | grep -A5 "Events" kubectl describe nodes | grep -A5 "Allocated resources" ``` **Solution**: Either increase cluster resources or reduce the resource requests in your values file. ### Operator Reconciliation Errors [#operator-reconciliation-errors] **Symptom**: `Degraded` condition on the NetBoxEnterprise **Diagnosis**: ```bash kubectl -n netbox describe netboxenterprise netbox kubectl -n netbox logs -l app.kubernetes.io/name=nbe-operator --tail=100 ``` The operator publishes Kubernetes events for each reconciliation step. Check events for details: ```bash kubectl -n netbox get events --field-selector reason=ReconcileError ``` ### Pods in CrashLoopBackOff [#pods-in-crashloopbackoff] **Diagnosis**: ```bash # Get pod logs (current crash) kubectl -n netbox logs # Get previous crash logs kubectl -n netbox logs --previous # Check pod details kubectl -n netbox describe pod ``` The operator automatically extracts the last 20 log lines from failing pods and includes them in the component status. ## Status Condition Reference [#status-condition-reference] | Condition | Status | Reason | Meaning | | --------------- | --------- | ---------------------- | ------------------------------------------------------ | | `Ready` | `True` | `AllComponentsReady` | All components healthy | | `Ready` | `False` | `ComponentsNotReady` | Some components still starting | | `Ready` | `False` | `ReconcileFailed` | Reconciliation error | | `Progressing` | `True` | `ApplyingResources` | Resources being applied | | `Progressing` | `True` | `WaitingForComponents` | Waiting for pod readiness | | `Progressing` | `False` | `ReconcileComplete` | Reconciliation finished | | `Degraded` | `True` | `ReconcileError` | Reconciliation failed | | `RoutingHealth` | `True` | `GatewayActive` | Served by the Gateway API | | `RoutingHealth` | `Unknown` | `GatewayProgramming` | Gateway created; controller not yet `Programmed` | | `RoutingHealth` | `False` | `GatewayNotProgrammed` | Gateway controller reports `Programmed=False` | | `RoutingHealth` | `True` | `IngressActive` | Served by operator-emitted Ingress | | `RoutingHealth` | `True` | `IngressFallback` | Gateway requested but unavailable; serving via Ingress | | `RoutingHealth` | `False` | `RoutingModeMismatch` | Resolved routing path is disabled; nothing emitted | See [Status & Conditions](/enterprise/helm/api/status-conditions/) for the full reference. ## Suspend Reconciliation [#suspend-reconciliation] To pause the operator while debugging (existing workloads keep running): ```bash kubectl -n netbox patch netboxenterprise netbox \ --type merge \ -p '{"spec":{"suspend":true}}' ``` Resume with: ```bash kubectl -n netbox patch netboxenterprise netbox \ --type merge \ -p '{"spec":{"suspend":false}}' ``` ## Useful kubectl Commands [#useful-kubectl-commands] ```bash # Get all resources managed by the operator kubectl -n netbox get all -l app.kubernetes.io/managed-by=netbox-operator # Watch pod status changes kubectl -n netbox get pods -w # Get Helm release status helm status netbox-enterprise -n netbox # Get Helm release history helm history netbox-enterprise -n netbox # Check CRD is installed kubectl get crd netboxenterprises.netboxlabs.com ``` ## Next Steps [#next-steps] * [Status & Conditions](/enterprise/helm/api/status-conditions) - Detailed status reference * [Monitoring](/enterprise/helm/configuration/monitoring) - Set up metrics for proactive monitoring