Troubleshooting

Diagnostic commands and solutions for common issues with NetBox Enterprise on Kubernetes.

Quick Diagnostics

Check Overall Status

# NetBoxEnterprise health
kubectl -n netbox get netboxenterprises -o wide

# All pods
kubectl -n netbox get pods

# Recent events
kubectl -n netbox get events --sort-by='.lastTimestamp' | tail -20

Check Operator Logs

kubectl -n netbox logs -l app.kubernetes.io/name=nbe-operator --tail=100

Increase verbosity for debugging:

operator:
  logging:
    level: "debug"
    format: "pretty"

Inspect NetBoxEnterprise Status

kubectl -n netbox describe netboxenterprise netbox

Look at the Status section for:

• Conditions: Ready, Progressing, Degraded
• Components: Individual component health
• Message: Human-readable status summary

Common Issues

CRDs Not Installed

Symptom: error: the server doesn't have a resource type "netboxenterprises"

Solution: Install the CRDs before deploying:

kubectl apply -f https://netboxlabs.com/docs/files/helm/netboxenterprises.netboxlabs.com-v2.0.0.yaml

Or install the CRD chart:

helm install nbe-crds \
  oci://registry.enterprise.netboxlabs.com/library/nbe-operator-crds \
  --namespace netbox

Registry Authentication Failures

Symptom: Pods stuck in ImagePullBackOff or ErrImagePull

Diagnosis:

kubectl -n netbox describe pod <pod-name> | grep -A5 "Events"

Solution: Verify your registry credentials:

# Check the secret exists
kubectl -n netbox get secret netbox-enterprise-helm-registry -o yaml

# Test registry access
docker login proxy.enterprise.netboxlabs.com \
  --username your-email@example.com \
  --password <your-license-id>

PostgreSQL Not Ready

Symptom: NetBox pods in CrashLoopBackOff, operator logs show database connection errors

Diagnosis:

# Check PGO pods
kubectl -n netbox get pods -l postgres-operator.crunchydata.com/cluster

# Check PGO logs
kubectl -n netbox logs -l postgres-operator.crunchydata.com/role=master --tail=50

# Check if the PostgresCluster is ready
kubectl -n netbox get postgrescluster

Solution:

• Ensure PGO is enabled (pgo.enabled: true) and the operator is installed
• Check storage: PGO needs a writable PVC — verify your StorageClass supports dynamic provisioning
• For external PostgreSQL: verify hostname, port, and credentials

Redis Not Ready

Symptom: NetBox pods starting but not fully functional, task queue errors

Diagnosis:

kubectl -n netbox get pods -l app=redis
kubectl -n netbox logs -l app=redis --tail=50

Resource Constraints

Symptom: Pods stuck in Pending state

Diagnosis:

kubectl -n netbox describe pod <pod-name> | grep -A5 "Events"
kubectl describe nodes | grep -A5 "Allocated resources"

Solution: Either increase cluster resources or reduce the resource requests in your values file.

Operator Reconciliation Errors

Symptom: Degraded condition on the NetBoxEnterprise

Diagnosis:

kubectl -n netbox describe netboxenterprise netbox
kubectl -n netbox logs -l app.kubernetes.io/name=nbe-operator --tail=100

The operator publishes Kubernetes events for each reconciliation step. Check events for details:

kubectl -n netbox get events --field-selector reason=ReconcileError

Pods in CrashLoopBackOff

Diagnosis:

# Get pod logs (current crash)
kubectl -n netbox logs <pod-name>

# Get previous crash logs
kubectl -n netbox logs <pod-name> --previous

# Check pod details
kubectl -n netbox describe pod <pod-name>

The operator automatically extracts the last 20 log lines from failing pods and includes them in the component status.

Status Condition Reference

Condition	Status	Reason	Meaning
Ready	True	AllComponentsReady	All components healthy
Ready	False	ComponentsNotReady	Some components still starting
Ready	False	ReconcileFailed	Reconciliation error
Progressing	True	ApplyingResources	Resources being applied
Progressing	True	WaitingForComponents	Waiting for pod readiness
Progressing	False	ReconcileComplete	Reconciliation finished
Degraded	True	ReconcileError	Reconciliation failed

Suspend Reconciliation

To pause the operator while debugging (existing workloads keep running):

kubectl -n netbox patch netboxenterprise netbox \
  --type merge \
  -p '{"spec":{"suspend":true}}'

Resume with:

kubectl -n netbox patch netboxenterprise netbox \
  --type merge \
  -p '{"spec":{"suspend":false}}'

Useful kubectl Commands

# Get all resources managed by the operator
kubectl -n netbox get all -l app.kubernetes.io/managed-by=netbox-operator

# Watch pod status changes
kubectl -n netbox get pods -w

# Get Helm release status
helm status netbox-enterprise -n netbox

# Get Helm release history
helm history netbox-enterprise -n netbox

# Check CRD is installed
kubectl get crd netboxenterprises.netboxlabs.com

Next Steps

• Status & Conditions — Detailed status reference
• Monitoring — Set up metrics for proactive monitoring