FAQ & Troubleshooting

Common Issues

Configuration errors are logged with the prefix Kubernetes backend configuration error: and stop the run before any data is sent to NetBox.

NODE_OBJECT_TYPE is missing or invalid:
```
NODE_OBJECT_TYPE is required: set it to "device" or "virtual_machine".
```
```
NODE_OBJECT_TYPE must be "device" or "virtual_machine", got "vm".
```
Solution: Set NODE_OBJECT_TYPE to device or virtual_machine. It has no default and is required in bootstrap mode too. Choose carefully, because changing it later leaves the records written under the previous type in NetBox. See Choosing a node object type and site.

CLUSTER_NAME is missing or too long:

CLUSTER_NAME must be provided in policy config: Kubernetes cannot report its own name, so there is nothing to derive it from.

CLUSTER_NAME is 95 characters; at most 89 are allowed, because the integration derives the tag 'kubernetes-<CLUSTER_NAME>' from it and NetBox limits a tag name to 100 characters.

Solution: Set CLUSTER_NAME to the name you want for the cluster in NetBox, at most 89 characters. Use a different name for each cluster.

SITE_NAME is missing in device mode:
```
SITE_NAME is required when NODE_OBJECT_TYPE is "device": NetBox identifies a device by name plus site, so this value decides whether the integration adopts an existing server record of the same name or creates a new one beside it.
```
Solution: Set SITE_NAME to the site the node devices should be placed in. If the site already contains devices with the same names as your nodes, the integration updates those devices. See SITE_NAME.
API server URL or token missing:
```
K8S_API_URL and K8S_TOKEN must be provided in policy config
```
Solution: Set K8S_API_URL and K8S_TOKEN in your configuration or .env file, and make sure the variables are passed to the agent container.
TLS verification fails when connecting to the API server:

Solution: The API server's certificate is normally signed by the cluster's own CA, which the agent host does not trust. Set K8S_CA_CERT to the cluster's CA certificate, as described in Get the API server URL and CA certificate. With no CA configured, the integration logs:
```
No CA certificate configured, so the API server's certificate must be signed by a CA already trusted by this host. Set K8S_CA_CERT to the cluster's CA if the connection fails to verify.
```
K8S_CA_CERT cannot be read:
```
K8S_CA_CERT is neither a PEM certificate nor a readable file: '/opt/orb/cluster-ca.pem'
```
Solution: The value must be either the PEM-encoded certificate itself or a path to a file that exists inside the agent container. If you use a path, check that the file is in the directory mounted at /opt/orb/. If you set SKIP_SSL: true instead, remove K8S_CA_CERT from the configuration.
The run fails with a 401 Unauthorized error:

Solution: The token is invalid or has expired. Tokens created with kubectl create token expire after the requested duration. Create a new token and update K8S_TOKEN, or use a long-lived token for a scheduled agent. See Create a service account and token.
The run fails with a 403 Forbidden error:

Solution: The service account cannot list nodes or read the kube-system namespace, and both are required. Apply the cluster role and binding from Create a service account and token, and check that the binding names the service account the token belongs to.

No LoadBalancer service addresses are recorded:

Cannot list services (403 Forbidden); no external service addresses will be recorded. Grant the service account 'get' and 'list' on services.

Solution: Grant get and list on services in the cluster role. The rest of the sync is not affected.

No service prefixes are recorded:
```
Cannot read ServiceCIDR objects (403 Forbidden); no service block will be recorded. Grant the service account 'get' and 'list' on servicecidrs in networking.k8s.io to record it.
```
```
This cluster does not serve the ServiceCIDR resource (networking.k8s.io/v1); no service block will be recorded
```
Solution: For the first message, grant get and list on servicecidrs in the networking.k8s.io API group. The second message means the cluster does not provide the ServiceCIDR resource, which is available from Kubernetes 1.33, so the integration has no source for the service block. In both cases, the rest of the sync is not affected.
The node label selector fails the run:
```
scope.node_labels: "hardware=baremtal" matched 0 of 12 nodes.
Labels present in this cluster: beta.kubernetes.io/arch, hardware, kubernetes.io/arch, kubernetes.io/hostname, kubernetes.io/os
```
```
scope.node_labels must be a Kubernetes label selector string, got list: ['hardware=baremetal']
```
Solution: A selector that matches no nodes is treated as an error, because a typo usually causes it. Check the selector against the listed label keys, and test it with kubectl get nodes -l "<selector>". Write node_labels as a single string, not a YAML list. A selector with invalid syntax fails with a message beginning scope.node_labels: "<selector>" rejected by the Kubernetes API:, followed by the API server's explanation.
Some LoadBalancer services are missing:
```
3 of 5 LoadBalancer services report a hostname endpoint rather than an address; these are not recorded in NetBox. Expected on EKS and on any platform whose load balancer is fronted by a DNS name.
```
Solution: This is expected. Only LoadBalancer services with an IP address in status.loadBalancer.ingress are recorded. Services whose load balancer reports a DNS name, as on Amazon EKS, are not recorded in this release.
A warning says a node has no matching ID:
```
Node 'worker-2' reports no status.nodeInfo.systemUUID, so it carries no matching ID: renaming it in Kubernetes will create a second NetBox record rather than moving this one.
```
Solution: The node is still synced, but it is matched by name. A similar warning is logged when several nodes report the same system UUID, which usually means virtual machines were cloned from a template without regenerating their UUID. See Node identity.
A warning says two nodes report the same address:
```
Nodes 'worker-1' and 'worker-2' both report address 10.0.0.21/32; recording it against 'worker-1' only, so the two do not rewrite each other's interface assignment on every run.
```
Solution: The address is assigned to the first node only. Check the nodes' network configuration if the duplicate is not intended.
Discovered objects do not appear in NetBox:

Solution: Discovered data is staged as deviations in NetBox Assurance and must be applied before it appears in NetBox. Navigate to Assurance → Active Deviations and apply the deviations.

Frequently Asked Questions

Q: Does the integration write anything to the cluster?

A: No. The integration is read-only. It needs only get and list on nodes, namespaces, services, and service CIDRs, and a token with that role cannot read pods or secrets.

Q: Can I use the integration to allocate prefixes from NetBox to my cluster?

A: No. Data flows from Kubernetes into NetBox only. The integration does not configure the cluster or its network plugin.

Q: Are pods, deployments, or persistent volumes synced?

A: No. Workloads are not modeled in this release. Individual pod IP addresses and service ClusterIPs are not recorded either. The integration records the cluster, its nodes, node addresses, pod and service blocks, and LoadBalancer service addresses.

Q: Are Kubernetes services created as NetBox services?

A: No. A NetBox service must belong to a device, virtual machine, or FHRP group, and a Kubernetes service belongs to the whole cluster. For LoadBalancer services, the external IP address is recorded as an IP address with the service's namespace and name in the k8s_namespace and k8s_service_name custom fields.

Q: Are namespaces created as tenants?

A: No. Namespaces are recorded only in the k8s_namespace custom field on LoadBalancer service addresses. The integration does not create tenants.

Q: Which Kubernetes distributions are supported?

A: Upstream (vanilla) Kubernetes. OpenShift and other distributions are not supported in this release.

Q: Should my nodes be devices or virtual machines?

A: Use device for bare-metal nodes and virtual_machine for nodes that run as virtual machines. If a hypervisor integration already syncs the machines your nodes run on, virtual_machine avoids competing with it. Decide before the first run, because changing it later leaves the old records in NetBox. See Choosing a node object type and site.

Q: Can I see which physical host or hypervisor a node runs on?

A: Not as a NetBox relationship. Each node records its system UUID in k8s_system_uuid and its cloud provider ID in k8s_provider_id. You can use these to match a node to its host in another inventory, but the integration does not link them in NetBox.

Q: Why do my nodes have no primary IP?

A: The integration does not set a primary IP on nodes. If it did, a node could be matched to an unrelated NetBox record that already uses the same address as its primary IP, and that record would be overwritten. The node's addresses are still synced and assigned to its kubernetes interface.

Q: Why is the interface called kubernetes instead of eth0?

A: Kubernetes does not report a node's network interfaces, so the integration creates one interface per node to hold its addresses. A real interface name such as eth0 could match and overwrite an existing interface on a server record already in NetBox.

Q: How is a node's role determined?

A: From its node-role.kubernetes.io/<role> label. For example, node-role.kubernetes.io/control-plane becomes Kubernetes Control Plane. Nodes with no role label, which includes most worker nodes, get Kubernetes Node. See Node role mapping.

Q: Why is a cordoned node still shown as active?

A: Node status comes from the node's Ready condition. A cordoned node is still Ready and in service, it just accepts no new pods, so it stays active. A node whose Ready condition is False or Unknown is offline.

Q: Why is the cluster-wide pod CIDR missing?

A: Kubernetes does not expose the cluster-wide pod CIDR through its API. The integration records each node's pod CIDR from spec.podCIDRs instead.

Q: Why do pod and service prefixes show as 100% utilized?

A: The prefixes have Mark utilized set, because the pod addresses and ClusterIPs inside them are not synced. Without it, the blocks would look empty and invite someone to allocate addresses from them.

Q: What is the VRF named after my cluster?

A: Each cluster's pod and service prefixes are placed in a VRF named after CLUSTER_NAME, so clusters that use the same default ranges stay separate in NetBox. Its route distinguisher, which starts with k8s:, is generated by the integration and is not used in your network. See Cluster VRF.

Q: Can I rename the cluster?

A: Yes. Change CLUSTER_NAME, and the existing cluster record and VRF are renamed, because both are keyed on the kube-system namespace UID rather than on the name.

Q: Will objects be deleted from NetBox if they are removed from the cluster?

A: No. The integration does not delete NetBox objects. Nodes removed from the cluster and released LoadBalancer addresses remain in NetBox. Handle these removals manually.

Q: How do I sync more than one cluster?

A: Add a policy for each cluster, each with its own API URL, token, and CLUSTER_NAME. Two different clusters must not use the same CLUSTER_NAME.

Q: Can I sync only some of my nodes?

A: Yes. Set scope.node_labels to a Kubernetes label selector, such as "hardware=baremetal". See scope Parameters.

Common Issues

Frequently Asked Questions

On this page