Docs

Handling API Load and Rate Limiting

How NetBox Enterprise handles bursts of API traffic, how to keep NetBox from running out of database connections, and where to set API rate limits.

How NetBox Enterprise handles API requests

Each NetBox pod runs a web server called Granian, with 4 worker processes.

  • Backpressure is how many requests a worker accepts at once.
  • Blocking threads is how many requests a worker runs at the same time. Each running request holds one PostgreSQL connection. By default, this is half the backpressure.

To set Granian options:

  • Embedded Cluster: in the Admin Console config, check Show Advanced Settings, then use NetBox Environment Overrides.
  • Helm: spec.netbox.env.

NetBox does limit the size of a single request:

  • GraphQL: 10 aliases per query by default. Adjust it with GRAPHQL_MAX_ALIASES, and from NetBox 4.6.1 GRAPHQL_MAX_QUERY_DEPTH. On Embedded Cluster, set them in the Admin Console under NetBox Python Configuration Overrides. On Helm, use spec.netbox.config.customPythonConfig.
  • REST API: results come back in pages.
  • Diode (NetBox's data-ingestion service) paces its own writes, so the guidance below is mainly for scripts and integrations.

Common symptoms

Each fix is set in one of three places: NBE settings, your own load balancer or gateway, or your scripts and integrations.

What you seeFixWhere you set it
PostgreSQL "too many clients" errors, or NetBox stops responding during a burstCap database connectionsNBE settings
One script or integration slows everyone downRate-limit at the load balancer or gatewayYour load balancer or gateway
Large reads or batch writes are slow or time outLighten requestsYour scripts and integrations
Scripts fail on a brief errorRetry with backoffYour scripts and integrations

Cap database connections

Where: NBE settings. From version 2.5.0, NBE sets the connection limit for its bundled PostgreSQL; you set the rest.

  1. Set GRANIAN_BACKPRESSURE. Each pod then opens about 4 × that number of connections, plus more when users work in branches (see step 2). What to set depends on your version:

    • 2.1 to 2.3: backpressure is 4, so no change is needed.
    • 2.4: backpressure defaults to 256, which can use every PostgreSQL connection under load. Set it to 4, and remove the override after you upgrade to 2.5.0.
    • 2.5.0 and later: NBE sets backpressure from the database's connection limit, or uses 4 with an external PostgreSQL. A value you set yourself takes precedence.
  2. Keep the total below PostgreSQL's max_connections. The total is pods × 4 × backpressure, plus room for background workers, other NBE services and PostgreSQL's reserved connections: about 28 on a single node, or about 42 on a 3-node install, where NBE runs extra copies of Diode and the login service. Add one for each background worker over 8.

    NetBox Branching, which NBE enables, opens one more connection for each branch a request works in and holds it for 5 minutes. The total leaves these out. If your users work in many branches at once, lower GRANIAN_BACKPRESSURE or raise max_connections.

    From NBE 2.5.0, bundled PostgreSQL sets the limit from its size preset:

    PostgreSQL presetmax_connections
    Small or Unlimited100
    Medium150
    Large200
    Extra Large300

    "Unlimited" is the default preset and gets the lowest limit. On NBE 2.4 and earlier, bundled PostgreSQL uses 100 whatever the preset. To raise the limit, pick a larger preset. On Helm with NBE 2.5.0 or later, you can set spec.postgresql.maxConnections instead. With an external PostgreSQL, size against your plan's limit.

  3. Add pods only while the total still fits. On Embedded Cluster, set Number of Replicas in the Admin Console (2 by default on Pro, 4 on Premium or Enterprise). On Helm, set spec.netbox.replicas. On a cluster with more than one node, use S3 media storage or a ReadWriteMany volume first. Otherwise, pods on other nodes won't start.

Rate-limit at the load balancer or gateway

Where: your own infrastructure, in front of NBE. NetBox and NBE have no rate-limit settings.

Install typeWhere to set rate limits
HelmYour ingress controller (add its rate-limit annotations under spec.ingress.annotations), your Gateway API gateway, or the load balancer in front
Embedded ClusterThe load balancer in front of the nodes. Not the bundled Traefik, which NBE manages

On Helm, spec.ingress.annotations applies to every Ingress NBE creates, including Diode's and Copilot's when they are enabled. A limit sized for API scripts then also limits Diode and Copilot. To limit only the NetBox API, set it at your gateway or load balancer instead.

Recommended settings:

  1. Cap concurrent requests, not just the request rate. Batching clients send fewer but heavier calls.
  2. Limit per API token (the Authorization header), not per IP address.
  3. Have the load balancer return HTTP 429 (Too Many Requests) when it turns a request away, so clients know to wait and retry. Some default to a different code, such as 503.
  4. Divide limits across load balancer copies. For example, three copies at 20 per second let through about 60.

Bulk creates use the same URL and method as single creates, so they can't get a separate limit. Bulk updates and deletes send PATCH, PUT or DELETE to the list URL (for example /api/dcim/sites/), so a gateway that matches on method and path can limit them separately.

Lighten requests

Where: your scripts and integrations. These are standard NetBox API options.

  • ?brief=true: returns a short version of each object.
  • ?fields= or ?omit=: returns only the fields you need.
  • ?background=true on bulk writes: runs the batch as a background job (NBE 2.4 and later).
  • Send smaller batches instead of one large burst.

Retry with backoff

Where: your scripts and integrations. NetBox doesn't retry for clients. Scripts should retry on 429, 503 and timeouts, waiting longer between each attempt.

On this page