TurboBulk
TurboBulk is a high-performance bulk data API for NetBox that loads, updates, deletes, and exports tens of thousands of objects per second, orders of magnitude faster than the REST API. It uses PostgreSQL COPY, staging tables, and Apache Parquet to achieve throughput of 5,000-25,000 rows/sec across all NetBox object types.
On NetBox Enterprise, the TurboBulk server-side plugin ships built into the nbe-core image. There is nothing to install on the server. You enable it through your deployment's configuration on a Premium-tier license, then connect with the client library.
Availability: Premium tier on NetBox Enterprise.
Client library, detailed docs, and examples: netboxlabs/netbox-turbobulk-public
Enabling TurboBulk
TurboBulk activates only when two conditions are both met:
- The plugin lever is on (it is on by default).
- The license is Premium tier.
The tier check is authoritative. On a license below Premium, TurboBulk does not load even if the lever is set directly. Read and bulk-export operations are available as soon as the plugin is active; the bulk write path (load and delete) is a separate opt-in that is off by default. See Write operations are disabled by default.
Embedded Cluster (KOTS admin console)
On an Embedded Cluster install, use the Administration Console:
- Open the Administration Console and go to Config.
- Find the TurboBulk section. If you do not see it, your license does not include the Premium tier that TurboBulk requires.
- Enable TurboBulk is on by default. Leave it on to keep the plugin active.
- To turn on the bulk write path, enable Enable Write APIs (bulk load / delete). This is off by default.
- Save the config and deploy the new version.
Helm
On a Helm install, set the TurboBulk values under netboxEnterprise.spec in your values.yaml:
netboxEnterprise:
spec:
turbobulk:
enabled: true
enableWrites: false
enabled(defaulttrue) is the plugin lever. It no-ops on licenses below Premium.enableWrites(defaultfalse) turns on the bulk write path. It cannot take effect unless the plugin is active.
Apply the change with helm upgrade. For how the netboxEnterprise values map to the NetBoxEnterprise resource, see Helm Values Reference.
Turning TurboBulk on or off triggers a database migration. Toggling the write path on or off does not.
Multi-node and HA storage
TurboBulk writes its bulk exports and reads its bulk uploads through NetBox's configured file-storage backend. NetBox Enterprise supports two backends: a node-local filesystem volume, or external S3-compatible object storage.
On a multi-node or HA deployment, TurboBulk's file storage must be external S3-compatible object storage, not a node-local filesystem volume. A node-local volume pins the workload to the node that holds it, which breaks on failover. This is the same external-storage requirement that HA itself has. Single-node installs can use the filesystem backend.
If you have not already configured external S3 storage for a multi-node deployment, see Storage Options.
When to use TurboBulk
| Use case | Example |
|---|---|
| Initial data population | Migrate an existing DCIM/IPAM into NetBox |
| Ongoing syncs | Nightly import of 500K IP addresses from an external source |
| Bulk exports for analytics | Export all devices + interfaces to a data warehouse |
| Large-scale changes | Re-assign 100K interfaces to a new VLAN group |
| Branch-based workflows | Load proposed changes into a NetBox branch, review, merge |
When NOT to use TurboBulk
- < 1,000 objects - the REST API is fine at this scale.
- Interactive single-object edits - use the NetBox UI or REST API.
- Workflows that depend on synchronous custom validators - TurboBulk does not execute Django model validators (DB constraints are still enforced).
- Workflows that depend on per-object webhooks firing in real time - events are dispatched asynchronously after the entire operation completes.
Capabilities
| Capability | Description |
|---|---|
| Bulk insert | Insert new rows from JSONL or Parquet files |
| Bulk upsert | Insert or update on conflict (configurable conflict fields or named constraints) |
| Bulk delete | Delete rows by primary key or custom key fields, with cascading FK nullification |
| Bulk export | Export full tables or filtered subsets to JSONL (gzipped) or Parquet |
| Export caching | Server-side cache with TTL - repeated exports return instantly via cache key |
| Dry-run validation | Validate data without committing; returns errors and rolls back |
| Validation modes | none (skip), auto (SQL pre-validation for IP/prefix), full (Django full_clean() per row) |
| Branching | Operate within a NetBox branch (requires netbox-branching plugin) |
| Changelogs | Generate ObjectChange records for audit trail (opt-in per operation) |
| Event dispatch | Fire webhooks and event rules after operation completes (async) |
| Save hooks | Replicate model save() side-effects as bulk SQL (computed fields, denormalization) |
All write operations are fully atomic - they succeed completely or roll back with no partial state.
Getting started
Prerequisites
- NetBox Enterprise on a Premium-tier license with TurboBulk enabled
- A NetBox API token with appropriate object permissions
- Python 3.10+
Install the client
pip install turbobulk-client
Quick example
import os
from turbobulk import TurboBulkClient
client = TurboBulkClient(
url=os.environ["NETBOX_URL"],
token=os.environ["NETBOX_TOKEN"],
)
# Bulk insert sites
data = [
{"name": "Site-A", "slug": "site-a", "status": "active"},
{"name": "Site-B", "slug": "site-b", "status": "active"},
]
result = client.load("dcim.site", data, mode="insert")
print(f"Inserted {result['rows_inserted']} sites")
# Bulk export all sites
export = client.export("dcim.site")
print(f"Exported {export['rows_exported']} sites")
For more examples, see the example scripts in the public repo.
Data formats
TurboBulk supports two wire formats:
| Format | Extension | Best for | Notes |
|---|---|---|---|
| JSONL | .jsonl, .jsonl.gz | Simplicity, small-to-medium datasets | One JSON object per line. Gzip supported. Default format. |
| Parquet | .parquet | Maximum throughput, 100K+ rows | Columnar, compressed, strongly typed. Requires PyArrow. |
Format is auto-detected from file content, Content-Type header, or filename extension. When in doubt, start with JSONL and switch to Parquet when throughput matters.
Important caveats
FK columns must use the _id suffix
This is the most common source of errors. Foreign key fields must reference the database column name (with _id suffix), not the Django relation name:
{"name": "switch-01", "site_id": 1, "role_id": 3, "device_type_id": 5}
Not: {"name": "switch-01", "site": 1, "role": 3, "device_type": 5}
Use GET /api/plugins/turbobulk/models/{model}/ to discover the exact field names and types for any model.
Custom validators are not executed
TurboBulk bypasses the Django ORM for writes. Database constraints (NOT NULL, UNIQUE, FK) are enforced by PostgreSQL. Built-in validation is available via validation_mode (auto or full), but custom model validators and clean() methods are not called.
Write operations are disabled by default
The bulk write path (load and delete) is off by default for safety. A single write call can modify or delete a large number of objects in one transaction and is difficult to reverse. Enable it only after you have validated your write workflows, ideally in a non-production environment first. Turn it on with the Enable Write APIs toggle (Embedded Cluster) or netboxEnterprise.spec.turbobulk.enableWrites (Helm).
Events are dispatched asynchronously
Webhooks and event rules fire after the entire operation completes, not per-object during the operation. Custom scripts are not triggered. Set dispatch_events: true in your request to enable event dispatch.
Limits
| Limit | Value |
|---|---|
| Maximum upload size | 1 GB |
| Job timeout | 1 hour |
| NetBox version | 4.5+ |
API endpoints
All endpoints are under /api/plugins/turbobulk/. Authentication is via NetBox API token.
| Method | Endpoint | Description |
|---|---|---|
GET | /models/ | List all available models and their capabilities |
GET | /models/{app_label.model_name}/ | Get full schema for a model (fields, types, constraints) |
POST | /load/ | Bulk insert or upsert from JSONL/Parquet |
POST | /delete/ | Bulk delete by primary key or custom key fields |
POST | /export/ | Bulk export to JSONL or Parquet (async job) |
GET | /jobs/{job_id}/ | Check job status and results |
GET | /jobs/{job_id}/download/ | Download export result file |
GET | /cache/{cache_key}/download/ | Download cached export by cache key |
For full request/response schemas and parameter details, see the API Reference.
Resources
| I want to... | Link |
|---|---|
| Get started with the client library | Quickstart |
| See working code examples | Examples |
| Read full API documentation | API Reference |
| Understand validation, hooks, and changelogs | User Guide |
| Use TurboBulk with NetBox Branching | Branching Guide |
| Troubleshoot errors | Troubleshooting |
| Report an issue or request a feature | GitHub Issues |
| Get support | Contact NetBox Labs |