Portable Schema

The portable schema feature allows Custom Object Type (COT) definitions to be exported as structured YAML documents, versioned in source control, and applied to other NetBox instances. This makes COT schemas shareable, auditable, and deployable across environments in a consistent and repeatable way. YAML is the recommended format; JSON is also accepted (see Schema Document Format).

Concepts

Schema Documents

A schema document is a mapping that fully describes one or more Custom Object Types - their names, metadata, and all field definitions. The document is self-contained: a reader does not need access to the originating NetBox instance to understand or validate it.

schema_version: "1"
types:
  - name: circuit
    slug: circuit
    verbose_name: Circuit
    verbose_name_plural: Circuits
    description: WAN circuit inventory
    fields:
      - { id: 1, name: carrier, type: text, required: true }
      - { id: 2, name: bandwidth_mbps, type: integer, validation_minimum: 0 }
    removed_fields: []

Schema IDs

Every CustomObjectTypeField carries a schema_id - a stable, monotonically increasing integer scoped to its parent COT. Schema IDs are the primary key used by the comparator and executor to match fields across export and apply cycles:

• Assigned automatically on first save if not set explicitly.
• Scoped per COT - field 3 in COT "circuit" is unrelated to field 3 in COT "device-profile".
• Never reused. When a field is deleted its schema_id is retired. The parent COT's next_schema_id counter only ever advances, even if fields are removed.
• Stable across renames - a field retains its schema_id when its name attribute changes, so the comparator correctly identifies a rename rather than a deletion plus addition.
• Assigned atomically using SELECT ... FOR UPDATE on the parent COT row to prevent race conditions under concurrent field creation.

Tombstones

When a field is removed from a COT, the comparator needs to distinguish "this field was intentionally deleted" from "this field is not in the schema yet." Tombstone entries in removed_fields provide that signal:

removed_fields:
  - { id: 4, name: legacy_carrier_code, type: text, removed_in: "2.0.0" }

A tombstone records the field's last-known id, name, type, and the version string when it was removed. The executor uses tombstones to drop the corresponding DB column; without a tombstone the comparator treats the absent field as ambiguous and emits a warning rather than a REMOVE.

Tombstones are only recorded by applying a document: deleting a field in the UI or the REST API doesn't add one. When a document is applied, its removed_fields are stored in the type's schema_document (replacing any stored earlier), and every later export includes them. To keep the full removal history, carry each tombstone forward into every later document, as an exported document does.

Schema Document Storage

CustomObjectType.schema_document stores the most recently applied schema document as a JSON blob. It is written by the executor after a successful apply (exporting doesn't change it), and read by the exporter when building tombstone lists. It also holds a snapshot of the type's base database columns, taken when the type is created, which manage.py upgrade_custom_objects uses to detect columns that changed in an upgrade.

Design Decisions

Integer schema_id Instead of UUID

Several alternatives were considered for the stable field identity value:

Integers were chosen because the intended workflow is single-source-of-truth: one canonical environment exports schemas, and downstream instances receive and apply them. In this hub-and-spoke model there is no opportunity for two instances to independently create fields and generate conflicting IDs, so UUID collision avoidance is unnecessary.

This follows the same convention as Protocol Buffers, which also uses integer field IDs and places the responsibility for incrementing them on the schema author.

Single-Source-of-Truth Distribution Model

The feature is designed around a hub-and-spoke topology:

1. A canonical environment (e.g., a development instance or a dedicated schema registry) defines and maintains COT schemas.
2. Schema documents are exported and committed to version control.
3. Downstream instances (staging, production) receive schema documents and apply them via the API.

The comparator and executor handle the downstream side: they diff an incoming schema against the live DB state and apply the delta atomically. There is no merge logic for reconciling independent changes from multiple peers.

Identifier Pattern Alignment

COT names (CustomObjectType.name) and field names (CustomObjectTypeField.name) must satisfy the pattern ^[a-z0-9]+(_[a-z0-9]+)*$. This is the same pattern used by the JSON Schema identifier definition, ensuring that any name accepted by the database will also pass schema validation and round-trip cleanly through export and apply cycles.

The pattern permits:

• circuit, bandwidth_mbps, carrier_code_v2

The pattern rejects:

• _private (leading underscore)
• foo_ (trailing underscore)
• test__field (double underscore)
• my-field (hyphen)

Schema Document Format

Schema documents may be written in YAML (recommended) or JSON - both parse to the same structure described below, and the preview and apply endpoints accept either. The validator for this structure lives at netbox_custom_objects/schema/cot_schema_v1.json, a JSON Schema definition - it validates the parsed document itself, independent of whether it arrived as YAML or JSON, and is used by the API endpoints before any DB access.

Top-Level Structure

COT Definition

Field Definition

All fields share these base attributes:

Attributes that match their defaults are omitted from exported documents to keep output minimal.

Field Types and Type-Specific Attributes

An array field's array_element_type and array_tuple_size, and a field's type changing to or from array, can't be changed by applying a document; doing so raises UnsupportedFieldChangeError. For example, {id: 3, name: points, type: array, array_element_type: integer, array_tuple_size: 3, array_min_length: 2} stores a list of at least two (x, y, z) integer points.

on_delete_behavior is one of "set_null" (default), "cascade", or "protect". It applies only to single object fields.

related_name names the reverse relation on the related object type, e.g. related_name: widgets lets a device object field be followed back as device.widgets. It's optional: with the default "", a non-polymorphic object field gets a generated name, and multi-object and polymorphic fields get no reverse relation. When a document omits it, an existing field keeps its name (documents exported before related_name was added don't include it); "" clears it. It uses lowercase letters, digits and single underscores (max 100 characters), and must be unique among the fields pointing at each object type. Changing it renames the reverse relation.

is_polymorphic defaults to false. When true, the field uses a generic foreign key and accepts references to any of the types listed in related_object_types. The is_polymorphic flag can't be changed after the field is created. Applying a document can change the set of allowed types, which the UI and REST API don't allow.

Related Object Type Encoding

related_object_type (and each entry in related_object_types) is encoded as a /-separated string:

• Built-in NetBox model: "dcim/device", "ipam/prefix"
• Custom Object Type: "custom-objects/<cot-slug>", e.g. "custom-objects/circuit"

Constraint Definition

constraints:
  - name: vendor_serial
    fields: [1, 2]
  - name: serial_ci
    fields: [2]
    case_insensitive: true
  - name: name_unique_top_level
    fields: [3]
    condition:
      - {field: 4, lookup: isnull, value: true}
  - name: end_after_start
    type: check
    rule:
      - {field: 6, lookup: gt, value: {field: 5}}
    error_message: The end date must be after the start date.

Constraints are matched by name. As with fields, a constraint in the DB but absent from constraints is left in place (with a warning) unless its name is listed in removed_constraints. Removing a constraint doesn't delete data, so it doesn't need allow_destructive.

Tombstone Record

id: 4
name: legacy_carrier_code
type: text
removed_in: "2.0.0"

removed_in is optional but recommended. The id value must match the original field's schema_id and must not appear in the active fields list.

Usage

Exporting a Schema

Use the Python API from netbox_custom_objects.schema.exporter. Before running either option, switch to your NetBox installation root and activate its virtualenv:

# Replace /opt/netbox with your NetBox installation root.
export NETBOX_ROOT=/opt/netbox
cd "$NETBOX_ROOT"
source "$NETBOX_ROOT/venv/bin/activate"

Option 1 - NetBox shell (recommended)

Run manage.py nbshell from the NetBox installation root. Django is fully initialised for you, and all models are importable immediately:

python3 netbox/manage.py nbshell

Then inside the shell, export specific COTs by slug:

from netbox_custom_objects.schema.exporter import export_cots
from netbox_custom_objects.models import CustomObjectType
import yaml

cots = CustomObjectType.objects.filter(slug__in=["circuit", "device-profile"])
document = export_cots(cots)
print(yaml.safe_dump(document, sort_keys=False))

Or export all COTs at once:

from netbox_custom_objects.schema.exporter import export_cots
from netbox_custom_objects.models import CustomObjectType
import yaml

cots = CustomObjectType.objects.all()
document = export_cots(cots)
print(yaml.safe_dump(document, sort_keys=False))

To run a script file non-interactively, pipe it in:

python3 netbox/manage.py nbshell < /path/to/export_cot.py

Option 2 - Standalone script

If you need to run a .py file directly (e.g. from a cron job or CI pipeline), you must bootstrap Django yourself before importing any NetBox or plugin code:

PYTHONPATH="$NETBOX_ROOT/netbox" python3 /path/to/export_cot.py

Where /path/to/export_cot.py is your script, containing:

import os
import django

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "netbox.settings")
django.setup()  # must be called before any model or app imports

from netbox_custom_objects.schema.exporter import export_cots
from netbox_custom_objects.models import CustomObjectType
import yaml

cots = CustomObjectType.objects.filter(slug__in=["circuit", "device-profile"])
document = export_cots(cots)
print(yaml.safe_dump(document, sort_keys=False))

export_cots returns a dict with schema_version and types. For a single COT without the document wrapper, use export_cot(cot).

Previewing a Schema (API)

POST /api/plugins/custom-objects/schema/preview/

Submit a schema document and receive a structured diff showing what would change, without modifying the database. The request body and response may be YAML or JSON - YAML is recommended and is what's shown here; send Content-Type: application/json (and, if you want a JSON response, Accept: application/json) to use JSON instead. A request with no Accept header gets a JSON response.

POST /api/plugins/custom-objects/schema/preview/
Content-Type: application/yaml
Accept: application/yaml
Authorization: Bearer nbt_<key>.<token>

schema_version: "1"
types:
  - name: circuit
    slug: circuit
    verbose_name_plural: Circuits
    fields:
      - { id: 1, name: carrier, type: text, required: true }
      - { id: 3, name: contract_ref, type: text }
    removed_fields:
      - { id: 2, name: bandwidth_mbps, type: integer, removed_in: "2.0.0" }

Response 200:

diffs:
  - slug: circuit
    name: circuit
    is_new: false
    has_changes: true
    has_destructive_changes: true
    cot_changes: {}
    field_changes:
      - op: add
        schema_id: 3
        db_name: null
        schema_def: { id: 3, name: contract_ref, type: text }
      - op: remove
        schema_id: 2
        db_name: bandwidth_mbps
        schema_def: { id: 2, name: bandwidth_mbps, type: integer, removed_in: "2.0.0" }
    warnings: []

has_destructive_changes: true indicates that applying this schema would drop at least one column. The preview endpoint never returns 409 - it is safe to call at any time.

Applying a Schema (API)

POST /api/plugins/custom-objects/schema/apply/

POST /api/plugins/custom-objects/schema/apply/
Content-Type: application/yaml
Accept: application/yaml
Authorization: Bearer nbt_<key>.<token>

allow_destructive: false
schema: { ... }

• allow_destructive (default false): must be true for the apply to proceed when the diff contains REMOVE operations. If false and removals are present, the endpoint returns 409 Conflict.
• The apply is fully atomic - a failure at any point rolls back all changes including newly created COT tables (PostgreSQL supports transactional DDL).
• Permissions: every apply requires both add and change permission on Custom Object Types, even when the document only updates existing types. Beyond that, permissions are checked as the UI and REST views check them, object-level constraints included: change permission on every existing Custom Object Type in the document (it rewrites their schema_document), add permission for new types, and add, change or delete permission on the fields and constraints being added, changed or removed. Missing permissions return 403 Forbidden before anything is changed; a result that falls outside a constraint returns 403 and is rolled back.
• On success, schema_document is persisted on each affected COT so tombstones are available for future export/diff cycles.
• Schema apply works inside a NetBox Branching branch: with a branch active, the Custom Object Type and field changes are made in the branch and the DDL runs against the branch's schema, so main is unchanged until the branch is merged. A failed or rejected apply is rolled back in the branch too.

Response 200:

applied: true
diffs: [ ... ]

Response 409 Conflict:

error: destructive_changes
detail: "Schema contains destructive field removals for COT(s): circuit. Pass allow_destructive=True to apply them."
destructive_slugs: [circuit]

Response 400 Bad Request, for a document that doesn't match the schema format (at most ten errors are listed):

schema_errors:
  - path: [types, 0, fields, 1, type]
    message: "..."

Other 400 Bad Request responses give an error code and a detail message:

error: unresolvable_reference
detail: "..."

Typical End-to-End Workflow

1. Define and iterate on COT schemas in a development environment using the NetBox UI or API.
2. Export the schemas to a YAML file and commit to version control.
3. Review the diff in the PR - because IDs are stable integers and defaults are elided, the diff is human-readable.
4. Preview the schema on a staging instance using the preview endpoint to confirm the diff matches expectations.
5. Apply the schema on staging (and then production), using allow_destructive: true only when column drops have been explicitly reviewed.

Field Deprecation Lifecycle

Fields can be marked deprecated without being removed, allowing a grace period before deletion:

id: 5
name: old_carrier_name
type: text
deprecated: true
deprecated_since: "2.1.0"
scheduled_removal: "3.0.0"

• deprecated: true marks the field as deprecated. It's informational only: the field keeps working, and values can still be entered and changed. NetBox doesn't act on it.
• deprecated_since records the version in which the field was deprecated.
• scheduled_removal signals to consumers when the field will be tombstoned.

These attributes can be set by applying a document or through the REST API, but not in the web UI.

Deprecation is non-destructive. The field remains in fields (not removed_fields) until it is actually deleted, at which point a tombstone entry should be added.

Comparator (Developer Reference)

netbox_custom_objects/schema/comparator.py - pure-read, no DB writes.

from netbox_custom_objects.schema.comparator import diff_document, diff_cot

diffs = diff_document(schema_doc)   # list[COTDiff]
diff  = diff_cot(type_def)          # COTDiff

COTDiff

Convenience properties: has_changes, has_destructive_changes, adds, removes, alters.

FieldChange

Properties: is_rename, is_type_change.

Matching Rules

• Fields are matched exclusively by schema_id. DB fields with no schema_id generate a warning, not a REMOVE.
• REMOVE is emitted only when a schema_id appears in the document's removed_fields. A field absent from both fields and removed_fields generates a warning.

Executor (Developer Reference)

netbox_custom_objects/schema/executor.py - writes to the DB.

from netbox_custom_objects.schema.executor import apply_document, apply_diffs

diffs = apply_document(schema_doc, allow_destructive=False)  # list[COTDiff]
apply_diffs(diffs, type_defs_by_slug, allow_destructive=False)  # lower-level

apply_document is the primary entry point. apply_diffs is available when diffs have been pre-computed by the comparator (e.g. for preview-then-apply flows).

All DB writes are wrapped in a single transaction.atomic() block. Any exception causes a full rollback.

Exceptions

DestructiveChangesError is raised before the transaction opens, so the DB is never touched. The other exceptions may be raised mid-transaction, triggering a full rollback.

Constraint Ordering

Removed and changed constraints are deleted before any field operations, and added and changed constraints are created after them, so a constraint can cover a field added in the same document and never blocks a field change that the document also makes.

Dependency Ordering

When a schema document contains multiple new COTs that reference each other via related_object_type: "custom-objects/<slug>", the executor performs a topological sort to ensure referenced COT tables exist before any referencing field is added. Cycles among new COTs raise CircularDependencyError.

Backfilling Pre-Existing Fields

Fields created before the portable schema feature was introduced have schema_id = null. Migration 0008_backfill_schema_ids assigns IDs to all such fields in PK order and updates each COT's next_schema_id counter accordingly. This migration runs automatically with manage.py migrate.

After the backfill, all existing fields participate in export and diff cycles normally.