How to Automate Proxmox Virtual Machine Configuration with NetBox and Ansible AWX/Tower/Automation Platform (AAP)

As I wrote in Part 1 of this Series, NetBox is the world’s most popular Network Source of Truth (NSoT) with tens of thousands of installations globally.  I also discussed the importance of Proxmox in the ecosystem of freely available virtualization technologies alongside the criticality of documenting desired states of virtual machines (VMs) to correspond with their operational states.  Finally, I discussed why and how you’d want to integrate NetBox and Proxmox for the sake of consistency and reliability.

As was the case before, netbox-proxmox-automation is free for all to use.  Version 1.1.0 was just released, and will help you get this automation rolling easily in NetBox.

Between part 1 of this series and this posting, NetBox Labs also delivered a weblog around Event-Driven Automation with NetBox and Ansible Automation Platform.  Building on that, this weblog focuses on how to automate and manage the state of VMs in Proxmox from NetBox – by way of using AWX/Tower/AAP.

As I discussed in part 1 of this weblog, the underlying automation is driven by using webhooks and event rules in NetBox. When you induce a change to a [Proxmox] VM in NetBox, this triggers an event (based on a matching event rule); the event rule then connects to an underlying webhook. However, while part 1 of this series used a Flask application to facilitate the Proxmox automation, this weblog focuses on using AWX/Tower/AAP to facilitate the Proxmox automation instead.

Why Ansible?

Ansible is ubiquitous in the realm of automation.  It is freely available, widely used, has hundreds of built-in modules, an immense community, and can integrate with practically everything.  Plus there are myriad other ways that Ansible is useful to engineers in the field: the same engineers who are loath to write code.  A simple playbook is all that’s required for any engineer to start their automation journey with Ansible.  Given the context of this weblog, Ansible integrates with both NetBox and Proxmox.  

Having written that, Ansible in and of itself is an automation engine, meaning that Ansible will facilitate your automation, but you need to tell Ansible how/when/why/where you want to automate.  This is very much the case when you write an Ansible playbook then carry out your automation by running ansible-playbook from the command line.

In the case of event-driven automation with NetBox, to use an automation engine like Ansible, there needs to be a conduit to deliver this automation.  And that’s where AWX comes into play.  AWX provides a web interface and API for Ansible; further, through centralized management and job templates, AWX is a perfect fit for event-driven automation with NetBox.

AWX is free, and must run in a containerized environment.  AWX is the upstream release of RedHat’s commercial offering: Ansible Automation Platform (formerly Ansible Tower).  The netbox-proxmox-automation should work regardless if you are using AWX, Tower, or AAP.

Instead of writing a web application to handle event-driven automation with NetBox – you can write a Flask application, use the example web application that was shipped with netbox-proxmox-automation, or do something with FastAPI, among many other options – AWX has the concept of job templates.  A job template defines a set of parameters for running an Ansible job; plus, job templates are accessible through an API.  This means that you can create a webhook in NetBox that points at an underlying (AWX) job template, then an event rule in NetBox that points at the underlying webhook.  As you define your Proxmox VMs in NetBox, this will trigger the automation in AWX, and AWX in turn will run a series of Ansible playbooks to handle the automation for you.  

netbox-proxmox-automation has included all of these Ansible playbooks so that you can automate your Proxmox VMs through NetBox events and AWX.  In addition, there are two convenience scripts that you can use to create all of the NetBox objects: based on how you’ve structured your project and job templates in AWX.

More of a visual learner? You can see a netbox-proxmox-automation demonstration of AWX here.

Using netbox-proxmox-automation

The netbox-proxmox-automation does not currently discover Proxmox VMs / add them to NetBox. Proxmox has an existing NetboxPlugin.pm that should handle this functionality for you, or you can use NetBox-Proxbox. NetBox-Proxbox is a NetBox plugin that can be used to discover Proxmox VMs and add them to NetBox.

Note that you must have a running AWX (or Tower/AAP) instance before you can use netbox-proxmox-automation.  You must also have:

  • A project defined in AWX
  • A set of credentials defined in AWX
  • A set of job templates defined in AWX
  • A working execution environment

More of a visual learner? You can see a netbox-proxmox-automation demonstration of AWX here.

Accessing netbox-proxmox-automation Documentation

When you clone netbox-proxmox-automation, you can create a documentation set from the existing repository. mkdocs is used to render the documentation, which will be accessible through http://localhost:8000 on your web browser of choice. Here’s how to get the netbox-proxmox-automation documentation rolling.

shell$ cd /path/to/netbox-proxmox-automation
shell$ python3 -m venv venv
shell$ source venv/bin/activate
(venv) shell$ pip install -r requirements.txt
(venv) shell$ mkdocs serve

You should then be able to access the docs, as shown in the output message, in your web browser of choice.

Example Usage of AWX

As noted earlier, the 1.1.0 release of netbox-proxmox-automation includes two convenience scripts – such that you don’t have to create objects, webhooks and/or event rules manually, in NetBox.  Before you can use these convenience scripts, you must have created an API user (and API key) in Proxmox, and also an API token in NetBox.  You will also need to have an AWX user (and its credentials) that is capable of querying both projects and job templates in AWX.

The convenience scripts in the netbox-proxmox-automation are called netbox_setup_objects_and_custom_fields.py and netbox_setup_webhook_and_event_rules.py.  (Special thanks to our partner and friends at sol1 for contributing to these scripts.)

Before you can run either of these scripts, you need to create a configuration file.  This configuration file contains information about your NetBox and Proxmox API information, as well as information about your AWX (or Tower/AAP) configuration.  You can find a sample configuration under the conf.d directory called netbox_setup_objects.yml-sample.  Copy this configuration file to a location of your choice, and call it whatever you want.  Regardless of where you store your configuration file, it should look something like this when you are done.

shell$ cat my-config.yml
proxmox_api_config:
  api_host: proxmox-ip-or-hostname
  api_port: 8006-or-whatever-port
  api_user: proxmox_api_user
  api_token_id: name_of_proxmox_api_token
  api_token_secret: proxmox_api_secret_token
  verify_ssl: false-or-true
netbox_api_config:
  api_proto: http # or https
  api_host: name-or-ip-of-netbox-host
  api_port: 8000-or-whatever
  api_token: netbox_api_secret_token
  verify_ssl: false-or-true
proxmox:
  cluster_name: proxmox-ve-or-whatever
netbox:
  cluster_role: Proxmox-or-whatever
  vm_role: "Proxmox VM or whatever"
automation_type: ansible_automation
ansible_automation:
  host: name-or-ip-of-awx-tower-aap-host
  http_proto: http-or-https
  http_port: 80-or-whatever
  ssl_verify: false # or true
  username: awx_user # should have correct permissions
  password: awx_password
  project_name: netbox-proxmox-ee-test1 # or whatever your ee is

NetBox Initial Configuration

Once you have written your configuration file, let’s start with netbox_setup_objects_and_custom_fields.py.  netbox_setup_objects_and_custom_fields.py will configure related objects and customizations related to Proxmox in NetBox (custom field choice sets and related custom fields).

shell$ cd /path/to/netbox-proxmox-automation/setup

shell$ python3 -m venv venv

shell$ source venv/bin/activate

(venv) shell$ pip install -r requirements.txt

(venv) shell$ ./netbox_setup_objects_and_custom_fields.py –config /path/my-config.yml
(Output suppressed)

When this automation has completed, your NetBox UI should look something like this.

Webhooks and Event Rules Initial Configuration in NetBox

Now let’s move on to netbox_setup_webhook_and_event_rules.py.  netbox_setup_webhook_and_event_rules.py will query your AWX (or Tower/AAP) instance for information about your project and job templates, and will create the related webhooks and event rules in NetBox.  Assuming that you used netbox-proxmox-automation for its Ansible playbooks when you created your project and job template, this process should be a snap.

If you are not already in a Python 3 venv, do this:

shell$ cd /path/to/netbox-proxmox-automation/setup

shell$ python3 -m venv venv

shell$ source venv/bin/activate

(venv) shell$ pip install -r requirements.txt

Once you are in the venv, make sure that your configuration is all set – then do this:

(venv) shell$ ./netbox_setup_webhook_and_event_rules.py –config /path/my-config.yml
(Output suppressed)

The result of this automation is that the webhooks should look like this in NetBox.

Let’s take a look at the webhook that’s called proxmox-clone-vm-and-set-resources-ansible-automation.  netbox_setup_webhook_and_event_rules.py will create all of the settings that you need for this NetBox webhook object.

  • name
  • HTTP Method
  • Payload URL
  • HTTP Content Type
  • Additional headers (for authentication to AWX/Tower/AAP)
  • Body Template, which is used to create ‘extra_vars’ that are passed along to AWX/Tower/AAP in the payload.
  • SSL verification (set to false in our configuration)

The event rules should look like this in NetBox.

Now let’s take a look at the event rule that’s called proxmox-clone-vm-and-set-resources-ansible-automation.  netbox_setup_webhook_and_event_rules.py will create all of the settings that are required in this NetBox event rule.

  • name
  • enabled
  • Conditions
  • Object Types
  • Event Types
  • Action (type Webhook, related webhook object)

AWX/Tower/AAP Automation Details

The following describes how specific event rules trigger Proxmox automation. 

  • When a new Proxmox VM is created in NetBox (VM name, status == ‘Staged’, name of Proxmox VM template selected, SSH public key populated), netbox-proxmox-automation will:
    • Clone a Proxmox VM (cloud-init) template to a new VM of ‘name’
    • Define any resources (vcpu, memory size) in the cloned Proxmox VM that were defined in NetBox
    • Define SSH key in the cloned Proxmox VM that was defined in NetBox
  • When a Proxmox VM is changed in NetBox (VM name, status == ‘Staged’, VM network interface created, primary IP set for network interface, SSH public key populated), netbox-proxmox-automation will:
    • Collect the IP address from Proxmox VM configuration in NetBox, and define ‘net0’ (ipconfig0) configuration in Proxmox for the VM. This will set the IP address and default gateway of the Proxmox VM
    • Define SSH key in the cloned Proxmox VM that was defined in NetBox (if not specified earlier)
  • When a Proxmox VM is changed in NetBox (VM name, status == ‘Staged’, VM disk ‘scsi0’ created with a size of 20000MB), netbox-proxmox-automation will:
    • Resize ‘scsi0’ in Proxmox to 20GB. (NOTE: Proxmox will not let you shrink disk sizes.)
  • When a Proxmox VM is changed in NetBox (VM name, status == ‘Staged’, VM disk ‘scsi1’ created with a size of 15000MB), netbox-proxmox-automation will:
    • Create disk ‘scsi1’ in Proxmox with a size of 15GB. (NOTE: Proxmox will not let you shrink disk sizes.)
  • When a Proxmox VM is changed in NetBox (VM name, status == ‘Staged’, VM disk ‘scsi1’ resize to 30000MB), netbox-proxmox-automation will:
    • Resize disk ‘scsi1’ in Proxmox to size 30GB. (NOTE: Proxmox will not let you shrink disk sizes.)
  • When a Proxmox VM is changed in NetBox (VM name, status == ‘Staged’, VM disk ‘scsi1’ removed), netbox-proxmox-automation will:
    • Remove disk ‘scsi1’ in Proxmox, for VM name.
  • When a Proxmox VM is changed in NetBox (VM name, status == ‘Active’), netbox-proxmox-automation will:
    • Start Proxmox VM with name VM name.
  • When a Proxmox VM is changed in NetBox (VM name, status == Offline), netbox-proxmox-automation will:
    • Stop Proxmox VM with name VM name.
  • When a Proxmox VM is removed from NetBox (VM name), netbox-proxmox-automation will:
    • Remove VM from Proxmox

Give netbox-proxmox-automation a try today!

You can see a netbox-proxmox-automation demonstration below.

Got feedback? I’d love to hear from you on the NetDev Slack or through feature or pull requests on GitHub.

Share the Post:

Related Posts