This article is part a 4-part series on Navigating Network Automation with NetBox. See the other parts covering: the Design Stage, the Build & Deploy Stages, and the Operate Stage.
We estimate that NetBox can be found in more than 10 thousand organizations worldwide, and we know that many of those organizations are successfully using NetBox to power their network automation initiatives.
In this blog post you’ll learn about:
- What is causing companies to reach for network automation
- How companies are investing more than ever before in network automation
- The real challenges companies are facing on their network automation journeys
- How NetBox Labs and the worldwide NetBox community are stepping up to help out
- The Network Lifecycle that we’ll be building on in the following months to highlight the patterns that companies are adopting to get the most out of network automation
The reasons to automate are everywhere
Network automation is growing in popularity as networking teams seek to get ahead of increasing demands on their capabilities and with this surge in attention we see the market responding, but what are these new demands being placed on networking teams that are causing them to reach for network automation?
In a recent survey of 354 IT professionals by Enterprise Management Associates (EMA) Shamus McGillicuddy, EMA’s Vice President of Research, found that the main initiatives causing companies to look at network automation were Cloud Migration (42%), Multi-Hybrid Cloud Architecture (42%), IoT/OT (35%), Regulatory Compliance (34%), AI (34%), and Zero Trust Security (29%). What I find interesting is that there isn’t really one leading driver here which is way ahead of the pack, they’re fairly tightly grouped which suggests that whatever networking teams are being asked to work on, they are finding that network automation could help.
The survey data tells a similar story when we look at the events and pains that trigger businesses to invest in network automation: Network Complexity (45%), Inefficient Operations (42%), Network Outages/Degradations (33%), Lack of Skilled Engineers (32%) and Security Breaches (31%). Again we see a reasonably tight grouping, which tells me that networking teams are being asked to do more, and when they try to do more they encounter a range of challenges where network automation could help.
Network automation investment is flowing, but many companies still struggle
To tackle this, companies are investing in network automation with 91% of respondents reporting that they have a specific budget set aside for network automation, and 75% planning to increase that investment over the next two years. While we suspect that this pattern might not hold across every industry, it’s still a strong signal that automation is top of mind, and wallet, for an increasing number of networking teams.
So networking teams are feeling the pain and investing accordingly and because of that the market is responding. Networking conferences and news outlets are featuring more and more automation with some focusing solely on automation content such as NetDevOps Days that NetBox Labs and our partners kicked off in 2023 and later merged with Network Automation Forum’s increasingly popular AutoCon series of conferences.
Alongside the massively increased conversation around network automation, we’re seeing the emergence of tooling and practices in networking, some new and some from existing disciplines outside of networking, stepping up and being explored to help networking teams succeed with automation.
So the challenges are real, investment is flowing and everyone is talking about and adopting network automation, so how is it going? For most it’s not going very well. According to the same survey only 18% of respondents rate their network automation strategies as a complete success citing the following main issues hampering their network automation journeys: Poor IT Leadership (31%), Staffing and Skills Gaps (27%), Budget issues (25%), Conflicts and Collaboration between groups (25%), and Security Policy Constraints (25%).
The future is already here – it’s just not evenly distributed
We can’t help you with all of the above challenges, but we can help with some. While the huge attention and investment in network automation should be viewed as a positive development, it can leave some networking teams asking: what does all this mean? and how do we get started?
At NetBox Labs our huge community gives us the privilege of peering into the challenges and successes of hundreds of companies on their network automation journeys and we’ve recognized patterns. Some of those patterns are captured in our Network Automation Reference Architecture, but while the architecture does a good job of giving a high-level view of the state of the art, it leaves much to the imagination around the exact implementation details and the “why?” behind each of the steps companies can take to move from zero automation to an appropriate amount of network automation for their needs.
I’d like to underline that last point: blindly automating parts of your network without thinking about the potential benefits of doing so might be fun but risks burning time with little ROI. Also, we hear that many engineers find it difficult to get time (or permission!) to work on automation anyway, making it all the more important to invest that time wisely.
To help networking teams navigate these decisions we’ll be releasing a ton of content over the next few months in which we’ll look at the patterns we see companies using to advance the automation of their networks with NetBox, with the goal of demystifying the terminology, the tooling and the architectures that successful networking teams have adopted. To get into the nitty gritty we’ll also be kicking off a sister series to the already popular NetBox Heroes podcast.
In Network Automation Heroes we’ll be speaking to the toolmakers, thinkers, designers and implementers from the NetBox community who are pushing network automation forward and harnessing their energy, insights and experience to help you understand how it all fits together.
First principles
So let’s close this blog post out by teeing up the framework we’ll use for the content that will follow. I’m a big sci-fi fan and I like to imagine automation like a bionic exoskeleton, it’s not doing anything you don’t already do and it’s not intended to replace you, it’s intended to make you faster and stronger, but we’ve got to learn how to get the most out of it to justify the investment and not accidentally break things with our newfound strength. To start stepping through how network automation can help you scale and manage your networks it therefore makes sense to start with how network engineers scale and manage their networks when there is no automation and build from there.
Let’s first consider the lifecycle that all networks go through: Design, Build, Deploy, and Operate.
Design
In the design phase we are asked to understand business requirements and translate them into additions or changes to our network to enable those requirements. We can split this into three categories:
Understanding Business Requirements Gathering and interpreting the needs of the business, including performance, budgetary, and compliance requirements. What behaviors should the network support after this change? How much do we have to spend on it and what constraints do we need to work within?
Generate Appropriate Designs Creating detailed network designs that can be practically implemented. These designs should be accurate enough that the Build stage can be carried out with maximal independence and minimal rework.
Issue Resolution Identification and resolution of any potential design issues before they impact the build or deployment stages. The earlier we find problems with our translation of business requirements to buildable designs the faster and less expensively they can be addressed.
It’s worth noting that the outputs from the design phase can vary greatly, we’ll cover that in the coming weeks.
Build
In this stage the teams responsible for building the network take the output from the Design stage and start translating it into action. This can include procurement teams buying and shipping equipment, datacenter teams racking and stacking, working with providers to connect circuits, and a whole host of other activities which must produce a network which adheres to the design plans, within budgetary and other constraints.
Issues that might be quickly addressed in the Design stage can be costly to address in the Build stage so this step is critically dependent on the quality of the designs and requires clear collaboration and communication ensuring that those doing the building work closely with those who created the design to accurately translate the network design into a physical or virtual infrastructure.
To reduce risk here some companies leave certain details of the design to the build stage, allowing those on the ground to improvise within the higher level design, reducing the potential fragility of this critical handover.
Deploy
Once the physical network is in place the logical network must be deployed. This typically falls into two broad categories of work.
Configuration Deployment The application of the correct configurations to network devices and setting up supporting systems (e.g., permissions, logging, and monitoring). In a zero automation situation this typically involves SSHing into devices to configure them or to push predefined configurations.
Validation and Testing Ensuring the network operates as intended before going live. Here we rely on the initial requirements and the intended design to verify that the network is performing as intended before handing it over to our users and the Operate stage.
Operate
This is the bread and butter of the Network Operations Centre (NOC) and includes handling input from the two main sources of information we have about whether our network is meeting its requirements: the network itself, and its users.
Continuous Validation and Maintenance Continuously validate the network to ensure it meets business requirements and perform routine maintenance.
Incident Response and Management Handling issues reported by users and making necessary adjustments or repairs.
It’s worth noting that not all changes to the network would follow this whole process, for example building out a datacenter might warrant following the full cycle from Design through Operate, whereas enabling a new services by adding an IP to an interface and assigning it to a VLAN might bypass a formal Design stage and skip straight to those responsible for the Operate stage.
The Network Lifecycle
Of course few networks are ever “done”, so the above stages are in fact a Network Lifecycle and constitute the foundational model that we’ll use alongside our Network Automation Reference Architecture as the basis for the content to follow. We’ll also see how companies who need to move fast often collapse these stages into more of a constant feedback loop than a set of well-defined hand overs, but that will become clear in the following weeks.
In this series you’ll see the automation patterns, techniques and tools that successful companies apply to take advantage of network automation across all of these stages, and as we do that we’ll see how the NetBox Labs Network Reference Architecture can provide a useful guiding light for your network automation initiatives.
Lastly, network automation is a fast moving space so this series is intended as a conversation, not a lecture. We’re excited to have already seen such a strong response in the NetoBox community; from the members of the NetBox Design Partner Program who worked on the initial network automation reference architecture, to all those who have stepped up to share their experiences for the upcoming blogs and videos, it seems like everybody is excited to converse and contribute.
If you’re reading this and you have feedback or ideas to share, I’d love to hear from you. You can find me on X (@mrmrcoleman) and on the NetDev Slack. Onwards!
In my next post, I dive in more depth into the Network Lifecycle, starting with the Design Stage.
