The Network of the Future Calls for a New Kind of Network Engineer

I have a few questions for all the network engineers out there: Do you suspect that your days should involve more than logging into switches and juggling vendor-specific management tools? Should you really have to rebuild your networking stack from the ground up every time you want to connect to a new public cloud or modify your network architecture?

If so, I’m here to tell you that it’s not just all in your head. There’s a real need to rethink network operations. In a world shaped by software-defined everything, multicloud and hybrid cloud architectures, and a deep inter-dependency between software engineers and network engineers, the old way doesn’t work.

Allow me to explain…

How the Ground Under the Network Engineers’ Feet Shifted

Until about 10 years ago, a network engineer’s role was pretty clearcut. It was to manage networking equipment using whichever set of proprietary tools the equipment vendors provided. The engineer got their CCIE or CCDE certifications, managed switches, and kept stuff running.

Then, software-defined networking became a thing, and slowly but surely life started to change. At first, the world of SDN wasn’t radically different from what preceded it (proprietary tooling still dominated), but SDN made it possible to automate more tasks using programming languages. This drove some network operators to learn things like Python and Ansible and moonlight as coders doing network fleet management.

Five or so years later, another wave of change hit. It became clear that organizations would no longer be using just one cloud or service provider. Multicloud and hybrid cloud became the trend and businesses now needed to manage multiple VPCs spread across multiple clouds; they needed to connect on-premises infrastructure to public clouds; they needed to load-balance within distributed infrastructures; and so on.

This turned the set of skills needed to automate using vendor-neutral, code-based tools from nice-to-have (like back in the early days of SDN) to essential. Complete automation became the future of networking.

As a result, today’s network teams need two fundamental skill sets: deep expertise in networking and deep expertise in software engineering. They need to know how everything works at the packet level and be able to automate everything in your network using code.

Network engineers have something you can't read about or get trained in: network operations experience and troubleshooting skills. It is understated in the industry how much it helps inform software development decisions on networking. Even amateur network scripting by a networking engineer with no understanding of development fundamentals, much less a CS background, is a force multiplier in how and what can be solved or configured once they learn how to interact with the network via software. This is especially true in smaller networks or companies that do not have a 24/7 NOC or even a "networking team.” As networks become more software defined, the ability for a network operator to “ask” the network questions about its state or health via a script or tools will be mandatory in the future.

There’s No Buzzword for It (Yet)

There are different labels you could apply to the world network engineers inhabit today—and the worldview they must embrace.

Some call it “NetDevOps,” to emphasize the blurred line between network engineering and software engineering. Although, there’s more at stake than simply teaching network operators to code or telling them to collaborate with DevOps engineers.

Others talk in terms of hybrid and multicloud networking, reflecting the role that a code-based approach to networking plays in powering complex cloud architectures.

Still others, focusing on code’s ability to drive powerful automations, simply call this new world “network automation.”

The truth is that the world that modern network engineers inhabit is all of these things and more. So far, no one has managed to come up with a label that accurately encapsulates all that modern network management is about. And that’s fine, because you don’t need a new buzzword to know that network operations have changed forever.

What’s So Good About the New Way

Rather than spend more time trying to find the right term for the concept, let’s focus on the benefits of this modern style of network engineering.

One is the ability to construct networks that are open and extensible. When you’re not wedded to proprietary tools or specific vendors, you can grow your network however you need, whenever you need.

Along similar lines is the virtually unlimited scalability this approach can give you. Using code to automate networking tasks lends itself to repeatable processes that can scale no matter how many endpoints, data centers, or clouds you include in your network.

Hybrid cloud enablement, too, goes hand-in-hand with modern network operations. Code-based, vendor-neutral tooling enables easy integration of on-premises or colocated infrastructure with public cloud resources via the network—no matter what your infrastructure configurations look like on either side, or which vendors are part of the equation.

Meanwhile, your network management tools evolve from proprietary, hard-to-customize solutions to extensible open source tools that you can tailor to your heart’s content. And because most modern network management tools are API-centric, you can manage all aspects of network operations via standardized APIs. You get to put reliance on vendor-specific languages or frameworks for every switch you manage behind you.

You’ll Need a New Network Engineering Organization

I’ve said a lot about the importance of network management tools that are open, extensible, API-centric, and vendor-neutral. They are indeed a core ingredient in modernizing network operations.

Equally important, however, is finding the right people–people who are willing and able to blur the line between network engineering and software engineering in how they work and think. You then need to organize in the right way to enable your new capability.

Admittedly, finding people that fit this profile is challenging. While there are some unicorns out there who know networking just as well as they know coding, they are just slightly less rare than actual unicorns. In my experience, many network engineers have acquired skills like scripting and Ansible, and some have made the switch, taking network software engineer roles. But getting network engineers to write software to run the network and scale it in production requires them to climb a steep learning curve. 

That’s why, at Equinix, we’ve found a more realistic approach, which is to build teams that include both network operators and software engineers, and then encourage them to cross pollinate. We’ve found that the skill sets that each engineer possesses at the outset aren’t as important as their philosophical outlook: when they accept that coding and network operation are inextricably linked, they pick up the requisite skills naturally enough.

Alternatively, I have been successful in training up software engineers to acquire the necessary network domain knowledge. But this approach requires a lot of upfront resource and time investment to create a strong, high-performing team.

Going forward, as more and more organizations—not to mention certification providers—recognize the importance of combining software engineering and network engineering skills, I’m optimistic that we’ll see more people who bring both types of expertise to the table from the start. But that’s not how most folks are trained today, and organizations need to adapt accordingly.

This comment by a security researcher on the big 2021 Facebook outage, where software abstractions on top of the network prevented most of the people with the training needed to resolve the issue from accessing the necessary infrastructure brings home the point:

Someone on the Facebook recovery effort has explained that a routine BGP update went wrong, which in turn locked out those with remote access who could reverse the mistake. Those who do have physical access do not have authorization on the servers. Catch-22.

— Steve Gibson (@SGgrc) October 4, 2021

We’ve Tried It, and It’s Great

To drive home the benefits of taking a new approach to network engineering, let me brag a little  about what we’re doing on this front at Equinix.

Previously, our network engineers, like most of their peers at other companies, lived in the world of vendor-specific tooling. This limited not only what our network operators could do, but also what our customers could do with the network resources we provided to them. Seamlessly connecting colocated infrastructure to public clouds wasn’t impossible for customers, but it was tough—because the underlying tooling we were working with was tough.

Those days are now over. Our cross-pollinated teams of network engineers and software engineers are building a new generation of networking services. Leveraging open tooling like OpenConfig, gNMI, and OVS, we are rewriting the network orchestration software for our data centers. We’re creating an architecture that lets operators issue commands across the infrastructure in a centralized way, regardless of the underlying hardware they’re dealing with.

This means that our network operators can work more efficiently, while our customers get features like one-click connectivity with public clouds.

And this is just the beginning. We are building a foundational network orchestration platform that will allow us to continue rolling out new networking features on a regular basis. We look forward to showcasing some of these innovations on our blog.

I’ll conclude by saying that, although there will always be a need for traditional network administrators, the future of network engineering hinges on the ability to align network expertise with software engineering expertise. This means not only bringing the right skill sets and tools to the table, but also making the organizational changes that allow network engineers to think like software engineers and vice versa.

We’ve been doing this for some time at Equinix, and we’re already seeing amazing changes. But we are still in early days, and I am excited to see what comes next!