Automation Helps Network Providers Weather Coronavirus Disruptions

Zoom’s videoconferencing platform has exploded in popularity as stay-at-home mandates have swept the globe, and some of the credit for being able to keep up with demand goes to automation.

“We have automation in place so that we can quickly scale our infrastructure – the network as well as the compute infrastructure – with very little human intervention,” said Alex Guerrero, senior manager of SaaS operations at Zoom.

That has translated into being able to maintain service levels at a time when traffic levels are in flux and physical access to network infrastructure is constrained.

Guerrero took part in a virtual panel last week designed to provide an inside look at network operations at Zoom, Netflix, Dropbox and Equinix during the pandemic.

“We have automation in place so that we can quickly scale our infrastructure – the network as well as the compute infrastructure – with very little human intervention,” Guerrero said. [already used this part of the quote]

Industry-wide, network programming skills have become more important as companies navigate the shift to software-defined networking and embrace automation to combat the complexity of IT infrastructure management.

Software fluency, important to administering automation, is a required skill for Zoom’s network team, which uses programming tools including Ansible and Puppet. Senior network engineers at Zoom need to “know more than Perl today. They need to be able to script in some other language,” Guerrero said.

Zoom isn’t alone in relying on automation to quickly scale networks and infrastructure. Data-center provider Equinix has seen traffic levels rise between 10% and 40% over the last few months, and its customers have been accelerating their plans to scale network capacity. Having the physical capacity, along with automation capabilities, has made it possible for Equinix to absorb the uptick in volume, said Bill Long, senior vice president of core product management at Equinix.

At Dropbox, network teams are also relying on automation capabilities to keep pace with increasing demand for the company’s cloud storage platform. “[Automation] is one of the main things that we’re doing with networking today,” said Dzmitry Markovich, senior director of engineering at Dropbox. “We rarely touch those devices manually.”

Panelists on the webcast also talked about how they’re supporting their own corporate IT staff and adjusting to new patterns of work.

“We are operating in a very uncertain time,” said Dave Temkin, senior vice president of network and systems infrastructure at Netflix. “One of the things that I’ve told my team, frankly, is that I don’t look at this as a race. I don’t even look at this as a power walk. We really don’t know how long we’re going to be in this mode for. And so I want to ensure that our customers have a great quality of service, but I also want to make sure that my teams feel supported.”

Going from working in an office to working from home, without the usual social interactions, can be a painful transition that requires support, Markovich said. “I’ve seen it in our company. I’ve seen it in different companies. I see how people struggle. So that’s important to keep in mind.”

Dropbox is taking the opportunity to learn best practices from its employees who are used to working remotely. Keeping track of productivity is also important. “We track our productivity every single week, because we make different decisions within the company on what the next week will look like,” Markovich said. “When everyone is working from home, it’s easy to lose momentum… You need to have a clear metric to understand.”

At Equinix, network teams are settling into remote work. “We have network operation centers around the world. Those are all now work-from-home. And it’s going great,” Long said.

While the pandemic has caused companies to quickly shift IT operations and investments, it’s not clear which changes will remain once travel restrictions are lifted. “How much of these shifts that we’re seeing now, that actually work pretty well – how many of those are going to stick and become the new normal?” Long said. “And how many are we going to revert to old ways of doing things?”

It’s too early to know, obviously. IT hiring – which is challenging even under normal conditions – is another unknown. “How do we even recruit? Hire? Onboard? Train?” said Kentik CEO Avi Freedman, who moderated the panelists. “There’s a lot we don’t know about how long this is going to last, which creates stress.”

One thing that seems clear is demand for software-fluent network engineers is unlikely to abate.

“When we hire network engineers and network automation people, we require everyone to be able to code,” Dropbox’s Markovich said. Five years from now, “there will be no job” for network engineers who haven’t learned network automation skills, he said.