How Zoom, Netflix, and Dropbox are Staying Online During the Pandemic

Zoom Scales Up

“Last couple weeks it’s been all hands on deck,”  Alex Guerrero, senior manager of SaaS operations at Zoom, said. To date, Guerrero’s team at Zoom has been focusing primarily on scaling up bandwidth in various places on its network. That’s meant peering with more carriers and ISPs, ordering more transit, and increasing bandwidth on existing interconnections, with a particular focus on doing more peering closer to end users.

“That’s mainly what I’m looking at: bandwidth and being as close to the customer as possible,” Guerrero said. “Our product can handle a lot of latency, but still, the closer you are to the eyeballs the better performance you’re going to get across the board.”

Highlighted Zoom actions:

• Maintaining the cushion of 50% more capacity than its maximum actual usage
• Boost network’s bandwidth through Equinix’s Cloud Exchange Fabric, the software-defined network interconnection platform
• Solving some challenges quickly scaling compute in its own data centers, due to the lockdown-related “supply chain issues” (scaling compute in the cloud hasn’t been a problem)
• Expanding local exchanges in the market it’s in: Zoom today is in 19 data centers around the world, and each facility is connected to the biggest exchange in the markets. Now, however, its network engineers are looking at second-biggest and in some cases third-biggest exchanges in those markets to bring its network closer to more end users.

As usage goes up, the platform is designed to scale both network and compute automatically, “with very little human intervention,”. Other than having to scale “a lot faster” than anticipated, “everything is kind of in our standard operating procedure,” he said.

Netflix Is Careful Not to Scramble

Overall, Dave Temkin’s (VP of network and systems infrastructure at Netflix) philosophy has been to avoid scrambling for resources to ensure other, more essential services can get them if they need – services like healthcare, e-learning, and video conferencing. Much like US officials have been pleading with the public to avoid hoarding surgical masks because hospital workers badly need them, Netflix doesn’t want to hoard servers and hog network capacity because it’s more important for a doctor to be able to see her patient remotely than for you to be able re-watch Breaking Bad in glorious 4K.

• Like Zoom, Netflix had no problems with scaling cloud capacity, but also did hit a snag last week when trying to get more servers into the ISP locations to increase the capacity of its Content Delivery Network. “We have had multiple fires at this point with our supply chain,” Dave said during the webinar.
• Switching to different server manufacturer location: Netflix’s primary server manufacturer is in Santa Clara, California, and earlier this month, when six Bay Area counties including Santa Clara issued a shelter-in-place order, Temkin’s team had 24 hours “to get as many boxes out of there as we could.” Those issues have since been resolved by switching to a different manufacturing location, he said.
• Scaling up the infrastructure that delivers content to users: While Netflix runs mostly on AWS, its platform is also a hybrid, because it operates its own content delivery network. Otherwise, the part of Netflix’s infrastructure that delivers content to users has been scaling up as designed. Temkin’s team has effectively “pulled forward” its growth plans for the coming holiday season, he said. “We don’t feel like we’re stressing our cloud infrastructure by the current events.”
• Things are different for the part of the company’s infrastructure that’s used to make content. “Right now (it’s not unique to us) most content production is shut down around the globe,” he said. 

“The internet itself seems to be scaling pretty well,” Temkin said. While there has been some strain – in some cases on interconnects, in others on last-mile networks – “generally, nothing is absolutely melting down.”

Dropbox Is Seeking Peers

Unlike Zoom and Netflix, Dropbox runs mostly in of its own data centers. The company moved its platform from AWS to its own computing facilities in 2015. However, the company continues to rely on AWS for unanticipated bursts in capacity and for some technological capabilities it wouldn’t make sense for Dropbox to build in-house.

The value of hybrid cloud platform is “you can always utilize public cloud capabilities and public cloud scale,” Dzmitry Markovich, senior director of engineering at Dropbox, said.

• Like Zoom and Netflix, the cloud storage and collaboration company’s platform has successfully relied on automation to scale along with the recent surge in demand. But there have been some operational challenges with “restricted access” by some vendors around the world, Markovich said. He didn’t specify what those challenges were, but many data center providers have reduced customer and vendor foot traffic in their facilities to prevent transmission of the coronavirus by allowing access only when it’s absolutely necessary and by rigorously screening visitors.
• Another challenge for Dropbox has been the shift of internet traffic from being highly concentrated in big hubs to a more distributed pattern, he said. Instead of having a lot of traffic coming from a thousand accounts in a university, for example, Dropbox is now seeing all those accounts access its platform from many different places, through many different networks. To address this, Markovich’s team has been analyzing its last-mile connectivity strategy and actively looking for more last-mile ISPs to peer with. Dropbox already peers “heavily,” but it’s now investing in even more peering relationships.

Read the full article here on Data Center Knowledge
Watch the Panel Discussion here on Kentik