Live From Rio 2016: NBC Olympics’ Content-Management Backbone Is Churning at Historic Rate

Highlights Factory offers “six to eight times the delivery of what [it] did in London”

For NBC Olympics, these Games are about far more than just the operations in Rio de Janeiro. Thousands of loggers, editors, producers, and more in numerous geographic locations across two continents are part of the Olympic Games content ecosystem on the NBC family of networks.

From left: Senior Digital Media Manager Matthew Green; VP, Post Operations and Digital Workflow Darryl Jefferson; and Engineer, Digital Workflows Kamal Bhangle have played key roles in NBC Olympics’ content-management infrastructure for the Rio Games.

A strong content-management infrastructure is essential to making all of that work, and this year’s system is accomplishing things never before seen in a system of its kind at an event of this nature. According to Darryl Jefferson, VP, post operations and digital workflow, NBC Olympics, this content-management system is handling “six to eight times” the delivery provided by the Highlights Factory system deployed for the 2012 London Olympics.

The Highlights Factory is the backbone of NBC Olympics’ success, supplying all the media assets required for the main primetime show, the daytime television product, various VOD offerings, digital-content destinations, and much more.

In Rio, the team has erected an eight-crate Avid ISIS for the Avid editing environment, two arms of Harmonic MediaGrid for the central SAN, an EMC Isilon storage array, an Aberdeen storage array, and a collection of EVS machines. All in all, NBC Olympics is prepared to handle and store a staggering 820 TB of main record capacity for the primary ingest wall.

Who will work on the content? Up to 180 users at the IBC in Rio, 35 at each of the four designated A venues (Maracanã Stadium, Olympic Stadium, gymnastics, and swimming); 170+ in Stamford, CT; 100 more in New York City; more than 50 in a facility in Orlando; another 40 in Hialeah, FL, and a handful in Colorado. All log into the asset-management system to do something with content during the Games.

One of the primary reasons for the dramatically increased system requirements is the vast array of viewing options that NBC Olympics offers viewers in the U.S., meaning that content developers are building assets for a seemingly endless list of viewing destinations. Each Olympics seems to bring new viewer behavior. At the 2010 Vancouver Olympics, the tablet didn’t exist. At the 2010 London Games, connected TVs weren’t yet a reality, and 4K was still very much in its infancy. These Games are introducing many more social-media–driven consumption points, such as Facebook Live and SnapChat.

Central ingest at NBC Olympics home in the IBC

“The world is changing,” says Jefferson. “We have gotten older since London, but also the appetite and internal needs and desires within the company for more stuff — tonnage, streaming hours, 360 video, VR, 4K (whether HDR or SDR) — [go] on and on. There’s all these layers that are beyond what we could have imagined when we were in London.”

The numbers are staggering but don’t begin to tell the tale. NBC is boasting 7,000 hours of content made available to U.S. viewers — a tremendous amount of raw tonnage — but, for the behind-the-scenes team, that doesn’t count such assets as ENG footage, 4K footage, etc.

“[This] probably is the new normal,” Jefferson opines, “and, honestly, if young people are going someplace other than the primetime TV show, we have to bring it to where they watch. That’s the goal.”

Prepping this content-management system has been a multi-year process dating to the 2012 London Olympics. That team, under Jefferson’s direction, was looking to make a massive transition to tying operations more seamlessly into NBC Sports Group’s new Stamford, CT, facility; previous methods had required designing a massive system and network that would be built and operate for two weeks of competition and then torn down. The team wanted to leverage what NBC Sports was already doing year-round with its existing systems, add some essential bells and whistles, and scale up from there.

To do that, the content-management group has essentially been running a mad scientists’ lab in “The Basement,” a sub-floor beneath the Stamford facility. Many of the systems in place in Rio were prefabbed, preconfigured, and tested there. Then the whole system was shipped to Rio essentially as one piece. It’s a method that NBC had never used before.

“We had all of the networking gear available and tied it into the running systems upstairs, such that we were able to test how this environment would react,” says Matthew Green, senior digital media engineer, NBC Olympics. “[It wasn’t] to full scale, but it was as close as reasonably possible. As a consequence to that, the baseline-type stuff we knew  was going to work and how everything was going to interact before we got down here.”

NBC Sports’ content-management infrastructure features many systems similar to those at the Stamford facility: EVS for playback; Avid for edit; After Effects, ChyronHego, and Cinema4D for graphics; and Pro Tools, InterplayMAM, and Central UX for asset management.

In The Basement, Jefferson and his team ran the system through numerous tests, including adding artificial latency, modeling high-stress environments, and simulating disaster scenarios. All had the system adequately prepared, but Green acknowledges that preparation can get you only so far.

“Simulations are one thing” says Green, “but putting actual people on actual systems distributed across two continents and five different facilities is what actually tells you if it is going to work.”

The system faced some early difficulties but was able to find its sea legs in the first few days of the Games. Jefferson says that everyone across the asset chain has performed admirably and that things are going very well.

“One of the best things about this industry is, in live television, you really get to see what people are made of,” he points out. “What is your team capable of, and what can they face down? I’m so deeply impressed with the people I work with. They are able to stare down a seemingly unwinnable situation every day and make it through that.”