Live From Super Bowl LIV: Fox Sports’ Kevin Callahan Discusses Rapid Evolution of Super Bowl Compounds
Tech exec says demands on the compound have more than doubled since the net’s previous production
If anyone wants to see just how rapidly the production around a Super Bowl is growing, a visit to the Fox Sports FATS (Feed Another Truck Something) hub that expands every time Fox Sports covers the Big Game will demonstrate it.
As with virtually every sports-related production, the efforts around the Super Bowl continue to grow and, along with it, the demands on the distribution network and router. Rightsholders want more access to signals, studio, and shoulder programming, and there is simply viewer demand for more content around major events like the Super Bowl.
“The shows are more complex, and the distribution requirements far exceeded what we could do with the router units in the mobile units,” says Kevin Callahan, VP, field operations and engineering, Fox Sports. “In 2011, we were able to do a Super Bowl without a FATS, and now the compound’s distribution needs exceed a 576×1152 video router, and we have doubled the size of the router we had three years ago in Houston. The good news is that, by doubling our capacity, we have been able to handle all of the requests that have come in and we haven’t had to say no to anyone.”
The FATS trailer this year was outfitted by Bexel’s ESS team, and Callahan says the Bexel and Fox teams worked closely to ensure all the resources were up to the task. At the core is an Evertz EQX router plus Evertz DA, multiviewers, and AJA FS-HDR converter/frame synchronizer.
“Bexel put together a great system,” says Callahan. “Bryan Kirby and Johnny Pastor got it up and running, and Derek Woodington and Jay Johnson are managing day to day for us.”
One of the unique challenges this year is the split compound: the main game trucks are located next to the stadium, and the remainder are across the street. That required the use of a cable bridge to connect the two compounds.
“Anytime a cable bridge is in play, you have to think about something like a truck that could drive through and take it down,” says Callahan. “So we have two sets of redundant fiber cable bridges to move the signals from the primary compound to the secondary one, but we also have an entire power system and transmission outputs with the main game trucks so that, if anything took down the bridge, the game would stay on-air.”
The FATS cabin does more than just distribute signals to the various entities onsite. It also connects the Hard Rock Stadium compound to the South Beach Studios, the Fox Pico facility in Los Angeles, and the disaster-recovery facility in The Woodlands, TX.
“We’ve also been able to take a lot of extra equipment, in particular robotic cameras and Sony replay servers, and install them in the racks in FATS rather than anywhere we could fit them,” says Callahan. “It makes for a nice controlled environment.”
FATS also plays an important role in the HDR side of the production: it is home to 200 channels of AJA tone-mapping gear that can tone-map SDR content to HDR and also upconvert it to 1080p if needed. Several transmission paths also are tone-mapped downward to SDR for the NFL Network broadcast, international broadcasters, and Van Wagner, which is handling the in-venue show.
The move to 1080p HDR and the use of the XAVC 100 codec had an effect on the EVS replay network. And Sony has stepped in as well, providing X310 monitors to help the mission-critical areas ensure that the HDR color space is correct.
“We have fewer EVS replay servers here than we did at the Super Bowl in 2017,” Callahan notes, “but we have 100 additional channels for a total of 396. And we’ve deployed an XT-VIA platform that is the largest deployment to date.”
The XT-VIA server platform has been crucial, he adds, because the XAVC codec is a resource hog on a CPU.
“The large number of high-frame-rate cameras we have will tax the replay servers, so we decided to use the XT-VIA platform as it has so much more processing power and bandwidth available,” says Callahan. “The network is so much faster, by three times, and it was the only way we could guarantee a flawless show on the EVS side so the operators wouldn’t have to worry about performance issues.”
He notes that power and transmission are always two of the things he worries about most. That is one reason he asked Bart Lucassen, head of power and telecom, HBS, to perform an audit of the system that Fox designed: “It’s always great to have another set of eyes look at these things.”
The transmission side this year has carrier diversity, with both AT&T and CenturyLink providing paths and 1080p HDR circuits to Los Angeles.
“On the power side, CAT Entertainment engineered a great solution for us,” he says. “We have multiple UPS systems running and even running in parallel to eliminate some points of failure. We have multiple technical power systems, each one with three generators running separately and backing up each other. And we have a completely separate domestic power system for non-technical needs.”
Also helping was a revamped Hard Rock Stadium infrastructure.
“With the renovations to be done, the cable infrastructure has helped us out a lot. There are multiple SMPTEs at every location so we had to run very little cable into the bowl,” says Callahan. “And WBL Services, who handles the fiber for the campus, runs fiber out, so it has been easier to get fiber connectivity to tailgate and other areas.”