Sports Graphics Forum: AR Enhances Virtually Every Aspect of a Production

Panelists from ESPN, BBC Sport, The Famous Group, MLB Network/NHL Network, Univision Deportes discuss their respective efforts

Coupled with player tracking and data, augmented reality (AR) is opening up new ways of storytelling for sports-TV production teams. It is also allowing studios to take advantage of advanced green-screen technology and virtual sets in a way that enhances nearly every aspect of a production, from pregame to postgame and beyond. The opportunities and challenges were part of the Augmenting Your Reality panel discussion at SVG’s annual Sports Graphics Forum, held in New York City last week.

Charlie Collin, senior manager, motion graphics and production enhancements, ESPN Creative Services, said that ESPN has spent the past few years focusing on how to use virtual reality and AR in a way that is more sustainable and can have a presence beyond one big game. College GameDay, for example, relies on AR week in and week out, and the efforts require more than just the buy-in of the graphics and camera department. On-air talent also has to be on board.

The Augmenting Your Reality panel at 2020 Sports Graphics Forum: (from left) SVG’s Kristian Hernandez, MLB Network/NHL Network’s Patrick Armstrong, ESPN Creative Services’ Charlie Collin, The Famous Group’s Andrew Isaacson, BBC Sport’s John Murphy, and Univision Deportes’ Alexis Salinas

“It’s important to remember that everybody is involved,” he added. “It’s all about the talent and them buying into it.”

John Murphy, creative director, motion, BBC Sport, concurred that getting on-air talent to buy in is important: because a green-screen environment doesn’t offer the typical physical set, talent needs to spend a bit more time rehearsing. BBC coverage of the Match of the Day for the EPL, he said, takes advantage of Unreal Engine to deliver photorealistic looks and, when combined with keying technology that has improved by leaps and bounds, allows four shows shot in the same stadium to have truly individual looks.

“Creatively,” he noted, “we are not where we want to be and are just scratching the surface, but we have a big summer coming up with the Euros and Tokyo Olympics.”

Even if AR is used for the day-to-day, it is still needed in the really big shows, and ESPN worked hard on an AR moment for the opening pre-tease of the College Football National Championship game at Mercedes-Benz Superdome in New Orleans in January.

“Our moment of glory stayed on-air for 30 seconds,” he said. “We were proud of that, and we kept selling the AR graphics for the rest of the game. We achieved our goals, but there is still room for improvement, like integrating further with the stadium feeds so we can have a full stadium takeover.”

Arguably the best example of how AR and mixed reality can take over a stadium was at Super Bowl LIV when the NFL celebrated the 100th anniversary with an on-field tribute to the 100 greatest players and 10 greatest coaches. Andrew Isaacson, EVP, The Famous Group, discussed the project, which posed three challenges: 110 names had to be mentioned in seven minutes; a third of those were deceased so needed to be referenced in a respectful way; and the rehearsal schedule for the on-field and real-world participation of former players and coaches was tight.

“We started with a 3D scan of [Miami’s Hard Rock Stadium],” he explained. “That was a digital blueprint for the software so we would know where the field, the concourses, and the videoboards are, as well as the field goals. And we had to do the rehearsal at 6:04 p.m. [the day before the game] to make sure the lighting worked for when the physical and virtual objects would come together.”

Isaacson also discussed a project done for a Ravens game where a virtual raven would fly around the stadium. That meant getting the camera operator to become part of the live theater and sell the shot as if the raven were unexpected.

“We told him he had to make it look like he was surprised, with a big whip pan,” he said. “The whole production team has to be educated to see how it comes together.”

AR as the ultimate analysis tool is also an evolving project, and Patrick Armstrong, supervising producer, production technologies, MLB Network/NHL Network, discussed how the networks use Ross Video’s Piero Sports Graphics Analysis Tool to overlay graphics and have them shown from one camera shot to the next. It’s a process that involves helping the analysts understand how the graphics will help tell the story and, occasionally, working with the vendor support team on technical efforts that can push the envelope.

“Piero and Ross have great customer support in place,” he noted. “They can get on the phone and solve any problems we have in a timely manner. The people on the other end of the line have used the software and can solve problems in a pinch.”

Isaacson added that backup servers and cameras are important additions to a live AR production. “The goal is to run a clean show. Having enough backups and a backup plan was our secret.”

Alexis Salinas, senior director, technical creative services, Univision Deportes, agreed, pointing out that, with a physical studio, an equipment failure is much more manageable. He discussed how the recently rebranded TUDN network had production teams in Mexico and the U.S. last year working more efficiently by sharing graphics packages.

“The new look is very 2D, so a challenge is how to put 3D with a 2D look,” he added. “But we have a common strategy and implementation with Unreal Engine across multiple territories.”

Much of TUDN’s sports-production efforts are around soccer, and Salinas said the lack of breaks in the action makes it difficult to trigger AR elements: “Our live-events team doesn’t want to use it too often during the game. You can put it up and miss a goal or other action.”