Live From Super Bowl LI: Intel’s Be The Player Is Set To Transform Analysis

The system requires 36 5K cameras strategically placed around the stadium

Intel has installed 38 5K cameras around NRG Stadium to capture images for the Intel Be The Player replay system.

One of the more anticipated advances for Super Bowl LI is the Be The Player technology that Fox Sports plans on using to give analyst Troy Aikman an option no one has ever had before: to see the field through the eyes of a quarterback on the field. Intel’s freeD technology was used at Super Bowl 50 by CBS Sports, but advances in technology in the past 12 months have taken the system to a whole new level.

“We had to redesign from the ground up the render algorithms to increase the render resolution,” says Diego Prilusky, director, product and creative, Intel Sports Group. “We doubled the amount of zoom just by changing the software architecture and having a big leap in software optimization on the hardware.”

The system requires 36 5K cameras to be placed strategically around the stadium to cover the action from all 360 degrees. Two additional cameras are on hand to capture background imaging. The advances in the past year have doubled the resolution and cut the processing time by 50%, and 36 computer processing servers are on hand to bring all the image data together.

The Replay Technologies production team at Super Bowl LI: (from left) front row: Amitai Sharon, Beni Menahem, Ilya Grinshpoun; second row: Diego Prilusky, Christi Stahl, Renan Schilman; third row: Tal Gamberg, Scott Katz, Vitto Zaidman; back row: John Battaglia, Yizhar Feldman

Scott Katz, producer, Be The Player, Fox, will be working with a small team of eight from Intel to create the magic. He will be watching the game and, if a play of interest occurs, will have only seconds to decide whether to start building a clip as the system overwrites itself. Once the decision is made to create a replay, he notifies the pilot, who operates as the editor and helps identify the points of interest in the frame.

For example, Katz will look at a wide image of the field that includes all the players. He can tap the quarterback (or any player) to identify that the replay will be seen from that perspective. He then taps on the wide receivers to set those players as the place to start and end the POV replay. The system then takes about 60 seconds to render the replay.

What the viewer sees is based on the player chosen for the POV and the start and end points. In the case of the quarterback, the replay would begin with a sweeping camera move from behind him down to his viewpoint, and then a still image would freeze the players. The viewer would see what the quarterback would see as he looked at each receiver. The replay ends with the virtual camera pulling back and a video of the rest of the play rolling.

The pilot of Be The Player replay system is essentially the editor of the replays, which will give viewers a POV from anywhere on the field.

The interesting thing about the system is that the POV perspective can be from any player and even a referee.

“There are no real limitations for where you start, end, and where you want to go,” says Prilusky. “We have been playing around with some other things.”

One of those things is showing what the referee sees on a play. Says Katz, “We did a ‘Be the Ref,’ and you can see what the refs are seeing, even something like a hold.”

The production team also includes a navigator, who plays the role of the engineer, another half dozen staffers who work in image processing, and an EVS operator in the production truck, who makes the clips available to the front-bench production team.

Given that Aikman will be analyzing the Be The Player replays, odds are that most of them will be from the QB perspective: the former Cowboys quarterback will easily be able to offer his thoughts on what Tom Brady or Matt Ryan is seeing. The challenge for Katz will be to ensure that a replay of the quarterback’s progression from one side of the field to the other goes from right to left or left to right.

Each camera used for the Intel Be The Player replay system has a dedicated server to process image data.

“It will be about two minutes from the time we grab the play [until] it makes it into the EVS server. So we aren’t going to try and get it in the first replay sequence,” says Katz. “But we can turn it around and have it ready after the next play.”

Getting to this point involved practicing with the system at Levi’s Stadium in San Jose, CA, (home of the San Francisco 49ers) and M&T Bank Stadium in Baltimore (home of the Ravens). The Foster Farms Bowl served as the first true test of the system deployed for Super Bowl LI.

According to Prilusky, the company is already looking at applications for other sports, including basketball and soccer. The key to using the system is ensuring that there is a place to hang the 5K cameras around the field of play.

“This is where we always aimed to be,” says Prilusky. “With the support from Intel, we were able to take the hardware and software implementation to the next level.”