Super Bowl LI Preview: Inside Fox Sports’ ‘Be The Player’ First-Person POV Replay Tech

Deployment at NRG Stadium will mark Intel system’s first use for replays from player perspective

Ever wonder what it’s like to stand in the pocket trying to find an open receiver as hulking NFL linebackers race toward you ready to annihilate you? Well, Fox Sports is looking to provide viewers with a first-hand view of just that during its Super Bowl LI coverage next month in Houston.

As previously reported by SVG, Fox Sports plans to use Intel 360 Replay technology to introduce Be The Player, a replay feature that displays a first-person POV perspective for any player on the field without a physical camera attached to the player. Developed by Intel and Fox Sports Labs, the first-of-its kind replay technology will provide viewers a unique field-of-vision perspective of the play from the player’s point of view.

Be The Player will provide viewers a field-of-vision perspective from the player’s POV during the play.

CLICK HERE for an in-game demo of the technology.

Be The Player will leverage the Intel 360 Replay system (branded freeD prior to Intel’s acquisition of tech upstart Replay Technologies last year) that has been deployed at a variety of major sports events: U.S. Open golf; MLB All-Star Game; NBA All-Star Game, Conference Finals, and Final; Super Bowl 50; the Rio Olympics; and primetime NFL games. However, the system has previously been used to create wide shots of the action; Super Bowl LI will mark the first time Intel has created first-person POV replays.

The system will deploy an array of 38 5K cameras throughout NRG Stadium, along with a bevy of Intel servers in a control room inside the stadium to create a 3D space in which replays can spin around the field and zoom in or out of key action during the play.

“On any sports broadcast, we try to provide the best viewing angles to analyze the game and tell the story in the right way. What this does is give us a vantage point that viewers have never seen before and, quite frankly, only a handful of people — solely NFL players — ever have,” says Zac Fields, SVP, graphic technology and innovation, Fox Sports. “It allows us to take a look back at a touchdown and show, from the quarterback’s perspective, how he read the defensive back and made the decision to throw the ball where he did. Obviously, we are going to have many other incredible angles and hyper-motion cameras, but, with this, we are now able to show [viewers] something they have truly never seen before.”

Months of Development Set To Pay Off
Be The Player has been in development for several months, with Fox Labs and Intel conducting three offline tests at Levi’s Stadium in Santa Clara, CA (where the Intel 360 system was already installed for San Francisco 49ers games), culminating with the Foster Farms Bowl on Dec. 28.

Intel will deploy an army of servers at NRG Stadium, similar to the rack pictured here at Petco Park during the MLB All-Star Game.

Fox previously worked with the Intel 360 system at the MLB All-Star Game in San Diego last summer (Fox RSN YES Network also previously deployed the system at Yankee Stadium) and was looking to kick it up a notch for Super Bowl LI.

“When we first started having conversations with Intel around implementing freeD replay for the Super Bowl, we wanted to find ways to take the system to the next level. Understanding the promise of the technology, which is the ability to put a camera anywhere, we proposed the idea of showing fans the point of view from any player on the field — in particular, the quarterback. Intel responded that, with a bit more development, they could have a version [of the system] in time for Super Bowl with the ability to turn these clips around quickly. Each time we tested it, the quality and speed at which clips can be turned around increased.”

The Workflow
Each camera will be connected via fiber (more than 5 miles of fiber in all) to a dedicated server in Intel’s production room at NRG Stadium, which will house half a dozen Intel staffers and a dedicated Fox Sports producer. Simultaneous camera data are processed together, composing every point in the stadium into individual volumetric pixels (or “voxels”), which are used to build a true three-dimensional reconstruction of the scene.

In addition to the individual camera servers, “management servers” provide the computer power to build the replay packages, which display a 3D reconstruction of the play and can be paused, zoomed in or out, and viewed from anywhere on the field.

The Intel 360 pilot (right) controls the cameras and selects angles and moments to zoom and pause, while the navigator monitors camera operation (pictured here at MLB All-Star Game at Petco Park).

For Super Bowl LI, two operators will be located inside the Intel replay-control room: the pilot, who creates the camera angles and selects individual angles and moments to zoom or pause, and the navigator, who monitors the system to ensure that all the cameras are shaded and working correctly. Fox will dedicate a producer on the Intel team and the production team in Fox Sports’ Encore mobile unit to choose clips that will be integrated into the telecast.

“Once the system is triggered, they will decide who the player of interest will be — whether it’s a quarterback or maybe linebacker — and determine which camera zones will provide the best viewing angle,” says Fields. “Then they decide what the person of interest should look at: is it a single spot on the field or panning around to two different spots? Intel then renders it down and, once it’s ready to go, pushes the clip to our EVS network for playout so we can add any telestration to help tell the story. From there, it’s just like a normal replay for the producer and director.”

In addition to the Be The Player feature, Fox Sports will also be able to create traditional wide-shot Intel 360 Replay clips to integrate into the Super Bowl LI telecast.

Cutting Down the Turnaround Time
According to Fields, turnaround time for Be The Player clips had been reduced to just two minutes during the final test at the Foster Farms Bowl last month.

“One of the most important factors is giving production a tool that tells relevant stories in a timely manner. You may be able to create a very cool piece of video, but, if you can’t turn it around in a timely fashion, then it’s irrelevant to the broadcast,” he explains. “While it’s not ready for instant replay yet, [two minutes] is still a very comfortable timeline for production. So, for example, after a touchdown, we can turn it around coming back from the commercial break.”

Although Fields declines to pinpoint a precise number, he does venture a guess that the system will make its way into Fox’s Super Bowl LI telecast two to five times plus several more occasions during the halftime and postgame shows.

“It will come down to when the system is relevant and when it is able to further the story,” he says. “We are very excited, but we don’t want to use it just to use it. As the game unfolds, I’m sure there will be opportunities to use it.”