Super Bowl LIV: Inside the Revolutionary ‘NFL 100 All-Time Team’ Mixed-Reality Pregame Presentation
The Famous Group was enlisted by the league to create a first-of-its-kind MR experience
All season long, the NFL had been teasing the reveal of the NFL 100 All-Time Team on Super Bowl Sunday. When the time finally came to unveil the 100 players and 10 coaches during the Super Bowl LIV pregame festivities, more than 100 million fans watching at home and the thousands at Hard Rock Stadium witnessed a seven-minute, multicamera mixed-reality presentation unlike anything they had seen before. The ambitious and groundbreaking mixed-reality presentation was the culmination of months of work by The Famous Group (TFG), the NFL and NFL Films, and Fox Sports.
— NFL (@NFL) February 2, 2020
“The league came to us with a very unique challenge, and we were extremely grateful for the opportunity,” says The Famous Group EVP Andrew Isaacson. “As premium content creators and storytellers, we see mixed reality as the latest tool in our toolbox to create things that have never been seen before. The NFL 100 [project] took a tremendous amount of technical and creative work, but we could not have been happier with the results.”
Months in the Making: How the Mixed-Reality Show Came Together
The Famous Group is no stranger to the Big Game, having serviced 15 Super Bowls and worked with the league as a creative-services provider regularly over the past two decades. However, the “NFL 100 All-Time Team” mixed-reality presentation proved to be one of the company’s most complex undertakings, requiring more than six months of preparation and a team of more than 40 people primarily based at TFG’s Culver City, CA, headquarters.
Last spring, the NFL invited the company to present at the league meetings in Phoenix on mixed reality, augmented reality, and the difference between the two. Isaacson detailed how AR simply overlays virtual objects on a real-world environment, while MR tracks computer-generated digital objects in a physical environment through a live camera feed, placing digital objects in a physical space with accurate camera tracking and matching the physical lighting/shadows. NFL Director, Event Presentation and Content, Tim Tubito took a particular interest in the seminar and followed up with Isaacson over the summer.
“Tim said the league was going to be doing this All-Time Team ceremony and that he would love to have us involved in coming up with ideas for how to pull it off,” says Isaacson. “We both believed this would be a great way to potentially bring mixed reality to the table.”
From a logistics perspective, the league and The Famous Group had to confront three primary challenges. First, how to present 110 members in just seven minutes since it was physically impossible to announce every name one-by-one in that time. Second, how to appropriately honor players/coaches who could not attend the ceremony or were deceased. And third, how to avoid elaborate physical sets/props to minimize the impact on the grass field and not interfere with all the rehearsals taking place at the stadium in the days leading up to the Super Bowl.
“Mixed reality ended up being a solution for all three of those challenges,” notes Isaacson, “and also provided a way to accomplish the most important part: telling a story about those 110 players and coaches.”
TFG also got a boost when it worked with the Baltimore Ravens to debut a small dose of its MR content at M&T Bank Stadium during a Thursday Night Football matchup in December. During that presentation, a giant, CGI raven swooped down into the stadium before kickoff and then appeared on the stadium’s videoboards at various times throughout the game.
“The Ravens project was very different from a creative standpoint, but it absolutely helped us build this [MR] pipeline,” says Isaacson. “[Ravens VP, Broadcasting,] Jay O’Brien and his team gave us carte blanche there in the control room and really allowed us to do a test run for what became the product you saw for the NFL 100 All-Time Team presentation.”
Inside the Operation: Tech Partners and NFL Films Help Pull It Off
Once the project was greenlighted, the first step was to capture a full 3D scan of Hard Rock Stadium, which was completed in December in partnership with True Point Laser Scanning. TFG also created a full 3D animatic model showing all the virtual elements and how they would interact with the real world, as well as specifying the camera angles and cuts between them.
“A big part of [the operation] was planning it all out in 3D and dialing in that 3D animatic over the course of the production,” says TFG Executive Creative Director and Partner Greg Harvey. “That served as our 3D visual storyboard and our guiding compass to put it all together. The show you saw on TV was about 90% the same as the final animatic that we put together. It laid out the camera cutting between the wide shot, close shots, and the mixed reality. It’s basically a guidepost for how the mixed-reality elements would be placed into the [real-world] environment, so it all has to be choreographed perfectly.”
The 3D scan of the stadium along with 3D animations created in Maxon’s Cinema4D software were migrated into TFG’s virtual-production environment powered by Epic Games’ Unreal Engine. The virtual 3D scenes were created in Unreal and brought into The Future Group’s Pixotope cross-reality production system, where the final MR integration was completed.
TFG also worked with longtime projection-mapping partner Quince Imaging, which built out the hardware system for the MR presentation. Quince Imaging Creative Director CJ Davis and the team built out a six-server system to drive the four broadcast cameras (including the Skycam) and two backup cameras that were enhanced with MR capability for the pregame presentation.
NFL Films wrote the script and constructed the edits for the presentation and was able to secure longtime Hard Knocks narrator Liev Schreiber to complete the voiceover. “NFL Films was instrumental in taking our concept and breathing life into it from a storytelling standpoint,” says Isaacson. “And the biggest key of all was Tim Tubito and his vision and his trust in us to pull this off on the biggest stage imaginable.”
740 Sound also completed the sound design for the project, layering in sound effects for the opening and closing animations of the presentation.
It’s Showtime: How It Played Out on Super Bowl Sunday
In the weeks leading up to the Super Bowl, TFG sent a team of more than a dozen to Hard Rock Stadium to complete final integration and calibration work. The company erected a production studio onsite with eight workstations: one for each camera (including the two backup cameras) and two producer stations.
“All that final integration and calibration work has to be done onsite,” Harvey explains. “The 3D model has to be calibrated correctly so it’s an exact replica [of the stadium]. Things like the glowing goalposts, all of the shadows, and the lighting: that all had to be done onsite, and it’s very difficult technical work to do. The team in Miami was going back and forth with our team here [in Culver City, CA] making adjustments. It was a very collaborative and very long process between us and The Future Group, but we were able to get it done.”
TFG also worked extremely closely with Fox Sports throughout Super Bowl Week to ensure that the MR experience was properly integrated into the broadcaster’s operation. In the week leading up to the game, the two groups conducted several rehearsals, including a final run-through featuring local high school football teams as stand-ins for the NFL 100 players honored. Fox Sports’ production team handled the live production of the seven-minute, multicamera presentation from inside the truck.
“Fox Sports was absolutely tremendous to work with, especially considering everything they had going on,” says Isaacson. “It’s one of those technologies where you can’t actually fully realize the end product unless you’re looking through the lens of the camera. So it was critical for us to get rehearsals where we had all four camera operators available. And it went right down to the last day: when we were doing rehearsals with stand-ins on Sunday afternoon, that was truly the first time anybody had seen the finished product.”
When the moment of truth finally arrived during the pregame festivities at Super Bowl LIV, viewers watching at home and on the videoboards inside the stadium were privy to the most ambitious use of mixed reality for a sports event to date. The field and goalposts lit up with glowing graphics, virtual video screens played out highlights of the NFL 100 greats, and virtual player-card graphics appeared as the team members were announced.
On the Horizon: A Bright Future for Mixed Reality
With Super Bowl LIV now in the books, the question becomes, What’s next for MR technology in the live sports production? The Famous Group sees the monumental effort in Miami as just the beginning of what is possible for live-sports experiences.
“We see this as a huge part of the live-event business moving forward,” says Harvey. “We have a lot of interest already — whether it is permanent installations of this type of technology or just one-off. We’re really just on the tip of the iceberg in terms of what can be done with this technology and how it can be used for entertainment pieces, sponsorship, and interactive experiences that were never possible before. I think the Super Bowl, because it’s so high-profile, really went a long way in helping establish what is possible for mixed reality.”