Crafting AR Innovation at the Super Bowl with Silver Spoon and Stage Precision

As the world’s eyes converge on the annual sporting and entertainment spectacular that is the Super Bowl, the pressure to deliver a flawless broadcast is undoubtedly sky-high. For Silver Spoon, a real-time graphics and animation company, the 2024 Super Bowl live broadcasts on CBS and Nickelodeon presented an unprecedented opportunity: to redefine the boundaries of what is possible in the realm of augmented reality and live broadcast. Central to this endeavour was Stage Precision, whose cutting-edge SP tracking and control solutions played a pivotal role in orchestrating several broadcast firsts during this year’s show.

“The scale and complexity of the Super Bowl demanded a bullet-proof solution that could seamlessly integrate across multiple broadcasts and platforms,” recounts Matthew Houstle, Director of Technology at Silver Spoon. “SP’s tracking solutions emerged as the linchpin of our operations, providing us with the precision and reliability we needed to execute our vision.”

Part of the vision was to bring the Super Bowl to a younger audience through the first-ever simulcast of the event on broadcast channel, Nickelodeon. The simulcast would recreate the action of the Super Bowl game in an alternate reality from the SpongeBob SquarePants world of Bikini Bottom, with over three hours of live AR coverage. Alongside the Nickelodeon broadcast, Silver Spoon handled all AR content and operations for both the CBS Studio shows at the Bellagio and the game broadcast on CBS, utilising a total of 36 Unreal Engine servers and a 32-person on-site crew. This included AR-enhanced touchdown celebrations and mixed reality scorecard signage across several Las Vegas landmarks.

“We had 15 tracked cameras across three shows,” elaborates Houstle. “Bringing SP into the fold was essential for streamlining our operations and ensuring seamless integration across all platforms.” Key to the management of such a complex project was the development of a unified backend system – a single server infrastructure that could manage the diverse array of cameras and broadcasts. “SP allowed us to consolidate our infrastructure into a single server, reducing backups across all shows and unifying our engineering team,” Houstle explains. “Their software not only facilitated camera tracking but also managed our server failover seamlessly.”

Axel Lambrecht, Co-Founder and Solutions Director at Stage Precision worked closely with the Silver Spoon team to realise their vision. “Axel and his team went above and beyond our expectations,” continues Houstle. “Their expertise and dedication were instrumental in building out an incredible system that was customised to be as comprehensive as possible.”

Running all camera tracking data through the unified backend of SP allowed Silver Spoon to craft an innovative server failover system, that not only gave greater visibility of the whole workflow, but also saved on costs. “Without SP, every render engine for each main camera required a backup server, and this gets expensive on a show of this scale,” explains Houstle. “We wanted to avoid this extra cost, so we built the system with a 50% backup idea going into the project and SP allowed us to set up a camera switching logic so that with a single button press the camera and server could be switched over instantly in case of failure.”

The calibration of cameras presented a unique challenge, particularly for hero shots of the stadium. SP’s camera calibration features and tools streamlined this process, allowing for quick and accurate calibration. The integration of numerous custom APIs ensured Silver Spoon could use their entire toolkit of technologies, yet have control, visibility and failover within a single source of truth in SP.

In the case of the Super Bowl, SPnet, an openly available protocol that allows real-time object-based data streams to connect seamlessly in SP, was adopted by Pixotope facilitating the control of multiple instances in only a few clicks.

“SP’s technology provided us with peace of mind, knowing that we could deliver a flawless broadcast to the largest audience in the world,” confirms Dan Pack, Managing Director at Silver Spoon. “Products like those from Stage Precision have levelled up the operations in our industry giving us complete confidence to deliver highly conceptual executions live on the biggest stages.”

The visibility and control made possible with SP have been another key to the successes at the Super Bowl. “With SP we have full visibility over diagnostics across the entire production and can identify problems quickly and solve them straight away. If something pops up, I want to be able to tell clients I know where the problem is and how long it will take us to fix it. With SP, I can do this.”