Live From Super Bowl LI: Inside SMT’s Development of the 1st & Ten Line for Skycam

The technology can be used for all sports, ad content, and more

SMT is bringing a Super Bowl first to tomorrow’s big game: it will be the first Super Bowl to place the virtual 1st & Ten line within images captured from the Skycam, a major technical accomplishment that creates new uses for augmented-reality graphics and more.

SMT’s Gerard Hall (left) and Patricia Hopkins (background) with the SMT production team at Super Bowl LI

Historically, 1st & Ten virtual insertion technology relies on reading data from a fixed camera position and matching that data (pan, tilt, and zoom) with a virtual map of the field of play.

“That is a one level of feat,” says SMT founder/Chairman Gerard Hall, “but [having] a moving camera where you don’t have fixed reference points and you need to rely on image recognition is another.”

The challenge in working with a moving camera in real time is that, when the subject in the image gets closer or further away, is it because the subject and camera are physically closer or did the camera zoom in or out? The new system has image-processing algorithms that know with some degree of accuracy where the camera is on the field; it analyzes visual anchors (a hash mark, yardage line, the yardage numbers on the field) to determine exactly where the virtual insertion needs to be placed.

“This technology can expand for all sports and things like advertising content or things like introductions of players on the field,” says Hall. “And this can also be used to bring image stabilization to a fixed camera that might be on something like a mast.”

It also could be used to replace the existing hardware-based insertion technologies that require extensive setup and calibration.

“With this,” says Hall, “you could just show up and within a day have the cameras good to go. And, in a world of distracted viewing, putting virtual ads into the actual game is going to become more and more important. In this case, we were just solving the problem of a 1st & Ten line, but this technology has a lot of other applications.”