Sportvision, ESPN Successfully Deploy 1st & Ten Line on Monday Night Football Spidercam

Monday Night Football viewers have been treated to an extra dose of the yellow 1st & Ten line this year, as ESPN and Sportvision have teamed up to bring the virtual line marker to the production’s Spidercam aerial camera system for the first time. This marks the next step in the decade-plus evolution of Sportvision’s 1st & Ten system that started back in 2004, when it was first deployed on Monday Night Football solely in replay situations.

Sportvision’s 1st & Ten line appears on the Spidercam on ESPN’s Monday Night Football

“Back then we couldn’t do it live because the technology wasn’t up to it, but we have all gotten smarter and the technology has gotten better – computers got faster, cameras capture more pixels,” says Sportvision CTO Marv White. “So we went to ESPN this past summer and said we were ready [to deploy 1st & Ten on Spidercam] and they said let’s do it. They are a group we’ve worked with all along and [Director] Chip [Dean] and [Producer] Jay [Rothman] are outstanding to work with because they understand new technology; they know that there are going to be surprises and that we deal with them, and that we somehow make it all work.”

From Lab Test to Live Telecast
Since the Spidercam is not a fixed camera system like the other four standard cameras (three up-cameras and the all-22 shot) in which the 1st & Ten line has long been available on for MNF, Sportvision had to, essentially, go back to the drawing board. Sportvision’s computer vision technology relies on knowing the camera’s precise location, but the Spidercam moves throughout the X/Y/Z axis – creating a unique challenge for White and his team.

“We have computer vision approaches to tracking and putting a first down line on a camera, but that relies on the camera being fixed and knowing exactly where the camera is,” explains White. “So we had to come up with much more complex mathematics in solving for the camera parameters, and [adapting] computer vision for finding objects in the [frame] that we could use to keep track of where the camera is and where it’s looking.

“Football is the place to start because they basically paint the coordinate system on the ground for us, so we have the yard lines, the hash marks, and the sidelines – we use all that,” he continues. “But, even using all that, it’s still a daunting project, and it’s still very much in development.”

ESPN and Sportvision began testing the virtual 1st & Ten and Line of Scrimmage lines on the Spidercam this past preseason. After proving out the technology over a handful of games, ESPN finally presented the virtual lines live on Spidercam early this season.

“Even after we knew it would work it actually took a couple more weeks before we went live,” says White, “because Chip said ‘I’m not doing this for eye candy, I’m doing this for strategic reasons; I’m doing it to tell the story.’”

Inside the MNF Truck
This season, the 1st & Ten system has typically gone live on Spidercam 20-30 times per game with White estimating that 10 of those instances are live and the others in replay via a dedicated EVS operator, Josh Johnson. The live Spidercam 1st & Ten shot most often used by Dean is the wide shot behind the quarterback as the team breaks the huddle and approaches the line of scrimmage.

“The fan gets to see what the quarterback is seeing,” says White. “That’s the advantage that Spidercam offers in the first place, but now they get to see what the quarterback is seeing even better than the quarterback sees it because they can see where the line of scrimmage and the first-down line are.”

In terms of replay, having the 1st & Ten system on the sweeping Spidercam shots allows the viewer to see exactly where the ball-carrier passes the first-down marker on close plays.

“Our operator who’s running the system, Todd Golebiewski, is an extremely important piece,” says White. “His first responsibility is to get the line off air if he thinks it’s not going to work. Then when we start a play, he’s got to make sure that our wire-frame field is perfectly aligned with the real video image of the field, so he may have to nudge it a bit.”

While White says the 1st & Ten system is not yet 100% available on every Spidercam shot, he says the system is becoming “more and more reliable every week.”

“It’s computer vision tracking the camera, but there’s a fair bit of human help,” says White. “As we get more and more automated, the help can be less and less skilled. But we’re still in the pretty early stages of computer vision doing the whole job. It’s not like we’re going to automate broadcasts here.”

A Collaborative Effort on MNF
“To make all this work is more than just Sportvision inventing technology,” says White. “ESPN worked very closely with us to make sure that the camera framing was right for telling the story and for making the lines show up right, and the EVS folks worked with us to make sure we recorded the right clips. So it really is a collaboration.”

White also tips his hat to the role that the Spidercam team has played in the development process.

“The Spidercam guys were very helpful,” continues White. “We work closely with [the operators], so that, before a play, they frame the shot wide, which gives our computer vision lots of stuff to latch onto and track. And then they just stay in touch with us and make sure that we’re good.”