MLBAM Edges Toward the Future With 8K, VR, AR, Intel 360 Demos at All-Star Game

MLB Advanced Media used this week’s All-Star festivities in San Diego as an opportunity to demo bleeding-edge technologies: 8K/UHD, virtual reality, augmented reality, and Intel 360 Replay Technology. Besides exploring these technologies, MLBAM investigated how its Statcast player/ball-tracking technology can make them more efficient and accurate in covering America’s Pastime.

“We are always looking to explore new [technologies], and it’s all part of the same vision,” said MLBAM EVP/CTO Joe Inzerillo at the Petco Park demo suite on Tuesday. “We are trying to have a comprehensive viewpoint on all these different technologies’ [coming] together and focus on the harder problems first, like Statcast and WiFi [at the ballpark]. Now that we have that foundation system, we can start to plug in these modules and really start to experiment with what’s going to be the most compelling experience for our fans.”

The 8K/UHD demo at MLBAM’s suite at Petco Park

8K/UHD on the Diamond
During Tuesday’s All-Star Game, MLBAM deployed a single ASTRODESIGN 8K camera at the high-home position to capture the entire field of play, sending the live feed to an 8K monitor inside a VIP suite at Petco Park. Statcast data was deployed to drive virtual-camera movements within the 8K frame to zoom in and pull out hi-res cutouts of the field. MLBAM sees this as a potential tool for MLB broadcasts, streaming experiences, and official review.

“This is the first time we can actually have an extremely high-resolution image of the entire field of play and then can zoom in on sections of it to use in broadcast or for officiating applications,” said Inzerillo. “What is really interesting is the [reduction] in motion blur. Since the camera is fixed, when you do this virtual move, it actually winds up being a clearer move than you can get from a real camera, which would have natural motion blur. When you start to think about it from an officiating standpoint, this becomes something that is really, really useful.”

VR at the Derby and FanFest, AR for the Game
Virtual reality had a sizable presence for MLBAM at All-Star this year, thanks to NextVR’s live production of the Home Run Derby on Monday and a VR demo at FanFest down the street at the San Diego Convention Center.

The virtual-reality demo was live at MLBAM’s suite at Petco Park

“Every technological advance has a lag time of what it’s like to do storytelling. I think that we’re at the point where there are not best practices for VR because we’re still feeling our way through it,” said Inzerillo. “The only way we’re going to find out is by doing more and more of these types of efforts to see what’s compelling and what resonates.”

The Derby VR experience was available on-demand on the MLB channel of the NextVR platform 15 minutes after the event’s conclusion and also offered live in the VIP demo area at Petco Park. It allowed viewers to select among the seven angles along with a switched feed that included live audio commentary by MLB.com analyst Mike Petriello. The experience also integrated MLBAM’s Statcast system, which virtually displayed distance of all home runs during the Derby.

The FanFest VR demo comprised a variety of VR experiences that had been shot at Petco Park three weeks previously, including batting practice inside the cage, a front-row seat at a game, and inside the bullpen as relief pitchers warmed up. Beginning at the “home screen” inside MLB Network’s studio, with monitors displaying the various experiences to choose from, the viewer used a laser-pointer–style selector to select an experience and, once inside the experience, looked to the ground to return to the home screen.

“There’s no question that baseball is a challenge relative to some other sports, because we have the biggest playing surface, the smallest ball, and we play outside — all things that are not in our favor when it comes to [VR production],” said Inzerillo. “And, while it’s cool to sit in the stands and watch a play, we’re also looking to create moments in time, such as inside the bullpen. Nobody really gets to see what happens in there, so, when you see a player warming up, you get a really good sense of presence that you wouldn’t get any other way.”

The suite at Petco Park also featured an augmented-reality demo that allowed the viewer to watch, via military-grade AR headset, a game overlaid over the real world.

“We think there is a place for virtual reality where you’re totally in your own environment, but we also think the world of augmented reality is actually much more interesting for some things,” Inzerillo pointed out. “The technology for [AR] is really coming along. The glasses we’re using here are the best I’ve ever seen; It doesn’t wall you off. So you can be in your own space or not — it’s your choice.”

Intel 360 Replay Stands To Benefit From Statcast
MLBAM also partnered with Intel on its 360 replay technology, which creates a 3D rendering of the field and a virtual view of a play from almost any angle. The 28-camera system was installed at Petco Park in advance of the All-Star Weekend. In addition to the demo at MLBAM’s suite, Intel 360 Replay Technology was featured on MLB.com and in telecasts of Sunday’s Futures Game on MLB Network, Monday’s Home Run Derby on ESPN, and Tuesday’s All-Star Game on Fox.

“San Diego has become one of our test beds to try to figure out how to do this best,” said Inzerillo. “We’re learning all the time. For example, shadows are really tough on the system’s imaging. But it’s increasingly cool the more we explore it. The notion of being able to actually take the Statcast data to drive the region of interest rather than render the entire field and structure is very intriguing. It could potentially automatically determine the interesting part of the frame based on data, which allows you to cheat a little bit so you know the play in advance.”

Statcast Continues To Evolve
With a season and a half under its belt, the MLB Statcast player/ball-tracking system continues to mature — in terms of both its library of data and the technology being used. At the Home Run Derby, for example, MLBAM deployed a federated radar system (provided by TrackMan) to track the cavalcade of balls flying across the field.

“The radar we were using was never designed for having multiple objects in flight at the same time because it could never happen in a baseball game,” Inzerillo explained. “So we actually went with a federated radar system. We ended up having just under 700 hits during the Derby, and every single one of them was tracked, and we had exact trajectory for all of them. From a technical standpoint, that was a specific use case for the Derby, but it’s also about trying to solve the federated-radar problem. How do you have more than one radar active at the same time in order to put them together and get some deeper data. We learned a ton that I think is going to translate into enhancements for Statcast down the road.”