Machine Learning, AR, VR, and the Future of Sports

Ongoing developments hold promise of new experiences in the future

Machine learning, augmented reality, and virtual reality and their roles within the sports environment was at the core of day two of the Sports Business Journal’s 2017 Neulion Sports Media & Technology Conference. And one thing is clear: while it is still early days for each of the technologies and hasty opinions can be made of success or failure there is no denying that the efforts are being made to transform the sports media landscape.

Everything from coaching to TV and Internet distribution, was the focus of a discussion with Rajiv Maheswaran, Second Spectrum, CEO at the Sport Business Journal Media& Technology Conference. Second Spectrum discussed an ongoing development that will impact arguably the entire sports content creation and distribution marketplace: machine learning.

Second Spectrum came to the fore in 2013 working with four NBA teams and now is being used league wide. And even old-school coaches and teams steeped in tradition are adopting technology as it can often be a difference maker on and off the court.

Second Spectrum entered the market when it looked to take raw data related to player and ball movement on the court and then add a layer of analysis and predictive statistics that can help transform the understanding of the game.

Maheswaran points to something as simple as identifying every pick and roll in a game and then offering a deeper dive into the results of the play, the matchups that worked (and didn’t), and more.

“Before machines identifying pick and rolls was done by people who would watch the games but they would only catch about 20% of them and they would only be correct in identifying a pick and roll properly 80% of the time so they were missing tons of them,” he says. “And then media partners like ESPN would be doing the same thing.”

An important part of getting by in for new technologies is working with the coaches and helping them understand how data points fit into the rest of their ecosystem. Once the coaches are on board then media partners start to make use of it and then, eventually, fans will be able to sort through data that interests them.

“The coaches don’t care how something gets done they just want someone to produce a video that they want,” adds Maheswaran. “And they have a need for accuracy. It won’t pass the smell test unless it is accurate. There is a lot of buzz around machine learning and there is no tougher customer than an NBA coach.”

With a league deal in place Second Spectrum has grown to 100 full time staffers (and about 60-part time) and has now installed its own capture system within NBA arenas.

“We can increase the pace of innovation so that what used to take hours and minutes is down to seconds,” he says. “We can innovate our own infrastructure and the league wants to move forward and do innovative things for every frame of video and every fan. So, we are always trying new things.”

One advantage of working with a ton of data and making it malleable is that when it comes to serving fans a league, team, or distribution partner will more easily be able to craft different experiences for non-fans, casual fans, and super fans.

Rajiv Maheswaran, CEO, Second Spectrum

“ESPN can get the audience the stories that are being talked about,” says Maheswaran. A clothing manufacturer, he explains, doesn’t just say this one shirt is perfect and a restaurant doesn’t just say this one meal is perfect.

“But sports broadcasting does that,” he says. “Myself, I would love to watch shooting probabilities floating over player’s heads. Or my wife want to know more about the players. And kids may want to see fire balls or cats. Until you have cats you are nothing on the Internet.”

The transformative nature of that data will help prevent traditional broadcasters from simply turning the Internet into a different form of cable.

“The Internet is really about what you want when you want it and in your own way and that is why machine learning is key,” says Maheswaran. “It’s about taking data and turning it into something people want.”

Unleashing the Power of AR
The power of machine learning is the ability to continually evolve with the user, becoming smarter as it continues to take data points and then use them to create a customized experience. The trick now is how to have a user interface that can seamlessly use that data and apply it in a practical and simple way.

That is why the recent developments in augmented reality (or AR) hold so much promise.

Most notably, recently released AR development kits from Apple and Android, coupled with smartphones that have the power to deliver augmented reality experiences, could unleash that power nearly everywhere. Fans in stands will be able to hold up a phone, point it a player, and see statistics and biographical data. Or they can snap a photo of a player like Steph Curry and instantly be transported to a specialized Web page or app, obviating the need for things like QR codes.

“And things like halftime shows will have augmented reality content for those in the venue that can be underwritten by a sponsor,” says Neil Mandt, Mandt Media, CEO and Founder. “The experience can be bigger and better for the fans.”

Mandt says that anyone who doubts AR only must consider that Apple is a believer.

“Apple is the most valuable company in the world and they gave away their AR software development kit and now developers are just starting to get involved,” he says. “And AR Core from Android has only been around for a few months and already things are moving fast.”

Mike Davies, Fox Sports, SVP of Technical and Field Operations agrees, adding that the developer kits are getting people within app development teams familiar with it. The challenge now is figuring out how customizing an AR experience can be harnessed. An example would be the ability to show the undulation of the greens during a golf tournament on a second screen device without taking the viewer outside of the broadcast.

“Instead you can immerse them in the broadcast,” he says.

There are also new sponsorship opportunities beyond the halftime show offerings mentioned earlier.
“It offers more immersive ways to put billboards in a broadcast or a virtual Ford truck on the field,” he says. “But we are in the exploratory stage of all of this and we are looking to other partners to see what they have been doing.”

He points to the original glowing puck used on hockey in the mid 1990s, virtual first and 10 lines, and virtual strike zones as examples. And to Maheswaran’s point of customization a new level of TV experience might also be possible as OTT services and apps become more tightly integrated with a TV set. And fans in the stands might also be able to bring those at home enhancements, like a first and 10 line, to their smartphone so that can hold it up and see the line on the field.

“You can make the in-venue experience more like TV,” he says.

Can VR Get Its Mojo Back?
The topic of virtual reality was also part of the discussion and Jamie Leech, MLBAM, VP, Games and VR, says that a VR game created for an Americana Festival in London and the 2017 MLB All-Star Game in Miami showed the potential of the format: a virtual home run hitting contest.

“One of the benefits of baseball is you can stand at the plate and simply swing the bat and hit home runs,” he says. Fans in London and Miami put on a pair of VR goggles and stepped up to a virtual plate in a virtual MLB stadium that replicated the actual stadium environment.

“People who experience good VR can see why it will be a success,” he says. “It’s not like 3D where people said it would be big because people were saying it will be big.”

There is still work to be done on the hardware and software side as a really good experience right now requires a $2,000 computer and an $800 headset. It also could require someone to switch their smartphone and that is a barrier as well.

But, says Leech, MLBAM is seeing an increase in engagement with their VR offerings.

“Video content in VR now has about 50% longer session times than previously and while the installed user base is small people are coming back to using it again and again,” he says. “But we are still at the embryonic stage.”

Derek Belch, STRIVR CEO and Founder, believes that VR is exactly where it should be in terms of adoption and the ever-improving technology and experience.

“The expectations a couple of years ago of where VR would be today was a little crazy,” he says. “Some say it is in the trough of disillusionment but things are happening. Consumption is only going to go up as devices get up there so there is no reason to be pessimistic but just realistic.”

He says the VR movement is about five years away from being mainstream, giving both the technology that allows VR to mature and deliver a better experience and for content providers to figure out what really works. But, he cautions, it needs commitment from leagues and rights holders to reach that point.

“MLBAM and the PGA Tour are the most forward thinking but VR is not going anywhere if the governing bodies don’t put more effort into it,” he says.