SVG@NAB Perspectives: NextVR Wrangles Next-Gen Audio’s Issues
The past, present, and future of broadcast-sports audio is sprinkled throughout the exhibit floors at the NAB 2016, as well as in the sun-drenched canyon between the Convention Center halls. And, in that sprawling corral of trucks, trailers, satellite vans, and drones, one vehicle contains a bit of each of those moments in time.
NextVR’s truck fits well with the others outside, but its matte-black finish against the gleaming whiteness of most of the trailers suggests that what’s inside is disruptive. You won’t see it at first glance: compartments and displays and racks all look familiar. But NextVR is actually a test bed for how sports will look and sound in a virtual-reality landscape.
NextVR SVP Ryan Sheridan says audio for virtual-reality production needs to be imagined from the ground up.
Ryan Sheridan, SVP, imaging and production technologies, NextVR, describes it: “This truck is one-third audio, one-third production, and one-third data center.”
Audio gets equal billing with video production and with data — the fuel of post-Billyball sports production — because virtual reality demands a higher level of it.
“We knew audio was going to be important, because the engines of game audio, higher-order audio like Atomos and DTS, are what we’re putting into it,” says Sheridan, referring to ambisonic, non–channel-based sound. “Working on head tracking” — the ability of the audio to follow the VR viewer’s field of vision — “in a live capacity proved to be harder than we anticipated. No one supports it. What’s out there now is all based around postproduction: you capture, you acquire it, you sweeten it, render it, and put it out there. But doing audio for VR live is far more complex.”
That’s because, instead of the conventional broadcast paradigm in which multiple cameras share a single audio mix, VR demands a separate audio mix for each camera, in order to follow its individually shifting POV as the viewer’s head moves. And when each of those mixes is a 7.1 immersive-audio mix, the amount of processing can be massive. Sheridan says platforms found in videogaming, such as Atomos and DTS, will be part of the processing used for VR’s audio, because they offer the kind of rendering power necessary to allow audio to keep up with a constantly changing set of perspectives.
The matte-black finish on NextVR’s hints of something disruptive inside.
“Audio quality is going to be as important as the video quality for VR, when it comes to immersive audio,” he says.
One familiar piece of equipment aboard the NextVR remote-production truck is a Studer Vista 1 audio console. Sheridan says the Vista X and Studer Infinity Core processing platforms, capable of more than 800 discrete audio channels, offer the bandwidth necessary to process the high channel counts of multicamera, multiple 7.1-surround productions. He says a typical production would use between 10 and 15 cameras, which would easily push the overall audio-channel count above 100 simultaneous channels.
But the big challenge is doing all of that in real time, which is what live sports productions demand. NextVR has already applied some of its efforts to that, working with the NBA and with Fox Sports on NASCAR productions; NextVR was at the Daytona 500, and more races are planned in VR. The company — infused with a $30.5 million round of funding from Comcast, Time Warner, and others late last year — already has 23 patents granted or pending for virtual-reality technology in content, delivery, transmission, and playback.
“We’ve solved a lot of issues, but there are lots more that have to be addressed before live sports can be produced [in VR] on a regular basis,” Sheridan says. “What the truck does is give us a lab to work those out.”