Tech Focus: Monitoring for Immersive Audio, Part 3 — What Will Drive It for Broadcast?

Sports usually propels tech transitions, but entertainment is close behind

In the inevitable chicken-and-egg equation that precedes any new shift in media technology, what will definitively propel immersive audio into the broadcast environment is still up for grabs. If it were a horse race, though, some might say sports is ahead of entertainment applications by a nose. Or an ear.

Click here for Tech Focus: Monitoring for Immersive Audio, Part 1 — Remote Production Gears Up
Click here for Tech Focus: Monitoring for Immersive Audio, Part 2 — Speaker Options for Remote Production

“Putting music [elements] into overhead speakers doesn’t do a lot for music mixing,” says Tom Holmes, a freelance A1 who mixes the broadcast music performances at many sports events, including the NHL Winter Classic, where he mixed performances by Kiss’s Ace Frehley and the Goo Goo Dolls. “Sports, on the other hand, is well-suited for immersive audio. You can put the announcer in the overheads, so it sounds like it does at the arena, coming from the [venue’s] PA system speakers, or basketball effects as they bounce around an arena. Sports will be much more of a driver for immersive than entertainment.”

John Harris, a partner at Music Mix Mobile, the remote-production company that mixes the broadcast audio for shows like the Grammy Awards and the CMA Awards, takes it one step further, noting that he has found virtually no interest among the entertainment-television producers he works with when it comes for immersive sound.

Dolby Atmos, currently the leading immersive codec, has been deployed in NBC Sports productions of Notre Dame football.

“The capabilities to produce [immersive broadcast audio] are there,” he says, “but, whenever I bring the idea up with producers, I get a blank stare. They don’t know what they’d do with it. I get the sense that the high-end [entertainment] shows are not even venturing to have a conversation around immersive sound.”

Harris says the difference between immersive and 5.1 surround, which entered the broadcast arena mainly through sports more than a decade ago and has become the standard for most broadcast productions of all kinds, is that no media category other than sports has made a valid case for what to put into immersive audio’s overhead channels.

“Even when we began to experiment with doing the Grammys in 5.1, [late music producer and Grammy Awards’ audio supervisor] Phil Ramone said we had to make sure that it also still sounded great in stereo,” Harris says. “We still do. I just don’t know what immersive brings to music on television.”

However, he has discovered that mixers for European broadcasters are experimenting with just that. He has shared mix studios with A1s for Deutsche Telecom and other networks mixing the Montreux Jazz Festival, among other events, using immersive codecs.

“The Montreux organization is very progressive, technically speaking, and they’ve been experimenting with immersive like that over there,” he notes. “It’s part of the conversation for entertainment over there, but not over here.”

Technical Obstacles
Paul Sandweiss, who mixes live-music shows including the American Music Awards, the Billboard Music Awards, and the Emmy and Oscars shows, as well as the halftime show at Super Bowl LII, is less than thrilled at how 5.1/AC-3 has been implemented on television, mainly because of what he contends have been inconsistent results with its fold-down to stereo. The solution is to provide a separate stereo mix along with a 5.1 mix, he says, noting that that is the workflow protocol to Netflix, for instance.

“Most performers we work with want us to finalize the mix with their input.,” he explains. “Once that is done they do not want others to have ways to change the mix downstream.”

Major-network live sports-audio mixes are done largely as 5.1 surround and downmixed automatically at the set-top–box stage in consumers’ televisions. They’re subjected to a number of potentially distortive artifacts and unintended processing along the way, drawing criticism from music artists performing on television.

However, Sandweiss believes that the problem cannot be solved at the distribution level: “There are wide disparities in those chips and their subsequent fold-down mixes, even differences in Dolby AC-3 chips made in different countries over the years.” Thus, he also feels that separate 5.1 and stereo mixes created at the source point are the best solution. However, he notes, that was tried at the 2006 Grammy Awards broadcast, with a 5.1 mix embedded with the (then-experimental) HD video feed and a stereo mix with the SD video.

“But it was decided that was not cost-effective or technically savvy,” he says, “and it was decided to feed only 5.1 on the broadcasts and the set top box would make everything wonderful. Unfortunately, that doesn’t really work. In sporting events, news coverage, and reality programming, it’s easy to anchor the dialog in the center speaker and fly lots of sound around the sound field, knowing that, even if the relationship between channel levels is misadjusted, the resulting mix will probably be tolerable. Turning the dialogue or center channel up and down 1-5 dB will make a difference but [will not] destroy the overall mix, as it will [in] a music performance if the vocal level is changed in relation to the music level within a performance by that amount.”

Those concerns, he says, extend to 7.1 and 5.1+4 surround and immersive formats as well.

Dolby, whose Atmos format is currently the leading immersive codec, is understandably broad in its expectations for both categories. Pointing out that Atmos complements 4K and HDR sports programming like NBC Sports’ productions of Notre Dame home football games, Dolby Senior Product Marketing Manager Rob France says, “We do see other genres outside of live sports driving adoption of immersive audio, such as episodic shows and live music. Sky in the UK delivered the Isle of Wight Music Festival to its subscribers in Dolby Atmos this year; NTT Docomo, the largest mobile operator in Japan, recently streamed Japan’s largest music festival in Dolby Atmos to compatible mobile devices; in China, iQIYI has broadcast several musical performances in Dolby Atmos.”

France also points out that all the major Hollywood studios now support Dolby Atmos for the home, as do streaming services iTunes, Netflix, Prime Video, and Tencent.

“Together,” he says, “we see these factors driving the adoption of Dolby Atmos as the leading solution for next-generation audio for both the entertainment and broadcast industries alike.”

For now, however, sports will likely remain the tip of the spear when it comes to uptake of new broadcast-audio formats, the testing ground for sound before entertainment productions take it mainstream, but with all the expected potential for mistakes and blind alleys along the way. As pioneers any endeavor say, to find the leader, follow the arrows.