Tech Focus: Immersive Sound, Part 1 — The Technology Is Increasingly Integrated Into Broadcast Sports

Updated platforms serve as conduits for next-generation audio formats

Immersive audio has become an integrated element in a range of high-profile broadcast-sports proof-of-concept projects, and the format is increasingly finding its way out of the industry bubble so that more viewers can access it.

• After two years of experimentation, NBC Sports provided Dolby Atmos immersive audio on all 2019 Notre Dame Football home games for DirecTV customers, part of a package with 4K video.
• In 2018, South Korean television and radio network SBS transmitted live UHD coverage of selected World Cup matches over ATSC 3.0 in immersive and interactive audio using Fraunhofer’s MPEG-H audio format — according to the broadcaster, the world’s first regular broadcast of immersive and interactive audio powered by that format.
• The National Hot Rod Association (NHRA) continued a three-year trial of Atmos for regular-season and finals races.
• In the first live-produced Dolby Atmos event in the U.S., the 2018 Winter Olympics were available to Comcast and DirecTV subscribers, and Atmos is being used in the UK by Sky Sports and BT Sport for Premier League and Championship League soccer on their respective 4K channels.

Audio Consoles Are Gateways
Although it seems that the shift into immersive audio is moving more rapidly than the transition to surround more than a decade ago, Atmos, DTS:X, MPEG-H, and other immersive broadcast formats arrive at a time when sports are increasingly listened to over earbuds and on mobile devices, and stereo televisions remain ubiquitous. (The National Association of Broadcasters does not track sales of consumer-television audio formats, but a spokesperson says that, “while a definitive answer has always proven elusive,” the majority of broadcast television today is likely produced, broadcast, and experienced in stereo.)

Audio-console manufacturers, though, are actively trying to accommodate the emerging immersive-broadcast market, largely by developing and integrating immersive updates into consoles that will have to continue working with stereo and 5.1 for years to come.

“We have reached the point where sports broadcasters are familiar with what immersive audio offers and what it can add to a broadcast,” observes Berny Carpenter, broadcast product specialist, Solid State Logic, which added an immersive-audio feature set to its System T broadcast desk as part of a software release in May 2018. “There is also the understanding that different content will require mixing in different ways. For instance, the listening experience of stadium sports traditionally viewed from the sideline can be enhanced with additional ambience signals in the height channels. This works especially well with arena sports, such as ice hockey, basketball, and boxing. Sports broadcasts that include first-person content, such as motorsports, offer the chance to use the height channels for more-localized content.”

He points out that System T’s immersive features were developed to extend the functionality across the console so that mixing in immersive formats doesn’t add additional levels of complexity or require workarounds. Key additions are support for larger path formats — including 5.1.2, 5.1.4, 7.1.2. and 7.1.4 — and extension of the panning tools to include intuitive controls for the additional height dimension. Most recently, the company added a built-in binaural encoder and support for A-and-B–format ambisonic signals, including the Sennheiser AMBEO microphone, transcoding it into a channel-based format.

“These tools make it easy to add an immersive ambient bed to a live sports production, all controlled from the desk,” Carpenter explains.

Growth in Applications
Practical applications of immersive broadcast audio are growing. For instance, SSL has taken part in recent EBU trials at the 2018 European Athletics Championships and the 2019 Eurovision Song Contest. After cinema and television sports, music is the latest target for immersive audio.

Lawo’s Patrick Warrington says that, with video technologies, such as 4K and UHD, gaining traction with consumers, the tools for creating immersive sound need to be ready.

“We see sports and large-scale entertainment as the main areas of focus for immersive audio,” says Carpenter. “These are often the flagship programs for broadcasters, with large viewing figures and the biggest drive for technological developments. At the consumer end, soundbars and receivers that support these technologies are becoming more readily available and widespread, which will help drive adoption.”

Lawo can accommodate immersive audio in its mc² series desks, with additional DSP cards or existing ones redeployed to allow larger internal-bus widths. It’s the same way surround was adapted to in the past, and Lawo’s Patrick Warrington, recently named senior director, technical business development, says it results in a mix matrix capable of up to a 22.2 configuration that can be allocated anywhere along immersive’s X, Y, and Z axes. That has been further enhanced by Lawo’s proprietary Hyper Panning feature, which offers control of a complete surround bundle — all channel parameters and eight-bar metering—via a single rotary control link that allows rotation of the audio through 360 degrees.

“This lets you take legacy 5.1 surround audio into an immersive space and lets it be able to be treated as a native audio format,” he says.

Warrington feels that, although the uptake even of 5.1 surround among consumers has been relatively light, the arrival of the height axis could make the difference, once ways of realizing it in living rooms become more effective and affordable, which they are beginning to, in the form of immersive soundbars and upfiring speakers.

Furthermore, he adds, enhanced video technologies, such as 4K and UHD, have already gained substantial traction with consumers. “As a result, there is the expectation that the audio will have to improve as well. Immersive sound is a good match for this, so the tools for creating it need to be ready.”

Many Possibilities
Calrec’s Apollo and Artemis consoles can use the company’s recently introduced ImPulse Core to actuate immersive-audio control. Its additional DSP power provides mix buses up to 12 channels each that accommodate various immersive configurations up to 7.1.4. An additional capability allows the various channels to be grouped similar to stems; this could; for instance, allow a “0.0.4” group — height channels only — to be assembled and applied to an existing 5.1 mix. The ImPulse Core also provides for three-axis panning.

What to do with all these channels and possibilities is still under development by broadcasters, according to Henry Goodman, director, product management, Calrec. “We’re still in a trailblazing period,” he says, noting such examples as NBC Sports’ work with Dolby Atmos on Notre Dame football home games. “[A1] Doug Deems came up with ways of miking the pitch and the crowds and developing the beds and ambience mixes for those matches as he went along, and that’s how it will be for most sports. There will be different approaches for immersive sound for each sport.”

Calrec’s Henry Goodman: “Sports will be the main driver of immersive.”

Goodman adds that the developmental arc of immersive’s mixing techniques will likely be similar to that of 5.1 nearly two decades ago. “We began with all sorts of ideas, including panning the ambience to match the camera pans,” an idea that had the unfortunate potential to induce vertigo in some viewers, he recalls. “It’ll be a similar learning experience for immersive. We’re trying to provide the best tools for those experiments.”

Although Atmos has established firm traction in the U.S. market, Europe is still up for grabs when it comes to immersive-audio formats, with DTS:X and MPEG-H also in contention in various regions. That diversity offers broadcasters options as well as challenges, Goodman points out. He notes the efforts of the ITU to develop an agnostic audio-definition model (ADM — ITU-R BS.2125-0) that can encompass the audio and data for all formats.

“That has been the challenge on the production side for immersive,” he explains. “There is a need to make immersive mixes translatable between broadcasters and countries, which will encourage uptake of immersive overall.”

Objects in Action
Lawo also is preparing to leverage the object-audio capabilities of both Dolby Atmos and Fraunhofer Institute’s MPEG-H immersive-audio format. Its Kick technology, which lets sound-effects capture automatically follow the ball around the pitch for soccer, anticipates the coming personalization of broadcast-sports audio.

“The delivery of metadata with Atmos and MPEG-H means that immersive audio and object-based audio, while separate things, are becoming bound together,” Warrington says.

Calling immersive and object audio an “integrated development,” SSL’s Carpenter says, “The same next-generation audio technology, such as Dolby Atmos or MPEG-H, is used to deliver the immersive content and audio objects. These technologies allow a broadcaster to adapt their audio offering from one production to the next, changing the format of the audio bed and the number of objects, such as different languages or commentary options, available to the listener. The recent EBU trials demonstrated this: the MPEG-H broadcast consisted of an immersive audio bed plus different language options [that is, objects] for the viewer to choose from. Growth will be hand-in-hand: as the adoption of immersive audio increases, so will the use of object-based audio within broadcast.”

Goodman cites the work now being done by the BBC on speech intelligibility on television as an even more compelling driver of immersive technology: the use of objectified audio to let an aging viewer population increase the volume levels of commentary relative the rest of the program’s audio.

“It’s a direct outgrowth of immersive audio technologies,” he notes. “Sports will be the main driver of immersive, but things like what the BBC is doing can offer benefits to an even larger audience.”

Click here for Tech Focus: Immersive Sound, Part 2 — Consoles Are Transitioning to Next-Gen Tech.

Click here for Tech Focus: Immersive Sound, Part 3 — Multichannel Mics Find a Place in Sports Broadcasts