Tech Focus: Immersive Audio, Part 1 — Waiting Out the Pandemic

The infrastructure is ready in the form of microphones and consoles

Immersive audio has become an integrated element in a range of high-profile broadcast-sports proof-of-concept projects, and the format has been increasingly finding its way out of an industry bubble so that more viewers can access it. For instance, NBC Sports has been providing Dolby Atmos immersive audio on its Notre Dame Football home games for the past three years, part of a package with 4K video. Most recently, this year’s U.S. Open golf tourney was produced with an immersive 5.1.4 audio mix: surround sound (5.1) plus four height speakers (.4).

The productions followed such events as South Korean television and radio network SBS’s live UHD coverage of selected World Cup matches over ATSC 3.0 in immersive and interactive audio using Fraunhofer’s MPEG-H audio format (according to the broadcaster, the “world’s first” regular broadcast of immersive and interactive audio powered by that format). The first live-produced Dolby Atmos event in the U.S. was the 2018 Winter Olympics available to Comcast and DirecTV subscribers. (It was planned for the now-delayed 2020 Summer Olympics as well.) Sky Sports and BT Sport are also using Atmos in the UK for Premier League and Championship League soccer on their 4K channels.

And the tools for immersive sound are now more widely available and increasingly affordable.

“Although they are still more specialty items, we’ve found, as more [companies] enter the immersive market with different approaches, like the Studio Technologies 792/3 [an audio-over-IP–enabled multichannel monitoring system], and the Focusrite R1 [a Dante, Pro Tools, and Thunderbolt system-monitoring interface supporting up to eight channels], that this area is starting to be a more competitive landscape,” says Joel Guilbert, technology development manager, Dale Pro Audio. “The addition of those devices, all three having eight-plus–channel monitor-control capabilities, increases the amount of professional tools available by a third.”

Microphones on Cameras

René Moerch, product manager, DPA Microphones, has seen his company’s 5100 Mobile Surround microphone used in a larger number of sports broadcasts in the past year, including for some NFL broadcasts. They are also showing up on cameras, though mainly as a way to add an LFE channel to handheld shots, and are in some cases replacing conventional shotgun microphones for Doppler shots in auto racing, where they can boost low-end sound by eliminating proximity-effect issues.

DPA Microphones’ René Moerch sees growth in use of surround microphones used for sports broadcasts, including for some NFL broadcasts.

These types of microphones have also become standard for European soccer broadcasts; they can set up a soundstage for the lengthy wide shots of a soccer pitch common in those productions. In some cases, a surround mic is installed in a stadium, usually in the rafters and pointed downward, to create sonic surround baseline sound. That’s rarely the case in the U.S., where most broadcasters bring their own, mostly conventional, microphone complements for remote broadcasts.

“When a wide shot zooms in,” Moerch explains, “the baseline surround audio remains in the shot, giving it continuity.”

Immersive sound is part of the Next Gen Audio initiative, part of the ATSC 3.0 portfolio, whose audio standards allow a full 7.1+4 implementation: seven channels of sound in a horizontal plane, one channel for a subwoofer, and four channels overhead.

The Sound Needs To Go Big

Like 5.1 surround sound, though, Next Gen Audio is being propelled from within the industry, by its technical corps. However, according to Berny Carpenter, broadcast product specialist, Solid State Logic, which added an immersive-audio feature set to its System T broadcast desk as part of a software release in 2018, large and marquee broadcast events — World Cups, Super Bowls, even Eurovision Song Contests —will raise consumer awareness about immersive sound on television, which he believes will be necessary to move immersive further into the mainstream.

“It will also require more products that incorporate immersive sound outside of the very high end,” he says, citing the proliferation and declining costs of products like Atmos soundbars and Amazon’s Echo Studio, a smart speaker that can deliver a 3D audio experience using Sony’s 360 Reality Audio, Fraunhofer’s MPEG-H audio decoder, and other immersive formats. “The System T now also integrates with Sennheiser’s AMBEO microphone, allowing it to directly decode the ambisonic signal to channel-based outputs, such as 5.1+4.”

But sports remains a focus for proponents of immersive sound.

“We have reached the point where sports broadcasters are familiar with what immersive audio offers and what it can add to a broadcast,” observes Carpenter. “There is also the understanding that different content will require mixing in different ways. For instance, the listening experience of stadium sports traditionally viewed from the sideline can be enhanced with additional ambience signals in the height channels. This works especially well with arena sports, such as ice hockey, basketball, and boxing. Sports broadcasts that include first-person content, such as motorsports, offer the chance to use the height channels for more localized content.”

Consoles Are Ready

Audio consoles have been working toward accommodating immersive sound for some time. For instance, Calrec’s Impulse Core, the next-generation audio-processing platform for its Apollo and Artemis control surfaces, can accommodate 5.1, 5.1.2, 5.1.4, 7.1, 7.1.2, and 7.1.4 formats in terms of input channels, buses, monitoring, and metering. Although the platform’s remote-operations features have been highlighted during the pandemic, Henry Goodman, director, support and market development, Calrec, considers that “serendipity as much as anything” and notes that both remote features and immersive capability were designed to make the product as flexible as possible. He expects the near future not only to include immersive-audio formats but also perhaps to see them become part of tiers of premium consumer services.

“I don’t expect that all broadcasters will be interested in the complexity and cost of adding immersive sound on all programming,” he says. “Immersive sound as a premium service makes more sense.”

However, that would continue to expand the range of broadcast-audio formats, and immersive does come in a variety of configurations: 7.1, 7.1.2, 7.1.4, and so forth. Goodman says that consoles have to give A1s the ability to mix and monitor in all of them and to do so using automated workflows. Impulse Core, for instance, can create automated downmixes from all immersive formats up to 7.1.4.

“The console remains a routing device,” he notes, “but immersive is going to increase the need for processing, which is something that Impulse Core is designed to do. As we found with 5.1, as you start doing larger numbers of channels discretely, there is more complexity of the workflows around that. Impulse Core is a way to get that processing to the console.”

Christian Struck, senior project audio manager, Lawo, believes that the pandemic has actually slowed integration of immersive sound, and he cites recent cutbacks, thanks to COVID-19, in the capital investment that Next Gen Audio will require.

“Immersive sound means more channels, and that requires additional spending,” he says, adding that remote operations take precedence during the pandemic. “Before that, clients will want to see more [proof of concept] done in real-world production environments.

“But the interest is there,” he continues. “We saw many requests for Atmos and MPEG-H compatibility before the pandemic. There are fewer of those now, but they’ll return in the future. Right now, REMI is on everyone’s mind.”

Lawo’s product line is ready to accommodate both immersive and remote contingencies. Introduced at IBC 2018, the Lawo A__UHD Core can address immersive mixing configurations up to 22.2 and offers different levels of remote-operation functionality. The latter capability is further extended with Lawo’s A__line nodes and PowerCore, designed to serve as IP audio stageboxes for mc² consoles and as audio extensions for the V__matrix ecosystem or as standalone IP audio gateways. PowerCore includes comprehensive processing and remote-mixing capabilities.

According to Struck, the main considerations for audio consoles for immersive operations have already been largely achieved: for example, high channel counts and the busing, monitoring, and mixing infrastructure to support them. Downmixing capability — to 5.1, stereo, and mono — has also been made part of console software; the remaining challenge is upmixing from those formats to immersive.

“Everything and everyone is waiting for the pandemic to end so we can move forward,” he says. “Right now, we’ve been interrupted. But long term, the move to immersive sound only makes sense.”

Click here for Tech Focus: Immersive Audio, Part 2 — Consoles Shift Toward Software-Based Solutions

Click here for Tech Focus: Immersive Audio, Part 3 — Multichannel Mics Take Their Place in Sports Broadcasting