Augmented Crowd Sound Gears Up for New Sports Seasons
Fledgling audio-industry sector ponders relevance in a post-COVID landscape
Last year, enhanced-crowd-sound systems helped make sports during COVID seem almost normal. This year, they’re poised to do the same — and, perhaps, more.
Having provided the NBA’s 2020 season what was widely regarded as a highly authentic-sounding augmented crowd-sound experience, Firehouse Productions is reprising that role for basketball in the 2021 season, which began in December, and is doing the same for the NHL’s new season, scheduled to tip off on Jan. 13.
Last year, both leagues played their seasons in so-called bubbles: the NBA in a multi-court venue in Orlando and the NHL in two venues, in Toronto and Edmonton. This year, their games will be played in the teams’ home venues, and that has brought about a change in deployment and operation of Firehouse Productions’ crowd-sound systems. Instead of three or four operators working a pair of iPads and two Launchpads, a crowd-sound mixer at each venue will be working solo on two Launchpad system controllers.
This year, all NBA teams are pumping the crowd audio through the venue PA systems, with some acquiring —through either purchase or rental — additional PA boxes focused on the arena’s courts, to energize the athletes. Among other changes this season is that use and operation of augmented crowd sounds are being left to individual teams’ discretion, versus being under the control of the league last season.
The NHL, which last year used a crowd-sound system developed by EA Sports and other partners, will use the same Launchpad-based controller as the NBA, with audio sources coming from Firehouse Productions’ own vault of venue- and team-specific recordings. That sound will be played through the hockey arenas’ existing PA systems, which will not use additional PA components to augment the venue sound.
Both leagues have mandated that the virtualized crowd sound be played as part of game-broadcast audio.
According to Firehouse Productions VP Mark Dittmar, about half of NBA teams have augmented their PA systems with court-facing speakers, in some cases with high-end concert-level systems.
“You’re trying to get the sound up to around 105 dB from 150 ft. away,” he says, “so you need a pretty powerful system. The NHL crowd sound will be played just from the arena’s existing center-cluster PA systems facing out onto the stands from below the scoreboards.”
He adds that the main challenge this season is less about applying its crowd-sound solution to an entirely new sport — hockey — than about readying the system for both leagues’ playing in a combined 60-plus arenas and curating the appropriate sounds for each of them in a highly compressed time span.
“We had three months to put together the systems for the NBA [bubble] last year; we had more like two weeks to get ready for everyone playing at home this year,” he says. “We had to train more mixers and provide technical support for them, too. And we had to develop more content, because the teams wanted more team-specific content, like chants and songs, and the sound design for the NBA and NHL is very different. The teams want to be able to try different things with the crowd sounds, so we’re giving them options.”
Streamlining the System
Unlike Firehouse Productions, which existed before COVID-19, Los Angeles-based Sonofans was largely a product of the pandemic, founded by Fred Vogler, a multiple–Grammy Award–winning audio engineer and mixer who had been working as principal sound designer and FOH mixer for the Hollywood Bowl and Walt Disney Concert Hall, among other roles. For him, fanless sports presented an opportunity, one that he quickly leveraged into producing the virtualized crowd sound for much of the MLB’s postseason, including the World Series, and for most of Fox Sports’ main NFL games during the past season, as well as some collegiate football, professional bowling, and the Mike Tyson–Roy Jones Jr. eight-round exhibition match in late November at Staples Center in Los Angeles.
Sonofans’ crowd-sound platform uses Ableton Live as the software engine; Akai triggering pads let two-person audio teams access team-, league- and venue-specific content from Apple Mac Mini computers and an array of hard drives. Vogler has also found ways to streamline the platform and make it more productive and efficient as it enters its second set of sports seasons. That includes adding pedals to the triggering pads, essentially letting a single operator have an additional “hand” or two to keep the audio tightly responsive to the action on the field or court.
“We’ve been looking at how we can reduce the number of operators needed and other system requirements, to make it more cost-effective,” he explains. “How we can make one person be able to do more, as long as the randomizing feature in the software keeps it from sounding the same.”
vCROWD Meets AI
UK-based Salsa Sound — which began life as a developer of automated audio mixing and augmented field-sound effects for broadcasters and such sports teams as the Premier League’s Manchester City — has extended its portfolio to include providing crowd sounds for U.S. sports teams and leagues —MLS and USL soccer, the NWSL (for which the company supplied crowd sound for CBS’s Challenge Cup broadcasts) — as well as for the Big Ten Network’s hockey, football, and men’s and women’s basketball. According to Rob Oldfield, co-founder/CEO, Salsa Sound, the company has provided the service to more than 100 games in both countries last fall alone.
He expects that Salsa Sound’s augmented-crowd-sound system, vCROWD, which was developed rapidly to meet the needs of suddenly silent stadiums, will eventually be integrated with the company’s AI-based automated mixing and field-effects technology.
“Currently, we don’t use our AI system for the virtual crowd,” Oldfield says. “Moving forward, however, we are hoping to automate the vCROWD control using AI with data from additional sources, such as remote fans, automatically recognizing the crowd emotion and intensity from their responses and triggering events automatically accordingly.”
Soccer’s large field of play — the typical football pitch is approximately half again as large as an NFL gridiron — benefits from this hybrid technology, because microphones around the field’s perimeter are too far away to capture the sound of kicks in the middle.
“Our artificial-intelligence engine can follow the ball just by analyzing the audio and can augment or replace the ball-kick sound live in real time with one of the several hundred sounds we have sampled,” says Ben Shirley, co-founder, Salsa Sound. He adds that the entire process takes place with about 80 ms of lag, which is more than covered by the latency of the video processing. “And it can do it faster and more precisely than trying to manually mix a larger number of microphones during a broadcast.”
These crowd-noise entrepreneurs aren’t sure how long demand for crowd-sound virtualization might last. Certainly, taming the pandemic will bring fans back into the stands, decreasing the need for it. But the technologies that have sprung up around it offer new options.
Noting Scottish football’s propensity for inflammatory religious-sectarian chants, Oldfield says the combination of automixing and crowd-sound systems can make it far easier to avoid embarrassing on-air moments.
“The way our automixing system works in terms of removing profanity is not to detect the profanity with a view to removing it per se,” he explains, “but rather to listen for the sounds that we do want to add in, such as ball-kicks, racket strikes, collision sounds, etc.”
Dittmar suggests that the ability to hear much more natural sound from the fields of play won’t necessarily disappear once real crowd sounds return and that broadcasters could find new ways to keep capturing and delivering it.
“Maybe contact microphones in helmets for football,” he speculates. “There is a market for this close-up sound that the lack of crowds has revealed.”
Ultimately, augmented crowd sounds could become a regular part of the sports-audio arsenal, says Vogler, pointing out that television has long relied on artificial audience-reaction sound, starting with laugh tracks on sitcoms 70 years ago. Furthermore, the sports application has made such audio smarter: he notes that Sonofans supplied synthetic, dynamic crowd sounds for two NBC game shows last year, including the LeBron James–co-produced The Wall.
“The crowds will come back,” he says, “but, at the same time, we’re all getting better and better at doing this. There may be ways what we do can be used for the benefit of the viewers and the networks.”