Disney, ESPN Send Virtual-Production Tech Into Sports Stratosphere With New ‘Catalyst Stage’ in Bristol

XR studio builds on concepts and technologies pioneered in Disney+’s The Mandalorian

Since the eye-melting release of the first season of Disney+’s The Mandalorian, virtual-production technology has been the hot new innovation in the Media & Entertainment sector. Now the technology has taken center stage for sports with the launch of Catalyst Stage at ESPN HQ in Bristol, CT. The first studio of its kind to enable both live multi-camera productions and cinematic-quality projects, the new immersive studio blends the physical and virtual worlds to unlock almost infinite possibilities for studio production.

Using Unreal Engine, Disney and ESPN are able to create ultra-realistic set environments.

“This is truly a game-changer for us when it comes to creativity, scalability, flexibility, and innovation,” says Joseph Ferretti, director, studio design and development, ESPN. “I can’t overstate its impact. I think the flexibility and the ROI are unprecedented when you consider the turnaround time and the creative possibilities. Using Catalyst Stage, we can do a high-end studio show and then do a cinematic commercial shoot the same day and, the next morning, be ready for a plate shoot or a content-enhancing feature for SportsCenter. Simply put, this is huge.”

The 11-million-pixel Catalyst Stage is capable of producing day-to-day live, multi-camera AR and XR productions in 1080p or UHD, as well as cinematic-quality and advertising projects in 4K. The studio is powered by proprietary software developed by the Disney Entertainment and ESPN technology teams, including the GRACE [graphics real-time automation and control environment] platform. Other key technologies deployed are Ghostframe, Unreal Engine, Disguise XR, Pixotope, and Mark Roberts Motion Control robotic camera systems.

Years in Development: How Catalyst Stage Came To Be

In 2021, ESPN opted to refresh Bristol’s Studio F, the first-ever HD SportsCenter studio. Two-thirds of the space was converted into a set for ABC’s college-football coverage, leaving a third available for something new.

“We started doing a lot of research into virtual production,” says Ferretti. “Then I saw Katy Perry do “Daisies” on American Idol, and that really caught my eye. We started exploring if we could do something like that here in Bristol, and the ball got rolling pretty quickly. We got executive buy-in and brought in folks from across ESPN and Disney to see what we could do in that space.”

The new XR studio creates a canvas for a wealth of creative possibilities.

In September 2021, ESPN conducted a technical shootout for the LED panels, processors, cameras, compositing, and other technology, and NEP came out on top as primary integrator. From there, Disney and ESPN technology, production, and creative teams got to work.

“This was truly a dream project for our team,” says Christiaan Cokas, director, Creative Automation + Studio Technology, Disney. “It’s exactly what we’ve been wanting to do for a long time. The timing was perfect. We felt like the tech was finally ready, especially Ghost Frame, which got us really excited. We could look production [teams] in the face and tell them this is significantly better than green-screen studio [technology] because you’ve got live multi-camera cutting. We can basically deliver your standard production workflow while making it film- and commercial-ready as well.”

Inside ESPN’s new Catalyst Stage in Bristol

Ferretti sees the launch of Catalyst Stage as the next step in a long progression of virtual-studio technology: from its use of depth monitors at the 2010 FIFA World Cup in Johannesburg through the launch of the Around the Horn virtual studio at ESPN’s Seaport Studios in NYC in 2018. “This is just the next logical step.”

Infinite Possibilities: XR Studio Opens the Creative Floodgates 

With Catalyst Stage, limitations attached to the access, building, and changing of physical environments are eliminated so that complex sets, which once took weeks or months to stand up or alter, can come together in days or even hours. Production teams can blend physical space with virtual elements to visually place talent in any location, real or imagined — from making Catalyst Stage a digital “twin” of an existing studio or stadium to a scenic hilltop or alien planet.

Catalyst Stage allows ESPN to create a digital twin of existing sets, allowing more flexibility and creativity in its studio-show operations.

“We have a huge spectrum of flexibility in terms of creating [sets],” says Cokas, “going from the Full 3D immersive environment that can be built in Unreal all the way to simple plate shoots that can be turned around quickly.”

Although the team is working in a virtual realm, Ferretti says, his team went into the initial studio-design process with the same sensibilities as for a physical studio. The team worked with top set designers like Jack Morton, Kramer, and MPKscenic to build these immersive environments.

Says Ferretti, “We were looking to create new environments for our XR stage that had the typical sensibilities in terms of scenic design and set elements and build past that because we have a whole virtual world where you can go almost anywhere. We can create an entire lobby, locker room, or practice facility, which we obviously wouldn’t be able to build on campus physically.”

The Hilltop set will be a primary environment used by Catalyst Stage.

The Catalyst Stage production team is able to cut to four cameras live while simultaneously shooting four different backgrounds between the wall and the floor. For the talent, the environment can be isolated to a single perspective that they can engage with directly. Meanwhile, directors can view all virtual-set cameras simultaneously just as they would be able to see the physical camera feeds on a traditional set.

“It starts with the producers and directors since they focus on the storytelling,” says Ferretti. “The director sits in the control room and sees composited outputs of all the cameras. They don’t have to dig into the tech and understand what’s under the hood; they just cut a show pretty much the same way they would in a physical studio.”

Directors are able to see all camera angles and virtual cameras just as they would in a traditional studio production.

To visually integrate the talent into the space with such elements as shadows and reflections on the floor, ESPN worked with Pixotope to incorporate an array of cameras that use machine-learning technology to triangulate on people and movement on the stage. That data is fed into Unreal Engine and allows ESPN to create those elements as graphics triggered by the talent’s gestures. The graphics are displayed on the wall LEDs so that the talent can see and interact with them directly.

“The talent should interact with a live shot on the LED wall the same as they would in a physical studio with a monitor,” says Ferretti. “Since we have all those physical elements like comfortable seating or a host desk within this virtual world, they can feel comfortable because they are already used to being on a physical set. The goal is for the technology to fall by the wayside so that they don’t have to think about it. They can focus on telling the stories and creating great content and leave it up to us to figure out what’s going on behind them.”

Although the technology is cutting-edge, ESPN aims to make talent as comfortable on Catalyst Stage as on a traditional set.

Megapixel’s HELIOS LED Processing Platform drives the displays and Ghostframe is functionality in HELIOS in partnership with ROE and AGS (Ghostframe is a combination of IP owned by three parties).  All five cameras used on the Catalyst Stage — four Grass Valley LDX 150 broadcast cameras, one ARRI AMIRA Live 35mm camera — are operated via Mark Roberts Motion Control robotic systems.

“We opted for robotics not only because of the space limitation but also for the flexibility,” says Cokas. “The depth in front of the stage is only about 10 ft. For us to work around that, we decided to go with these robotics. That has been a key element for us because it would have been tough to fit a jib in there but we can get still a lot of compelling, creative shots.”

All five cameras are mounted on Mark Roberts robots.

The Backend: Showing GRACE in the Studio, Power With 26 High-Octane Servers

With 26 real-time servers at work, Catalyst Stage renders 11 million pixels, refreshes 7,600 times every second, and uses the largest and most complex Disguise XR system ever built for television production. The servers process data at speeds as high as 120 Gbps and transfer data five times faster than traditional servers. ESPN moves 4K video-tracking data from each of the studio’s five robotic cameras plus any additional graphics and imagery and passes it all through the servers before the final product is displayed on the LED wall.

26 real-time servers power the new XR studio.

“I think the most important factor here is that we are using established systems that you can trust for daily production,” says Cokas. “I can honestly say that I trust the technology, which isn’t always the case with [innovations] like this.”

The key component differentiating Catalyst Stage from similar XR studios is the use of Disney’s GRACE platform, which allows operators to drive a virtual 3D broadcast studio with the same tools that control a virtual monitor. GRACE combines automated tools with Unreal Engine to make technically complex changes to the space very quickly without a big crew or specialized experts. As a result, production, operations, and creative teams can easily and efficiently select and move among multiple studio environments.

Disney’s GRACE platform plays a key role in Catalyst Stage’s operation.

“The whole point of using the GRACE toolset was to make our production operators feel comfortable,” says Cokas, “as if it’s the same show they do every day. They come in and load a set, which is fairly straightforward. Then they have their lighting controls, prop controls, video ins, and pretty much everything else that they would have in their typical operation.”

To make the system as similar to traditional studio operation as possible, Cokas and his team worked with a handful of superusers to obtain feedback and optimize the interface and controls.

Disney’s GRACE system in action

“We tried to make it flexible so that, no matter where you’re sitting in the production process, you can get to the tools you need,” says Cokas. “Those superusers have become our head trainers are spreading that knowledge, which will allow us to expand to more users. We’re improving our tools while also keeping our production partners comfortable so that they can worry about storytelling and about operation.”

Looking Ahead: Multi-Use Space With Potential for a Full-Time Studio Show

In the near term, Ferretti says, Catalyst Stage will be used primarily for specialty segments in SportsCenter and other studio shows. However, he envisions a future in which the studio could be home to its own daily or weekly studio show.

In the near term, ESPN will deploy Catalyst Stage for specialty segments like its Gear Up features.

“At the core,” he adds, “I think our goal is to service as many different shows as possible, as well as doing the cinematic content for the creative and advertising teams. But we’d also love to find a daily or weekly show that works. We have played around with some pilots and haven’t quite found the right fit yet, but we would love to find a show that we can pull off in Catalyst Studio.”

Although Catalyst Studio was built initially for ESPN, Disney says the advances in the project will extend to other parts of the company.

“I think it’s fair to compare this [technology] to when the iPhone came out in 2000,” says Cokas. “It literally changed the world. This is going to have that kind of impact on production. It’s the Swiss Army knife of production because it can do it all. And it’s ready to handle anything you throw at it today. We’re not just bolting everything together and hoping for the best; the technology feels like it’s actually ready for primetime.”