Riot Games Boosts At-Home Production at League of Legends Mid-Season Invitational

The esports producer transmitted 34 sources from Paris to L.A. with just an 800-Mbps IP path

The concept of at-home production is nothing new for Riot Games, which has been leveraging its West Los Angeles esports-production facility in some fashion for its live productions around the globe for several years. However, over the past 18 months, the videogame publisher has been working on taking the productions to the next level with a new low-bandwidth IP contribution scheme that would allow more than 30 sources to be sent to Los Angeles in near real time from many more countries around the world.

Three weeks ago, Royal Never Give Up hoisted the League of Legends Mid-Season Invitational trophy at Zenith Arena in Paris.

The fruits of these efforts were realized last month during the League of Legends (LoL) Mid-Season Invitational in Paris: during three six-hour days of competition, Riot Games successfully transmitted 34 1080p60 live sources to L.A. at a latency of just 175 ms. The show was cut and produced live out of Riot Games’ control room in Los Angeles and distributed around the world on streaming platforms Twitch and YouTube, linear broadcast, and simulcast in 12 languages around the globe.

“One of the big topics in the entire broadcast industry is sustainability, and Riot is leading the charge with the tech needed to achieve this,” says Mitch Rosenthal, senior manager, broadcast operations, esports, Riot Games. “Our company goal is always to be the most player-focused company in the world. This [drives] us to build solutions that cater to all of our players. Reaching that goal pushed us in the direction of a REMI[, or at-home,] workflow. That means we can continue to produce great content that provides the widest player reach, giving us more to reinvest in the broadcast and grow the product globally.”

Sustained Success Requires a Sustainable At-Home Model
As a global brand, Riot Esports produces live esports events all over the world every year, making at-home production a logical game plan. However, the cost of securing the 10-Gbps fiber circuit necessary to deploy the at-home model (using J2K compression) and the total lack of such connectivity in many less developed countries spurred Riot Games to begin searching for a workflow in which it could transmit 30-plus sources over a 1-Gbps pipe.

Riot Games successfully transmitted 34 1080p60 live sources to L.A. at a latency of just 175 ms.

“Fundamentally, if we want to continue to do events all over the globe long term, then we need to figure out ways to work towards better sustainability and global compatibility,” says Ray Panahon, technical lead, esports, Riot Games. “The fact of the matter is that a lot of countries simply can’t always provide the reliable 10-gig connection we need for [at-home production] and, when they can, it is commonly cost-prohibitive. We have to think about how we do this in the future in places like Shanghai, São Paolo, and Saigon [with bandwidth constraints]. So we started looking at using MPEG-4 contribution-encoding hardware in place of J2K as a codec for REMI.”

The Path to 1-Gig: Haivision Helps Lead the Way
Over the past 18 months, the Riot Games technical and engineering teams explored multiple vendors’ MPEG-4/H.264 encoding/decoding solutions in an effort to overcome the primary obstacle: minimizing or eliminating video drift and maintaining lip sync over tens of encoder streams. Riot eventually collaborated with Haivision and devised an H.264 compression scheme capable of transmitting three dozen 1080p60 video paths using 1 Gbps or less while synchronizing audio and video within a single frame.

In February, Riot broadcast engineers Max Trauss and Michael Caal lead the first test of the technology during Week 9 of its North American LCS spring season, in which 34 video sources were sent from the L.A. live studio audience through the Riot Direct network to Japan and back into its control room (approximately 12,000 miles). The team successfully produced two days of games of more than six hours each with under–500-ms latency.

Much of the production team was located back at Riot Games HQ in Los Angeles for the at-home production workflow used for the MSI.

“The primary prerequisite was ultra-low latency and synchronization at MPEG-4 compression that would get us under that gigabit,” says Panahon. “Haivision was dedicated to overcoming the initial technical hurdles , so they were a huge piece of the puzzle.”

The workflow went live last month at the LoL Mid-Season Invitational, where 34 sources were sent 5,500 miles from Paris to L.A. over an 800-Mbps point-to-point MPLS IP path with just 175-ms lag. As a result, it was “business as usual for the production team,” notes Panahon.

“The impact on the production team was zero,” says Rosenthal. “They had to work through it at first, and the test we did in Japan was a huge step forward, but it had no impact whatsoever for [Mid-Season Invitational in] Paris. And it saved hundreds of thousands of dollars leaving the crew work from L.A. The high-compression–transmission workflow also reduced the requirements for venue bandwidth, which was another huge cost reduction.”

More Than Just Video: Prompting, Shading, Audio Mixing, and Switching From L.A.
The at-home workflow isn’t limited to video feeds, however. Riot also remotely transported the teleprompter over IP: actual scrolling and prompting was performed in Los Angeles and fed to an Autocue Qbox portable scroll system in Paris.

Riot Games successfully tested remote audio mixing using a Calrec RP1 production system in Paris and a Calrec Artemis console (pictured here) in Los Angeles.

Riot also tested remote camera shading for its Grass Valley remote-control panels by creating a wide-area network that allowed the Grass Valley cameras in Paris to be shaded from L.A. via IP. In addition, the 10 POV cameras (Blackmagic Design Micro Studio 4K) hanging over the player’s monitors were remotely shaded from Los Angeles via Blackmagic IP software for the Blackmagic ATEM switcher.

On the audio side, Riot successfully tested remote audio mixing using a Calrec RP1 production system in Paris and a Calrec Artemis console in Los Angeles.

Although, for the LoL Mid-Season Invitational, a SAM Kahuna switcher was deployed onsite in Paris and operated by a technical director for in-venue–screen switching, at future events, Riot is exploring deployment of a switcher frame at the remote site with the panel and TD back home.

“This test was a proof of concept. [We knew it] was going to be successful in theory, but we just had to prove it in reality,” says Matt Donovan, broadcast engineering manager, Riot. “At Riot, one of our core values is challenging convention, and this workflow represents a great example of that spirit. This is evolutionary; I don’t think it’s a revolutionary thing: we took an existing workflow and infrastructure and streamlined it and will continue to do this on the entire broadcast.”

New At-Home Workflow Provides Roadmap for Global Expansion
Although the launch of the new contribution scheme was a success in Paris, the real benefits are likely to be seen in nations outside of North America and Western Europe with less connectivity and infrastructure. According to Panahon, one of Riot’s most significant challenges in catering to its esports fans has been in emerging countries in Southeast Asia and Latin America, where the interest in gaming is high but the bandwidth available is low.

34 sources were sent 5,500 miles from Paris to L.A. over an 800-Mbps point-to-point MPLS IP path with just 175-ms lag.

“The common REMI workflows that we have today are not replicable in these countries,” he explains. “But now that this model’s in place, it’s really given us the capability to establish a workflow architecture and a methodology by which we can [guarantee] to our broadcast and production engineering teams that they will be able to produce the level of shows that you see being produced in the EU and North America without having to compromise or spend a fortune. Best of all, there can be better parity in terms of how the production and creative teams deliver these experiences to our players.”

With the League of Legends World Championship coming up in the fall, Riot’s at-home effort is likely to continue to open up new opportunities for live events in new markets.

“I think the real excitement comes when we are in a more challenging environment, such as Southeast Asia or Latin America, where the connectivity is a lot more restricted,” says Donovan. “We could have had all the bandwidth that we wanted at this Paris venue, but this test was about proving that remote broadcasts are options for all our Riot regions that want to up their production value.”

Only the Beginning: Riot Looks To Continue Down At-Home Path
The production in Paris was far from a one-off. According to Rosenthal, Riot’s at-home plans for future remote esports productions are full steam ahead. Not only will the H.264 IP at-home workflow continue, but the company is also looking to eliminate the need for a truck onsite and shift to a modified flypack model.

“Moving forward, we plan to go only to places where this model will work,” says Rosenthal. “We’re now very confident that there’s anywhere in the world that we can do this and that is absolutely the mantra going forward: when we’re on the road, we’re only going to do REMI.”

MSI was streamed in 15+ languages, across 14 digital platforms and broadcast on two TV channels.

Panahon adds that he not only sees more large-scale esports events taking place in less developed countries as a result of at-home production but is hopeful that others in live sports production will see the possibilities presented by the new Riot esports workflow and follow suit.

“It’s clear that, all of a sudden, there’s a lot more opportunity for all of our regions to do REMI and book more events with a much larger choice of venues and cities,” he says. “I’m hoping that the industry will take note and realize that you don’t always need to have big circuits and massive codecs to [ensure] high quality. At the end of the day, if you watched the show [from Paris] for three days, you saw that the quality looked amazing. We’re hoping that the industry takes notice and will grow from there.”