TranSPORT 2017: At-Home Production Has Arrived. Now What?
As the at-home model grows, connectivity and data-transfer requirements are exploding
Whether you call it at-home, home-run, REMI, mic’ers, or — as the Europeans do – remote production, one fact is clear: the concept of using facilities and crew at your broadcast center to aid in production of live sports shows in the field is skyrocketing. As the number of at-home productions rises exponentially, demand for reliable connectivity and tools for data transfer between the remote production and the fixed facility back home is exploding.
A panel at SVG’s TranSPORT conference in New York last week addressed the at-home–production phenomenon, with leaders from Big Ten Network, Fox Sports, Pac-12 Networks, NBC Sports/Olympics, Aldea Solutions, and The Switch. Much of the discussion — led by Chris Connolly, senior director, transmission engineering and operations, NBC Sports Group — focused on the risk vs. reward of deploying the at-home production model.
“The risk/reward discussion is interesting,” said Pac 12 Networks VP of Production Operations and Technology Michael Harabin. “[Some have said] these are very risky productions and things are going to go wrong. But we just counted up the single points of failure and determined that there were far more possible things that could go wrong on a regular production than on a remote production, where we have our best engineers in-house, multiple pieces of equipment to back up. We just found it to be much more secure — including the transmission — than a regular production.
“When you see all these trucks pulling in [for large multitruck productions] ,” he concluded, “it’s like you’re looking at dinosaurs moving around.”
College Conference Nets Leading the Way
Big Ten Network and Pac-12 Networks have largely led the way when it comes to at-home production: BTN launched its first at-home shows in 2009, Pac-12 immediately after its launch in 2012. This year, BTN and Pac-12 will produce 600 events (including 12 football games) and 415 events (including three football games), respectively, using the at-home model.
At-home production has allowed BTN to significantly increase the number of events it is able to produce overall, growing its slate of live productions from 400 when the network launched in 2007 to 600+ events.
“Doing at-homes has allowed us to produce more events live, especially Olympic [sports],” said Wes Goldstein, director of engineering, Big Ten Network/Fox Sports. “That growth has [been a result] of being able to produce some of our lower-tiered events, as well as some higher-end events like football. There is no event that we are shying away from at this point.”
On the Pac-12 Networks side, Harabin was quick to note that, although many of these at-home productions are lower-profile sports, they are still provided rather sizable production complements, including five cameras, two channels of EVS replay, and 16 channels of audio for 5.1 surround sound.
“These may be smaller sports, but they are still produced in a very big way. Just because it’s an at-home doesn’t mean it’s diminished in any way,” he pointed out. “We have a number of smaller productions that sometimes deserve [more equipment], and, if we have to fly that equipment and crew out or rent it from our vendors, we would never allocate that complement of equipment against that event. But having it all in San Francisco, if we get to January and our conference No. 1 is playing No. 2 [in basketball], we may want to allocate another EVS operator or enhanced graphics. … So we are able to add production value that we would never be able to justify before [for these events].”
Both BTN and Pac-12 have the benefit of dedicated IP networks (built on Internet2) connecting their broadcast center (BTN in Chicago, Pac-12 in San Francisco) with all the college campuses and venues in their respective conferences. This offers an invaluable advantage over other sports broadcasters (Fox Sports SVP, Technical and Field Operations, Mike Davies joked, “We call it cheating.”), which must coordinate connectivity to the broadcast center for every single at-home production.
“We have that as our advantage,” said Goldstein, “but the disadvantage is, we are always fighting all of the other content going on this network. These networks aren’t specifically for us; they were put in place to be research networks. We are always trying to find better ways of compression. Trying to get 10 cameras back to Chicago over 500 megs is a challenge. So utilizing H.264 technology and anything that comes down the road is something that we’re always looking at.”
At-Home Rule #1: Consistency Is Key
Fox Sports produces a number of events using its home-run (its term) production model, from lower-profile events on its FS1 and FS2 cable channels to onsite studio shows at larger events to digital needs on massive shows like U.S. Open golf, where it used BTN’s Chicago facility (Fox owns 51% of BTN) to produce all its live-streaming content (Featured Group, Featured Holes, etc.). In those experiences, Fox is quickly learning that deploying consistent facilities across these productions is integral to success.
“I would say the biggest challenge in home-run production is communications and audio,” said Davies. “Sending back video is fairly straightforward, but the complexity of interfacing a communications system between two points can be your undoing many times. I think coming up with a recipe for consistency in your facilities [is key] so that you’re not inventing new things every single time you do it. That’s what we are trying to figure out and get to: finding the ingredients. Consistent facilities are really what makes a successful [home-run] production.”
BTN has gone as far as building and deploying its own at-home mobile units, a fleet that has grown to seven, in order to keep its at-home production as consistent as possible across the board.
Establishing At-Home Connectivity at ‘Off the Grid’ Venues
Although major stadiums and arenas usually boast the fiber connectivity necessary for at-home production, major colleges, golf courses, and racetracks do not. Establishing the necessary connectivity for at-home production at such venues requires plenty of coordination with vendors and network carriers.
The Switch has been a major player in the growth of the at-home business in recent years, providing broadcasters with connectivity from both major venues and connectivity-deficient venues like NASCAR and PGA TOUR stops.
“Especially at events like NASCAR and PGA TOUR, it is hard [to establish connectivity], and you have to work well ahead with the network carriers to make sure you have dual-diverse paths out,” said The Switch CEO Scott Beers. “The hardest part of home-running from the carrier’s standpoint is that clients want to see consistency every single time they use it. For example, if they go into a stadium, they want to know that they can [set up] and it’s going to come out exactly the same way on the other side and that all the paths are backed up across the country, all the audio [channels] are hitting on the exact same spot. So we have built software and templates to build all the primary and backup paths. If you don’t do that, you’re never going to keep up with demand.”
The Switch continues to put an emphasis on the at-home–production market and recently launched its Cumulus cloud-based at-home–production service, which will be based at a new Burbank, CA, operations center. On the panel, Beers said he believes this shared-services model will be vital to the future of the at-home model and the live-sport-production business in general.
For International Events, Latency Is a Killer
Establishing reliable connectivity for an at-home production becomes that much more complicated for international events, in which the pipe to the broadcast center must run through multiple countries. Not only do major carriers vary from country to country — along with the price of bandwidth — increased latency also becomes a major hurdle.
“If you’re in a single country and you have a repetitive game [at the same facility] every weekend, that’s [not as difficult]. But, at Aldea, we deal a lot with international events that one day take place in Poland and the next day take place in Mexico,” said Aldea Solutions CEO Lionel Bentolila. “The most important thing we look at is the diversity — [establishing] two totally diverse fiber routes — and latency. Within one country, you may have latency of 30-40 ms. But, when you have a feed that is originating from Russia and going to Chile by fiber, like we did in June, then you’re looking at a different problem where you have latency in the 150 ms.”
However, as technology grows and at-home workflows continue to become more sophisticated, Aldea is seeing some broadcasters produce large international events exclusively from home with zero crew onsite. In September, TV Globo opted not to travel any crew to New York for US Open tennis; instead, the Brazilian broadcaster worked with Aldea to bring back several feeds and produce the show entirely from home.
“In the case of international events, like the Olympics and World Cup, things can be extremely costly. So whatever you can save on people and give you more flexibility is very much appreciated,” said Bentolila. “What we’ve seen in the last five years is that we are bringing more and more feeds back home. At the end of the day, as soon as they become comfortable and develop consistency, we will see more and more [at-home production].”