Live From FIFA World Cup: Amazing AR Studio Leads ITV Sport Production for Russia 2018
The undoubted star of the Russia World Cup presentation from ITV Sport has been the dramatic ‘cathedral’ studio in Red Square, with a backdrop of St Basil’s ahead and the Kremlin on the left. ITV Sport’s studio uses various elements of AR to enable a small studio space with restricted height to look much larger and see the full height of St Basil’s Cathedral.
It also incorporates a UHD 180 camera feed from host broadcaster HBS to transport the viewer into the venue. An intensive work effort in a tight timescale with partners Deltatre (AR system), Gearhouse (studio build), Stype (Red Spy optical camera tracking), Telegenic (OB facilities) and White Light (LED screens, D3 server and lighting) went into the intricacies of the design, especially for a temporary setup. A trucking system for the LED screens was built to enable screens to be moved in, across the window, in a short space of time.
BAFTA Award-winner Paul McNamara, Senior Director/Executive Producer Major Events for ITV Sport, held overall creative responsibility for the ambitious and challenging set, with design by Paul Sudlow and set construction by Scott Fleary. Alex Dinnin was graphics/AR designer and lighting directors are Chris Hollier and Andy Cottey. Vision supervision is provided by Mark Dunning and Adrian Turpin, with Nicos Chrisdoulou as studio project manager.
ITV Sport Technical Consultant Paul Bateman, who played a central role in planning its World Cup operations across Red Square and the IBC, told SVG Europe “The set receives a virtual extension in the shape of an augmented reality ceiling designed by Alex Dinnin to be like the inside of a chapel, complete with a virtual window fed by two HD cameras. Two side LED screens have a further two HD cameras to provide views to the left and right of the real window, providing an unrivalled panoramic view of Red Square.
“To complete the effect, the AR ceiling and two side screens use the tracking to provide perspective movement, giving the impression the windows are real.
“The same system is then applied to the incoming UHD stadium feed, but with one significant difference: three further LED screens are placed in front of the actual window, with the stadium feed mapped across the five LED screens and virtual ceiling.
“deltatre’s delivery uses eight Vizrt Engines with Stype Red Spy providing the tracking. A further two Engines provide the onscreen graphics and match clock, and the IBC is set up for post-production NLE rendering,” said Bateman.
Leading the team in the Telegenic truck at Red Square are Senior Unit Engineer Jason Langford and Deputy Head of Vision Tom Read. Langford told SVG Europe, “there are four HD cameras, two centre cameras and two side cameras, and they give us our ‘AR paintbox’. And then we move around within that vista, with data from the track camera – all working in real time.
“The D3 server is handling all the content going to the screens. And we’ve got another screen that comes in the back wall to do the perspective tracking, when in Red Square tracking mode. The data from the jib camera is sent to deltatre, and they send us track data to match the Red Square perspective,” said Langford.
“We also do a stadium tracking mode with data from the Stype camera, and when we’re in full stadium mode it looks like we’re faking a perspective through the window. We build up the entire set in a 3D world and then manipulate around that. And that’s it in a nutshell!” he said.
“The studio has been very tough,” ITV Sport Technical Director Roger Pearce told SVG Europe. “We’ve been through the depths and come out of it with something I think we can be proud of. It’s been the biggest technical challenge and the biggest technical achievement.
“And it’s been about the people: the Telegenic engineers, the White Light guys, deltatre. Tom Reed from Telegenic, whom I’ve known for a long time, said ‘this has re-energised me.’ All the contractors had to work really closely together.
“Part of the challenge was joining the real world with the AR world, and everything has got to be extremely accurate and precise as to how it joins up, with perspective tracking between virtual cameras and static cameras. And then you add on the AR of the ceiling, which is absolutely virtual and has to match in with all the screens below, with colour balancing.
“The vision engineers and shaders have done a fantastic job,” said Pearce. “We’ve really been through the depths where everyone has thought, ‘oh my God this will never work’ to ‘my God we’ve done it!’. You had a load of people who have never worked together before, and you’re trying to do something new and on the edge. People were saying, ‘I don’t think this can be done’ – and that was two weeks out! From a technical point of view, we’re very proud of the studio.”
At the IBC and around the country
In total ITV Sport has around 150 people in Russia, plus some ancillary staff for security and translation. Within the IBC, which is located at Moscow’s Crocus Expo International Exhibition Centre, Gearhouse has provided ITV with facilities including production offices, a master control room and a transmission gallery. This is then linked to ITV’s studio in Red Square.
Work on the studio took over a year, with a mock-up built in London before everything was shipped to Russia to ensure there would be no delays once it arrived on site. There is also a full talkback capability between the studio and ITV’s production team at the IBC.
At the IBC, ITV Sport has five Avid edit suites linked to shared storage and a media management
operation. All match feeds are recorded on a local EVS system which is integrated via fibre with the EVS servers at the Red Square Studio. Facilities supplied and supported by Gearhouse Broadcast include a separate gallery for producing the simultaneous games and an MCR which deals with the routing of all signals to and from the studio, offices in London and from the host broadcaster.
“Around the country, ITV Sport has two main reporting teams,” said Bateman, “filing via LiveU and also able to transfer files or feed from venues using HBS facilities. And ITV Sport has made some changes to our four-camera ‘fibre team’ which goes to the venues for chosen matches. It’s using host-supplied GigE circuits, with Lawo V_remotes to give multiple video feeds to and from the venue, instead of using the traditional VandA circuits from the host broadcaster.
“And for the first time we’re using Dante over fibre around the venue to provide more flexible comms and IFBs.”
Reliability and resilience of circuits is critical. Globecast is managing connectivity from the IBC in Moscow back to London via diverse routes, and providing feeds to and from the studio in Red Square. This includes a data pipe for linking EVS systems together.
“For the first time,” said Bateman, “a GigE connection has been available to the venues and
Globecast has managed this link for us using Lawo V_remote units to provide video circuits and communications. Backups for the main transmission fibres include a satellite uplink from Red Square and a low bit-rate IP connection as a final backup,” he said.
“Another thing we’re doing a lot more of here, as we did in Paris for the Euros in 2016, is linking up the EVS system here with the EVS at the studio – so it’s not just sending files or video back and forth. In Paris we had a dark fibre between the two: this time we’ve got a Gig (of our ten Gig circuit) allocated to data transfer.
“It means that if a clip comes in here, rather than taking up video circuits to the studio, if it’s not needed urgently we can just file transfer and it goes quicker than real time and pops up at the other end as a file on the EVS at the studio. That’s worked really well,” he said.
“Globecast has booked international diverse fibres from here back to the UK. The actual fibre is provided by VIDI and Globecast provides the terminal equipment at each end. And then the same to Red Square: we’ve got a Gig E and a 10 Gig, and Globecast manages that. And this time the host HBS, as well as providing VandAs to the venues, they’re also providing Gig Ethernet, which gives us more flexibility,” said Bateman.
“Gig Ethernet is expensive,” added Pearce, “but we’ve used it just to prove we could! And in fact it has worked out because we’ve got three cameras on it and comms, from the venues. That’s a good way to dip our toe into ‘almost remote’.
“We have a pitch-eye position, a position in the flash for post-match interviews and then other platforms as well – such as gantry or players’ tunnel. It means we can get all those pictures back in at the same time.
“Between tournaments, you can forget how complex and challenging it is to bring these massive events to kick-off,” said Pearce. “We always try to introduce the latest technology to support the vision of our production team, but the people we work with make all the difference when it comes to the operational phase.
“We ask a lot of them all, and we are very lucky to have built a core team of freelancers and supplier staff that want to work with us, and understand our mix of creative engineering and editorial ambition whilst working to tight timescales — and continually under high operational pressure. We hope all the team are proud of ITV’s FIFA World Cup coverage,” he said.