SVG Tech Insight: Inserting AR Into the Remote Production Workflow

This fall SVG will be presenting a series of White Papers covering the latest advancements and trends in sports-production technology. The full series of SVG’s Tech Insight White Papers can be found in the SVG Fall SportsTech Journal HERE.

Augmented Reality (AR) provides numerous compelling opportunities for broadcasters to differentiate their coverage, especially in the areas of explanatory graphics and/or sponsorship opportunities. To date, however, such solutions have involved specific crew for rigging, equipment that requires precious rack unit space, and a lengthy set up process, all of which translates into additional costs and time penalties for productions.

Using first-generation technology, typically the load out would involve one operator per camera, in addition to the camera person themselves. There was also a penalty in terms of equipment; our first-generation camera bar that was mounted onto the camera itself weighed 2.65lb (1.2kg) and the rack-mounted server units that accompanied each unit weighed 24.25lb (11kg). The camera bar was also required to be tethered to the server via ethernet, further decreasing flexibility and choice in deployment.

While not hugely problematic for many AR use cases, such as realtime pre-visualization on movie sets, the increased footprint that AR’s deployment necessitates when it comes to live sports and events has seemed increasingly out of step with the meta trend in the industry towards remote production. Sport OB in particular started to accelerate its pivot towards remote workflows throughout the course of the past two years driven by the twin motors of cost-effectiveness and increasing environmental considerations and/or legislation.

Our analysis showed that what was required to keep AR in the spec sheets and at the forefront of future production’s plans in terms of the live sports market was:

• Miniaturization of equipment
• Lowered bandwidth requirements
• Remote operation
• Decrease of personnel
• Swift deployment

This development effort was already well underway before the COVID-19 pandemic added an urgency to remote production adoption. The resulting acceleration of the R&D cycle means that those issues have all been addressed and that AR can now be comfortably deployed as part of best practice remote workflows due to impressive innovation in the following fields. Indeed, these apply to much of the equipment that will be considered by OB companies over the coming years as remote production workflows journey from mainstream acceptance to majority deployment.

Miniaturization

Form factor is an issue when it comes to OB manifests, with smaller being better and enabling less vehicle transport. Weight is an issue when it comes to number of crew required to maneuver kit and, in AR’s case, in the feasibility of camera-mounted equipment.

Progress here has been rapid. The Mk2 version of our camera bar, for instance, weighs a mere 0.64lb (290g), while the processor has been shrunk down to 1.98lb (900g). This means it can be camera mounted too, doing away with the previously required ethernet tether. This also makes AR deployable on wireless handheld units, Steadicam, spider cams, etc. It also gives productions the flexibility to use the same unit both in the studio and pitch-side.

Lower bandwidth

By moving many of the processing requirements of the AR workflow onto an onboard unit, the bandwidth requirements of the link from the camera have been shrunk by a factor of close to 1,000. This means that AR units can be added to the remote production workflow with no significant overhead on bandwidth requirements. Indeed, the raw data requirement for an AR workflow in this case is as low as 16KBps per camera.

The heavy lifting part of any AR workflow occurs at the rendering stage when the data from the real world image is combined with the graphics elements for output. This can occur at the production hub alongside other operations such as switching, EVS, audio mixing, etc.

Remote control

One of the goals of remote production is to have steadily less kit at the ground and that which is essential to be there become increasingly controlled remotely by operators. Remote camera positions are becoming increasingly normalized, while kit that was previously considered essential onsite, such as EVS, is now routinely kept back at base. The same is true for technologies such as AR. This needs to fulfill set-and-forget requirements to be successful, with the solution engineered so it can be controlled — and trouble-shot if necessary — remotely from base or another cloud-based site.

Lowered footfall

The obvious desired knock on effect from this is a lowered need for physical presence at the site. Pre-COVID this was primarily on a cost and environmental basis; post-COVID, this is a socially distanced requirement. Rigging and operation are increasingly tasks undertaken by the same crew and equipment needs to be designed to facilitate a low headcount, in terms of both its set up and its operation. In keeping with the remote operation requirement detailed above, once our AR solution/s are set up by an engineer, they can leave the site.

Fast deployment

If specialists have to be onsite, the current environment dictates that the sooner they can be offsite, the better. Set and forget is less of an appealing workflow paradigm if the set process takes hours and hours to complete. Solutions need to be engineered for fast and accurate setup and calibration. Progress here in the AR field has been rapid; with one broadcaster we are working with deploying our solution, we are able to have a virtual studio up and running in over 90 possible locations in under an hour.

These and other developments mean that AR does not have to be considered as a pre-COVID addition to a broadcast that now has too much overhead and operational risk to be considered viable. Rather, broadcasters can not only add AR elements to their productions, they can ramp them up and deploy the technology safely under pandemic conditions — either purely informational or increasing advertising and sponsorship opportunities — without increasing onsite footprint. This is not only an immediate benefit for them now during the current pandemic as they look to gain traction for the returning leagues, but will prove a significant asset in helping ensure the success of the carbon neutral productions that will be required in the future.