SVG Tech Insight: Control and Monitoring in Distributed Sports Production

This spring, SVG will be presenting a series of White Papers covering the latest advancements and trends in sports-production technology. The full series of SVG’s Tech Insight White Papers can be found in the SVG Spring SportsTech Journal HERE.

Introduction

Sports broadcasting is one of the most innovative parts in the industry in terms of production values and adoption of leading-edge technology.

For example, it has been pioneering IP remote production to produce high-quality live events with less equipment and fewer people onsite, thereby reducing costs. This also enables production staff to produce more than one event or show on the same day without having to travel, while using for their remote productions the same equipment and systems they would for their in-house shows.

Although, remote production is just one aspect of an even bigger trend happening in the industry. With IP technology now being used in production facilities (LANs) and between facilities (WANs), every production facility, every piece of equipment, and every production professional can be involved in the production of live content — almost regardless of their geographical location. Add to this the storage and processing capabilities offered by the cloud, and genuinely distributed production can now become a reality.

Management is key

Creating the technical infrastructure to deliver the benefits of distributed production for sport coverage is not trivial.

The focus is often on the physical media transport (video, audio, ancillary data, intercom), and associated considerations around standards (e.g. SMPTE ST 2110), optimum encoding (e.g. JPEG XS) and protection (e.g. SMPTE ST 2022-7) amongst others.

However, the management layer is often overlooked, or at least underestimated. And yet, that layer is critical, not least because setting up and configuring equipment is potentially tricky (especially in an IP environment where very many IP and multicast addresses are involved for example), as well as time-consuming. These issues stand in the way of achieving some of the main objectives of remote production, i.e. reducing the number of people onsite, and being quick and nimble.

Management is a wide-ranging

Management covers a deceptively large range of requirements, relating amongst others to setting up and configuring equipment; controlling and monitoring that equipment; and orchestrating and securing media flows across the media network.

For example, broadcast control includes, amongst others, tally/UMD management, alias management, endpoint parameterization, and scheduling. Media network management includes SDN (software defined networking) control, network configuration, PTP configuration, IP address management, device discovery and configuration (NMOS), and more. Monitoring also covers aspects such as alarm management, status information, network monitoring, PTP monitoring, analysis and trending, troubleshooting, logging/accounting, and so on.

All of these are complex enough in a contained environment — such as an OB truck — but they become even more challenging in a distributed environment like remote production.

Multi-segment management

In its simplest form, remote production involves two locations (the location where the event takes place and the central facilities), connected by a wide area network (WAN). So, from a control and monitoring perspective, such a production involves three distinct “segments” that need to be managed both individually and together. More complex productions, for example where multiple games are taking place simultaneously in different locations, could obviously involve many more segments as well as other technologies such as mobile 5G and Cloud — adding greatly to the management complexity.

In considering the management layer for a particular remote production, it’s important to ascertain in the first instance where the various production staff will be located. In a remote production, most of the equipment will be located in the central facility, as sharing resources is one of the benefits sought. However, when it comes to production staff, while it may be desirable to have them also in the central facility, it may also not be possible for logistical reasons — for example where the producer needs to liaise in person with the event organizers. This obviously creates quite a challenge in terms of workflows and therefore media flow orchestration, as the equipment and the people controlling it will be separated geographically by many miles and technically by a WAN.

The next consideration is that each of the segments mentioned above will have its own specificities, which will influence how it can be managed.

The WAN connection for example will typically be based on IP technology — since the telcos providing that network connectivity migrated to IP at the turn of the century. But the level of the control over the network available to the broadcaster or production company will vary greatly. In some cases, it will be possible to control the path of the signals through the network (SDN) to ensure true path diversity (SMPTE ST 2022-7). In most situations, though, the network connection will be offered “as-is” by the telco with no option to control flows inside the WAN.

The event site and the central facilities themselves may be based on IP or SDI technology, or indeed a mixture of both. This means that the management needs to be flexible enough to manage production seamlessly across technologies — adding further complexity to the challenge.

A further issue is that the setup in the location could be permanent, semi- permanent, or temporary. Clearly, the less permanent the arrangement, the more dynamic the management needs to be and — as mentioned before — the simpler and automated the setting up and configuration must be — as it will probably be carried out by non-specialists onsite. This is where, for example, advanced IP address management can play a key role, by dramatically simplifying handling of the thousands of multicast addresses required for the multitude of simultaneous audio, video, and data streams.

And of course, total reliability is paramount, as it’s expensive to sort issues out remotely.

Management options

With the right facilities management and broadcast control products, it is possible to extend the management capabilities from local to remote locations. So, for example, Sony’s Live System Manager (LSM) can perform XPT control, device monitoring, and tally distribution remotely, across an IP WAN.

When it comes to managing the actual flows of signals, various options are available.

The simplest option in some ways is to have a static set-up — meaning that the central facilities and the remote locations are effectively “married”. Simplicity comes at cost though: changing or expanding the set up — for example, reducing or adding signals — is time-consuming and costly. NMOS IS-04/IS-05 is likely to be required, to enable the joint-requests to be carried out.

The next option is to add an orchestration and SDN control capability, such as Nevion’s VideoIPath, to enable the dynamic set-up of streams across the WAN. This makes it easier to add or remove streams. It also enables multiple locations to be connected dynamically through the same gateways. The local networks (onsite and in the central facilities) remain statically configured.

Building on this, it is possible to bring in one overall management system, controlling the network end-to-end (the local networks on site and centrally, and the WAN connectivity). This allows all streams to be connected dynamically, providing full redundancy not only for the video and audio signals, but also for data. This solution is perfect for remote production but has limitations in the context of a distributed production (e.g. where multiple studios are connected across the WAN): as the management is centralized, the individual locations cannot easily operate independently.

This leads to a final model, which involves a federated management, i.e. separate instances of the management system in each location, collaborating together to manage the flows end-to-end. Not only does this gives more flexibility in managing the locations both independently and together, but also provides additional resilience by distributing the management responsibility.

Conclusion

Remote and distributed production brings undoubted benefits to broadcasters in terms of cost savings and nimbleness. While much of the focus in creating the infrastructure to support these is on the media transport issues, management is probably the most crucial and complex aspect. If done wrong, it can create costly complexities that will reduce or even negate the benefits of the remote or distributed production. With the right approach, the right systems, and the right expertise, media network management can create an environment in which remote locations can be considered simply as extensions of the central facilities, ensuring that not only the expected benefits are realized, but also that the production values are unmatched.