The production Media Data Center: end-to-end monitoring and orchestration for all-IP broadcast infrastructures

This article discusses the main challenges on the way to an ALL-IP uncompressed video infrastructure and the way a latest-generation end-to-end multi-vendor orchestration platform helps to deal with them.

Introduction

With the ongoing migration in the broadcast industry towards an ALL-IP infrastructure, media companies are facing tremendous new challenges. The move to ALL-IP is not only another shift in technology, like the migration from SD to HD for example. This time it is not about replacing boxes and SDI with ethernet cables. Media over IP enables companies to leverage new technologies and redefine operational practices. COTS hardware that supports multi-format media can be implemented and both the network and the applications can be virtualized. The next logical step is then to deploy those media solutions in the cloud. However, the faster technology evolves, the more flexibly media companies will have to act and adapt their operational procedures, which also means that the demands for the management of such a transition get all the higher. End-to-end visibility of your operational environment and the possibility to orchestrate resources and services dynamically are key to making the transition successful.

Challenges

SMPTE 2022-6 has been out there for a while and the SMPTE 2110 set of standards have been published recently. Data plane interoperability has already been adopted well in the media industry. However, when it comes to the orchestration part, a lot of work remains to be done. The industry has yet to adjust to NMOS IS-04, to discover and register compliant streams, and the other NMOS standards (IS-05 and IS-06) for connection management and network control to reserve and secure services are not yet finished.

The fact that linear media flows, which need to transmit in real time and switch quickly and frame-accurately, use a network that has a non-linear nature leads to a situation where those network technologies and topologies do not always behave as they should. Frame drops, packet delay variation, bursts, latency and jitter have to be measured carefully, and new tools are also required to identify signals and their behavior within multicast streams.

Content and network security is another topic that has to be addressed in an ALL-IP world. Ecosystems are not closed any longer: on-premises, off-premises and hybrid installations are the new default setup. Fine-grained authorization and authentication mechanisms as well as IT cyber security guidelines are crucial to achieve a high level of protection for high-value media content.

Today there is no “one-fits-all” workflow to switch uncompressed video-over-IP streams. The industry still settles for best practices. IGMP or static multicast routes with source-, destination- or switch-based signal switching lead to a vast variety of technical solutions. Vendor proprietary SDN controllers create new silos in operations and very often only support non-blocking spine/leaf or monolithic network topologies.

An end-to-end orchestration layer is required to address those challenges, in order to to allow media companies to truly benefit from a real-time production media data center.

A solution – workflow orchestration for media over IP MCR operation and live productions

Media flows and services need to be orchestrated end to end in multi-vendor environments. For maximum agility, such an overlying layer needs to run unconditionally and fully transparently across any vendor and technology boundaries. It must be aware of all resources, their capabilities, availability and constraints. To orchestrate a modern media over IP environment, capacity needs to be managed in both blocking and non-blocking network topologies to avoid any oversubscription of the available network bandwidth. Security and access control mechanisms need to be implemented together with a centralized management of resource pools, which can be legacy hardware, software or even virtual resources like pools of multicast addresses.

In a typical MCR operation, most feeds, compressed or uncompressed, and their associated resources can be booked ahead of time and neither are time critical nor have to be frame accurate. Such a scheduled operation leaves enough time for an orchestrator to make thoughtful decisions. Capacity and resource availability are guaranteed at any given point in time. An orchestrator manages all resources directly, and additional third-party SDN controllers are not involved.

However, for live productions today, the requirements are different. Most uncompressed video-over-IP implementations are based on a non-blocking infrastructure. This relies on the fact that typical vendor-specific broadcast controllers for ALL-IP do not handle capacity and the underlying network topology constraints. This leads to a massive bandwidth underutilization and overprovisioning of the overall network infrastructure for most of the daily situations in live production. At the same time, the need for fast switching with constant execution times is contradictory to a blocking network environment. It is nevertheless possible to bring these needs together.

While a real-time controller sends commands to physical or logical devices with no guarantee that the actions can and will happen, an orchestrator is deterministic in that it guarantees capacity and resource availability. An orchestrator also books and guarantees capacity ahead of time and manages the full stack of applications and all underlying network elements and physical layers. Many SDN controllers available for media applications are tied to specific technology from a single vendor, which conflicts with a modern multi-vendor approach, which is vital in an ever faster evolving technology landscape. An open independent orchestrator can interoperate either with every product directly or with multiple SDN controllers from different vendors.

For live production with uncompressed video over IP, both worlds of SDN controllers and orchestrators need to be combined in a meaningful way, especially in a blocking network infrastructure: thoughtful decision-making of an orchestrator together with an SDN controller that executes quickly and frame-accurately for well-defined scenarios, such as an upcoming studio production.

The overarching NMS orchestrator acts as a master and forwards only those sources and destinations that are required for a specific scenario to one or more underlying real-time SDN controllers. It calculates a worst-case scenario by taking all possible connections into account with their bandwidth requirements. After that, this SDN controller can execute switch commands for uncompressed multicast flows as quickly as possible and deliver a constant execution time in that predefined non-blocking environment, without having to check capacity or other constraints. In case additional sources or destinations are required for a scenario, the SDN controller can ask the orchestrator for the additional resource and bandwidth, or the signal switch can be executed directly on the orchestrator (including some delay as the orchestrator has to do a capacity and resource check). This actually creates a non-blocking island in an overall blocking network infrastructure and ensures that multiple SDN controllers can be combined into a single ecosystem.

For a successful operation in ALL-IP, it is also essential to identify the video and audio signals in a media network for monitoring purposes. A multi-layer approach is required to visualize those streams from source to destination. Flow management (NetFlow SFlow) to detect streams in combination with edge device information and routing tables from the switch fabric must be united in a single solution and user interface that can provide an easy overview of flows and that allows operators to investigate the root cause of potential stream issues.

On top of this, both content and network security need to be kept in mind as well. An umbrella orchestrator with built-in workflow automation is the foundation for fine-grained authorization and authentication mechanisms towards the media over IP network, both on operator and on device level. New devices can be automatically detected and configured with a baseline setup before they are added to the production network. The use of ACL lists, secure VLANs for device provisioning and other SDN network access control protection mechanisms, like automatically turning off SFPs when they are not in use, significantly help to adapt to standard IT cyber security guidelines.

Last but not least, for true 360° visibility and management of a media facility, the operations and business layer must be taken into account as well. An uncompressed video network does not stand on its own any longer. To truly allow end-to-end orchestration, it needs to be integrated with ingest and playout, media asset management systems, job management, and ticketing and billing platforms, just to name a few.

More than ever, a single consolidated fully independent and agile NMS/OSS platform is a strategic key to successfully make the transition to media over IP in the broadcast industry. It is an effective response to all key challenges any broadcast and media operation is facing.