{"id":167859,"date":"2019-08-20T13:19:37","date_gmt":"2019-08-20T17:19:37","guid":{"rendered":"https:\/\/www.sportsvideo.org\/?p=167859"},"modified":"2019-08-20T13:19:37","modified_gmt":"2019-08-20T17:19:37","slug":"sports-content-management-forum-the-challenge-of-high-capacity-archive-storage","status":"publish","type":"post","link":"https:\/\/www.sportsvideo.org\/2019\/08\/20\/sports-content-management-forum-the-challenge-of-high-capacity-archive-storage\/","title":{"rendered":"Sports Content Management Forum: The Challenge of High-Capacity Archive Storage"},"content":{"rendered":"

High-capacity archive storage was the focus of a panel of industry leaders at the 2019 SVG Sports Content Management Forum, held in New York City. Among the topics were the impact of data tape, SSD, NVME, and advances in disk-drive density on the storage platform for large-scale archiving over the past year, along with cloud-based storage and where it fits into the mix.<\/p>\n

Tab Butler, senior director, media management and post production,\u00a0MLB Network<\/strong>, said that scale and time are the two biggest challenges MLB Network is facing.<\/p>\n

\"\"<\/a>

From left: Quantum\u2019s Eric Bassier, MLB Network\u2019s Tab Butler, IBM\u2019s David Taylor, Media Translation\u2019s Jay Yogeshwar, and Spectra Logic\u2019s Hossein ZiaShakeri<\/p><\/div>\n

\u201cIt seems like the hardware refresh cycle happens far too often and in too short a period of time,\u201d he said. \u201cAnd, as time goes on, we are realizing there is much more value in content we never thought of [as] being of value, and [it ends] up being just as important to the viewership as a walk-off home run. It is a situation where we now are cataloging and keeping more angles as technology makes it easier for us to create and consume content. When we started, we thought we were doing a lot by keeping four or five copies of every game, coming in [at] five hours for every hour of play. But now we are at 10 hours for every hour of play for a regular game while a showcase game will drive that up to 20 hours.\u201d<\/p>\n

The MLB Network archive is 1.2 million hours of content, with all of it available to the production team via proxy video. And, with everyone accustomed to searching for video content on their phones and finding it within seconds, having to wait three minutes for a video file to be pulled out of a system is unacceptable. That means that content historically stored on tape has to be moved to primary disk storage. In addition, those proxy-video\u2013storage systems will wear out after spinning for five years, which means that the entire process needs to be started again.<\/p>\n

Eric Bassier, senior director, products, Quantum<\/strong>, noted that a lot of customers say they have 100 PB that they need to keep forever. And part of that is due to the difficulty of predicting what content will have value at what time. \u201cAs an industry, we\u2019ve made progress,\u201d he said, \u201cbut it\u2019s a difficult problem to solve.\u201d<\/p>\n

David Taylor, executive cloud architect, IBM Storage and Software Defined Storage Solutions, IBM<\/strong>, cited the other big problem: everything is going to 4K and even 8K. He described the content as coming in \u201clike a fire hydrant with upwards of 6,000 gallons per minute and the hydrant never closes and you also can\u2019t spill a drop. We\u2019re doing a lot in AI and machine learning to identify where the valuable content is.\u201d<\/p>\n

Nick Gold, VP, marketing, Catalog DNA<\/strong>, noted the role of AI and machine learning, whereby the machines can learn to predict where certain kinds of content needs to be stored given its possible use scenario.<\/p>\n

\u201cWith a lot of the new reality-TV shows,\u201d said Taylor, \u201cthey have so much content that editors can\u2019t go through it, but they need to know what content was in focus or who was talking. It\u2019s about mining for valuable content so the editors are productive.\u201d<\/p>\n

Media Translation CEO<\/strong> Jay Yogeshwar <\/strong>said that, because archive technology will need to be replaced at some point, the goal is to figure out when that move will happen and whether a change improve workflows or create new monetization opportunities.<\/p>\n

\u201cOne of the areas I am interested in,\u201d he explained, \u201cis how to transition from one to another without causing disruption by doing bulk migrations in the background. I am also interested in virtualization with things like an abstraction layer for an archive-management team that emulates the current system.<\/p>\n

\u201cYou abstract the tape libraries and then also abstract the object storage and move the burden of the technology,\u201d he continued. \u201cThis has been a movement for a long time now, like FIMS (or Framework Interoperable Media Services). The point of that is, how can we create vendor neutral archives where tools can be created and then plugged into it?\u201d<\/p>\n

Hossein ZiaShakeri, SVP, business development and strategic alliances, Spectra Logic<\/strong>, said that he sees two tiers: production and archive, or storage in perpetuity. The production tier, which is typically the most costly, includes editing, rendering, and other functions that need quick access to files. The archive tier is where the mass is, and the more automation brought into that area, the greater the efficiencies.<\/p>\n

\u201cObject-based storage is truly the way we as the platform do that,\u201d he said. \u201cOne of the main attributes of object storage is, it is not connected to the actual storage media. It could be tape, disk, or cloud, and that is one of the great things about object storage. But it does include metadata that is abstracted from the application that brings portability and also allows for a simple API so that things can be automated when desired.\u201d<\/p>\n

Key is an agile environment that can change as needs change, ZiaShakeri added. \u201cThat is so important because things are changing quite a bit. And the key to automation is having the right tools. How do you learn about and catalog your data, and then how do you take advantage of it so you can apply the right automation process? That\u2019s where we are putting a lot of our resources.\u201d<\/p>\n

Bassier noted that it is helpful to break out what is the right medium to use and then how to manage the data on it. Tape, digital tape, disks, and hard and solid-state drives are the mediums of choice, price making solid-state drives less attractive. Cloud-based systems also rely on those mediums.<\/p>\n

\u201cTo keep content for 30 years, we believe tape is a great storage medium, but it takes time, sometimes minutes, to get files,\u201d he said, adding, \u201cBut a lot of characteristics of tape as a storage medium are very good. And, as metadata gets tied together with the content data, it becomes easier to write software that can manage that data across different storage mediums. And that is a development that will help solve our problems.\u201d<\/p>\n

Taylor noted another piece that has to be considered: how to manage the most-valuable content because customers don\u2019t want to be beholden to a cloud vendor to get that content back. \u201cDo you send proxies to the cloud but keep the asset under your own control?\u201d<\/p>\n

Butler said MLB Network\u2019s approach to the cloud involves AWS and has helped out greatly because AWS is also a CDN for the network\u2019s needs.<\/p>\n

\u201cIt puts our content at the right resolution close to our consumer, whoever it is,\u201d he said. \u201cWe\u2019ve worked with them since 2011, and all of the games are in the cloud and highly searchable for internal operations. Those files are proxy [versions], but we also have all of the proxy files on premises.\u201d<\/p>\n

One attraction of a big archive in the cloud is that machine learning can be applied to the content and a rich set of metadata created. That allows the content to be correlated with things like social media, and the correlations can be used to change local workflows, helping create automated tasks and speeding searches and discoverability.<\/p>\n

ZiaShakeri noted that some of his clients have migrated to the cloud because of all the promises surrounding it but have returned to more-traditional storage.<\/p>\n

\u201cIf you can visualize a virtual entity that encompasses on-premises storage, off-premises storage, and cloud but with the right tools,\u201d he explained, \u201cthen there is no reason certain intelligence aspects of a storage platform cannot exist on premises and in the cloud and in sync.<\/p>\n

:But the key is object-based storage,\u201d he continued. \u201cOnce you have that, there are so many different things you can do because the system has visibility of all of the assets, regardless of where they are. A framework like what we are used to when [we] google something can work with a perpetual-storage platform, and then you can decide how you want to use the cloud.\u201d<\/p>\n

Gold added that the personnel and the processes that are developed are as important as the tech stack under it: \u201cThey need just as much focus but are often overlooked.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"

High-capacity archive storage was the focus of a panel of industry leaders at the 2019 SVG Sports Content Management Forum, held in New York City. Among the topics were the […]\n More<\/a><\/p>","protected":false},"author":5,"featured_media":163715,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":"","_links_to":"","_links_to_target":""},"categories":[9248,2155],"tags":[16209,1373,16210,773,3307,673,16168],"acf":[],"_links":{"self":[{"href":"https:\/\/www.sportsvideo.org\/wp-json\/wp\/v2\/posts\/167859"}],"collection":[{"href":"https:\/\/www.sportsvideo.org\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.sportsvideo.org\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.sportsvideo.org\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/www.sportsvideo.org\/wp-json\/wp\/v2\/comments?post=167859"}],"version-history":[{"count":2,"href":"https:\/\/www.sportsvideo.org\/wp-json\/wp\/v2\/posts\/167859\/revisions"}],"predecessor-version":[{"id":167861,"href":"https:\/\/www.sportsvideo.org\/wp-json\/wp\/v2\/posts\/167859\/revisions\/167861"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.sportsvideo.org\/wp-json\/wp\/v2\/media\/163715"}],"wp:attachment":[{"href":"https:\/\/www.sportsvideo.org\/wp-json\/wp\/v2\/media?parent=167859"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.sportsvideo.org\/wp-json\/wp\/v2\/categories?post=167859"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.sportsvideo.org\/wp-json\/wp\/v2\/tags?post=167859"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}