Thursday Night Football Deep Dive: Amazon Prime Video Embraces HDR, AI, Next-Gen Stats in Year 2 of Exclusive NFL Package

A major goal is to enable fans to watch the game the way coaches and players watch it

Top Takeaways:

• All TNF games will be produced and delivered in 1080p HDR.
• The Prime Vision alternative stream will now feature Prime Targets, Defensive Alerts, Fourth Down Territory, and Field Goal Target Zone data-visualizations.
• Prime has once again deployed an intricate transmission and live-streaming scheme to offer reliability, low latency, and synchronization for viewers.
• AI/ML will power X-Ray, Rapid Recaps, and new Key Moments interactive features.

Amazon Prime Video kicks off its lone preseason game tonight in preparation for its 16-game NFL regular-season slate, and viewers can once again expect plenty of high-tech production and storytelling tools to be part of the coverage. After winning its first Sports Emmy Award following the inaugural season of its exclusive primetime NFL package, Prime is looking to crank up the tech quotient yet again in year two.

“The bar for primetime NFL coverage is really high,” says Betsy Riley, senior coordinating producer, live events, Prime Video, “and we believe that we have a strong foundation in place after last year. We have 50-plus cameras, a state-of-the-art production-truck fleet, and some very cool features and unique ways to watch the game. With all that in place, we’re now focused on how we can make football fans’ Thursday nights even more special.”

In addition to producing and distributing all games in HDR for the first time, Prime has upped the ante on its use of AI and NFL Next-Gen Stats (powered by AWS) to drive in-game data visualizations on both the primary broadcast and Prime Vision alternative feed, as well as to create automated real-time highlights throughout the game.

HDR From Glass to Glass: Prime Enters the HDR Era

After producing TNF games in 1080p HDR but distributing them in SDR last year, Prime will be end-to-end HDR this season. In the lead-up to the 2022 season, Prime worked with Game Creek Video to build the native-IP Prime One and Prime Two mobile units (along with the Prime Three office trailer), which were 4K HDR–ready from the outset. With all games produced in HDR last season, Prime conducted extensive HDR transmission tests throughout the year. However, tonight’s preseason opener will be the first TNF game to reach streaming viewers in HDR.

Amazon’s Prime One mobile units from Game Creek Video are back for their second season of TNF.

“We’re delivering games in native HDR,” says Riley. “That means from the camera lens all the way through to what the customers will see on screen is in HDR. We received feedback that our picture quality was excellent in SDR, but we think people are going to be wowed by the picture quality this year.”

Each week, Prime One (A, B, C, D, and E units) will produce the game, and Prime Two (a standalone truck) will serve TNF’s traveling pre/half/post studio show. Per Amazon’s protocols, both mobile units are fully redundant in every facet, from the routing core to the cameras. In addition to 1080p HDR video, all games and studio shows are produced in Dolby 5.1 Surround Sound. Besides the primary game feed, these trucks are responsible for producing the Prime Vision enhanced-graphics feed, Spanish-language feed, and, on most weeks, a talent-driven alternative presentation (TNF in the Shop and TNF With Dude Perfect) — all in 1080p HDR.

The Prime One and Prime Two debuted last season and are among the most technologically advanced mobile units on the road today.

Although multiple broadcasters have already been producing NFL games in HDR, Prime believes it holds a major advantage due to Amazon’s live-streaming–technology prowess and the fact that TNF’s DTC streaming model is not constrained by local broadcast affiliates and/or MVPDs, which can affect the video quality of the final product that fans see on their TVs and devices.

“Since we are a streamer,” says Riley, “we have the ability to reach our customers more directly, and we don’t have to risk degradation of signals by going through local affiliates. That affords us the opportunity to deliver these pristine pictures all the way to the home.”

A Leap Forward: AI, NFL Next-Gen Stats, and the Future of Data Visualization

After debuting to much praise in 2022, the analytics-fueled Prime Vision alternative stream will see a number of enhancements this year. Prime Vision derives stats, data, and analytics from three live-data sources to enhance the broadcast. Custom logic and real-time graphic overlays are then used to offer viewers an insightful way to view the game.

As part of the Next-Gen Stats system, RFID chips in player shoulder pads are calibrated with the dedicated all-22 camera, allowing fans to see where all skill players are lined up before each play. Prime Vision then offers real-time recommendations based on thousands of data points on game scenarios along with player and coach tendencies. Prime’s analytics team deploys AI and ML to provide viewers with deeper insights into the decision giving each team the best chance of winning in real time.

“With Prime Vision,” says Sam Schwartzstein, TNF analytics expert, Prime Video, “we’re not just providing analytics; we are genuinely telling a story with that data. We’re not throwing metrics out to the fan; they’re just watching the game in real time. They don’t want to watch a spreadsheet; they want to watch football. That’s what’s so beautiful about what this process.”

TNF’s Prime Vision broadcast will heavily incorporate “Prime Targets” graphics, previously known as Open Receiver, to identify open receivers.

This season, Prime Vision will offer four new data-visualization features: Prime Targets, Defensive Alerts, Fourth Down Territory, and Field Goal Target Zones.

Prime Targets highlights open receivers running routes down the field, and, once a receiver hits required benchmarks, a green virtual-marker graphic lights them up to indicate that they are open. Although this feature was available on the broadcast last year, Prime Targets (previously branded Open Receiver) has been improved for this season and will be deployed on all pass plays during the Prime Vision broadcast.

“This year,” says Schwartzstein, “we are tracking when a player is more likely going to convert a first down, not just when are they open. We can track this stat to figure out which players are getting open more often and which coordinators are scripting plays to get specific players open more often.

“For example],” he continues, “Dak Prescott throws into tight windows at among the highest rates [in the league]. Using Prime Targets, we can know if another player was actually open on that play but Dak chose to throw it into a tight window instead. It allows you to watch the game exactly how coaches and players are watching it.”

Prime Vision has added several new data-visualization features for the second season of TNF.

Developed in-house, the ML/AI-powered Defensive Alerts feature visually identifies defenders likely to blitz prior to the snap, allowing the viewer to watch the defense from the quarterback’s perspective. During the offseason, Amazon’s machine-learning team created a neural network to process all the players’ X/Y coordinate data generated by Next-Gen Stats tracking chips, their relationship to each other, and their acceleration in order to identify which players the quarterback and the offensive line should be most worried about blitzing.

“We can identify specific blitzing threats and where the quarterback’s eyes should be,” says Schwartzstein. “The model is able to predict blitz threats better than the NFL offensive lines and quarterbacks on passing downs. And, I have to admit, it’s actually better than me. I came up with a rule set to predict blitzes and went up against the model, and it crushed me every time.”

To identify trends, the system has been trained on over the past three seasons, and, as it tracks plays during the season, the model can retrain itself on new blitzes and new ideas coming into the league, thus improving over time. Prime has also enlisted an expert panel of former NFL players, quarterbacks, offensive linemen, and coaches to annotate plays to ensure that the model is reading each play correctly.

“We’ll never be able to predict a 100% of blitzes, but we can come close,” says Schwartzstein. “We have all this great real-time pre-snap processing [with Defensive Alerts]. Then the snap happens, and you’ll be able to see Prime Targets taking place. It’s truly a directive viewing experience that allows you to watch the game exactly how a quarterback does.”

TNF director Pierre Moossa at the front bench calling the shots

Two other new virtual graphics — Fourth Down Territory and Field Goal Target Zones — will be incorporated into Prime Vision to provide context around how and why coaches make decisions on the field.

On third downs, in addition to displaying the yellow first-down line, Prime will overlay a blue line indicating exactly where the offense has to reach to make a statistically accurate decision to go for it on fourth down rather than punt or attempt a field goal.

“Fourth Down is the place that analytics [are used] the most,” says Schwartzstein, “but TV has been explaining it reactively after a fourth-down call happens. That’s not how teams are using analytics [in these situations]; teams are being proactive. We’re going to mirror what teams do on their play calls. On third down, the analytics expert is in the coach’s headset saying, ‘If you get to this yard line, it makes sense to go for it on fourth.’ We’re going to show that in real time whenever [a team] is in fourth-down territory.”

While virtual field-goal target lines are nothing new in NFL coverage, Prime is rethinking the way they are deployed in the broadcast. Instead of a single line indicating a kicker’s career long-field or average distance, TNF will feature three lines, each showing when the field-goal–success rate is 25%, 50%, and 75%. These probabilities are determined by Next-Gen Stats, which incorporates live weather patterns and the altitude of the stadium (since the ball travels farther in high altitudes).

“We are educating the viewer about not just where the team needs to get to but how confident they should feel in making the kick,” says Schwartzstein.

Riley adds, “We want to help our audience understand how the teams are seeing the game and how the quarterbacks and coaches are hearing the game in their helmet or headset. Our goal is to bring data, information, and analytics into the broadcast to complement the storytelling so that fans can gain a deeper appreciation for what coaches and players are actually experiencing.”

Next-Level Streaming Experience: Reliability, Low-Latency, Synchronization, Interactivity

Last season, Prime presented the most exclusively streamed NFL games in history, with TNF regularly drawing audiences of more than 10 million viewers. To handle this unprecedented scale, Prime deploys multiple production and transmission hubs around the country as well as the Amazon Video Operations Center in Seattle — a technical production hub that monitors thousands of devices in real time to ensure high quality regardless of the device the viewer is watching TNF on.

“We serve more than a billion different devices [at Prime Video],” says Eric Orme, director, live events, Prime Video. “When we build things, we do it at a global scale in order to offer an error-free, high-quality experience. But, when we signed the Thursday Night Football deal in 2022, we knew it was going to be much bigger than any [live-streaming operation] we had ever done before. We invested heavily in the availability and resiliency of our services and built out new features. This year, we believe that viewer experience is going to be even better.”

Prime Video will once again roll out an onsite studio show for all 16 weeks of Thursday Night Football.

As is the case for its TNF onsite facilities, Amazon is rabid about redundancy when it comes to its transmission scheme. All four broadcast streams — primary, Spanish, Prime Vision, alternative commentary — have geographically diverse, quad-redundant transmission paths. Prime’s engineering team selects different formats and encoding profiles so that the widest possible array of devices receives an optimal video stream. As a result of this complex transmission scheme, Prime manages a whopping 144 feeds every Thursday night.

Prime has also gone to great length to reduce the latency of its livestreams by using proprietary AWS software that allows TNF to be streamed using the more lightweight UDP (User Datagram Protocol) rather than HTTP Live Streaming. As a result, Orme says, TNF streams now reach the viewer 10 seconds or less behind real time.

“That’s a massive accomplishment whether you’re talking about the internet or broadcast TV,” he says. “Not every single CDN or ISP supports UDP at this time, but we have been working closely with them to make that happen. We think about 70% or so of our streams will actually be our native ultra-low-latency experience, which is very exciting. Even if you’re in that remaining 30%, you’re still getting a fantastic experience and low latency compared with a lot of other services. And we’ll continue to shrink that latency window for our customers moving forward.”

Boasting the largest production gallery and monitor wall on the road, Prime One’s production-control room is unlike any truck-based PCR seen before it.

In addition, every frame is synchronized from source so that all devices receive the TNF feed at the exact same time, allowing fans to avoid spoilers and experience the game in unison. During live coverage, Prime’s engineering team is continually adjusting and monitoring key metrics to optimize the viewing experience.

“We can make instant automated decisions about the bitrate that you’re getting,” says Orme. “If we detect that you’re about to start buffering, we can instantly give you a lower bitrate to make sure that you don’t miss a single second of action. On a macro level, we work with a variety of CDNs and ISPs, and we’re constantly monitoring their networks so that, if their network is starting to get saturated, we can route traffic around that to make sure that our customers have the best possible experience.”

Inside the truck on Thursday night (Photo by Jason Hanna/Amazon Sports)

In addition to low latency, reliability, and synchronization, Prime has put a major focus on interactivity and is using AI-based tools to enhance the TNF viewing experience. Amazon’s X-Ray feature provides viewers with real-time access to a treasure trove of live statistics and data during the game, while Rapid Recap automatically generates more than a dozen two-minute highlights for viewers to catch up on plays during a game. New this year to TNF is Key Plays, a feature that provides viewers with in-game highlights and key moments on demand at the touch of a button.

“Key Plays is something we’re really excited for this year,” says Orme. “If you miss anything or want to go back, you can very quickly just [click a] button and see a key play versus trying to have to fumble through the [timeline].”

Orme and company will be put to an even greater test this season, when the new “Black Friday” NFL game on Nov. 24 will be available for all fans to stream for free — whether they are Prime members or not. With one of the season’s most anticipated games available outside the paywall on an unofficial U.S. holiday, the Prime team expects massive viewership.

“Because we’re a tech company,” says Riley, “we have this unique opportunity to think about football and how we deliver it to fans in a very different way. Our access to Amazon’s technological resources and ability to work closely with our engineering counterparts afford us the opportunity to think about football storytelling and the fan experience differently than I think any [rightsholder] has before.”