Netflix is adding high-quality audio feeds to its catalog, and introducing adaptive audio streaming to make sure its members are getting the best-possible sound based on the speed of their internet connection. The streaming service announced Tuesday that it is upgrading the quality of its 5.1 and Dolby Atmos audio tracks, with members with compatible devices now receiving audio streams with bitrates of up to 640 kbps and 768 kbps, respectively.
The inspiration for the upgrade came from a listening session with the Duffer brothers, the masterminds behind “Stranger Things,” explained Netflix production sound technology manager Scott Kramer during a recent interview with Variety. The duo was listening to a car chase from the opening scene of the first episode of “Stranger Things 2,” and something seemed off. “They were unhappy with the way their mixes sounded,” Kramer recalled. “It was just a little bit mushed.”
Related
Netflix’s team took the criticism at heart, and immediately tried to figure out what went wrong. As it turned out, the audio in question could be improved a lot by going back to the mixing board — but once Netflix’s engineers started to look into the issue more broadly, they realized that they could do a lot to improve the overall audio quality of the service.
Up until then, Netflix had been streaming 5.1 audio with 192 kbps using the Dolby Digital Plus codec. The result sounded pretty good on 5.1-capable home theater setups, but there was room for improvement. “We knew we could do better,” said Phill Williams, Netflix’s senior software engineer for audio algorithms.
That’s because with 192 kbps, Netflix wasn’t achieving what audio codec engineers call perceptual transparency, a term that describes a level of quality that even expert listeners can’t tell apart from the original master recording. During testing, Williams’ team discovered that they could effectively dupe expert listeners with 5.1 streams encoded with 640 kpbs.
Williams demonstrated the difference between these two audio bitrates during a recent listening demo with Variety that used a recording of applause as a sample sound. With 192 kbps, some parts of the audio sounded a bit wetter, less differentiated. With 640 kbps, everything was crystal clear, virtually indistinguishable from an uncompressed version.
There was only one problem: More than tripling the bitrate of the audio track could result in buffering, especially for Netflix users with slower broadband connections. That’s why Netflix decided to take a cue from the work its engineers had been doing in the video space, where the company has long optimized streaming for a variety of bandwidth constraints.
The result: Instead of just settling on one bitrate for audio, Netflix encoded each of its 5.1 audio tracks with multiple bitrates, in order to switch back and forth between them based on streaming conditions — something that’s also known as adaptive streaming. “We started making a bitrate ladder,” said Williams. He didn’t want to disclose how many different bitrates Netflix is now using for its adaptive audio streaming.
Netflix executives involved in the efforts also kept mum on future plans for adaptive audio streaming on the service, including whether they’re looking to bring any of this to mobile phones and other devices that rely on stereo audio. However, Netflix’s past efforts to optimize video streaming give us some idea of how the company may improve adaptive audio streaming over time.
Netflix has been streaming videos with adaptive streaming technology for many years now. More recently, engineers at the company realized that there was no reason to stream each and every video with the same bitrates, regardless of the complexity of the source material.
Simply put, an animated show just didn’t need as many bits as a highly complex action movie. That’s why in 2015, the company began to re-encode its entire catalog with an eye for these complexities. In the following years, Netflix went even further and encoded each and every scene in each and every video based on the complexity of the depicted imagery.
The same strategies may not translate 1:1 to the world of audio, but it’s clear that the company could make use of some of the unique qualities of each video’s audio track to further optimize streaming.
Case in point: Without commenting on the specifics of the codecs used by Netflix, Williams acknowledged that there are significant differences in the difficulties to encode different audio source material. Rock music for instance is fairly easy to encode, he said, while acoustic instruments and singing voices with a lot of expression, can be a lot more challenging. That’s why the inventors of MP3 used “Tom’s Diner,” and not “Lust for Life,” as their reference song of choice.
For now, Netflix is focusing on regular adaptive audio streaming for both 5.1 and Dolby Atmos. The company has been testing both for some months with a small subset of its audience, primarily to make sure that higher bitrates didn’t trip up the Netflix player on a wide range of devices. And in recent weeks, it quietly rolled out the new bitrates to Netflix’s entire subscriber base, safe for a few older legacy devices that don’t work with the new bitrates.
Chances are, most Netflix members haven’t noticed a difference right away — but the changes clearly haven’t been lost on some critical ears with a vested interest in the best-possible sound. Said Kramer: “The Duffers were really excited about it.”