Like rust, compression experts never sleep. 17 years ago, a newer and more efficient codec made its debut. MPEG-4 H.264 (aka Advanced Video Codec) promised 50% compression efficiency over MPEG-2…and delivered it! By that time, HD video was becoming widely adopted across a multitude of delivery platforms, but available bandwidth was slow to keep up. With the growth of video streaming a few years later, MPEG-4 H.264 became the codec of choice and is supported on everything from tablets and smartphones to laptops, smart TVs, camcorders, and DSLRs.
The only problem was that 4K video became the latest flavor by the turn of this past decade. And to make the job even more difficult, high dynamic range video with its associated wider color gamut was part of the 4K package. But bandwidth hadn’t kept up! Hence, the High-Efficiency Video Codec (HEVC) was rolled out in 2013, again promising 50% more compression efficiency over H.264. HEVC requires quite a bit of computing power to pull off that trick, but it works (and so does its close relative, Google’s VP9 codec). And it’s not cheap to license.
HEVC is used for the UHD Blu-ray optical disc format and for streaming 4K content from just about everywhere except YouTube, which being owned by Google employs the VP9 “open” codec. But the licensing costs spurred more tech types to come up with yet another codec, known as the Alliance for Open Media (AOM) AV-1 codec. This is a royalty-free codec that competes with HEVC but is intended solely for streaming video over Internet connections.
Since the people developing high-resolution video formats always seem to be a few steps ahead of the codec people, more codecs have been proposed in an attempt to catch up. Essential Video Coding (EVC, or MPEG-5) was developed as an alternative video codec for streaming and OTT, but with a streaming performance at least equivalent to HEVC. MPEG-5 Part 2, Low Complexity Enhancement Video Coding (LC-EVC), is yet another MPEG standard and is intended to provide enhanced compression efficiency for existing MPEG-based video codecs.
To top it off, the successor to HEVC H.265 is now preparing to take the stage. The Versatile Video Codec, or VVC, is designed for maximum compression efficiency across all compatible devices and platforms with a specific focus on applications like high-dynamic range, high frame rate video, the 360-degree video for virtual and augmented reality, and 8K UHD-2 video.
Using spatial-only image metrics for reference, VVC is about 40% more efficient than HEVC for UHD & HD compression. However, a VVC reference encoder has about ten times the complexity of an HEVC reference encoder, while a VVC reference decoder is about twice as complex as an HEVC reference decoder.
We should mention that HEVC, AV-1, EVC, LC-EVC, and VVC are software-intensive codecs, unlike the older MPEG-2 and H.264 AVC codecs that are still in widespread use. All five employ larger coding block sizes and need super-fast CPUs and plenty of memory to analyze and compress video streams. By the way, H.264 AVC is no slouch – it’s had 26 updates since it was first rolled out in 2003.
Okay, so you’re getting a headache from all these abbreviations. (We are, too.) The takeaway here is that methods for enabling the transport of compressed, high-bit-rate video over everything from broadcast airwaves, Wi-Fi, 5G, and broadband are continuously being refined. You’ll go through life blissfully unaware of which particular codec is being used to stream The Marvelous Mrs. Maisel or let you watch Clemson’s and Alabama’s football teams slug it out on your iPhone. (You will, however, notice any impairments to video quality as a result of excessive compression and complain vigorously to your service provider about them!)
Will there ever be a unified codec? That’s the goal, but no one knows how, when, or even if it will happen. In the ongoing quest for efficiency, codec designers are now implementing artificial intelligence (AI) to perform the lightning-quick analysis of incoming video and decisions on which picture elements to compress, by how much, and for how long.
We already use a simple version of AI to dynamically adjust bit rates of multiple programs in a stream, based on constantly-changing available network bandwidth (dynamic stream shaping and adaptive bit rate encoding are two examples). It stands to reason that a unified codec – one based on advanced AI that can optimize the delivery of high-resolution video across any network or platform – should emerge at some point and rid us of the “alphabet soup” of codec formats.
Dream on…