Onward and Upward

We recently attended the annual Society of Motion Picture & Television Engineers technology conference in Los Angeles. SMPTE has been holding this event for decades and it attracts the best and the brightest to talk about advancements in everything from television and motion picture production to human visual science, advances in audio, video compression and transmission, and lately, promoting the field to women and college graduates.

One invited papers session, in particular, stood out this year, and its focus was on 8K television. Regular readers of this blog may remember that much of the research and development in this area is being undertaken by NHK, the national Japanese broadcasting network. NHK commenced their work way back in 1995, first achieving 4K resolution in camera sensors in 2004 and then introducing their first 8K camera sensor in 2006.

Since then, they’ve designed and built a 4-pound 8K camera system, pushed sensor speeds to as high as 480 Hz, and created a simultaneous downconversion product that ingests 8K video and spits it out at native resolution, 4K resolution, and Full HD resolution. As you can imagine, that requires a ton of processing power!

At this year’s session, one speaker from NHK detailed their work in next-generation camera sensors for 8K that incorporate a unique organic photoconductive film (OPF) layer to boost sensitivity while keeping noise to a minimum. That’s a real challenge when working with small camera sensors – Super 35 sensors for 4K (Ultra HD) production are already jammed full of tiny photosites, a design headache for camera engineers. Now, imagine you’re asked to double the number of pixels in the same size sensor, where the average pixel measures about 3 micrometres!

The second paper described a new 1.25” 8K camera sensor that can record video at frame rates as high as 480 Hz, or eight times as fast as conventional sensors. Using this sensor, fast motion can be captured with minimal blurring and very fine detail. The captured video is down-converted in-camera to 120 Hz for eventual recording and playback. As you might guess, the data flowing from the camera sensor is a gusher: Uncompressed, with 10-bit 4:2:2 color sampling, it approaches 100 gigabits per second (Gb/s), or more than twice as fast as the latest version of HDMI (2.1) can handle.

The final NHK paper talked about setting up the world’s first full-time 4K/8K satellite broadcasting system, which was launched in December of 2018. Aside from the technical challenges of bandwidth (both left-hand and right-hand circular polarization of the radio waves was necessary to carry all of the signal data), there was an additional obstacle: Many residents live in older apartment buildings, making the cable infrastructure upgrade process difficult. It was eventually solved by installing parallel lines of plastic optical fiber (POF, or Toslink) alongside existing coaxial cable, telephone, and power lines.

Where is the relevance to our industry? Consider that, ten years ago, Ultra HD and 4K video was largely a lab experiment to many of us. In 2019, we were just getting used to managing Full HD signals over signal distribution and interfacing systems, wrestling with color bit depth and varying frame rates, not to mention the limitations and foibles of HDMI.

Yet, three years later, the first commercial Ultra HD monitors washed up on our shores. A decade later, Ultra HD became the default resolution for most commercial AV monitors and displays for consumer televisions. Just as we did in 2009, we’re wrestling with the same signal management issues, color bit depths, refresh rates, and a whole new version of HDMI…which isn’t even ready to handle the higher bit rates that 4K video requires for higher frame rates and higher color bit depths.

So, while we fuss, argue, complain, and try to adjust to this latest jump in resolution to 4K, there is a country that is already working at TWICE that has video resolution for acquisition, editing, storage, and distribution to the home. There’s no reason to think that we’ll catch up to them eventually – the first 8K televisions already launched to the North American market earlier this year, and we’re seeing early interest in 8K displays for specialized installations like command and control, surveillance, visualization and augmented reality, and (of all things) 3D visualization using autostereo displays.

Skeptics can scoff all they want, but this never-ending push upward and onward in spatial resolution isn’t going to stop. If anything, additional momentum will be provided by enhancements like high dynamic range, wider color gamuts, and high frame rate video. (Did you know that, as display screens get larger and fields of view become wider, any flicker in images created by judder and slower frame rates becomes increasingly noticeable? NHK studied this phenomenon and concluded that a minimum frame rate of 80 Hz was required for 4K and 8K on large displays.)

And as usual, we’ll be expected to interface and transport these signals. The SMPTE SDI standard for UHD (12G SDI) is already inadequate for signal-wire serial digital connections, having a maximum data rate of 11.88 Gb/s. This has resulted in 8K camera manufacturers employing four separate 12G SDI ports and some light compression in-camera to record 8K/60 video with 10-bit 4:2:2 color (uncompressed data rate of 47.7 Gb/s). And it’s also revived interest in the latest SMPTE standard for SDI, 24G (23.76 Gb/s, likely over optical fiber).

This should be interesting to watch, particularly since our industry is still working around the six-year-old TMDS-based HDMI 2.0 interface (18 Gb/s), is still largely allergic to optical fiber, and is promoting an AV/IT video codec that’s barely fast enough to squeeze 4K/60 10-bit 4:4:4 video through a 10-gigabit network switch.

Do you feel the need for speed yet?