New Quality-Of-Experience Measurement Technologies: Streamlining How Videos Are Delivered To Consumers

Video distribution services have been growing exponentially in the past decade. Cisco Visual Network Index Report predicts that globally, IP video traffic will be 82 percent of all consumer Internet traffic by 2020, up from 70 percent in 2015. Such a gigantic scale of data transmission has been supported by a vast investment of money and resources, and has also helped major industrial players grow their revenue immensely. But are consumers happy with the quality of videos delivered to their TVs, tablets and cellphones? A recent survey by Limelight Networks shows that 39 percent of video consumers are considering changing their providers in the next 12 months due to poor video quality. Major video distributors are still struggling on how to manage consumers’ quality-of-experience (QoE) when visualizing the video content being delivered to them.

To manage visual QoE, we first need to measure it, and this is exactly what’s lacking in the current industry.

Human subjective testing is not an ideal option due to the large volume of video data. Therefore, predicting human perception of video quality automatically and instantly using computing systems is the direction to go. This is an extremely challenging problem that requires deep understanding of the human visual system and advanced digital signal processing algorithms, as well as smart design and efficient implementation of the systems. In practice, quick remedies have often been used. For example, quality-of-service (QoS) parameters such as video bitrate, bit error rate, package drop rate, and network delay have been used to approximate perceptual QoE. Device playback behavior parameters such as the duration and frequency of video freezing may also be added into the equation. Such quick remedies are sometimes helpful in obtaining a rough idea about how the overall video delivery system performs, but is far from what we need in terms of accuracy and versatility to optimize video delivery systems and manage the visual QoE of individual users. For example, a common mistake of the current industry is to equate bitrate with quality, and use bitrate to define the level of services. However, using the same bitrate to encode different video content could result in dramatically different visual quality. Even worse, the actual user QoE varies depending on the device being used to display the video, another factor that cannot be taken into account by bitrate-driven video delivery strategies. 

Fortunately, innovative solutions have emerged and show great promise to solve the long-standing problem of automatic visual QoE prediction. One outstanding example is the structural similarity (SSIM) index, which uses signal processing technologies to simulate the behaviors of the human visual system in a highly efficient manner, and achieves significantly better quality prediction accuracy than traditional approaches.

The SSIM index is just the beginning of the story. Many new breakthroughs have been made on top of SSIM in the past 13 years. This has led to the recent SSIMplus method, which not only improves upon SSIM in terms of accuracy and speed, but also combats many more challenges. For example, using SSIMplus, now wee can compare videos across different spatial resolution, frame rate, dynamic range and viewing devices, a feature that has never been achieved before. It is also able to incorporate video playback parameters such as freezing and quality switching to produce highly accurate QoE predictions of streaming video on a per-user, per-device and per-view basis. SSIMWave’s real-time software products based on SSIMplus have been adopted by major broadcasters and video distributors to monitor and manage the quality of video delivered through the entire video delivery chain.

With the fast spread of automatic QoE assessment tools such as SSIMplus in real-world systems, the industry is now gradually changing its mindset from bitrate-driven to quality-driven, such that every component in the whole video distribution system is designed and operated to best serve the purpose of delivering the optimal visual QoE to end consumers.

The benefits of streamlining the video delivery ecosystem this way are manifold: 

• Consumers will be able to experience better quality videos.
• Improved user experience results in improved user engagement, which benefits the video distributors for better video monetization.
• Content producers and video distributors are now able to set target QoE levels in their service level agreements.
•  The resource utilization of the video delivery systems may be optimized to achieve the target visual QoE service levels with the minimal resources allocated, largely reducing the overall delivery cost.

ABOUT THE AUTHOR

Dr. Zhou Wang, PhD, PEng, FIEEE, FCAE, is a professor in the Dept. of Electrical & Computer Engineering at the University of Waterloo. He, along with Professor Alan Bovik of University of Texas at Austin, Dr. Hamid Sheikh of Texas Instruments, and Professor Eero Simonelli of New York University, authored the original SSIM paper published in 2003. It has received over 14,000 citations (Google Scholar), and received the prestigious IEEE Signal Processing Society Best Paper Award in 2009 and Sustained Impact Paper Award in 2016. Additionally, the authors received a Primetime Engineering Emmy Award for their disruptive invention, which, according to the Academy of Television Arts and Sciences, has been “used to test and refine video quality throughout the global cable and satellite TV industry, and directly affects the viewing experiences of tens of millions of viewers daily”. 

You can reach Dr. Wang at zhou.wang@uwaterloo.ca or connect with him on LinkedIn.