Industry Workshop: Meta Media Foundation
Short Introduction about Meta’s Media Foundation Team and the Workshop
SVT-AV1 for Every Media Workload at Meta
Over the past years, we have invested in maturing SVT-AV1 to meet the diverse encoding requirements across Meta — from video-on-demand and still image compression (AVIF) to real-time communication and on-device upload. This talk summarizes the key optimizations, quality improvements, and integration work that brought SVT-AV1 to production readiness for each of these use cases, and shares results and lessons learned along the way.
Adopting AVIF at Scale: 2 Years Later
Optimizing the image experience across Meta’s Family of Apps comes with challenges that are hard to find anywhere else. Every day, we handle billions of image uploads and trillions of image download requests on our CDN. This means that image compression is more important to us than ever in the world where visual quality matters but also where every millisecond of latency counts. In this talk we’re going to share the latest updates on adopting AVIF in our Image Infrastructure, deep dive into codec performance aspects that matter in different parts of our image pipelines and present challenges we encounter with image quality understanding between formats.
Scaling Visual Metadata and Metrics: Powering the Video Quality Flywheel
Meta processes billions of user-generated videos, and our pipelines generate a variety of metrics and metadata to optimize encoding settings and compression efficiency. We have been investing in producing these quality metrics and visual metadata — shot boundaries, borders, motion scores — as first-class signals. This talk describes how we do it at scale, and the diverse use cases that these signals enable. We begin with what it takes to make a state-of-the-art quality metric cheap enough to run on everything, drawing on recent work optimizing UVQ for scale. We then discuss how visual metadata supports video enhancement model training and inference. Finally, we trace the full loop, where quality metrics and metadata inform every stage of development, optimization, and deployment of HDR enhancement models. The result is a quality flywheel whose benefits compound — new use cases drive the development of new signals, and each new signal benefits every use case that follows.
Audio Enhancement, Understanding and Quality Assessment at Scale
Audio is a critical component of how users experience media. In this talk, we will discuss how a deeper understanding of the audio scene and its content can inform enhancement methods, leading to better downstream experiences. We will also cover how we build these models to operate at scale.
DCT Tokenized GFVC with Adaptive AC channels Updates
Generative Face Video Coding (GFVC) is a promising solution for very low bit rate talking head video communication, with wide applications in business and social media use cases. In this work we developed a light weight learning framework based on blockwise DCT tokenization and token energy profiling for group-wise partial convolution learning. This framework significantly reduces the complexity for high frame rate video deployment, with adaptive AC channel updates that signals critical detail and local info with graceful rate-distortion trade-offs. Initial results demonstrated that this solution significantly reduces complexity of the GFVC while offering coding gains in both objective and subjective loss metrics.
