Skip to main content

Very-Low-Complexity Deep Reference Frame Generation for VVC Inter Prediction

By
Tian Shu; Xinxin Chen; Wenzhuo Zhang; Nianxiang Fu; Zhenzhong Chen

Deep reference frame (DRF) generation improves VVC inter prediction by synthesizing additional reference pictures. Existing Neural Network-based Video Coding (NNVC) DRF configurations provide high- and low-complexity operating points, but their model complexity remains high for resource-constrained coding scenarios. This paper presents very-low-complexity DRF (VL-DRF), a configuration adopted by the Joint Video Experts Team (JVET) for VVC inter prediction and the first neural inter-coding tool enabled by default in the NNVC reference software. VL-DRF reduces complexity by integrating multiscale feature extraction and coarse-to-fine bidirectional flow refinement into a unified synthesis network, eliminating the external optical-flow module. To improve practical inference efficiency, VL-DRF uses channel widths aligned with the target backend's SIMD granularity constraints. A selective temporal-layer activation strategy further controls aggregate workload by jointly accounting for synthesis difficulty at larger temporal distances and generator invocation frequency. Under the NNVC-17.1 random-access configuration, VL-DRF achieves BD-rate savings of 0.55%, 0.59%, and 0.52% for the Y, U, and V components, respectively, using 9.7 kMAC/pixel and 0.257 M parameters.

Read on IEEE Xplore