Optimal Prompt Guided Multimodal Learning for Blind Omnidirectional Image Quality Assessment

Multimodal learning has garnered significant attention in the era of large models, particularly for their remarkable zero-shot generalization capabilities. This approach has also been extensively explored in the field of image quality assessment. However, its application in blind omnidirectional image quality assessment (BOIQA) remains relatively underexplored. Additionally, most existing BOIQA models rely on computationally intensive viewport or patch sequence generation, thereby limiting model flexibility and efficiency. To address these challenges, we propose an intuitive and effective BOIQA model with Optimal Prompt guided Multimodal Learning (OPML) that processes raw omnidirectional images in Equirectangular Projection (ERP) format without requiring any transformation. Specifically, OPML consists of an optimal prompt selection module for eliminating quality-irrelevant information, which mitigates the limitations of objective models when encoding lengthy text descriptions, and a hierarchical visual degradation modeling branch for capturing visual quality features, along with a multimodal quality capturing module for fusing multimodal features and predicting the global quality of an omnidirectional image. The proposed model demonstrates competitive performance on two large-scale OIQA databases, outperforming most state-of-the-art methods, and also shows great generalizability in group Maximum Differentiation (gMAD) competition methodology.