MSDS: Deep Structural Similarity With Multiscale Representation

Deep-feature-based perceptual similarity models have demonstrated strong alignment with human visual perception in Image Quality Assessment (IQA). However, most existing methods operate at a single spatial scale, implicitly assuming that structural similarity at a fixed resolution is sufficient, which may lead to inconsistent performance under frequency-diverse distortions, undermining the consistency of quality predictions. In this letter, we establish Deep Structural Similarity with Multiscale Representation (MSDS) as a minimal multiscale extension of DeepSSIM. MSDS independently computes DeepSSIM at multiple scales and fuses the resulting scores with a few learnable global weights, improving the robustness of single-scale models while preserving the original feature representation. Experiments on multiple benchmark datasets demonstrate that, compared with the single-scale baseline, the proposed method achieves consistent and statistically significant improvements, suggesting that learnable multiscale fusion is an effective way to improve the consistency of deep perceptual similarity models under diverse distortion conditions.