HVLCSR: Heuristic Visual Localization and Causal Semantic Reasoning for Multi-Label Remote Sensing Scene Classification

Multi-label remote sensing (RS) scene classification aims to describe RS images according to the contained land covers. Existing methods typically focus on two aspects: visual learning to extract class-relevant regions and semantic modeling to uncover potential label relationships. However, as observation ranges expand and spatial resolution improves, variations in the quantity, scale, and spatial distribution of land covers become increasingly pronounced, which limits the effectiveness of existing methods when applied to modern RS images. To respond to the above difficulties and improve final results, a heuristic visual localization and causal semantic reasoning (HVLCSR) framework is proposed, which consists of a multi-scale prompt-guided visual localization module (MPVLM) and a causal semantic relationship building module (CSRBM). Specifically, MPVLM first employs a non-discriminative localization mechanism to mitigate insufficient feature discriminability in RS scenarios. Subsequently, tailored regularization and multi-scale weighting schemes are incorporated to further address the challenge of large variations in land covers. Therefore, MPVLM is capable of extracting visual features associated with land covers efficiently. Correspondingly, inspired by human cognitive strategies, CSRBM progressively interprets the content of RS images by reasoning from the global scene down to specific land covers. It begins by capturing scene clues from the input image. Next, land covers are divided into typical and indistinct groups based on their inherent attributes and relevance to the identified scene. Ultimately, hierarchical “scene $\rightarrow $ typical $\rightarrow $ indistinct” semantic relationships are constructed based on the state space model. Extensive comparative experiments and ablation studies implemented on three public benchmark data sets demonstrate the superiority of our HVLCSR. Our source codes are available at https://github.com/TangXu-Group/HVLCSR