Skip to main content

Asymmetric Cross-Scale Interaction for Compact Speech Emotion Recognition

By
Yijie Zhu; Ruijun Gu; Wenyang Bi; Yu Han; Zhenghao Xie

Speech emotion recognition (SER) requires transient acoustic cues and longer temporal context. We investigate whether coarse-scale context can improve recognition by updating pooled summaries. Our asymmetric cross-scale Mamba network (ACS-Mamba) transfers fine-scale sequence features to coarser resolutions and then calibrates utterance summaries from coarse to fine. The default sequence backbone has width 32, including when 768-dimensional pretrained features are supplied. With mel-frequency cepstral coefficients (MFCCs), ACS-Mamba achieves 72.30% unweighted average recall (UAR) on four-class IEMOCAP with 0.087 M parameters. Across five session-disjoint folds and three seeds, it improves UAR over flat fusion with matched fusion parameter counts by 0.75 percentage points. Frozen emotion2vec-base features yield 79.04% IEMOCAP UAR and 49.57% MELD weighted F1 with 0.230 M downstream parameters. At 606 valid frames on a Tesla T4, the MFCC model reduces median latency by 21.8% relative to cross-scale attention with comparable recognition accuracy. These results support sequential summary calibration for compact SER.

Read on IEEE Xplore