Speech Emotion Classification Using Attention-Based LSTM

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

Speech Emotion Classification Using Attention-Based LSTM

Yue Xie; Ruiyu Liang; Zhenlin Liang; Chengwei Huang; Cairong Zou; Björn Schuller

Automatic speech emotion recognition has been a research hotspot in the field of human-computer interaction over the past decade. However, due to the lack of research on the inherent temporal relationship of the speech waveform, the current recognition accuracy needs improvement. To make full use of the difference of emotional saturation between time frames, a novel method is proposed for speech recognition using frame-level speech features combined with attention-based long short-term memory (LSTM) recurrent neural networks. Frame-level speech features were extracted from waveform to replace traditional statistical features, which could preserve the timing relations in the original speech through the sequence of frames. To distinguish emotional saturation in different frames, two improvement strategies are proposed for LSTM based on the attention mechanism: first, the algorithm reduces the computational complexity by modifying the forgetting gate of traditional LSTM without sacrificing performance and second, in the final output of the LSTM, an attention mechanism is applied to both the time and the feature dimension to obtain the information related to the task, rather than using the output from the last iteration of the traditional algorithm. Extensive experiments on the CASIA, eNTERFACE, and GEMEP emotion corpora demonstrate that the performance of the proposed approach is able to outperform the state-of-the-art algorithms reported to date.

SPS on Twitter

  • The SPS Webinar Series continues of 29 March when Dr. Mauricio Delbracio presents "A Walk Through Image Deblurring:… https://t.co/H1dNvuFgRv
  • COMING SOON: Join us on 9 March when Mr. Sayantan Dutta presents "Novel Prospects of Image Restoration Inspired by… https://t.co/LVYqeWEmLg
  • Happy from SPS! Thank you for doing your part towards furnishing a fairer, more equitable world for your c… https://t.co/63tIxNQQaR
  • There's still time to register your team for the 2023 IEEE Signal Processing Cup! Visit our website and register no… https://t.co/lgOQUjNPbe
  • There is still time to join the 5-Minute Video Clip Contest! Visit our website to learn more and submit your videos… https://t.co/aVUNYfTEF2

SPS Videos

Signal Processing in Home Assistants


Multimedia Forensics

Careers in Signal Processing             


Under the Radar