Speech Emotion Classification Using Attention-Based LSTM

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

Speech Emotion Classification Using Attention-Based LSTM

Yue Xie; Ruiyu Liang; Zhenlin Liang; Chengwei Huang; Cairong Zou; Björn Schuller

Automatic speech emotion recognition has been a research hotspot in the field of human-computer interaction over the past decade. However, due to the lack of research on the inherent temporal relationship of the speech waveform, the current recognition accuracy needs improvement. To make full use of the difference of emotional saturation between time frames, a novel method is proposed for speech recognition using frame-level speech features combined with attention-based long short-term memory (LSTM) recurrent neural networks. Frame-level speech features were extracted from waveform to replace traditional statistical features, which could preserve the timing relations in the original speech through the sequence of frames. To distinguish emotional saturation in different frames, two improvement strategies are proposed for LSTM based on the attention mechanism: first, the algorithm reduces the computational complexity by modifying the forgetting gate of traditional LSTM without sacrificing performance and second, in the final output of the LSTM, an attention mechanism is applied to both the time and the feature dimension to obtain the information related to the task, rather than using the output from the last iteration of the traditional algorithm. Extensive experiments on the CASIA, eNTERFACE, and GEMEP emotion corpora demonstrate that the performance of the proposed approach is able to outperform the state-of-the-art algorithms reported to date.

SPS on Twitter

  • The IEEE Journal of Selected Topics in Signal Processing is now accepting papers for a Special Issue on Learning-Ba… https://t.co/ZWLSpRNAPz
  • Last chance to vote for your favorite video in the IEEE SPS 5-Minute Video Contest! Check out the videos from our t… https://t.co/GeHlfhSeYi
  • CALL FOR PROPOSALS: The IEEE Workshop on Automatic Speech Recognition and Understanding is now soliciting proposals… https://t.co/gzYreLyroa
  • authors have started uploading their conference slides and posters to IEEE SPS SigPort! Get a sneak pea… https://t.co/XGvnfdrHIb
  • DEADLINE EXTENDED: The IEEE Journal of Selected Topics in Signal Processing is accepting papers for a Special Issue… https://t.co/E89M7bEFlu

SPS Videos

Signal Processing in Home Assistants


Multimedia Forensics

Careers in Signal Processing             


Under the Radar