Time-Frequency Masking Based Online Multi-Channel Speech Enhancement With Convolutional Recurrent Neural Networks

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

Time-Frequency Masking Based Online Multi-Channel Speech Enhancement With Convolutional Recurrent Neural Networks

By: 
Soumitro Chakrabarty; Emanuël A. P. Habets

This paper presents a time-frequency masking based online multi-channel speech enhancement approach that uses a convolutional recurrent neural network to estimate the mask. The magnitude and phase components of the short-time Fourier transform coefficients for multiple time frames are provided as an input such that the network is able to discriminate between the directional speech and the noise components based on the spatial characteristics of the individual signals as well as their spectro-temporal structure. The estimation of two different masks, namely, ideal ratio mask (IRM) and ideal binary mask (IBM), along with two different approaches for incorporating the mask to obtain the desired signal are discussed. In the first approach, the mask is directly applied as a real valued gain to a reference microphone signal, whereas in the second approach, the masks are used as an activity indicator for the recursive update of power spectral density (PSD) matrices to be used within a beamformer. The performance of the proposed system with the two different estimated masks utilized within the two different enhancement approaches is evaluated with both simulated as well as measured room impulse responses, where it is shown that the IBM is better suited as an indicator for the PSD updates while direct application of IRM as a real valued gain leads to a better improvement in terms of short term objective intelligibility. Analysis of the performance of the proposed system also demonstrates the robustness of the system to different angular positions of the speech source.

SPS on Twitter

  • Registration is now live for the 2020 IEEE 6th World Forum on Internet of Things! Meet attendees from industry, the… https://t.co/1T7vQhAazS
  • Early bird registration for ends on Monday, 24 February. Register today and save, and save even more with… https://t.co/dzlSXdN4y8
  • The IEEE Journal of Selected Topics in Signal Processing is now accepting original manuscripts for a Special Issue… https://t.co/mXKh41of5A
  • Join us on Tuesday, 25 February for a new webinar, “Enabling Identity-Based Integrity Auditing and Data Sharing Wit… https://t.co/rfpjVkEv09
  • The 2020 IEEE International Conference on Autonomous Systems will take place in Montréal on 12-14 August 2020 and w… https://t.co/ePFEWYagwP

SPS Videos


Signal Processing in Home Assistants

 


Multimedia Forensics


Careers in Signal Processing             

 


Under the Radar