Keyword Search Using Attention-Based End-to-End ASR and Frame-Synchronous Phoneme Alignments

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

Keyword Search Using Attention-Based End-to-End ASR and Frame-Synchronous Phoneme Alignments

Runyan Yang; Gaofeng Cheng; Haoran Miao; Ta Li; Pengyuan Zhang; Yonghong Yan

Attention-based end-to-end (E2E) automatic speech recognition (ASR) architectures are now the state-of-the-art in terms of recognition performance. However, despite their effectiveness, they have not been widely applied in keyword search (KWS) tasks yet. In this paper, we propose the Att-E2E-KWS architecture, an attention-based E2E ASR framework for KWS that can afford accurate and reliable keyword retrieval results. First, we design a basic framework to carry out KWS based on attention-based E2E ASR. We adopt the connectionist temporal classification and attention (CTC/Att) joint E2E ASR architecture and exploit the spike posterior property of CTC to provide the keywords time stamps. Second, we introduce the frame-synchronous phonemes modeling and use the dynamic programming (DP) algorithm to provide alignments between E2E grapheme outputs and phoneme outputs. We call this alignment procedure dynamic time alignment (DTA), which can provide the proposed Att-E2E-KWS system with more accurate time stamps and reliable confidence scores. Third, we use the Transformer, a self-attention-based encoder-decoder neural network, in place of conventional recurrent neural networks in order to yield more parallelizable models and increased training speed. We conduct comprehensive experiments on English and Mandarin Chinese. To the best of our knowledge, this is the first practical Att-E2E-KWS framework, and experimental results on Switchboard and HKUST corpora show that our proposed Att-E2E-KWS systems significantly outperform the CTC E2E ASR based KWS baselines.

SPS on Twitter

  • The SPS Webinar Series continues of 29 March when Dr. Mauricio Delbracio presents "A Walk Through Image Deblurring:…
  • COMING SOON: Join us on 9 March when Mr. Sayantan Dutta presents "Novel Prospects of Image Restoration Inspired by…
  • Happy from SPS! Thank you for doing your part towards furnishing a fairer, more equitable world for your c…
  • There's still time to register your team for the 2023 IEEE Signal Processing Cup! Visit our website and register no…
  • There is still time to join the 5-Minute Video Clip Contest! Visit our website to learn more and submit your videos…

SPS Videos

Signal Processing in Home Assistants


Multimedia Forensics

Careers in Signal Processing             


Under the Radar