Feb
11
Date: 11-February-2026
Time: 10:00 AM ET (New York Time)
Presenter: Dr. Hung-yi Lee
Abstract
This talk highlights recent advancements in Spoken Language Models (SLMs), focusing on enabling text-based Large Language Models (LLMs) to seamlessly process and generate speech while retaining their universal capabilities. Starting from traditional text-based LLMs, we explore methods to integrate speech comprehension and generation without causing catastrophic forgetting of their original skills. We introduce novel speech representation learning techniques specifically tailored for SLMs and present analyses of their internal representations. Additionally, we discuss benchmark evaluations designed for SLMs. Finally, we will discuss how to enable SLMs to think and speak simultaneously.
Biography
Hung-yi Lee received the M.S. and Ph.D. degrees from National Taiwan University (NTU), Taipei, Taiwan, in 2010 and 2012, respectively.
He is professor of the Department of Electrical Engineering at National Taiwan University (NTU), with a joint appointment at the Department of Computer Science & Information Engineering of the university. His recent research focuses on developing technology that can reduce the requirement of annotated data for speech processing (including voice conversion and speech recognition) and natural language processing (including abstractive summarization and question answering).
Dr. Lee won Salesforce Research Deep Learning Grant in 2019, AWS ML Research Award in 2020, Outstanding Young Engineer Award from The Chinese Institute of Electrical Engineering in 2018, Young Scholar Innovation Award from Foundation for the Advancement of Outstanding Scholarship in 2019, Ta-You Wu Memorial Award from Ministry of Science and Technology of Taiwan in 2019, and The 59th Ten Outstanding Young Person Award in Science and Technology Research & Development of Taiwan. He is a Fellow of International Speech Communication Association (ISCA). He owns a YouTube channel teaching deep learning technology in Marian, which has more than 350,000 subscribers.
