Member Highlight: Dr. Chanwoo Kim

Member Highlight: Dr. Chanwoo Kim
Professor, Department of Artificial Intelligence
Korea University, Seoul, Korea

Dr. Chanwoo Kim is a professor at the Department of Artificial Intelligence at Korea University. Till 2023, he has been a corporate Executive Vice President (EVP) at Samsung Researchleading the Language and Voice Team (LVT). He joined Samsung Research as a corporate Vice President (VP) heading the speech processing Lab in Feb. 2018.

He has been leading research on end- to-end speech recognition, end-to-end text-to-speech (TTS), Natural Language Understanding (NLU), Language Modeling (LM) , speechenhancement, key-word spotting, and so on at Samsung Research. Most of these research outcomes have been commercialized for Samsung products. He was a senior software engineer at the Google speech team between Feb. 2013 and Feb. 2018. He worked on acoustic modeling for speech recognition systems and enhancing noise robustness using deep learning techniques. While working for Google, he contributed to data-augmentation and acoustic modeling of Google speech recognition systems. He contributed to the commercialization of various Google AI speakers and Google speech recognition systems. He was a speech scientist at Microsoft from Jan. 2011 to Jan. 2013.

Dr. Kim received his Ph. D. from the Language Technologies Institute of the School of Computer Science at Carnegie Mellon University in Dec. 2010. He received his B.S. and M.S. degrees in Electrical Engineering from Seoul National University in 1998 and 2001,respectively. Dr. Kim’s doctoral research was focused on enhancing the robustness of automatic speech recognition systems in noisy environments. Between 2003 and 2005 Dr. Kim was a SeniorResearch Engineer at LG Electronics, where he worked primarily on embedded signal processing and protocol stacks for multimedia systems. Prior to his employment at LG, he worked for EdumediaTek and SK Teletech as an R&D engineer.

We approached Dr. Chanwoo Kim to learn more:

Q: Why did you choose the industry in signal processing?

The first time I encountered signal processing was in 1997. At that time, I was a third-year undergraduate student, and I took the Signals and Systems course. The textbook we used was Signals and Systems by Alan V. Oppenheim, Alan S. Willsky, and Ian T. Young. This course introduced me to the fascinating world of signal processing and had a profound impact on my entire career. In particular, I was captivated by the mathematical beauty of the theories behind signal processing.

The concepts of digital filters, spectral analysis, and the underlying mathematics intrigued me deeply. These ideas were not only intellectually stimulating but also elegant in their simplicity and utility. It was during this time that my interest in speech processing grew stronger. I decided to major in speech signal processing and speech recognition when I entered graduate school. Around 1999-2000, speech recognition was still considered a particularly challenging task due to the vast variability between speakers, the adverse effects of noise, and the difficulty of recognizing named entities, among other issues. These challenges intrigued me, and I set my goal as building a robust speech recognizer. I decided to continue my doctoral research in speech recognition. I was fortunate to be accepted into a Ph.D. program at Carnegie Mellon, where I had the opportunity to dive deep into this exciting and challenging field.

Q: How does your work affect society?

During my tenure in industry, I worked on the acoustic modeling aspect of Google's speech recognizer used in Google Home and Google Assistant, as well as various speech and language engines for Samsung Bixby, including the on-device speech recognition engine. I believe the technology I worked on contributed to the widespread use of voice assistants, which many people worldwide use daily to set alarms, listen to music, operate home appliances, check the weather, and more.

When we began AI speaker projects in 2014, the first major issue we encountered was the far-field speech recognition problem. In far-field environments, speech recognition accuracy is negatively affected by reverberation, voices from interfering speakers, TVs, and other noise sources. I believe that the far-field speech recognition technologies I contributed to - such as robust feature extraction and far-field data augmentation using room acoustic simulators - have made AI speakers more effective. These technologies are useful not only for AI speakers but also for other home appliances with voice control capabilities, such as TVs, air conditioners, vacuum cleaners, and more. In addition, my team and I worked on on-device speech and language engines with low latency. I believe our work improved voice assistants by enhancing latency, privacy, and personalization capabilities, while also reducing server costs from the company's perspective.

Q: What challenges have you had to face to get to where you are today?

I worked in industry after completing my Ph.D. studies. In industry, it is crucial to make contributions to real products that customers will eventually find useful. When we first began developing the Google Home device in 2014, one of the most significant technical challenges was the lack of sufficient far-field data. To address this problem, I worked on an acoustic simulator for large-scale data augmentation to simulate far-field environments. My background in signal processing was invaluable in designing such a system. When we started this project, we thought the approach seemed risky, but eventually, it turned out to be very successful.

When I joined Samsung in 2018, I needed to adjust my strategy. Samsung is a device company that manufactures roughly 500 million devices a year, so it is important to fully leverage its strengths in device manufacturing. I decided to focus my team’s efforts on on-device processing. In early 2018, when I joined Samsung, the commercialized on-device speech recognition and speech synthesis engines were based on the Hidden Markov Model (HMM), and they had significant limitations in recognizing large vocabulary speech and generating high-quality speech, respectively. I believed that the newly introduced full neural end-to-end speech recognition and speech synthesis techniques might be good candidates for next-generation on-device engines. However, at the time, there were obstacles to commercialization, despite their promising results in academic papers. Specifically, in the case of speech recognition, challenges included enabling streaming capability without sacrificing accuracy, efficiently recognizing named entities, and applying domain biasing, including personal information. For text-to-speech, computation costs were a major obstacle. At Samsung, my team and I overcame these problems one by one, and eventually, on-device speech recognition engines began to be commercialized for many Samsung products starting in 2019. On-device speech synthesis engines started being commercialized for various Samsung products beginning in 2020.

Q: What advice would you give to scientists/engineers in signal processing?

I would like to offer the following advice to students and early-stage scientists and engineers in the signal processing field:

1. Set the Long-term Career Goals

When thinking about your career in signal processing, it’s very important to aim high and set long-term goals that reflect your true aspirations. By envisioning where you want to be in the next 10 or 20 years, you will become more self-motivated which will help you grow professionally.

2. Choose a Career Goal that Resonates with Your Interests

It’s essential to choose a career goal that genuinely excites and motivates you. Signal processing is a broad field with various applications across industries such as telecommunications, healthcare, and robotics, to name a few. Finding what specifically sparks your interest—whether it’s speech recognition, image processing, or machine learning algorithms—ensures that you remain motivated in the face of challenges

3. Strengthen the Mathematical and Theoretical Foundations

One of the cornerstones of becoming a successful engineer in signal processing is having a solid grasp of the fundamental mathematical and theoretical principles. Signal processing is built on complex concepts from areas such as Fourier analysis, linear algebra, and probability theory. By reinforcing these foundational areas, you position yourself to understand the intricacies of advanced signal processing techniques, making it easier to innovate and solve real-world problems.

Q: Is there anything else that you would like to add?

I have a 7-year daughter named Jane Kim. When she tries to watch her favorite animation on YouTube, she says, 'Hi Bixby, please play My Little Pony on YouTube' using the Smart TV at home. This is truly inspirational to me. Even though I left Samsung in December 2023, it reminds me of the hard work I did with my wonderful team members.

I would like to thank my advisors during my Ph.D. and master’s studies, Prof. Richard Stern and Prof. Wonyong Sung. During my time in the industry, I am grateful to my team members and managers, including Dr. Yifan Gong, Dr. Vincent Vanhoucke, Dr. Michiel Bacchiani, Prof. Daniel Lee, and Prof. Sebastian Seung. Especially, I would like to thank Dr. Seungwhan Cho, who hired and encouraged me at Samsung, and strongly supported me and our team at Samsung Research. Without his support, it might not have been possible to commercialize various on-device AI engines for Samsung products.

Finally, I would like to thank my parents. My father was an electrical engineer, and that may be one of the reasons I chose this field of study. When I was a young child, I remember being fascinated watching him solve a set of linear equations using Cramer’s rule to solve electric circuit problems. This inspired me to start my career in electrical engineering.