Getting to Know Your Fellow Researchers: Douglas O'Shaughnessy

June, 2017

What sparked your interest in speech and language processing?

I got into the field at the end of my undergrad study at MIT. At the time (1971), MIT had the option to be admitted to graduate study without doing a Bachelor’s thesis. I did not view a Bachelor thesis as worth doing, and was sure I wanted to do graduate study in some field of Electrical Engineering. Thus, the most efficient path (as always of great interest to an engineer) was to find a Master’s thesis supervisor at MIT. Computer science (CS) was really taking off in the late 1960’s, of course, and I liked that area a lot, but I wanted an application area, not pure CS. Artificial intelligence (AI) was an area I also liked, and thus I explored options in AI at MIT, eventually settling on speech as a topic that fit a lot of my interests and had great potential. There was a solid group at MIT (Dennis Klatt, Ken Stevens, Jon Allen) that inspired me as well. I also liked the idea of using linguistics (Morris Halle, associate of Noam Chomsky, was on my thesis jury).

How do you think speech and language processing is changing the society for the new generation?

The last decade has witnessed a massive increase in the use of voice dialog over phone links. Just a few years ago, most organizations required callers to listen to extensive messages and then respond to a series of questions by pushing keypad buttons. This style of interaction still persists and is not user-friendly, and is a source of frustration to many. Hands-free interaction by voice is far more pleasant. With voice inquiry in the home as well becoming more prevalent, the general public is far more comfortable talking to machines than ever before. Lastly, the new generation is far more tuned into use of PDAs than us older folk, who recall life without microwave ovens, remote controls (for cars, doors, TVs), VCR/PVR, MP3 players, PCs and push-button phones. I sometimes wonder how we passed the time back then.

What is your holy grail in speech and language processing? When will we achieve it?

I do not particularly like use of phrases like “holy grail” (or “gold standard” and “ground truth”) that can easily be expressed more clearly and have origin in other domains with no relevance to speech. (“Holy grail” is particularly egregious, as it refers to a desired but unattainable object that would do nothing to benefit people’s lives even if found). The speech field has been trying to achieve truly fluent voice dialog with machines for 50 years now, starting perhaps with motivation from cinema (2001 and Star Trek sparking one’s interest). I doubt that there will be any single specific, significant breakthrough that will lead us there. We have seen ASR (automatic speech recognition) go through some major swings of activity, from LPC in the 1970s to MFCCs, HMMs and trigram language models in the 1980s, and now to DNNs. However, through it all, we have always known (or should have known) that any true emulation of human speech communication should involve a combination of exploiting the immense and increasing power of computers with a solid understanding of how humans produce and perceive speech (and language in general). I surmise that we could achieve acceptable voice dialog, for general conversation even in noisy conditions, in perhaps another 10-15 years. We are already there for many practical but limited conditions. Extensions to include many degraded acoustic environments will take rather more time.

Do you have any specific advice for students, junior faculty or others early in their careers?

Do not neglect teaching. Too many of us focus almost entirely on research, and try to avoid teaching courses. Teaching helps one to organize ideas, see problems more clearly. Students often ask questions that are pertinent but sometimes overlooked by the teacher. Familiarize yourself with work outside your thesis domain. Breakthroughs are rarely done by staying narrowly in one area. Related fields often develop techniques that can apply more broadly.

What development in the field has most surprised you? Was there a hard problem that turned out to be easy? An easy problem that proved surprisingly difficult?

Three developments actually:

I resisted hidden Markov models (HMMs) when they first appeared in the mid-1970s, and so I was somewhat surprised to see them replace LPC/DTW as the standard for ASR for the last 30 years, notwithstanding the inherent flaws of the first-order model assumption and exponential durational density. Of course, their exploitation of statistics overcame the fatal flaw of the deterministic and time-consuming DTW.
I always thought that GMMs (Gaussian mixture models) were a strange way to model the complex densities one finds when modeling diverse conditions (e.g., Gaussians are great for indentically-distributed RVs, but not for handling speaker independence).
MFCCs made sense in terms of exploiting critical-band aspects of audition, but their final step of inverting the Fourier transform back to the time domain made less sense to me, as human audition does nothing of the sort and any physical interpretation beyond the first two coefficients was impossible.

Any problem seems hard before one solves it, and then seems trivially easy afterward…

Which of your publications is your favorite? Why?

My textbook “Speech Communication: Human and Machine”, surely. I spent years on this, first to learn the breadth of the field, and then to express the concepts clearly. Repeated revisions taught me how to express ideas well. I had offered the speech course at McGill University starting in 1978, but kept apologizing to my students for not having a suitable text; Flanagan’s great book was an inspiration and the Rabiner and Schafer text was great for applications, but neither included the human processes at the level I needed.

Publications & Resources

Conferences & Events

Professional Development

Community & Involvement

About IEEE SPS

For Volunteers