Heather Roberta Pon-Barry (Harvard University), “Inferring Speaker Affect in Spoken Natural Language Communication” (2013)

Heather Roberta Pon-Barry (Harvard University), “Inferring Speaker Affect in Spoken Natural Language Communication”, Advisor: Prof. Stuart M Shieber (2013)

The field of spoken language processing is concerned with creating computer programs that can understand human speech and produce human-like speech. Regarding the problem of understanding human speech, there is currently growing interest in moving beyond speech recognition (the task of transcribing the words in an audio stream) and towards machine listening --interpreting the full spectrum of information in an audio stream. One part of machine listening, the problem that this thesis focuses on, is the task of using information in the speech signal to infer a person's emotional or mental state.

In this dissertation, the approach is to assess the utility of prosody, or manner of speaking, in classifying speaker affect. Prosody refers to the acoustic features of natural speech: rhythm, stress, intonation, and energy. Affect refers to a person's emotions and attitudes such as happiness, frustration, or uncertainty. The author focuses on one specific dimension of affect: level of certainty. The goal is to automatically infer whether a person is confident or uncertain based on the prosody of his or her speech. Potential applications include conversational dialogue systems (e.g., in educational technology) and voice search (e.g., smartphone personal assistants).

For details, please contact the author or visit the thesis page.