Plenary Talk: Inside Out and Outside In: Human Communication Through Multimodal Signals in Health and Breakdown
This presentation, delivered by Shrikanth Narayanan at the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2026), investigates human communication through the lens of multimodal signal processing. The primary goal is to examine how complex neurocognitive, physiological, behavioral, and social processes manifest across biophysical human systems during interpersonal interactions. By analyzing the dynamics of the brain–body–behavior framework, the talk explores how individual intent and emotion shape both the generation and interpretation of communicative information within various sociocultural contexts.
To support this framework, the presentation leverages recent technological advancements in multimodal sensing, signal processing, and machine learning. These computational tools provide a data-driven methodology for modeling the human "communication stack," enabling researchers to decode fine-grained physiological and behavioral patterns. The central evidence focuses on identifying critical signal perturbations and anomalies that correspond to specific neurocognitive, psychological, developmental, or environmental challenges across an individual's lifespan.
Ultimately, the key takeaway is that engineering advanced signal processing and computational models directly enhances our understanding of human communication in both health and disease. By mapping multimodal signals to underlying psychological and physiological states, these biological insights can lead to innovative diagnostic, monitoring, and therapeutic interventions for health disorders and developmental challenges.
