May
28
Date: 28-May-2026
Time: 1:30 PM ET (New York Time)
Presenter: Dr. Gourav Datta
About this topic:
The rapid emergence of generative AI and vision-language models (VLMs) is transforming multimodal perception and reasoning, yet their deployment at the edge remains fundamentally constrained by bandwidth, memory movement, and energy costs. From a signal processing perspective, this challenge is not merely computational; it is a problem of representation, compression, and information flow across hierarchical hardware systems. Conventional architectures separate sensing from representation learning, forcing high-dimensional visual signals to be transmitted and stored before meaningful dimensionality reduction occurs, thereby incurring excessive off-chip communication and DRAM energy.
This webinar presents a near-sensor, model–hardware co-design framework centered on in-sensor autoencoder-based compression and region-aware activation reduction. By integrating entropy-regularized objectives, quantization-aware optimization, and semi-analytical system energy modeling, we treat early visual feature extraction as a constrained signal transformation problem, where spatial-channel dimensionality, entropy, and hardware memory costs are jointly optimized. The proposed approach enforces compact, compressible latent representations at the sensor boundary, effectively reshaping the rate–distortion–energy trade-off while preserving downstream task fidelity. Extending these principles to vision-language models, we discuss how entropy-constrained visual encoding and hardware-aware representation shaping can reduce bandwidth and memory requirements in multimodal pipelines, enabling more scalable edge deployment of generative workloads.
About the presenter:
Gourav Datta received the Ph.D. degree from University of Southern California in 2023.
He is currently an Assistant Professor with the ECSE Department of Case Western Reserve University (CWRU). Before that, he was an Applied Scientist at Amazon AGI, where he contributed to enhancing the video understanding capabilities of Amazon Nova, a next-generation foundation model. His research interests include energy and latency efficient algorithm-hardware co-design for machine learning at the edge.
Dr. Datta received the USC Viterbi School of Engineering’s highest achievement award, The 2024 William Ballhaus Best Ph.D. Dissertation Award, the IEEE Graduate Fellowship on Applied Superconductivity 2022, the best Research Assistant (RA) award from USC ECE, and the Annenberg fellowship during his Ph.D. tenure. He has published more than 45 peer-reviewed papers in top-tier venues including Nature Scientific Reports, Frontiers in Neuroscience, ICLR, ECCV, TCAS-I, DATE, WACV, among others, and received a best paper award nomination at VLSI-SoC 2022. Lastly, his research on in-sensor computing has been highlighted multiple times by Edge Impulse, a leading semiconductor IP company for TinyML, and USC Viterbi School of Engineering.
