Search results

Enhancing speech signals in complex acoustic environments remains a critical challenge in audio processing. The presenters recent work presents several innovative approaches to tackle key issues in this domain. They introduced a novel audio zooming technique based on deep learning, shifting from traditional direction-based beamforming to a user-defined, adjustable 3D region for sound capture. This advancement enables precise and flexible audio acquisition, supporting real-time applications such as remote conferencing, education, and live streaming.

Audio-visual machine learning models are transforming video understanding by effectively integrating auditory and visual data, enabling diverse applications such as audio-visual scene analysis. A prominent example within this field is visual sound source localization, which identifies the spatial location of sounds in visual scenes, applicable to both environmental audio and speech. Recent methods leverage various intra and inter-modality model design choices, training-invariance strategies and data augmentations, leading to significant performance improvements.

Ambisonics offers a signal-independent and mathematically rigorous framework for representing spatial audio. Widely adopted in industry and deeply studied in academia, Ambisonics models the sound field around a listener’s head using spherical harmonics. This makes it particularly well suited for headphone-based applications such as music playback, virtual reality (VR), and augmented reality (AR). This webinar provides an overview of Ambisonics recording and encoding.

A speech signal captured in an enclosed space, such as a conference room, will inevitably contain reverberant components due to reflections from the walls, floor, and ceiling. These components degrade the perceived quality of the speech signal and cause issues in applications like hands-free teleconferencing and automatic speech recognition (ASR). The goal of “dereverberation” is to reduce these components while preserving the direct signal, thereby minimizing detrimental effects.

Multi-microphone noise reduction and speaker separation are among the most extensively researched topics in audio signal processing. Traditionally, spatial filters—often referred to as beamformers—have been designed to satisfy specific optimization criteria, such as Minimum Variance Distortionless Response (MVDR), Speech Distortion Weighted Multichannel Wiener Filter (SDW-MWF), and Linearly Constrained Minimum Variance (LCMV). Incorporating acoustic propagation through the use of Relative Transfer Functions (RTFs) for each speaker of interest has proven beneficial.

Beyond Diagonal (BD) RIS has recently been proposed and proven theoretically to enhance channel gain and coverage. The performance of BD-RIS architectures and modes depends heavily on the accuracy of the channel state information. However, the conventional least squares (LS) estimator requires significant training and disregards the built-in block Kronecker structure of the composite channel. In this talk, the presenter links the channel estimation problem for BD-RIS to a tensor decomposition problem.

Extremely large-scale antenna arrays (ELAA) play a critical role in enabling the functionalities of next generation wireless communication systems. However, as the number of antennas increases, ELAA systems face significant bottlenecks, such as excessive interconnection costs and high computational complexity. Efficient distributed signal processing (SP) algorithms show great promise in overcoming these challenges. In this talk, the presenters will provide a comprehensive overview of distributed SP algorithms for ELAA systems, tailored to address these bottlenecks.

The intersection of speech and language models offer unique opportunities and challenges. This talk provides a comprehensive walkthrough of speech-language model research from NVIDIA NeMo. We cover several types of models such as attention-encoder-decoder Canary-1B, and LLM-based architectures such as SALM or BESTOW. In particular, we highlight the challenges in training and inference efficiency of such models and propose robust solutions via 2D bucketing and batch size OOMptimizer.

Visible light communication (VLC) can facilitates multiple functionalities such as illumination, high-speed data communications, and localization. Based on the VLC technology, visible light positioning (VLP) systems estimate positions of VLC receivers based on light-emitting diode (LED) transmitters at known locations. VLP provides a viable alternative to radio frequency (RF)-based positioning systems by providing inexpensive and accurate localization services.

Integrated sensing and Communication (ISAC) is emerging as a foundational component of next-generation wireless networks, marking a paradigm shift by integrating two distinct functionalities—sensing and communication—into a single platform with fully shared hardware and radio resources. In the ISAC community, the term “sensing” is frequently used interchangeably with “localization”, which refers to the estimation of meaningful parameters that describe the state of target objects, such as their position, velocity, and potentially orientation.