Farfield speech recognition has become a popular research area in the past few years, from more research focused activities such as the CHiME Challenges, to the launches of Amazon Echo and Google Home. This talk will describe the research efforts around Google Home. Most multichannel ASR systems commonly separate speech enhancement, including localization, beamforming and postfiltering, from acoustic modeling. In this talk, we will introduce a framework to do multichannel enhancement jointly with acoustic modeling using deep neural networks. Inspired by beamforming, which leverages differences in the fine time structure of the signal at different microphones to filter energy arriving from different directions, we explore modeling the raw time-domain waveform directly. We introduce a neural network architecture which performs multichannel filtering in the first layer of the network and show that this network learns to be robust to varying target speaker direction of arrival, performing as well as a model that is given oracle knowledge of the true target speaker direction. Next, we show how performance can be improved by factoring the first layer to separate the multichannel spatial filtering operation from a single channel filterbank which computes a frequency decomposition. We also introduce an adaptive variant, which updates the spatial filter coefficients at each time frame based on the previous inputs. Finally we demonstrate that these approaches can be implemented more efficiently in the frequency domain.

The insatiable demand for media rich content and the increasing availability of advanced devices such as smart phones, tablets, etc., has forced the mobile communications eco system to consider the next generation solutions to address these needs. 5G, already in early commercial deployment, is responding to these needs with options such as Small Cells, HetNets, Carrier Aggregation, Machine-to-Machine, Internet-of-Things, Relays, Device-to-Device, massive MIMO and operation in the vast spectrum available in the millimeter wave range, among others. In this talk, I will review some of the opportunities and challenges inherent to these higher bands, and how they can be best addressed to deliver practical solutions to the challenges outlined above in 5G and beyond.