Skip to main content

Deep-learning-based audio-visual speech enhancement

1 February 2023

We all experienced the discomfort of communicating with our friends at a cocktail party or in a pub with loud background music. When difficult acoustic scenarios like these occur, we tend to rely on several visual cues, such as lips and mouth movement of the speaker, in order to understand the speech of interest.

Read more

PANNs: Large-scale Pretrained Audio Neural Networks for Audio Pattern Recognition

1 January 2023

Audio pattern recognition is an important research topic in the machine learning area, and includes several tasks such as audio tagging, acoustic scene classification, music classification, speech emotion classification and sound event detection. In this blog, we introduce pretrained audio neural networks (PANNs) trained on the large-scale AudioSet dataset. These PANNs are transferred to other audio related tasks. We investigate the performance and computational complexity of PANNs modeled by a variety of convolutional neural networks. We propose an architecture called Wavegram-Logmel-CNN using both log-mel spectrogram and waveform as input feature.

Read more

Frontal-Centers Guided Face: Boosting Face Recognition by Learning Pose-Invariant Features

1 January 2023

Recent years, face recognition has made a remarkable breakthrough due to the emergence of deep learning. However, compared with frontal face recognition, many deep face recognition models still suffer serious performance degradation when handling profile faces. To address this issue, we propose a novel Frontal-Centers Guided Loss (FCGFace) to obtain highly discriminative features for face recognition. Most existing discriminative feature learning approaches project features from the same class into a separated latent subspace.

Read more

Estimation in Multi-Object State Space Model

1 April 2022

A brief introduction to state estimation in multi-object system that arises from applications where the number of objects and their states are unknown and vary randomly with time. State space model (SSM) is a fundamental concept in system theory that permeated through many fields of study.

Read more