IEEE JSTSP Article

Deep learning methods haverevolutionized speech recognition, image recognition, and natural language processing since 2010. Each of these tasks involves a single modality in their input signals. However, many applications in the artificial intelligence field involve multiple modalities.

Speech analysis could provide an indicator of Alzheimer's disease and help develop clinical tools for automatically detecting and monitoring disease progression. While previous studies have employed acoustic (speech) features for characterisation of Alzheimer's dementia, these studies focused on a few common prosodic features, often in combination with lexical and syntactic features which require transcription.

Clinical literature provides convincing evidence that language deficits in Alzheimer's disease (AD) allow for distinguishing patients with dementia from healthy subjects. Currently, computational approaches have widely investigated lexicosemantic aspects of discourse production, while pragmatic aspects like cohesion and coherence, are still mostly unexplored.

Obstructive sleep apnea (OSA) is a sleep disorder in which pharyngeal collapse during sleep causes complete (apnea) or partial (hypopnea) airway obstruction. OSA is common and can have severe implications, but often remains undiagnosed. The most widely used objective measure of OSA severity is the number of obstructive events per hour of sleep, known as the apnea-hypopnea index (AHI).

Obstructive Sleep Apnea (OSA) is a sleep breathing disorder affecting at least 3–7% of male adults and 2–5% of female adults between 30 and 70 years. It causes recurrent partial or total obstruction episodes at the level of the pharynx which causes cessation of breath during sleep. 

Approximately one-fifth of the world's population suffer or have suffered from voice and speech production disorders due to diseases or some other dysfunction. Thus, there is a clear need for objective ways to evaluate the quality of voice and speech as well as its link to vocal fold activity, to evaluate the complex interaction between the larynx and voluntary movements of the articulators (i.e., lips, teeth, tongue, velum, jaw, etc), or to evaluate disfluencies at the language level.

Panoramic videos are becoming more and more easily obtained for common users. Although these videos have 360∘ field of view, they are usually displayed with perspective views, which needs the saliency informations for viewing angle selection. In this paper, we propose a saliency prediction network for 360∘ videos. Our network takes video frames and optical flows in cube map format as input, thus it does not suffer from image distorations of panoramic frames. 

Panoramic videos are becoming more and more easily obtained for common users. Although these videos have 360∘ field of view, they are usually displayed with perspective views, which needs the saliency informations for viewing angle selection. In this paper, we propose a saliency prediction network for 360∘ videos. Our network takes video frames and optical flows in cube map format as input, thus it does not suffer from image distorations of panoramic frames. 

Nowadays, 360° video/image has been increasingly popular and drawn great attention. The spherical viewing range of 360° video/image accounts for huge data, which pose the challenges to 360° video/image processing in solving the bottleneck of storage, transmission, etc. Accordingly, the recent years have witnessed the explosive emergence of works on 360° video/image processing.

Recent years have witnessed the rapid development of virtual reality (VR). Above 90% of VR content is in the form of 360° video, also called omnidirectional video or panoramic video. Generally speaking, 360° video offers immersive and interactive viewing experience, as the viewers are able to freely move their heads in the range of 360° × 180° to access different viewports.