Block Division Convolutional Network With Implicit Deep Features Augmentation for Micro-Expression Recognition

Despite the development of computer vision techniques, the micro-expression (ME) recognition task still remains a great challenge because MEs have very low intensity and short duration. However, the ME recognition is of great significance since it provides important clues for real affective states detection. This paper proposes a novel Block Division Convolutional Network (BDCNN) with the implicit deep features augmentation. In detail, BDCNN learns from four optical flow features computed by the onset and apex frames of each video. It innovatively divides each image into a set of small blocks in the deep learning model, then the convolution and pooling operations are performed on these small blocks in sequence. To handle the small sample size problem in the micro-expression data, this study uses the improved implicit semantic data augmentation algorithm in the deep features space. Experiments are conducted on three publicly available databases, viz, CASME II, SMIC, and SAMM. Experimental results show that our model outperforms the state-of-the-art methods by attaining the accuracy of 84.32% and F1-score of 82.13% on the 3-class datasets, and the accuracy of 81.82% and F1-score of 75.46% on the 5-class datasets, respectively. Our source code is publicly available for non-commercial or research use at https://github.com/MLDMXM2017/BDCNN.

Introduction

The Facial expression has been playing an increasingly important role in many fields, such as judicial system [1], [2], driver safety [3], and police interrogation [4]. In general, the facial expression is divided into two basic types: macro-expression (MaE) and micro-expression (ME). MaEs represent the normal facial expressions, which occur at high intensity of facial muscles with long duration (about 2-3 s) [5]. On the contrary, MEs have very short duration (1/25 s to 1/2 s) [6], and it involves relatively fewer facial regions and lesser muscle movements [7].