Multimodal Emotion Recognition Using Deep Neural Networks

Abstract

The change of emotions is a temporal dependent process. In this paper, a Bimodal-LSTM model is introduced to take temporal information into account for emotion recognition with multimodal signals. We extend the implementation of denoising autoencoders and adopt the Bimodal Deep Denoising AutoEncoder modal. Both models are evaluated on a public dataset, SEED, using EEG features and eye movement features as inputs. Our experimental results indicate that the Bimodal-LSTM model outperforms other state-of-the-art methods with a mean accuracy of 93.97%. The Bimodal-LSTM model is also examined on DEAP dataset with EEG and peripheral physiological signals, and it achieves the state-of-the-art results with a mean accuracy of 83.53%.