Cross-subject EEG-based emotion recognition with deep domain confusion

At present, the method of emotion recognition based on Electroencephalogram (EEG) signals has received extensive attention. EEG signals have the characteristics of non-linear, non-stationary and low spatial resolution. There are great differences between EEG signals collected from different subjects as well as the same subjects from different experimental sessions. Therefore, it’s difficult for traditional emotion recognition methods to achieve high recognition accuracy. To tackle this problem, this paper proposes a cross-subject emotion recognition method based on convolutional neural network (CNN) and deep domain confusion (DDC). Firstly, the Electrodes-frequency Distribution Maps (EFDMs) is constructed from EEG signals, and the residual blocks based deep CNN is used to automatically extract the features related emotion recognition from the EFDMs. Then, the difference of the feature distribution between source and target domain are narrowed by the DDC. Finally, the EEG emotion recognition task is realized with EFDMs and CNN. On SEED, we set up two experiments, the proposed method achieved an average accuracy of 90.59% and 82.16%/4.43% for mean accuracy and standard deviation under conventional and cross-subject experimental protocols, respectively. Finally, this paper uses the gradient-weighted class activation mapping (Grad-CAM) to get a glimpse of what features the CNN has learned during the training from EFDMs, and obtained the conclusion that the high frequency EEG signals are more favorable for emotion recognition.