Machine and Deep Learning Techniques for Emotion Recognition Using EEG Signals and Computer Games

Abstract

Emotion recognition using electroencephalography (EEG) signals is a rapidly growing research area in affective state analysis, owing to its ability to directly reflect the underlying neural activity associated with participants’ internal emotional states. However, this task remains challenging due to significant inter-subject variability and the inherently non-stationary nature of EEG signals, which makes the extraction of stable and discriminative emotional patterns particularly diffcult. To address these challenges, this study proposes two classification approaches. The first is a Deep Learning (DL) model that integrates Convolutional Neural Networks (CNN), Multi-Head Attention mechanisms (MHA), and Bidirectional Long Short-Term Memory (BiLSTM) networks to effectively learn spatial and temporal representations from EEG signals. The second approach is a Global Stacking Ensemble framework that leverages multiple base classifiers to enhance performance, stability, and generalization capability. The proposed models are evaluated on the GAMEEMO dataset, which contains four emotional classes, under two experimental settings: Subject-Dependent and Subject-Independent using a Leave-One-Subject-Out (LOSO) cross-validation strategy. Experimental results show that the DL model with Intra-Subject Fine-Tuning achieves an accuracy of 97.78% in the Subject-Dependent setting and 60.31% in the Subject-Independent setting. In contrast, the ensemble model achieves accuracies of 95.31% and 81.70%, respectively. The experimental results clearly demonstrate the effectiveness of Deep Learning (DL) models in capturing highly discriminative emotional representations from EEG signals under Subject-Dependent settings, while Ensemble Learning (EL) methods consistently achieve superior generalization ability and enhanced robustness against inter-subject variability. Quantitatively, the proposed approach outperforms existing state-of-the-art methods by 11.26% and 3.39% in the Subject-Dependent and Subject-Independent settings, respectively.