Student Emotion Recognition System for Virtual Class Engagement Using Deep Learning and Multimodal Fusion

This paper proposes an AI-based Student Emotion Recognition and Engagement Monitoring System, which analyzes the facial emotions and other relevant factors to determine the level of student engagement during online classrooms. This system uses deep learning algorithms such as EfficientNet-B0 and MobileNetV3 to recognize facial emotions and then uses a regression-based fusion model to determine the level of student engagement by analyzing the facial features and other relevant factors such as sleep time, stress level, motivation level, and screen time. This system operates in the background and provides a summary of the level of engagement of students to the teacher only, indicating the level of class and student disengagement. From the experimental results, it is clear that the fusion of the proposed system enhances the robustness of the student engagement prediction system.