Multimodal Emotion recognition

Emotion recognition is essential in enhancing human-computer interaction across various applications. This project aims to develop a multimodal emotion recognition system that processes both audio and video data to classify emotions into six categories: Happy, Sad, Angry, Fearful, Disgusted, and Surprised. The dataset used for this project is sourced from Kaggle, containing audio and video recordings of individuals exhibiting different emotional expressions. The system employs three advanced algorithms: Convolutional Neural Networks (CNN) combined with Conformer for video data, Long Short-Term Memory (LSTM) networks for sequential audio data processing, and Support Vector Classifier (SVC) for emotion classification. A Flask-based web application offers an interface that allows users to upload their media for emotion classification. The goal is to achieve high accuracy in emotion detection by leveraging the complementary features of both audio and video. The system is evaluated based on accuracy, precision, recall, and F1-score metrics. This approach highlights the potential of multimodal models in improving emotion recognition capabilities.

Keywords: Emotion Recognition, Multimodal Learning, Audio Classification, Video Classification, CNN, Conformer, LSTM, SVC, Flask, Deep Learning.