Multimodal Emotion Classification Using Machine Learning and Deep Learning
Authors:-Professor Mr. V. K. Sabari Rajan, B. Mukesh, C. Narendra, M. Shivanand, B. Ajay Kumar
Abstract- In the rapidly evolving field of artificial intelligence, emotion recognition has emerged as a pivotal area of research, with significant applications in human-computer interaction, mental health analysis, and social robotics. This project focuses on the development of a multimodal emotion recognition system capable of classifying emotions from text, audio, images, and live video. The system employs advanced machine learning algorithms tailored to each modality: BERT for text, CNN and LSTM for audio, and CNN for both images and live video frames. Each modality is designed to recognize a set of core emotions, with slight variations to account for the unique characteristics of each data type. The text module identifies emotions such as anger, fear, joy, love, surprise, and sadness, while the audio, image, and live video modules detect emotions including angry, disgust, fear, happy, neutral, and surprise. The system architecture encompasses dataset creation and preprocessing, model training, and emotion classification. User interaction is facilitated through a web interface, allowing users to input text, audio, images, or live video and receive real-time emotion classification results. This multimodal approach enhances the accuracy and robustness of emotion detection, providing a comprehensive tool for analyzing human emotions across different communication mediums.
