HAND GESTURE AND VOICE CONVERSION FOR DEAF AND DUMB USING DEEP LEARNING

The "Hand Gesture and Voice Conversion for Deaf and Dumb Using Deep Learning" project aims to bridge the communication gap between deaf and hard-of-hearing individuals and the general public by leveraging advanced artificial intelligence (AI) and deep learning techniques. The system integrates hand gesture recognition through a pre-trained model with speech-to-text conversion, providing real-time communication between users with speech impairments and those without.

This project utilizes a combination of hand gesture recognition using the American Sign Language (ASL) dataset and YOLO v8 for efficient and accurate sign language gesture detection. The model recognizes specific gestures such as "okay", "peace", "thumbs up", "thumbs down", "call me", "stop", "rock", "live long", and "fist", converting them into corresponding text, enabling real-time communication. Additionally, a speech-to-text conversion module is included, which uses advanced speech recognition libraries to transcribe spoken language into text. The system is developed using Flask, creating an efficient and user-friendly platform for individuals with hearing impairments to communicate effectively.

To enhance the classification accuracy of the hand gestures, InceptionNet is incorporated as a classifier. The system analyzes and classifies gestures into predefined categories, ensuring high accuracy in gesture recognition.

By providing solutions for both visual (sign language) and auditory (speech) communication, this system ensures a holistic approach to addressing the diverse communication needs of the deaf and dumb community. The system not only fosters inclusivity and accessibility but also promotes seamless interaction between individuals with and without hearing impairments. This project is a step toward creating a more inclusive society through advanced AI technologies.

Keywords: Speech-to-Text, Hand Gesture Recognition, InceptionNet,  YOLO v8, ASL Dataset, Flask, Deep Learning, Accessibility, Communication Enhancement, AI-based System