Silent speech recognition for mute people using lip movement in MATLAB

This work presents an image processing and deep learning–based system for real-time lip movement recognition aimed at assisting mute and speech-impaired individuals in basic communication. The proposed framework employs a webcam to capture continuous facial video streams, from which the mouth region is automatically detected and extracted using robust image preprocessing techniques. Key preprocessing steps include face detection and accurate mouth localization to ensure consistent and reliable region extraction. Convolutional Neural Networks (CNNs) are then utilized to learn discriminative spatial features from lip shapes and movement patterns associated with specific words and expressions. The system is trained to classify a predefined set of commonly used words and gestures, including Bad, Beautiful, Bye, Dance, Dead, Family, Good, Hai, Hello, You are welcome, House, What, You, Incomprehension, and Prohibition-crossing. During real-time operation, the captured mouth movements are processed frame by frame, and the trained CNN model predicts the corresponding word with high accuracy. The classified output is displayed instantly, enabling effective non-verbal communication. The proposed MATLAB-based implementation demonstrates the feasibility of integrating image processing and deep learning techniques for practical lip-reading applications. Experimental observations indicate that the system performs reliably under real-time conditions and varying facial poses, making it a cost-effective and accessible solution for enhancing communication support for mute individuals in everyday environments.

Keywords: Silent speech recognition, lip movement analysis, image processing, convolutional neural network (CNN), MATLAB implementation.