American Sign Language Recognition Based on Machine Learning and Neural Network

Numerous disabilities such as deaf and mute are suffered from not being capable of communicating with normal people, it is necessary to find a way to solve this problem. A feasible method is Sign Language Recognition (SLR) which is a sort of pattern recognition technique. In this paper, machine learning and deep learning methods are applied to recognize and classify American Sign Language (ASL), and only 24 English letters are classified because letter J and Z require fingers to move. 

First, Principal Component Analysis (PCA) and manifold algorithms are used to do dimension reduction to accelerate the training of machine learning and visualize it. Second, various machine learning methods such as Random Forest Classification (RFC), K- Nearest Neighbor (KNN), Gaussian Naïve Bayes (GNB), Support Vector Machine (SVM), and Stochastic Gradient Descent (SGD) are applied to classify the pattern. Since the SVM algorithm has several hyperparameters, this study uses the Grid Search method to find the best combination of hyperparameter which lead to predicting more accurately. 

It is found that different dimensionality reduction algorithms have unequal effects on the accuracy of each prediction model, and it can be concluded that the manifold algorithm is the best dimension reduction algorithm only for KNN but not for other prediction models, and PCA is much more feasible than KNN applied in such machine learning algorithms except KNN. Two deep learning algorithms such as Convolutional Neural Networks (CNN) and Deep Neural Networks (DNN) are also used in classification and their accuracy is highest among such algorithms mentioned above.

Keywords; Sign Language Recognition; Manifold; Machine learning; CNN; Dimension reduction