Authors: Assistant Professor. Sukanya H N, Adithya N, Akash H S, Farazulla Khan, G P Chinmayaradhya
Abstract: Sign language is the primary communication medium for deaf and hard-of-hearing individuals, yet it remains largely inaccessible to the general public, creating a persistent commu-nication barrier. This paper presents a real-time sign language detection system that leverages computer vision and machine learning to recognise hand gestures and convert them into readable text or speech with minimal latency. The proposed framework follows a structured processing pipeline comprising data acquisition, key-frame extraction, skin-colour-based hand segmentation, face-region elimination, morphological filtering, and noise reduction. Discriminative spatial features are derived using fuzzy triangular membership functions, and gesture recognition is performed by a K-Nearest Neighbour (Mediapipe) classifier trained on a self-collected dataset of two-handed dynamic signs. For real-time operation, the system employs the MediaPipe library for hand-landmark detection and a Convolutional Neural Network (CNN) trained with TensorFlow/Keras for gesture classification. Experimental evaluation demonstrates an overall gesture recognition accuracy of approximately 92%, with a high-confidence detection of 99.6% for the “Peace” gesture and an average detection-plus-translation latency of approximately 150 ms per frame. The system requires no specialised sensors or gloves, making it cost-effective and practically deployable in educational institutions, healthcare facilities, and public service environments. Results confirm the feasibility and effectiveness of the proposed approach as an assistive communication solution for hearing-impaired individuals.
