Authors: Harshini R, Poojabharani S, Sanjana S, Saranya R
Abstract: Lip reading is the task of recognising spoken words from the visible movements of the lips without using any audio signal. This paper presents a real-time visual speech recognition system that captures live video from an ordinary webcam, locates the mouth region, and predicts the spoken word or phrase as text. The system combines 3D convolutional layers, which learn spatial and temporal lip-movement features, with bidirectional LSTM layers that model the whole sequence in both directions, and it is trained with Connectionist Temporal Classification (CTC) loss so that no frame-level alignment is needed. A lightweight pipeline of MediaPipe landmark detection, mouth-region preprocessing and a sliding frame buffer allows the model to run on a standard computer. On a small custom dataset of four phrases, the system achieved a word error rate (WER) of 16.67% and a character error rate (CER) of about 4–5%. The results indicate that a compact, audio-free lip-reading system is feasible for assistive communication and for noisy environments.
DOI: https://doi.org/10.5281/zenodo.23615156
