INDIAN SIGN LANGUAGE INTERPRETATION USING CNN AND MEDIAPIPE WITH TEXT-TO-SPEECH INTEGRATION

Abstract

Indian Sign Language (ISL) is of great importance in communication for deaf and speech-impaired people from all around the nation of India. However, the lack of universally available interpretation tools has resulted in an obstacle to communication between ISL users and the general population. This research presents a real-time ISL interpretation system that consists of CNN and MediaPipe hand tracking to process gestures and convert them to natural language text. Finally, Text-to-Speech (TTS) technology is integrated to allow for the output of the gestured information to hearing individuals for supportive interaction. The proposed model takes in live video input, extracts hand landmarks based on MediaPipe output, and passes these landmarks into a trained CNN to make gesture classification. Analytically, with high accuracy on a broad, machine-readable ISL dataset of commonly used signs, the system achieves high accuracy. After classification, it translates the output into readable text and converts it into speech to form a high-fidelity communication bridge between hearing and non-hearing people. The way it integrates gesture detection, deep learning-based recognition, and speech synthesis, is to enable real-time intuitive and efficient two-way communication. As a lightweight, scalable system suitable for deployment on mobile and wearable devices, it largely conforms to what is found in daily use as an educational, social, and professional tool. The system shows great promise in improving social inclusion, accessibility, and independence of hearing and speech-disabled individuals in India.