Sign Language Translator

Real-time hand gesture recognition that translates sign language into text and speech

Real-time hand gesture detection and translation interface in action

Project Overview

The Hand Language Translator is an innovative assistive technology project that bridges the communication gap between sign language users and non-signers. Using computer vision and machine learning, the system recognizes hand gestures in real-time and translates them into both written text and spoken audio.

This project demonstrates the practical application of artificial intelligence in solving real-world accessibility challenges. By combining OpenCV for image processing, decision tree-based machine learning models for gesture recognition, and Google Text-to-Speech for audio output, the system creates a complete translation pipeline that operates in real-time.

The translator is designed to be accessible and user-friendly, requiring only a standard webcam for operation. It showcases proficiency in computer vision, machine learning model selection and optimization, and the integration of multiple technologies.

Challenges & Solutions

Model Training & Accuracy
Initial attempts using neural networks resulted in poor accuracy and long training times. Solved by switching to a Random Forest model, which provided better accuracy with significantly less training time and computational resources.
Text-to-Speech Implementation
Native TTS libraries were unreliable across operating systems. Resolved by implementing Google Text-to-Speech (gTTS), which provides consistent, high-quality voice output on all platforms.
Real-Time Video Processing Latency
Hand gesture detection had significant lag during real-time video capture. Fixed by implementing frame skipping and optimizing the image preprocessing pipeline, reducing latency from ~500ms to under 100ms per prediction.

Technologies Used

Python
Mediapipe
NumPy
gTTS
OpenCV
Threading
Scikit-learn
Pickle

Project Demonstration

Video coming soon…