MyLingo
Real-time sign language recognition in the browser, converting signs to text and speech.
- test accuracy
- 95%
- training images
- 87,000+
- hand and face tracking
- 60fps
- Problem
- Deaf and hard-of-hearing users need a way to be understood by people who don't know sign language.
- What I built
- Trained a custom ASL alphabet model in Python and TensorFlow, then deployed it for in-browser inference with TensorFlow.js. MediaPipe Holistic handles tracking, feeding a three-tier classifier (pretrained, custom and trained ASL models). Includes a custom gesture creator, a canvas skeleton overlay and text-to-speech through the Web Speech API.
- Stack
- React
- Vite
- TensorFlow.js
- MediaPipe
- Python
Live demo of MyLingo (opens in a new tab) Source on GitHub for MyLingo (opens in a new tab)