Case study · Summer 2025
Real-Time ASL Translator
An offline-first multimodal pipeline translating webcam ASL gestures into English text and speech.
- Discipline
- Multimodal AI · Accessibility
- Technology
- OpenCV · Gemma 3n · FastAPI · Ollama
Overview
I combined OpenCV input processing with Gemma 3n inference behind a FastAPI service, then deployed the pipeline locally through Ollama for privacy and low-connectivity use.
System map
How the pieces connect
The challenge
What made the problem worth solving
A real-time accessibility tool has to connect perception, language, and presentation while respecting the privacy and connectivity constraints of the people it is meant to support.
The approach
Turning the problem into an engineering plan
- 01
Captured and prepared live gesture input through OpenCV.
- 02
Used Gemma 3n to connect multimodal observations to English output.
- 03
Ran the system locally via Ollama to avoid sending sensitive camera input to a cloud service.
Result
The prototype connected multimodal AI with a concrete human need and pushed me toward local-first systems where privacy, responsiveness, and access matter.