SignFlow

In virtual communication—whether in meetings, classrooms, healthcare, or remote work—Deaf and hard-of-hearing individuals still struggle to fully access spoken conversations. Captions lack context, human interpreters are not always available, and sign language is more than just text—it’s expressive, visual, and emotional. SignFlow aims to fill this accessibility gap. SignFlow is an AI-powered Chrome extension that listens to live speech during video calls, understands the meaning using Google Gemini, converts speech to structured sign language using NLP, and generates realistic sign animations through Opus AI video generation tools. These generated sign videos are either retrieved from our sign phrase library stored in Qdrant vector database, or dynamically produced using AI when no pre-recorded sign exists. The entire processing pipeline is hosted and deployed using Surge, ensuring fast, scalable, real-time performance inside the browser. Instead of showing subtitles, SignFlow displays a sign language avatar or animated gesture videos on screen, signing naturally with hand shapes, facial expressions, and flow. The MVP version translates 30–50 basic phrases, supports real-time sign rendering, and integrates smoothly with Zoom, Google Meet, and Microsoft Teams. Key Technologies Used: 🎙️ Gemini API (Speech Understanding & NLP) – accurately extracts meaning and context from spoken language 💬 Custom NLP Models – structure speech into sign grammar format 🎥 Grok / AI Video Generation – create or fetch sign language animations 📁 Qdrant Vector DB – store and retrieve sign gestures efficiently using semantic search 🌐 Chrome Extension Framework – integrates AI-powered sign translation directly into the browser SignFlow is more than an accessibility tool—it’s a real-time communication bridge that makes conversations not just understandable, but truly inclusive.