FourVoices

Streamlit
application badge
Created by team Groow on July 10, 2026
Video Captioning

FourVoices turns any short clip (30s–2 min) into four captions, each in a distinct voice: formal, sarcastic, humorous-tech, and humorous-non-tech. Most captioners trade accuracy for tone — push a funny voice and the model starts inventing things that aren't in the video. FourVoices decouples the two with a "ground once, style four times" pipeline. First, ffmpeg samples representative frames from the clip. Gemma's multimodal vision reads those frames and produces a neutral, grounded description of exactly what is visible — objects, actions, and setting — with no tone or invented detail. Each of the four voices is then generated from that same grounded description, so the content stays faithful while the tone varies sharply. A self-check verifies every caption on the two axes the judge scores — accuracy (faithful to the grounded facts) and tone (unmistakably in its target voice) — and automatically regenerates any caption that drifts. The system runs on Gemma through the Fireworks AI API and on AMD Developer Cloud, uses structured JSON output to keep captions clean even with reasoning-heavy models, and ships as a fully containerized, MIT-licensed app with a simple batch runner and HTTP API. It develops and evaluates for free against a local Gemma (via Ollama) and swaps to hosted Gemma with a single environment variable — no code change. The result: captions that are faithful to what's on screen and unmistakably in voice, across every clip. (≈1,480 chars, ~230 words — well within 600–2000 and over the 100-word min.)

Category tags: