Gemma Captioner: Four Voices, One Truth

Most video captioning agents write all four styles in a single pass, so one visual mistake poisons every caption. And the funnier a caption tries to be, the more it tends to invent: a tech joke reaches for packets and uptime that were never there, a sarcastic line flips what actually happened. Track 2 scores both accuracy and tone, so a caption that reads well can still be wrong. We built it the other way around. Instead of adding style at the end, the agent uses Google Gemma-4-31B to look at the video once. It turns sampled frames into a grounded record of what is actually there: it transcribes on-screen text like signs and lane numbers, names concrete colors and counts, and flags whatever it cannot be sure of. A self-check drops anything the frames do not support, and those facts are frozen. Only then do four parallel Gemma-4 text passes rewrite the same verified facts into formal, sarcastic, humorous-tech, and humorous-non-tech voices. A joke can change the tone, but never the events, because the humor voice has to map its metaphor onto a real visible detail. The hard part of this task is speed. Long UHD clips are the trap: a clip that finishes too late is graded as a placeholder. So the agent streams only the frames it needs from each video instead of downloading the whole file, which takes a 97MB two-minute clip from a 20-to-40-second download down to about ten seconds, and it captions clips concurrently while always leaving a complete results.json on disk. Every requested style is always filled, so a slow call never costs a zero. Gemma does the real work. Every scene fact and every caption comes from Gemma-4-31B; take it out and there is no product. It ships as a public linux/amd64 Docker image that runs on its own, reading /input/tasks.json and writing /output/results.json, and it captions the full set of clips well inside the 10-minute budget.