GemmaLens — Gemma 4 Video Captioning Agent

GemmaLens is a video captioning agent built for Track 2 of the AMD Developer Hackathon. Given any short video clip, it generates four distinct captions — formal, sarcastic, humorous_tech, and humorous_non_tech — that faithfully describe the content while matching each requested tone. Every word in the pipeline is produced by a single model: Gemma 4 31B IT. The architecture uses a deliberate two-pass design. Pass A sends uniformly sampled, deduplicated video frames to Gemma 4's vision capability, producing a rich factual scene description. Pass B takes that description and generates all four styled captions in one structured JSON response, with an explicit rule: every caption must reference concrete visual details, ensuring high scores on both accuracy and style match. This separation matters. The expensive multimodal call happens once per clip. Caption retries for malformed JSON or weak style output are cheap text-only calls that never re-send images. Style quality improves because the model writes from a clean factual anchor rather than juggling vision and tone simultaneously. Reliability is engineered at every layer. A valid safety-net output file is written before any processing begins, so even a total crash produces complete JSON. Multiple API credentials rotate automatically on quota errors. Per-clip timeouts degrade gracefully: retry, then fewer frames, then description-anchored fallback captions. A global deadline guard flushes the best available results before the ten-minute limit. The container always exits cleanly. The agent handles diverse content with no clip-specific logic. Frame sampling adapts to duration, and histogram-based deduplication maximizes visual information per model call.