DeepSync: Captioning Agent

Our agent tackles Track 2: given a video clip and a requested style, it produces a caption that both accurately reflects what's happening and matches the tone asked for. The challenge is generalising across wildly different content — nature, sports, food, people, urban scenes — while nailing four very different voices. We built it as a two-stage pipeline. Stage A watches the clip and produces a single grounded, factual description of the scene: subjects, setting, actions, on-screen text, and how things change over time. We sample frames adaptively — more frames for longer or busier clips, scene-aware placement so motion is captured, not missed — and feed them with timestamps to a vision-language model. Getting this description accurate matters, because every caption is built on it. Stage B then restyles that description into the four requested styles. Rather than generating all four together (which makes them blur into each other), each style is generated in isolation from its own carefully tuned style guide, with multiple candidates per style. A separate judge model then vetoes anything that invents details or drifts into the wrong tone, and picks the sharpest, most scene-specific caption. The whole thing runs unattended in a Docker container: it reads tasks from /input, processes every clip concurrently within a strict time budget, and guarantees a valid caption for every requested style — with graceful fallbacks so a single failure never sinks the run. We focused hard on not overfitting to sample clips, validating on fresh, varied footage so it holds up on unseen content.