Hacienda - Video Captioning Agent

Hacienda is an autonomous multi-style video captioning agent built for Track 2 of the LabLabAI x AMD Developer Hackathon. Given a batch of video URLs, it produces four stylistically distinct captions per clip - formal, sarcastic, humorous_tech, and humorous_non_tech - fully unattended, inside a single Docker container, within the harness's 10-minute wall-clock budget. The core design principle is "see precisely, then write with style." Each clip is downloaded and sampled into 16 uniformly spaced high-resolution keyframes with FFmpeg. A vision model (MiniMax M3 on Fireworks AI's AMD-powered inference) then produces a structured, factual scene analysis covering setting, subjects, actions, environment, mood, key visual elements, and temporal flow. Finally, a text model (GPT-OSS 120B) turns that single verified analysis into all four captions at once. Because the style writers never see raw pixels, they cannot invent visual details the analysis did not establish - the formal caption stays precise while the three humorous styles keep their jokes grounded in what is actually on screen. Reliability was treated as a first-class feature: video downloads retry with backoff and write atomically, API calls retry on rate limits and server errors with separate longer timeouts for heavy vision requests, every task retries its full pipeline before degrading, and results.json is snapshotted after every completed task so even a hard kill leaves valid captions for all finished work. On the public 15-clip validation set the full batch completes with zero fallbacks in under half the time budget. The codebase is deliberately small and reviewable: two pipeline variants (the two-stage submission path and an alternative brief-verify-write pipeline), one HTTP client, and zero heavyweight frameworks - everything configurable through environment variables.