FrameSense: Multi-Style Video Captioning Agent

FrameSense takes a video URL and writes four captions of it, each in a different tone: formal, sarcastic, tech humour, and everyday humour. It works by pulling eight frames evenly spaced across the clip with ffmpeg, shrinking them, and handing the whole sequence to a vision model in order. The point of spreading the frames across the full duration is that you get the motion — a cat walking toward the camera, traffic moving through an intersection — instead of just whatever a single thumbnail happened to catch. Captions that describe a still frame tend to miss what the clip is actually about. The model does one pass to work out what's genuinely in the clip, then writes the four captions off that same observation. That way all four are describing the same real thing and only the tone changes, which is what the scoring cares about: accuracy comes from the grounding, style match comes from being strict about each register. The everyday-humour style, for instance, is explicitly banned from using any tech words — left alone, models drift and start making jokes about buffering in a caption about a kitten. Most of the engineering went into not failing. Clips download and process in parallel so twelve of them finish inside the ten-minute limit, and there's a hard internal deadline that forces the results file to be written no matter what the network is doing. Every failure path — a dead download, a model returning junk, an API timeout — falls back to a usable caption instead of a missing one. A missing style scores zero; a mediocre caption doesn't. So the container always writes valid JSON and always exits clean.