.png&w=256&q=75)
🤝 Top Collaborators
🤓 Latest Submissions
Video Captioning Agent
Clarity is a video captioning agent built for the AMD Developer Hackathon ACT II — Track 2. Given a short video clip, it generates one caption in each of four distinct styles: formal, sarcastic, humorous-tech, and humorous-non-tech. The core design challenge in this track is that asking a model to "watch and be funny" in a single step tends to produce captions that are creative but factually wrong. Clarity solves this with a three-phase architecture that separates seeing from writing: Phase 1 — Perception: The clip is downloaded, and roughly 8 evenly spaced frames are sampled across its duration. These frames are sent to a vision-capable model on Fireworks AI, which produces a purely factual scene description — no jokes, no style, just what's actually visible. Phase 2 — Style Generation: That factual report is passed into four separate, focused calls to a language model on Fireworks AI, one per requested style. Because every style call is grounded in the same facts, the humor and sarcasm never drift into inaccurate territory. Phase 3 — Integration: An orchestrator runs both phases for every task, guarantees every requested style is present even under partial failure, validates the JSON schema, and writes the final results file. A key engineering finding along the way: the available models are heavy reasoning models that produce long internal "thinking" text before answering. Plain prompt instructions weren't reliable enough to control this, so Clarity uses Fireworks' JSON mode to grammar-constrain every response directly into clean, parseable output — a much more robust fix than parsing free text. The agent is fully containerized, builds for linux/amd64 via an automated GitHub Actions pipeline, and was tested against the three official example clips plus additional clips spanning different content types to guard against overfitting.
13 Jul 2026
