.png&w=256&q=75)
🤝 Top Collaborators
.png&w=640&q=75)
🤓 Latest Submissions
ClipTone : Multi-Style Video Captioning Agent
This project is a video captioning agent built for Track 2 of the AMD Developer Hackathon. Given a short video clip, it generates four distinct styled captions formal, sarcastic, humorous tech, and humorous-non tech, that must both accurately reflect the video content and convincingly match the requested tone. The pipeline extracts evenly spaced keyframes and audio from each clip, transcribes any speech using Groq Whisper for additional context, and generates all four captions in a single structured JSON call to a Fireworks AI vision model rather than four separate calls, reducing token usage and rate-limit exposure. If the primary vision model fails or returns malformed output, an automatic fallback model takes over, and a lightweight retry handles transient errors. A key differentiator is the self-evaluation loop: after generating captions, the agent scores its own output on the same two dimensions the judge uses accuracy and style match and automatically regenerates any style that scores below threshold, before the results are ever submitted. The system also runs clips concurrently with a per-clip timeout, keeping total runtime well within the 10-minute limit even across a full batch of clips. It was tested locally across twelve diverse clips spanning animals, sports, people, food, and static scenes, ranging from 10 to 80 seconds in duration, with no crashes, timeouts, or malformed output, and verified end-to-end inside the built Docker container against the official example clips.
13 Jul 2026
