GemmaClip

GemmaClip is an AI video captioning agent built for Track 2 of the AMD Developer Hackathon ACT II. The system takes video URLs from the required tasks.json input, downloads each clip, probes its metadata, and extracts representative frames using an AKS-lite temporal-binned frame selection strategy. Instead of captioning directly from raw frames, GemmaClip first asks Gemma 4 to produce structured factual evidence about the scene, subjects, actions, setting, objects, mood, and camera details. This evidence-first design reduces hallucinations and keeps captions grounded in the actual video content. GemmaClip is a glass-box video captioning system built for the AMD ACT II Hackathon. Instead of asking one model to describe an entire video in a single step, GemmaClip separates the process into inspectable stages. It reads the video metadata, selects six representative frames using four temporal anchors and two high-change moments, and evaluates a bounded audio segment before deciding which multimodal route to use. The selected inputs are sent to specialized Gemma 4 models. Gemma 4 26B A4B handles visual evidence, Gemma 4 12B Unified can combine frames and usable audio, and Gemma 4 31B turns validated evidence into grounded captions. The system produces formal, sarcastic, humorous_tech, and humorous_non_tech caption styles while keeping them tied to the same structured observations. GemmaClip includes two user experiences. Quick Caption provides an automatic end-to-end workflow, while Gemma Lab exposes the Video, Frames, Audio, Evidence, Captions, and Compare stages. Users can inspect timestamps, selection reasons, audio decisions, verified observations, unsupported claim types, routing metadata, fallbacks, and saved experiment snapshots. This makes errors easier to understand, reproduce, and improve. The live application uses Fireworks for hosted Gemma inference, with an AMD Developer Cloud deployment path using ROCm and vLLM for self-hosted execution.