Dragon Den

Dragon Den is a Dockerized AI agent built for the AMD Developer Hackathon Act II (Track 2: Video Captioning). The agent watches short video clips (30s–2min) and generates captions in four distinct personality-driven styles. At its core is a multi-stage pipeline. It downloads each video, and runs parallel FFmpeg extraction for frames and audio. Scene detection via histogram analysis identifies transitions, then adaptive sampling selects representative frames for the vision model. Dragon A — a VLM (Kimi K2 on Fireworks AI) — examines the frames and produces structured JSON observations: subjects, actions, objects, environment, camera movement, and OCR text. A second review pass with a fast instruct-tuned model catches hallucinations, while a cross-batch OCR filter discards text that only appears in a single batch. Meanwhile, faster-whisper transcribes the audio track. The merger combines vision and audio into a single canonical JSON — the source of truth. Dragon B then rewrites that canonical summary into four styles, each personified by a named dragon character. - Trí Long (formal) — analytical data-dragon, speaks like a researcher documenting a discovery - Lão Quân (sarcastic) — ancient star-dragon, permanently unimpressed, dry and weary - Gemma Trí Tech (humorous tech) — dragoness engineer, cracks programming jokes and tech references - Du Rong (humorous non-tech) — young overconfident dragon, dramatic tales about everyday life A validation loop scores each caption with a judge LLM; low-scoring captions trigger revision retries. Results are written to /output/results.json in the competition-specified format. The entire run must complete within 10 minutes. The agent also includes an optional FastAPI web UI (app.web) served on port 5173 with a tabbed interface — video player with dragon commentary overlay, extracted frames grid, and canonical JSON viewer.