.png&w=256&q=75)
🤝 Top Collaborators
.png&w=640&q=75)
🤓 Latest Submissions
.png&w=828&q=75)
StyleCap: Multi-Style Video Captioner
StyleCap is a fully automated, two-stage video captioning pipeline built for the AMD Fireworks AI Hackathon. Stage 1 — Scene Understanding: Given a video URL, the pipeline uses ffmpeg to extract a set of evenly-spaced frames directly from the remote stream without downloading the full file. These frames are passed to a Fireworks vision-language model (kimi-k2p6), which analyzes them concurrently and returns a structured scene description as JSON — covering the setting, actors, objects, events, and visible text. Stage 2 — Multi-Style Caption Generation: The structured scene data is then fed to four specialized LLMs running in parallel, each configured with distinct personas and sampling parameters to produce captions in four unique voices: Formal — precise, documentary-style narration Sarcastic — dry, deadpan wit Humorous Tech — analogies drawn from software engineering Humorous Non-Tech — relatable, observational comedy Robustness by Design: A key engineering challenge was that all reasoning models on the Fireworks catalog emit chain-of-thought prose before their final answer. The pipeline handles this gracefully through a multi-layer defense: discarding separate reasoning fields, stripping think tags, and extracting JSON from markdown fences — ensuring every video always produces a complete, valid caption set even under model misbehavior. Deterministic templated fallbacks guarantee no style is ever silently missing from the output.
13 Jul 2026
