Photon

Photon Video Captioner is an AI agent for Track 2 that watches any video clip and generates captions in four distinct styles: formal, sarcastic, humorous_tech, and humorous_non_tech. The pipeline is built to generalize across arbitrary content — nature, urban scenes, animals, people, sports, food, weather, and technology. It first samples 8–20 frames from the clip using ffmpeg, seeking directly over the video URL to avoid downloading full 4K files where possible, with a full-download fallback. All frames are sent to a vision-language model in a single request that returns a purely factual scene description, which drives caption accuracy. A second LLM call then rewrites that description into all four styles at once as a single structured JSON object, driving style match. The agent is fully containerized, reads tasks from /input/tasks.json and writes /output/results.json, and runs well within the 10-minute limit by processing clips in parallel with a soft internal deadline. It is provider-agnostic through an OpenAI-compatible client (Fireworks AI primary, Groq fallback) and is hardened with retries, provider failover, reasoning-output stripping, and per-style fallbacks so a single failure never produces an empty caption.