Video captioner agent

A Dockerized agent that automatically generates styled captions for video clips using Fireworks AI's vision and language models. The agent processes video files through a multi-stage pipeline combining audio transcription, frame sampling, and multi-modal AI inference. How it works: The agent reads a tasks.json file from the /input directory containing video URLs and metadata. For each task, it downloads the video clip, extracts audio using ffmpeg, and transcribes speech with Fireworks' Whisper model (whisper-v3-turbo). It then samples evenly-spaced JPEG frames from the video (adaptive count, capped at 16), and feeds both the frames and the transcript to a Fireworks vision-language model (kimi-k2p6) for grounded observations. Finally, a Fireworks text model (qwen3p7-plus) generates captions in four distinct styles: formal, sarcastic, humorous_tech, and humorous_non_tech. Output: All results are saved to /output/results.json with structured captions for each task, including timestamps and styled variations. The agent exits with code 0 on success or non-zero if any task fails. Key Features: Multi-modal AI pipeline: Combines visual frame analysis with audio transcription for contextually accurate captions Four caption styles: Formal, sarcastic, humorous_tech, and humorous_non_tech for versatile use cases Dockerized deployment: Self-contained container with all dependencies (Python, ffmpeg, AI models) Parallel processing: Handles up to 3 tasks concurrently within a 10-minute runtime budget Configurable: Environment variables for models, frame limits, and transcription toggle Lightweight: Optimized Docker image under 10GB with slim Python and ffmpeg