UniKL - AMD ViXion

Video Caption Agent is an AI powered tool that watches video clips and generates captions in four distinct styles: formal, sarcastic, humorous_tech, and humorous_non_tech. Built for the AMD Developer Hackathon: Act II Track 2, this agent uses Fireworks AI's minimax-m3 model to understand video content through native video input. The pipeline follows a two-stage approach: 1. Video Understanding: The model watches the video directly and produces a detailed factual description of the setting, subjects, actions, and mood. 2. Style Generation: The description is rewritten into each requested caption style in a single API call, with intelligent key normalization to handle variations in model output. Key optimizations include: - Single-pass generation requiring only 2 API calls per video - Automatic handling of common misspellings without retries - Graceful failure handling that continues processing remaining tasks The agent processes videos in a Docker container, reading tasks from /input/tasks.json and writing results to /output/results.json, making it easily deployable and scalable. Features: - Supports four caption styles: formal, sarcastic, tech-humour, non-tech humour - Handles videos up to 2 minutes in length - Native video understanding without frame extraction - Fast processing with optimized API calls - Robust error handling and fallback mechanisms The project demonstrates efficient use of multimodal AI models for video understanding and creative text generation, all packaged in a production-ready container.