CaptionForge AI

CaptionForge AI is a production-ready video captioning platform built for the AMD Developer Hackathon Track 2. It processes videos through a modular Extract-Reason-Style multi-stage AI pipeline: Extract - Uses imageio-ffmpeg to extract audio and Pillow/imageio to sample 6 keyframes from videos Reason - Transcribes audio using Whisper-v3 and analyzes visual content using Kimi-k2p6 (both via Fireworks AI on AMD MI300X GPUs) Style - Transforms factual captions into four distinct styles using DeepSeek-v4-pro: Formal - Objective, professional documentation Sarcastic - Dry, theatrical irony Humorous-Tech - Programming metaphors Humorous-Non-Tech - Everyday observational comedy Evaluate - Local CaptionCritic validates for hallucinations and style adherence Key Innovations: Decoupled architecture enables flexibility and maintainability Cloud AMD MI300X GPU acceleration via Fireworks AI enables serverless deployment Multi-model fallback chain (DeepSeek → Kimi → GPT-OSS) for reliability Automated reasoning cleanup removes leaked chain-of-thought from model outputs Style-specific evaluation ensures quality across all four required styles Technology Stack: Backend: FastAPI + SQLAlchemy + SQLite Frontend: React + TypeScript + Vite + Tailwind CSS AI/ML: Whisper-v3, Kimi-k2p6, DeepSeek-v4-pro (all via Fireworks AI) Deployment: Vercel (serverless) + Docker