iSquare - Multi-Model Video Caption Agent
iSquare is a multi-model video captioning agent that generates accurate captions for short video clips in four required styles: formal, sarcastic, humorous_tech, and humorous_non_tech. The goal is to keep captions factually grounded while still making each style distinct and creative. Styled captioning is challenging because humor or sarcasm can easily introduce hallucinated details. iSquare reduces this risk by using a verification-first pipeline with separate stages for video understanding, visual verification, fine-detail inspection, evidence merging, caption generation, and internal validation. The system first processes the video and extracts technical metadata such as duration, frame rate, resolution, and audio-track status. It then selects representative frames using scene-aware sampling where possible, with fallback sampling for simpler clips. These frames help verify visible objects, actions, setting, text, and small details that may be missed in normal video analysis. After evidence is collected, the pipeline creates a verified fact base that becomes the only source of truth for caption writing. This ensures that humor and sarcasm come from real observations rather than invented content. An internal validation layer can also check whether captions remain grounded and match the requested style. The agent is packaged as a Docker container for the evaluation environment. It reads tasks from /input/tasks.json, generates captions for every requested style, and writes valid results to /output/results.json. It also includes atomic output writes, retry handling, graceful fallback behavior, and a safety timeout to preserve valid output during constrained evaluation runs.
Category tags:
For Innovators & Creators