OpenAI Whisper

Top Builders

Explore the top contributors showcasing the highest number of app submissions within our community.

OpenAI Whisper

The Whisper models are trained for speech recognition and translation tasks, capable of transcribing speech audio into the text in the language it is spoken (ASR) as well as translated into English (speech translation). Whisper has been trained on 680,000 hours of multilingual and multitask supervised data collected from the web. Whisper is Encoder-Decoder model. Input audio is split into 30-second chunks, converted into a log-Mel spectrogram, and then passed into an encoder. A decoder is trained to predict the corresponding text caption, intermixed with special tokens that direct the single model to perform tasks such as language identification, phrase-level timestamps, multilingual speech transcription, and to-English speech translation.

General
Relese date	September, 2020
Author	OpenAI
Repository	https://github.com/openai/whisper
Type	general-purpose speech recognition model

Start building with Whisper

We have collected the best Whisper libraries and resources to help you get started to build with Whisper today. To see what others are building with Whisper, check out the community built Whisper Use Cases and Applications.

Tutorials

👉 Discover more Whisper Tutorials on lablab.ai

Boilerplates

Kickstart your development with a GPT-3 based boilerplate. Boilerplates is a great way to headstart when building your next project with GPT-3.

Python Whisper Boilerplate Whisper gpt3 email generator
Streamlit Whisper, GPT-3 Boilerplate Whisper gpt3 sentiment analysis
React app to upload audio file boilerplate Uploading audio file to an API endpoint
Whisper Flask API Boilerplate Whisper API with Flask
Whisper Flask API with GPT-3 Boilerplate Whisper API with Flask and GPT-3
Whisper Streamlit Boilerplate Automatic Speech Recognition using OpenAI's Whisper

Libraries

Whisper API libraries and connectors.

Whisper API Whisper API reference
OpenAI Node.js API library for Whisper The OpenAI Node.js library provides convenient access to the OpenAI API from Node.js applications
OpenAI Python Library for Whisper The OpenAI Python library provides convenient access to the OpenAI API from applications written in the Python language

Edit on GitHub

OpenAI Whisper AI technology Hackathon projects

Discover innovative solutions crafted with OpenAI Whisper AI technology, developed by our community members during our engaging hackathons.

Video Captioner

Video Captioner V2 is a 5-stage AI pipeline that generates accurate, multi-style captions for silent videos by analyzing visual content with Gemma 4 31B vision model. The Problem: Most captioning tools rely on audio transcription, which fails for silent videos. Track 2 requires describing what's IN the video visually. Our Solution: - Stage 1: Adaptive Frame Extraction (9-15 frames based on duration) - Stage 2: Vision Analysis with Gemma 4 31B - Stage 3: Frame-to-frame comparison for scene changes - Stage 4: Thinking Board documentation - Stage 5: Caption generation + scoring + re-generation for quality Caption Styles: Formal, Sarcastic, Humorous-Tech, Humorous-NonTech Results: 6/6 videos processed, 97s average, auto-scoring ensures quality.

TruLens-by Mithridates

TruLens is an AI-powered, real-time companion that seamlessly bridges the gap between streaming media and ground truth. In an era overflowing with fast-talking influencers and political debates, separating fact from fiction is increasingly difficult. TruLens solves this by acting as a live truth filter. It listens to any audio stream, instantly transcribes speech, and uses advanced context-aware LLMs to analyze claims on the fly. Results are delivered directly to your screen via a sleek Picture-in-Picture overlay without disrupting your viewing experience. Featuring mid-stream summarization and persistent session histories, TruLens empowers you to consume digital media with absolute confidence.

Emergency Smart Dispatcher

In Thailand, a 191 emergency call travels through three relays — the provincial center answers, phones the district station, which then radios the patrol officer covering the area. Every relay costs minutes, real phone lines carry no GPS, and panicked callers describe locations in local slang no map understands. Emergency Smart Dispatcher cuts all of that down to one click. When a caller speaks, Whisper transcribes Thai speech in near real time while DeepSeek V4 (via Fireworks AI) triages the incident as they talk: type, urgency 1–5, sentiment, and recommended units, following a rubric we defined. Location is resolved in three layers — device GPS when available, a RAG landmark base that matches colloquial and even mispronounced place names by sound rather than spelling, and a geocoding ladder that trims filler words until the map finds the spot. The system then identifies the exact duty officer for that patrol zone and, with one click, sends the full picture to the officer's screen: urgency, live transcript, incident map, and turn-by-turn navigation. The officer acknowledges and reports back; the dispatcher can answer the call over WebRTC and send human-verified notes — AI proposes, a human finalizes. Every layer has a fallback: we validated Whisper large-v3 on an AMD Instinct MI300X (ROCm), and if the cloud drops mid-call the system switches to the local model baked into our single Docker image. Officer rosters are transparent demo data; production integrates real duty rosters. Built by team emergency-smart-dispatcher for AMD Developer Hackathon: ACT II.

ReachIQ AI - Competitive Intelligence Engine

ReachIQ AI is a Generative Growth as a Service platform for YouTube creators. For this hackathon, we built a Video RAG-powered Competitive Intelligence Engine: it ingests competitor videos, transcribes them with Whisper, extracts on-screen text via OCR, generates embeddings, and stores everything in a Supabase pgvector knowledge base. Using semantic retrieval, the engine surfaces hook patterns, CTA strategies, thumbnail trends, and content gaps — then synthesizes them into a full executive intelligence report via Gemma 3 4B, deployed and validated on AMD Developer Cloud using ROCm 7.2 and vLLM. This turns hours of manual competitor research into seconds of AI-accelerated strategy.

ClipContext

ClipContext takes a short creator video and turns it into platform-ready metadata: ten candidate titles, ten candidate descriptions, and ten candidate hashtag sets are generated per video, each pool independently ranked by an AI discriminator, with the top 5 of each pool surfaced to the creator — plus, optionally, a direct upload of the analysed video to the creator's own YouTube channel with whichever candidates they picked. Every candidate is written in a style pulled from real trending videos — not a generic "make it punchy" instruction. ClipContext always analyzes worldwide YouTube trends in the video's own niche; if the creator gives their channel handle, it uses that specific creator's own top-performing videos instead. Local-first preprocessing: Video validation, audio extraction, 1 FPS frame scanning, visual-quality scoring, and perceptual-diversity frame selection all run locally before any paid AI call. Evidence-grounded generation: Titles/descriptions/hashtags are required to trace back to the video's actual transcript and visuals — not free-associated from a topic string. Genuine diversity, not ten rewordings. Each of the 10 candidates per pool is generated against a distinct strategy (question, bold claim, curiosity gap, number-led, story, technical, emotional, SEO-minimal, creator-voice, and more). Two trend sources: ClipContext always mines worldwide YouTube trends in the video's own niche for a real, data-derived style profile (typical title/description/hashtag structure, SEO vocabulary, tone) — and, if a creator hands over their channel, it swaps in that specific creator's own top-performing videos instead, so the output sounds like them, not a generic trending-video template. Independent AI ranking: A second model scores and ranks each candidate pool against the video's ground truth and real trend benchmarks, with a stated reason per score — the top 5 per pool are what reach the results page.

Persona Studio AI

Project Overview & Vision: PersonaStudio AI introduces an efficient "Write Once, Run Everywhere" paradigm for digital media assets. Traditional content creation workflows are heavily fragmented; adapting a single master video for diverse platforms and audiences usually demands repetitive human effort or computationally expensive AI re-parsing. PersonaStudio AI addresses this bottleneck by decoupling heavy video ingestion from the generation layer. Built for the AMD Developer Hackathon (Unicorn Track), the platform analyzes a raw video exactly once to extract its semantic essence into a highly compressed, persistent JSON payload called Content DNA. Dual-Path Ingestion & Multi-Model Architecture: The core innovation lies in its flexible Understanding Engine, which maps video data via two selectable methods: Whisper Transcript Path: Leverages ffmpeg and Groq (Whisper Large v3) to extract audio and generate precision timestamped transcripts, which are compiled into the Content DNA using Fireworks AI LLMs. Gemma 4 Vision Path: Samples evenly-spaced frames using ffmpeg and passes them directly to a vision-capable Gemma 4 model via OpenRouter, mapping visual context without requiring any audio transcription. Decoupled Transformation at Scale: Both ingestion channels converge on the identical ContentDNA schema persisted as a JSONB object in Supabase Postgres. Once saved, users can instantly trigger /generate requests to transform that single structural blueprint into platform-specific LinkedIn posts, blog articles, short-form scripts, or captions customized by tone and persona. Because the Transformation Engine queries the static database payload instead of re-processing the source media, the system minimizes API latency, avoids redundant AI compute overhead, and optimizes throughput on scale.