OpenAI Whisper

The Whisper models are trained for speech recognition and translation tasks, capable of transcribing speech audio into the text in the language it is spoken (ASR) as well as translated into English (speech translation). Whisper has been trained on 680,000 hours of multilingual and multitask supervised data collected from the web. Whisper is Encoder-Decoder model. Input audio is split into 30-second chunks, converted into a log-Mel spectrogram, and then passed into an encoder. A decoder is trained to predict the corresponding text caption, intermixed with special tokens that direct the single model to perform tasks such as language identification, phrase-level timestamps, multilingual speech transcription, and to-English speech translation.

General
Relese date	September, 2020
Author	OpenAI
Repository	https://github.com/openai/whisper
Type	general-purpose speech recognition model

Start building with Whisper

We have collected the best Whisper libraries and resources to help you get started to build with Whisper today. To see what others are building with Whisper, check out the community built Whisper Use Cases and Applications.

Tutorials

Boilerplates

Kickstart your development with a GPT-3 based boilerplate. Boilerplates is a great way to headstart when building your next project with GPT-3.

• Python Whisper Boilerplate Whisper gpt3 email generator
• Streamlit Whisper, GPT-3 Boilerplate Whisper gpt3 sentiment analysis
• React app to upload audio file boilerplate Uploading audio file to an API endpoint
• Whisper Flask API Boilerplate Whisper API with Flask
• Whisper Flask API with GPT-3 Boilerplate Whisper API with Flask and GPT-3
• Whisper Streamlit Boilerplate Automatic Speech Recognition using OpenAI's Whisper

---

Libraries

Whisper API libraries and connectors.

• Whisper API Whisper API reference
• OpenAI Node.js API library for Whisper The OpenAI Node.js library provides convenient access to the OpenAI API from Node.js applications
• OpenAI Python Library for Whisper The OpenAI Python library provides convenient access to the OpenAI API from applications written in the Python language

---