Eleven Labs AI Hackathon

Hackathon Kick-Off

The founders at NewNative and lablab.ai will introduce participants to our mission and the opportunities you have as a member of the lablab.ai community.

NewNative Introduction

Let's go through the Technologies for the Hackathon and how you can learn it.

Technology and Challenge

Mati Staniszewski, CEO of ElevenLabs, will give an introductory presentation to hackathon participants on the advanced voice model you'll be using to build projects. Get excited and be ready to start creating!

Get to know ElevenLabs!

We will guide you through the website, and Discord community - to make sure you will succeed at the Hackathon

Hackathon Guide

After the Kick-Off stream, focus on finding your team for the Hackathon and start brainstorming your idea!

Team Formation time!

Join Q&A space on lablab.ai Discord server for open session to discuss any questions you may have

Open Q&A with mentors

Submission Deadline

shrimple

bungerr

Toven

Whispy is an accessibility tool built for voice chat accessibility. Using multiple models running concurrently, we can completely substitute a user in a voice chat. Users of Whispy can stick to using their preferred input method, whether that be Speech to text, or Text to speech, and other users in the voice chat continue to use the platform as is. This seamless integration into the Discord platform for our Demo allows users to have complete, real-time, and thorough conversations via Text or Voice, regardless of their preference. 

We leverage ElevenLabs streaming API and an audio queue to return any written text to the users of the voice call with a custom TTS voice. Text users can choose from all default voices, and their preferences are stored in the bot files. Our solution allows for text to be streamed back into the voice call rapidly, ensuring fluid conversation.

Additionally, OpenAI's Whisper large model is analyzing and transcribing audio from any number of users in a voice call, separated out by speaker, and returning their speech as text into the same channel as the ElevenLabs user is typing in. This essentially replicates the Voice Call audio into a text conversation.

For international users, both ElevenLabs and Whisper models can handle other languages, mostly limited to the Whisper supported languages. Our demo showcases Spanish as a secondary.

Whispy is an accessibility tool built to enable users of any ability to join and use a voice chat. Leveraging ElevenLab’s API, we deliver TTS to the voice call. Users in the voice call are transcribed per user by Whisper model. 

"shrimple team on Eleven Labs AI Hackathon Hackathon"

Team Idea

Submission