Eleven Labs AI Hackathon

Hackathon Kick-Off

The founders at NewNative and lablab.ai will introduce participants to our mission and the opportunities you have as a member of the lablab.ai community.

NewNative Introduction

Let's go through the Technologies for the Hackathon and how you can learn it.

Technology and Challenge

Mati Staniszewski, CEO of ElevenLabs, will give an introductory presentation to hackathon participants on the advanced voice model you'll be using to build projects. Get excited and be ready to start creating!

Get to know ElevenLabs!

We will guide you through the website, and Discord community - to make sure you will succeed at the Hackathon

Hackathon Guide

After the Kick-Off stream, focus on finding your team for the Hackathon and start brainstorming your idea!

Team Formation time!

Join Q&A space on lablab.ai Discord server for open session to discuss any questions you may have

Open Q&A with mentors

Submission Deadline

PhoMemes

Rahel Gunaratne

A tool for language learning. 

Conversation mode:
1. Give basic roleplay scenario's
2. Evaluate conversation
3. Proper grammar/word usage

Practice mode:
1. Read sentences
2. See your pronunciation mistakes
3. Play the audio of both ElevenLabs and your audio to compare the difference

It uses a local proxy server with:
 - ElevenLabs for realistic TTS
 - OpenAI for LLM completions and transcriptions
 - For the pronunciation, I used Montreal forced alignment to get transcription intervals. It generates aligned phones with the transcription.

The Montreal Forced Aligner (MFA) is a tool used in speech processing and linguistics to align speech recordings with their corresponding transcriptions. It takes a speech recording and a corresponding text transcript as input and automatically aligns the words in the transcript with their corresponding segments in the audio.

1. Phones are generated (using MFA) for both the user recorded message and the ElevenLabs TTS.
2. Damerau-levenshtein distance is computed between the words and the phones of each word to get the difference in pronunciation.
3. The shortest-edit path is interpreted as replacing, inserting, deleting or transposing a word/phone.

i.e. Do you have mispronunciation patterns like stressing your T's. This is done by comparing the generated phones to voices by ElevenLabs. You can learn different accents or languages by changing the voice/language of the ElevenLabs voice.

To create realistic conversational roleplay and evaluation like Duolingo. But to also try and capture pronunciation errors (using the state of the art Montreal Forced Aligner) and use Elevenlabs to easily and clearly articulate those errors. 


"PhoMemes team on Eleven Labs AI Hackathon Hackathon"

Team Idea

Submission