GPT-4 Powered App Creation and Evals Hackathon

Start

TruEra Opening Words

Hackathon Guide and Lablab Platform Walk-through

OpenAI Workshop

Workshop - Retrieval Augmented Generation with Pinecone

Workshop - Advanced Evaluation Methods with TruLens

General Q&A session on Discord

Workshop - Make your app reliable, cost-efficient, and fast with Portkey

Submissions Deadline

No matter what it is, we are going to break the record with generative AI!

Breaking

Ibtesam

Jaweria

AHMAD TALHA ANSARI

quantumai

MuhammadAsad_Ishfaq

Rumaisa

The ongoing dialogue between humans and AI not only showcases the remarkable capabilities of current technologies but also illuminates the future possibilities of AI-human synergy, promising an era where AI enhances human creativity, decision-making, and problem-solving in unprecedented ways.

Our hackathon project explored the interaction between humans and Large Language Models (LLMs) over time, developing a novel metric, the Human Interpretive Number (HIN Number), to quantify this dynamic. Leveraging tools like Trulens for groundedness analysis and HHEM for hallucination evaluation, we integrated features like a custom GPT-5 scene writer, the CrewAI model translator, and interactive Dall-E images with text-to-audio conversion to enhance understanding.

The HIN Number, defined as the product of Groundedness and Hallucination scores, serves as a new benchmark for assessing LLM interpretive accuracy and adaptability. Our findings revealed a critical inflection point: LLMs without guardrails showed improved interaction quality and higher HIN Numbers over time, while those with guardrails experienced a decline. This suggests that unrestricted models adapt better to human communication, highlighting the importance of designing LLMs that can evolve with their users. Our project underscores the need for balanced LLM development, focusing on flexibility and user engagement to foster more meaningful human-AI interactions.

In this hackathon, our group sought to answer one question: How do LLMs interpret us, and over a period of time, what is the result of our interaction with them?

Breaking Free

"Breaking team on GPT-4 Powered App Creation and Evals Hackathon Hackathon"

Team Idea

Submission