TruEra Challenge: Build LLM Applications

Get Prepared With TruLens!

Evaluation: TruLens empowers you to assess the quality of your LLM-based applications by evaluating inputs, outputs, and internal workings. It offers built-in feedback mechanisms, including groundedness, relevance, and toxicity, while being adaptable to custom evaluation needs.

Tracking: TruLens provides essential instrumentation for a range of LLM applications, from question answering to retrieval-augmented generation and agent-based solutions. This instrumentation allows you to monitor diverse usage metrics and metadata, providing valuable insights into your model's performance.

TruLens Resources

To kickstart your journey with TruLens, explore the following resources:

GitHub Repository - Access the TruLens codebase and contribute to its development.
Quick Start Guide - A step-by-step guide to quickly set up and start using TruLens-Eval.
GitHub Repo - Visit the TruLens website for comprehensive project information and updates.

User Case Guides

TruLens is adaptable to a variety of LLM applications, including:

Any LLM App
RAGs (Retrieval-Augmented Generative models)
LLM Agents
Dev to Prod

Get started with TruLens today and unlock the potential of your LLM-based applications.

Google Cloud Credits

Access Google Cloud credits to build with your hackathon project. We provide a few ways to get access to Vertex AI products for Hackathon Participants:

Entry Google Cloud Benefits

$300 free tier for all users, allowing exploration of Vertex AI without charges.
Free credits for students, educators, and institutions, supporting educational initiatives.
Dedicated free credits for researchers, enabling AI-driven data analysis and model training.

Hackathon Team Credits

Teams formed for the TruEra Challenge will qualify for $30 in credits per team member to utilize throughout the Hackathon.

These credits will be issued shortly after the start of the hackathon through several stages. It is advantageous for participants to promptly join a team at the beginning of the hackathon to expedite the credit allocation process.

Alternative Infrastructure for the Hackathon

If a team reaches their credit limits on their accounts and requires additional credit coverage, lablab.ai offers infrastructure support throughout the hackathon. This serves as a backup option, ensuring teams have uninterrupted access and comfort while using Google Cloud Vertex AI products.

To access the service account created for the hackathon, submit the form on the hackathon team page. Ensure you're the team leader of a registered team before completing the form. Your access details will be emailed to you, allowing for uninterrupted project development.

Hackathon Challenge

Your challenge is to utilize the power of AI observability to enhance your LLM-based application, with cutting-edge technology from TruEra and Google Cloud Vertex AI.

Submission Requirements

Your project must utilize both TruLens technology and Google Cloud Vertex AI products to be eligible for submission and participation in the evaluation process.

We recommend using the LlamaIndex framework and Zilliz vector database for your Hackathon app. This choice supports scalability and ensures direct assistance from Zilliz and LlamaIndex teams, potentially boosting your app's success!

Hackathon Prize

🥇 Hackathon Winner: $4,000 in cash prize + $5,000 Google Cloud credits
🥈 2nd Place: $3,000 in cash prize + $3,000 Google Cloud credits
🥉 3rd Place: $2,000 in cash prize + $2,000 Google Cloud credits
2 Hackathon Finalists: $2,000 Google Cloud credits

Furthermore, each team that submits a solution will receive $100 in Google Cloud credits per team member. These credits will be distributed only after the submission deadline. Please note that up to 500 participants who are part of a team that submits a solution will be eligible to receive the credits. The credits will be distributed to the email address used by the participant to register on lablab.ai.

Your challenge is to utilize the power of AI observability to enhance your LLM-based application, with cutting-edge technology from TruEra and Google Cloud Vertex AI.

Please note: The total combined value of both cash prizes and Google Cloud credits is up to $73,000 USD.

Cash prizes are awarded in USD. While we endeavor to expedite the distribution process with our partners, the delivery of prizes may take up to 90 days. lablab will not be held responsible for delays or inability to deliver prizes for this event as it relies solely on the event partners (TruEra & Google Cloud) to receive the total prize amount and distribute it.

Additionally, while the lablab team will endeavor to make their best effort, there is no guarantee of entry, offer, or reward of any of giveaways or prizes, and such entry, offer, or reward will be subject to their respective availability, conditions, and certain criteria that will need to be met. lablab reserves the right to amend, modify or cancel any of the items or terms of this event or even the event in its entirety. While lablab is not obligated to, it will endeavor to inform participants within a reasonable timeframe of such changes.

Success Criteria

Address a real-world problem from any industry of your choice.
Showcase your application's adept utilization of TruLens by TruEra.
Your application should feature a comprehensive observability suite and score impressively on evaluations.
Tell a compelling story about how you harnessed observability to pinpoint opportunities for enhancing your application.
While it doesn't have to be production-ready, your application should demonstrate practicality and user-friendliness, leaving users excited about its potential.

Technologies

TruLens by TruEra

TruLens is a versatile toolkit by TruEra, encompassing TruLens-Eval for LLM and LLM-based application evaluation, and TruLens-Explain for deep learning explainability. These tools offer developers valuable insights to optimize AI applications effectively.

Partner's Technologies

Google Cloud Vertex AI

Vertex AI provides the tools you need for swift innovation through generative AI. With AI solutions, powerful Search and Conversation capabilities, and access to 100+ foundational models, it's your gateway to streamlined AI development.

Zilliz

Zilliz offers a cutting-edge Vector Database tailored for Enterprise-grade AI and LLM applications. This advanced technology serves as a robust foundation, empowering developers to optimize their AI projects with efficiency and scalability.

LlamaIndex

LlamaIndex, an open-source data framework, streamlines the connection between custom data sources and large language models, enhancing AI applications by enabling data ingestion, indexing, and querying capabilities.

Get Started With TruLens!

To dive deeper with the Hackathon technology, check out guide on the evaluation and tracking of LLM experiments with TruLens.

Build better AI with TruLens - an open source platform for testing and analyzing large language models. This video guide provides an overview of key TruLens features like comprehensive testing, explainability analysis, and robust metrics. See how TruLens helps minimize risks and ensures greater accuracy in your LLM experiments.

Transitioning Your Submission from the TruLens & Google Cloud Vertex AI 48-Hour Hackathon

If you've previously participated in the TruLens & Google Cloud Vertex AI 48-Hour Hackathon, your creations can be the part on the main challenge. Develop your submission for the main challenge, adhering to these key requirements:

Define your chosen feedback functions for evaluating your application's success.
Submit evaluations from your initial prototype and the final version, highlighting improvements made with TruLens.
Focus on building retrieval augmented generation (RAG) applications, in line with the main challenge's objectives.
Ensure you highlight your project's development timeline, starting from the micro hackathon through to the advancements during the main event.
To join the TruEra Challenge with the same team, create a new team on the hackathon dashboard and invite your team members.

This is your chance to elevate your previous work and make it shine even brighter. Let's push the boundaries of AI innovation together!