Falcon Barsita

Who are we? We are a new startup dedicated to revolutionizing the restaurant industry with cutting-edge AI solutions. This Hackathon provides us an opportunity to showcase an early concept of our chatbot (still significantly in development) built on top of Falcon LLM. The Problem As the restaurant industry continues to recover from the coronavirus pandemic, it is confronting numerous challenges, with the foremost and most significant being high labor costs. The Solution We introduce Falcon Barista, an order-taking bot for coffee shops and restaurants, designed to converse with customers in the most human-like manner possible. Although it is still under development, this bot is envisioned as an affordable alternative for restaurants to replace manual labor at counters, drive-throughs, and over the phone. What makes Falcon Barista better? The primary innovation behind Falcon Barista lies in its minimal compute requirements, thereby maximizing cost savings. While many chatbots are built on top of LLMs with over 100 billion parameters, Falcon Barista operates on the much more compact fine-tuned Falcon-7B LLM, which consists of only 7 billion parameters. This efficiency is realized by employing smaller fine-tuned BERT models in tandem with the Falcon LLM: the Falcon-7B LLM guides the conversation while BERT manages information extraction, such as identifying food items and their quantities. Falcon Barista utilizes a quantized version of the Falcon-7B LLM and can be deployed on a single GPU with 16GB RAM. Furthermore, it boasts Automatic Speech Recognition and Text-to-Speech capabilities, allowing for conversations with customers that mimic human-like interactions. Challenges (Due to lack of compute resources) 1. Significant latency (~10s). 2. The BERT model, still in the process of fine-tuning, can easily become confused. 3. Falcon-7B requires further fine-tuning for more efficient conversation management.