AskTheWeb AI Website QnA Assistant

AskTheWeb: AI-Powered Website Question Answering System AskTheWeb (also known as WebMind AI) is an advanced Question-Answering application designed to tackle information overload by transforming static websites into interactive, conversational experiences. Built using Streamlit, the application leverages the speed and intelligence of Google Gemini 2.0 Flash to understand and synthesize web content in real-time. The core functionality relies on a robust RAG (Retrieval-Augmented Generation) pipeline designed for accuracy and persistence. When a user inputs a URL, the system employs Requests and BeautifulSoup to scrape and clean the HTML data, acting as an efficient ETL transformer to remove messy code and script tags. This cleaned text is converted into high-dimensional vector embeddings using Google’s GenAI SDK and stored in ChromaDB, which acts as the application's "Long-Term Memory". This architecture allows users—such as students, researchers, and analysts—to ask complex natural language questions and receive instant answers that are grounded specifically in the context of the provided URL. Unlike standard chatbots, AskTheWeb ensures that answers are relevant to the specific source material provided. We also addressed significant technical challenges during development, specifically ensuring compatibility with cloud environments. We implemented a custom solution using pysqlite3-binary to patch SQLite version incompatibilities on Streamlit Cloud, ensuring the vector database runs smoothly in production. The result is a scalable, modular tool that makes researching the web faster and more intuitive.