Khemamorn Chan@morn

Embark on a journey into the intricate world of company 10K filings with The Financial Pandalist – your premier ally in deciphering financial intricacies! Are you seeking expert advice or in-depth analysis on a particular company's 10K filing? Look no further! Our platform offers a seamless experience – just upload the 10K document of your choice and dive into a personalized chat with our financial experts. That is what we aim to do in our project. We mainly employ Retrieval-Augmented Generation (RAG) to our project. Our technology stack encompasses TruLens as an evaluation tool, LLM and Embedding model from VertexAI, LangChain for crafting diverse chain types, and ChromaDB serving as the vector store. Our goal is to compare four distinct chain-type techniques: Stuff, Map Rescue, Refine, and Map Re-rank. To assess their effectiveness, we employ the RAG Triad concept and evaluate each chain type across three dimensions: Context Relevance, Groundedness, and Answer Relevance. For evaluation purposes, we employ approximately 20 questions related to financial analysis. Leveraging TruLens, we observe that the Stuff and Refine chain types score notably high on Groundedness and Answer Relevance, respectively, while the others fall short. To enhance overall performance scores, several factors, such as varying prompts, need further investigation.