Agentic Feedback System for TalkToLoop

Over 100 million people are forcefully displaced, with 72 countries in crisis and over 300 million needing humanitarian aid. A record $48 billion has been allocated, but this funding falls short. Misallocation of resources further compounds the problem, reducing value and productivity growth. To ensure funds are allocated effectively and fraud minimized, it’s crucial to include those in need by gathering their feedback and complaints on local humanitarian projects. This presents challenges: (1) How can we centralize data from numerous organizations and projects? (2) How do we enable marginalized groups with low connectivity, education, or niche languages to share their concerns? (3) How can we make the data accessible and actionable? Talk to Loop is a modern independent charity, dedicated to tackling those issues. We have set up pipelines to allow populations in crisis zones to voice their concerns despite their lack of internet connectivity, or education, and regardless of their demographics and language. The pipeline includes classifying incoming data and tagging it with metadata to make it findable and actionable. A key step involves separating sensitive data from general feedback—protecting data that, if exposed, could endanger its author, like reports on rape, fraud, or child endangerment. Currently, this process is manual and unscalable, requiring humans to review each feedback piece and assign metadata manually. Our project automates this process, by leveraging a fine-tuned Llama model, and an agentic system using that model to assess whether the feedback is sensitive, annotate it, and analyze the sentiment of the text inside it. Our hope is that we can see a dramatic improvement in the time it takes to process the data, ensuring that the people in need of assistance receive it as fast as possible, and allows us to scale our operations to the point where the data is dictating resource allocation on a global scale.