Vote First, Then Answer

Every question sent to a language model costs something — but not every question needs the same amount of firepower. This project is a routing agent built around one core decision: a combined vote, where simple pattern rules and semantic similarity against a library of example questions contribute together into a single score, rather than one step gatekeeping the next. When that vote is confident, the decision is immediate — straight to a free local model for easy questions, straight to a paid model for genuinely hard ones, with no wasted attempts either way. When the vote is genuinely unsure, the local model gets a real chance to prove itself: it answers the question twice, and is only trusted if both answers actually agree with each other. If it hesitates, the bigger model takes over instead. Getting here also meant catching real mistakes along the way — a routing rule that was confidently wrong on certain questions, a local model that ran fine on one machine but crashed on another over a missing CPU instruction, a token cap that silently cut answers short instead of finishing them. Each one got tracked down and fixed with real evidence, not guesses. The whole thing was built and stress-tested against real constraints: a small memory budget, a strict time limit, and every kind of messy or unexpected input we could think to throw at it, from empty prompts to attempted prompt injections. The goal throughout stayed simple — answer correctly, spend as little as possible, and never fail loudly when it can fail gracefully instead. (~1,550 characters, ~250 words) Categories / Event Tracks Track 1: General-Purpose AI Agent Technologies Used Python, Docker, llama.cpp / llama-cpp-python, Qwen2.5-3B-Instruct (GGUF, 4-bit quantized), Fireworks AI (gpt-oss-20b), sentence-transformers (all-MiniLM-L6-v2), httpx Want any of these adjusted — shorter, more casual, a different title option — before you paste them in?