RouteMind: Token-Efficient AI Routing Agent

RouteMind is a hybrid, token-efficient routing agent built for the AMD/Fireworks hackathon. Instead of sending every task to an expensive large language model, RouteMind tries the cheapest path first and escalates only when necessary. The pipeline: an incoming task first checks a semantic cache (FAISS + MiniLM embeddings) for near-duplicate prior requests, served instantly at zero cost. On a cache miss, a lightweight classifier scores task difficulty and type, then a local model (Ollama, qwen2.5:1.5b) attempts the task with a self-consistency confidence check. If confidence clears a per-difficulty threshold, the local answer is returned for free. Only when local confidence is insufficient does RouteMind escalate to Fireworks AI, using a two-tier model ladder (gpt-oss-20b, then gpt-oss-120b) to keep even escalated calls as cheap as possible. Every request is logged with its route, token cost, and confidence score. A /stats endpoint aggregates this into real metrics: in testing, 68% of requests resolved locally at zero cost, 26% were served from cache, and only 5% required Fireworks escalation — an estimated 93% token reduction compared to routing everything through a large model by default. The system is fully containerized (Docker Compose), includes disk-backed cache persistence across restarts, automatic fallback if the local model is unreachable, and a self-eval harness for regression testing. Development required real debugging: diagnosing a classifier bug that misclassified trivial arithmetic as high-difficulty, and adapting the Fireworks escalation ladder after discovering the original planned Gemma model had lost serverless access mid-development — documented transparently in the README's Known Limitations.