GemmaRoute The Financial Firewall for EnterpriseAI

GemmaRoute is the financial firewall for enterprise AI infrastructure. The core problem: enterprises today route every customer support query, including trivial ones like "What are your business hours?" to expensive frontier cloud models, burning margins and wasting compute. GemmaRoute solves this with a 4-layer hybrid routing pipeline powered by Gemma 2 running on AMD ROCm hardware. Layer 0 is an instant pre-classifier that detects high-signal vocabulary (legal, SLA, fraud, contract) in zero milliseconds and bypasses the AI classifier entirely for unambiguous queries. Layer 1 runs a local Gemma 2B semantic classifier on AMD hardware, assigning every query a complexity tier. A confidence gate ensures low-confidence results are automatically bumped to a stronger tier. Layer 2 routes to the cheapest capable model: local Gemma 2B for simple queries at zero cost, and cloud models via Fireworks AI for medium and complex queries only when necessary. Layer 3 runs a local Gemma 2B quality judge that scores every response from 0.0 to 1.0 against a 4-criterion rubric. Any response scoring below 0.75 triggers automatic escalation to the next tier, with a hard cap of 2 escalations to prevent runaway costs. The system includes a production-grade circuit breaker that trips after 3 consecutive cloud failures and instantly reroutes all traffic to local AMD hardware for 60 seconds. Exponential backoff retries transient errors at 1s, 2s, and 4s intervals. A hop budget prevents infinite escalation loops. Gemma 2 powers all three intelligence layers: the routing brain, the local executor, and the quality judge, making GemmaRoute the only submission using Gemma in three distinct roles simultaneously. Built with FastAPI, LangGraph StateGraph, SQLAlchemy async, and a Next.js 16 frontend deployed on Vercel. 52 automated tests, all passing. Live demo at gemmaroute.vercel.app.