Confidence-Cascade Routing Agent

Our project, the "Confidence-Cascade Routing Agent," is designed to solve the exact problem enterprises face today: balancing AI capabilities with strict API cost controls. Built for Track 1 of the AMD Developer Hackathon, this agent tackles a wide variety of natural language tasks (from factual Q&A to math and code debugging) while aggressively minimizing expensive API calls to premium models. The core innovation is our Confidence-Cascade Architecture. By default, the agent attempts to solve every incoming task locally using a lightweight, 4-bit quantized gemma-3-4b-it model running entirely on the CPU within the container's tight 4GB RAM limit. We implemented domain-specific evaluation logic: the agent inspects the local model's output and assigns it a confidence score. If the local answer passes our hard validators (e.g., proper JSON formatting, exact string matching) or clears our confidence threshold, the agent keeps the local answer—costing absolutely zero API tokens. Only when the local model fails or expresses uncertainty does the agent escalate the task. It routes these difficult edge cases through the Fireworks AI proxy to a massive gemma-4-31b-it deployment. We also engineered custom token budget multipliers to handle the 31B's "thinking" mode, ensuring it has enough runway to reason through complex math and code before returning the final answer. The result is a robust, highly competitive router that achieved 87.5% accuracy on the practice suite while keeping 37.5% of tasks entirely local, drastically reducing total token usage without sacrificing capability.