Cheap Route

CheapRoute is a cost-aware LLM routing agent that answers most tasks for free and only pays for the hard ones. Every incoming prompt first runs on a small Gemma model hosted locally on-device. Rather than trusting that answer blindly, CheapRoute scores its own confidence using three signals: token-level log-probabilities, self-agreement across multiple samples, and a format check that vetoes malformed or truncated output. When the combined confidence clears a tuned threshold, the local answer ships and the task costs zero remote tokens. Only genuinely uncertain cases escalate to a frontier model over an API. This design targets the common reality that most requests are easy: simple Q&A, classification, short code fixes, and summaries that a 1–4B local model handles well, while a minority of logic and math problems justify a bigger model. A wall-clock deadline cap bounds the local-plus-remote path so every response returns within the grading time limit, and the whole agent ships as a single self-contained Docker image with a remote-only fallback mode for safety.