SmartRouter — cut your LLM bill

Every LLM app today sends every request to a remote model—even tasks a calculator or simple rule could solve for free. SmartRouter treats routing as a verification problem: answers ship from the cheapest tier only when they can be proven correct, escalating only when needed. Tier 0 (0 tokens): ~30 deterministic solver families covering arithmetic, percentages, unit conversions, dates, primes, factorials, combinatorics, Roman numerals, statistics, equations, logic puzzles, sentiment, rule-based NER, and extractive summarization. Every solver self-gates and escalates on ambiguity. Tier 0.5: Program repair via AST rewrites and mutation search. Patches ship only if they pass tests extracted from the prompt. Tier 0.75: Template-based code generation (Fibonacci, FizzBuzz, validators, etc.) verified with self-tests and prompt examples. Tier 1 (0 tokens): Local Gemma 3 via Ollama answers factual, summarization, sentiment, and NER tasks. Self-consistency and self-critique verify confidence, while preprocessing strips filler so simple requests stay free. Tier 2: Fireworks models are tried from cheapest to strongest. Each answer is verified before escalation. Prompts are compressed, token caps enforced, and blank outputs blocked. SmartRouter also decomposes compound prompts, learns routing from live traffic, serves semantic cache hits with numeric safety guards, and degrades gracefully under token budgets. It's a drop-in OpenAI-compatible proxy: change only base_url. Every response includes a routing trace and a dashboard showing routing, tokens, and cost savings. Results: On a 40-task benchmark, SmartRouter matched the all-remote baseline's 100% accuracy using 0 Fireworks tokens. 31/40 tasks were solved deterministically and the rest by local Gemma, backed by 301 unit tests and a fully containerized implementation of the official Track 1 grading contract.