TokenRouter: Cost-Efficient Multi-Model AI Agent

TokenRouter is a containerized general-purpose AI agent built for enterprise cost control: not every task needs a premium model. It reads tasks from /input/tasks.json, classifies each into one of eight capability categories, and routes it to the cheapest Fireworks model that still clears the LLM-judge accuracy gate—then writes /output/results.json. The core insight is that routing is nearly free if done locally. Category detection runs entirely on local regex/structural rules (zero scored tokens), across three axes: (1) an ordered rule set that distinguishes, e.g., "debug this code" from "write code" from a factual question, including a multiple-choice structural detector for deductive-reasoning tasks; (2) a calibrated model table mapping each category to its cheapest sufficient model, with automatic fallback if a model isn't in the launch-day allowlist; and (3) an input-size gate that escalates to a larger-context model only when an input genuinely wouldn't fit. Token efficiency comes from terse, category-specific system prompts and output-token ceilings tuned per category. Every Fireworks call is bounded by a per-task wall budget so no request breaches the 30-second limit, with a one-tier fallback for robustness. Results: 100% accuracy on our development set and 86% across public benchmarks (TriviaQA, GSM8K, SST-2, XSum, CoNLL-2003, LogiQA, HumanEval), at roughly 360 tokens per task. The final image is a 161 MB linux/amd64 container that starts in seconds, reads all credentials and the model allowlist from the environment at runtime, and hardcodes no answers—so it generalizes to unseen prompt variants.