.png&w=256&q=75)
🤓 Latest Submissions
TokenRouter: Cost-Efficient Multi-Model AI Agent
TokenRouter is a containerized general-purpose AI agent built for enterprise cost control: not every task needs a premium model. It reads tasks from /input/tasks.json, classifies each into one of eight capability categories, and routes it to the cheapest Fireworks model that still clears the LLM-judge accuracy gate—then writes /output/results.json. The core insight is that routing is nearly free if done locally. Category detection runs entirely on local regex/structural rules (zero scored tokens), across three axes: (1) an ordered rule set that distinguishes, e.g., "debug this code" from "write code" from a factual question, including a multiple-choice structural detector for deductive-reasoning tasks; (2) a calibrated model table mapping each category to its cheapest sufficient model, with automatic fallback if a model isn't in the launch-day allowlist; and (3) an input-size gate that escalates to a larger-context model only when an input genuinely wouldn't fit. Token efficiency comes from terse, category-specific system prompts and output-token ceilings tuned per category. Every Fireworks call is bounded by a per-task wall budget so no request breaches the 30-second limit, with a one-tier fallback for robustness. Results: 100% accuracy on our development set and 86% across public benchmarks (TriviaQA, GSM8K, SST-2, XSum, CoNLL-2003, LogiQA, HumanEval), at roughly 360 tokens per task. The final image is a 161 MB linux/amd64 container that starts in seconds, reads all credentials and the model allowlist from the environment at runtime, and hardcodes no answers—so it generalizes to unseen prompt variants.
13 Jul 2026
