.png&w=256&q=75)
🤓 Latest Submissions
TokenCascade
TokenCascade is built on one idea: in an enterprise AI stack, most tasks don't need a frontier model — they need the cheapest model that gets the answer right. The agent classifies each incoming task into one of eight capability categories (factual Q&A, math reasoning, sentiment, summarization, NER, code debugging, logic puzzles, code generation) using a lightweight, instant classifier, then routes it up an escalation ladder. Categories the local model is measured to handle reliably are answered by a quantized Qwen2.5-3B running entirely inside the container on CPU — zero Fireworks tokens. Accuracy-critical categories (math, logic, code) escalate to frontier models on Fireworks AI (MiniMax-M3 for general reasoning, Kimi K2 for code), selected at runtime from the allowed model list, with terse system prompts and per-category output caps so every remote token is spent deliberately. A global time watchdog keeps the run inside the execution budget: if local inference threatens the clock, remaining tasks flush to the fast remote path, and a never-empty fallback guarantees every task ships an answer. The result: measured on the practice set, roughly 1,000 Fireworks tokens for a full task batch with high answer accuracy — the routing pattern enterprises actually want, demonstrated end to end on AMD infrastructure with all model choice, category routing, and budgets tunable by environment variable, no rebuild required.
13 Jul 2026
