Most AI agents send every question to the same model, no matter how simple or complex it is. That wastes tokens on easy tasks that never needed a powerful model in the first place. This agent takes a different approach. Every incoming task first goes through a classifier that figures out what kind of question it is: factual knowledge, math, sentiment, summarization, named entity recognition, code debugging, logical reasoning, or code generation. Based on that, it decides where the task should go. Five of these categories (factual knowledge, sentiment, summarization, math reasoning, and named entity recognition) are handled entirely by a small language model that runs locally inside the container, on CPU, using no Fireworks tokens at all. The model is Qwen2.5-1.5B-Instruct, quantized down to about 940MB so it fits comfortably within the grading environment's memory limits. The remaining three categories (code debugging, code generation, and logical reasoning) are routed to Fireworks, since these tasks benefit more reliably from a larger model's reasoning ability. I tested this agent against all eight official practice tasks plus several additional prompts written in different phrasing than the originals, to make sure the classifier generalizes and doesn't just memorize the examples. Every task, across both local and remote paths, was answered correctly. The result is an agent that keeps its Fireworks token usage low by only paying for the tasks that actually need it, while still passing the accuracy bar on every category.
Category tags: