TERA: Hybrid Token-Efficient Routing Agent

TERA (Token-Efficient Routing Agent) is a hybrid, multi-stage routing system designed for ACT II — Track 1 of the AMD AI Developer Program. Operating within a strict CPU-only container limit (2 vCPUs / 4GB RAM), TERA utilizes an optimized local Qwen-2.5 1.5B model running via Ollama alongside a remote DeepSeek-V3 API to dynamically balance cost and accuracy. Key Architecture & Features: Dynamic Task Routing: Automatically detects task difficulty and category, delegating lightweight categories (sentiment, summarization, simple facts) to the local model, while routing advanced categories (coding, logic, NER) to remote endpoints. In-Container Model Footprint: GPU backends (CUDA/Vulkan runtimes) were trimmed to compress the Ollama footprint. Direct --chown COPY instructions eliminated duplicated layers, keeping the final Docker image at a lightweight 2.28 GB. Token Savings (46.3%): Integrates regex prompt compression, semantic response caching, and custom system instructions requesting direct answers to eliminate conversational preamble and save completion tokens. Rule-Based Math: Trivial arithmetic is solved locally for 0 scored tokens before routing. Under full evaluation, TERA scores 95.1% accuracy (39/41 tasks passed), resolving 51.2% of tasks locally for 0 scored tokens, with a peak memory footprint of only 1,466 MB and initialization boot time of 1.0 second.