ScubaaaaRouter

Our project is an intelligent, cost-efficient LLM routing agent designed to solve the common trade-off between performance and API costs in Generative AI applications. At its core, the agent orchestrates requests between a lightweight, locally-hosted model (Qwen 4B running on CPU via GGUF and llama-cpp-python) and a powerful remote frontier model. To determine the most efficient route for each query, the system employs a multi-step verification pipeline. First, a blazing-fast 51M-parameter SupraRouter model acts as an initial pre-screener, instantly flagging complex logic or math questions for immediate escalation without wasting local compute. For remaining tasks, the local Qwen model generates an initial response alongside a zero-cost deterministic format validator. If a local model confidently answers and passes the structural checks (e.g., producing valid code or correctly formatted summaries), the response is returned immediately. Only when the local model is uncertain or fails validation does the agent compress the prompt and escalate to the remote model.