Inbound Routing
This project is a hybrid token-efficient routing agent built for the AMD Developer Hackathon ACT II (Track 1), designed to minimize LLM inference costs without sacrificing accuracy. At its core is the Inbounding Router, a four-tier cascading pipeline that processes every query through semantic caching, prompt compression, complexity classification, and cascade verification before committing to an expensive model call. Layered on top is PHANTOM, a novel optimization system comprising three sub-components: a Shannon entropy-based confidence predictor that aborts unpromising local generations by token 8, a speculative parallel race executor that runs local and remote models simultaneously when routing confidence is ambiguous, and a dynamic budget enforcer that caps token spend per call based on predicted output length. Together, these systems answer a question most routing agents ignore: not just where to send a query, but whether it's already going wrong — and how to stop paying for it the moment it does. Benchmarks show up to 69% token savings with 96%+ accuracy retention and sub-10ms routing latency.
Category tags:
.png&w=96&q=75){kind=small}
.png&w=96&q=75){kind=small}
.png&w=96&q=75){kind=small}
.png&w=96&q=75){kind=small}
.png&w=96&q=75){kind=small}
For Innovators & Creators