Token Gate

TokenGate is a hybrid LLM router built for the AMD Developer Hackathon ACT II that dynamically routes user prompts between a lightweight local model (Ollama/llama3.2:1b-instruct-q4_K_M) and paid cloud APIs (Fireworks AI) to maximize accuracy while minimizing token costs... The workflow operates through a multi-stage decision pipeline. Incoming prompts are first classified by category (code, math, reasoning, creative, factual Q&A, general) using keyword analysis. Most tasks are initially sent to the local model, which returns structured responses with confidence scores. The system evaluates this confidence against a configurable threshold. For math and reasoning tasks, a self-consistency check re-runs the prompt at a different temperature — if the two answers disagree, the task escalates to the cloud. Tasks requiring long-form outputs bypass local inference entirely and go straight to the cloud... When escalation occurs, tasks are routed to specialized cloud models — code tasks to kimi-k2p7-code and general tasks to minimax-m3 — with budget guards preventing overspending. The system is containerized with Docker using a multi-stage build that bakes the compact local model into the image, keeping the compressed size under 800 MB for fast pull times within the hackathon's 10-minute constraint. It supports batch processing, an interactive CLI, and a Streamlit web UI. The scoring model rewards both accuracy and token efficiency, making this hybrid approach a strategic competitive advantage... Team: Feil Jasper Doria, Julianna Raine Lacaden, Francie Galapate, Vincent Rafael Fajardo.