Routing Agent
The Cascade Routing Engine implements a three-phase inference pipeline: Local Phase: Queries are routed to a local model endpoint (default: Qwen/Qwen2.5-7B-Instruct on AMD MI300X) Self-Evaluation Phase: The local model evaluates its own response accuracy on a 0-100 scale Escalation Phase: If accuracy falls below 80%, queries escalate to Fireworks AI with intent-based model selection Zero-cost local inference when quality threshold is met (80%+ accuracy) Real-time Server-Sent Events (SSE) streaming for live process tracking Intent-based routing - automatically selects optimal Fireworks model: >conversational / declarative / computational → lightweight tier (8B model) >rag (research/lookup queries) → frontier tier (70B model) Cost transparency - real-time token usage and pricing display Graceful fallback - serves local response with disclaimer if cloud escalation fails
Category tags:
.png&w=96&q=75){kind=small}
For Innovators & Creators