Hybrid-Router

This project is a cost-aware intelligent routing system for Fireworks AI models that automatically selects the most appropriate large language model for each incoming request. Rather than sending every prompt to the same model, the router analyzes the input and dispatches it to the model that offers the best balance of accuracy and token efficiency. The system supports eight core task categories: factual question answering, mathematical reasoning, sentiment analysis, text summarization, named entity recognition (NER), code generation, code debugging, and logical reasoning. A lightweight routing layer determines the optimal model from the allowed Fireworks models based on task characteristics and learned routing policies. To further reduce inference costs, the router employs task-specific prompt optimization, structured output constraints, and token-aware generation settings. An evaluation pipeline benchmarks each available model across representative prompts, measuring accuracy and token consumption. These results are used to continuously improve routing decisions and maximize performance while minimizing API usage. The entire solution is packaged as a Dockerized service, making it easy to deploy and evaluate in standardized environments. The architecture is designed to achieve high accuracy while significantly reducing token consumption compared to a single-model approach, making it suitable for production AI systems where cost, scalability, and performance are equally important.