.png&w=256&q=75)
🤓 Latest Submissions
.png&w=828&q=75)
NeuralGate:Intelligent Multi-Model AI Routing Eng
NeuralGate solves a core AI tradeoff: routing every query to a costly cloud model wastes money, while relying only on small local models sacrifices quality. NeuralGate eliminates this by intelligently classifying each query and routing it to the most cost-efficient capable model, leveraging AMD GPU hardware for local inference with cascading fallback to larger models when needed. 3-Tier Classification Engine (100% uptime): SLM Classifier — A QLoRA fine-tuned Llama-3.2-1B (llama3-router), trained on 5,000+ synthetic prompts across Math, Coding, Research, and Casual Chat; runs on Ollama with millisecond latency. ChromaDB Semantic Fallback — Vector similarity matching using local nomic-embed-text embeddings if Tier 1 fails. Regex Safety Net — Deterministic keyword matching, zero dependencies, guarantees output always. Routing: Configs live in routing_rules.yaml. Math→gemma-4-31b-it, Coding→kimi-k2p7-code, Research→gemma-4-26b-a4b-it, Casual→minimax-m3 — each with fallback models. Failures cascade silently, verified via failure-simulation testing. Dashboard: React/Vite dashboard tracks model usage, cost savings, and classification distribution, backed by SQLite in WAL mode (150 concurrent writes, zero lock errors). MLOps: MLflow-backed retraining pipeline (validation → preprocessing → training → registry promotion) with rollback support. Quality: FastAPI backend, 84 tests, 76.48% coverage, CI/CD via GitHub Actions, Docker Compose deployment. A production-ready pattern for cost-aware AI on accessible hardware.
12 Jul 2026
