Velora

Velora is a token-efficient hybrid AI inference pipeline built for the AMD Developer Hackathon Track 1. It implements a three-layer optimization strategy: 1. Persistent Normalized Cache All responses are stored in a cache keyed by normalized prompts (lowercased, punctuation-stripped). On repeated or similar queries, the agent returns answers instantly with no API call made — zero remote tokens consumed. 2. Confidence-Cascaded Speculative Routing Each incoming task is classified by type (factual, math, sentiment, summarisation, NER, debugging, logic, codegen) and first attempted by a locally-running Gemma 2B Q4 model via llama-cpp-python . If the local model clears the confidence threshold and passes structural validation (JSON schema check, Python syntax compile check), the answer is accepted at zero token cost. Only tasks that fail verification are escalated to Fireworks — MiniMax M3 for knowledge and math tasks, Kimi K2 for complex code and logic. 3. Symbolic Prompt Compression Before sending to any model, prompts are compressed using task-specific templates that replace verbose natural language with compact symbolic notation, reducing input token overhead while preserving full task semantics. Stack: Python 3.12, uv, llama-cpp-python, Pydantic, OpenAI SDK, Docker ( linux/amd64 )