.png&w=256&q=75)
🤓 Latest Submissions
.png&w=828&q=75)
Intelligent Assistant
Project Overview & Architecture This project introduces a high-efficiency, zero-RAM hybrid LLM routing engine engineered explicitly for strict container environments limited to 4 GB RAM and 2 vCPUs. Traditional architectures rely on running smaller local models to intercept simple queries, but this approach triggers immediate Out-Of-Memory (OOM) fatal crashes on low-spec hardware. Core Innovation Our solution completely eliminates local model memory overhead by using pure Python micro-heuristics to parse input complexity in milliseconds. To maximize token efficiency without sacrificing accuracy, the router implements Dynamic System Prompt Injection (Soft Capping). Instead of using hard token cuts that truncate answers and cause API failures, the engine dynamically adjusts the system prompt instructions based on the task type—forcing single-word labels for classifications and concise responses for text processing. This installs intelligent natural "brakes" on the models, drastically lowering token consumption while successfully passing rigorous accuracy testing gates.
13 Jul 2026
