Thymus
Thymus is a lightweight hybrid token-efficient router designed to maximize accuracy while minimizing token costs in multi‑task LLM pipelines. It dynamically routes user queries across local and remote models on LLM providers. The general architecture of the router is as follows. 1. Classify: a fast regex/keyword pass handles the obvious cases for free; anything ambiguous falls back to the local model for classification. 2. Infer locally: the local model (Qwen2.5-3B-Instruct, GGUF, CPU) answers the query and self-reports a routing preference and confidence. 3. Verify: a task-aware verifier sanity-checks the answer (JSON structure, code execution, entity coverage, self-consistency, depending on task type). 4. Decide: LOCAL (free, done), SHEPHERD (local answer used as context, one cheap cloud pass to double check), or RESET (send straight to Fireworks).
Category tags:
.png&w=96&q=75){kind=small}
For Innovators & Creators