🤓 Latest Submissions
.png&w=828&q=75)
penny — zero-token-first LLM routing agent
Track 1 rewards the agent that clears the accuracy gate while spending the fewest remote tokens — and local inference is free. penny is built around that asymmetry: the optimal move is knowing, per task, that you do not need a remote call at all, and proving it before answering. Every task flows through a zero-cost funnel: an exact cache, deterministic matchers (arithmetic, percentages, unit fractions) that solve and verify in one pass, then a two-pass local stage — a fast Gemma 3 1B answers the whole batch, and tasks that fail verification are re-answered by Gemma 3 4B, swapped into the same 4GB memory budget so both models are never resident together. Math is solved by program-aided reasoning: the model writes Python and penny executes it in a sandbox — executed code beats small-model mental arithmetic, and this path passes all five public validation rubrics including the multi-step warehouse problem. Per-category verifiers check every answer for free: sandboxed code execution against examples parsed from the prompt, numeric cross-checks, format compliance (exact bullet counts, labelled NER entities including dates), and seeded self-consistency. Only tasks that still fail after both local tiers escalate to Fireworks — ranked globally by verifier weakness, capped per category — so the common path spends zero remote tokens. A boot-time micro-benchmark adapts the model tier to the judge machine's speed; deadline modes degrade gracefully under the 10-minute batch budget; the container always writes valid results.json and exits 0. 240+ unit tests, fully offline apart from the sanctioned Fireworks path, MIT licensed.
13 Jul 2026
