Fallthrough

Fallthrough is a router for batches of mixed AI tasks. It looks at each prompt, works out what kind of task it is, and chooses the cheapest reliable way to answer it. Any answer produced entirely inside the container uses zero Fireworks tokens. Tasks with a provable local solution stay inside the container. Arithmetic is evaluated using a restricted AST interpreter, simple list operations such as sorting, counting, deduplication, minimum and maximum are handled directly, and supported transitive logic problems are solved using a relation graph. These local solvers only return an answer when they can determine it reliably. If they cannot, the task falls through to a remote model. Tasks that genuinely require language understanding are sent to Gemma through the Fireworks endpoint using short prompts tailored to the task. Gemma handles work such as factual questions, summarisation, sentiment analysis, entity extraction, reasoning and code generation. The returned answers are then checked where possible for things such as valid labels, numeric formats, entity spans and Python syntax. Fallthrough does not assume that local execution is always better, or that every task should be sent to the largest model. The repository includes an evaluation harness for comparing routes using accuracy, Fireworks token usage, latency, invalid outputs and failures. Local vLLM inference is also supported for benchmarking, but it is only added to the final routing policy if the results show that it is reliable enough. Each task is processed independently, so one failure cannot stop the rest of the batch. The final results are written atomically in the required output format.