
Gulliver's Router reduces cloud usage by handling suitable prompts locally. It sends work to the cloud when the expected improvement in answer quality justifies the extra cost. Under the hood, it uses small classifiers trained on judged local and cloud responses. They flag prompts where the local model is likely to fall short, identify the task category, and let the runtime use specialist paths for jobs like named entity extraction and arithmetic. The result is a router that stays practical for resource-constrained environments. It also features a bailout if time draws near to the 10 minute mark - evacuating all local tasks to the cloud.
13 Jul 2026