tokentaxi

This project is a highly optimized, resource-aware AI agent designed specifically for the AMD Hackathon Track 1, where the goal is to maximize accuracy while aggressively minimizing API token usage. Operating safely within a strict 4GB RAM limit, the agent utilizes a native Python transformers pipeline to run a lightweight Qwen2.5-0.5B-Instruct model locally. A dynamic LangGraph orchestrator intercepts every incoming task, functioning as a "smart router." It forwards all basic text-processing tasks (like factual Q&A, sentiment analysis, and NER) exclusively to the free local model, ensuring zero API token expenditure for easy questions. However, to protect accuracy scores, it intelligently bypasses the local model for highly complex tasks- such as calculus, logic puzzles, and code generation- routing them instead to the Fireworks API. To mathematically minimize the token penalty on those API calls, the agent employs a suite of extreme optimization techniques. Before a prompt ever reaches the Fireworks API, it is aggressively compressed using a TF-IDF sentence pruner and a custom stopword-stripper that removes filler text without losing context. Furthermore, the router automatically selects the absolute smallest capable model for each specific category and enforces brutal output token caps, forcing the LLM to skip conversational filler and output only the raw answer. Finally, the system features a lightning-fast, dependency-free in-memory Python dictionary cache. This memoization layer instantly catches duplicate hackathon prompts, returning correct answers for exactly zero compute and zero tokens, making the agent incredibly cost-efficient.