ArmMind: NL Task Planner for Robot Pick & Place

Robotic pick-and-place systems are typically hard-coded: every task sequence is scripted by an engineer, and operators can't simply tell the arm what to do. ArmMind solves this by placing an LLM agent between the human operator and the robot's perception and control stack. Given a plain-English instruction like "pick up the blue gear and place it in bin 2," ArmMind reasons over the current workspace state (objects and bins) and outputs a structured, validated JSON action plan specifying exactly which object to pick and where to place it, along with its reasoning and confidence level. This project directly extends a prior capstone: an OpenCV-based robotic arm that performed autonomous object detection and pick-and-place sorting using hard-coded task logic. ArmMind replaces that fixed logic with an agentic reasoning layer powered by Gemma via Fireworks AI, running on AMD infrastructure, so the same perception pipeline can now be driven by natural language instead of pre-written rules. The result is a step toward natural-language-controllable industrial robotics, applicable to manufacturing, warehouse sorting, and other automated handling tasks.