Edward

Edward is a local-first AI desktop assistant built for the IBM Bob Hackathon. Inspired by Jarvis from Iron Man, Edward runs silently in your system tray and activates instantly via a hotkey (Win+Shift+E) or wake word ("Hey Edward"). IBM Bob serves as the primary reasoning engine. For fully offline operation, Edward switches to Ollama running local LLMs directly on your hardware. Screen understanding is handled by Moondream, an on-device vision model that describes what is on your screen without uploading anything to an external service. Users can switch between four engine modes in real time from the UI: IBM Bob Only, Ollama Only, Hybrid Local (Moondream + Ollama), and Hybrid Gemini (Moondream + Gemini 2.5). Edward goes far beyond a chat interface. It controls your mouse and keyboard autonomously via a FastAPI computer control server, letting IBM Bob execute tasks on your behalf by opening apps, typing text, and clicking buttons. It captures a screenshot before every query so responses are grounded in what is actually on your screen. The assistant also includes 30+ slash commands for instant local tasks: searching Google, Bing, or YouTube, controlling Spotify, reading the clipboard, saving screenshots, checking system info, and opening any of 20+ supported applications by name. An AES-128 encrypted password vault stores credentials locally and never touches a network. Voice input uses faster-whisper for speech to text. Voice output uses ElevenLabs for text to speech. The UI is built in PyQt6 with a dark Iron Man inspired overlay.