EchoWalk: On-Device Guidance for Low-Vision Users

Imagine walking through an unfamiliar room with your eyes closed. You need to know what is ahead, what is around you, and how to reach the chair someone mentioned — without cloud latency or sending your camera feed anywhere. EchoWalk is built for that moment. On a Galaxy S25 Ultra, one shared camera pipeline feeds a central ModeManager that decides when to warn, when to describe, and when to search — all on the Snapdragon NPU via ExecuTorch and Qualcomm QNN. Safety Radar runs continuously. Depth Anything V2 and YOLOv10 fuse on the Hexagon NPU: not just what is there, but how far and whether it is a trip hazard or a wall you can trail. Spatial audio and haptics place obstacles in space; a VoiceWarningEngine speaks when it matters. A live bounding-box overlay helps sighted helpers follow along in demos. Scene Description is on demand — tap the preview, the Describe button, or long-press Volume Up. A short burst of frames runs through a Places365 classifier and pairs the room label with live YOLO directions: "You appear to be in a living room — couch on your left, TV ahead." Auto-describe announces stable scene changes hands-free. The full SmolVLM-500M stack is integrated and validated through handoff scripts; richer VLM captions are ready for the next aligned build. Find Mode is voice-first. Long-press Volume Down, say "find the bottle," and the app maps your words to everyday object labels. It scans the room, guides you turn by turn, warns about obstacles in your path, highlights the target on screen, and remembers where it last saw it so the next search starts with a hint. Accessibility is front and center: lock-screen access, screen-on at launch, spoken onboarding with a first orientation from live radar, eyes-free volume shortcuts, and double-tap to repeat your last description. No cloud. No upload. Your home never leaves your pocket.