SixthSense: Haptic Vision for the Blind

SixthSense is a wearable that helps blind and low-vision people sense obstacles around them and find a clear path. A phone is mounted on the chest and watches the way ahead. On-device models turn what the camera sees into a simple readout: how near obstacles are in the left, center, and right zones, what objects are present, and whether the path is clear. That readout drives a vibration belt worn at the waist, which buzzes on the side of the nearest obstacle so the user can feel which way to move. The point is that knowing something is close is not enough. A basic vibrating cane buzzes whenever anything is near, so in a crowd it buzzes constantly without telling you where the gap is. SixthSense reads each zone separately and steers the user toward open space, so it stays useful in busy areas. The user can also ask what is ahead and hear a short spoken answer, or point the camera at a sign and have its text read aloud. The vision runs on the phone. YOLOv11n detects objects and tags each to a left, center, or right zone. Depth-Anything-V2 estimates how near things are, which sets how hard the belt buzzes. Qwen2.5-0.5B answers spoken questions about the scene, and ML Kit reads text on demand. The models run through ExecuTorch as compiled files on the phone, offline, on a Qualcomm Snapdragon 8 Elite, with the option to run on the Hexagon NPU. The phone sends a small directional packet over Bluetooth to an ESP32, which drives the belt motors. Cost is the main reason it exists. Smart canes and glasses run from about $850 to over $2,000, and only one in ten people who need assistive technology can get it, dropping to about five percent in lower-income countries. SixthSense uses a phone the user already has and a sub-$20 belt, with room to reach about $50 at scale, putting this within reach of people who are priced out today.