AEROi: helping blind people move around with LiDAR, voice and haptics

Problem

Independence for blind people depends on knowing what is around them

A white cane finds what is directly ahead. It can't say that the obstacle is a chair, that the door is a metre away, or which way to turn. Many blind people end up asking a carer or a passer-by, which costs them independence and privacy.

Recent Pro iPhones include a LiDAR scanner that measures distance to the surfaces in front of it. I wanted to see whether that sensor, with voice and haptics, could give blind people more awareness of their surroundings.

Research

Few people knew of assistive apps, and fewer believed they help

I ran a Google Forms survey aimed at carers and at people who knew someone blind, asking about daily routines, the technology they used, and whether they preferred voice or haptic feedback. It received 39 responses. Only a quarter knew of any assistive apps for blind people, and fewer than four in ten of those who answered thought such apps really help.

From this and from desk research into LiDAR, I wrote a proto-persona: Nethan, a 26-year-old blind law student who hates having to ask strangers or his carer for help and wants more privacy. Because this was a proto-persona built on assumptions rather than interviews with blind people, I treated everything that followed as hypotheses to test.

Framing

Three hypotheses, and one riskiest assumption

Instead of a single problem statement, I framed the work as Lean UX hypotheses. 1. LiDAR and AI can give Nethan accurate information about his surroundings, so he comes back to the app. 2. GPS, the accelerometer and the gyroscope can give support workers the location and health data they need to care for someone. 3. A wearable device with AI text-to-speech gives immediate feedback and, in effect, a new sense.

The canvas identified the riskiest assumption: older people, whether blind users or carers, would struggle to understand the interface. That shaped the decisions below.

Decision 1

Voice and vibration lead, and the screen only backs them up

A blind user can't read a busy screen, so every guidance screen carries a single large symbol and one short spoken line, such as 'Walk 15 steps straight from the current position'. Distance is also given as sound: a higher pitch means more space, a low pitch means an obstacle is close, and the phone or glasses vibrate when something is right in front. Object detection names what the camera sees, such as a chair, a door or raw metal.

The large symbols are there for people with some sight and for carers looking on. The trade-off is that audio can fail in a noisy street, which is why haptics give the most urgent warning, not voice alone.

Decision 2

Separate modes for the blind user and the support worker

My survey was aimed at carers, and the Lean UX canvas named care workers who need accurate data as a key customer. Instead of a separate app, a support worker chooses their own mode at sign-in. It shows the person's live location, activity, health status and an SOS function, while the blind user's mode stays voice-led and minimal.

This is where the hardest tension in the project sits. Nethan wants more privacy, and a tracking feature takes some away. A real version would need the blind user to control who sees what and when, and it would need consent built in from the start. My prototype doesn't solve that yet.

Decision 3

Phone first, with glasses as the hands-free version

LiDAR already exists on the iPhone 12 Pro and later, so the app works on a phone with no new hardware. But a blind person walking often has a cane in one hand, so holding a phone up is awkward. I modelled a concept pair of glasses with a LiDAR sensor at the bridge and a haptic motor and speaker in the arm. They connect over Bluetooth, and a short calibration (height, weight and body type) tunes the sensor to the wearer.

The glasses are speculative. I haven't validated their range, battery life, cost or comfort. Indoor mode lets people use the phone at home without wearing them.

Solution

The prototype: speak a destination, follow the cues, hear what's around you

I built the hi-fi prototype in Adobe XD with a dark interface, high-contrast green and violet, and recorded text-to-speech prompts. The cut below shows setup, user selection, step-by-step guidance, the support worker's map and object detection.

Validation

A small test showed the flows were clear to sighted users, so blind users are the next test

Two sighted participants used the clickable prototype on screen to complete three tasks. They gave System Usability Scale scores of 85 and 72.5 (mean 78.75). They also rated how far they agreed with each of my three hypotheses, from 80% to 100%. With two sighted people, this only shows that the flows made sense to them.

What I'd do next: recruit blind and low-vision participants through a charity such as the RNIB, test with VoiceOver on a real device, and measure whether they reach a destination without help, how many obstacles they miss, and how many times they have to ask someone. I'd also run a separate study on whether users trust the tracking in support worker mode.

Reflection

What I'd do differently

I designed for blind people without speaking to any, and that is the biggest gap. Next time I'd start with interviews and co-design sessions with blind and low-vision people, before choosing any technology. I'd also design the voice interaction first, with the visual interface derived from it, and I'd keep the support worker's data off by default.