Berkeley scientist designs tools for the visually impaired

Mar 2, 2016

 

Dr. Joshua Miele's morning commute to Smith-Kettlewell Eye Research Institute takes about an hour—as long as no one gets in the way. In fact, most people move out of his way when they see him coming, because Miele is blind.

 

He’s one of about 140,000 blind and visually impaired people in the Bay Area. Less than three quarters of those people are employed, and only a few work in tech, like Dr. Miele.

Uncommonly Universal

His office in Pacific Heights is full of tactile art, project binders, awards, and audio equipment. On a round meeting table, magnetic building toys sit in a cluster—just something to fiddle with on long calls. This is where Miele does the work he’s uniquely suited to do, designing accessible technologies for the blind.

“If you want to design a good boat,” he says, “you don’t ask somebody who doesn’t know the first thing about sailing. Blind people must be integral to the design process, not just as users that do the testing at the end, but as designers and engineers who do the thinking at the very beginning.”

Miele says most developers don’t even think about accessibility until it’s too late, because so few are trained in universal design. That’s a design principle that says, ‘find a single solution that will consider the broadest possible spectrum of human ability.’ It’s something Miele strives for every time he takes something on, though he mainly focuses on visual impairment.

“I want to build cool stuff for blind people that gives them the ability to do the things that they want to do, and I feel incredibly lucky that I'm given that privilege and opportunity.”

He can pretty much choose his projects, too, because Dr. Miele directs his own innovation lab here at Smith-Kettlewell.

The Moment Things Changed

But getting to this place in his life wasn’t easy.

“I grew up in New York. I was a sighted kid until I was almost five, and then I was burned in a very violent act where somebody threw sulfuric acid over my head.”

That’s the chemical that powers car batteries. A single drop can dissolve concrete. Thankfully, specialists at a San Antonio military hospital were able to save Miele’s life. But not his eyes. His left eye is now a synthetic blue-gray, and as he speaks, the shadow of his right eye darts beneath the scar tissue that covers most of his face, all haloed by his thick, curly black hair.

“My personal story is shocking and frightening and painful and often it’s difficult for people to get past that, I think.”

Although, he says where he lives, people seem to take it in stride.

“Difference and diversity is much more accepted, even embraced here. Being a burned, one-eyed blind guy in Berkeley doesn't even get you a second glance most of the time.”

The Pursuit of Science

The acceptance he found in Berkeley went inward, too. Coming to study physics at UC Berkeley at age 18 changed his entire outlook.

“I met a whole community of cool blind people who I respected and liked and who were creative and smart and funny,” he says. ”I realized that I did want to be a blind person. I could be proud of being a blind person.”

Even so, working in accessibility wasn’t Miele’s plan. His dream was space science, so he studied physics and interned at NASA. But every time he took on something new, he struggled. He wasn’t able to do a lot of things that sighted students took for granted, like reading a cafeteria menu, or riding a bike to class. Or, more importantly, being able to use the school’s scientific software, which he had to do, to earn his PhD.

“The first thing I had to do as a graduate student in psychoacoustics was to develop a set of sonification and tactile representation tools for this package called Mat Lab. It’s basically like excel on steroids,” he explains. “I needed to be able to look at that data and so I made some software tools that would turn that data into sound and tactile charts."

He explains that having to do that much extra preparation for almost everything you take on, gets you thinking.

“I started learning more about what types of technologies were available for blind people and realizing that there wasn't enough cool stuff. I realized that I wanted to be the one making those decisions about what access of the future would look like.”

Maps for Blind Commuters

Now, after working in this field for a couple of decades, Miele says coming up with product ideas is the easy part of his job. All he has to do is pay attention to the ways the blind community is underserved. Like having to navigate things like a busy BART train station, at rush hour. His solution? A complete set of accessible BART station maps.

They took ten years to develop and were made in partnership with the LightHouse for the Blind, where Miele did two back-to-back terms as President of their Board of Directors.

The maps are designed to accommodate people with all degrees of visual impairment, even if they can't read braille. They also work in tandem with a specially adapted LiveScribe smart pen that has a camera in the tip and a speaker to read out extra map information.

The pen is simple to use; you just press it against these clusters of tiny dots on the map, and it reads out things like street names, and where to find escalators, and which bus stops where. That makes it possible for blind people to plan their route before they get to the BART station.

Miele explains. “You can say if I'm getting off two doors back from the center, which way do I need to turn to find the stairs, do I turn right or do I turn left? And that information alone is worth the price of entry.”

A Keyboard Literally at Your Fingertips

One of Miele’s earlier projects are these electronic gloves that look like something from a sci-fi movie. Thick red wires connect sensors on the fingers to circuit boards at the wrists. They’re called WearaBraille, and the gloves operate wirelessly, so once they’re paired with a device, a blind person can tap or type on any flat surface to send a text, or open apps, or answer a call.

The gloves work as plug-and-play with his iPhone. It uses accessibility software called Voice Over to read out whatever’s on screen. Miele runs this kind of screen reading software on all his devices.

Voicing Video

“Welcome to YouDescribe, a free accessibility tool.”

His laptop’s voice tells us he’s just logged on the webpage of his most recent invention called YouDescribe. It’s a technology that addresses a core need: the ability for blind people to watch internet videos using audio description.

Description is a second voice-over track that explains any essential visual information, like someone rolling their eyes or throwing a kiss. The tech isn’t that new — it’s been around since 1981, but it was really only done by professional production companies for large movie theaters, or major TV networks. But what do you do when videos start to show up everywhere online? To a blind viewer, most of those videos will sound like little more than music with the occasional door closing or exchange of dialog.

It’s an issue since every day, over a billion users watch hundreds of millions of hours of YouTube videos, but only a fraction of them include audio description.

“For most people who are putting up videos on YouTube, accessibility is really the last thing on their minds,” Miele says.

So once again, Miele’s frustration lead to inspiration. His lab designed a way to let sighted people create description tracks on anything YouTube hosts. Registered users log on YouDescribe.org, find the video they want to share, pause it at the right spots and record themselves explaining how to tie a single windsor, or bake a quiche, or watch Maru the famous internet cat trying to squeeze his puffy body into smaller and smaller boxes. (Or whatever else you might use Youtube for.)

Getting Buy In

And because YouDescribe uses crowdsourcing to generate content, the described videos are available world-wide, for free. If a blind person wants to watch a specific video, they can tweet it with the hashtag #ydRequest and their message goes out to the community of describers.

“I’m really proud of what it is and the possibilities it represents,” Miele says. “It’s a really cool technology.”

The FCC agrees. In 2014 they awarded YouDescribe a Chairman's triple A innovation award. But the project’s grant from the US Department of Education has recently ended, which means that Miele needs to find new partnerships so he can continue making the software better, and give it wider reach.

Making His Way

At the end of his day, Miele grabs his iPhone and his white cane and heads downstairs to walk the least hilly route to the downtown BART station. Rush hour is only beginning, and the platform is packed. Miele stops at the edge of the crowd.

“I’m going to basically do what a sighted person would never do,” he says. Then he turns and presses ahead through the wall of bodies, and a path opens up before him.

When the right train arrives, Miele squeezes on board.

Once we’re sardined into the full train car with other commuters, Miele explains why he just cut through a packed crowd and then boarded the train despite the lineups.

“If you can see you’ll just wait until there’s an opportunity and then slide through the gaps," he says. "But when you can’t see those opportunities, you have to create your own. The only way to do it in any kind of effective way is to either stand around and feel helpless, or to just say excuse me, pardon me, coming through,”

It’s a philosophy that Dr Miele uses to navigate his entire life. And by finding these new ways to create opportunities for himself, he’s also clearing a path to success for other blind people.