Putting people at the center of AI design
Mira Lane, the lead of Microsoft’s Ethics & Society team, is charged with ensuring that the principles articulated at the highest levels of the company to guide the responsible use of AI find their way into the heads of researchers conducting user testing and the hands of engineers writing code. It starts with asking the right questions, she said.
Her team of philosophers, engineers, security experts, designers and trainers works closely with product teams to consider what data or models should be used, who might be directly or indirectly affected by a new technology, what kinds of people should be interviewed to identify unintentional harms and how those insights can be folded into product design.
“The thing that we’re trying to do is help people design technology in a really intentional way, so you really understand what the effects of the tech are and can look around the corner to how it might be used or misused,” Lane said.
For teams that incorporate AI into productivity tools, one of the most important principles is to keep people at the center of the process.
“We bring a lot of focus to making sure the experiences we’re delivering are actually valuable,” said Penny Collisson, principal design research manager for Office. “We have lots of conversations with customers where we never even mention AI. We’re talking about understanding the expressed or latent needs or pain points that people have and then we go back and try to think about how AI could fit in.”
Microsoft has developed 18 best practices that researchers and product designers use to guide their work. But a lot of that work involves listening to people with different levels of tech adoption, socioeconomic backgrounds, geography, physical abilities or attitudes about AI and privacy.
If you talk to people with learning disabilities, for instance, some have a fear of starting with a blank page. That insight helped guide improved dictation offerings in Word for the web, which makes it easier to create content with one’s voice and use speech-to-text to get thoughts down on paper.
Creating good user experiences with AI is more complicated than asking people for feedback on whether they prefer one type of control over another, or which interface is easier to navigate, said Jon Friedman, Microsoft corporate vice president for design and research.
“The kicker and power of AI is that everyone’s experience is unique. Before, we were designing for the mean because solutions were closer to one size fits all. And now we are designing each thing to be a special size to fit each individual,” Friedman said.
“So making sure we’re talking to a much broader set of people and hearing everyone’s voice is really important to give people what they truly need,” he said.
In one example, Microsoft designers and engineers who were interested in building a better screen reader for people who are blind or with low vision built a relationship with the Washington State School for the Blind and began interviewing and observing how those students consume information and approach tasks for the day. That work led to Play My Emails in Outlook mobile, which turned out to also be useful for anyone who wants a jump on their day but can’t safely look at a screen while commuting or cooking breakfast for kids.
Through interviews and equipment that simulated the experience of having macular degeneration, the design team began to understand the massive cognitive load that’s required to listen for pertinent information among a sea of extraneous details like dates and time stamps and even punctuation marks that screen readers include as they scan from left to right.
“It was like listening for a needle in a haystack, and the fatigue level was really high,” Friedman said.
So the team used AI to offer the most important information upfront and in a much more conversational way. Having Cortana, Microsoft 365’s personal productivity assistant, tell you that someone sent you an email in the past hour about scheduling a meeting this afternoon is more useful than knowing the precise time stamp, Friedman said.
Play My Emails also provides summary information like how many unread emails are in your inbox and how long it would take to listen to them. That helps people decide if they have enough commute time or brain space while they’re rushing to get out of the house to focus on the task.
“We started on this path because we thought inclusive design was an important philosophy that we needed to start living and breathing in product,” Friedman said. “But the team quickly realized that there’s a lot of instances where people are situationally blind or looking at screens when it’s not safe, and that’s when they realized this is something that could be useful for people in a lot of different contexts.”