Posted on Leave a comment

Creating a future for all: An interview with Saqib Shaikh, the force behind Seeing AI

Saqib Shaikh lost his sight at the age of seven, fell in love with computers as a schoolboy in Britain and grew up to become a top software engineer with an inspirational mission.

Standing at the intersection of artificial intelligence (AI) and inclusive design, he believes we can create intelligent machines to empower millions of people around the world with disabilities to achieve more and live enhanced lives. The knowledge gained from targeting and solving the problems of those with special needs, he says, can only drive technological innovation that benefits everyone across society.

Saqib has had a lifelong relationship with advancing digital technology. At a school for children who are blind or with low vision, he learned self-reliance and developed a burning sense of curiosity. As a 10-year-old, he was given a rudimentary talking PC, and that led him to learn how to program. His intellectual romance with computer science blossomed at university where he doggedly overcame all sorts of day-to-day challenges on campus to graduate top his class with a master’s degree in AI.

A dozen or so years ago, Saqib joined Microsoft and quickly proved his prowess as an engineer by helping to create products, services, and apps that many of us use every day, like Bing and Cortana.

His quest nowadays is to create greater accessibility and inclusion – to level the playing field for everyone. As the driving force behind Microsoft’s Seeing AI project, he is exploring how AI can enable people who are blind or with low vision to achieve more with freedom and confidence.

His team launched the Seeing AI app in 2017, giving those who cannot see a new way to understand the world through the cameras on their smartphones. Since then, it has helped customers with more than 10 million tasks. A user merely points his or her phone, and the app vocally says what it sees. It might be in a room, on a street, in a mall, or an office – customers are using the app in all sorts of situations. With facial-recognition technology, the app can name friends and acquaintances, describe physical appearances of people and even predict their moods.  It can read printed text in books, newspapers, menus, and signs aloud. It can even identify banknotes.

Saqib currently works and lives in London. We caught up with him on a recent visit to Singapore where he told audiences about how technology has helped him realize his potential and how it promises to improve the lives of everyone – and not just those disabilities.

“There are a lot of problems. But for every problem, there is a solution,” he says.

Posted on Leave a comment

With Code Jumper, experts look to jumpstart computer science interest for kids who are blind

None of this would come as a surprise to the students at New College Worcester. As they assembled their programs, some recalled computer science classrooms in which they had been told to do touch typing – which is a method of typing by muscle memory instead of looking at the keyboard – while the other kids in their classes used block coding to write basic computer programs. Others shared the frustrations of trying to learn more complex coding skills like Python or JavaScript without the grounding in more simplified coding systems many other students were getting to use.

With Code Jumper, they said, they were immediately able to experiment and build programs.

“I just felt very independent, and I liked that,” said Daniel, who at 11 years old already knows he wants to pursue a career in computer science. “It kind of made me inspired to do more coding.”

On this day, Daniel was collaborating with another student, Rico. The 12-year-old’s favorite subjects are IT and science, and he said that in his previous school he only got to do touch typing. With Code Jumper, he was able to write an actual program for the first time.

“To just do coding, it was a fun experience,” he said.

As the kids worked on their programs, Jonathan Fogg, head of computing and IT at New College Worcester, walked around the classroom, helping them with things like debugging. Fogg also watched with evident delight as they ran their programs for him. He said Project Torino was unlike anything he’d been able to provide the kids before now.

“There really isn’t an equivalent to this physical way of programming,” Fogg said.

The early access to basic coding skills is important, Fogg said, because many kids who are blind or low vision are drawn to careers in computer science. He thinks that’s partly because many of the skills kids with low vision develop to navigate the world make them good at the kind of computational thinking that’s helpful for a computer science career. And, he said, traditionally it has been a career that is more accessible to people who are blind or low vision, because of tools such as screen readers.

But at a young age, he said, he’s found that many kids are afraid to start playing around with a computer, especially if they know it’s an expensive, fragile machine.

“If they’re not confident, they won’t have a go at the computer because they’re afraid they’ll break it,” he said. “But once they’ve gotten over that barrier, then they’ve been successful. Project Torino reinforces that – they can’t break it and they can do all these really, really cool things.”

Daniel and Rico sitting at a classrom table, helping each other feel the buttons and knobs on brighlty colored plastic pods
From left, Daniel and Rico were part of a group of students at New College Worcester in Worcester, UK, who participated in a beta test of the technology behind Code Jumper. Photo by Jonathan Banks.

Magnets, blocks and lots of trial and error

It’s a couple of days later, and Theo is sitting in a small resource room at the school he attends, Kings College School, also in the UK. Theo, who is blind, has been part of the Code Jumper project for years – in fact, he was one of the first group of students who collaborated with Morrison and others to develop the system.

“I helped them choose what kind of buttons to use,” he said as his hands moved swiftly along the bright plastic pieces, assembling a program as he chatted easily about the computer science skills he’s developed over the years.

The bright plastic pods, oversized dials and thick cords the students are using today is a far cry from the original ideas Morrison and her collaborator, senior researcher Nicolas Villar, had when they first started thinking of a physical programming language.

Their original idea was to create a physical programming language that mimicked block coding, complete with actual blocks and magnets. It didn’t work at all. The kids either lined the blocks up in a row and didn’t do anything else, or they grew frustrated by the magnets falling off the table and getting lost.

Microsoft senior researcher Nicolas Villar in a blue jacket stands against a glass wall, looking into the camera
Microsoft senior researcher Nicolas Villar. Photo courtesy of Microsoft.

So, Morrison, Villar and the other team members began meeting regularly with a small group of kids and getting their ideas. Based on the kids’ feedback and ideas, they switched to bigger plastic shapes that fit easily into kids’ hands, and they created surfaces the kids could rub or squeeze in order to recognize and interact with them.

With the new designs, the kids immediately began exploring ways to put them together and write short programs that made sounds.

Through the work with the young collaborators, Villar said he started to see the technology from the kids’ perspectives. For example, the kids with some vision benefited from bright, contrasting colors. They also found that kids like to work together, guiding each other’s hands, so they built the pods to be about the size of two kids’ hands.

“They were really helping us invent,” Villar said.

Once they had figured out what Morrison called “the kid way of engaging with things,” they set to work making sure that the system also would teach kids the basics of coding, such as how to create a sequence and what steps you need to take in order to debug.

The researchers also created detailed guidance that teachers without a computer science background could use to help kids develop coding skills, since they couldn’t expect all schools to have an IT specialist available to work with students who are blind or low vison.

For Villar, the impact of Code Jumper has extended far beyond just this project.

“It’s opened my eyes to different perspectives, different ways of focusing or seeing the world,” he said.

For Theo, the involvement with Project Torino has been life changing as well. He’s now doing more complex coding, including recently writing a hangman game in Python – something he says he couldn’t have done without the basics he learned using Code Jumper. Equally important to him, he’s made new friends with other kids in school who are also interested in coding.

“It’s been really good socially for us,” said Elin, his mother.

A smiling Theo sits at a table in a small library, both hands touching the plastic knobs
Theo demonstrates a program he created with the technology behind Code Jumper as his mother looks on. Photo by Jonathan Banks.

Big plans

In late 2017, Villar flew to Kentucky to show the leadership at American Printing House for the Blind a demonstration of the project. Craig Meador, APH’s president and a longtime educator, said he was immediately drawn to how intuitive and interesting the system would be to kids.

“If you put this in a classroom, not only is the blind student going to be using this but every student in the classroom is going to want a crack at this,” he said. “From a teacher’s perspective, that’s all you ever really want – something that’s inclusive.”

Skutchan, APH’s technology director, said the fact that it’s immediately accessible for kids with no prior computing experience is important. In the past, he said, kids who are blind or with low vision who wanted to get into coding had to first learn other computing skills, such as how to type and use a screen reader. That made it harder to get started at a young age – and potentially closed the door to a field that could have provided a career for them.

“I think coding is an equalizing field to be in,” he said.

Meador and Skutchan have grand plans for Code Jumper, including developing a curriculum and figuring out distribution and other levels of support. APH plans to release Code Jumper first in the US, UK, Canada and Australia, and then to distribute it throughout the world.

They say it’s the type of system they may have dreamed of – but would never have had the resources to create themselves.

“We’ve never really had anything that’s been able to give students so many different ways they can experiment and learn about coding,” he said, “and it translates so well into actual skills.”

Related:

Allison Linn is a senior writer and editor at Microsoft. Follow her on Twitter.

Top Image: Student Victoria, left, and Jonatha Fogg, right, head of computing and IT at New College Worcester, discuss the program Victoria created during a beta test of the technology behind Code Jumper. Photo by Jonathan Banks.

Posted on Leave a comment

343 Industries collaborates with Limbitless Solutions to offer Halo-themed ‘Bionic Arm’ prosthetics to deserving young heroes

One of our core missions here at the studio is to “inspire heroes and deliver wonder.” Normally, this is most often and understandably applied to the nature of making games. Indeed, creating new worlds to explore, new mysteries to solve, and heroes through which to experience it all is always an amazing and rewarding endeavor. There are, however, times where opportunities to inspire heroes and deliver wonder present themselves outside the scope of game code, cutscene scripts, or sandbox balancing.

Today we’re excited to talk a little bit about one such opportunity, as 343 Industries is proud to announce a collaboration with Limbitless Solutions to provide new Halo-themed, 3D-printed prosthetics to be made available to young heroes who – like the Master Chief himself – also routinely and triumphantly beat the odds.

Limbitless Solutions is a non-profit organization that creates an affordable option for prosthetic-limb devices for children. Using rapid prototyping technologies ‑ 3D printing specifically ‑ prosthetic arms that are fully functional and capable of gripping objects and various gestures are created and fitted to children at a fraction of the cost compared to other prosthetics on the market. The incredible and dedicated team over at Limbitless began in 2014 as a group of University of Central Florida students who wanted to help a family whose son was born without most of his right arm, and since Limbitless has grown to a full-time operation based at UCF in Orlando, Fla. 

The Limbitless “Bionic Arm” prosthetics are all at once functional, cost-effective, and expressive, and perhaps best of all, Limbitless donates their innovative prosthetic arms completely free of charge to recipients and their families. Limbitless is “devoted to building a generation of innovators who use their skills and passion to improve the world around them.” To learn more about their program and also see and hear inspiring stories of affiliated families, please make sure to visit Limbitless-Solutions.org.

THE HALO CONNECTION

With all that firmly in mind, we’re incredibly honored to have the opportunity to collaborate with Limbitless Solutions to offer a unique bit of Halo flavor in our shared goal of inspiring heroes. Beginning in 2019, Limbitless will add two special Halo-themed options to their already-amazing lineup of designs for their 3D-printed Bionic Arms. First up is a fantastic recreation of the Master Chief’s own legendary Mk. VI armor, designed to help turn any eager recipient into an awesomely augmented, galaxy-saving superhero.

In addition, Limbitless will also offer a “multiplayer” variant of the Mjolnir-inspired and personalized prosthetic, allowing a wide range of creative color customization to create a Spartan style as unique and heroic as the recipients themselves. We’re infinitely honored to have been approached Limbitless Solutions to help further their ongoing mission to make heroes out of humanity and look forward to delivering hope and helping hands beginning early next year.

Again, for more information on Limbitless Solutions and their innovative and empowering projects, please make sure to visit Limbitless-Solutions.org on the web, or on Twitter at @Limbitless3D. You can also learn more about their amazing studio and team by checking out their YouTube channel. For all the latest Halo news and updates, keep dialed into Halo Waypoint on the web, as well as @Halo on Twitter.

Posted on Leave a comment

Present more inclusively with new live captions & subtitles in PowerPoint

Live presentations can be thought-provoking, inspirational, and powerful. A great presentation can inspire us to think about something in an entirely different way or bring a group together around a common idea or project. But not everyone experiences presentations in the same way. We may speak a different language from the presenter, or be a native speaker in another language, and some of us are deaf and hard of hearing. So, what if speakers could make their presentations better understood by everyone in the room? Now they can with live captions & subtitles in PowerPoint.

In honor of the United Nations International Day of Persons with Disabilities, we’re announcing this new feature—powered by artificial intelligence (AI)—which provides captions and subtitles for presentations in real-time. Live captions & subtitles in PowerPoint supports the deaf and hard of hearing community by giving them the ability to read what is being spoken in real-time. In addition, captions and subtitles can be displayed in the same language or in a different one, allowing non-native speakers to get a translation of a presentation. At launch, live captions & subtitles will support 12 spoken languages and display on-screen captions or subtitles in one of 60+ languages.

Live captions & subtitles in PowerPoint brings:

  • The power of AI to presenters, so they can convey simple and complex information across subjects and topics.
  • Speech recognition that automatically adapts based on the presented content for more accurate recognition of names and specialized terminology.
  • The ability for presenters to easily customize the size, position, and appearance of subtitles. Customizations may vary by platform.
  • A peace of mind with security and compliance knowing that the feature meets many industry standards for compliance certifications.

The feature joins other accessible features in Office 365, like automatic suggestions for alt-text in Word and PowerPoint, expanded availability of automatic closed captions and searchable transcripts for videos in Microsoft Stream, enhancements to the Office 365 Accessibility Checker, and more.

Here’s what one of our customers had to say:

“We are constantly looking for new ways of ensuring that the Government of Canada sets the highest possible standards as an accessible and inclusive workplace. We welcome such positive advances in technology, like this feature, that allows everyone, and notably those with disabilities, to better communicate ideas. They help break down barriers and lead to greater inclusiveness to the benefit of individuals and society as a whole.”
—Yazmine Laroche, deputy minister responsible for Public Service Accessibility

Live captions & subtitles in PowerPoint will begin rolling out in late January 2019 and will be available for Office 365 subscribers worldwide for PowerPoint on Windows 10, PowerPoint for Mac, and PowerPoint Online.

Posted on Leave a comment

Xbox Adaptive Controller one of TIME’s ‘Best Inventions of 2018’

For people with limited hand and arm mobility, it can be tough to play video games, which are generally controlled using small buttons and joysticks. And while some gamers and small companies have engineered hacks, major gaming companies have largely remained on the -sidelines—until now. Inspired by an internal hackathon and informed by work with groups like the Cerebral Palsy Foundation, Microsoft developed the Xbox Adaptive Controller ($100), an oversize version of its classic rig designed to empower as many gamers as possible. Its main buttons, for example, are roughly 4 in. in -diameter—large enough to be pressed with an elbow or chin, if necessary. It also features ports to accommodate additional aids, like a foot pedal. Xbox designer Chris Kujawski urges others to follow suit: “We hope [our controller] becomes a catalyst for inclusiveness in the gaming industry.” —Samantha Cooney

Posted on Leave a comment

How your company can be a positive force for disability employment

by Jessica Rafuse, Microsoft Accessibility Senior Program Manager

NDEAM is a wrap, what’s next?

As National Disability Employment Awareness Month comes to an end, I reflect on the last month and ask, “Now what?!” Making a positive impact on the unemployment and underemployment rates for people with disabilities is a challenge that must extend beyond NDEAM and must be a collaborative effort. So how do we keep the momentum going?

To help answer the question, we gathered experts and influencers from over 75 different organizations from across the Seattle-area to the second annual Microsoft DisAbility Employment Symposium.

As Bri Sambo from T-Mobile put it, we must “lean on the community we have and collaborate with other companies.” The Symposium generated discussion on common challenges, lessons learned, and innovation in disability employment. We celebrated people with disabilities and committed to our ongoing efforts towards inclusion. Then we said goodbye. As the last guest exited the building, the lights dimmed in the conference room, and I realized, “Now, the real work begins.”

What can your company do to continue the journey to positively impact disability employment? Three things: get Ready, get Set, Hire!

GET READY

1. Nurture a culture of inclusion

A common theme across the Symposium was the importance of nurturing a culture of inclusion. Celebrating people with disabilities within your organization will ignite a sense of disability pride that results in a more inclusive and productive workplace. Here are a few things to consider:

  • Recognize Talent: People with disabilities are an asset to the workplace. From innovation to troubleshooting to project management skills, each individual with a disability brings with them a unique set of skills that they have honed as a result of their disabilities. Remember that 70% of disabilities are non-apparent, so you like have someone in your workplace today who is already adding value to the team.
  • Create opportunity for connection: To build a culture you first need to build a community. Our Disability Employee Resource Group is a long standing group of employees, who have helped to define disability inclusion at Microsoft. If you are looking for a first step in this journey, start here. Just a few passionate employees with common experiences can support your business in the creation of truly inclusive culture.
Disability Employment Symposium Opening Panel – in order left to right: Jessica Rafuse (Microsoft), Keith Clark (Seattle Lighthouse), Megan Mauney (Amazon), Becky Curran (Disability: IN) and Bri Sambo (T-Mobile)

DisAbility Employment Symposium Opening Panel – in order left to right: Jessica Rafuse (Microsoft), Keith Clark (Seattle Lighthouse), Megan Mauney (Amazon), Becky Curran (Disability: IN) and Bri Sambo (T-Mobile)

GET SET

2. Invest in accessibility

Another common theme we saw during the event was companies referencing the importance of embedding accessibility into the fabric of their companies as key to recruiting and retaining talent with disabilities. So how do they do it?

  • Be creative, be frugal, and be resourceful. “Investment” does not always mean “budget” allocation. Utilize resources that are available for free on public platforms. For example, watch Introduction to Disability and Inclusion to learn the basics, and educate your colleagues by using Microsoft Accessibility Training Resources particularly our At a Glance series, which provides bite-sized accessibility trainings.
  • Leverage technology to empower people with disabilities. The role of technology is indisputable in empowering all people, including people with disabilities. From accessible career websites to assistive technologies like screen readers (e.g., Narrator and JAWS) for people who are Blind or Low Vision, accessible technology can make your company more attractive to talent with disabilities. As an individual, you also have a role in making your workplace more accessible with a few simple tricks. Try using the Accessibility Checker before sending an email or turn on Translator within PowerPoint during your next meeting. If you want to know more about accessibility features check out our Accessibility Feature Sway and at the Microsoft Accessibility Site.
  • Accessibility in all levels of your company. Senior leaders are immensely influential as accessibility champions. Ignite your leader’s passion for accessibility by sharing personal stories about how your company’s business has positively impacted the lives of people with disabilities. In support of NDEAM, Microsoft CEO, Satya Nadella, celebrated with six accessibility feature demos: Inclusive Presentations, Seeing AI, Xbox Adaptive Controller, Learning Tools, Soundscape, and Eye Control.

HIRE

3. Hire someone with a disability today!

The timing is now. While nurturing your company culture and integrating accessibility into all that you do, actively seek candidates with disabilities to join in your journey. People with disabilities are just the problem solvers you need to provide feedback on your efforts as you strive for improvement.

  • Transparency in recruiting: candidates appreciate transparency in your company’s inclusion efforts and clear timelines on the accessibility journey. For hiring managers, open communication is key to confront bias and to allow for discussion around your company’s policies for workplace accommodations and benefits.
  • Identifying gaps: partner with your internal talent acquisition team, HR, legal, or other stakeholders to understand the process for working with candidates with disabilities. Ensure that your organization has a process for soliciting and responding to requests for accommodations. Offer trainings to all members of a recruiting ecosystem and co-create resource guides that are specific to your business. Don’t wait for the processes and policies to be perfect, “Hiring talent with disabilities is just common sense, just ask us what we need.”- Nyle DiMarco
  • Resources abound: check out the Disability Equality Index (DEI), which serves as an neutral benchmarking tool to evaluate, measure, and improve your company’s disability inclusion efforts. We are sharing even more of our learnings through our Disability Inclusion Sway and the Microsoft Inclusive Hiring Site.
Jenny Lay-Flurrie and Nyle DiMarco onstage at Disability Employment Symposium

Jenny Lay-Flurrie and Nyle DiMarco onstage at DisAbility Employment Symposium

Thank you again to all the companies, organizations, and disability influencers who participated in our DisAbility Employment Symposium and to those who helped develop these learnings. For those who were not able to join us in-person, Disability:IN can help you to find other businesses in your area that are also committed to disability inclusion. Be curious, be bold, and be collaborative. Hire someone with disabilities and they will guide you along the way.

Posted on Leave a comment

Devs and manufacturers: Enhance accessibility with Microsoft’s Eye Control

October 31, 2018 | By Microsoft blog editor

Eye Control is an exciting technology in Windows 10 that allows customers to use their eyes to control an on-screen mouse, keyboard and text-to-speech experience. This is a technology that empowers people with limited mobility such as people living with ALS.

As part of the National Disability Employer Awareness Month (NDEAM), Microsoft’s CEO Satya Nadella has shared a demo video showing the capabilities enabled by the APIs (Application Programming Interface) associated with the Eye Control. Thanks to the additional Eye Drive Library, which emulates a joystick via eye tracking, the video shows prototypes of a remote-controlled car and an electric wheelchair, which can then both be controlled by eye tracking. Those are research prototypes designed to illustrate how this technology can be used to empower people, and appeal to developers, innovators and makers to download these open source libraries and see what you can do.

By highlighting the Eye Control features in Windows and its APIs, we encourage developers and eye tracking device manufacturers to harness the power of eye control for their own applications and products to enhance accessibility and utilize a new mode of interacting with a Windows-based computer. We are keen to see where the development of this technology is leading us in the future.

To learn more about Eye Control APIs for Windows and Eye Drive Libraries referenced in the video visit: Eye Control APIs and Eye Drive Library. To learn more about the research team behind this technology visit Microsoft Enable.

If you have any questions about the demo or the technology, please reach out to the Disability Answer Desk.

[embedded content]

Posted on Leave a comment

New accessibility improvements in Skype 8.0

Skype is designed to keep you connected to the people who matter the most and help you do the work you need to do. We take your feedback very seriously and are grateful to our active accessibility community who point us to areas of improvement to live up to our vision. Building on the work we announced earlier this year, we’ve been listening to your feedback and have continued to work to make the latest version of Skype (version 8.0) even more accessible.

Based off your suggestion to make the transition to the new version easier, we created the Accessibility support for Skype page with details on how to easily use a screen reader to move from Skype version 6 or 7 to Skype version 8.0. The page also demonstrates keyboard shortcuts for Skype version 8.0, as well as how to complete key scenarios in Skype with a screen reader, including chat, adding expressions, and navigating a profile.

In addition, since our last update on accessibility, we made a few more improvements to the latest version of Skype, including:

  • Focus assist, which announces when you receive an incoming call for keyboard only and screen reader users.
  • Screen reader announcements from users in the active call list.
  • Navigational focus announcements within the contact list and settings to improve flow.
  • Announcing the name, content, and actions of a pop up dialog.
  • New message announcements when users are in another chat.
  • Smoother navigation that takes a more natural left-to-right and top-to-bottom path.
  • Displaying additional information when sending and receiving messages. For example, Skype now announces when messages are sent and when messages you attempt to send have failed.

We are committed to continually improving our software and services to meet the needs of all our customers. Please share your thoughts and ideas via the Microsoft Accessibility UserVoice or contact the Disability Answer Desk for real-time support via phone, chat, or ASL videophone—your feedback is always appreciated. If you are an early adopter—and comfortable with early preview releases—please consider joining the Skype Insider community.

Posted on Leave a comment

Announcing the first AI for Accessibility grantee: Zyrobotics

By Wendy Chisholm, Principal Accessibility Architect

In May, Microsoft CEO Satya Nadella announced AI for Accessibility to put AI tools in the hands of developers, universities, NGOs and inventors to accelerate the development of accessible and intelligent AI solutions to benefit people with disabilities around the world.

Today, we are announcing the first AI for Accessibility grantee, Zyrobotics. They are developing unique solutions for accessible science, technology, engineering and math (STEM) education, like ReadAble Storiez, that are adaptive to the diverse needs of students. We are thrilled to have them as part of our program and we can’t wait to see the impact they will have!

Zyrobotics CEO Dr. Johnetta MacCalla and CTO Dr. Ayanna Howard alongside Microsoft CTO Satya Nadella

ReadAble Storiez is a reading fluency program for students with diverse learning needs, which also helps fill in the gaps for students from low-income homes who may not have access to speech-language or occupational therapists. By creating custom speech models with Microsoft Cognitive Services and Azure Machine Learning, they aim to identify when a student needs feedback, much like an occupational therapist would recognize and provide.

Zyrobotics is a team of educators and developers who have a passion for creating technology that will enhance the quality of life for children. Last week, Satya had the opportunity to meet with Zyrobotics CEO Dr. Johnetta MacCalla and CTO Dr. Ayanna Howard in Atlanta, Georgia, and learn more about their work and passion for helping all students become better readers.

As Dr. Howard shared with us, “In our experience, because every child is different, you have to let them look at technology in different ways, more than just visual and auditory. Our focus is to incorporate interaction and personalization. The usage of AI/machine learning algorithms ensures that our apps are accessible and can be adapted to the personal needs of each child. We are excited about how this grant and partnership with Microsoft will help us address significant gaps for engaging each child based on their unique strengths and abilities.”

Look out for more grant-winning projects to be announced on the Accessibility Blog, the AI for Accessibility website and our social channels in the coming months. For those getting started, here are three of the top questions we have received from applicants:

When can I apply?
Now! We are accepting submissions on a rolling basis over the next five years. Grant proposals can be submitted via the online application form. Applicants will receive a follow up email within 30-60 days.

What is Microsoft looking for in applications?
We are looking for revolutionary ideas that use AI and align with any of our three areas of focus:

  • Employment. How can we positively impact the employment rate for people with disabilities through more intelligent technology?
  • Daily Life. How can we increase access to technology for people with disabilities while also decreasing the cost of such technology?
  • Communication & Connection. How can AI help improve the speed, accuracy, and convenience of communication for people with disabilities?

We are looking for individuals or teams who are not only passionate about making the world more inclusive, but also firmly rooted in the communities they intend to benefit. We want to see ideas that are developed by or with people with disabilities, not just for us.

What can I receive through the program?
First, check that your idea meets the following criteria:

  • Relevance: AI and/or machine learning are core to the project success;
  • Impact: Clearly elevates either employment, daily life, or communication & connection for someone with a disability;
  • Feasible: Is deliverable in a one-year timeframe—this could be phase 1 of the project but there should be tangible deliverables completed each quarter that are wrapped up in one year;
  • Capable: Your team should have the knowledge and skill to deliver the idea OR you partner with someone who can fill in the gaps;
  • Sustainable: The project has room to grow beyond the first year of funding;
  • Showcase-able: We want to talk about whatever you deliver!

The AI for Accessibility program provides monetary grants as well as technology and expertise depending on the potential impact and needs of each project. This can include:

  • Access to Microsoft Azure AI resources and Developer Support. To estimate the Azure computing resources you need, use the Azure calculator;
  • The costs of crowd-sourcing and cleaning data needed to get your project up and running;
  • Guidance from a Microsoft team of AI, project management, and accessibility experts who can help guide you to scale to a wide audience and deliver the impact you envision.

If you have additional questions about the awards program or the application process, please visit the AI for Accessibility FAQ.

Remember: anyone can apply! From a tinkerer in a basement to an established corporation, from students to professionals. If you are worried you don’t have what it takes, find someone who does and partner with them on an application. So, no excuses. 😊

Over the past eighteen months, Microsoft has fostered a growing $115 million AI for Good initiative that puts AI in the hands of inspired individuals developing real-world outcomes that tackle some of society’s most critical challenges. Comprised of AI for Accessibility, AI for Earth and AI for Humanitarian Action, we are hopeful the world will see what a compelling force for good AI can be.

We cannot wait to see what you create by or with people with disabilities to empower everyone to achieve more. 

Posted on Leave a comment

Water Music: Blind and low vision paddlers make waves using Soundscape

By: Amos Miller, Microsoft AI & Research

Imagine you’re paddling on the lake, you feel the warm sun on your face, listen quietly to the calm ripples of the water. You can hear a sound beacon, in 3D, on the water, just in front of you and over to the right; you know exactly where it is, you steer the kayak towards it, paddle like crazy and catch it, collecting 25 points.

When your ability to perceive the space around you is enhanced in a natural, intuitive way, the urge to get out and enjoy that heightened sense of independence is irresistible. We experienced that feeling a few weeks ago on a beautiful sunny day on Lake Sammamish at the Microsoft Soundscape kayaking PaddlePalooza. What was that all about? Let me share the story.two kayaks on lake Sammamish

In my June post, I noted the continued excitement about the Soundscape beacon from people who are blind or have low vision. Many of you have asked to be able to place the Soundscape audio beacon anywhere. In response, we recently introduced Soundscape markers. With Soundscape you can now quickly and easily mark-up anything in your surroundings that you care about. That could be, for example, your front door or the post box at the top of your street, or you can mark-up key landmarks in your area to help with learning a route. You might even mark the entrance to the park or the picnic spot on the lawn.

With your markers in place you can hear them through Soundscape or set a beacon to them, so you can always know where they are. This is a wonderful way for you to personalize the areas you walk in and build up confidence to go beyond what you are familiar with, always knowing that you have your markers for orientation. Do give it a try and tell us what you think. It’s a lot of fun.

We then thought, well, if you can place a marker anywhere what if we marked up a lake and used the markers for orientation on the water while kayaking? Hmmm. Ok, so this sounds new, and we figured it would be something that takes empowerment to a completely new level!

As part of the Microsoft One Week hackathon, we marked up seven virtual beacons on the lake. In partnership with Outdoors For All, we invited a group of people with blindness or low vision to join us for a kayaking scavenger hunt.

[embedded content]

Click here for the audio description of the Soundscape Kayaking Scavenger Hunt video.

Wow! What an extraordinary day. When you wake up in the morning and by the end of the day realize you have done something you never imagined would be possible.

Now for the cool statistics! Of the 13 competing teams, 3 had only sighted paddlers, and 10 had blind or low vision paddlers. 11 of the 13 teams successfully found all seven beacons (covering an area roughly within a circle of 1000 meters in diameter) and scored the maximum points. The winning team completed the course in just 16 minutes!! The best sighted team came in sixth place at 25 minutes!

people with their kayaks on the shoreline of Lake Sammamish

“We were having a blast, the sun was shining, the water was calm and warm, I had an attentive partner, and I was independently captaining a kayak and successfully found all seven beacons, earning the maximum 140 points in 25 minutes…the exhilaration of being able to compete, to experience the freedom of paddling on the water, and truly being the navigator of our team is something I’ll remember always,” noted Cindy Van Winkle, a participant from the Seattle Lighthouse. You can read more about her experience on her blog.

The kayaking experience motivated us to reflect, once again, on what we mean by the sense of independence and about presence and empowerment. Have people who are blind kayaked before? Sure, they do all the time. However, are they really the driver? On this occasion they indeed were the driver in every sense of the term.

Firstly, in the teams with a paddler who is blind, only the blind participant knew where the beacons were located; the sighted companions had no indication where they were, and therefore they were not able to help. That meant that the paddler who is blind didn’t even need to ask for that independence, it was built into the game. Secondly, the participants didn’t have an agent whispering in their ear, go left a bit, straight, right, nor did they receive any form of instructions they had to listen to and process. Instead, Soundscape, using 3D audio, enabled the participant to effortlessly hear exactly where the beacons were, leaving it for them to decide where to steer. This transfer of control not only creates a true sense of independence, but the participants in fact performed much better. I talk about this phenomenon in my TedX talk where I describe the feeling I had flying a glider.

A few months ago, at an event in Montana I had the pleasure of meeting Sharon. As we made our way out to learn how to use the Soundscape beacon, I noticed that Sharon stayed back. When I checked with her I understood she chose not to join because she is hard of hearing. After we established that she was still able to hear the high pitch of the Soundscape beacon, she gave it a try, and, with her husband walking along with her, off they went and within a few minutes made it to her destination. When I caught up with them I noticed she was overcome with emotion, this had been the first time in many years she truly felt that sense of independence and that she was in the driver’s seat. For her, this is what mattered at that moment, that sense of joy and empowerment; and, knowing that her life would now be different.

There is no greater gift that technology can provide than a true sense of independence.

I must say that the impact the kayak hack has really surprised and motivated us to think about how we can bring Soundscape to more people and more organizations to really benefit from the value it provides and the empowering changes it has started to enable for people. This is something that we are looking into, and if you, or an organization you know, run programs or activities in which you believe Soundscape can help then do get in touch with us at soundscapefeed@microsoft.com.

Before wrapping up let me highlight some of the other recent updates we made to Soundscape:

  • We’ve been working to light up the experience while you’re travelling, whether in a bus, a car, or a train. To minimize interruptions and avoid getting in the way of whatever else you may be doing, Soundscape now identifies when you are in a moving vehicle and adjusts callouts to include only major landmarks, and update you only when you change roads.
  • Battery savings! You no longer need to close Soundscape to save battery; you can now just select the “Sleep” button on the home screen to stop Soundscape from using GPS and your data plan.
  • For our low vision users, you no longer need to hold the phone flat when pressing buttons; instead, you can now bring the phone right up close and press buttons, and everything will continue working correctly.

screenshot of the Soundscape homescreen

We can’t wait to hear what you make of them, please do continue to write to us on soundscapefeed@microsoft.com and of course, check our FAQ’s on our website for more detailed information and guidance.

In closing we welcome our friends from down under to the Soundscape community! Soundscape was launched in Australia on September 12, 2018, with a great partnership with Vision Australia and the extended community of people who are blind. Based on our learnings from the launch in Australia we continue to look at how we can bring Soundscape to other countries and aim to support the local language where possible. We will be sure to provide further updates as soon as we can.

We continue to be humbled by the response to Soundscape, we recognize that this is just the start and there is more work to be done; but please keep the feedback coming, on anything from the app, to the content we produce, to the way we can engage with you. Let’s continue to empower everyone, everywhere with the benefits of technology.

Amos and the Soundscape team.

Microsoft Soundscape is available for free on iOS and iPhone in the US, UK and Australia.