One day, young Heather came home from junior high school with an idea. She’d often toss out ideas to her father for what she could be when she grew up. Back then, he was endearingly tough to please, she remembered. Not in an unloving way, but in a way that emboldened Heather to challenge herself.
Her dad was reading in his tattered, gray arm chair. She touched his arm gently and signed, “Dad! What if I become a sign-language interpreter?”
He peered at her over his book, set it down, and signed, “No.”
“What? I thought you’d be excited. I’ve been interpreting for you and mom my whole life,” she pressed. “Why not?”
“Too safe for you,” said Royce. “Believe in yourself. Do what makes you uncomfortable.”
While Heather walked out of the room thinking that it was strange that he wouldn’t want her to choose interpreting as a career, she knew that he was right. So even though she interpreted for then-president Barack Obama during a 2015 national monument dedication in Chicago, she respected his commitment to working hard for the sake of the family. So, she would keep looking for something that challenged her.
After her sophomore year of high school, Heather went to a summer engineering program at Chicago State University. There, she got to dig in to the hardware of all the gadgets that she loved. As she soldered the circuit board for a phone, she thought about how so many people who have hearing loss, at that time, couldn’t use the phone easily—the feedback on hearing aid devices made people on the other end of the line sound like they were in a construction zone. “Wow, I could make accessible technology and really change people’s lives.”
“Today, I never take it for granted that I can send my dad a text if he doesn’t see me standing at the door. The world of pagers and mobile phones really changed our world,” Heather said.
It turns out that there was a name for being an inventor of technology for people who are Deaf or hard-of-hearing. She came home that summer and told her dad, “Engineer. I’m going to be an engineer.” At that point, Heather had never heard of any female, black engineers. Surely that qualified as uncomfortable.
Royce said nothing, his kind eyes narrowing in on hers. He smirked, shrugged, and then walked down the narrow hallway to rest up for the evening’s shift at the post office.
Saqib Shaikh lost his sight at the age of seven, fell in love with computers as a schoolboy in Britain and grew up to become a top software engineer with an inspirational mission.
Standing at the intersection of artificial intelligence (AI) and inclusive design, he believes we can create intelligent machines to empower millions of people around the world with disabilities to achieve more and live enhanced lives. The knowledge gained from targeting and solving the problems of those with special needs, he says, can only drive technological innovation that benefits everyone across society.
Saqib has had a lifelong relationship with advancing digital technology. At a school for children who are blind or with low vision, he learned self-reliance and developed a burning sense of curiosity. As a 10-year-old, he was given a rudimentary talking PC, and that led him to learn how to program. His intellectual romance with computer science blossomed at university where he doggedly overcame all sorts of day-to-day challenges on campus to graduate top his class with a master’s degree in AI.
A dozen or so years ago, Saqib joined Microsoft and quickly proved his prowess as an engineer by helping to create products, services, and apps that many of us use every day, like Bing and Cortana.
His quest nowadays is to create greater accessibility and inclusion – to level the playing field for everyone. As the driving force behind Microsoft’s Seeing AI project, he is exploring how AI can enable people who are blind or with low vision to achieve more with freedom and confidence.
His team launched the Seeing AI app in 2017, giving those who cannot see a new way to understand the world through the cameras on their smartphones. Since then, it has helped customers with more than 10 million tasks. A user merely points his or her phone, and the app vocally says what it sees. It might be in a room, on a street, in a mall, or an office – customers are using the app in all sorts of situations. With facial-recognition technology, the app can name friends and acquaintances, describe physical appearances of people and even predict their moods. It can read printed text in books, newspapers, menus, and signs aloud. It can even identify banknotes.
Saqib currently works and lives in London. We caught up with him on a recent visit to Singapore where he told audiences about how technology has helped him realize his potential and how it promises to improve the lives of everyone – and not just those disabilities.
“There are a lot of problems. But for every problem, there is a solution,” he says.
With Code Jumper, they said, they were immediately able to experiment and build programs.
“I just felt very independent, and I liked that,” said Daniel, who at 11 years old already knows he wants to pursue a career in computer science. “It kind of made me inspired to do more coding.”
On this day, Daniel was collaborating with another student, Rico. The 12-year-old’s favorite subjects are IT and science, and he said that in his previous school he only got to do touch typing. With Code Jumper, he was able to write an actual program for the first time.
“To just do coding, it was a fun experience,” he said.
As the kids worked on their programs, Jonathan Fogg, head of computing and IT at New College Worcester, walked around the classroom, helping them with things like debugging. Fogg also watched with evident delight as they ran their programs for him. He said Project Torino was unlike anything he’d been able to provide the kids before now.
“There really isn’t an equivalent to this physical way of programming,” Fogg said.
The early access to basic coding skills is important, Fogg said, because many kids who are blind or low vision are drawn to careers in computer science. He thinks that’s partly because many of the skills kids with low vision develop to navigate the world make them good at the kind of computational thinking that’s helpful for a computer science career. And, he said, traditionally it has been a career that is more accessible to people who are blind or low vision, because of tools such as screen readers.
But at a young age, he said, he’s found that many kids are afraid to start playing around with a computer, especially if they know it’s an expensive, fragile machine.
“If they’re not confident, they won’t have a go at the computer because they’re afraid they’ll break it,” he said. “But once they’ve gotten over that barrier, then they’ve been successful. Project Torino reinforces that – they can’t break it and they can do all these really, really cool things.”
Magnets, blocks and lots of trial and error
It’s a couple of days later, and Theo is sitting in a small resource room at the school he attends, Kings College School, also in the UK. Theo, who is blind, has been part of the Code Jumper project for years – in fact, he was one of the first group of students who collaborated with Morrison and others to develop the system.
“I helped them choose what kind of buttons to use,” he said as his hands moved swiftly along the bright plastic pieces, assembling a program as he chatted easily about the computer science skills he’s developed over the years.
The bright plastic pods, oversized dials and thick cords the students are using today is a far cry from the original ideas Morrison and her collaborator, senior researcher Nicolas Villar, had when they first started thinking of a physical programming language.
Their original idea was to create a physical programming language that mimicked block coding, complete with actual blocks and magnets. It didn’t work at all. The kids either lined the blocks up in a row and didn’t do anything else, or they grew frustrated by the magnets falling off the table and getting lost.
So, Morrison, Villar and the other team members began meeting regularly with a small group of kids and getting their ideas. Based on the kids’ feedback and ideas, they switched to bigger plastic shapes that fit easily into kids’ hands, and they created surfaces the kids could rub or squeeze in order to recognize and interact with them.
With the new designs, the kids immediately began exploring ways to put them together and write short programs that made sounds.
Through the work with the young collaborators, Villar said he started to see the technology from the kids’ perspectives. For example, the kids with some vision benefited from bright, contrasting colors. They also found that kids like to work together, guiding each other’s hands, so they built the pods to be about the size of two kids’ hands.
“They were really helping us invent,” Villar said.
Once they had figured out what Morrison called “the kid way of engaging with things,” they set to work making sure that the system also would teach kids the basics of coding, such as how to create a sequence and what steps you need to take in order to debug.
The researchers also created detailed guidance that teachers without a computer science background could use to help kids develop coding skills, since they couldn’t expect all schools to have an IT specialist available to work with students who are blind or low vison.
For Villar, the impact of Code Jumper has extended far beyond just this project.
“It’s opened my eyes to different perspectives, different ways of focusing or seeing the world,” he said.
For Theo, the involvement with Project Torino has been life changing as well. He’s now doing more complex coding, including recently writing a hangman game in Python – something he says he couldn’t have done without the basics he learned using Code Jumper. Equally important to him, he’s made new friends with other kids in school who are also interested in coding.
“It’s been really good socially for us,” said Elin, his mother.
In late 2017, Villar flew to Kentucky to show the leadership at American Printing House for the Blind a demonstration of the project. Craig Meador, APH’s president and a longtime educator, said he was immediately drawn to how intuitive and interesting the system would be to kids.
“If you put this in a classroom, not only is the blind student going to be using this but every student in the classroom is going to want a crack at this,” he said. “From a teacher’s perspective, that’s all you ever really want – something that’s inclusive.”
Skutchan, APH’s technology director, said the fact that it’s immediately accessible for kids with no prior computing experience is important. In the past, he said, kids who are blind or with low vision who wanted to get into coding had to first learn other computing skills, such as how to type and use a screen reader. That made it harder to get started at a young age – and potentially closed the door to a field that could have provided a career for them.
“I think coding is an equalizing field to be in,” he said.
Meador and Skutchan have grand plans for Code Jumper, including developing a curriculum and figuring out distribution and other levels of support. APH plans to release Code Jumper first in the US, UK, Canada and Australia, and then to distribute it throughout the world.
They say it’s the type of system they may have dreamed of – but would never have had the resources to create themselves.
“We’ve never really had anything that’s been able to give students so many different ways they can experiment and learn about coding,” he said, “and it translates so well into actual skills.”
Top Image: Student Victoria, left, and Jonatha Fogg, right, head of computing and IT at New College Worcester, discuss the program Victoria created during a beta test of the technology behind Code Jumper. Photo by Jonathan Banks.
One of our core missions here at the studio is to “inspire heroes and deliver wonder.” Normally, this is most often and understandably applied to the nature of making games. Indeed, creating new worlds to explore, new mysteries to solve, and heroes through which to experience it all is always an amazing and rewarding endeavor. There are, however, times where opportunities to inspire heroes and deliver wonder present themselves outside the scope of game code, cutscene scripts, or sandbox balancing.
Today we’re excited to talk a little bit about one such opportunity, as 343 Industries is proud to announce a collaboration with Limbitless Solutions to provide new Halo-themed, 3D-printed prosthetics to be made available to young heroes who – like the Master Chief himself – also routinely and triumphantly beat the odds.
Limbitless Solutions is a non-profit organization that creates an affordable option for prosthetic-limb devices for children. Using rapid prototyping technologies ‑ 3D printing specifically ‑ prosthetic arms that are fully functional and capable of gripping objects and various gestures are created and fitted to children at a fraction of the cost compared to other prosthetics on the market. The incredible and dedicated team over at Limbitless began in 2014 as a group of University of Central Florida students who wanted to help a family whose son was born without most of his right arm, and since Limbitless has grown to a full-time operation based at UCF in Orlando, Fla.
The Limbitless “Bionic Arm” prosthetics are all at once functional, cost-effective, and expressive, and perhaps best of all, Limbitless donates their innovative prosthetic arms completely free of charge to recipients and their families. Limbitless is “devoted to building a generation of innovators who use their skills and passion to improve the world around them.” To learn more about their program and also see and hear inspiring stories of affiliated families, please make sure to visit Limbitless-Solutions.org.
THE HALO CONNECTION
With all that firmly in mind, we’re incredibly honored to have the opportunity to collaborate with Limbitless Solutions to offer a unique bit of Halo flavor in our shared goal of inspiring heroes. Beginning in 2019, Limbitless will add two special Halo-themed options to their already-amazing lineup of designs for their 3D-printed Bionic Arms. First up is a fantastic recreation of the Master Chief’s own legendary Mk. VI armor, designed to help turn any eager recipient into an awesomely augmented, galaxy-saving superhero.
In addition, Limbitless will also offer a “multiplayer” variant of the Mjolnir-inspired and personalized prosthetic, allowing a wide range of creative color customization to create a Spartan style as unique and heroic as the recipients themselves. We’re infinitely honored to have been approached Limbitless Solutions to help further their ongoing mission to make heroes out of humanity and look forward to delivering hope and helping hands beginning early next year.
Live presentations can be thought-provoking, inspirational, and powerful. A great presentation can inspire us to think about something in an entirely different way or bring a group together around a common idea or project. But not everyone experiences presentations in the same way. We may speak a different language from the presenter, or be a native speaker in another language, and some of us are deaf and hard of hearing. So, what if speakers could make their presentations better understood by everyone in the room? Now they can with live captions & subtitles in PowerPoint.
In honor of the United Nations International Day of Persons with Disabilities, we’re announcing this new feature—powered by artificial intelligence (AI)—which provides captions and subtitles for presentations in real-time. Live captions & subtitles in PowerPoint supports the deaf and hard of hearing community by giving them the ability to read what is being spoken in real-time. In addition, captions and subtitles can be displayed in the same language or in a different one, allowing non-native speakers to get a translation of a presentation. At launch, live captions & subtitles will support 12 spoken languages and display on-screen captions or subtitles in one of 60+ languages.
Live captions & subtitles in PowerPoint brings:
The power of AI to presenters, so they can convey simple and complex information across subjects and topics.
Speech recognition that automatically adapts based on the presented content for more accurate recognition of names and specialized terminology.
The ability for presenters to easily customize the size, position, and appearance of subtitles. Customizations may vary by platform.
A peace of mind with security and compliance knowing that the feature meets many industry standards for compliance certifications.
“We are constantly looking for new ways of ensuring that the Government of Canada sets the highest possible standards as an accessible and inclusive workplace. We welcome such positive advances in technology, like this feature, that allows everyone, and notably those with disabilities, to better communicate ideas. They help break down barriers and lead to greater inclusiveness to the benefit of individuals and society as a whole.” —Yazmine Laroche, deputy minister responsible for Public Service Accessibility
Live captions & subtitles in PowerPoint will begin rolling out in late January 2019 and will be available for Office 365 subscribers worldwide for PowerPoint on Windows 10, PowerPoint for Mac, and PowerPoint Online.
For people with limited hand and arm mobility, it can be tough to play video games, which are generally controlled using small buttons and joysticks. And while some gamers and small companies have engineered hacks, major gaming companies have largely remained on the -sidelines—until now. Inspired by an internal hackathon and informed by work with groups like the Cerebral Palsy Foundation, Microsoft developed the Xbox Adaptive Controller ($100), an oversize version of its classic rig designed to empower as many gamers as possible. Its main buttons, for example, are roughly 4 in. in -diameter—large enough to be pressed with an elbow or chin, if necessary. It also features ports to accommodate additional aids, like a foot pedal. Xbox designer Chris Kujawski urges others to follow suit: “We hope [our controller] becomes a catalyst for inclusiveness in the gaming industry.” —Samantha Cooney
by Jessica Rafuse, Microsoft Accessibility Senior Program Manager
NDEAM is a wrap, what’s next?
As National Disability Employment Awareness Month comes to an end, I reflect on the last month and ask, “Now what?!” Making a positive impact on the unemployment and underemployment rates for people with disabilities is a challenge that must extend beyond NDEAM and must be a collaborative effort. So how do we keep the momentum going?
As Bri Sambo from T-Mobile put it, we must “lean on the community we have and collaborate with other companies.” The Symposium generated discussion on common challenges, lessons learned, and innovation in disability employment. We celebrated people with disabilities and committed to our ongoing efforts towards inclusion. Then we said goodbye. As the last guest exited the building, the lights dimmed in the conference room, and I realized, “Now, the real work begins.”
What can your company do to continue the journey to positively impact disability employment? Three things: get Ready, get Set, Hire!
1. Nurture a culture of inclusion
A common theme across the Symposium was the importance of nurturing a culture of inclusion. Celebrating people with disabilities within your organization will ignite a sense of disability pride that results in a more inclusive and productive workplace. Here are a few things to consider:
Recognize Talent: People with disabilities are an asset to the workplace. From innovation to troubleshooting to project management skills, each individual with a disability brings with them a unique set of skills that they have honed as a result of their disabilities. Remember that 70% of disabilities are non-apparent, so you like have someone in your workplace today who is already adding value to the team.
Create opportunity for connection: To build a culture you first need to build a community. Our Disability Employee Resource Group is a long standing group of employees, who have helped to define disability inclusion at Microsoft. If you are looking for a first step in this journey, start here. Just a few passionate employees with common experiences can support your business in the creation of truly inclusive culture.
DisAbility Employment Symposium Opening Panel – in order left to right: Jessica Rafuse (Microsoft), Keith Clark (Seattle Lighthouse), Megan Mauney (Amazon), Becky Curran (Disability: IN) and Bri Sambo (T-Mobile)
2. Invest in accessibility
Another common theme we saw during the event was companies referencing the importance of embedding accessibility into the fabric of their companies as key to recruiting and retaining talent with disabilities. So how do they do it?
Leverage technology to empower people with disabilities. The role of technology is indisputable in empowering all people, including people with disabilities. From accessible career websites to assistive technologies like screen readers (e.g., Narrator and JAWS) for people who are Blind or Low Vision, accessible technology can make your company more attractive to talent with disabilities. As an individual, you also have a role in making your workplace more accessible with a few simple tricks. Try using the Accessibility Checker before sending an email or turn on Translator within PowerPoint during your next meeting. If you want to know more about accessibility features check out our Accessibility Feature Sway and at the Microsoft Accessibility Site.
Accessibility in all levels of your company. Senior leaders are immensely influential as accessibility champions. Ignite your leader’s passion for accessibility by sharing personal stories about how your company’s business has positively impacted the lives of people with disabilities. In support of NDEAM, Microsoft CEO, Satya Nadella, celebrated with six accessibility feature demos: Inclusive Presentations, Seeing AI, Xbox Adaptive Controller, Learning Tools, Soundscape, and Eye Control.
3. Hire someone with a disability today!
The timing is now. While nurturing your company culture and integrating accessibility into all that you do, actively seek candidates with disabilities to join in your journey. People with disabilities are just the problem solvers you need to provide feedback on your efforts as you strive for improvement.
Transparency in recruiting: candidates appreciate transparency in your company’s inclusion efforts and clear timelines on the accessibility journey. For hiring managers, open communication is key to confront bias and to allow for discussion around your company’s policies for workplace accommodations and benefits.
Identifying gaps: partner with your internal talent acquisition team, HR, legal, or other stakeholders to understand the process for working with candidates with disabilities. Ensure that your organization has a process for soliciting and responding to requests for accommodations. Offer trainings to all members of a recruiting ecosystem and co-create resource guides that are specific to your business. Don’t wait for the processes and policies to be perfect, “Hiring talent with disabilities is just common sense, just ask us what we need.”- Nyle DiMarco
Jenny Lay-Flurrie and Nyle DiMarco onstage at DisAbility Employment Symposium
Thank you again to all the companies, organizations, and disability influencers who participated in our DisAbility Employment Symposium and to those who helped develop these learnings. For those who were not able to join us in-person, Disability:IN can help you to find other businesses in your area that are also committed to disability inclusion. Be curious, be bold, and be collaborative. Hire someone with disabilities and they will guide you along the way.
Eye Control is an exciting technology in Windows 10 that allows customers to use their eyes to control an on-screen mouse, keyboard and text-to-speech experience. This is a technology that empowers people with limited mobility such as people living with ALS.
As part of the National Disability Employer Awareness Month (NDEAM), Microsoft’s CEO Satya Nadella has shared a demo video showing the capabilities enabled by the APIs (Application Programming Interface) associated with the Eye Control. Thanks to the additional Eye Drive Library, which emulates a joystick via eye tracking, the video shows prototypes of a remote-controlled car and an electric wheelchair, which can then both be controlled by eye tracking. Those are research prototypes designed to illustrate how this technology can be used to empower people, and appeal to developers, innovators and makers to download these open source libraries and see what you can do.
By highlighting the Eye Control features in Windows and its APIs, we encourage developers and eye tracking device manufacturers to harness the power of eye control for their own applications and products to enhance accessibility and utilize a new mode of interacting with a Windows-based computer. We are keen to see where the development of this technology is leading us in the future.
Skype is designed to keep you connected to the people who matter the most and help you do the work you need to do. We take your feedback very seriously and are grateful to our active accessibility community who point us to areas of improvement to live up to our vision. Building on the work we announced earlier this year, we’ve been listening to your feedback and have continued to work to make the latest version of Skype (version 8.0) even more accessible.
Focus assist, which announces when you receive an incoming call for keyboard only and screen reader users.
Screen reader announcements from users in the active call list.
Navigational focus announcements within the contact list and settings to improve flow.
Announcing the name, content, and actions of a pop up dialog.
New message announcements when users are in another chat.
Smoother navigation that takes a more natural left-to-right and top-to-bottom path.
Displaying additional information when sending and receiving messages. For example, Skype now announces when messages are sent and when messages you attempt to send have failed.
We are committed to continually improving our software and services to meet the needs of all our customers. Please share your thoughts and ideas via the Microsoft Accessibility UserVoice or contact the Disability Answer Desk for real-time support via phone, chat, or ASL videophone—your feedback is always appreciated. If you are an early adopter—and comfortable with early preview releases—please consider joining the Skype Insider community.
By Wendy Chisholm, Principal Accessibility Architect
In May, Microsoft CEO Satya Nadella announced AI for Accessibility to put AI tools in the hands of developers, universities, NGOs and inventors to accelerate the development of accessible and intelligent AI solutions to benefit people with disabilities around the world.
Today, we are announcing the first AI for Accessibility grantee, Zyrobotics. They are developing unique solutions for accessible science, technology, engineering and math (STEM) education, like ReadAble Storiez, that are adaptive to the diverse needs of students. We are thrilled to have them as part of our program and we can’t wait to see the impact they will have!
ReadAble Storiez is a reading fluency program for students with diverse learning needs, which also helps fill in the gaps for students from low-income homes who may not have access to speech-language or occupational therapists. By creating custom speech models with Microsoft Cognitive Services and Azure Machine Learning, they aim to identify when a student needs feedback, much like an occupational therapist would recognize and provide.
Zyrobotics is a team of educators and developers who have a passion for creating technology that will enhance the quality of life for children. Last week, Satya had the opportunity to meet with Zyrobotics CEO Dr. Johnetta MacCalla and CTO Dr. Ayanna Howard in Atlanta, Georgia, and learn more about their work and passion for helping all students become better readers.
As Dr. Howard shared with us, “In our experience, because every child is different, you have to let them look at technology in different ways, more than just visual and auditory. Our focus is to incorporate interaction and personalization. The usage of AI/machine learning algorithms ensures that our apps are accessible and can be adapted to the personal needs of each child. We are excited about how this grant and partnership with Microsoft will help us address significant gaps for engaging each child based on their unique strengths and abilities.”
When can I apply? Now! We are accepting submissions on a rolling basis over the next five years. Grant proposals can be submitted via the online application form. Applicants will receive a follow up email within 30-60 days.
What is Microsoft looking for in applications? We are looking for revolutionary ideas that use AI and align with any of our three areas of focus:
Employment. How can we positively impact the employment rate for people with disabilities through more intelligent technology?
Daily Life. How can we increase access to technology for people with disabilities while also decreasing the cost of such technology?
Communication & Connection. How can AI help improve the speed, accuracy, and convenience of communication for people with disabilities?
We are looking for individuals or teams who are not only passionate about making the world more inclusive, but also firmly rooted in the communities they intend to benefit. We want to see ideas that are developed by or with people with disabilities, not just for us.
What can I receive through the program? First, check that your idea meets the following criteria:
Relevance: AI and/or machine learning are core to the project success;
Impact: Clearly elevates either employment, daily life, or communication & connection for someone with a disability;
Feasible: Is deliverable in a one-year timeframe—this could be phase 1 of the project but there should be tangible deliverables completed each quarter that are wrapped up in one year;
Capable: Your team should have the knowledge and skill to deliver the idea OR you partner with someone who can fill in the gaps;
Sustainable: The project has room to grow beyond the first year of funding;
Showcase-able: We want to talk about whatever you deliver!
The AI for Accessibility program provides monetary grants as well as technology and expertise depending on the potential impact and needs of each project. This can include:
The costs of crowd-sourcing and cleaning data needed to get your project up and running;
Guidance from a Microsoft team of AI, project management, and accessibility experts who can help guide you to scale to a wide audience and deliver the impact you envision.
If you have additional questions about the awards program or the application process, please visit the AI for Accessibility FAQ.
Remember: anyone can apply! From a tinkerer in a basement to an established corporation, from students to professionals. If you are worried you don’t have what it takes, find someone who does and partner with them on an application. So, no excuses. 😊
Over the past eighteen months, Microsoft has fostered a growing $115 million AI for Good initiative that puts AI in the hands of inspired individuals developing real-world outcomes that tackle some of society’s most critical challenges. Comprised of AI for Accessibility, AI for Earth and AI for Humanitarian Action, we are hopeful the world will see what a compelling force for good AI can be.
We cannot wait to see what you create by or with people with disabilitiesto empower everyone to achieve more.