Posted on Leave a comment

AI and preventative healthcare: Diagnosis in the blink of an eye

In his office in suburban Beijing, Zhang proudly demonstrated the physical part of Airdoc’s system – a small desktop device that looks similar to a scanner a neighborhood optometrist might use for a routine eye exam.

You sit on a stool, lean forward, place your chin on a padded brace, and stare into the darkness of an eyepiece. The algorithm then takes over, precisely adjusting the angle of your head until a green cross comes into focus in the gaze of your right eye. A moment later there’s a bright, but not uncomfortable, flash of white light. The process is repeated for your left eye.

The machine has just taken high-resolution medical-grade images of both your retinas. It instantly sends them to the cloud where it takes 20 to 30 milliseconds (about the same time as an eye blink) of computation to analyze both.

Above: Taking a test at Airdoc’s Beijing office.

Moments later an impressively detailed diagnostic dashboard is sent to your smartphone. It rates from low to medium to high your susceptibility to a long list of diseases. If there is a problem, it urges you to seek professional medical help.

Right now, it can search for 30 diseases. More machine learning will soon boost that number to 50, and eventually, it could go beyond 200.

Zhang regards his system as a gamechanger because of its potential to deliver at scale and relieve stretched medical resources. To date, it has scanned more than 1.12 million people, mostly in China, but also in the United States, India, Britain, and parts of Africa. “Airdoc users are all over the world.  We hope our deep learning technology can prevent all kinds of disease.”

China, with a population of 1.3 billion, only has about 1,100 eye doctors who are qualified to analyze retinal images. So, the challenge of providing adequate diagnostic services is truly massive – and perhaps no more so than for the epidemic of diabetes.

Authorities estimate as many as 114 million Chinese have diabetes – but only 30 percent of them know that. The other 70 percent are unaware and, without early detection, will eventually be struck down with serious maladies, like blindness, strokes and other potentially fatal conditions.

“Diabetic retinopathy, or DR, is one of the most common and serious complications of diabetes. Once patients feel symptoms, they are already in a severe stage of DR and will go blind without proper treatment,” says Dr. Rui Li Wei (pictured in top image) of Shanghai’s Changzheng Hospital, one of several major medical institutions that now routinely uses Airdoc’s technology as a quick, accurate, and simple diagnostic tool.

Posted on Leave a comment

Pioneers in AI series launches with interview of Apache Spark inventor Matei Zaharia

Matei Zaharia, Chief Technologist at Databricks & Assistant Professor of Computer Science at Stanford University, in conversation with Joseph Sirosh, Chief Technology Officer of Artificial Intelligence in Microsoft’s Worldwide Commercial Business


At Microsoft, we are privileged to work with individuals whose ideas are blazing a trail, transforming entire businesses through the power of the cloud, big data and artificial intelligence. Our new “Pioneers in AI” series features insights from such pathbreakers. Join us as we dive into these innovators’ ideas and the solutions they are bringing to market. See how your own organization and customers can benefit from their solutions and insights.

Our first guest in the series, Matei Zaharia, started the Apache Spark project during his PhD at the University of California, Berkeley, in 2009. His research was recognized through the 2014 ACM Doctoral Dissertation Award for the best PhD dissertation in Computer Science. He is a co-founder of Databricks, which offers a Unified Analytics Platform powered by Apache Spark. Databricks’ mission is to accelerate innovation by unifying data science, engineering and business. Microsoft has partnered with Databricks to bring you Azure Databricks, a fast, easy, and collaborative Apache Spark based analytics platform optimized for Azure. Azure Databricks offers one-click set up, streamlined workflows and an interactive workspace that enables collaboration between data scientists, data engineers, and business analysts to generate great value from data faster.

So, let’s jump right in and see what Matei has to say about Spark, machine learning, and interesting AI applications that he’s encountered lately.

Video and podcast versions of this session are available at the links below. The podcast is also available from your Spotify app and via Stitcher. Alternatively, just continue reading the text version of their conversation below, via this blog post.

Joseph Sirosh: Matei, could you tell us a little bit about how you got started with Spark and this new revolution in analytics you are driving?

Matei Zaharia: Back in 2007, I started doing my PhD at UC Berkeley and I was very interested in data center scale computing, and we just saw at the time that there was an open source MapReduce implementation in Apache Hadoop, so I started early on by looking at that. Actually, the first project was profiling Hadoop workloads to identify some bottlenecks and, as part of that, we made some improvements to the Hadoop job scheduler and that actually went into Hadoop and I started working with some of the early users of that, especially Facebook and Yahoo. And what we saw across all of these is that this type of large data center scale computing was very powerful, there were a lot of interesting applications they could do with them, but just the map-reduce programming model alone wasn’t really sufficient, especially for machine learning – that’s something everyone wanted to do where it wasn’t a good fit but also for interactive queries and streaming and other workloads.

So, after seeing this for a while, the first project we built was the Apache Mesos cluster manager, to let you run other types of computations next to Hadoop. And then we said, you know, we should try to build our own computation engine which ended up becoming Apache Spark.

JS: What was unique about Spark?

MZ: I think there were a few interesting things about it. One of them was that it tried to be a general or unified programming model that can support many types of computations. So, before the Spark project, people wanted to do these different computations on large clusters and they were designing specialized engines to do particular things, like graph processing, SQL custom code, ETL which would be map-reduce, they were all separate projects and engines. So in Spark we kind of stepped back at them and looked at these and said is there any way we can come up with a common abstraction that can handle these workloads and we ended up with something that was a pretty small change to MapReduce – MapReduce plus fast data sharing, which is the in-memory RDDs in Spark, and just hooking these up into a graph of computations turned out to be enough to get really good performance for all the workloads and matched the specialized engines, and also much better performance if your workload combines a bunch of steps. So that is one of the things.

I think the other thing which was important is, having a unified engine, we could also have a very composable API where a lot of the things you want to use would become libraries, so now there are hundreds maybe thousands of third party packages that you can use with Apache Spark which just plug into it that you can combine into a workflow. Again, none of the earlier engines had focused on establishing a platform and an ecosystem but that’s why it’s really valuable to users and developers, is just being able to pick and choose libraries and arm them.

JS: Machine Learning is not just one single thing, it involves so many steps. Now Spark provides a simple way to compose all of these through libraries in a Spark pipeline and build an entire machine learning workflow and application. Is that why Spark is uniquely good at machine learning?

MZ: I think it’s a couple of reasons. One reason is much of machine learning is preparing and understanding the data, both the input data and also actually the predictions and the behavior of the model, and Spark really excels at that ad hoc data processing using code – you can use SQL, you can use Python, you can use DataFrames, and it just makes those operations easy, and, of course, all the operations you do also scale to large datasets, which is, of course, important because you want to train machine learning on lots of data.

Beyond that, it does support iterative in-memory computation, so many algorithms run pretty well inside it, and because of this support for composition and this API where you can plug in libraries, there are also quite a few libraries you can plug in that call external compute engines that are optimized to do different types of numerical computation.

JS: So why didn’t some of these newer deep learning toolsets get built on top of Spark? Why were they all separate?

MZ: That’s a good question. I think a lot of the reason is probably just because people, you know, just started with a different programming language. A lot of these were started with C++, for example, and of course, they need to run on the GPU using CUDA which is much easier to do from C++ than from Java. But one thing we’re seeing is really good connectors between Spark and these tools. So, for example, TensorFlow has a built-in Spark connector that can be used to get data from Spark and convert it to TFRecords. It also actually connects to HDFS and different sorts of big data file systems. At the same time, in the Spark community, there are packages like deep learning pipelines from Databricks and quite a few other packages as well that let you setup a workflow of steps that include these deep learning engines and Spark processing steps.

“None of the earlier engines [prior to Apache Spark] had focused on establishing a platform and an ecosystem.”

JS: If you were rebuilding these deep learning tools and frameworks, would you recommend that people build it on top of Spark? (i.e. instead of the current approach, of having a tool, but they have an approach of doing distributed computing across GPUs on their own.)

MZ: It’s a good question. I think initially it was easier to write GPU code directly, to use CUDA and C++ and so on. And over time, actually, the community has been adding features to Spark that will make it easier to do that in there. So, there’s definitely been a lot of proposals and design to make GPU a first-class resource. There’s also this effort called Project Hydrogen which is to change the scheduler to support these MPI-like batch jobs. So hopefully it will become a good platform to do that, internally. I think one of the main benefits of that is again for users that they can either program in one programming language, they can learn just one way to deploy and manage clusters and it can do deep learning and the data preprocessing and analytics after that.

JS: That’s great. So, Spark – and Databricks as commercialized Spark – seems to be capable of doing many things in one place. But what is not good at? Can you share some areas where people should not be stretching Spark?

MZ: Definitely. One of the things it doesn’t do, by design, is it doesn’t do transactional workloads where you have fine grained updates. So, even though it might seem like you can store a lot of data in memory and then update it and serve it, it is not really designed for that. It is designed for computations that have a large amount of data in each step. So, it could be streaming large continuous streams, or it could be batch but is it not these point queries.

And I would say the other thing it does not do it is doesn’t have a built-in persistent storage system. It is designed so it’s just a compute engine and you can connect it to different types of storage and that actually makes a lot of sense, especially in the cloud, with separating compute and storage and scaling them independently. But it is different from, you know, something like a database where the storage and compute are co-designed to live together.

JS: That makes sense. What do you think of frameworks like Ray for machine learning?

MZ: There are lot of new frameworks coming out for machine learning and it’s exciting to see the innovation there, both in the programming models, the interface, and how to work with it. So I think Ray has been focused on reinforcement learning which is where one of the main things you have to do is spawn a lot of little independent tasks, so it’s a bit different from a big data framework like Spark where you’re doing one computation on lots of data – these are separate computations that will take different amounts of time, and, as far as I know, users are starting to use that and getting good traction with it. So, it will be interesting to see how these things come about.

I think the thing I’m most interested in, both for Databricks products and for Apache Spark, is just enabling it to be a platform where you can combine the best algorithms, libraries and frameworks and so on, because that’s what seems to be very valuable to end users, is they can orchestrate a workflow and just program it as easily as writing a single machine application where you just import a bunch of libraries.

JS: Now, stepping back, what do you see as the most exciting applications that are happening in AI today?

MZ: Yeah, it depends on how recent. I mean, in the past five years, deep learning is definitely the thing that has changed a lot of what we can do, and, in particular, it has made it much easier to work with unstructured data – so images, text, and so on. So that is pretty exciting.

I think, honestly, for like wide consumption of AI, the cloud computing AI services make it significantly easier. So, I mean, when you’re doing machine learning AI projects, it’s really important to be able to iterate quickly because it’s all about, you know, about experimenting, about finding whether something will work, failing fast if a particular idea doesn’t work. And I think the cloud makes it much easier.

JS: Cloud AI is super exciting, I completely agree. Now, at Stanford, being a professor, you must see a lot of really exciting pieces of work that are going on, both at Stanford and at startups nearby. What are some examples?

MZ: Yeah, there are a lot of different things. One of the things that is really useful for end users is all the work on transfer learning, and in general all the work that lets you get good results with AI using smaller training datasets. There are other approaches as well like weak supervision that do that as well. And the reason that’s important is that for web-scale problems you have lot of labeled data, so for something like web search you can solve it, but for many scientific or business problems you don’t have that, and so, how can you learn from a large dataset that’s not quite in your domain like the web and then apply to something like, say, medical images, where only a few hundred patients have a certain condition so you can’t get a zillion images. So that’s where I’ve seen a lot of exciting stuff.

But yeah, there’s everything from new hardware for machine learning where you throw away the constraints that the computation has to be precise and deterministic, to new applications, to things like, for example security of AI, adversarial examples, verifiability, I think they are all pretty interesting things you can do.

JS: What are some of the most interesting applications you have seen of AI?

MZ: So many different applications to start with. First of all, we’ve seen consumer devices that bring AI into every home, or every phone, or every PC – these have taken off very quickly and it’s something that a large fraction of customers use, so that’s pretty cool to see.

In the business space, probably some of the more exciting things are actually dealing with image data, where, using deep learning and transfer learning, you can actually start to reliably build classifiers for different types of domain data. So, whether it’s maps, understanding satellite images, or even something as simple as people uploading images of a car to a website and you try to give feedback on that so it’s easier to describe it, a lot of these are starting to happen. So, it’s kind of a new class of data, visual data – we couldn’t do that much with it automatically before, and now you can get both like little features and big products that use it.

JS: So what do you see as the future of Databricks itself? What are some of the innovations you are driving?

MZ: Databricks, for people not familiar, we offer basically, a Unified Analytics Platform, where you can work with big data mostly through Apache Spark and collaborate with it in an organization, so you can have different people, developing say notebooks to perform computations, you can have people developing production jobs, you can connect these together into workflows, and so on.

So, we’re doing a lot of things to further expand on that vision. One of the things that we announced recently is what we call machine learning runtime where we have preinstalled versions of popular machine learning libraries like XGBoost or TensorFlow or Horovod on your Databricks cluster, so you can set those up as easily as you can set up as easily as you can setup an Apache Spark cluster in the past. And then another product that we featured a lot at our Spark Summit conference this year is Databricks Delta which is basically a transactional data management layer on top of cloud objects stores that lets us do things like indexing, reliable exactly once stream processing, and so on at very massive scale, and that’s a problem that all our users have, because all our users have to setup a reliable data ingest pipeline.

JS: Who are some of the most exciting customers of Databricks and what are they doing?

MZ: There are a lot of really interesting customers doing pretty cool things. So, at our conference this year, for example, one of the really cool presentations we saw was from Apple. So, Apple’s internal information security group – this is the group that does network monitoring, basically gets hundreds of terabytes of network events per day to process, to detect intrusions and information security problems. They spoke about using Databricks Delta and streaming with Apache Spark to handle all of that – so it’s one of the largest applications people have talked about publicly, and it’s very cool because the whole goal there – it’s kind of an arms race between the security team and attackers – so you really want to be able to design new rules, new measurements and add new data sources quickly. And so, the ease of programming and the ease of collaborating with this team of dozens of people was super important.

We also have some really exciting health and life sciences applications, so some of these are actually starting to discover new drugs that companies can actually productionize to tackle new diseases, and this is all based on large scale genomics and statistical studies.

And there are a lot of more fun applications as well. Like actually the largest video game in the world, League of Legends, they use Databricks and Apache Spark to detect players that are misbehaving or to recommend items to people or things like that. These are all things that were featured at the conference.

JS: If you had one advice to developers and customers using Spark or Databricks, or guidance on what they should learn, what would that be?

MZ: It’s a good question. There are a lot of high-quality training materials online, so I would say definitely look at some of those for your use case and see what other people are doing in that area. The Spark Summit conference is also a good way to see videos and talks and we make all of those available for free, the goal of that is to help and grow the developer community. So, look for someone who is doing similar things and be inspired by that and kinda see what the best practices are around that, because you might see a lot of different options for how to get started and it can be hard to see what the right path is.

JS: One last question – in recent years there’s been a lot of fear, uncertainty and doubt about AI, and a lot of popular press. Now – how real are they, and what do you think people should be thinking?

MZ: That’s a good question. My personal view is – this sort of evil artificial general intelligence stuff – we are very far away from it. And basically, if you don’t believe that, I would say just try doing machine learning tutorials and see how these models break down – you get a sense for how difficult that is.

But there are some real challenges that will come from AI, so I think one of them is the same challenge as with all technology which is, automation – how quickly does it happen. Ultimately, after automation, people usually end up being better off, but it can definitely affect some industries in a pretty bad way and if there is no time for people to transition out, that can be a problem.

I think the other interesting problem there is always a discussion about is basically access to data, privacy, managing the data, algorithmic discrimination – so I think we are still figuring out how to handle that. Companies are doing their best, but there are also many unknowns as to how these techniques will do that. That’s why we’ll see better best practices or regulations and things like that.

JS: Well, thank you Matei, it’s simply amazing to see the innovations you have driven, and looking forward to more to come.

MZ: Thanks for having me.

“When you’re doing machine learning AI projects, it’s really important to be able to iterate quickly because it’s all about experimenting, about finding whether something will work, failing fast if a particular idea doesn’t work.


And I think the cloud makes it much easier.”

We hope you enjoyed this blog post. This being our first episode in the series, we are eager to hear your feedback, so please share your thoughts and ideas below.

The AI / ML Blog Team

Resources

Posted on Leave a comment

How cloud AI can power a prosthetic arm that sees objects

Based on “Connected Arms”, a keynote talk at the O’Reilly AI Conference delivered by Joseph Sirosh, CTO for AI at Microsoft. Content reposted from this O’Reilly Media website.

[embedded content]

There are over 1 million new amputees every year, i.e. one every 30 seconds – a truly shocking statistic.

The World Health Organization estimates that between 30 to 100 million people around the world are living with limb loss today. Unfortunately, only 5-15% of this population has access to prosthetic devices.

Although prostheses have been around since ancient times, their successful use has been severely limited for millennia by several factors, with cost being the major one. Although it is possible to get sophisticated bionic arms today, the cost of such devices runs into tens of thousands of dollars. These devices are just not widely available today. What’s more, having these devices interface satisfactorily with the human body has been a massive issue, partly due to the challenges of working with the human nervous system. Such devices generally need to be tailored to work with each individual’s nervous system, a process that often requires expensive surgery.

Is it possible for a new generation of human beings to finally help us break through these long-standing barriers?

Can prosthetic devices learn to adapt to us, as opposed to the other way around?

A Personalized Prosthetic Arm for $100?

In his talk, Joseph informs us about how, using the combination of:

  • Low-cost off-the-shelf electronics,
  • 3D-printing, and
  • Cloud AI, for intelligent, learned, personalized behavior,

it is now becoming possible to deliver prosthetic arms at a price point of around $100.

Joseph takes the smartARM as an example of such a breakthrough device. A prototype built by two undergraduate students from Canada who recently won the first prize in Microsoft’s Imagine Cup, the smartARM is 3D-printed, has a camera in the palm of its hand and is connected to the cloud. The magic is in the cloud, where a computer vision service recognizes the objects seen by the camera. Deep learning algorithms then generate the precise finger movements needed to grasp the object near the arm. Essentially, the cloud vision service classifies the object and generates the right grip or action, such as a pincer action to pick up a bunch of keys on a ring, or a palmar action to pick up a wineglass. The grip itself is a learned behavior which can be trained and customized.

The user of the prosthetic arm triggers the grip (or its release) by flexing any muscle of their choice on their body, for instance, their upper arm muscle. A myoelectric sensor located in a band that is strapped over that muscle detects the signal and triggers the grip or its release.

Simple, Adaptable Architecture

The architecture of this grip classification solution is shown below. The input to the raspberry pi on the smartARM comes from camera and the muscle sensor. These inputs are sent to the Azure Custom Vision Service, an API in the cloud which has been trained on grip classifications and is able to output the appropriate grip. This grip is sent back to an Arduino board in the smartARM which can then trigger the servo motors that realize that grip in the physical world, i.e. as soon as the smartARM gets the signal to do so from the muscle sensor.

This is an adaptable architecture. It can be customized to the kinds of movements you want this arm to generate. For instance, the specific individual using this prosthetic can customize the grips for the objects in their daily life which are the ones they care the most about. The muscle sensor -based trigger could be replaced with a speech trigger, if so desired.

Summary

AI is empowering a new generation of developers to explore all sorts of novel ideas and mashups. Through his talk on “Connected Arms”, Joseph shows us how the future of prosthetic devices can be transformed by the power of the cloud and AI. Imagine a world in which all future assistive devices are empowered with AI in this fashion. Devices would adapt to individuals, rather than the other way around. Assistive devices will become more affordable, intelligent, cloud-powered and personalized.

Cloud AI is letting us build unexpected things that we would scarcely have imagined.

Such as like an arm that can see.

The AI / ML Blog Team

Posted on Leave a comment

AI for Earth: Helping save the planet with data science

Wee Hyong Tok is a data scientist. He has a passion for numbers, a faith in technology – and a mission that might make a superhero think twice.

“I want to save the Earth,” he says matter-of-factly. “That seems like a very bold statement. But, I strongly believe that artificial intelligence (AI) can play an important role in monitoring the health of our planet.”

Singapore-born and educated, Wee Hyong has been a data guy and techie all his working life – first in academia, and later with Microsoft in China and the United States where he helped create ground-breaking products in the cloud.

For more than a year now, he has been leading an elite global research team for AI for Earth – a five-year, US$50 million Microsoft initiative that supports, and partners with, environmental groups and researchers. They are tackling some of the world’s most intractable problems by marshaling the immense power of AI, machine learning (ML), and the cloud.

Wee Hyong Tok, Principal Data Science Manager, AI & Research.

In a recent interview during a quick visit back to Singapore, Wee Hyong summed up the challenge: We live on planet Earth, and yet we know very little about it.

We have limited time to learn how to conserve its resources. Fresh water supplies are being dangerously overexploited. Land is being exhausted and degraded to produce more food for more people in ever-growing cities. Thousands of species are fading fast into extinction as their habitats disappear in a whirl of industrialization and a haze of pollution. The oceans are choking on plastics and the carbon-charged climate is changing. Precious things that are vital to our existence are under threat and, if lost, might never come back.

I strongly believe that AI can play an important role in monitoring the health of our planet.

When we hear such things, most of us tend to shrug helplessly. Such problems just seem too big, too hard, and too scary to fix. But Wee Hyong and his colleagues at AI for Earth and Microsoft Research are convinced that solutions can come in our time – if data, technology, and imagination are put to work.

“I am an optimist,” he says before describing the technical complexities surrounding his team’s quest. “We can learn how to leverage AI to solve some of the sustainability challenges facing humanity today.”

Asia’s elusive and endangered Snow Leopard. Photo: Peter Bolliger.

Boiled down, AI for Earth aims to create sustainable solutions across four areas that are key to the health of the planet and the future of humankind: agriculture, water, biodiversity, and climate change.

Wee Hyong proudly points to some early breakthroughs. The Farm Beats project is pioneering new data-driven agriculture to guide farmers in India and the United States on where and when to plant crops for the greatest yield.

Equally impressive are the strides being made in land cover mapping – traditionally a time-consuming, expensive tool that is essential for environmental management and precision conservation. Recently, the entire United States was mapped by machine-learning algorithms that processed nearly 200 million aerial images in just over 10 minutes. Done the usual way, such a project would have taken many months and cost a fortune. Deployed globally and locally, this new way of mapping could revolutionize how we mitigate the effects of urbanization, pollution, deforestation, and even natural disasters.

Endangered species are also being given new hope. Traditionally, analysts pore over of thousands of images taken from satellites, drones or camera traps in the wild to study the range, populations, and behaviors of animals otherwise rarely seen by humans. It’s laborious work that takes time, skill, and concentration. “Try spotting a herd of zebra on the African savannah from a satellite image,” Wee Hyong says. “it’s not easy.”

High resolution imagery of zebra on the African savannah. Photo: Courtesy of Save The Elephants

Now computers can take on this role thanks to deep learning techniques that enable them to make sense of the thousands of pixels in an image. This is freeing up the expert time of scientists to do and study more. It’s already adding invaluable knowledge about elusive snow leopards in Kyrgyzstan in Central Asia and dwindling elephant populations in Congo in Africa where AI is also being used to in the fight against the twin scourges of poaching and the ivory trade.

Project Premonition uses insects as de facto “field biologists”. The project uses AI to analyze blood that mosquitoes take from animals across an  an ecosystem to glean valuable data. To achieve this, AI for Earth is developing drones that autonomously locate mosquito hotspots, robotic traps to collect specimens, and cloud-scale genomics and machine learning algorithms to identify each animal bitten.

The rise of the intelligent cloud and the ability to deploy machine learning models to the intelligent edge is accelerating and enabling new exciting possibilities to study and save wildlife from the remotest corners of the Earth to suburban backyards.

African bush elephants with Mount Kilmanjaro in the background. Picture: Courtesy of Save the Elephants

It goes beyond just technology, right? They want to tell their kids they are trying to save the Earth.

Pursuing research is worthy in itself, but real value comes when a solution is launched into action in the real world. It is here that Wee Hyong’s motivation shines through: He wants to leave the world in better shape for his two children – and for all children in the world.

The same goes for his team of data scientists and software engineers who left exciting and satisfying roles in commercial product development to join AI for Earth.

“Every single person who came for a job interview said they wanted to be able to tell their kids and families that they were serving a higher purpose. It goes beyond just technology, right? It goes beyond just new deep learning techniques and approaches, or whatever. They want to tell their kids they are trying to save the Earth.”

Posted on Leave a comment

Breakthrough ideas wanted: Join the Microsoft AI Idea Challenge for cool prizes, exposure

This post is authored by Tara Shankar Jana, Senior Technical Product Marketing Manager at Microsoft.

All of us have creative ideas – ideas that can improve our lives and the lives of thousands, perhaps even millions of others. But how often do we act on turning those ideas into a reality? Most of the time, we do not believe in our ideas strongly enough to pursue them. Other times we feel like we lack a platform to build out our idea or showcase it. Most good ideas don’t go beyond those initial creative thoughts in our head.

If you’re a professional working in the field of artificial intelligence (AI), or an aspiring AI developer or just someone who is passionate about AI and machine learning, Microsoft is excited to offer you an opportunity to transform your most creative ideas into reality. Join the Microsoft AI Idea Challenge Contest today for a chance to win exciting prizes and get your project featured in Microsoft’s AI.lab showcase. Check out the rules, terms and conditions of the contest and then dive right in!

The Challenge

The Microsoft AI Idea Challenge is seeking breakthrough AI solutions from developers, data scientists, professionals and students, and preferably developed on the Microsoft AI platform and services. The challenge gives you a platform to freely share AI models and applications, so they are reusable and easily accessible. The ideas you submit are judged on the parameters shown in the figure below – essentially half the weight is for the originality of your idea, 20% for the feasibility of your solution, and 30% for the complexity (i.e. level of sophistication) of your implementation.

The Microsoft AI Challenge is accepting submissions between now and October 12th, 2018.

To qualify for the competition, individuals or teams are required to submit a working AI model, test dataset, a demo app and a demo video that can be a maximum of three minutes long. We encourage you to register early and upload your projects soon, so that you can begin to plan and build out your solution and turn in the rest of your materials on time. We are looking for solutions across the whole spectrum of use cases – to be inspired, take a look at some of the examples at AI.lab.

Prizes

The winners of the first three places in the contest will respectively receive a Surface Book 2, a DJI Drone, and an Xbox One X.

We hope that’s motivation to get you started today – good luck!

Tara

Posted on Leave a comment

Can sound help save a dwindling elephant population? Scientists using AI think so.

Scientists with the Elephant Listening Project estimate that Africa’s population of forest elephants has dropped from roughly 100,000 animals in 2011 to fewer than 40,000 animals today. But those numbers are largely based on indirect evidence: ivory seizures, signs of poaching and labor-intensive surveys that are too expensive to be done regularly.

The Elephant Listening Project has spent more than three decades researching how elephants use low-frequency rumbling sounds to communicate with one another. More recently, those scientists began to use acoustic sensors at research sites to inform population estimates and, ultimately, to track and protect forest elephants across their ranges in Central and West Africa.

If scientists find, for example, that at specific times of year elephants are using clearings in an unprotected logging concession to access scarce minerals or find mates, scientists can work with the loggers to schedule their work to minimize disturbance and reduce conflicts.

But there has been a bottleneck in getting data out of these remote African forests and analyzing information quickly, says Peter Wrege, a senior research associate at Cornell who directs the Elephant Listening Project.

“Right now, when we come out of the field with our data, the managers of these protected areas are asking right away, ‘What have you found? Are there fewer elephants? Is there a crisis we need to address immediately?’ And sometimes it takes me months and months before I can give them an answer,” says Wrege.

Conservation Metrics began collaborating with the Elephant Listening Project in 2017 to help boost that efficiency. Its machine learning algorithms have been able to identify elephant calls more accurately and will hopefully begin to shortcut the need for human review. But the volume of data from the acoustic monitors, shown in the spectrogram below, is taxing the company’s local servers and computational capacity.

Microsoft’s AI for Earth program has given a two-year grant to Conservation Metrics to build a cloud-based workflow in Microsoft Azure for analyzing and processing wildlife metrics. It has also donated Azure computing resources to the Elephant Listening Project to reduce its data-processing costs for the project. The computational power of Azure will speed processing time dramatically, says Matthew McKown, the CEO of Conservation Metrics. The platform also offers new opportunities for clients to upload and interact with their data directly.

It currently takes about three weeks for computers to process a few months of sound data from this landscape-scale study, says McKown. Once the Azure migration is complete later this year, that same job may take a single day.

“It’s a huge improvement. We’re really interested in speeding up that loop between having equipment monitoring things out in the field and going through this magic process to convert those signals into information you can send into the field where someone can take action,” says McKown. “Right now, that process can take a really long time.”

Posted on Leave a comment

Why Microsoft and National Geographic Society are working together to advance conservation science with computer science

Camper sitting at creek with night sky above
Photograph by Devlin Gandy/National Geographic

By Dr. Jonathan Baillie, chief scientist at National Geographic Society, and Dr. Lucas Joppa, chief environmental scientist at Microsoft

Yesterday, Microsoft and National Geographic Society announced a new, joint grant program that will equip explorers working on the most challenging environmental issues of the 21st century with the most advanced technologies available today. “AI for Earth Innovation” grants will fund new solutions that leverage AI and cloud technologies to monitor, model and manage Earth’s natural resources. Application forms are available today, here, to anyone working at the intersection of computer science and environmental science, especially in the areas of agriculture, biodiversity, climate change and water.

As scientists that have spent our entire respective careers focused on conservation, we’ve come to believe that increased adoption of technology, including AI, is critical to make the progress needed – and at the pace needed – to protect our planet. From producing the foundational estimates on how rapidly species are going extinct to determining the effectiveness of our current conservation efforts, we realized that progress was slow or impossible without deploying scalable technology solutions.

There have been some notable success stories – including those we featured in a book we jointly published on the role of protected areas in conservation. But they are, frustratingly, the exception to the rule.

Now, in our roles as chief scientists at global organizations (for science and exploration and innovative technology, respectively), we hope to solve the root cause of that frustration. That is the goal of this partnership, and why Microsoft and National Geographic Society are bringing $1 million and access to our experts and technology together in this new partnership.

While different, both organizations are focused on pushing the boundaries of science and exploration for the benefit of the planet. National Geographic is synonymous with science and exploration. For 130 years, the organization has opened minds to the natural world, challenged perceptions of what is possible and set the stage for future exploration. For more than 35 years, Microsoft, too, has explored and pushed forward the boundaries of what technology can do, and what it can do for people and the world.

Our organizations have a unique combination of expertise in conservation and computer science, capacity building and public engagement, providing incredible potential to drive fundamental change. We will work together to empower people everywhere to respond to some of the most challenging environmental issues of the 21st century.

We realize that to some, it may seem counterintuitive to try to protect the planet with technology. It’s true that past industrial revolutions and technology development has directly contributed to our current climate crisis. Certainly, we recognize it’s not a panacea. But we’re fundamentally optimistic, because over the course of human history, every solution to a major societal challenge has been the result of human ingenuity and new technologies. It’s been the combination of scientific exploration and technological advances that has fueled new discoveries and led to major breakthroughs in our understanding of the planet and life on Earth. It’s as true today as it was when National Geographic Explorer Bob Ballard discovered new forms of life at the bottom of the ocean using then-cutting edge underwater remotely operated vehicle (ROV) technology.

Lately, innovation in technology has far outpaced anything imaginable before, but scientific knowledge isn’t keeping pace. We have often imagined a future where that is no longer the case, and our individual organizations have worked tirelessly to change this, too.

By partnering, we’re ready to move from imagining to enabling. With AI and the cloud, researchers can stay focused on new discoveries, rather than data collection and sorting. Their findings can more easily be shared with other researchers around the world, creating new economies of scale that accelerate and improve the state of conservation science in near-real time.

While there are only a handful of grants, the program is structured to provide exponential impact. By ensuring that all models supported through this grant follow an open source approach and are publicly available, we will allow environmental researchers and innovators around the globe to take advantage of these new innovations immediately and directly in their own vital work.

For the health of our planet and our future, we all need to get back to nature with the help of technology. Microsoft and National Geographic are ready to put our tools and skills to work for researchers working to make that more sustainable future a reality. Come join us!

Tags: ,

Posted on Leave a comment

Microsoft and National Geographic form AI for Earth Innovation Grant partnership

New grant offering will support research and scientific discovery with AI technologies to advance agriculture, biodiversity conservation, climate change and water

REDMOND, Wash., and WASHINGTON, D.C. — July 16, 2018 — On Monday, Microsoft Corp. and National Geographic announced a new partnership to advance scientific exploration and research on critical environmental challenges with the power of artificial intelligence (AI). The newly created $1 million AI for Earth Innovation Grant program will provide award recipients with financial support, access to Microsoft cloud and AI tools, inclusion in the National Geographic Explorer community, and affiliation with National Geographic Labs, an initiative launched by National Geographic to accelerate transformative change and exponential solutions to the world’s biggest challenges by harnessing data, technology and innovation. Individuals and organizations working at the intersection of environmental science and computer science can apply today at https://www.nationalgeographic.org/grants/grant-opportunities/ai-earth-innovation/.

National Geographic logo“National Geographic is synonymous with science and exploration, and in Microsoft we found a partner that is well-positioned to accelerate the pace of scientific research and new solutions to protect our natural world,” said Jonathan Baillie, chief scientist and executive vice president, science and exploration at the National Geographic Society. “With today’s announcement, we will enable outstanding explorers seeking solutions for a sustainable future with the cloud and AI technologies that can quickly improve the speed, scope and scale of their work as well as support National Geographic Labs’ activities around technology and innovation for a planet in balance.”

“Microsoft is constantly exploring the boundaries of what technology can do, and what it can do for people and the world,” said Lucas Joppa, chief environmental scientist at Microsoft. “We believe that humans and computers, working together through AI, can change the way that society monitors, models and manages Earth’s natural systems. We believe this because we’ve seen it — we’re constantly amazed by the advances our AI for Earth collaborators have made over the past months. Scaling this through National Geographic’s global network will create a whole new generation of explorers who use AI to create a more sustainable future for the planet and everyone on it.”

The $1 million AI for Earth Innovation Grant program will provide financial support to between five and 15 novel projects that use AI to advance conservation research toward a more sustainable future. The grants will support the creation and deployment of open-sourced trained models and algorithms that will be made broadly available to other environmental researchers, which offers greater potential to provide exponential impact.

Qualifying applications will focus on one or more of the core areas: agriculture, biodiversity conservation, climate change and water. Applications are open as of today and must be submitted by Oct. 8, 2018. Recipients will be announced in December 2018. Those who want more information and to apply can visit https://www.nationalgeographic.org/grants/grant-opportunities/ai-earth-innovation/.

About the National Geographic Society

The National Geographic Society is a leading nonprofit that invests in bold people and transformative ideas in the fields of exploration, scientific research, storytelling and education. The Society aspires to create a community of change, advancing key insights about the planet and probing some of the most pressing scientific questions of our time, all while ensuring that the next generation is armed with geographic knowledge and global understanding. Its goal is measurable impact: furthering exploration and educating people around the world to inspire solutions for the greater good. For more information, visit www.nationalgeographic.org.

About Microsoft

Microsoft (Nasdaq “MSFT” @microsoft) enables digital transformation for the era of an intelligent cloud and an intelligent edge. Its mission is to empower every person and every organization on the planet to achieve more.

For more information, press only:

Microsoft Media Relations, WE Communications for Microsoft, (425) 638-7777,

rrt@we-worldwide.com

Note to editors: For more information, news and perspectives from Microsoft, please visit the Microsoft News Center at http://news.microsoft.com. Web links, telephone numbers and titles were correct at time of publication, but may have changed. For additional assistance, journalists and analysts may contact Microsoft’s Rapid Response Team or other appropriate contacts listed at http://news.microsoft.com/microsoft-public-relations-contacts.

Posted on Leave a comment

TextWorld, an open-source project for generating text-based games, can train and test AI agents

Today, fresh out of the Microsoft Research Montreal lab, comes an open-source project called TextWorld. TextWorld is an extensible Python framework for generating text-based games. Reinforcement learning researchers can use TextWorld to train and test AI agents in skills such as language understanding, affordance extraction, memory and planning, exploration and more. Researchers can study these in the context of generalization and transfer learning. TextWorld further runs existing text-based games, like the legendary Zork, for evaluating how well AI agents perform in complex, human-designed settings.

Figure 1 – Enter the world of TextWorld. Get the code at aka.ms/textworld.

Text-based games – also known as interactive fiction or adventure games – are games in which the play environment and the player’s interactions with it are represented solely or primarily via text. As players moves through the game world, they observe textual descriptions of their surroundings (typically divided into discrete ‘rooms’), what objects are nearby, and any other pertinent information. Players issue text commands to an interpreter to manipulate objects, other characters in the game, or themselves. After each command, the game usually provides some feedback to inform players how that command altered the game environment, if at all. A typical text-based game poses a series of puzzles to solve, treasures to collect, and locations to reach. Goals and waypoints may be specified explicitly or may have to be inferred from cues.

Figure 2 – An example game from TextWorld with a house-based theme.

Text-based games couple the freedom to explore a defined space with the restrictions of a parser and game world designed to respond positively to a relatively small set of textual commands. An agent that can competently navigate a text-based game needs to be able to not only generate coherent textual commands but must also generate the right commands in the right order, with little to no mistakes in between. Text-based games encourage experimentation and successful playthroughs involve multiple game losses and in-game “deaths.” Close observation and creative interpretation of the text the game provides and a generous supply of common sense are also integral to winning text-based games. The relatively simple obstacles present in a TextWorld game serve as an introduction to the basic real-life challenges posed by text-based games. In TextWorld, an agent needs to learn how to observe, experiment, fail and learn from failure.

TextWorld has two main components: a game generator and a game engine. The game generator converts high-level game specifications, such as number of rooms, number of objects, game length, and winning conditions, into an executable game source code in the Inform 7 language. The game engine is a simple inference machine that ensures that each step of the generated game is valid by using simple algorithms such as one-step forward and backward chaining.

Figure 3 – An overview of the TextWorld architecture.

“One reason I’m excited about TextWorld is the way it combines reinforcement learning with natural language,” said Geoff Gordon, Principal Research Manager at Microsoft Research Montreal “These two technologies are both really important, but they don’t fit together that well yet. TextWorld will push researchers to make them work in combination.” Gordon pointed out that reinforcement learning has had a number of high-profile successes recently (like Go or Ms. Pac-Man), but in all of these cases the agent has fairly simple observations and actions (for example, screen images and joystick positions in Ms. Pac-Man). In TextWorld, the agent has to both read and produce natural language, which has an entirely different and, in many cases, more complicated structure.

“I’m excited to see how researchers deal with this added complexity, said Gordon.”

Microsoft Research Montreal specializes in start-of-the art research in machine reading comprehension, dialogue, reinforcement learning, and FATE (Fairness, Accountability, Transparency, and Ethics in AI). The lab was founded in 2015 as Maluuba and acquired by Microsoft in 2017. For more information, check out Microsoft Research Montreal.

This release of TextWorld is a beta and we are encouraging as much feedback as possible on the framework from fellow researchers across the world. You can send your feedback and questions to textworld@microsoft.com. Also, for more information and to get the code, check out TextWorld, and our related publications TextWorld: A Learning Environment for Text-based Games and Counting to Explore and Generalize in Text-based Games. Thank you!

Posted on Leave a comment

Facial recognition technology: The need for public regulation and corporate responsibility

All tools can be used for good or ill. Even a broom can be used to sweep the floor or hit someone over the head. The more powerful the tool, the greater the benefit or damage it can cause. The last few months have brought this into stark relief when it comes to computer-assisted facial recognition – the ability of a computer to recognize people’s faces from a photo or through a camera. This technology can catalog your photos, help reunite families or potentially be misused and abused by private companies and public authorities alike.

Facial recognition technology raises issues that go to the heart of fundamental human rights protections like privacy and freedom of expression. These issues heighten responsibility for tech companies that create these products. In our view, they also call for thoughtful government regulation and for the development of norms around acceptable uses. In a democratic republic, there is no substitute for decision making by our elected representatives regarding the issues that require the balancing of public safety with the essence of our democratic freedoms. Facial recognition will require the public and private sectors alike to step up – and to act.

We’ve set out below steps that we are taking, and recommendations we have for government regulation.

First, some context

Facial recognition technology has been advancing rapidly over the past decade. If you’ve ever seen a suggestion on Facebook or another social media platform to tag a face with a suggested name, you’ve seen facial recognition at work. A wide variety of tech companies, Microsoft included, have utilized this technology the past several years to turn time-consuming work to catalog photos into something both instantaneous and useful.

So, what is changing now? In part it’s the ability of computer vision to get better and faster in recognizing people’s faces. In part this improvement reflects better cameras, sensors and machine learning capabilities. It also reflects the advent of larger and larger datasets as more images of people are stored online. This improvement also reflects the ability to use the cloud to connect all this data and facial recognition technology with live cameras that capture images of people’s faces and seek to identify them – in more places and in real time.

Advanced technology no longer stands apart from society; it is becoming deeply infused in our personal and professional lives. This means the potential uses of facial recognition are myriad. At an elementary level, you might use it to catalog and search your photos, but that’s just the beginning. Some uses are already improving security for computer users, like recognizing your face instead of requiring a password to access many Windows laptops or iPhones, and in the future a device like an automated teller machine.

Some emerging uses are both positive and potentially even profound. Imagine finding a young missing child by recognizing her as she is being walked down the street. Imagine helping the police to identify a terrorist bent on destruction as he walks into the arena where you’re attending a sporting event. Imagine a smartphone camera and app that tells a person who is blind the name of the individual who has just walked into a room to join a meeting.

But other potential applications are more sobering. Imagine a government tracking everywhere you walked over the past month without your permission or knowledge. Imagine a database of everyone who attended a political rally that constitutes the very essence of free speech. Imagine the stores of a shopping mall using facial recognition to share information with each other about each shelf that you browse and product you buy, without asking you first. This has long been the stuff of science fiction and popular movies – like “Minority Report,” “Enemy of the State” and even “1984” – but now it’s on the verge of becoming possible.

Perhaps as much as any advance, facial recognition raises a critical question: what role do we want this type of technology to play in everyday society?

The issues become even more complicated when we add the fact that facial recognition is advancing quickly but remains far from perfect. As reported widely in recent months, biases have been found in the performance of several fielded face recognition technologies. The technologies worked more accurately for white men than for white women and were more accurate in identifying persons with lighter complexions than people of color. Researchers across the tech sector are working overtime to address these challenges and significant progress is being made. But as important research has demonstrated, deficiencies remain. The relative immaturity of the technology is making the broader public questions even more pressing.

Even if biases are addressed and facial recognition systems operate in a manner deemed fair for all people, we will still face challenges with potential failures. Facial recognition, like many AI technologies, typically have some rate of error even when they operate in an unbiased way. And the issues relating to facial recognition go well beyond questions of bias themselves, raising critical questions about our fundamental freedoms.

Politics meets Silicon Valley

In recent weeks, the politics of the United States have become more intertwined with these technology developments on the West Coast. One week in the middle of June put the issues raised by facial recognition technology in bold relief for me and other company leaders at Microsoft. As the country was transfixed by the controversy surrounding the separation of immigrant children from their families at the southern border, a tweet about a marketing blog Microsoft published in January quickly blew up on social media and sparked vigorous debate. The blog had discussed a contract with the U.S. Immigration and Customs Enforcement, or ICE, and said that Microsoft had passed a high security threshold; it included a sentence about the potential for ICE to use facial recognition.

We’ve since confirmed that the contract in question isn’t being used for facial recognition at all. Nor has Microsoft worked with the U.S. government on any projects related to separating children from their families at the border, a practice to which we’ve strongly objected. The work under the contract instead is supporting legacy email, calendar, messaging and document management workloads. This type of IT work goes on in every government agency in the United States, and for that matter virtually every government, business and nonprofit institution in the world. Some nonetheless suggested that Microsoft cancel the contract and cease all work with ICE.

The ensuing discussion has illuminated broader questions that are rippling across the tech sector. These questions are not unique to Microsoft. They surfaced earlier this year at Google and other tech companies. In recent weeks, a group of Amazon employees has objected to its contract with ICE, while reiterating concerns raised by the American Civil Liberties Union (ACLU) about law enforcement use of facial recognition technology. And Salesforce employees have raised the same issues related to immigration authorities and these agencies’ use of their products. Demands increasingly are surfacing for tech companies to limit the way government agencies use facial recognition and other technology.

These issues are not going to go away. They reflect the rapidly expanding capabilities of new technologies that increasingly will define the decade ahead. Facial recognition is the technology of the moment, but it’s apparent that other new technologies will raise similar issues in the future. This makes it even more important that we use this moment to get the direction right.

The need for government regulation

The only effective way to manage the use of technology by a government is for the government proactively to manage this use itself. And if there are concerns about how a technology will be deployed more broadly across society, the only way to regulate this broad use is for the government to do so. This in fact is what we believe is needed today – a government initiative to regulate the proper use of facial recognition technology, informed first by a bipartisan and expert commission.

While we appreciate that some people today are calling for tech companies to make these decisions – and we recognize a clear need for our own exercise of responsibility, as discussed further below – we believe this is an inadequate substitute for decision making by the public and its representatives in a democratic republic. We live in a nation of laws, and the government needs to play an important role in regulating facial recognition technology. As a general principle, it seems more sensible to ask an elected government to regulate companies than to ask unelected companies to regulate such a government.

Such an approach is also likely to be far more effective in meeting public goals. After all, even if one or several tech companies alter their practices, problems will remain if others do not. The competitive dynamics between American tech companies – let alone between companies from different countries – will likely enable governments to keep purchasing and using new technology in ways the public may find unacceptable in the absence of a common regulatory framework.

It may seem unusual for a company to ask for government regulation of its products, but there are many markets where thoughtful regulation contributes to a healthier dynamic for consumers and producers alike. The auto industry spent decades in the 20th century resisting calls for regulation, but today there is broad appreciation of the essential role that regulations have played in ensuring ubiquitous seat belts and air bags and greater fuel efficiency. The same is true for air safety, foods and pharmaceutical products. There will always be debates about the details, and the details matter greatly. But a world with vigorous regulation of products that are useful but potentially troubling is better than a world devoid of legal standards.

That’s why Microsoft called for national privacy legislation for the United States in 2005 and why we’ve supported the General Data Protection Regulation in the European Union. Consumers will have more confidence in the way companies use their sensitive personal information if there are clear rules of the road for everyone to follow. While the new issues relating to facial recognition go beyond privacy, we believe the analogy is apt.

It seems especially important to pursue thoughtful government regulation of facial recognition technology, given its broad societal ramifications and potential for abuse. Without a thoughtful approach, public authorities may rely on flawed or biased technological approaches to decide who to track, investigate or even arrest for a crime. Governments may monitor the exercise of political and other public activities in ways that conflict with longstanding expectations in democratic societies, chilling citizens’ willingness to turn out for political events and undermining our core freedoms of assembly and expression. Similarly, companies may use facial recognition to make decisions without human intervention that affect our eligibility for credit, jobs or purchases. All these scenarios raise important questions of privacy, free speech, freedom of association and even life and liberty.

So what issues should be addressed through government regulation? That’s one of the most important initial questions to address. As a starting point, we believe governments should consider the following issues, among others:

  • Should law enforcement use of facial recognition be subject to human oversight and controls, including restrictions on the use of unaided facial recognition technology as evidence of an individual’s guilt or innocence of a crime?
  • Similarly, should we ensure there is civilian oversight and accountability for the use of facial recognition as part of governmental national security technology practices?
  • What types of legal measures can prevent use of facial recognition for racial profiling and other violations of rights while still permitting the beneficial uses of the technology?
  • Should use of facial recognition by public authorities or others be subject to minimum performance levels on accuracy?
  • Should the law require that retailers post visible notice of their use of facial recognition technology in public spaces?
  • Should the law require that companies obtain prior consent before collecting individuals’ images for facial recognition? If so, in what situations and places should this apply? And what is the appropriate way to ask for and obtain such consent?
  • Should we ensure that individuals have the right to know what photos have been collected and stored that have been identified with their names and faces?
  • Should we create processes that afford legal rights to individuals who believe they have been misidentified by a facial recognition system?

This list, which is by no means exhaustive, illustrates the breadth and importance of the issues involved.

Another important initial question is how governments should go about addressing these questions. In the United States, this is a national issue that requires national leadership by our elected representatives. This means leadership by Congress. While some question whether members of Congress have sufficient expertise on technology issues, at Microsoft we believe Congress can address these issues effectively. The key is for lawmakers to use the right mechanisms to gather expert advice to inform their decision making.

On numerous occasions, Congress has appointed bipartisan expert commissions to assess complicated issues and submit recommendations for potential legislative action. As the Congressional Research Service (CRS) noted last year, these commissions are “formal groups established to provide independent advice; make recommendations for changes in public policy; study or investigate a particular problem, issue, or event; or perform a duty.” Congress’ use of the bipartisan “9/11 Commission” played a critical role in assessing that national tragedy. Congress has created 28 such commissions over the past decade, assessing issues ranging from protecting children in disasters to the future of the army.

We believe Congress should create a bipartisan expert commission to assess the best way to regulate the use of facial recognition technology in the United States. This should build on recent work by academics and in the public and private sectors to assess these issues and to develop clearer ethical principles for this technology. The purpose of such a commission should include advice to Congress on what types of new laws and regulations are needed, as well as stronger practices to ensure proper congressional oversight of this technology across the executive branch.

Issues relating to facial recognition go well beyond the borders of the United States. The questions listed above – and no doubt others – will become important public policy issues around the world, requiring active engagement by governments, academics, tech companies and civil society internationally. Given the global nature of the technology itself, there likely will also be a growing need for interaction and even coordination between national regulators across borders.

Tech sector responsibilities

The need for government leadership does not absolve technology companies of our own ethical responsibilities. Given the importance and breadth of facial recognition issues, we at Microsoft and throughout the tech sector have a responsibility to ensure that this technology is human-centered and developed in a manner consistent with broadly held societal values. We need to recognize that many of these issues are new and no one has all the answers. We still have work to do to identify all the questions. In short, we all have a lot to learn. Nonetheless, some initial conclusions are clear.

First, it’s incumbent upon those of us in the tech sector to continue the important work needed to reduce the risk of bias in facial recognition technology. No one benefits from the deployment of immature facial recognition technology that has greater error rates for women and people of color. That’s why our researchers and developers are working to accelerate progress in this area, and why this is one of the priorities for Microsoft’s Aether Committee, which provides advice on several AI ethics issues inside the company.

As we pursue this work, we recognize the importance of collaborating with the academic community and other companies, including in groups such as the Partnership for AI. And we appreciate the importance not only of creating data sets that reflect the diversity of the world, but also of ensuring that we have a diverse and well-trained workforce with the capabilities needed to be effective in reducing the risk of bias. This requires ongoing and urgent work by Microsoft and other tech companies to promote greater diversity and inclusion in our workforce and to invest in a broader and more diverse pipeline of talent for the future. We’re focused on making progress in these areas, but we recognize that we have much more work to do.

Second, and more broadly, we recognize the need to take a principled and transparent approach in the development and application of facial recognition technology. We are undertaking work to assess and develop additional principles to govern our facial recognition work. We’ve used a similar approach in other instances, including trust principles we adopted in 2015 for our cloud services, supported in part by transparency centers and other facilities around the world to enable the inspection of our source code and other data. Similarly, earlier this year we published an overall set of ethical principles we are using in the development of all our AI capabilities.

As we move forward, we’re committed to establishing a transparent set of principles for facial recognition technology that we will share with the public. In part this will build on our broader commitment to design our products and operate our services consistent with the UN’s Guiding Principles on Business and Human Rights. These were adopted in 2011 and have emerged as the global standard for ensuring corporate respect for human rights. We periodically conduct Human Rights Impact Assessments (HRIAs) of our products and services, and we’re currently pursuing this work with respect to our AI technologies.

We’ll pursue this work in part based on the expertise and input of our employees, but we also recognize the importance of active external listening and engagement. We’ll therefore also sit down with and listen to a variety of external stakeholders, including customers, academics and human rights and privacy groups that are focusing on the specific issues involved in facial recognition. This work will take  up to a few months, but we’re committed to completing it expeditiously .

We recognize that one of the difficult issues we’ll need to address is the distinction between the development of our facial recognition services and the use of our broader IT infrastructure by third parties that build and deploy their own facial recognition technology. The use of infrastructure and off-the-shelf capabilities by third parties are more difficult for a company to regulate, compared to the use of a complete service or the work of a firm’s own consultants, which readily can be managed more tightly. While nuanced, these distinctions will need consideration.

Third, in the meantime we recognize the importance of going more slowly when it comes to the deployment of the full range of facial recognition technology. Many information technologies, unlike something like pharmaceutical products, are distributed quickly and broadly to accelerate the pace of innovation and usage. “Move fast and break things” became something of a mantra in Silicon Valley earlier this decade. But if we move too fast with facial recognition, we may find that people’s fundamental rights are being broken.

For this reason, based in part on input from the Aether Committee, we’re moving more deliberately with our facial recognition consulting and contracting work. This has led us to turn down some customer requests for deployments of this service where we’ve concluded that there are greater human rights risks. As we’re developing more permanent principles, we will continue to monitor the potential uses of our facial recognition technologies with a view to assessing and avoiding human rights abuses.

In a similar vein, we’re committed to sharing more information with customers who are contemplating the potential deployment of facial recognition technology. We will continue work to provide customers and others with information that will help them understand more deeply both the current capabilities and limitations of facial recognition technology, how these features can and should be used, and the risks of improper uses.

Fourth, we’re committed to participating in a full and responsible manner in public policy deliberations relating to facial recognition. Government officials, civil liberties organizations and the broader public can only appreciate the full implications of new technical trends if those of us who create this technology do a good job of sharing information with them. Especially given our urging of governments to act, it’s incumbent on us to step forward to share this information. As we do so, we’re committed to serving as a voice for the ethical use of facial recognition and other new technologies, both in the United States and around the world.

We recognize that there may be additional responsibilities that companies in the tech sector ought to assume. We provide the foregoing list not with the sense that it is necessarily complete, but in the hope that it can provide a good start in helping to move forward.

Some concluding thoughts

Finally, as we think about the evolving range of technology uses, we think it’s important to acknowledge that the future is not simple. A government agency that is doing something objectionable today may do something that is laudable tomorrow. We therefore need a principled approach for facial recognition technology, embodied in law, that outlasts a single administration or the important political issues of a moment.

Even at a time of increasingly polarized politics, we have faith in our fundamental democratic institutions and values. We have elected representatives in Congress that have the tools needed to assess this new technology, with all its ramifications. We benefit from the checks and balances of a Constitution that has seen us from the age of candles to an era of artificial intelligence. As in so many times in the past, we need to ensure that new inventions serve our democratic freedoms pursuant to the rule of law. Given the global sweep of this technology, we’ll need to address these issues internationally, in no small part by working with and relying upon many other respected voices. We will all need to work together, and we look forward to doing our part.

Tags: , , ,