How Europe’s clinicians and patients are using data and AI to fight cancer

Fabian Bolin was just 28-years-old when he found out he had leukemia. A promising actor, the diagnosis of cancer made him feel as if he suddenly lost control of his future and that nothing could help him regain it.

His experience is all too common.

Each year, there are an estimated 3.7 million new cases of cancer and 1.9 million deaths from the disease in Europe. According to the World Health Organization, despite making up only one eighth of the total global population, Europe bears a quarter of the world’s cancer cases. In fact, cancer is the second leading cause of death across the region behind cardiovascular disease.

While Europe is home to some of the best and most established healthcare systems in the world, cancer remains a formidable opponent. Today, leading healthcare providers and organizations are using technology such as artificial intelligence (AI) to engage and support patients, empower doctors and accelerate research. Moving us one-step closer to help manage and conquer the disease.

Giving power back to the patient
When Fabian was first diagnosed, he felt powerless and began sharing his experiences on social media. The response was so great that he helped launch WarOnCancer, a social network for cancer patients and relatives.

Group shot of people smiling while wearing war on cancer tshirts

The original platform comprised of a 150-member strong blogging community, who represented 40 types of cancer, highlighted that most cancer patients suffer from low self-esteem and depression. With this insight, WarOnCancer is working with six partners in the pharmaceutical and broader life science industry to develop and test a new mobile app, which aims to become a global social network for cancer patients.

Scheduled to launch during 2019, the app will allow members to share their data and track how the industry uses this data in research. Through the power of Microsoft Azure, WarOnCancer can analyze this data to detect flaws and benefits experienced by different groups of patients depending on where, and how, they are treated.

“During my treatment and interactions with specialists, I was astounded to learn that almost half of clinical trials in oncology are delayed because it’s hard to find patients who meet the right criteria for that particular trial,” said Fabian. “Despite the vast majority of patients willing to share their data for clinical trials, many don’t know these are even taking place or aren’t properly informed how their data will be used. This disconnect can literally be the difference between finding a life-saving treatment or not.”

“The long-term goal is to build a ‘matchmaking’ type service for clinical trials and patients. This will increase the number of successful clinical trials, spearhead the pharmaceutical R&D-process, tailor treatment schedules and medication around a cancer patient’s needs, and ultimately save lives,” says Sebastian Hermelin, co-founder and head of WarOnCancer’s industry partnerships.

Helping doctors deliver early-detection, and increase precision and accuracy

The benefits of early cancer detection are clear. Not only does it result in a higher survival rate, but it helps minimize treatment side effects. While the process varies in every country, standard breast cancer screening typically occurs every two years and involves the mammography of women within a certain age bracket.

However, the effectiveness of mammography dramatically decreases when examining ‘dense’ breasts with a higher percentage of fibroglandular tissue. To address this challenge, the Veneto Institute of Oncology (IOV) is using a new breast density assessment tool from Volpara that has the potential to help millions of people. Leaping beyond the limits of a traditional mammogram, the cloud-based solution assesses images of a patient’s breast tissue, honing in on its density.

“Since dense breast tissue and lesions both appear white on X-rays, it is difficult to detect cancer in women with dense breasts. Moreover, it has been proven that women with dense breasts have higher risk of developing breast cancer compared to women with low breast density,” says Gisella Gennaro, Medical Physicist at the Venetian Institute of Oncology. ““But now, through advanced image analysis, we can automatically and objectively assess women’s breast density, use it to estimate their risk of developing breast cancer, and provide them with personalized imaging protocols such as using ultrasound in the event that breast density hinders cancer detection.”

“Without advanced image computing, it would be impossible to get such fast and accurate analysis. Over the next five years: we plan to examine more than 10,000 women; see an increase in cancer detection rates; a decrease in interval cancers; and sustainable screening costs. It’s truly a step forward towards precision medicine,” says Francesca Caumo, Director of Breast Radiology Department at the Venetian Institute of Oncology.

Back in Stockholm, Fabian and his team are tireless in their mission to improve the lives for everyone affected by cancer. It has been almost four years since his initial diagnosis and the journey to date has been nothing short of courageous. Alongside first-rate treatment and family support, data has also proved a somewhat hidden helping hand.

Whether its researchers, clinicians or patients – together with cloud computing and AI – humanity’s war on cancer has never been as fierce.

For more information on how Data&AI are helping clinicians, researchers and patients to make healthcare more efficient, click here.


Podcast series explores how AI can help solve society’s toughest challenges

YouTube Video

A podcast series sponsored by Microsoft on how artificial intelligence is helping people solve previously intractable societal challenges launches Monday, Feb. 4, on This Week in Machine Learning and AI. The six-episode “AI For the Benefit of Society with Microsoft” series highlights how AI breakthroughs are advancing work in environmental sustainability, precision medicine, accessibility and life-saving humanitarian assistance.

Hosted by Sam Charrington, the podcast episodes cover technologies and people using AI to pinpoint communities that are at risk of famine before it strikes, help children with autism get additional communication tools, fight climate change through sustainable forest management and develop chatbots to efficiently connect refugees with legal services. They also explore cross-cutting themes around AI and ethics, including how to account for bias in data, ensure new technologies work for the broadest range of users and build a culture of responsible innovation.

Episodes will be available on the following dates at the This Week in Machine Learning and AI website and on Spotify, iTunes and Google Play.

  • Feb. 4: AI for Humanitarian Action (podcast, transcript)
    With Justin Spelhaug, Microsoft general manager for Technology for Social Impact
  • Feb. 6: AI for Accessibility
    With Wendy Chisholm, Microsoft principal accessibility architect, and AI for Accessibility grantee InnerVoice
  • Feb. 8: AI for Earth
    With Lucas Joppa, Microsoft chief environmental officer, and AI for Earth grantee SilviaTerra
  • Feb. 18: AI for Healthcare
    With Peter Lee, corporate vice president, Microsoft Healthcare
  • Feb. 20: Human-Centered Design
    With Mira Lane, Microsoft partner director–ethics and society
  • Feb. 22: Fairness in Machine Learning
    With Hanna Wallach, principal researcher at Microsoft Research


Jennifer Langston writes about Microsoft research and innovation. Follow her on Twitter.


AI & IoT Insider Labs: Helping transform smallholder farming in Kenya

This blog post was authored by Peter Cooper, Senior Product Manager, Microsoft IoT.

From smart factories and smart cities to virtual personal assistants and self-driving cars, artificial intelligence (AI) and the Internet of Things (IoT) are transforming how people around the world live, work, and play.

But fundamentally changing the ways people, devices, and data interact is not simple or easy work. Microsoft’s AI & IoT Insider Labs was created to help all types of organizations accelerate their digital transformation. Member organizations around the world get access to support both technology development and product commercialization, for everything from hardware design to manufacturing to building applications and turning data into insights using machine learning.

Here’s how AI & IoT Insider Labs is helping one partner, SunCulture, leverage new technology to provide solar-powered water pumping and irrigation systems for smallholder farmers in Kenya.

Affordable irrigation for all


Kenyan smallholdings face some of the most challenging growing conditions in the world. 97 percent rely on natural rainfall to support their crops and livestock—and the families that depend on them. But just 17 percent of the country’s farmland is suitable for rainfed agriculture. Electricity is unavailable in most places and diesel power is often financially out of reach, so farmers spend hours every day pumping and transporting water. This limits them to low-value crops like maize and small yields, all because they lack the resources to irrigate their crops. Additionally, irrigation technologies have an important role to play in reducing the impact agriculture has on the earth’s freshwater resources, especially in Africa.

SunCulture, a 2017 Airband Grant Fund winner, believed sustainable technology could make irrigation affordable enough that even the poorest farmers could use it without further aggravating water shortages. The company set out to build an IoT platform to support a pay-as-you-grow payment model that would make solar-powered precision irrigation financially accessible for smallholders across Kenya.

How SunCulture’s solution works

SunCulture’s RainMaker2 pump combines the energy efficiency of solar power with the effectiveness of precision irrigation, making it cheaper and easier for farmers to grow high-quality fruits and vegetables. Using the energy of the sun, the SunCulture system pulls water from any source—lake, stream, well, etc.—and pumps it directly to the farm with sprinklers and drip irrigation.

This cutting-edge solution combines ClimateSmart™ solar and lithium-ion energy storage technology with cloud-based remote monitoring and optimization software developed with support from AI & IoT Insider Labs. It’s a powerful platform that makes it simple and cheap to deploy off-grid energy and connected solutions.

Farmers get the information they need to make good irrigation decisions at scale, without the costs involved in sending agronomy experts into the field. How? SunCulture processes a steady flow of sensor data, like soil moisture, pump efficiency, solar battery storage, and other factors, that is analyzed within Microsoft Azure’s cloud environment. This sensor data is combined with data from SunCulture’s network of 2,000 hyperlocal weather stations to leverage Azure machine learning tools and provide simple, real-time, precision irrigation recommendations directly to the farmer via text messaging (SMS).


The platform also enables real-time locking and unlocking of devices that makes the pay-as-you-grow model feasible. The platform is smart enough to shut off pumps automatically when power levels are getting low on a cloudy day, or when optimal irrigation thresholds are reached.

How farmers are benefiting from SunCulture

SunCulture’s pay-as-you-grow revenue model allows farmers to make small, monthly payments until they own their precision sensor-based irrigation system outright, empowering even the region’s poorest smallholder farmers to take control of their environment.

On average, SunCulture customers enjoy a 300 percent increase in crop yields and a 10x increase in annual income. Farmers with livestock double their milk yield, earning an extra $3.50/day in income from milk alone. The 17 hours per week they used to spend moving water manually is now directed to better tending their crops and livestock. At a price point of $1.25/day for the RainMaker2 with ClimateSmart™, a farmer’s investment is recouped quickly, and profit starts flowing from increased agricultural productivity.

Download SunCulture’s case study to learn more.


Microsoft CTO Kevin Scott on Venture Beat: Understanding AI is part of being an informed citizen in the 21st century

Microsoft CTO Kevin Scott believes understanding AI in the future will help people become better citizens.

“I think to be a well-informed citizen in the 21st century, you need to know a little bit about this stuff [AI] because you want to be able to participate in the debates. You don’t want to be someone to whom AI is sort of this thing that happens to you. You want to be an active agent in the whole ecosystem,” he said.

In an interview with VentureBeat in San Francisco this week, Scott shared his thoughts on the future of AI, including facial recognition software and manufacturing automation. He also detailed why he’s “cautiously optimistic” about the ways people will devise to use intelligent machines and why he thinks Cortana doesn’t need a smart speaker to succeed.

However vital staying informed about the evolution of AI may be to the average person in the century ahead, Scott concedes it’s not an easy thing to do.

“It’s challenging, because even if you’re a person with significant technical training, even if you’re an AI practitioner, it’s sort of challenging to keep up with everything that’s going on. The landscape is evolving really rapidly,” he said.

Technologists who make and use AI today also have a duty to help people better understand what’s possible and make their work accessible, so Scott is writing a book about how AI can be a force for good for the economy in rural America.

In recent years, AI has proliferated across health care and homes, as well as governments and businesses, and its continued expansion could redefine work roles for everyone. News and public education initiatives to help citizens understand AI are important, and technologists should make their work more accessible, but Scott believes it’s not enough for businesses using AI to be disruptive in their industry.

“We have to think about how there’s balance here,” he said. “You can’t just create a bunch of tech and have it be super disruptive and not have any involvement … you have to create value in this world, and it can’t just be shareholder value.”

A ‘cautiously optimistic’ view of facial recognition

One subject that has drawn much attention from average citizens and Microsoft is facial recognition software and the potential for government overreach.

On Tuesday, the American Civil Liberties Union (ACLU) — along with a coalition of human rights and other organizations — called for major tech companies, including Microsoft, to abstain from selling facial recognition technology to governments, because doing so would inevitably lead to misuse and discrimination against religious and ethnic minority groups.

Microsoft declined to respond directly to the letter but pointed to past actions that represent its point of view. Analysis last year found facial recognition systems from Microsoft, as well as Face++ in China, were not capable of recognizing people with dark skin, particularly women of color, at the same rates as white people. Just weeks after Microsoft made improvements to the Face API’s ability to identify people with dark skin tones last summer, president Brad Smith declared that the government needs to regulate facial recognition software. Then last month the company laid out six principles it will use to govern the use of facial recognition software by its customers, including law enforcement agencies and governments, such as fairness, transparency, and accountability.

Microsoft is currently on track to implement the plan on schedule, Scott said.

Though facial recognition software could be used for nefarious purposes by businesses and governments and can drum up fears of technologically powered police states, Scott likes to think of the upside when it comes to facial recognition software use cases.

“There’s this fine line between … that boundary; there are clearly some things that you just shouldn’t allow. Like, you shouldn’t have governments using it as a mechanism of oppression. No one should be using it to discriminate illegally against people, so I think it’s a good debate to have, but I’m usually on the cautiously optimistic side of things — I actually have faith in humanity,” he said. “I believe if you give people tools, the overwhelming majority of the uses to which they will be put are positive, and so you want to encourage that and protect against the negative in a thoughtful way.”

Potential positive use cases he cites include improving security in buildings, understanding who’s in a meeting, or verifying that a person handling dangerous machinery is certified to do so.

He also offered a theoretical example based on what he observed when his wife was in the hospital last year. Just two nurses were tasked with managing an entire a hospital recovery ward, where patients were prescribed a precise regiment of ambulatory activity.

A computer vision system assigned to this task could alert nursing staff if a patient was seen in common areas too often, signaling too much activity, or if they hadn’t been seen out of their room, indicating that they were not getting enough activity.

In addition to a belief that understanding AI makes for more informed citizens, Scott emphasized that AI experts need to do more to share the positive outcomes that can come from technology like facial recognition software.

The Terminator often comes to mind in worst-case scenarios with AI, but sharing a Star Trek vision of the future is important too, Scott said, because telling positives stories helps people grasp those possibilities.

“Folks who are deeply in the AI community need to do a better job trying to paint positive pictures for folks, [but] not in a Pollyanna way, and not ignoring the unintended consequences and all the bad things that could be amplified by AI,” he said.

Scott’s book on AI in rural America

Scott believes a book will help expound on his point of view “that AI can and should be a beneficial thing for rural America.” A Microsoft spokesperson declined to share the book title or scheduled release date details.

To write the book, Scott said he began by thinking about how to define AI for his grandfather, a former appliance repairman, farmer, and boiler room mechanic during World War II.

“I think if my granddad were alive he’d be curious about AI, and part of my process is figuring out how I would explain it to him, because he wasn’t a computer scientist. And I think it’s part of your set of responsibilities these days as a tech person to try to do more of that, to make the things that you’re working on more accessible,” he said.

The book will likely draw on Scott’s experiences growing up in rural Virginia.

When asked which form of AI he believes is likely to have a more positive impact than anticipated, Scott pointed to manufacturing automation in rural areas. It’s easy to imagine advanced robotics being a disruptive factor in manufacturing, but it can also level the playing field worldwide, making it possible to establish business anywhere.

“I have talked with dozens of both small and large companies over the past couple of years, and in every last one of these conversations the thing that I’m seeing is that automation is this sort of equalizing factor, like a piece of advanced automation that runs in Shenzhen costs about the same as it does in some little rural town [in the U.S.],” he said.

“That’s this thing I think people haven’t really fully wrapped their heads around, this whole agile manufacturing movement, where you’ve got lots of these small companies that are now able to make things [and] that are repatriating jobs to the U.S. from overseas, just because they’re deploying all of this automation and their unit cost of production is dropping.”


From seed to sip: How Anheuser-Busch InBev uses AI to drive growth

Anheuser-Busch InBev (AB InBev) is using artificial intelligence to drive growth and innovation across all dimensions of its global brewing business.

The brewer of Budweiser, Corona, Stella Artois and more than 500 other beer brands has built a worldwide analytics platform on the Microsoft Azure cloud, enabling the company to draw data driven-insights about everything from optimal barley growing conditions to drivers of successful sales promotions.

Tassilo Festetics, AB InBev’s vice president for global solutions, shared insights about the company’s AI strategy at a recent AI in Business event in San Francisco, which Transform edited into an abbreviated Q&A.

How is Anheuser-Busch InBev using AI today?

Tassilo Festetics, AB InBev’s vice president for global solutions.
Tassilo Festetics.

FESTETICS:  The question is not going to be where we deploy AI, but where is it not going to be deployed, because we see it in so many different fields.

Can you share an example or two?

FESTETICS: Smart Barley, which is one of our platforms, enables us to work with farmers to use artificial intelligence to improve their yields, reduce water usage, reduce fertilizer usage and create a much more sustainable environment. We started there five years ago.

Now we see AI in the customer-facing area with chatbots and voice. Customers are expecting to have the same frictionless interaction with every company that they’re also having in their private life. Conversational bots that allow your customers to interact with your company in that way are a basic machine learning algorithm.

We also use AI in our supply chain and back-office operations. We use Azure to simplify tasks that people are performing every day and to make people’s lives much more focused on real added-value activities rather than just on transactional activities.

How did you get started in your AI transformation?

FESTETICS: When our company was born, the cloud was not there yet. Microsoft was even not there yet. We were born in 1366. So obviously we are not a digital company. We are a company that’s being digitized.

Our company has grown over time, as a large global organization our data landscape was fragmented. For us the first step was really looking at how we basically get data together, how do we harmonize it, how do we platform it. When we looked at the entire data infrastructure we said, ‘OK let’s just not touch it. Let’s hope it doesn’t break.’

We basically rebuilt everything totally as if we were starting a new company today. With advantages in technology and the cloud you can do that. And that saves a lot of time and allows you certainly to be much more agile. But for sure it’s the biggest barrier to get that data right at the beginning.

How did you develop AI expertise within AB InBev?

FESTETICS: We were very lucky that our senior management understood very early on that this was something that we should work on. So we were early to invest in new resources to join the company, because obviously we didn’t have them around. But then we also started to develop and improve the capabilities of our own people.

Last year I took my entire team to Berkeley. We spent a week on just machine learning. And it was very fun, because normally if I take my team anywhere, they are very — well, they know a lot of things. So if you put them in a room with professors after one day they will probably be explaining to the professors how life works.

In this course about machine learning you could hear a pin drop after the lesson, because everybody was still processing. And that’s, I think, the important part — that you really continue learning and you continue to build those capabilities inside of your company.

By getting new people in and by developing new skills in your people you start to see different approaches to problem solving. These people will start to find ways to deploy new technologies, new methodologies inside of the company to provide better customer service, better waste management, improved ROI on certain activities. It really starts a different way of thinking.

What advice would you give to companies that are just getting started with AI?

FESTETICS: Really start looking at your data early, because data is the fundamental part. There is no AI without data. Then start looking at the areas where you have the best business cases, where can you drive the most value for your company.

Top photo: At the Conversations in AI event, Microsoft CVP of Azure Julia White leads a panel discussion with (left to right) Tassilo Festetics, vice president, global solutions, Anheuser-Busch InBev; Abhishek Pani, senior director of AI product and data science, Adobe; Jack Brown, SVP of product and software, Arccos Golf; and Fiona Tan, SVP of customer technology, Walmart Labs.


  • Watch how Anheuser-Busch InBev taps data for even better beer.
  • Read more about how customers including ABInBev are using AI.
  • Learn more about Microsoft’s AI tools for businesses.

Fishy business: Putting AI to work in Australia’s Darwin Harbour

Identifying and counting fish species in murky water filled with deadly predators is a difficult job. But fisheries scientists in the Northern Territory are working on an artificial intelligence project with Microsoft that has incredible potential for marine science around the world.

Your mission should you choose to accept it, is to go into one of Australia’s largest harbours and count the fish. Think this sounds daunting? You don’t know the half of it.

First, there’s the water. There’s a lot of it in Darwin Harbour – five times more than Sydney Harbour, to be precise. Heavy tides swell more than seven metres then retract, leaving little visibility in their wake.

And if you think you’ve got some occupational hazards at work, try getting your job done in an environment teeming with some of the world’s most intimidating apex predators – saltwater crocodiles, along with tiger, bull and hammerhead sharks. More than 300 salties are caught in the harbour each year.

This is the daunting task of the Department of Primary Industry and Resources for the Northern Territory Government, as it goes about ensuring fisheries resources are sustainably managed and developed for future generations.

Identifying and counting fish species in murky water filled with deadly predators makes diving to count fish species impossible.

Murky water filled with deadly predators like the saltwater crocodile make diving to count and identify fish species impossible.

“If you’re in the water with a crocodile you aren’t taking a calculated risk. You’re going to be a statistic. That’s it. If you’re in the water and he’s there, he wants you and you’re gone.” – Wayne Baldwin, Research Technical Officer, NT Fisheries

If shooting fish in a barrel is a metaphor for something all-too-easy, the correct metaphor for something exceptionally challenging might be counting fish in Darwin Harbour. Yet the NT Fisheries team, led by Dr Shane Penny, Fisheries Research Scientist, do it every day. As the old saying goes, you can’t manage what you can’t measure, so their work begins with knowing how many fish there are.

But they were bogged down by the time it took to wade through hours of underwater footage. The team needed to assess the abundance of critical fish species faster and more accurately, while maintaining a safe distance from deadly predators.

A meeting of the minds

It was from these murky depths that an innovative project showed the potential for artificial intelligence (AI) to support the important work being done by this team of marine biologists. Amid rising debate about the potential impact of AI on society, a collaboration between these scientists and Microsoft engineers became an opportunity to test out its powers as a force for good. Could technology hold the key to safely, accurately and rapidly counting fish – giving the NT Fisheries team more time to devote to analysing this data and improving the sustainable management of NT fish stocks?

The NT Fisheries team had high hopes. They had been using a baited remote underwater video (BRUV) to help with high-risk data gathering. The camera allows the team to see what’s in the water without going in. But even with BRUV on their side, the task was formidable.

Using a GoPro, researchers at NT Fisheries begin the process of assessing critical fish species.

Shane Penny, Fisheries Research Scientist and his team using baited underwater cameras.

“We’ve had quite a few problems with sharks coming in and taking the baits away. Tawny sharks have learned how to open our baits and suck it all out before we have a chance to collect any video.”
– Wayne Baldwin, Research Technical Officer, NT Fisheries

Then there was the sheer quantity of work involved. Once the video is collected, terabytes of footage must be viewed, and its content scoured and quantified. To put this in perspective, a single terabyte would store 500-hours of your favourite movies. The team was identifying vast quantities of different fish species and tracking their behaviour. This diversity and the murkiness of the water meant classification was often far from simple.

Steve van Bodegraven, a Microsoft machine learning engineer and Darwin local, worked with the NT Fisheries team over several months to see whether computer vision would be up to the ambitious task of identifying fish in underwater images.

In a similar way to how tags are suggested for friends and relatives in the photos you upload to social media – through repeated exposure and the discovery of patterns – the project’s success depended on feeding the system with training images. Along the way they had to confront an array of unusual problems. For example, how would Microsoft’s AI solution respond to fish like gold-spotted cod that can change colour to blend into their environment?

“We went in and talked to them about how they work and the challenges they face,” van Bodegraven says. “From that we tried to figure out how we could help. Everything we do is explorative, so we don’t necessarily have solutions out of the box.”

Three months and thousands of images later, results are encouraging to the scientists. To date the system is showing great potential, having learnt to identify 15 different species, from black jewfish to golden snapper which are under careful management to rebuild breeding stocks.

fisheries gif

The AI solution automates the laborious process of counting local fish stocks by progressively learning to identify different varieties of fish.

“We threw a few test images of fish it’s never seen before and it’s managed to pull those out and differentiate them from the fish it does know about. Once we had that first positive identification of a fish, we really felt we were onto something. From there it was just a matter of finding the right tools to improve and optimise.”
Dr Shane Penny, Fisheries Research Scientist

With each new fish analysed, the power of the machine learning technology increases. Samantha Nowland, the team’s Darwin-born research assistant, sees the potential for such systems to change the game in marine management.  NT has some of the most pristine waters in the world with healthy populations of endangered species such as sawfish and sharks. The development of this technology and its availability may help other areas of the world to improve their understanding of aquatic resources and ensure they are managed sustainably.

Beyond the harbour

While there’s already talk of using the system to create a global database of fish species, the NT Fisheries team is focused on analysing trends, coming up with management plans and expanding its reach.

“It’s going to help us monitor any marine species in Darwin Harbour and around the region,” Penny says. “We have a lot of endangered species and many more where we don’t have enough data. We need research projects that can identify species accurately.”

Microsoft’s van Bodegraven hopes it will open people’s eyes to the transformative potential of AI in fisheries and marine management and beyond. The project has already piqued the interest of fisheries departments across Australia, while the possibility of using the technology to monitor other animal species, like the iconic Kookaburra, is being actively explored.

Microsoft is also exploring how it could support similar projects elsewhere. By making the technology available via open source platform GitHub, the technology giant is encouraging others to build AI solutions that address their unique scenarios.

“Projects like this set a new precedent. Hopefully it will make people curious and give them the confidence to explore the application of AI in their industries,” van Bodegraven says. “It’s going to change industries and societies. The potential is only limited by imagination.”

Steve van Bodegraven, Machine Learning Engineer at Microsoft and Dr Shane Penny, Fisheries Research Scientist at NT Fisheries review the identified fish species using the AI solution.

Feeding the world with AI-driven agriculture innovation

In the 1950s and 1960s, plant biologist Norman Borlaug famously led the “Green Revolution,” developing high-yield grains that helped drive up global food production when paired with innovations in chemical fertilizers, irrigation, and mechanized cultivation. By so doing, Borlaug and his peers helped save a billion people from starvation. However, this new form of farming was not sustainable and created multiple environmental issues.

Today, farmers are using technology to transform production again, driven by the need to feed more with less and to address the impacts of industrial farming on the environment. Currently, nearly half of current food produced, or 2 billion tons a year, ends up as waste, while an estimated 124 million people in 51 countries face food insecurity or worse. In addition, new sources of arable land are limited, fresh water levels are receding, and climate change puts pressure on resources and will lower agricultural production over time. Governments need to solve these issues swiftly, as the world’s population is slated to grow from 7.6 billion to 9.8 billion 2050. Agencies and companies will need to team with growers to drive a 70 percent increase in food production.

The good news is that we’re now in the midst of a second Green Revolution that’s part of the Fourth Industrial Revolution. Here’s how technology innovation, driven by big data, the Internet of Things (IoT), artificial intelligence (AI), and machine learning, will reap a more bountiful harvest.

A vision for AI in agriculture

Farmers are deploying robots, ground-based wireless sensors, and drones to assess growing conditions. They then capitalize on cloud services and edge computing to process the data. By 2050, the typical farm is expected to generate an average of 4.1 million data points every day.

AI and machine learning interpret findings for farmers, helping them continually tweak crop inputs to boost yields. Farmers can use AI to determine the optimal date to sow crops, precisely allocate resources such as water and fertilizer, identify crop diseases for swifter treatment, and detect and destroy weeds. Machine learning makes these activities smarter over time. It can also help farmers forecast the year ahead by using historic production data, long-term weather forecasts, genetically modified seed information, and commodity pricing predictions, among other inputs, to recommend how much seed to sow.

Such precision farming technology augments and extends farmers’ deep knowledge about their land, making production more sustainable. Advanced technology can increase farm productivity by 45 percent while reducing water intake by 35 percent. However, the key is ensuring equitable access: Often the communities that need AI the most lack the physical and technology infrastructure required to support it.

Connecting communities with broadband

Access to high-speed connectivity and reliable power are still challenges in many parts of the world. That’s one reason Microsoft and its partners are bringing affordable broadband to rural communities in countries such as Colombia, India, Kenya, South Africa, and the United States through the Airband Initiative.

When communities are connected, farmers can benefit from AI and machine learning, even if they lack internet access to their individual farms. Microsoft employee Prashant Gupta and his team used advanced analytics and machine learning to create a Personalized Village Advisory Dashboard for 4,000 farmers in 106 villages and a Sowing App for 175 farmers in a district in the southeastern coastal state of Andhra Pradesh in India. Farmers with simple SMS-enabled phones can access Sowing App recommendations, which apply AI to data such as weather and soil conditions to optimize planting times. Farmers who followed the AI-driven advice increased yields by 30 percent over those who adhered to traditional planting schedules.

Using IoT and AI on individual farms

Farmers with connectivity can use IoT to get customized recommendations. The Microsoft FarmBeats program, driven by principal researcher Ranveer Chandra, has developed an end-to-end IoT platform that uses low-cost sensors, drones, and vision/machine learning algorithms to increase the productivity and profitability of farms. FarmBeats is part of Microsoft AI for Earth, a program that provides cloud and AI tools to teams seeking to develop sustainable solutions to global environmental issues.

In the United States, FarmBeats solves the problem of internet connectivity by accessing unused TV white spaces to set up high-bandwidth links between a farmer’s home internet connections and an IoT base station on the farm. Sensors, cameras, and drones connect to this base station, which is both solar- and battery-powered. To avoid unexpected shutdowns due to battery drain, the base station uses weather forecasts to plan its power usage. Similarly, drones leverage an IoT-driven algorithm based on wind patterns to help accelerate and decelerate mid-flight, reducing battery draw.

IoT data processing—for bandwidth-hogging information like drone videos, photos, and sensor feedback—is done by a PC at the farmer’s home. The PC performs local computations and consolidates findings into lower-memory summaries, which can be distributed over bandwidth more easily, while also serving as a backup during network outages.

AI for everyone means more food for the world

Over time, AI will help farmers evolve into agricultural technologists, using data to optimize yields down to individual rows of plants. Farmers without connectivity can get AI benefits right now, with tools as simple as an SMS-enabled phone and the Sowing App. Meanwhile, farmers with Wi-Fi access can use FarmBeats to get a continually AI-customized plan for their lands. With such IoT- and AI-driven solutions, farmers can meet the world’s needs for increased food sustainably—growing production and revenues without depleting precious natural resources.

Be the first to know about new advancements in the Microsoft AI farming initiative. Follow us at FarmBeats.

To stay up to date on the latest news about Microsoft’s work in the cloud, bookmark this blog and follow us on TwitterFacebook, and LinkedIn.


AI transforms photo management for Japanese pro baseball

Sports stars are among the most photographed people on the planet today. Their on-field performance, style, gestures, and facial expressions are almost continuously captured digitally for fans, the media, commercial use, and, ultimately, posterity.

It’s not unusual for thousands of pictures to be shot from all angles at any professional encounter nowadays. So, a typical season is likely to produce virtual mountains of images for major clubs and competitions in most sports.

Now, professional baseball in Japan is turning to artificial intelligence and the cloud to handle the magnitude of what has been a laborious and time-consuming task – photo management.

Sports photos can have immediate, lasting, and lucrative value – but only if they are kept in well organized and cataloged collections that can be accessed efficiently. IMAGE WORKS – a service of iconic Japanese photo giant, Fujifilm – manages the Nippon Professional Baseball’s (NPB) cloud-based Content Images Center (CIC).

Here curators sort images, identify players in each image and tag those images with that information. It sounds simple, but the volume of imagery now being produced is huge. The usual way of managing this is simply not keeping up.

To understand why let’s look at the special place baseball holds in modern Japan where it has been a wildly popular game since the 1930s. While its rules differ slightly from those of America’s favorite pastime, the NPB is to Japan is what Major League Baseball (MLB) is to the United States. The NPB consists of two top professional leagues: the Central League and the Pacific League. Each has six teams, and each holds 146 games a season, playing on most days of the week from March to October. Each league then holds its own playoffs, which are followed by the seven-game Nippon Series Championship between the two league champions – in a spectacle similar to that of World Series in the United States.

The automatic player name-tagging function can often identify players even in images that do not show their faces.

There is a steady deluge of images from across the country for much of the year with about 3,000 images shot at each game. After the crowds have left the stadiums, curators from each team typically select about 300 photographs. They then spend around four hours manually identifying and tagging player information to each picture.

That sort of timing can be a problem in our fast-paced world. Demand for images is usually at its highest in realtime or near realtime – that is, during or immediately after, each game. Fans and media can quickly lose interest in content from a past game once a new one begins. So, not only is the job of player image identification massive, it needs to be done fast.

Now AI has stepped up to the plate. Developers from Fujifilm and Microsoft Japan have devised a solution: an automatic player name-tagging function that identifies and tags images much faster than people can, and in greater volumes.

Since June 2018, it has been in a trial that has focused on just five baseball teams – including Hiroshima Toyo Carp, which has won the Central League championship eight times, and the Nippon Series three times. The trial was such a success, the function will be used for all NPB teams in the 2019 season.

Its photo analysis capabilities are based on pre-trained AI from Microsoft Cognitive Services and a deep learning framework from the Microsoft Cognitive Toolkit. Specifically, facial recognition using the Microsoft Cognitive Services Face API is combined with a unique determination model built on the Microsoft Cognitive Toolkit.

This enables the classification of images into four types—batting, pitching, fielding, and base running. Often, it can also determine a player’s name when his face is not visible in an angled or side shot. Azure Durable Functions and Automatic Player Name Tagging, and a final manual check by people has reduced overall processing time from the traditional four hours to just 30 minutes.

A sample of IMAGE WORKS baseball photo collection

Through its developmental stages, Microsoft Japan provided a ResNet neural network model from Microsoft Research, its research and development arm. It also held several hackathons with Fujifilm Software, which is the developer of IMAGE WORKS. Repeated verification exercises saw player recognition accuracy rates jump to over 90%.

“With the power of Azure and deep learning, we have been able to create an AI solution that makes of our photo service so much more efficient and faster. And, that is good for our customers,” said Riki Sato, Team Leader of the Advanced Solution Group at IMAGE WORKS. His colleague Daichi Hayata hailed the collaboration between IMAGE WORKS team and Microsoft Japan. “This was the first time we have dealt with deep learning, and we could do it with basic knowledge,” he said.

Fujifilm Imaging Systems now has plans to widen its use to amateur baseball leagues and then other sports. It might also be applied to the content needs outside the sports world. And, it is looking at the use of video analysis through Azure Video Indexer.

Microsoft Japan is committed to helping companies and organization embrace digital transformation with AI and is considering how to use this combination of pre-trained AI and a customizable deep learning framework in other fields, such as medicine.


More ways to improve patient care with AI and blockchain

Whether you’re interested in using Artificial Intelligence (AI) and Machine Learning (ML) to drive better health outcomes, reduce your operational costs, or improve fraud detection, one way you can better unlock these capabilities is through leveraging blockchain.

In my last blog, “Improving Patient Care through AI and Blockchain – Part 1,” I discussed several opportunities for blockchain to help advance AI in healthcare, from sourcing more training data from across a consortium, to tracking provenance of data, improving the quality of AI with auditing, and protecting the integrity of AI using blockchain. In this second blog, take a look at four more reasons to consider blockchain for advancing AI in healthcare.

  1. Shared models
    In cases where constraints exist that preclude the sharing of raw training data from across a consortium of healthcare organizations, for legal or other reasons, it may be possible to incrementally train shared models, enabled by the blockchain. In this approach the AI / ML models themselves can be shared across the network of healthcare organizations in the consortium, rather than the raw training data, and these shared models can be incrementally trained by each organization using its training data, and within its firewall. Blockchain can then be used to share the models as well as metadata about training data, results, validations, audit trails, and so forth.
  2. Incentivizing collaboration using cryptocurrencies and tokens
    Cryptocurrencies and tokens on blockchain can be used to incent and catalyze collaboration to advance AI / ML in healthcare. From sharing of training data, to collaboration on shared models, results, validations, and so forth, healthcare organizations can be rewarded with cryptocurrencies or tokens proportional to their participation and contribution. Depending on how the blockchain is setup these cryptocurrencies or tokens could be redeemed by participating healthcare organizations for meaningful rewards, or monetized. This can be useful in any AI / ML blockchain initiative both as an accelerant, and could also be critical to overcome potential impediments and reservations to collaboration that can arise where the size / value of contributions from organizations across the consortium are asymmetrical.
  3. Validating inference results and building trust fasterBefore AI / ML models can be used for patient care they must be validated to ensure safety and efficacy. A single organization validating a model alone will take more time to achieve an acceptable level of trust than would be the case for a consortium of healthcare organizations concurrently collaborating to validate a shared model. Blockchain can be used to coordinate and collaborate around such validation to increase synergy, minimize redundant efforts, accelerate validation, and establish trust in a new model faster.
  4. Automation through smart contracts and DAOsExecutable code for processing transactions associated with AI / ML, whether procurement of training data or otherwise, can be implemented on blockchains in the form of smart contracts. DAOs (Decentralized Autonomous Organizations) such as non-profits can also be built using smart contracts to automate whole enterprises that can facilitate advancing AI / ML in healthcare at scale.

Keep the conversation going

If you’re interested in using AI, ML, or blockchain for healthcare, you know that new opportunities are constantly surfacing and with it come a whole host of new questions. Follow me on LinkedIn and Twitter to get updates on these topics as well as cloud computing, security, privacy, and compliance. If you would like to explore a partnership as you work to implement AI and/or blockchain for your healthcare organization, we’d love to hear from you.

For more resources and tips on blockchain for healthcare, take a look at part 1 of this series here.


Microsoft agrees to acquire conversational AI and bot development company, XOXCO

Conversational AI is quickly becoming a way in which businesses engage with employees and customers: from creating virtual assistants and redesigning customer interactions to using conversational assistants to help employees communicate and work better together. According to Gartner, “By 2020, conversational artificial intelligence will be a supported user experience for more than 50 percent of large, consumer-centric enterprises.”* At Microsoft, we envision a world where natural language becomes the new user interface, enabling people to do more with what they say, type and input, understanding preferences and tasks and modeling experiences based on the way people think and remember.

Logo of XOXOCOToday, we are announcing we have signed an agreement to acquire XOXCO, a software product design and development studio known for its conversational AI and bot development capabilities. The company has been paving the way in conversational AI since 2013 and was responsible for the creation of Howdy, the first commercially available bot for Slack that helps schedule meetings, and Botkit, which provides the development tools used by hundreds of thousands of developers on GitHub. Over the years, we have partnered with XOXCO and have been inspired by this work.

We have shared goals to foster a community of startups and innovators, share best practices and continue to amplify our focus on conversational AI, as well as to develop tools for empowering people to create experiences that do more with speech and language.

The Microsoft Bot Framework, available as a service in Azure and on GitHub, supports over 360,000 developers today. With this acquisition, we are continuing to realize our approach of democratizing AI development, conversation and dialog, and integrating conversational experiences where people communicate.

Over the last six months, Microsoft has made several strategic acquisitions to accelerate the pace of AI development. The acquisition of Semantic Machines in May brought a revolutionary new approach to conversational AI. In July, we acquired Bonsai to help reduce the barriers to AI development by combining machine teaching, reinforcement learning and simulation. In September, we acquired Lobe, a company that has created a simple visual interface empowering anyone to develop and apply deep learning and AI models quickly, without writing code. The acquisition of GitHub in October demonstrates our belief in the power of communities to help fuel the next wave of bot development.

Our goal is to make AI accessible and valuable to every individual and organization, amplifying human ingenuity with intelligent technology. To do this, Microsoft is infusing intelligence across all its products and services to extend individuals’ and organizations’ capabilities and make them more productive, providing a powerful platform of AI services and tools that makes innovation by developers and partners faster and more accessible, and helping transform business by enabling breakthroughs to current approaches and entirely new scenarios that leverage the power of intelligent technology.

We’re excited to welcome the XOXCO team and look forward to working with the community to accelerate innovation and help customers capitalize on the many benefits AI can offer.

*Gartner, Is Conversational AI the Only UX You Will Ever Need?, 25 April 2018

Tags: , , , ,