Tag: AI for Earth

Woodland Park Zoo, Seattle: streamlining camera traps

Manoj Sarathy, a high school student, is working with Dr. Robert Long of Seattle’s Woodland Park Zoo to develop an AI tool that processes data from wildlife camera traps to monitor species in the Pacific Northwest and in the Rocky Mountains. Using machine learning, this resource will be able to identify and index a wide variety of animals in a much shorter timeframe than if processed manually. Camera traps can generate large amounts of data, some of which are false positives – for example, when something triggers the camera, but nothing meaningful is photographed. This AI tool examines the images and eliminates images that don’t require more detailed inspection.

[Subscribe to Microsoft on the Issues for more on the topics that matter most.]

Cornell University, Ithaca, New York: listening to the sound of the rainforest

Dr. Laurel Symes and Dr. Holger Klinck are leading a project that uses insect sounds to monitor rainforests. Working out of the Cornell Bioacoustics Research Program in Ithaca, New York, their team of researchers uses AI to monitor insect sounds to develop a better understanding of the dynamics of these habitats. Tropical rainforests are home to great swathes of biodiversity, in part, because they are inaccessible to humans. The combination of plant density and high canopies makes these rainforests a challenging environment for traditional study techniques. This team’s first focus is on neotropical rainforest katydids, or bush crickets. The research will then move beyond insects to birds, monkeys and other vocal animals.

ETH Zurich: making the rainforest everyone’s joint concern

David Dao is a doctoral candidate at ETH Zurich, the Swiss Federal Institute of Technology. He is building AI and data systems to help fight deforestation. Historically, figuring out which body is responsible for looking after a section of forest has been a challenge. Dao and his team came up with an innovative idea: Make everyone a caretaker of the forest. This system allows people to make a financial investment in the forest’s wellbeing, with the hope of gaining a return. This gives – giving everyone an incentive to help fight deforestation.

University of Alberta in Canada: grizzly bears and the right to roam

Clayton Lamb is a researcher at the University of Alberta in Canada, specializing in grizzly bears. Lamb is particularly interested in the scope of the bears’ territory – or lack of it – as grizzlies only occupy a fraction of the land they formerly roamed. Much of the remaining population also faces competition for land use from humans. Lamb’s work uses AI and machine-learning tools to create a comprehensive analysis of the human and environmental factors that could contribute to the limiting of grizzly bear density.

iNaturalist: Using AI for species identification

iNaturalist is a social media platform that helps people identify the plants, animals and insects they see anywhere in the world. With badges and features that offer a gaming element, users capture biodiversity data and share their findings. The iNat’s Seek app was created with families in mind with features to mask location data and other potentially sensitive information about children using the app. There are an estimated 10 million species in the world, but only 1.8 million have been discovered and just 90,000 analyzed. So there’s plenty of scope for getting involved.

Colorado State University: Inspiration from above

The Community Collaborative Rain, Hail and Snow Network, CoCoRaHS for short, works with thousands of trained volunteer observers, including children, to gather data on rain, snow and hail. Using tools like plastic rain gauges, citizen scientists measure rain or snowfall each morning and submit their readings to the CoCoRaHS database. Making this data available to weather forecasting services is helping improve severe weather models, making it easier to issue accurate and timely alerts when needed.

City University of New York: Looking to the future 

Just off the Pacific coast of North America lies the Ocean Observatories Initiative (OOI) Regional Cabled Array. It sits atop the Juan de Fuca tectonic plate, collecting oceanographic data. Dr. Dax Soule at City University of New York uses that data to engage students from a variety of backgrounds. One such study concerns the Axial Seamount, which combines a volcanic hotspot with a mid-ocean ridge. Using cloud-based tools to access this data and then conduct important research, Soule is helping to inspire the next generation of oceanographic scientists.

For more on AI for Earth, visit Microsoft AI for Earth. And follow @MSFTIssues on Twitter.  

Episode 72, April 17, 2019

We hear a lot these days about “AI for good” and the efforts of many companies to harness the power of artificial intelligence to solve some of our biggest environmental challenges. It’s rare, however, that you find a company willing to bring its environmental bona fides all the way to the C Suite. Well, meet Dr. Lucas Joppa. A former environmental and computer science researcher at MSR who was tapped in 2017 to become the company’s first Chief Environmental Scientist, Dr. Joppa is now the Chief Environmental Officer at Microsoft, another first, and is responsible for managing the company’s overall environmental sustainability efforts from operations to policy to technology.

Today, Dr. Joppa shares how his love for nature and the joy of discovery actually helped shape his career path, and tells us all about AI for Earth, a multi-year, multi-million dollar initiative to deploy the full scale of Microsoft’s products, policies and partnerships across four key areas of agriculture, water, biodiversity and climate, and transform the way society monitors, models, and ultimately manages Earth’s natural resources.

Related:

---

Final Transcript

Lucas Joppa: We’d been investing in the AI for Earth space, from a pure research perspective, for almost a decade, and it was really when Microsoft started to go all-in in AI that there was a conversation saying look, we’ve been investing in this for ten years, isn’t now the time? If not now, then when? And so I put together a memo called AI for Earth which is how we could take this out of research, take this out of incubation, deploy it across the entire company, and then allow the full kind of scope and scale of a Microsoft to put that in the hands of partner organizations all around the world.

Host: You’re listening to the Microsoft Research Podcast, a show that brings you closer to the cutting-edge of technology research and the scientists behind it. I’m your host, Gretchen Huizinga.

Host: We hear a lot these days about “AI for good” and the efforts of many companies to harness the power of artificial intelligence to solve some of our biggest environmental challenges. It’s rare, however, that you find a company willing to bring its environmental bona fides all the way to the C Suite. Well, meet Dr. Lucas Joppa. A former environmental and computer science researcher at MSR who was tapped in 2017 to become the company’s first Chief Environmental Scientist, Dr. Joppa is now the Chief Environmental Officer at Microsoft, another first, and is responsible for managing the company’s overall environmental sustainability efforts from operations to policy to technology.

Today, Dr. Joppa shares how his love for nature and the joy of discovery actually helped shape his career path, and tells us all about AI for Earth, a multi-year, multi-million dollar initiative to deploy the full scale of Microsoft’s products, policies and partnerships across four key areas of agriculture, water, biodiversity and climate, and transform the way society monitors, models, and ultimately manages Earth’s natural resources. That and much more on this episode of the Microsoft Research Podcast.

Host: Lucas Joppa, welcome to the podcast.

Lucas Joppa: Thanks for having me here.

Host: You’re the Chief Environmental Officer at Microsoft. First off, does anyone else have that, or is that unique to this company?

Lucas Joppa: I think to the best of my knowledge it’s unique to this company. I actually came from a role that was unique to this company as well. So, before I was the Chief Environmental Officer, I was the Chief Environmental Scientist. And as a former Microsoft researcher, I always told myself I didn’t want a job that didn’t have the word “scientist” in it, but as long as I kept environment in the Chief Environmental Officer, then I’m good. There are companies that have, you know, Chief Sustainability Officers and things like that, but those generally have a much more narrow purview to them than the role that I currently have.

Host: Perhaps within the company and the sustainability practices of the company or…?

Lucas Joppa: I think, you know, a traditional Chief Sustainability Officer really thinks about sustainability within the four walls of an organization. How do you reduce the negative environmental impact of an organization’s business practices? And I think that that is a very important aspect of the role. I call it a wholly necessary, but entirely insufficient criterion and for success in the environmental space. Microsoft’s 130,000 employees or so, and that’s pretty small if you look at the 7-plus billion people in the world. So, we’ve got to think about how we take our products, our policies, our partnerships, and use those to really expand our impact all around the world. We’re a tech company. We’re obsessed with scale, right? That’s what we need to be seeking with sustainability as well.

Host: Your title is kind of a spoiler alert to what kind of big problems you are looking to solve. And I usually ask that kind of off-the-bat, but give us a “Virtual Earth 3-D” view of the work you do. What gets you up in the morning?

Lucas Joppa: Ultimately, at the highest level, what gets me up in the morning is the same thing that always has, as far back as I can remember, which is just like kind of incredible sense of wonder about the world. I think, you know, I’ve just always seen my place in the cosmos and on the planet as this tiny infinitesimal spec and just, you know, been fascinated by what else is out there. And as I started to think about that more, you know, some people are super-interested in what’s up there, up in the sky, right? In outer space. I was always interested in what’s here, at home, on planet earth. And then, of course, what’s the human species’ role in that? What impact are we having? How much of that life have we discovered? That’s what, intrinsically, gets me up. I think the thing that then gets me out of bed and gets me to work every day is, what can we do to extend our knowledge base? How can we go out and actually accelerate human discovery of the rest of life on earth and then how can we use some of our tools and our science to mitigate the impact of our own human activities on earth’s natural systems? Because that last bit, that worry about human impact, that’s actually kind of what keeps me up at night, so…

Host: Part of what drives the work of researchers, and the people who work with them to bring their efforts to life, is what we don’t know, that quest for discovery, the quest for knowledge. So, what are the big knowledge gaps in environmental science, and how is computer science helping to narrow those gaps? Just general right now.

Lucas Joppa: Sure. I think if you want to couch it in the framing of the big problem that we have right now is that human society is ultimately facing probably the greatest challenge human society has ever faced. What I mean by that is, we somehow have to figure out how to adapt to and mitigate changing climates, ensure resilient water supply, sustainably feed a human population, currently at seven – growing to ten – billion people, all while stemming a catastrophic loss of biodiversity that’s going on. And so, what you have to do is, you have to know about all of earth’s systems, how we use those systems to sustain our human activities, and then, of course, how we can mitigate and manage those systems to accelerate human progress. I think just one little example that I spent a lot of time on earlier in my research career is just looking at, for instance, the number of species on earth. Actually, we’ve only scientifically discovered something like two million species, scientists estimate. It’s probably something like ten, fifteen million. We’ve only even discovered something like 20%, at best, of life on earth, and of that, you know, if look at what we actually know, a lot of that discovery is just, kind of, a specimen in a drawer with, you know, a Latin name written on it. If you look at, like, how many of those species have we actually kind of studied and we understand their populations, how their populations are faring, done kind of a full conservation assessment for any particular species, we’ve done that for about a hundred thousand species. Of the potentially ten million or more that are out there! And so, we live in this world where, you can just ask your phone how to get to the nearest Starbucks and it’s got to run a least-cost-path algorithm to find out, is it shorter to go twenty steps to the right or thirty steps to the left to get to the nearest one…

Host: Yeah.

Lucas Joppa: …that’s kind of our day-to-day problems, but, when you take a step back and you ask the human position within the rest of life on earth, I mean, we talk about an information age and, you know, information overload. Well, it comes to our understanding of environmental systems. It’s a complete and total information drought.

(music plays)

Host: Well, let’s plunge in right away and talk about AI for Earth which is a pivotal program and one that you’re spearheading here at Microsoft. The program was launched in 2017 and at the time they said it’s a five-year, fifty-million-dollar, cross-company effort to deploy the full-scale of Microsoft’s AI technologies, capabilities and research in four key areas.

Lucas Joppa: Um-hum.

Host: So, let’s start macro.

Lucas Joppa: Um-hum.

Host: And then we’ll get micro in a bit. First, give us an overview of AI for Earth, why it exists and what its core areas of focus are.

Lucas Joppa: Fundamentally, AI for Earth exists to change the way that human society monitors, models, and ultimately, manages, earth’s natural systems. That’s what we’re trying to do. We’re trying to take a technology focus on doing that. Now, when you think about the scope and the scale of the problem, how big earth is, how few people there are for such a huge sphere, and how many species there are and how many complex relationships, you realize that we need to find scalable mechanisms to monitoring and modeling earth’s natural systems. And when you’re embedded inside the tech sector as I’ve been, inside the research arm of a company like Microsoft, you realize just how rapid and incredible the progress we’re making in collecting data, analyzing data, delivering insights to users. And so, it just, for me, became clear that Microsoft should be focusing its attention in that space on environmental systems and that we should really focus on agriculture, water, biodiversity and climate change. Those four issues are inseparable from each other, by the way. I mean obviously, you need water to grow crops and you need species to pollinate crops, and you need predictable climates to be able to plant crops predictably. And so those all kind of bundle together under that higher rubric of AI for Earth and this five-year, fifty-million-dollar commitment to deploying Microsoft’s 35 years, now, of research investments in the key areas of artificial intelligence.

Host: Talk a little bit more about this idea of five years, because it’s not five years. Explain what you mean by that.

Lucas Joppa: Sure, I mean, you know, anybody that works in the tech sector understands that five years is actually a geological age, right? Five years is about as far into the future as we can possibly imagine. If you look at the research-to-product-to-deployment cycle that’s going on right now, you quickly understand that a five-year commitment in the tech sector means that you are committing to deploying technologies out into a particular space, in our case, environmental sustainability, technologies that haven’t even been invented yet! I mean, in some cases, maybe technologies that haven’t even been dreamed of yet! And so, for me, five years just is really something so far off in the horizon that it’s kind of a statement of saying, as long as we can imagine doing tech, we imagine doing tech for nature, right? The fifty-million-dollar part is important of course, but it’s way more than that because of all the other resources that come across Microsoft. And what was really interesting is everyone talks about the fifty-million-dollar, five-year commitment, but nobody talks about how we kind of launched it, with a much smaller commitment. We launched it with a two-million-dollar commitment about five months prior to the main announcement. And this is what I love about the tech sector in general, Microsoft in particular: we launched it with a two-million-dollar investment just to kind of see what the demand was. The demand was overwhelming.

Host: Right.

Lucas Joppa: And we were able to scale to meet it as quickly as we saw it. I think that for me just kind of blew my mind about the power of a large organization. You can put something out in the market, you can see market response, market demand and you can just immediately rise up and meet it. I’d been in Microsoft Research leading research programs at the intersection of environmental and computer science. That’s another thing. We’d been investing in this space, in the AI for Earth space, from a pure research perspective, for almost a decade, and it was really when Microsoft, from a corporate level, really started to go all-in in AI that there was a conversation about, just like every technology that we incubate inside Microsoft Research, there’s a question about the human tech transfer, for me, of saying look, we’ve been investing in this for ten years, isn’t now the time? If not now, then when? And so I put together a memo called AI for Earth which is how we could take this out of research, take this out of incubation, deploy it across the entire company, and then allow the full kind of scope and scale of a Microsoft to put that in the hands of partner organizations all around the world. We went from nothing to something like two hundred and fifty grantees working in all seven continents, sixty-plus countries around the world, thirty-eight or so states in the United States. For me, it’s kind of the program equivalent of seeing some feature that you developed, deployed in software that’s used by a billion people, right?

Host: Right.

Lucas Joppa: You know, just to see that incredible growth and that’s because of the thirty-five years of investment that we’ve been putting in this space.

Host: Well, we talked about four areas of focus, and I’d like to switch over and talk about the three, what you call “pillars of support” for the program. Tell us about the pillars of this program. How did you decide on them, why are they important?

Lucas Joppa: So, AI for Earth from its very beginnings, had three pillars and those pillars are super-purposeful. And they were put in place as the result of years of work in this area of recognition of what the difficulties are. And the pillars are simply access, education and co-innovation. And what I mean by that is, from an access perspective, is that most organizations that work on environmental topics aren’t the large enterprise customers of the world. They are the small, scrappy, non-profits, the chronically underfunded government agencies, the academics, the small kind of social goods start-ups. And these are organizations where resources are incredibly tight. And they often just don’t have the ability to get out ahead on kind of the digital transformation journey that they would like to because they just don’t have that little bit of excess capital that they would need to crystalize the process. And so, for me, the number one thing going into AI for Earth was just this recognition that budget simply cannot be a barrier to people using our tech for environmental sustainability. And so, the significant aspect of our budget, of that fifty-million dollars, is just to ensure that budget isn’t a barrier, that anybody who is taking kind of a machine learning- or AI-first approach to solving environmental sustainability challenges in the four areas of ag, water, biodiversity or climate, we want to get our tech in their hands. That’s great, but it ignores the fact that also, while resources are short, most of those places doing the best work, their employees didn’t graduate from the world’s leading computer science departments. Their employees graduated from the world’s leading environmental science departments. Often, some of our best tech still requires a minor in computer science to use fully effectively, you know. And so we immediately recognized that just putting tech in people’s hands that don’t know how to use it is kind of a fool’s errand. You’ve got to ensure that you follow that up with educational curricula and community building. And so, we started putting together things like AI for Earth education summits, bringing grantees from all over the world together, both in-person, and digitally. We’ve got some fantastic top technical talent on the team that also leads that education effort. And so, engineering isn’t absent from education. And then the last bit is just simply a recognition of reality, which is that the tech sector has a lot of the tech talent. And that human skill and capacity, we need to be able to put that into play as well. The thing is, we know a lot about tech and a little bit about the environment. Our partners know a whole lot about the environment and a little bit about tech. And that intersection there is where innovation actually happens, right? It’s not just collaboration. Collaboration is just like, hey, me helping you get something done or vice versa, right? But this idea that there’s something that comes out of it that’s greater than the sum of its parts, that’s what innovation for me means. And so, we looked across, and those were the three pillars. Obviously, always more to do, but…

Host: AI for Earth, writ large, has several specific projects within that really flush out how this impacts the real world. Tell us about these. It’s a lot to cover, but maybe a brief overview of each would just give our listeners sort of the basis for where you are heading with this. Silvia Terra is a big one.

Lucas Joppa: Yes. Silvia Terra is a fantastic organization. Silvia Terra is actually the name of the company, not just the project. And what they’re really looking to do is to provide a species level tree-count product across the entire United States. And this is super important because, when you think about climate change mitigation or land conservation, you have to know what is where, how much is there, and how fast is it changing. And we fundamentally don’t know that information about earth’s natural resources or the natural resources of the United States. Silvia Terra is using high-resolution satellite and aerial imagery and convolutional deep neural networks to be able to build up that data product from looking down, train up models that are able to assess not just, is there a tree there or not, but what species of trees that is, how big that tree is, what its carbon potential, etc., etc., that really is allowing people to have kind of an unprecedented view into the state of their local forest. And Silvia Terra’s work on single tree species counting and identification is just kind of a subset of a larger problem that some of my research colleagues and I here in MSR have been tackling over the past couple of years, which is trying to build a land cover product. So, not just where are the trees, but where are the forests, the fields, the built-in urban environments, the creeks and the streams and the lakes? All of that, so that you can look down and not just say, oh, that pixel is red or green or blue or it looks like a tree, but we can bring all that together for the United States at a one-meter resolution. That’s something like ten trillion pixels that you need to count up. And it’s a really hard computer science problem as well. And so, building up things like a forest map of species, a national land cover map, all of these sorts of things start to get you towards that kind of “queryable earth” that I think we’re all interested in. If you look at you know, the way we want to answer a lot of questions, it’s kind of from looking down at the world and satellite imagery and the size of these data sets, you know we’re talking about just petabytes and petabytes and petabytes of data flowing down. And so, you’ve got to store those data, you got to be able to get them into memory really quick. You know, you’ve got to be able to train up algorithms really efficiently and then you’ve got to be able to do evaluation or deploy those algorithms over even larger amounts of data. It’s a super non-trivial task getting those data labeled. I mean, it’s just a problem in and of itself. And so, you know, when people ask why environment? I mean, even if you didn’t care about an environment, from a computer science perspective, these are some of the hardest problems out there.

Host: Talk a little bit… give a nod to Farm Beats and Project Premonition because those are really two cool things that are going on.

Lucas Joppa: I’ve watched those projects, Farm Beats and Project Premonition, grow from ideas to what they are right now. I mean, Farm Beats… an incredible sensor and data-fusion project that’s really looking to revolutionize the way that farmers collect data about their field and make harvesting and growing decisions based on extremely new advances in artificial intelligence, bringing that all together. I mean, it’s kind of crazy to see the results of some of these projects because ultimately, it kind of distills down to some graphs on a browser dashboard.

Host: Right.

Lucas Joppa: And what that hides is just the incredible amount of engineering, of actual physical electronic sensors in the field, communicating over really novel communication pathways like TV white space which Ranveer is such an incredible kind of advocate and proponent for. Bringing that into centralized, on-prem compute centers, figuring out what matters, what needs bigger compute. Doing all that kind of local intelligence processing. Sending up information and data sets to our larger compute clusters in the cloud. And ultimately, just allowing somebody to come, click, and see what’s going on out in their field just from, you know, balloons floating in the air, drones flying around and sensors out there, it’s kind of incredible. And Ethan, another fantastic researcher in Microsoft Research, he’s really interested in how we monitor the biosphere, as he would say. And his kind of path into it is through understanding that we have a scale problem. He originally started out wanting to understand diseases and how we can prevent epidemics with this perspective that epidemics are so catastrophic because we don’t see them coming.

Host: Right.

Lucas Joppa: And they often come from our unbuilt natural environments. And they’re housed by the rest of life on earth and so if you want to go out and sample that, well you need to take a technology approach. But still, tech isn’t going to allow you to sample every animal on the planet everywhere. But he had the same simple idea that many have had before him which is that evolution has already provided these data samplers in the form of biting insects. Mosquitos and their kind of devilish ilk, right? And so, the question is well, how do you insert yourself in that data sampling process? If you are a metagenomicist, you see mosquitos and the blood samples they collect as simply roaming data collectors. So, now you need to figure out how to collect those data. And the engineering that Ethan and his team have put together in building these entirely new, next-generation mosquito traps that are running machine-learning algorithms on-board, that can do species-level recognition from wing beat frequency patterns, decide which species they want to collect, and then pipe those into a full metagenomics compute engine that, again, spits out some graphs and information about what diseases and other species might be out in that environment, you know, the scope and the scale, it’s… you know, as somebody who speaks about these a lot, one of the things I really struggle to get across… it seems just this horrible disservice to this entire legacy of research and innovation that goes into providing something so simple and so powerful, that people just don’t see everything that goes behind it, right?

Host: Right.

Lucas Joppa: And I think that’s kind of one of the cool things about the digital transformation that the world’s undergoing is, all of the incredibly useful applications that we have in our lives that make our lives lighter, but are powered by things that most people will never be capable of truly kind of grokking, you know, the scope and the scale of work that goes into this stuff.

Host: I love that you said the word grok. It’s one of my favorite words. It’s such a sci-fi word.

Lucas Joppa: That’s right.

Host: I think we get why the “earth” part of this program is important. But let’s talk a bit more about the AI part. What does AI do for the earth that we can’t? I think we framed this in a previous conversation as an “efficiency” story but agreed that that doesn’t quite do it justice.

Lucas Joppa: Yeah, I mean I think the way I look at it is like ultimately, we’ve got this massive scale problem on the size of the earth, the amount of living organisms on earth, the complexity of their relationships… So, there’s this massive scale problem, how do we scale? And then we have a resource problem, which is that the organizations that focus on this, for better or worse, aren’t necessarily the most well-resourced organizations in the world.

Host: Right.

Lucas Joppa: And so, where has tech always had a transformative impact? It’s in the efficiency story. And what is efficiency? Well, efficiency allows you to either do the same for less or do more for the same. And, maybe in the enterprise space when you’re worrying about margins and quarterly profits, you are really interested in doing, at the very least, the same for less, and maybe even more for less. But when you’re interested in understanding the rest of life on earth, when you’re interested in collecting data about natural systems and building models about them, and you’re wildly resource-constrained, the question you are asking yourself every day is, how do I do orders of magnitude more for the same? And technology allows you to do that. One nice story from a partner organization between the AI for Earth program, Microsoft Research and a small, non-profit called The Chesapeake Bay Conservancy, this is that land cover mapping…

Host: Right.

Lucas Joppa: …project. And, you know, we had this ambition to build a one-meter resolution land cover map for the entire United States. That’s plowing through something like ten trillion pixels, twenty-five, thirty terabytes of data, and that was an aspiration shared with this small, nineteen-person nonprofit called Chesapeake Bay Conservancy. And they were way out ahead of us mind you, because they actually had done stuff, and we just thought about it. And what they had done was they had built a one-meter resolution land cover map for the Chesapeake Bay Watershed, which is just a small fraction of the United States. But it had taken them well over a million dollars and over a year and a half just to produce that one data product. And so that meant that, by the time they got done, they were almost already out-of-date, right? And so, we came in and we said, hey, look, you know, the way you are doing that, we think that there’s a better way. It turned out to be really hard. It’s led to some top papers in, you know, leading conferences like CDPR and ICLR and you know, we’ve made some significant machine learning progress as well. But ultimately, we still have a long way to go on our algorithmic kind of accuracy side of things. But from just a pure infrastructure perspective, we’re able to train-up an algorithm and deploy it on a cluster of eight hundred FPGAs, so field-programmable gate arrays, which is some really cool architecture that we’ve been shipping all across Azure, and plow through all ten trillion pixels. So, go from the Chesapeake Bay to the entire United States, twenty-five terabytes of data, ten trillion pixels, ten minutes, forty-two dollars. So, that gets to, how do I do more for the same, or in this case, way more for way less? And so, that’s just the efficiency story. How do you give an individual or a small organization superpowers? You either give them a huge check or you give them technology.

(music plays)

Host: Several researchers on the pod have talked about this idea of humans in the loop. And it kind of speaks to the fact that technology isn’t taking us over, it still needs us, if we personify it, but you turned that on its head a little bit. Talk to me about your take on who’s in the loop and how and why.

Lucas Joppa: Yeah, I mean, I think what’s kind of funny is, you know, researchers are always so far out ahead that they started talking about humans in the loop probably two decades before it was relevant. They were talking about humans in the loop when we were still trying to figure out how to put algorithms in the loop, right? Where we were doing most of it, but algorithms were just kind of helping. I think, where a lot of the worry is coming, from an AI and ethics perspective or anything else, is around what the future might hold when we need to purposefully build humans into the loop so that we have these spot checks and balances and things like that. So, I still think we completely live in a world where it’s algorithms in the loop of human activity, algorithms in the loop of human life. That’s the world I want to live in. I don’t want to live in a world where it’s humans in the loop as if I’m some sort of secondary citizen to a…

Host: Addendum.

Lucas Joppa: …yeah, addendum to this digital substrate that I’m just you know allowed to exist in. You know, it’s kind of a corollary to something else that just frustrates me all the time which is that so many of the things that we don’t know about are things that we ourselves have built. So, ask me where all the world’s dams are. I don’t know. Ask me where all the world’s roads are. We built these things and we don’t even know where they are, or the state of them. And so, when you get back to this whole human in the loop concept, I fundamentally have a problem with imagining a future where I’m an addendum to something that we built. And just to prove how far-off I think we are from that is that, the things we built, as I said, like, are kind of addendums in our lives still, and I think we’d be much better focused in figuring out how to put algorithms in the loop to help us understand what we’ve built, the impact that it’s having on environmental and social systems and to build our societies around that. And that’s going to be really important as we start to ask algorithms what we should do about managing our natural systems. Where should we put renewable versus non-renewable energy? Where should we put high-density urban growth centers versus protected areas and national parks? Where should we do, you know, any number of things? Well, we don’t know enough about our own biology to tell an algorithm how we’d make that decision…

Host: Right.

Lucas Joppa: …and we don’t know enough about algorithms to understand fully why they’d make that decision. So, I think there just needs to be this awareness of, like, you know what, once the output comes out, if it feels right to me, if it looks good to me, then I’m going to continue to interrogate it with human systems and I’m going to take it one step at a time. But that’s just my opinion.

Host: When I asked you what gets you up in the morning, you basically said it was the wonder of it all, the wonder of discovery. But we’ve hit the “worry” part of the podcast. This is where I ask the guests what keeps you up at night? And aside from all the concerns about conservation and stewardship of our planet, you’re developing and deploying some incredibly powerful technologies in which we’re putting a lot of trust.

Lucas Joppa: Hmmm.

Host: What could possibly go wrong?

Lucas Joppa: Hmmm. Well, I mean, what keeps me up at night is both what could possibly go wrong, and that we won’t do it fast enough! My former manager and long-time mentor, Eric Horvitz, who leads Microsoft Research, you know, he has a long history in the health space at the intersection of health and medicine and machine learning. And one of the things that he consistently says is, people worry about the issues of algorithms in these loops and that’s completely valid and understandable, but what’s the human cost of not doing it? I think that same thing is true in environmental space, which is just, the opportunities are so incredible to help better understand and manage earth’s natural systems that we have got to be throwing basically everything that we have at it. So, my first worry is that we won’t. My first worry is that all the other tech companies won’t follow Microsoft’s lead and build up an AI for Earth program. That all of the environmental scientists and organizations won’t see tech as one of the significant breakthrough solutions in their space. And that we’ll stagnate. We won’t make the progress that we all know we need to make. So, that’s the first thing that keeps me up at night. There’s that secondary worry which is, what could go wrong? Well, I mean if history shows you anything about human activities, it’s that a whole lot could go wrong. You know, I worry more about our lack of understanding of how nature works and doing something that inadvertently has a bit of a Schrodinger’s Cat kind of effect in the sense of like are you observing what would have been observed in the absence of observation, right? I don’t actually know if that’s what Schrodinger said or not, but it sounded good…

Host: Something about a dead cat…

Lucas Joppa: Yeah, something about a dead…

Host: Or a not-dead cat.

Lucas Joppa: Yeah, something about two cats. And a box. So that’s really kind of one of the first design principles for everything that we do is, in the absence of full understanding, do you feel like you are really taking that like do no harm kind of approach? And that’s one of the reasons that we’re really interested in these kind of aspects of remote monitoring, whether that’s like taking images from above or, you know, passively listening to ecosystems with acoustic sensors, or monitoring environmental systems through the lens of an organism like a mosquito that many places are trying to eradicate anyway. And really thinking about, are we deploying technology in a way that isn’t going to further disrupt the system because the reason we’re putting technology in the system is to figure out how to fix it and stop the problems. And so, those are my two worries. You know, I work a lot, so the time between worrying, falling asleep, and wondering and waking up and getting going again is bit too short, so sometimes I feel like those two Ws go hand-in-hand.

Host: So, I like to hear the stories of the guests that come on this show, personal, professional, academic, how you ended up doing what you’re doing. How did you get started and how did you end up doing what you are doing now?

Lucas Joppa: You know, I grew up a kid that just loved being out in the woods, never was a technology kind of family. I’ve never had a TV. Didn’t have a computer until undergrad. Never learned how to program until my PhD when I realized that I probably should. But I was always just interested in how it kind of all worked. And then I started paying more attention to the fact that, in many cases, it wasn’t working, i.e., humans were negatively influencing these systems. I did my undergrad in wildlife ecology. One of the beauties of the American education system is the general education requirements. I took a general education requirement called On the Extinction of Species. I sat through that first class. I listened to this professor talk about the state of the world and I was just completely and totally hooked. Changed my major. Spent two years in the Peace Corps. Saw some of these environmental issues just kind of raw, up front, in person. Came back. Did my PhD in ecology and environmental science. That’s when I really leaned into tech, because I realized that there was no way I was going to answer the questions that I wanted to ask like, do protected areas protect? Well, there’s a lot of protected areas, all around the world. You want to ask that across the entire earth? Across every year? You’re gonna be running code to do those sorts of things. And just through kind of serendipity and contacts, ended up at Microsoft Research in Cambridge, in the UK. I had an incredible time there. And now I have this opportunity to oversee Microsoft’s global sustainability ambitions. Everything from our built environments to our technology deployment. It’s kind of a crazy thing look back and see.

Host: From a kid in the woods, to where you sit now. Among the trees here in Redmond. All right Lucas, as I said at the beginning of the podcast, you’re the Chief Environmental Officer at Microsoft. Here’s your chance to say whatever you want to our listeners. I often frame this in terms of parting thoughts, wisdom, advice, inspiration, but you can say anything you want. So, go.

Lucas Joppa: I get asked this question a lot and the thing that I always say is, don’t take the easy path which is, what do you need to succeed? And it’s easy to say, oh, I need money! Or I need resources! But when I’m talking to an audience like this, an audience of machine learning experts, computer science professionals and researchers all around the world, the message I try to consistently deliver is, all I really need is for every single person, at some point in the day to think about how they could deploy their core competency. To just think about, hey, you know what? I’m a homomorphic encryption guy. Or I’m a deep neural nets person. Or I’m a UIUX person. That’s what I’m great at! Now, how can I deploy that to help make progress in one of these massive areas of societal challenge? How can I think about accelerating human progress in the areas of ag, water, biodiversity and climate change? If you just think about that, even if you don’t have an idea today or tomorrow, if you just think about that every day, then I’m satisfied. Then we’ve made an incredible start on the problem. I don’t need every single person to pick up an identical brick. I need every person to commit to doing what they’re good at. I think so many people look, and they say, on environmental sustainability, oh, what we need is better policy. And you go over to the public sector and what they say, what we need are technology solutions. And the truth is we need all of that. And we need the people who are best at each of those things to do what they’re best at. The Microsofts of the world need to contribute tech. The public sector needs to contribute policy. And, you know, double-click all the way down in the tech sector to whatever kind of role that you have in a company like Microsoft. That’s what the world needs. The world needs what you’re best at, deployed to help solve some of these environmental challenges.

Host: Lucas Joppa, thank you for coming on the podcast.

Lucas Joppa: Yeah, thank you so much for having me. It’s a huge pleasure and a great opportunity.

(music plays)

To learn more about Dr. Lucas Joppa and how Microsoft is putting its powerful Cloud and AI tools into the hands of those working to solve environmental challenges, visit Microsoft.com/research

By Marianne Phillips

From the melting glaciers in the Arctic Circle to algae blooms in the previously pristine Lake Atitlan to ecosystem change around Murchison Falls to penguin tracking in Antarctica, the need for greater knowledge about the world around us and how it’s changing is both apparent and acute. Today, in partnership with the National Geographic Society, we’re excited to announce the winners in the Microsoft and National Geographic Society AI for Earth Innovation Grant program, announced in July.

These 11 changemakers will, with the help of artificial intelligence (AI), uncover new insights to better monitor, model and ultimately manage the world’s natural resources more effectively and sustainably. These grantees, like our 200-plus AI for Earth grantees globally, are testing innovative applications of AI in scientific exploration and research to help solve critical environmental challenges. We introduce you to these innovative leaders and their inspiring work below, in four categories — climate change, biodiversity conservation, agriculture and water.

The data is clear — climate change is real and demands a global response. But what actions will be most effective? And how can progress be measured? These two grantees are helping answer these questions with AI.

Gaining a better understanding of Earth’s melting glaciers

Melting glaciers provide some of the clearest visual evidence of the Earth’s warming temperatures. Unfortunately, collecting data and measuring these changes can only be done crudely, making it tricky to measure glacier surface change and melt rates, never mind project those changes into the future.

This is a frustration Joseph Cook knows all too well. In the last decade, he has conducted more than 12 Arctic field expeditions, mostly in field camps on the Greenland Ice Sheet, and pumped out findings and research insights in scientific journals, documentaries and conferences around the globe.

These experiences led Cook to machine learning, specifically applying it to optical data from drones and satellites to explore the changing cryosphere. The key is capturing the complexity of ice surface evolution, from pristine snow to dust-covered, algae-covered and waterlogged ice. By training the algorithms to recognize how the different surfaces reflect certain wavelengths of light — wavelengths that can be measured by satellites as well as by drones — precision study of vast areas becomes feasible. After testing the algorithms on imagery from custom-built drones, Cook’s team will then apply them to satellite remote-sensing data, enabling them to scale up to entire glaciers.

Predicting climate-related migration in Africa

People around the world are already adapting their behavior to deal with a changing climate, including changing their address. These actions can provide helpful clues as to how certain geographies are experiencing climate change and how people will respond. But predicting when, where and how climate change may induce mass migration of people is incredibly difficult because historical patterns in many countries are undocumented.

Solomon Hsiang helped break new ground with studies showing that climate change both increases conflict at all levels of society and correlates to dramatic drops in productivity. Now, Hsiang’s team is applying AI to see how climate changes affected migrations across Africa in the past, with an eye to the future.

The team is digitizing 1.6 million aerial photographs from surveys of 18 African countries taken since 1943. By applying machine learning to the scanned images, the team is reconstructing a chronicle of population density, urban extents and land use across Africa over time. With statistical tools, economic theory and climate model projections, they can use that chronicle to estimate migration risk across Africa in the future. By understanding who’s at risk, where and why, society — particularly governments — will finally have information that can change the present and adapt for the future.

The latest reports indicate the situation is dire for life on Earth. Considering the small fraction of species that have been classified out of the estimated millions of species on the planet, the situation is likely far worse. Time is short, resources are thin and the data exists on a planetary level. Sounds like a job for changemakers armed with AI, like these four grantees working on biodiversity projects.

Tracking Antarctic penguin populations

Antarctica and its surrounding oceans are a remote and pristine wilderness under threat from the impacts of climate change, with melting ice and warming temperatures changing the habitat and feeding grounds for local wildlife populations — including penguins. Penguin populations are tricky to track. Their remote habitat presents difficulties for both collecting and processing data, and identifying their colonies often means looking for hard-to-find markers, like guano stains against a similarly colored rock.

Heather Lynch, a quantitative ecologist and an associate professor at Stony Brook University, thinks that AI can be used to improve, if not solve, both data challenges. Specifically, she plans to use computer vision to find guano stains in satellite images. This will help inform and develop classification algorithms to generate population estimates for penguin colonies.

“With this penguin project, we have one of the first real examples where we can automatically get and process satellite imagery, generate population estimates and deliver that information through decision support tools — making it as readily available as a weather forecast,” explains Lynch. Her hope? A better forecast for penguins once conservationists and policymakers have data easily and continuously available.

Identifying bird songs in acoustic field recordings

To protect threatened species, scientists need to gather a lot of data, often more than can be captured by direct field observers. Indirect observation methods — microphones, camera traps and other autonomous sensors — are indispensable tools in closing this gap. But these transformative tools come with a new challenge — more is only more if you can parse through it all to discern some insights.

Bird songs are particularly hard to classify due to their complexity. But bird populations are important to monitor because they provide an early warning of unhealthy environments. Justin Kitzes, an assistant professor at the University of Pittsburgh’s Department of Biological Sciences, is pioneering AI-assisted survey and classification of bird songs to better understand bird population changes.

His OpenSoundscape software will provide acoustic analysis on a laptop, cloud service or supercomputer, along with a trained classifier library for around 600 bird species found in the United States. The data collection enabled by this work will help to form the foundation for effective conservation of birds and other vocal species on a changing planet.

Monitoring insect sounds in tropical rainforests

While the benefits of rainforests are well known — vast biodiversity, as well as carbon storage and oxygen production — a detailed understanding of species and ecosystems contained within is still largely out of reach, as high canopies and dense vegetation make it hard to observe wildlife. Holger Klinck and Laurel Symes of Cornell University’s Bioacoustics Research Program are looking to AI-powered acoustic monitoring of insects as a way of better understanding the dynamics of rainforest habitats.

Insects typically produce a single mating call that varies little within a species, simplifying identification and classification. They also rarely travel far from where they’re born, making them a good indicator of the health of their local environment.

Klinck’s team is focusing first on neotropical rainforest katydids, a diverse species that occupies a central position in tropical food webs. How the wide variety of katydids interacts with the rest of the forest species, both plants and animals, can provide lots of information about the overall ecosystem. Klinck aims to scale beyond insects to other species, including birds, monkeys and other vocal animals, to help advance conservation of tropical rainforests.

Capturing and sharing data on lion populations

Effective animal conservation requires knowing what you have and where. But some animals travel widely, ignoring artificial boundaries such as national borders and conservation reserves. Those borders constrain researchers for travel and data sharing, as well as sheer resources, which makes it challenging to get accurate population numbers. Are they counting different animals in each area, or the same ones ranging over a wider area? What happens if the herd migrates across a border in the middle of the field work? Answering these questions is even harder when the animals have few distinguishing features — lions all look nearly the same even to highly trained researchers.

Making that easier is the task of the Lion Identification Network of Collaborators (LINC), the brainchild of biologist Stephanie Dolrenry and systems engineer Justin Downs. It brings together lion research through a custom web application that combines a collaborative database, innovative AI search capabilities and social media tools. LINC creates a platform for interaction and data sharing among conservation efforts, citizen scientists and government institutions, to help shape and inform conservation policy.

Currently, the LINC team is further developing AI techniques to identify individual lions. One technique maps the lion’s facial features with more precision than humans can see; the other can identify the whisker patterns, which like fingerprints are unique to each lion. Better identification will enable more coordinated research and conservation efforts.

Farmland around the world is ground zero for one of the greatest challenges of our time — feeding 3 billion more people than today, without many acres of new arable land, while using less water, dealing with more pests and managing unpredictable growing seasons. Solving this is possible, with a lot of human ingenuity and new technologies already being tested by our three grantees.

Improving irrigation efficiency in Uganda

To get more yield with fewer resources, precision is key — particularly with a precious resource like water. One way to know how much water crops really need is by measuring how much water evaporates from the soil and plant surfaces — the evapotranspiration (or ET) rate. Without that, farmers are most likely watering inefficiently, but it’s hard to measure.

Torsten Bondo and Radoslaw Guzinski are exploring how to generate field-level ET measurements using machine learning and satellite imagery. They’re developing an open-source algorithm that can merge data from optical and thermal satellites as well as meteorological data to determine the right amount of water for effective irrigation. The system will be tested on a large national irrigation project in Uganda, where water is becoming scarce because of a growing population and climate change.

The team predicts that this technique could reduce water use by up to 30 percent for the same yield within the district. They hope the technique will pave the way for better irrigation practices in other drought-prone countries around the globe — helping save water and providing food for generations to come.

Mapping agricultural land and water use

As climate change disrupts weather patterns, rainfall is becoming more unreliable. Farmers are drilling more wells for center-pivot irrigation — a method where crops are watered with sprinklers rotating around a central source. However, this approach can lead to lowered or even drained water tables, salination of coastal aquifers, land subsidence and disruption to ecosystems.

Kelly Caylor, a professor of ecohydrology at the University of California, Santa Barbara, is investigating how much water is being used from these groundwater sources. He is developing a web tool that uses machine learning to identify active crop fields in satellite imagery and geospatial analysis tools to monitor how crops change over time. Knowing where the crops are growing and for how long, and then correlating that to weather data, the system can also infer how much water is being used.

With a better understanding of how much groundwater is used by center-pivot irrigation will come opportunities to develop more optimal and efficient practices, as well as policies for better water stewardship. With the online map and tools, farmers, water resource managers, policymakers and the public will be better able to make agriculture more land and water efficient.

Detecting land cover changes in Uganda

Uganda’s Murchison Falls National Park is a safe home for iconic species like elephants, giraffes, hippos and chimpanzees. Nearby Lake Albert and the surrounding area support fishing and agriculture and are rich in other natural resources, like oil, that also provide income to the country. Managing natural resources, especially when they are potentially in competition, is essentially impossible to do without knowing what changes are happening to the land. Around Murchison Falls, land cover mapping was last done more than a decade ago.

Ketty Adoch, a geographical information systems specialist, is working to address this. A huge advocate for applying technology tools to geographic data, she will use supervised classification and machine learning on satellite images to detect changes in the shape or size of land cover types, like shadows from changing tree cover. She’ll conduct these analyses both historically and, once promising algorithms have been developed, on an ongoing basis for the coming decade.

The key outcomes — algorithms and maps to document the findings — will enable researchers, conservationists and technologists to monitor land cover change in the area, see the impact of oil activities and support conservation efforts going forward.

Freshwater makes up less than 2 percent of the water on the planet, and just 1 percent of that is readily accessible. That’s a lot of pressure on an incredibly important and very finite resource. Two of our grantees are leading the way on new techniques to measure, monitor and manage this precious resource more effectively.

Providing early warning of harmful algal bloom outbreaks

For many years, the waters of Lake Atitlán in the Guatemalan highlands were pristine, a landmark for natural beauty and biodiversity. However, in 2009 the lake experienced the first of several harmful algal blooms (HABs) — out-of-control colonies of algae that suck oxygen out of the water and make it potentially toxic to life.

Africa Flores describes that first HAB in Lake Atitlán as a wakeup call for action to preserve its precious biodiversity. But Guatemala has limited resources and means to investigate and better understand the causes and help predict and prevent future outbreaks. Thankfully, Flores’ work as a research scientist at the Earth System Science Center at the University of Alabama in Huntsville allows her to focus on this very issue.

Flores and her team will conduct deep analyses on image datasets from different satellites. Machine learning will help them to identify the variables that could predict future algal blooms. Knowledge on what those triggers are can turn into precise preventative action, not just in the lake in Flores’ home country but also in other freshwater bodies with similar conditions in Central and South America.

Detecting and mapping small dams and reservoirs

Millions of dams and reservoirs around the world deliver drinking water and hydropower generation, but if not built and managed thoughtfully, they can pose risks to the environment. And surprisingly few are mapped systematically, which makes it essentially impossible to understand how these interact and affect one another and the ecosystems around them.

At the Natural Capital Project, a small team — Gretchen Daily, Lisa Mandle, Richard Sharp and Charlotte Weil — is out to change the status quo. The team is combining remote sensing data with machine learning to develop a model that can detect smaller dams and reservoirs. To do this, they will locate thousands of already-mapped dams on high-resolution satellite images using the dams’ coordinates and then use machine learning to project where other dams are located based on shared characteristics in the satellite imagery.

The result will be an open-source algorithm for detecting dams and reservoirs, as well as sample results — all made freely available to the broader conservation and sustainable development community. Knowing where the dams and reservoirs are located will contribute to mitigating their impact, to conserving and managing hydrological ecosystem services, and to planning development more sustainably.

To learn more about the Microsoft and National Geographic Society partnership, visit  https://www.nationalgeographic.org/grants/grant-opportunities/ai-earth-innovation/.

To learn more about AI for Earth, visit https://www.microsoft.com/en-us/aiforearth/.

Tags: AI for Earth, climate change, Environment