12-01-2018, 09:10 PM
Fishy business: Putting AI to work in Australia’s Darwin Harbour
<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/12/fishy-business-putting-ai-to-work-in-australias-darwin-harbour.jpg" width="1920" height="1080" title="" alt="" /></div><div><header class="f-hero "> </header>
<p><!-- .f-hero --></p>
<div class="features-body">
<section class="f-content-section f-content-entry m-rich-content-block">
<p><h2>Identifying and counting fish species in murky water filled with deadly predators is a difficult job. But fisheries scientists in the Northern Territory are working on an artificial intelligence project with Microsoft that has incredible potential for marine science around the world.</h2>
</p>
</section>
<section class="f-content-section f-content-entry"> </section>
<section class="f-content-section f-content-entry m-rich-content-block">
<div>
<p>Your mission should you choose to accept it, is to go into one of Australia’s largest harbours and count the fish. Think this sounds daunting? You don’t know the half of it.</p>
<p>First, there’s the water. There’s a lot of it in Darwin Harbour – five times more than Sydney Harbour, to be precise. Heavy tides swell more than seven metres then retract, leaving little visibility in their wake.</p>
<p>And if you think you’ve got some occupational hazards at work, try getting your job done in an environment teeming with some of the world’s most intimidating apex predators – saltwater crocodiles, along with tiger, bull and hammerhead sharks. More than 300 salties are caught in the harbour each year.</p>
<p>This is the daunting task of the Department of Primary Industry and Resources for the Northern Territory Government, as it goes about ensuring fisheries resources are sustainably managed and developed for future generations.</p>
</p></div>
</section>
<figure class="f-figure-clean"><img width="1920" height="1080" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/12/fishy-business-putting-ai-to-work-in-australias-darwin-harbour.jpg" class="animate-viewport c-image" alt="Identifying and counting fish species in murky water filled with deadly predators makes diving to count fish species impossible." /></p>
<p><figcaption class="f-figcaption c-caption-1">Murky water filled with deadly predators like the saltwater crocodile make diving to count and identify fish species impossible.</figcaption></p>
</figure>
<figure class="f-figure f-figure-alt ">
<div class="f-content-section-wide f-content-entry">
<blockquote class="f-blockquote">
<p>“If you’re in the water with a crocodile you aren’t taking a calculated risk. You’re going to be a statistic. That’s it. If you’re in the water and he’s there, he wants you and you’re gone.” – <strong>Wayne Baldwin, Research Technical Officer, NT Fisheries</strong></p>
</blockquote></div>
</figure>
<section class="f-content-section f-content-entry m-rich-content-block">
<div>
<p>If shooting fish in a barrel is a metaphor for something all-too-easy, the correct metaphor for something exceptionally challenging might be counting fish in Darwin Harbour. Yet the NT Fisheries team, led by Dr Shane Penny, Fisheries Research Scientist, do it every day. As the old saying goes, you can’t manage what you can’t measure, so their work begins with knowing how many fish there are.</p>
<p>But they were bogged down by the time it took to wade through hours of underwater footage. The team needed to assess the abundance of critical fish species faster and more accurately, while maintaining a safe distance from deadly predators.</p>
<h3>A meeting of the minds</h3>
<p>It was from these murky depths that an innovative project showed the potential for artificial intelligence (AI) to support the important work being done by this team of marine biologists. Amid rising debate about the potential impact of AI on society, a collaboration between these scientists and Microsoft engineers became an opportunity to test out its powers as a force for good. Could technology hold the key to safely, accurately and rapidly counting fish – giving the NT Fisheries team more time to devote to analysing this data and improving the sustainable management of NT fish stocks?</p>
<p>The NT Fisheries team had high hopes. They had been using a baited remote underwater video (BRUV) to help with high-risk data gathering. The camera allows the team to see what’s in the water without going in. But even with BRUV on their side, the task was formidable.</p>
</p></div>
</section>
<figure class="f-figure-clean"><img width="1920" height="1080" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/12/fishy-business-putting-ai-to-work-in-australias-darwin-harbour-1.jpg" class="animate-viewport c-image" alt="Using a GoPro, researchers at NT Fisheries begin the process of assessing critical fish species." /></p>
<p><figcaption class="f-figcaption c-caption-1">Shane Penny, Fisheries Research Scientist and his team using baited underwater cameras.</figcaption></p>
</figure>
<figure class="f-figure f-figure-alt ">
<div class="f-content-section-wide f-content-entry">
<blockquote class="f-blockquote">
<p>“We’ve had quite a few problems with sharks coming in and taking the baits away. Tawny sharks have learned how to open our baits and suck it all out before we have a chance to collect any video.”<br />– <strong>Wayne Baldwin, Research Technical Officer, NT Fisheries</strong></p>
</blockquote></div>
</figure>
<section class="f-content-section f-content-entry m-rich-content-block">
<div>
<p>Then there was the sheer quantity of work involved. Once the video is collected, terabytes of footage must be viewed, and its content scoured and quantified. To put this in perspective, a single terabyte would store 500-hours of your favourite movies. The team was identifying vast quantities of different fish species and tracking their behaviour. This diversity and the murkiness of the water meant classification was often far from simple.</p>
<p>Steve van Bodegraven, a Microsoft machine learning engineer and Darwin local, worked with the NT Fisheries team over several months to see whether computer vision would be up to the ambitious task of identifying fish in underwater images.</p>
<p>In a similar way to how tags are suggested for friends and relatives in the photos you upload to social media – through repeated exposure and the discovery of patterns – the project’s success depended on feeding the system with training images. Along the way they had to confront an array of unusual problems. For example, how would Microsoft’s AI solution respond to fish like gold-spotted cod that can change colour to blend into their environment?</p>
<p>“We went in and talked to them about how they work and the challenges they face,” van Bodegraven says. “From that we tried to figure out how we could help. Everything we do is explorative, so we don’t necessarily have solutions out of the box.”</p>
<p>Three months and thousands of images later, results are encouraging to the scientists. To date the system is showing great potential, having learnt to identify 15 different species, from black jewfish to golden snapper which are under careful management to rebuild breeding stocks.</p>
</p></div>
</section>
<figure class="f-figure-clean"><img width="960" height="640" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/12/fishy-business-putting-ai-to-work-in-australias-darwin-harbour.gif" class="animate-viewport c-image" alt="fisheries gif" /></p>
<p><figcaption class="f-figcaption c-caption-1">The AI solution automates the laborious process of counting local fish stocks by progressively learning to identify different varieties of fish.</figcaption></p>
</figure>
<figure class="f-figure f-figure-alt ">
<div class="f-content-section-wide f-content-entry">
<blockquote class="f-blockquote">
<p>“We threw a few test images of fish it’s never seen before and it’s managed to pull those out and differentiate them from the fish it does know about. Once we had that first positive identification of a fish, we really felt we were onto something. From there it was just a matter of finding the right tools to improve and optimise.”<br />– <strong>Dr Shane Penny, Fisheries Research Scientist</strong></p>
</blockquote></div>
</figure>
<section class="f-content-section f-content-entry m-rich-content-block">
<div>
<p>With each new fish analysed, the power of the machine learning technology increases. Samantha Nowland, the team’s Darwin-born research assistant, sees the potential for such systems to change the game in marine management. NT has some of the most pristine waters in the world with healthy populations of endangered species such as sawfish and sharks. The development of this technology and its availability may help other areas of the world to improve their understanding of aquatic resources and ensure they are managed sustainably.</p>
<h3>Beyond the harbour</h3>
<p>While there’s already talk of using the system to create a global database of fish species, the NT Fisheries team is focused on analysing trends, coming up with management plans and expanding its reach.</p>
<p>“It’s going to help us monitor any marine species in Darwin Harbour and around the region,” Penny says. “We have a lot of endangered species and many more where we don’t have enough data. We need research projects that can identify species accurately.”</p>
<p>Microsoft’s van Bodegraven hopes it will open people’s eyes to the transformative potential of AI in fisheries and marine management and beyond. The project has already piqued the interest of fisheries departments across Australia, while the possibility of using the technology to monitor other animal species, like the iconic Kookaburra, is being actively explored.</p>
<p>Microsoft is also exploring how it could support similar projects elsewhere. By making the technology available via open source platform GitHub, the technology giant is encouraging others to build AI solutions that address their unique scenarios.</p>
<p>“Projects like this set a new precedent. Hopefully it will make people curious and give them the confidence to explore the application of AI in their industries,” van Bodegraven says. “It’s going to change industries and societies. The potential is only limited by imagination.”</p>
</p></div>
</section>
<figure class="f-figure-clean"><img width="1920" height="1080" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/12/fishy-business-putting-ai-to-work-in-australias-darwin-harbour-2.jpg" class="animate-viewport c-image" alt="Steve van Bodegraven, Machine Learning Engineer at Microsoft and Dr Shane Penny, Fisheries Research Scientist at NT Fisheries review the identified fish species using the AI solution." /></figure>
</div>
<div class="m-content-placement f-related-stories">
<h2 class="c-heading-5">More Stories</h2>
</p></div>
</div>
<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/12/fishy-business-putting-ai-to-work-in-australias-darwin-harbour.jpg" width="1920" height="1080" title="" alt="" /></div><div><header class="f-hero "> </header>
<p><!-- .f-hero --></p>
<div class="features-body">
<section class="f-content-section f-content-entry m-rich-content-block">
<p><h2>Identifying and counting fish species in murky water filled with deadly predators is a difficult job. But fisheries scientists in the Northern Territory are working on an artificial intelligence project with Microsoft that has incredible potential for marine science around the world.</h2>
</p>
</section>
<section class="f-content-section f-content-entry"> </section>
<section class="f-content-section f-content-entry m-rich-content-block">
<div>
<p>Your mission should you choose to accept it, is to go into one of Australia’s largest harbours and count the fish. Think this sounds daunting? You don’t know the half of it.</p>
<p>First, there’s the water. There’s a lot of it in Darwin Harbour – five times more than Sydney Harbour, to be precise. Heavy tides swell more than seven metres then retract, leaving little visibility in their wake.</p>
<p>And if you think you’ve got some occupational hazards at work, try getting your job done in an environment teeming with some of the world’s most intimidating apex predators – saltwater crocodiles, along with tiger, bull and hammerhead sharks. More than 300 salties are caught in the harbour each year.</p>
<p>This is the daunting task of the Department of Primary Industry and Resources for the Northern Territory Government, as it goes about ensuring fisheries resources are sustainably managed and developed for future generations.</p>
</p></div>
</section>
<figure class="f-figure-clean"><img width="1920" height="1080" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/12/fishy-business-putting-ai-to-work-in-australias-darwin-harbour.jpg" class="animate-viewport c-image" alt="Identifying and counting fish species in murky water filled with deadly predators makes diving to count fish species impossible." /></p>
<p><figcaption class="f-figcaption c-caption-1">Murky water filled with deadly predators like the saltwater crocodile make diving to count and identify fish species impossible.</figcaption></p>
</figure>
<figure class="f-figure f-figure-alt ">
<div class="f-content-section-wide f-content-entry">
<blockquote class="f-blockquote">
<p>“If you’re in the water with a crocodile you aren’t taking a calculated risk. You’re going to be a statistic. That’s it. If you’re in the water and he’s there, he wants you and you’re gone.” – <strong>Wayne Baldwin, Research Technical Officer, NT Fisheries</strong></p>
</blockquote></div>
</figure>
<section class="f-content-section f-content-entry m-rich-content-block">
<div>
<p>If shooting fish in a barrel is a metaphor for something all-too-easy, the correct metaphor for something exceptionally challenging might be counting fish in Darwin Harbour. Yet the NT Fisheries team, led by Dr Shane Penny, Fisheries Research Scientist, do it every day. As the old saying goes, you can’t manage what you can’t measure, so their work begins with knowing how many fish there are.</p>
<p>But they were bogged down by the time it took to wade through hours of underwater footage. The team needed to assess the abundance of critical fish species faster and more accurately, while maintaining a safe distance from deadly predators.</p>
<h3>A meeting of the minds</h3>
<p>It was from these murky depths that an innovative project showed the potential for artificial intelligence (AI) to support the important work being done by this team of marine biologists. Amid rising debate about the potential impact of AI on society, a collaboration between these scientists and Microsoft engineers became an opportunity to test out its powers as a force for good. Could technology hold the key to safely, accurately and rapidly counting fish – giving the NT Fisheries team more time to devote to analysing this data and improving the sustainable management of NT fish stocks?</p>
<p>The NT Fisheries team had high hopes. They had been using a baited remote underwater video (BRUV) to help with high-risk data gathering. The camera allows the team to see what’s in the water without going in. But even with BRUV on their side, the task was formidable.</p>
</p></div>
</section>
<figure class="f-figure-clean"><img width="1920" height="1080" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/12/fishy-business-putting-ai-to-work-in-australias-darwin-harbour-1.jpg" class="animate-viewport c-image" alt="Using a GoPro, researchers at NT Fisheries begin the process of assessing critical fish species." /></p>
<p><figcaption class="f-figcaption c-caption-1">Shane Penny, Fisheries Research Scientist and his team using baited underwater cameras.</figcaption></p>
</figure>
<figure class="f-figure f-figure-alt ">
<div class="f-content-section-wide f-content-entry">
<blockquote class="f-blockquote">
<p>“We’ve had quite a few problems with sharks coming in and taking the baits away. Tawny sharks have learned how to open our baits and suck it all out before we have a chance to collect any video.”<br />– <strong>Wayne Baldwin, Research Technical Officer, NT Fisheries</strong></p>
</blockquote></div>
</figure>
<section class="f-content-section f-content-entry m-rich-content-block">
<div>
<p>Then there was the sheer quantity of work involved. Once the video is collected, terabytes of footage must be viewed, and its content scoured and quantified. To put this in perspective, a single terabyte would store 500-hours of your favourite movies. The team was identifying vast quantities of different fish species and tracking their behaviour. This diversity and the murkiness of the water meant classification was often far from simple.</p>
<p>Steve van Bodegraven, a Microsoft machine learning engineer and Darwin local, worked with the NT Fisheries team over several months to see whether computer vision would be up to the ambitious task of identifying fish in underwater images.</p>
<p>In a similar way to how tags are suggested for friends and relatives in the photos you upload to social media – through repeated exposure and the discovery of patterns – the project’s success depended on feeding the system with training images. Along the way they had to confront an array of unusual problems. For example, how would Microsoft’s AI solution respond to fish like gold-spotted cod that can change colour to blend into their environment?</p>
<p>“We went in and talked to them about how they work and the challenges they face,” van Bodegraven says. “From that we tried to figure out how we could help. Everything we do is explorative, so we don’t necessarily have solutions out of the box.”</p>
<p>Three months and thousands of images later, results are encouraging to the scientists. To date the system is showing great potential, having learnt to identify 15 different species, from black jewfish to golden snapper which are under careful management to rebuild breeding stocks.</p>
</p></div>
</section>
<figure class="f-figure-clean"><img width="960" height="640" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/12/fishy-business-putting-ai-to-work-in-australias-darwin-harbour.gif" class="animate-viewport c-image" alt="fisheries gif" /></p>
<p><figcaption class="f-figcaption c-caption-1">The AI solution automates the laborious process of counting local fish stocks by progressively learning to identify different varieties of fish.</figcaption></p>
</figure>
<figure class="f-figure f-figure-alt ">
<div class="f-content-section-wide f-content-entry">
<blockquote class="f-blockquote">
<p>“We threw a few test images of fish it’s never seen before and it’s managed to pull those out and differentiate them from the fish it does know about. Once we had that first positive identification of a fish, we really felt we were onto something. From there it was just a matter of finding the right tools to improve and optimise.”<br />– <strong>Dr Shane Penny, Fisheries Research Scientist</strong></p>
</blockquote></div>
</figure>
<section class="f-content-section f-content-entry m-rich-content-block">
<div>
<p>With each new fish analysed, the power of the machine learning technology increases. Samantha Nowland, the team’s Darwin-born research assistant, sees the potential for such systems to change the game in marine management. NT has some of the most pristine waters in the world with healthy populations of endangered species such as sawfish and sharks. The development of this technology and its availability may help other areas of the world to improve their understanding of aquatic resources and ensure they are managed sustainably.</p>
<h3>Beyond the harbour</h3>
<p>While there’s already talk of using the system to create a global database of fish species, the NT Fisheries team is focused on analysing trends, coming up with management plans and expanding its reach.</p>
<p>“It’s going to help us monitor any marine species in Darwin Harbour and around the region,” Penny says. “We have a lot of endangered species and many more where we don’t have enough data. We need research projects that can identify species accurately.”</p>
<p>Microsoft’s van Bodegraven hopes it will open people’s eyes to the transformative potential of AI in fisheries and marine management and beyond. The project has already piqued the interest of fisheries departments across Australia, while the possibility of using the technology to monitor other animal species, like the iconic Kookaburra, is being actively explored.</p>
<p>Microsoft is also exploring how it could support similar projects elsewhere. By making the technology available via open source platform GitHub, the technology giant is encouraging others to build AI solutions that address their unique scenarios.</p>
<p>“Projects like this set a new precedent. Hopefully it will make people curious and give them the confidence to explore the application of AI in their industries,” van Bodegraven says. “It’s going to change industries and societies. The potential is only limited by imagination.”</p>
</p></div>
</section>
<figure class="f-figure-clean"><img width="1920" height="1080" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/12/fishy-business-putting-ai-to-work-in-australias-darwin-harbour-2.jpg" class="animate-viewport c-image" alt="Steve van Bodegraven, Machine Learning Engineer at Microsoft and Dr Shane Penny, Fisheries Research Scientist at NT Fisheries review the identified fish species using the AI solution." /></figure>
</div>
<div class="m-content-placement f-related-stories">
<h2 class="c-heading-5">More Stories</h2>
</p></div>
</div>