05-06-2019, 11:19 PM
Accelerating the journey from automated to autonomous systems
<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2019/05/accelerating-the-journey-from-automated-to-autonomous-systems.jpg" width="1024" height="538" title="" alt="" /></div><div><div><img src="http://www.sickgaming.net/blog/wp-content/uploads/2019/05/accelerating-the-journey-from-automated-to-autonomous-systems.jpg" class="ff-og-image-inserted"></div>
<p>Microsoft’s autonomous systems platform overcomes some of these challenges by using a unique approach called <a href="https://blogs.microsoft.com/ai/machine-teaching/">machine teaching</a>. It relies on a developer’s or subject matter expert’s knowledge — someone who may not have a background in AI but understands how to steer a drill or keep the airflow in an office building at safe levels — to break a large problem into smaller chunks.</p>
<p>Instead of having reinforcement learning algorithms explore how to solve a problem randomly or naively, which could take forever, that person uses a programming language called <a href="https://docs.bons.ai/guides/inkling-guide.html">Inkling</a> to show the system how to solve simpler problems first and provide clues about what’s important. This shortcuts the learning process and enables the algorithms to hit on a solution much faster.</p>
<p>Microsoft’s platform also enables non AI-experts to establish and tweak the reward system, which is key to arriving at a solution that truly works. And it selects and configures the algorithms to tackle the task, eliminating the need for machine learning experts to custom build solutions.</p>
<p>For instance, team members worked with <a href="http://www.se.com">Schneider Electric</a>, a global company working to digitally transform energy management in homes, buildings and industries, to test whether AI could help reduce the carbon footprint of HVAC systems that are used to heat and cool large commercial buildings.</p>
<p>“Schneider is very focused on sustainability, and <a href="https://www.eesi.org/files/climate.pdf">large buildings are a top contributor</a> to carbon pollution. So there’s a really important mandate to make HVAC systems more energy efficient,” said Barry Coflan, senior vice president and chief technology officer for Schneider Electric’s EcoBuildings Division.</p>
<p>Centered on a <a href="https://www.schneider-electric.com/en/partners/alliances/microsoft.jsp">longstanding relationship</a>, a proof-of-concept test was conducted using the Microsoft toolchain and Schneider supplied simulation to <a href="https://aidemos.microsoft.com/machineteaching/smart-building">train an AI system</a> to autonomously run the HVAC systems that controlled airflow and heating in a conference room. It had to balance saving energy with other goals, such as keeping the temperature comfortable for people inside and making sure there’s enough fresh air to keep carbon dioxide levels from building up.</p>
<p>Optimizing for all those factors — which are controlled by different physical systems — requires far more intelligence than a simple thermostat, says Microsoft’s Hammond. The system has to account for environmental variables that are constantly changing: energy costs that fluctuate throughout the day, people coming and going from the room, what the outside weather is doing, the physics of how air flows.</p>
<p>Using a machine teaching approach, Schneider and Microsoft experts first taught the reinforcement learning system to control temperature well. Then the AI system learned how to control air flows to keep air quality at healthy levels. Then it learned to consider how room occupancy affected those outcomes.</p>
<p>Taking all those factors into account, Microsoft’s AI system was able to reduce energy consumption in the room by about 20 percent, while preserving comfort and high air quality when it mattered. The teams are now embarking on a second phase of collaboration to scale the simulation across different types of rooms and further boost energy savings.</p>
<p>Coflan said the laddered approach to teaching and the ability to layer in different rewards enabled Schneider Electric to understand how the AI system was learning and track which factors contributed to the biggest gains.</p>
<p>“A lot of what we do has safety ramifications so we really need to understand how the AI system is making decisions,” Coflan said. “This approach lets you see how the system is getting smarter and gives you an audit trail that is essential for safety and reproducibility. Our customers would want that too — you can’t just put a system out there and say ‘Trust us.’”</p>
<p><button class="cookie-consent-btn">Click here to load media</button></p>
<p class="wp-caption-text"><em>Microsoft’s autonomous systems platform uses a simulated warehouse environment in AirSim to train an intelligent forklift to pick up and deliver loads autonomously while recognizing and avoiding other obstacles. This video illustrates the vision for a future warehouse with pre-trained, intelligent forklifts assisting people in everyday activities.</em></p>
<h2><strong>Running simulation at scale in Azure</strong></h2>
<p>Because no company can afford to let a robot or an intelligent control system make millions of mistakes in a real-world factory or wind farm or highway as it is learning, reinforcement learning algorithms need to practice in a simulated environment that can replicate the thousands or millions of different real-world scenarios they might encounter.</p>
<p>The Microsoft toolchain also includes <a href="https://blogs.microsoft.com/ai/microsoft-shares-open-source-system-training-drones-gadgets-move-safely/">AirSim,</a> an open source simulation platform originally developed by Microsoft researchers to use AI to teach drones, self-driving cars or robots to learn in high fidelity simulated environments. Or, the team can work with customers to train autonomous systems using existing industry-specific simulators.</p>
<p>In either case, running these data-hungry simulations in the Azure cloud enables the system to test thousands of different decision-making sequences in parallel, which allows the AI models to learn what does and doesn’t work much faster.</p>
<p>“If I have the ability to spawn thousands of simulations at once and in each one the pedestrian crossing the street is different and the curve of the road is different, suddenly the AI system is able to gather much more diverse experience in a short amount of time ,” said Ashish Kapoor, Microsoft principal research manager. “Azure gives us the ability to run these simulations at scale, which is really important.”</p>
<p>AirSim also allows developers to train different AI and control tools to solve different parts of more complex problems. In helping develop autonomous forklifts for Toyota Material Handling, for instance, researchers broke the task down into sub-concepts that are simpler to learn and debug: navigating to the load, aligning with the pallet, picking it up, detecting other people and forklifts, delivering the pallet, returning to the charging station.</p>
<p>In these complex scenarios, Kapoor said, it may make sense to use reinforcement learning to train a forklift on basic control tasks, like picking up a pallet. Machine teaching helps the system learn in progressively more difficult steps, such as aligning the lift horizontally and then finding the proper angles.</p>
<p>But other parts of the problem might be better solved by entirely different tools like obstacle detection and avoidance algorithms, robotics path planning or classical control techniques. Decomposing the larger task into smaller ones allows developers to select and deploy the best tool for that particular job.</p>
<p>“We are working to provide a comprehensive platform for customers who want to build intelligent autonomous systems, covering development, operation and end-to-end lifecycle management,” Hammond said.</p>
<p><em>Top image: An experimental version of the Sarcos Guardian S, a visual inspection robot that can be used in disaster recovery or for industrial inspections, has learned to avoid obstacles and climb stairs on its own using Microsoft’s autonomous systems platform. Photo by Dan DeLong for Microsoft.</em></p>
<h2><strong>Microsoft Build 2019 — related autonomous systems links:</strong></h2>
<p><em> Jennifer Langston writes about Microsoft research and innovation. Follow her on </em><a href="https://twitter.com/langstonjen"><em>Twitter</em></a></p>
</div>
<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2019/05/accelerating-the-journey-from-automated-to-autonomous-systems.jpg" width="1024" height="538" title="" alt="" /></div><div><div><img src="http://www.sickgaming.net/blog/wp-content/uploads/2019/05/accelerating-the-journey-from-automated-to-autonomous-systems.jpg" class="ff-og-image-inserted"></div>
<p>Microsoft’s autonomous systems platform overcomes some of these challenges by using a unique approach called <a href="https://blogs.microsoft.com/ai/machine-teaching/">machine teaching</a>. It relies on a developer’s or subject matter expert’s knowledge — someone who may not have a background in AI but understands how to steer a drill or keep the airflow in an office building at safe levels — to break a large problem into smaller chunks.</p>
<p>Instead of having reinforcement learning algorithms explore how to solve a problem randomly or naively, which could take forever, that person uses a programming language called <a href="https://docs.bons.ai/guides/inkling-guide.html">Inkling</a> to show the system how to solve simpler problems first and provide clues about what’s important. This shortcuts the learning process and enables the algorithms to hit on a solution much faster.</p>
<p>Microsoft’s platform also enables non AI-experts to establish and tweak the reward system, which is key to arriving at a solution that truly works. And it selects and configures the algorithms to tackle the task, eliminating the need for machine learning experts to custom build solutions.</p>
<p>For instance, team members worked with <a href="http://www.se.com">Schneider Electric</a>, a global company working to digitally transform energy management in homes, buildings and industries, to test whether AI could help reduce the carbon footprint of HVAC systems that are used to heat and cool large commercial buildings.</p>
<p>“Schneider is very focused on sustainability, and <a href="https://www.eesi.org/files/climate.pdf">large buildings are a top contributor</a> to carbon pollution. So there’s a really important mandate to make HVAC systems more energy efficient,” said Barry Coflan, senior vice president and chief technology officer for Schneider Electric’s EcoBuildings Division.</p>
<p>Centered on a <a href="https://www.schneider-electric.com/en/partners/alliances/microsoft.jsp">longstanding relationship</a>, a proof-of-concept test was conducted using the Microsoft toolchain and Schneider supplied simulation to <a href="https://aidemos.microsoft.com/machineteaching/smart-building">train an AI system</a> to autonomously run the HVAC systems that controlled airflow and heating in a conference room. It had to balance saving energy with other goals, such as keeping the temperature comfortable for people inside and making sure there’s enough fresh air to keep carbon dioxide levels from building up.</p>
<p>Optimizing for all those factors — which are controlled by different physical systems — requires far more intelligence than a simple thermostat, says Microsoft’s Hammond. The system has to account for environmental variables that are constantly changing: energy costs that fluctuate throughout the day, people coming and going from the room, what the outside weather is doing, the physics of how air flows.</p>
<p>Using a machine teaching approach, Schneider and Microsoft experts first taught the reinforcement learning system to control temperature well. Then the AI system learned how to control air flows to keep air quality at healthy levels. Then it learned to consider how room occupancy affected those outcomes.</p>
<p>Taking all those factors into account, Microsoft’s AI system was able to reduce energy consumption in the room by about 20 percent, while preserving comfort and high air quality when it mattered. The teams are now embarking on a second phase of collaboration to scale the simulation across different types of rooms and further boost energy savings.</p>
<p>Coflan said the laddered approach to teaching and the ability to layer in different rewards enabled Schneider Electric to understand how the AI system was learning and track which factors contributed to the biggest gains.</p>
<p>“A lot of what we do has safety ramifications so we really need to understand how the AI system is making decisions,” Coflan said. “This approach lets you see how the system is getting smarter and gives you an audit trail that is essential for safety and reproducibility. Our customers would want that too — you can’t just put a system out there and say ‘Trust us.’”</p>
<p><button class="cookie-consent-btn">Click here to load media</button></p>
<p class="wp-caption-text"><em>Microsoft’s autonomous systems platform uses a simulated warehouse environment in AirSim to train an intelligent forklift to pick up and deliver loads autonomously while recognizing and avoiding other obstacles. This video illustrates the vision for a future warehouse with pre-trained, intelligent forklifts assisting people in everyday activities.</em></p>
<h2><strong>Running simulation at scale in Azure</strong></h2>
<p>Because no company can afford to let a robot or an intelligent control system make millions of mistakes in a real-world factory or wind farm or highway as it is learning, reinforcement learning algorithms need to practice in a simulated environment that can replicate the thousands or millions of different real-world scenarios they might encounter.</p>
<p>The Microsoft toolchain also includes <a href="https://blogs.microsoft.com/ai/microsoft-shares-open-source-system-training-drones-gadgets-move-safely/">AirSim,</a> an open source simulation platform originally developed by Microsoft researchers to use AI to teach drones, self-driving cars or robots to learn in high fidelity simulated environments. Or, the team can work with customers to train autonomous systems using existing industry-specific simulators.</p>
<p>In either case, running these data-hungry simulations in the Azure cloud enables the system to test thousands of different decision-making sequences in parallel, which allows the AI models to learn what does and doesn’t work much faster.</p>
<p>“If I have the ability to spawn thousands of simulations at once and in each one the pedestrian crossing the street is different and the curve of the road is different, suddenly the AI system is able to gather much more diverse experience in a short amount of time ,” said Ashish Kapoor, Microsoft principal research manager. “Azure gives us the ability to run these simulations at scale, which is really important.”</p>
<p>AirSim also allows developers to train different AI and control tools to solve different parts of more complex problems. In helping develop autonomous forklifts for Toyota Material Handling, for instance, researchers broke the task down into sub-concepts that are simpler to learn and debug: navigating to the load, aligning with the pallet, picking it up, detecting other people and forklifts, delivering the pallet, returning to the charging station.</p>
<p>In these complex scenarios, Kapoor said, it may make sense to use reinforcement learning to train a forklift on basic control tasks, like picking up a pallet. Machine teaching helps the system learn in progressively more difficult steps, such as aligning the lift horizontally and then finding the proper angles.</p>
<p>But other parts of the problem might be better solved by entirely different tools like obstacle detection and avoidance algorithms, robotics path planning or classical control techniques. Decomposing the larger task into smaller ones allows developers to select and deploy the best tool for that particular job.</p>
<p>“We are working to provide a comprehensive platform for customers who want to build intelligent autonomous systems, covering development, operation and end-to-end lifecycle management,” Hammond said.</p>
<p><em>Top image: An experimental version of the Sarcos Guardian S, a visual inspection robot that can be used in disaster recovery or for industrial inspections, has learned to avoid obstacles and climb stairs on its own using Microsoft’s autonomous systems platform. Photo by Dan DeLong for Microsoft.</em></p>
<h2><strong>Microsoft Build 2019 — related autonomous systems links:</strong></h2>
<p><em> Jennifer Langston writes about Microsoft research and innovation. Follow her on </em><a href="https://twitter.com/langstonjen"><em>Twitter</em></a></p>
</div>