Hackers get help from Garage interns with new features in program for quickly building apps

Following the initial release of Web Template Studio in May earlier this year, Web Template Studio 2.0 is now available with additional services, inspired by community feedback and built by a second team of Garage interns. The VS Code extension helps hackers create full stack web apps quickly, now with a broader range of front and back end service options. Try Web TS 2.0 and share feedback for new feature requests on GitHub and check out the full story on the Windows Developer Blog or the walk-through video on YouTube.

Tag teaming development

This team of Garage interns, based out of the Vancouver, BC, pioneered a new approach for the unique internship program, picking up where a first team of interns left off to build on a product and refine its direction. Unlike in a traditional internship, Garage interns hear pitches from sponsoring teams who outline challenging engineering projects interns will tackle with a team. Typically, interns create new projects from the ground up to be released by Microsoft or partners, or add new capabilities to scaled Microsoft products. In this case, the Web Template Studio team from the Summer 2019 cohort accepted the baton from the Winter Cohort and continued iterating on the product features and direction.

Web Template Studio screenshotIn addition to managing the opensource feedback provided following the launch at Build 2019, the interns spoke to current users, student developers, hackers, and more to understand where Web TS could be enhanced. In addition to requests to deepen the bench of supported frameworks, the team also honed in on the value the solution could provide to the novice developer with a foundation of experience creating web apps.

Web Template Studio was created with hackers in mind; it’s ideal for rapid prototyping and spinning web apps up quickly at hackathons. In fact, the team made early versions of Web TS 2.0 available to hackers across Microsoft’s global, annual hackathon in July. The intern teams confirmed that Web TS is most useful for developers who have some background in creating web apps with specific frameworks–in their research, about 20 hours of coding time–enough to select their preferred front and back end without the tedium of wading through endless forums to identify how to stitch them together quickly in a time crunch setting. Web TS continues to be a great solution to get started with Azure with its simple wizard.

Adding new services to Web Template Studio

The team listened closely to developer feedback on GitHub and expanded the front and back end frameworks supported from React.js and Node.js available at the initial launch to also include Angular, Vue, and Flask. In an update this morning, the team also added App Service support to make it even easier to create web apps powered by Azure for storage and cloud hosting.

Throughout the summer, the team added a number of new features, namely adding additional frameworks.

  • Angular support
  • Flask support
  • Vue support
  • App Service support

For the full details of new features and how to use them, check out the Windows Developer Blog.

You can see a walk-through of Web TS on YouTube.

Apply to the Garage Internship

The Garage is hiring for the 2020 Winter & Summer seasons! Here you can learn more details about the internship and how to apply.

Travel app and Microsoft Garage project Outings graduates, bringing new destination search experiences to Bing

The Garage empowers Microsoft employees to explore demand for new products and features through tangible experiments. Outings, a Microsoft Garage project, an app for iOS and Android released in December 2017 and has since generated the feedback, usage, and buzz required to confirm the sponsoring Bing team hypotheses that travel and destination search is an area for continued investment. The team has integrated insights from user’s interest and feedback into and Bing Search apps, and the project is officially graduating from The Garage.

An experiment designed to explore new search

Bing has always had a strong culture of experimentation and the product was built from the ground up to enable efficient experimentation. “Our culture has always been: experiment, fail fast, learn fast—our infrastructure is built to equip engineers to flight and A/B test new features,” shares Jyot Patel, the Engineering Manager of Outings and a Principal Software Engineer on the Bing team.

While the team had access to this state-of-the art testing infrastructure, they built dedicated mobile apps to more acutely test an experience that presents users with possible destinations of interest. Outings identifies rich content from online travel blogs and curates it into bite-sized attractions, or outings, that users can browse through. It’s powered by a data pipeline that mines meta-data in popular travel blogs, which debuted in the Outings experience. The focused apps allowed the team to hone the relevance of the new data type and get clear feedback and ratings on the experience to zero in on users’ interest in curated travel content.

Once they observed the positive feedback for Outings, the team knew they had to scale the experience to more users on both desktop and mobile.

Outings insights scaled to and Bing mobile experiences

Oswaldo Ribas is a Principal Program Manager working on The Bing Attractions and Outdoors team which helps users discover attractions and destinations that they might want to visit, see, and experience, and in the process, help them map their experience. “Our goal is to continuously improve these scenarios.” He worked closely with the Outings team to explore what they’d learned about user experience and their technical approach through their experiment. “The work will surface in a number of places including the search results page and the Maps vertical.”

Bing has long had dedicated search result experiences for maps, travel, and destinations, and the insights and technology from Outings will fold into Bing in three chief ways.

  • Enhanced Search Result UX Now users will see attractions arranged in a carousel with better images, brief descriptions, and different topics synthesizing what the attractions are popular for.
  • Relevance The data pipeline fine-tuned in Outings powers more relevant search for destinations and attractions. Since the pipeline gathers meta-data from millions of travel blogs, the experience can surface the most popular attractions in particular areas
  • Improved Query Triggering Additional key terms in the search query that will cue these search result experiences. For example, you don’t just have to search the Space Needle; you can search things to do in Seattle or things to do near me

Jyot continues, “All the feedback and reviews we got, the first impression of the app was really positive. They really enjoy the content. But even if this content is already available, you’ll have to scour the internet for that information. There’s a need to have all this rich content about destinations in one place.” Now, the opportunity to access that content from a simple, single place will be available on a far broader scale.

“The positive feedback told us that there is quite a bit of demand for rich, attraction and destination content to be presented in a put-together fashion. Obviously, there’s a lot of opportunity for us to invest in travel and destinations moving forward. It confirmed suspicions that we have, basically,” adds Oswaldo.

The team is excited to bring these enhanced search results to a broader user base and continue to improve the experience. “Moving forward, we’re acquiring more and more content around attractions in France, Italy, Spain and the next countries that we’re after. I’m looking forward to seeing all that rich data surface on Bing,” he continues.

Congratulations to the entire Outings team and thanks to the users who can now see their feedback integrated into a larger search platform.

Outings fans, find your next outing on Bing

With this experiment complete, the Outings mobile app on iOS and Android will sunset on July 31, 2019. The enhanced search relevance will be available on Bing desktop and mobile experiences for English in the United States.

The Outings mixed reality experience highlighting Maps SDK will remain available for users who want to continue to use the sandboxed, sample app. The Outings Garage project will remain active but will mainly serve to illustrate the power of Maps SDK to MR developers.


Meet Azure Resource Graph, latest Microsoft Garage Wall of Fame inductee

For the 2017 company-wide Hackathon, Chirag Gupta and Gaurav Kapila saw an opportunity to bring visibility to and alignment behind an improved and more in-depth analysis of resource allocation for large enterprise customers on Azure. At the time, the two were working on log analytics and Azure Governance for their day jobs. They were gathering evidence on what were the gaps for enterprise adoption of Azure, and what could help the experience for customers with Azure subscriptions at scale. With their 2017 Hackathon project, Cloud Map, Chirag and Gaurav recruited a team to create a solution that provided Azure Resource exploration through a graphical interface as well as advanced query language for searching across Azure resource properties. When showing their hack project to various groups in the company, they found that everyone was dealing with one problem – helping large-scale, enterprise customers better understand their own Azure environments by bringing better visibility to existing issues or potential problems.

“Hackathon helped us really get these internal teams together. Our project has a lot of dependencies and partners inside Microsoft.” Gaurav explained. “We needed to get these teams in the same room and ask, ‘would something like Cloud Map be helpful to you? Does this solve your problems?’” Chirag added.

Their project resonated with teams and especially Group Manager of Azure portal, Leon Welicki, who not only joined their project but became the sponsor, key in guiding the project towards public preview and general availability as Azure Resource Graph. Alex Zakonov, who was the Director of Application Insights in 2017, also sponsored the project with engineers from his team contributing to the project early on. The amount of validation the project received was so powerful that the portal team adopted Azure Resource Graph completely, with the top search bar in Azure portal pulling data solely from Azure Resource Graph.

“We coded like crazy in 3-4 days during Hackathon, we were elbow to elbow, sitting together, so we developed a really close relationship. Fantastic full stack collaboration between everybody involved, which set a very solid foundation of two teams that worked like one.” Leon recalled his team working side by side with the Graph team during the Hackathon and how each member of the hack project worked across disciplines to achieve a compelling demo that showed their vision for the project. “We continue to have a fantastic partnership going on driving two parts of the product – front-end and back-end.”

Azure Resource Graph - dashboard displaying bar charts for VMs, SQL database, and disks counts

In 2018, Chirag again participated in the Hackathon to experiment with what their team had built. Their 2018 Hackathon project, Microsoft Graph for Azure, investigated ways Azure Resource Graph’s reach could be extended by integrating Microsoft Graph to allow exploration using Microsoft Graph API’s.

In September of 2018 at Microsoft Ignite, Azure Resource Graph was officially released as part of Azure Governance, a set of services for customers that help build and scale applications quickly while making sure their environments are secure, compliant, and cost effective. Customers are able to access Azure Resource Graph by directly searching in the Azure portal search bar, querying and filtering on multiple resource properties, and seeing an in-depth and even visual representation of their resources across Azure subscriptions. All to ensure customers have a clearer picture of their activity and environments across Azure for smarter and more efficient management to save time and money.

At Build 2019, the team released the Azure Resource Graph explorer view in Azure portal. In addition to allowing customers to create custom queries, they could now also create dashboards based on the Graph. The experience was also made available on the Azure mobile application, expanding Azure Resource Graph’s footprint.

Even after public preview and general availability release, expect the team to continue improving and extending Azure Resource Graph. The team is collaborating closely with Azure portal to provide robust data visualizations and build improved Azure management experiences that are critical for IT admins, cloud admins, DevOps, and enterprise businesses everywhere.


Interactive art: ‘Public Radio’ installation lets you pick the music

Public Radio on display at District Hall
Public Radio on display at District Hall (photo credit: New American Public Art)

Standing 10 feet tall, elegantly sloped along its front face with the soft light of rainbow LEDs glowing behind half inch of frosted acrylic, “Public Radio” is a unique art installation you’ll find at District Hall’s outdoor plaza in the Seaport, Boston’s home for innovation. Turn the metal-rimmed dials to tune to a different station as the corresponding LEDs light up on this giant radio, powered by a tiny but mighty BBC micro:bit computer, inviting the public to play with and listen to the latest broadcasts and music. Built by New American Public Art in the Microsoft Garage at the New England Research and Development Center, Public Radio is an interactive art installation with microprocessor parts and a futuristic look that encourages a spirit of community, working and grooving together.

A sense of community, inspiring movement and spontaneous interaction

Somerville-based New American Public Art (NAPA) is a multi-disciplinary art studio that builds interactive projects and installations. NAPA answered the 2016 Public Space Invitational Digital Challenge put forth by Boston’s Mayor’s Office of New Urban Mechanics in collaboration with Microsoft. After 6 months of fabricating and building, the larger-than-life radio was ready for show. It has been living in The Garage at NERD as a centerpiece that demonstrates how technology and art can converge.

“When you scale up a common object and place it in the public, you often get uncommon and surprising results. How will the public decide among themselves what to listen to?” says Dan Sternof Beyer, Creative Director and Co-Founder of NAPA.

From NAPA’s original project proposal, the idea is “Music inspires movement, creates a sense of place, and provides an opportunity for spontaneous interaction with other people.” NAPA designed the project in collaboration with creative engineer Andrew Ringler. Chris Templeman, Microsoft Garage program manager, had connections with the local studio prior to joining Microsoft. Templeman introduced NAPA to the bevy of hardware and fabrication tools in the Garage maker lab. “The Garage at NERD has embraced local artists since we opened our doors last year. We’re constantly inspired by art and how that creativity and imagination can help make better products and spark new ideas.”

Public Radio in The Garage at NERD

Public Radio in The Garage at NERD (photo credit: New American Public Art)

“I think people will be excited to experience the Public Radio because it’s an innovative way to activate a public space using a combination of interactive art and technology,” says Aimee Sprung, Director of Civic Engagement at Boston’s Microsoft office. “The build-out at Microsoft Garage has been both an educational and exciting experience and we’re proud to have sponsored this program as part of our commitment to driving civic engagement in Boston.”

More than art – bringing technology to life

Public Radio was freshly installed outside District Hall on April 5th and will be on display for the public to interact with through September. For the past year, however, it has stood as a compelling example of innovative tech-meets-art in The Garage at NERD, showcased in interactions with the community and local schools. “We often use micro:bit in workshops where we invite local STEM students,” explains Templeman. “When we have students come in and learn how to program on the micro:bit, we use the Public Radio to illustrate what you can do with it to run such large and interactive projects. It brings the tech to life in a creative and tangible way.”

Employees from Microsoft’s Education Windows and Devices team have also been engaging public schools and collaborating with The Garage to bring students in to engineer escape rooms. “We are teaching kids engineering by building escape rooms using the micro:bit.” Tisha Nguyen, a Solutions Professional on the Education team explains. “This hands-on experience outside the traditional classroom setting is extremely valuable as we continue to prepare the next generation of Data Scientists, Engineers, and Statisticians.” The effort was made possible with the help of engineers from Office who designed the escape room experiences, Jean-Yves Ntamwemezi, Arthur Berman, and Eric Frackleton.

Public Radio outside of District Hall

Public Radio outside of District Hall (photo credit: New American Public Art)

With Public Radio now leaving The Garage space and headed to its new home, there are still plenty of creative sources for inspiration left. The recent collaboration of MIT, The Met, and Microsoft has spurred new relationships with other art institutions such as the Museum of Fine Arts in Boston. “We are thrilled to work with local artists and museums to figure out different ways tech can help push the boundaries of creativity,” says Linda Thackeray, Director of The Garage.

The details and specifications of the Public Radio are all available open-source on Github, including how to get the micro:bit connected with the radio boards, sensors, and LEDs. Learn more about the Public Radio from the official press release.


Garage projects team up to illustrate the power of mixed reality

A customizable 3D Map canvas for MR creators

MapsSDK_Screenshot_01Maps SDK offers a host of developer-friendly controls, allowing creators to focus their energy on innovative MR experiences. Developers can drag the map control into a Unity scene to build on an out-of-the-box map complete with 3D terrain and configurable city and location labels sourced from Bing Maps, rather than start from scratch and create their own. From there, they can overlay additional geo-anchored content that users can explore via a table-top map experience with built-in panning, zooming, and navigation tools.
 In effect, Maps SDK creates a canvas against which developers can illustrate content in new and interesting ways. Whether developers are layering in data visualizations, custom 3D objects/terrain, or other geo-anchored content, the project gives them a head start so they can spend less time on the basics, and more time bringing a creative MR map experience to life.

Explore places near and far, now in mixed reality

“We are constantly thinking about the intersection between maps and new mediums,” shared Brian Kircher, a Senior Software Engineer on the Maps team and developer for Maps SDK. “We not only wanted to empower MR developers, but also inspire them.” By day, Brian, David Buerer, and Jesse Levine, work on the Maps team and sit next to Oswaldo Ribas, the current PM of Microsoft Garage project Outings. “Last summer, we were brainstorming how we could illustrate what a 3D map control could do when it hit us that Outings was the perfect experience to bring to mixed reality,” shared Jesse, a Program Manager II. Pairing up with the Outings team, the three pitched an Outings mixed reality experience to a group of Bing interns.
The original Outings Garage project launched last December, and presents users with points of interest by pulling in rich pictures and relevant descriptions from digital travel journals so they can explore local gems and far-away destinations through a Tinder-esque, card UI. The team of interns was able to easily create a similar travel exploration experience in a few short weeks by leveraging Maps SDK and the location-based data that powers the Outings iOS and Android apps. While the sand-boxed sample app is not connected to the mobile experiences (locations saved on mobile will not appear in the MR experience), Outings fans will recognize the same point-of-interest cards, now overlayed geographically against the rich, 3D terrain from Maps SDK.

• Discover points of interest via search or panning/zooming as the crow flies across a 3D, tabletop globe
• Save favorite destinations to build a bucket list or travel itinerary
• Explore famous natural and urban landmarks in 3D

The Maps SDK team polished the new experience and enhanced the SDK, incorporating feedback from the interns and Outings team, as well as other teams around Microsoft working on innovative 3D experiences. The team has already collected feedback from first-party developers, but is excited to open feedback up to third-party developers creating anything from productivity to entertainment experiences.

Try it now and share your feedback

Maps SDK is now available and accepting feedback via GitHub. The sample experience, Outings, is a UWP app for mixed reality headsets and HoloLens, and now available for download and feedback via the Microsoft Store.

By devs, for devs: Meet new Garage projects XAML Studio and Team Retrospectives

Today, we announce two new Garage projects built by developers, for developers. Two teams, inspired by their roles building developer tools, worked on ideas to enhance the developer experience during Microsoft’s annual Hackathon. XAML Studio, a Microsoft Garage project, makes iterative prototyping easy with real-time UI module changes. Team Retrospectives, a Microsoft Garage project, closes the loop on continuous iteration, enabling an Agile retrospectives experience within Azure DevOps. Whether devs are starting a new project or reflecting on how to work more efficiently as a team, they can use these projects to get a little more productive.

From Design to Development

XAML Studio enables developers to rapidly prototype UWP XAML by previewing and adjusting XAML in real-time before easily copying it over into Visual Studio.
Having spent the better part of a decade listening to developers around the world and as a developer himself, Michael Hawker has spent a lot of time thinking about how to make dev tools even more useful. Michael is a Senior Software Engineer working on the Partner, Analytics, and Essential Experiences, or PAX, team and contributes to the Windows Community Toolkit. He found himself wanting a faster way to edit XAML UI and set out to build a project that could do so in Microsoft’s 2017 Hackathon. The annual event gives Microsoft employees the opportunity to explore and build new ideas. Several teams continue working on the projects beyond the week-long experience to make their way into the hands of customers via Garage projects.
Xaml Studio projectMichael created an interactive XAML sandbox to test out small code changes without having to compile or rebuild projects first, much like what WPF developers had access to for years. “When you’re in Visual Studio, you have a variety of tools available to make coding and laying out your UI easy, but it’s not always straightforward to visualize the final result. So, I thought to myself, ‘What could I do to add to this experience?’” shared Michael, the creator of the project. “XAML Studio is about that quick prototyping loop. Using this tool, I can type my XAML and see how it renders right away. Then, I can interact with it and explore how it behaves before it goes in my app.”
XAML Studio follows a number of Garage projects that facilitate rapid prototyping. Launched in the January 2018, Garage intern project Ink to Code, enables developers to sketch wire frames and kickstart their app in Visual Studio. Both Ink to Code and XAML Studio help developers transition from design to code by creating concrete development assets that can be used in Visual Studio. Where Ink to Code provides a way to create several prototypes very quickly, reducing repitive steps across multiple UI sketches, XAML Studio accelerates prototyping through easy-to-see iteration. Developers can now quickly fine-tune their prototypes, aided by an interactive preview window.
XAML Studio also hosts a variety of tools, including a binding debugger that can illustrate how UI is going to use the data behind it and if problems arise, where those might be occurring. Check out the full set of features.

By Developers, For Developers

The team behind Team Retrospectives also got their start at the One Week Hackathon, inspired by their roles in the Azure Production Infrastructure Engineering organization, or Azure PIE, where they focus on ways to improve the Azure developer experience. “We knew we wanted to work on something that would help devs and tackle something that we could actually create a working prototype for during the timeframe of the Hackathon,” shared Andy Stumpp, Senior Software Engineering Lead in Azure PIE and the development lead on the project.
Photo of the Team Retrospectives team at the Microsoft Hackathon
True to the “One Week” name, the team landed on a simple idea that could make their own projects more productive and could be created in the space of a few days: building an experience that integrated Agile retrospectives into Azure DevOps.
Many Microsoft developer teams use Agile methods to build, deploy, and continuously improve upon their products, as well as create tools that empower developers to do the same. Azure DevOps is a set of modern development services that make it easier to plan, collaborate, and ship faster, and many of its features support Agile. “I love Azure DevOps. It’s a one-stop shop,” shares Perth Charernwattanagul, a Senior Software Engineer in Azure PIE and the technical lead for Team Retrospectives. “It has everything I need from starting a project to releasing it; I don’t need 5 or 10 different products, I can just use Azure DevOps, and everything is streamlined and coordinated.”

Closing the Agile Loop

The team loved using Azure DevOps to organize their iterations, the first phase of Agile in which developer teams determine and execute work items, but they found that they also wanted to integrate retrospectives, the closing step in Agile, designed to fuel improvement and upcoming work items in the next iteration. “Retrospectives are a key part of the Agile process,” adds Andy. “They help you figure out what you can do better and the next steps in your new sprint.”

Team Retrospectives Screenshot

The Team Retrospective hackers created a project in Microsoft’s internal Hackathon tool and quickly attracted other developers around the company who shared a similar problem. They achieved their goal of creating a working prototype during the hackathon and immediately began testing it during their everyday roles. The team found that not only did Team Retrospectives streamline work items within Azure DevOps, it also made collaboration more efficient. “I’m based in Vancouver, and the rest of our team sits in our headquarters in Redmond,” Perth explains. “Our tool made it easy to collaborate digitally, and with the option to add our feedback in advance, we didn’t have to reschedule our meeting if someone couldn’t make it.”

Increased Collaboration and Reflection

Team Retrospectives is an Azure DevOps extension that enhances collaboration and productivity within the Azure Boards service:
  • Collect – Create a board, connect from any device and start submitting positive or negative feedback. The interface makes it easy to manage boards and feedback cards.
  • Group – Organize similar items into categories to facilitate discussion.
  • Vote – Prioritize feedback when team members vote on the items that are most important to them.
  • Act – Create follow-up action items in your team’s Azure DevOps project. Track previous work items and view roll-up status to ensure your team is progressing.
Since building and using Team Retrospectives, the team has found themselves becoming more effective. For example, shares Perth, “Having this extension baked right into Azure DevOps makes it effortless to have these retrospective conversations, and we’re simply prioritizing it more. We’ve become faster, more impactful, and more observant. We were able to hit a difficult deadline by quickly noticing that we were distributing too many work items to one person.”
“Team Retrospectives makes sure that we’re not just shipping features, but always learning from them.” –Jeff Braunstein, Senior Program Manager, Azure PIE and PM, Team Retrospectives
The team also rolled the project out to other Microsoft groups for feedback and testing, both within their own organization as well as to several teams working on the Microsoft Store. With this release, the team is eager to get feedback from developers beyond Microsoft, especially around how the team can support different Agile styles and methods. “Every team conducts retrospectives their own way; we want to know what customizable features can help teams perform at their best,” continues Perth.

Try XAML Studio & Team Retrospectives


Meet Ear Hockey: Newest Garage project infuses inclusive design into the classic arcade game

Three years ago, a team of Microsoft employees joined together on a quest to make gaming more fun and accessible to people who are blind or have low vision. After years of hacking on machine learning algorithms in their spare time, talking with customers in the blind and low vision community, and partnering with gaming experts, the team releases their first title: Ear Hockey, a Microsoft Garage project. With in-game audio cues that enable gamers to play with or without for visuals, Ear Hockey was crafted using inclusive design to promote accessibility in game development.

When spatial audio meets accessible game development

Ear Hockey is a Pong-style, first-person paddle game, where players use their keyboard or controller to hit the ball back and forth with a partner or computer player—but with a twist on the arcade classic air hockey: it uses unique spatial audio cues, powered by Microsoft’s HRTF spatial audio engine for Unity, to signal to the player where they are in the play-space and where the ball is traveling. The team was inspired by  and worked with Daniel Kish, an expert in echolocation and President of World Access for the Blind, to craft special sounds for ball movement and contact with the paddle or other play space surfaces to help orient players. For example, a fine-tuned combination of low-frequency humming and high-frequency ticking, along with other contextual sound effects, makes the ball easier to locate with audio alone.
As they developed Ear Hockey, the team also consulted with Senior Program Manager Brannon Zahad, who has over 16 years of experience in the gaming space and works on Accessibility R&D. “One of the biggest challenges we have in the industry right now is: when a developer wants to make a game, it’s really overwhelming to think about all the different things they need to do to make their game accessible,” describes Brannon. “For the longest time, there’s been a perception that it would be impossible to build a first-person perspective game accessible to people who are blind, for example. The work this team is doing to evolve those perceptions and build technology that empowers game developers is, frankly, incredible.”
The grassroots team, which goes by Audio Augmented Reality Gaming, or AARG, originally set out to tackle making games more accessible for people who are blind or have low vision. After winning the Ability Hack category at Microsoft’s annual Hackathon in 2017, they met with several people working towards similar goals across Microsoft, including the Microsoft Researchers who developed Project Triton. The spatial audio solution creates realistic reverb effects based on objects in a video game’s map using sophisticated machine learning algorithms. The AARG team recognized the potential for this kind of audio capability for accessible game development, and has since been partnering closely to explore how this could improve not only the gamer’s experience, but also the developer’s experience in building a more inclusive game. The game studio who developed Gears of War 4, The Coalition, used Project Triton to bring next-level sound to life, featuring the possibilities of advanced reverb effects; the AARG team quickly realized they could help inspire even more innovation in this area by building their own title.

Inclusive gaming

The Ear Hockey attends the CSUN Assistive Technlogy ConferenceAARG is comprised of a passionate set of team members from every corner of the company, many of whom have never formally worked on game development. To ensure Ear Hockey was a fun experience for both people who are blind or low vision, as well as people who are sighted, the team attended one of the world’s largest accessibility summits, the CSUN Assistive Technology Conference, where they showcased a few experimental games to members of the blind and low vision community. Players of all experience types provided feedback that helped shape the game. “For me, I realized the importance of orienting the player. We’d taken for granted how critical it is to provide that information at the beginning of the experience,” shared Patrick Farley, a Content Developer in the Cloud and AI group and one of the core architects of Ear Hockey. “Since then, we’ve enhanced our tutorial with a lot more detail.” For Boris Baracaldo, a Software Engineer and the other primary game developer, he noticed there was room for more customization with respect to ambient noise. Ear Hockey gamers can now adjust the volume of background music because of the invaluable customer feedback the team received.
In addition to soliciting feedback from customers and employees from the blind and low vision community, the team also designed Ear Hockey with a visual experience, subtitles, and support for the Xbox Adaptive Controller, another Hackathon project, in an effort to make the game more inclusive for a wide range of players. This has influenced how the team approaches developing for accessibility in their day-to-day activities. “I’m a Software Developer in Dynamics and this has completely shifted my approach to making products accessible,” continued Boris, who has found new ways to apply inclusive design to the products he works on in his full-time role.

Try it Out

With Ear Hockey now available worldwide, the team is eager to get feedback from players and drive a larger conversation around accessible gaming and game development. When asked about the team’s journey over the past 3 years, original organizer and Senior Mechanical Engineer Brandon Arteaga spoke of their excitement to offer this fun experience and help promote awareness about the power of echolocation. “We support Daniel Kish’s mission to teach echolocation as a skillset which can empower members of the blind and low-vision community to explore the world more freely. Anything we can do to shine a light on his work would be a dream come true.” Wilson Dreewes, a Senior Software Developer in AARG added, “Our goal is to showcase feature sets and what can be done in this space. We hope that the games we’re releasing now would be fun in and of themselves, but also that they inspire more innovation in game development.” So, have fun playing! Please try out Ear Hockey and share any feedback to the team on the game or accessible game development via UserVoice. You can also read about the full game features on the Garage Workbench.


New Garage project Earth Lens pairs AI with aerial imagery to aid disaster relief, environment

This summer, Microsoft’s AI for Earth team and the office of the AI CTO decided to challenge a group of Garage interns to pair AI with aerial images to make a difference. Today, we announce the release of their summer project: Earth Lens, a Microsoft Garage project, a Xamarin-based, open source project for iPad that identifies, tracks, and analyzes objects in aerial imagery to assist in scenarios such as disaster relief and environmental conservation. Earth Lens releases on the heels of Microsoft’s announcement to commit over $40 Million to new initiative to leverage AI in efforts to save lives, joining a suite of programs known as AI for Good, including AI for Earth, AI for Accessibility, and now, AI for Humanitarian Action.

Earth_Lens_Screenshot_2Imagine yourself as a data analyst working to identify objects in aerial imagery. Maybe you’re tracking fishing vessels through the ocean to support ocean sustainability and protect marine ecosystems, or perhaps you’re looking at cities that have been hit by a natural disaster to determine where to allocate your relief efforts. In either case, you would need a dedicated team to comb through thousands of satellite images, manually identifying, tracking, and analyzing relevant objects. Extracting insights from imagery data is an expensive process that could take upwards of days, delaying the decision-making process in mission critical situations.

Inspired to make this process more efficient, the two co-sponsors decided to team up to pitch Garage interns on tackling this problem by leveraging AI. The Garage Internship Program offers teams of 5-6 university students the opportunity to design and build their own project in response to a challenge by sponsoring Microsoft groups. The Vancouver-based group of interns loved the AI for Good pitch. “In the start of May, our sponsors came to us with an idea that immediately sparked interest among the six of us: an opportunity to leverage AI to help with environmental efforts headed by researchers and conservationists,” shared Michelle Chen, Program Manager intern for Earth Lens.

AI for Good

The team was especially inspired by the work of the organizations helping victims of natural disasters by providing satellite imagery to hasten relief. She continues, “Our vision for Earth Lens was to use technology and AI to transform the way humanitarian work is conducted. The rise of automated image recognition has empowered humanitarian organizations to triage damage, and prioritized areas that require immediate help.”

Equipped with a trained machine learning model from the AI for Earth team, the interns set out to build a minimally viable product over the course of their remaining 16-week internship. They approached design with a customer obsessed mindset and interviewed prospective users like to the Red Cross, OceanMind, and FarmBeats to identify useful features and iterate on their prototype. The team quickly realized that, in addition to disaster relief, such a tool might be useful in a variety of social impact and sustainability applications such as agriculture, forestry, fishing, and land conservation.

The team built an iOS app that offers a host of features that humanitarians and environmental researchers can leverage to accelerate their analyses:

• Automatically identify, classify, and label objects in satellite images, encircling like objects in color-coded bounding boxes
• Count and aggregate objects identified by the machine learning model
• Toggle classes on/off to focus in on specific objects
• View images over time alongside a data visualization chart to identify trends and patterns in a Time Series mode
• Use the app remotely, without an internet connection

With the Earth Lens source code and corresponding instructions now available on GitHub, researchers will be able to build their own iOS apps that automatically extract information from a large dataset and scan that analysis in a useful and interactive view. As a result, this open source project can be useful in various industrial or agricultural applications to further Microsoft’s commitment to sustainability. In a disaster relief scenario, another minute could mean another life lost.

An Internship Where You Can Make an Impact

Earth_Lens_Team_ImageThis Garage project illustrates how developers can use AI to make a difference and the team hopes they’ve built something that will spark humanitarian creativity in others. “We all had the privilege and opportunity to experience what it means to work at Microsoft and be a part of something larger than us,” notes Seara Chen, a Software Developer intern who worked on model integration. “Our entire team agrees that this internship was an amazing experience filled with learning, surprises and a lot of problem-solving.”

Interested in trying Earth Lens? Check out the source code on GitHub, where you can also find instructions on how to build the iPad application and leverage the machine learning model that powers the project. You can learn more or provide feedback via GitHub.

The Garage Internship Program is hiring. You can learn more about openings and explore past Garage intern projects: Seeing AI, Ink to Code, Mobile Chest X-ray.


MileIQ team launches new Microsoft Garage project to simplify expense management

In the fall of 2015, a startup team committed to making mileage logging easier joined Microsoft to expand their ability to empower the self-directed worker. Today, the team that created MileIQ now announces an iOS app to simplify expense management. Spend, a Microsoft Garage project, is now available for download in the US App Store.

The Microsoft Garage has released dozens of projects in the four years following the October 2014 launch of its Garage project program. Several projects have gone on to become features of flagship Microsoft products or new, branded products in their own right. Both self-organized, grassroots teams and arms of Microsoft product groups have leveraged the program to collect user feedback, add new features, and refine their approach.

Spend joins a host of Garage projects that have narrowed in on a specific target customer to understand their needs and build scenarios that tackle acute problems. Sports Performance Platform worked with the Seattle Seahawks, Benfica, Cricket Australia, and Real Madrid among other major sports teams and organizations. Video Indexer—the once-titled Video Breakdown and Garage project alum—synthesized multiple Microsoft Cognitive Services into an improved experience and is now offered as a standalone Azure Media Service. Microsoft Kaizala began its journey focusing on mobile collaboration scenarios, getting its big break working with the Indian state government to organize a massive holiday event attended by over 20 million people.

Hassle-Free, On-the-go Expense Management

Spend is mobile-first and built with the user in mind, making it simple to track expenses for reimbursements or taxes

• Quickly manage all your purchases for your expense reports with automatically tracked expenses from a connected credit card, debit card or bank account
• See purchases in a feed and easily classify expenses as business or personal with a single swipe
• Edit purchases or bulk classify expenses through the web dashboard
• Create accurate reports with only a few clicks for the week, month, or another customer period
• Fully customize reports which are available in either spreadsheet or PDF, commonly-used formats compatible with leading accounting and expense management software
• Snap pictures of receipts and attached to purchase with additional features for easy cash purchase management
• Track confidently: Spend uses 256-bit encryption, bank-level security, with Microsoft certifications

Spend, a Microsoft Garage project

The team has long focused on creating solutions that simplify work and empower self-directed workers, and built its first product MileIQ to enhance the mileage logging experience. When you’re on the road a lot, it can be time-consuming and chaotic to organize mileage and gas information for reimbursements or tax deductions. Now, the team is also turning its attention to another pain point for this audience: expense management.

“Keeping, sorting, and tracking paper receipts is annoying and inefficient. Spend uses intelligent features to bring receipt and expense tracking to the modern era,” says Heman Chawda, Product Manager for Spend. Built for mobile and designed around those always on the go, Spend takes a fresh approach to expense management. “It’s a great opportunity for us to be able to explore this space,” shared Nat Robinson, General Manager of MileIQ. “We are a growing team at Microsoft working on a number of experiences that empower small and medium businesses. The Microsoft Garage gives us the chance to offer a simplified expense management experience and really hone the value we provide. This is an important area of investment for us—we’re really excited to take this first step.”


Microsoft New England teams up with customers to create high-impact product features

Microsoft Garage has been a force in affecting positive change within the company, practicing new ways of thinking and spreading a growth mindset culture to employees as well as partners outside the company. Microsoft’s global Hackathon is a key player in CEO Satya Nadella’s culture change priorities driving innovations that empower customers to achieve more. The Garage at NERD, located in Cambridge, MA is one of many Garage locations worldwide making big strides in challenging conventional ways while furthering business impact.

Garage NERD Hackathon 2018 Science Fair

The Garage at NERD during the Hackathon 2018 Science Fair

The New England Research and Development Center (NERD) is steeped in a rich community of forward-thinking institutions that are molding some of the best minds of the future. One of the newer Garage locations, The Garage – NERD is in a unique position of having the perfect vantage point to measure the pulse of research giants like MIT right next door and invite these institutions and local Boston communities into the Garage space to collaborate and share learnings.

“It’s about the people,” said Linda Thackeray, Director of The Garage at NERD. “We’re here to help employees and teams drive employee-led innovation at the company, for products and other areas employees are passionate about. We do that by creating opportunities to experiment and collaborate, not only within the Microsoft family, but with the local community as well. Look where we are! The greater Boston area is loaded with unique and robust talent and industry tech.”

The product development teams at NERD are at the forefront of the customer obsession initiative. The Docs Collaboration team took advantage of Microsoft’s annual Hackathon to create a proof of concept that could change the way Office builds app features. Ben Wilde, principal program manager on the Docs team, led the hack project which focused on adding key features to Office apps to enable richer and smarter collaboration. “A handful of us have been on a bit of a crusade to ensure that we’re solving real, high-impact customer problems that we’ve identified during actual conversations with our users, understanding what challenges they face when collaborating and coming up with creative solutions to eliminate their friction.” Though the project was about Office collaboration features, their team’s journey started months before Hackathon when the Docs team as a larger group began talking with customers in a more concentrated effort to get to the root of customer needs.

“For me personally the Hackathon has emboldened me and proved that the process works – I feel even more convinced now. This is something we’re all passionate about and we truly believe this is how we should be driving features.”

Wilde and team documented all their learnings and research, including customer conversations, and how they approached customer research as a process. A playbook of how to build features that solve actual customer problems, with data and feedback to validate what features should be prioritized. “At a high level, the process we went through started with building empathy and understanding user needs – the foundational sit-down-and-talk-to-customers approach. And more important than talking is listening, understanding where they feel friction, where things work and don’t work,” Wilde explained. From those conversations the team created feature concepts that they showed to customers and investigated which were the most impactful for collaboration. Those ideas became the key features that were prototyped during Hackathon and showcased at The Garage NERD Science Fair at the end of Hackathon, where their project won the Business Impact award.

Hackathon Project Team Photo- Cambridge, MA Science Fair

Hackathon 2018 Science Fair, Business Impact award winners at Cambridge, MA: Terrell Cox (NERD Site GM), Lyndsy Stopa, Ben Wilde, Daniel Chattan, Anunaya Pandey, Garrigan Stafford, with David Ku, CVP

Lyndsy Stopa, a senior software engineer on the Docs team, was excited to talk about how the work they did began a larger conversation around what would need to happen to get these customer-backed features into product. “The more people that see this successful proof of concept the better. If you asked us, we’d love to build these features into the product tomorrow. At the same time, we also know the limitations.” While details of the features can’t be revealed, the work they did exploring the existing codebase and hooking up other Microsoft technologies for a working prototype, in true hack fashion, allowed them to think about possible implementation paths.

Anunaya Pandey, software engineer on the project, recalled the many discussions they had while showing their work to colleagues. “The fact that it was a hackathon meant that we could focus on what the experience for the customer was going to be like, versus how this was all to be implemented as part of a product. And that’s generally the opposite of how we traditionally do things. We wanted to be able to show how this was going to look inside of an app.”

“Hackathon for us is one step in a larger journey where we’re trying to shift culture and get people on board with this approach to building features.” Daniel Chattan, principal software engineer also on the Docs team, shared other takeaways. “We had an intern, Garrigan Stafford, who jumped in to work on the hack project. He was from a different product team, didn’t know our code base at all – and he was able to make changes very quickly. It challenged a lot of our preconceptions that there was a big on-ramp to be able to do these things and for new people to get into this space.”

The project also used Microsoft cognitive services and machine learning to interpret and infer user intent while collaborating and suggest actions at opportune moments. More recently as a larger group, the Docs team has expanded to include more data scientists with a machine learning focus. Chattan explained how they got more out of cognitive services than functional impact. “This was a great way to demonstrate thinking differently about how you can pull these things like AI and ML into thinking about a feature. Rather than think, ‘OK, we have this feature, now how do we fit cognitive services into it?’ the approach that we presented is ‘We have a feature we’re building that’s trying to accomplish something specific for the user, how can cognitive services help the user as opposed to helping the feature?’ It leads to a more natural connection of those two things.”

Project team photo, casual talking in Garage Maker space

Garrigan Stafford, Anunaya Pandey, Ben Wilde, and Daniel Chattan in the Garage – NERD Maker space

It’s about a different mindset, open to change, embracing experimentation, and validating with customers. All are values that the project team is championing within their immediate organization, and values that Garage continues to foster within the company.

Post-hackathon, the team is also seeing a shift happening in their work group in how people are approaching feature development by being more incremental. “Let’s get to a minimum bar functionality,” explained Chattan, “then iterate again, then iterate again, which allows us to move very quickly and see the results of what we’re building and do fast course-corrections. We’re building more incrementally to incorporate user learning into the process, so we can learn as we go instead of creating a big monolith that a user sees for the first time only after it’s finished.”

It may sound like a simple idea, practicing this level of customer-obsession, but when trying to steer feature development there are other factors that come into play, such as demand for resources, and prior commitments that must be brought to completion before any new work can begin. The project team will continue their customer-validation approach for new iterations of the hack project, while relentlessly pursuing the same in their day jobs for current features they’re working on.

“For me personally the Hackathon has emboldened me and proved that the process works – I feel even more convinced now. This is something we’re all passionate about and we truly believe this is how we should be driving features.” Wilde explained. This begs the question then – how does management feel about it? “Management is in spiritual agreement. Everyone we’ve talked to is supportive and clearly sees the value of the features and the approach. But it’s difficult to say right now if we can pull together the necessary resources.”

With time, the team remains optimistic that their features, and their processes, will be embraced more widely as they continue their journey. Just as Wilde and his team are on a mission to affect culture-change in product development within Office, The Garage pursues culture-change for the entire company, so individuals, groups and organizations can provide greater business impact using Garage programs. This week, The Garage at NERD opened its doors to HUBWeek, a weeklong Boston festival where entrepreneurs, artists, and academics come together to collaborate on creating an inclusive and diverse future for art, science, and tech. Another example of the connective force of The Garage, and how a vibrant community continues to grow, full of doers, thinkers, and change-makers that will shape the next big ideas.

Read more about The Garage – NERD, check out a project shipped by Garage interns, and learn more about internship opportunities.