Posted on Leave a comment

Seventeenth century French artifact arrives in Seattle for an immersive exhibition, powered by Microsoft

Visitors can explore the Mont-Saint-Michel through an AI and mixed-reality-powered experience at Seattle’s Museum of History & Industry

Museum visitors explore the Mont-Saint-Michel through an AI and mixed-reality-powered experie

SEATTLE — Nov. 21, 2019 Seattle’s Museum of History & Industry (MOHAI) and Microsoft Corp. on Thursday announced the opening of a new exhibit, “Mont-Saint-Michel: Digital Perspectives on the Model,” which features a unique blend of 17th and 21st century technology.

Powered by Microsoft AI and mixed-reality technology as well as the recently released HoloLens 2 device, the interactive exhibition transports visitors into a holographic tour of the picturesque Mont-Saint-Michel, a medieval monastery perched atop a remote tidal island off the coast of Normandy, France.

The virtual experience is complemented by a physical relief map of the Mont-Saint-Michel, an intricate, three-dimensional model of the landmark. Entirely crafted by hand in the 1600s by the resident Benedictine monks, the 1/144-scale model precisely depicts the monument in such intricate detail that maps like this were considered valuable strategic tools to leaders like Napoleon and King Louis XIV, who considered the maps military secrets and hid them from public view.

“The Museum of History & Industry is honored to share this icon of world history, enhanced by leading-edge technology, to create a unique experience born of innovations both past and present,” said Leonard Garfield, MOHAI’s executive director. “More than 300 years separate the remarkable relief map and today, but the persistent human drive toward invention and creativity bridges those years, reflecting the unbroken quest for greater understanding and appreciation of the world around us.”

The opening of the exhibit is timed with the 40th anniversary of the Mont-Saint-Michel being designated as a UNESCO World Heritage Site. This is the first time the relief map, as well as the mixed-reality experience, has been in North America.

“The relief maps were technological marvels of Louis XIV and Napoleon’s time. It’s exciting to see how we can blend old and new technology to unlock the hidden treasures of history, especially for younger generations,” said Brad Smith, president of Microsoft. “This exhibit provides a unique model for preserving cultural heritage around the world, something Microsoft is committed to through our AI for Good program.”

The Mont-Saint-Michel experience is an example of Microsoft’s AI for Cultural Heritage program, which aims to leverage the power of AI to empower people and organizations dedicated to the preservation and enrichment of cultural heritage. Microsoft is working with nonprofits, universities and governments around the world to use AI to help preserve the languages we speak, the places we live and the artifacts we treasure. For example, earlier today Microsoft announced it is working with experts in New Zealand to include te reo Māori in its Microsoft Translator application, which will enable instant translations of text from more than 60 languages into te reo Māori and vice versa. This will be one of the first indigenous languages to use the latest machine learning translation technology to help make the language accessible to as many people as possible. The AI for Cultural Heritage program is the fourth pillar of Microsoft’s AI for Good portfolio, a five-year commitment to use AI to tackle some of society’s biggest challenges.

The relief map is on loan to MOHAI from the Musée des Plans-Reliefs in Paris, which houses more than 100 historically significant and well-preserved relief maps. The relief map of Mont-Saint-Michel is considered the museum’s crown jewel.

“One of the challenges in the history of art is the relationship with the public. To gain the attention, to capture the view or the interest of the public, is not always evident,” said Emmanuel Starcky, director, Musée des Plans-Relief. “With the HoloLens technology, you have now the possibility to realize immersive experiences in art, where you still see the reality but have more information about it. It will be a unique experience for the American public to discover the relief map, its condition in the 17th century and its evolution through three centuries, as well as reflect on the purpose of those relief maps.”

Drawing from hundreds of thousands of detailed images, Iconem, a leader in the digital preservation of cultural heritage sites, used Microsoft AI to create a photorealistic 3D digital model of the historic structure. Then, French mixed-reality specialists at HoloForge Interactive developed a unique Microsoft HoloLens experience to draw people into the artifact like never before.

The “Mont-Saint-Michel: Digital Perspectives on the Model” exhibit, including both the original relief map and mixed-reality experience, will be on display at MOHAI Nov. 23, 2019 through Jan. 26, 2020.


MOHAI is dedicated to enriching lives through preserving, sharing, and teaching the diverse history of Seattle, the Puget Sound region, and the nation. As the largest private heritage organization in the State of Washington; the museum engages communities through interactive exhibits, online resources, and award-winning public and youth education programs.  For more information about MOHAI, please visit, or call (206) 324-1126. Facebook: Twitter: @MOHAI

About Microsoft

Microsoft (Nasdaq “MSFT” @microsoft) enables digital transformation for the era of an intelligent cloud and an intelligent edge. Its mission is to empower every person and every organization on the planet to achieve more.

For more information, press only:

Microsoft Media Relations, WE Communications, (425) 638-7777,

Museum of History & Industry PR, Wendy Malloy, (206) 324-1126, ext. 150,

Note to editors: For more information, news and perspectives from Microsoft, please visit the Microsoft News Center at Web links, telephone numbers and titles were correct at time of publication but may have changed. For additional assistance, journalists and analysts may contact Microsoft’s Rapid Response Team or other appropriate contacts listed at

Posted on Leave a comment

Microsoft Mixed Reality Capture Studios create holograms to educate and entertain

“We move the camera for you, versus you are the camera,” Waskey says. “It’s a very similar process to creating a video, but in the end, you’re getting a hologram.”

The process brings in several cross-company initiatives and strengths. Most recently, they’ve added Azure to the mix to boost the work.

“One of the pieces of tech that really proves to be important is the way we compress and package the results,” says Steve Sullivan, general manager of the Mixed Reality Capture Studio. “Some other providers struggle with heavy files that are hard to get to consumers. But we can crunch it down to about the size of a video stream from Netflix. So basically, anywhere you can stream Netflix, you can stream our holograms. And that puts us onto phones, web, HoloLens and – you name it. We’ve built players for every major device and platform, so if you’re a creator or performer wanting to reach an audience, we can get you there.”

Azure is a new element, Sullivan says, that gives the company and its partners the advantage of better scale and broader reach. It allows them to process much more data, as they’re able to rely on the cloud to process more content, more flexibly, than an on-site render farm. This also allows them to provide services to other capture providers who may already have their own infrastructure.

“Even those who capture with different technology can process their data in our Azure pipeline to leverage our compression and playback ecosystem, giving all creators more options,” Sullivan says.

And now, they’re taking this technology on the road with two mobile stages, for more informal sessions. New as of this year, these packaged configurations are easier to deploy and able to take advantage of the cloud, versus huge on-premise computing. The mobile capture studios can have fewer cameras (64 instead of 106), are lighter in weight and built for portability.

“This gives us the opportunity to go where the action is,” says Waskey, pointing to events like the British Open, in which Dimension demonstrated golfers’ swings.

“When we look at sports as a particular scenario, we’ve generally seen it at a very flat angle. Golf is truly three-dimensional, there’s an arc and plane that gets described, there’s a full range of motion to hit the ball. Every golfer is subtly different in how they approach that task. One of our partners, Dimension, used a mobile stage and the power of Azure to process very fast huge amounts of data to show golfers’ angles that are normally ‘too dangerous to film.’”

Posted on Leave a comment

Airbus reaches new heights with the help of Microsoft mixed reality technology

What is mixed reality?

Mixed reality takes digital information beyond two-dimensional screens to a three-dimensional experience by using holograms, which are images made of light and sound.

Microsoft’s HoloLens headset is a culmination of breakthroughs in artificial intelligence (AI), hardware design and mixed reality development. It allows people to interact with holograms in physical space, meaning that they can view and manipulate holographic images on their own in the air or in combination with real physical objects.

The recent release of HoloLens 2 takes the mixed reality experience a step further, allowing users to manipulate holograms the same way they would handle physical objects. The headset also offers eye tracking that can sense when a user’s eyes land on a particular location and produce relevant digital information, as well as automatic scrolling as the user reads. Users can log in via iris recognition, making sharing among multiple people easy and secure.

Airbus engineers use HoloLens mixed-reality headsets for training.
Hololens 2 helps Airbus designers accelerate the validation process – when they determine whether their designs are fit for industrialization – by 80 percent.

Putting mixed reality to work

Airbus has seen impressive results in its trials and deployments of Microsoft’s mixed reality technology in training, design and manufacturing.

“Mixed reality can help us to increase quality, safety and security,” Dumont says. “The level of human error is significantly reduced, and in aerospace, increased quality is increased safety—and needless to say, security goes with that.”

Mixed reality allows aerospace trainees to learn in an immersive virtual environment without the need for an actual physical aircraft or parts. This 3D environment can offer features that real-life training cannot, such as the ability to view elements in three dimensions from any angle.

HoloLens helps Airbus designers virtually test their designs to see if they are ready for manufacture. Mixed reality speeds up the process substantially, decreasing the time spent by 80 percent.

Mixed reality technology can also help workers on the production line access crucial information while keeping their hands free. Digital information, such as instructions or diagrams, can be overlaid on a real piece of machinery to aid in complex or hard-to-reach tasks. These kinds of mixed-reality solutions have allowed Airbus to cut manufacturing time by a third while improving quality.

Mixed reality empowers employees to execute their jobs in the most efficient and ergonomic way possible, and this contributes directly to performance improvements, according to Barbara Bergmeier, who is head of operations at Airbus Defense and Space.

“By having the right information at the right time in hands free mode, not only does quality increase, but also safety, and this is what we are looking for. Quality without consideration of the well-being of our workers is not possible,” she says.

An Airbus engineers uses a HoloLens headset to perform maintenance on an aircraft.
Mixed reality technology can be used to help Airbus production workers access information and instructions while their hands are occupied.

Working together to evolve mixed reality

Not only is Airbus creating solutions for its workforce, but it has built off-the-shelf solutions for its customers, so they can also benefit from Airbus’ expertise in building mixed reality solutions. Starting at the Paris Air Show, Airbus will be selling these in partnership with Microsoft on HoloLens 2.

“HoloLens 2 was born from the inspiration that it be designed for the customer, by the customer,” says Alex Kipman, technical fellow in Microsoft’s Cloud and AI group. “Airbus has long been a strategic partner in building the future of mixed reality solutions for an industrial environment and we have learned a lot from them. We are thrilled to continue our partnership as we embark on this next era of computing, the era of mixed reality and artificial intelligence.”

The first new solution offered under this partnership is a mixed reality training program first released with Japan Airlines (JAL). It helps maintenance operators and cabin crews learn in a 3D holographic environment and access instructions, heads-up and hands-free, while on the job.

In addition, Airbus will launch a collaborative map solution that allows participants from the defense and aerospace fields to virtually connect, quickly share space data and interact with complex virtual environments to plan and prepare ahead of missions.

Airbus is working on requests from other customers for mixed reality maintenance, training and remote collaboration solutions.

An aerial view of Airbus jets.
Over the next 20 years, Airbus aims to build 20,000 aircraft.

Leading in real life

Airbus’ collaboration with Microsoft on mixed reality goes beyond helping the company reach its internal goals. Such technological innovation is crucial to Airbus’ larger objective to become a world leader in digital services for the aerospace industry.

“We are very optimistic about this future collaboration with Microsoft based off what we’ve done in the last four years,” Dumont says. “This is really a way for us to lead our digital transformation. It’s multifold, but the use of mixed reality and HoloLens 2 are one of the key assets for Airbus in the future.”

Top photo: Holographic technology from Microsoft will be key to helping Airbus manufacture more aircraft faster. 

Posted on Leave a comment

Mixed reality year in review — and what’s to come in 2019

Hi everyone!

I hope this month’s blog finds you well. Before we dive in, I wanted to take a quick moment to wish you all a happy holiday season and a great end to 2018. May the next few weeks bring you an abundance of joy and time well spent with those closest to you.

As we come to the end of 2018, I’ve been reflecting on what an incredible year it’s been for mixed reality. Not just for the team at Microsoft, but for the developers, customers, and partners who have joined us on this journey.

The next year is going to be a big one for mixed reality—and below I will share some of what’s on my mind as we get ready for 2019. But first, let’s take a quick look back on 2018 and the year that was!

We ❤ MR

  • Dynamics 365 and MR have come together: Earlier this year, we shared a broad vision for business applications at Microsoft. As part of that, we focused quite a bit on why mixed reality has joined the Dynamics 365 family. Customers are at the center of Dynamics 365, and we see them reimagining their businesses to remove silos and leverage the vast amounts of data available across their organization in new, powerful ways. Mixed reality provides unique value because it helps employees visualize this powerful data in the context of their jobs, so they can produce greater impact for their organizations.
  • We launched Dynamics 365 Remote Assist and Dynamics 365 Layout: I am very proud of our team for building and launching our first two mixed reality business applications! These two applications provide out-of-the-box value for our customers using Microsoft HoloLens and help address some of the unmet needs of Firstline Workers. Firstline Workers represent more than two billion people who are in roles that make them the first points of contact between a company and the world it serves. They are often the first to engage, the first to represent a company’s brand, and the first on the scene to address a problem. These two mixed reality solutions help our customers use technology to understand data in context to solve problems and make decisions more quickly.
  • HoloLens expanded to 41 markets and is available to rent: We made it easier to get your hands on HoloLens by expanding into the United Arab Emirates and Singapore, as well as making it available to rent in the US and Canada so companies can evaluate prior to purchase or temporarily increase their inventory to support tradeshows or events. We’re thrilled to work with many more customers and partners around the world!

Our partners ❤ MR!

  • Trimble’s Hard Hat Solution and Trimble Connect: Our long-term partner, Trimble, announced Trimble Connect for HoloLens and a new hard hat solution for HoloLens that improves the utility of mixed reality for practical field applications. Trimble has paid close attention to supporting HoloLens as a high-value tool for Firstline Workers and continues to increase their impact on the market.
  • CAE, Visual3D, TeraRecon, Pearson, and others deliver game-changing mixed reality healthcare solutions: From more insightful ultrasounds and new ways of learning to performing live surgeries using mixed reality, many healthcare companies are embracing mixed reality to transform the way they innovate and support and help others.
  • PTC and BAE use mixed reality to change the way employees learn: BAE Systems makes the electric propulsion systems for HybriDrive® buses. BAE is working with PTC, a Microsoft Mixed Reality Partner, using their ThingWorx Studio to create mixed reality solutions that dramatically improve the efficiency of Firstline Workers. BAE and PTC used ThingWorx Studio to easily create a guided, step-by-step training solution for HoloLens to teach workers how to assemble a green energy bus battery. Using these tools, BAE can now create guides for Firstline Workers in just a few hours at a tenth of the cost, training new people 30–40 percent more efficiently.

Our customers ❤ MR!

  • Chevron: Chevron is already achieving real, measurable results with its global HoloLens deployment. Previously they were required to fly in an inspector from Houston to a facility in Singapore once a month to inspect equipment. Using Dynamics 365 Remote Assist, they can now put any inspector or expert anywhere in the world in under a minute.
  • ZF: A great way to see Remote Assist and Layout in action is through the work of ZF Group, a German car parts manufacturer headquartered in Friedrichshafen and an early partner on our journey. We have worked closely with ZF Group to ensure their valuable insight from Firstline Workers using Dynamics 365 Remote Assist and Dynamics 365 Layout are built directly into the product. In developing these apps with customers, we are not only looking to make great software—we’re looking to empower the Firstline Workers using the apps.
  • Toyota: With their mixed reality solution, Toyota can now take their existing 3D CAD data used in the vehicle design process and project it directly onto the vehicle for measurements, optimizing existing processes, and minimizing errors. This has transformed a process which previously took one to two days and multiple people to execute into a task requiring four hours and one person.
  • Mae Jemison and Defying Gravity: Women in Space: This year, Microsoft, Dr. Jemison, and Smithsonian Magazine joined forces with the Intrepid Sea, Air & Space Museum to sound a rallying cry for representation and inclusion on Smithsonian Magazine’s Museum Day. They came together to create Defying Gravity: Women in Space, a mixed reality experience at the Intrepid that shares the stories of previously unsung women who’ve made critical contributions to the US space program.

What the future holds

Wow! What a year! As we look to 2019, I see so much opportunity for us to work with all of you to continue to define what the future of computing looks like with mixed reality. There are opportunities to reshape the way we work, learn, and visualize our world. Here are a few areas I am particularly excited about:

  • For many of our customers, equipment and processes are mission critical, and any downtime can have an enormous impact on their business, so ensuring employees are working safely and efficiently is paramount. With Microsoft HoloLens, employees can experience realistic 3D training at scale, and integrated with their existing productivity tools. This provides companies the flexibility to train their employees at lower costs but with high-quality results. This is an urgent area for customers as their workforce shifts, and we see huge opportunity with mixed reality.
  • Beyond Microsoft, the growth of mixed reality as a category this year has truly been inspiring. There are now more end points that allow people to visualize data in the context of the real world, whether that be through a headset or a mobile device. With Dynamics 365, we deeply understand a customer’s workflow and believe there are contextual mixed reality solutions that are relevant across all the devices they use in their work – and we’re going to make sure we’re ready to help.
  • Through the insights we’ve learned from our customers and partners about their industries and workflows, it’s clear to us that there’s an incredible opportunity to better connect their physical space into their business applications. We believe that our customers will benefit from real-time insights from sensors, IoT devices, and computer vision enabled devices. More to come here as we learn about the highest value scenarios.

Looking forward to working with you all in 2019!

Posted on Leave a comment

Toyota and thyssenkrupp breaking new ground with mixed reality

It’s been almost two months since we officially joined the Dynamics 365 family with the general availability of our first two mixed reality business applications: Dynamics 365 Remote Assist and Dynamics 365 Layout.

We are thrilled with the response from our partners and customers since our launch. It’s so much fun learning from and hearing about how others are forging their way using mixed reality.

For this month’s post, I wanted to take a quick moment to highlight a couple customers that are early adopters and long-term partners: Toyota and thyssenkrupp. Both companies are breaking new ground while helping us usher in the era of mixed reality.


Toyota is well known for their values of quality, safety and continuous improvement. With significant amounts of 3D CAD data, they understand the power of bringing the digital and physical worlds together. When they learned of mixed reality and Microsoft HoloLens, they began to look at processes that could be transformed.

As part of the painting process for their vehicles, Toyota performs a process called “film coating thickness inspection” to manage the thickness of the paint to ensure consistent quality of coating on every vehicle — a process that is performed regularly, even after mass production has started.

Before using HoloLens, Toyota performed film coating thickness inspection manually with automobile-sized paper patterns, similar to those used in dressmaking. The automotive pattern has approximately 500 holes, punched as measurement points based on existing CAD drawings. Because of the size, complexity and manual nature of the patterns, they were often error prone and took days to create. And, due to risk of dust and dirt from the paper patterns, the vehicles being inspected had to be removed from the manufacturing facility, adding more than a day to production timelines.

With the HoloLens solution, Toyota can now take their existing 3D CAD data used in the vehicle design process and project it directly onto the vehicle for measurements, optimizing existing processes and minimizing errors. In addition, the process can be standardized and replicated across their global operations. Using mixed reality, a process which previously took one to two days and multiple people to execute now takes four hours and one person.

Toyota proved the value of mixed reality and is currently trialing Dynamics 365 Layout to improve machinery layout within their facilities and Dynamics 365 Remote Assist to provide workers with expert support from off-site designers and engineers.


We’ve partnered closely with thyssenkrupp since the very early days of HoloLens, jointly developing mixed reality solutions that could be implemented at scale. First, in the earliest iterations of Dynamics 365 Remote Assist, they were able to drastically improve response time, increase efficiency and raise elevator uptimes for many of their 24,000 thyssenkrupp elevator-service engineers by providing hands-free remote assistance — the length of their service calls was reduced fourfold. We continue to work closely with thyssenkrupp on current and future Dynamics 365 mixed reality applications as well as extending existing custom applications into broadly available solutions.

For example, thyssenkrupp has recently announced the broad rollout of HoloLinc, a first-of-its-kind, fully digitized sales process for the stair lift industry. After a pilot in The Netherlands with more than 300 successful installations to date, the HoloLinc solution is being rolled out from this month onward in the U.K., Germany, Belgium, France, Italy, Spain and France, with Norway and Japan to follow next year.

The rollout on a global level is seen as a major milestone in mass industrialization innovation. To date, thyssenkrupp Elevator has equipped 120 salespeople with the HoloLinc toolkit, which comprises a Microsoft HoloLens, a tablet, a portable printer, and other technical accessories. What’s most exciting is that HoloLinc improves a process for customers who need it most, as mobility issues have a huge impact on quality of life. By using mixed reality, a process which formerly took around 40 to 70 days can now be done in just 14 days. Now that is true digital transformation of a long-standing process!

I am looking forward to sharing more about the work our customers and partners are doing with mixed-reality business applications. As always, I’m available on Twitter (@lorrainebardeen) and eager to hear about what you’re doing with mixed reality.

Posted on Leave a comment

Bringing mixed reality and Dynamics 365 together

Hey everyone!

I hope this note finds you well. For those who have been following and participating in our mixed reality journey, welcome to our new home in the Dynamics 365 blog! For those new to us, on behalf of everyone who works on mixed reality business applications at Microsoft it is our pleasure to meet you.

My name is Lorraine Bardeen, and I am the general manager of engineering for Mixed Reality Business Applications at Microsoft. I have the pleasure of working for James Phillips in the Business Applications Group and the privilege to lead a world-class team of engineers working to bring mixed reality to people and organizations across the globe.

I have been working on the mixed reality business for over six years and I still show up at work every day excited to work with customers, partners and developers to innovate and solve real problems using mixed reality. With mixed reality, we can understand data in context and simplify workflows to extend human ability. When this happens workers feel more effective, businesses see more progress and everyone has a chance to participate in the modern workplace.

Today marks an important day for all of us on the mixed reality business applications team. Today, we officially mark general availability of our first Dynamics 365 business applications: Dynamics 365 Remote Assist and Dynamics 365 Layout.

To celebrate this important milestone I wanted to write this post and share more about why mixed reality and Dynamics 365 have come together, what people can expect from these first mixed reality business applications, and how they are already being used and deployed.

Why mixed reality and Dynamics 365 have come together

A couple of weeks ago I had the opportunity to speak to press and analysts in San Francisco about mixed reality business applications at Microsoft. As part of that moment in time, Alysa Taylor talked about our vision for business applications and I had the chance to talk specifically about mixed reality. In particular I talked about why mixed reality and Dynamics 365 have come together and I wanted to share some of that with you here as well.

With Dynamics 365, customers are reimagining their business processes by leveraging modern, unified, intelligent and adaptable solutions that bring together the vast amounts of data across their organization to empower their employees with new tools to help them feel more effective at work. Factory and field service worker. Patient and provider. Storefront and supply chain. No silos — just customers, products, employees and data more closely connected than ever before. Mixed reality represents a totally new part of this solution because of the capabilities it provides.

We recognize that these employees need information in context to apply their knowledge and craft. Not only on a 2-D screen — but information and data in context, at the right place, and at the right time, that they can use hands-free so employees can produce even greater impact for their organizations. We’ve focused on integrating these mixed reailty business applications with the common data service that underlies Dynamics 365 so that companies get the full value of high-value data connecting the full workflow.

Introducing the general availability of Dynamics 365 Remote Assist

With today’s release of Dynamics 365 Remote Assist we are taking an important step forward in helping address some of the current, unmet needs of Firstline Workers. Firstline Workers represent more than 2 billion people in roles that make them the first points of contact between a company and the world it serves, between a company and its products. They are often the first to engage, the first to represent a company’s brand, the first on the scene to address a problem.

With Dynamics 365 Remote Assist we can enable technicians and remote experts to solve problems in real time with heads-up, hands-free video calling, annotations, and file sharing. By identifying and addressing issues accurately we can also eliminate the need for costly travel expenses while improving operational efficiency.

Organizations will be able to communicate securely with industry-leading identity and security measures, including Azure Active Directory. In addition, they will be able to leverage work order data from Dynamics 365 Field Service on-site using the common data service.

For business decision-makers looking for new ways to empower their employees and create more collaborative ways of problem-solving, this is pretty exciting stuff! But enough talking about it — here is Microsoft Dynamics 365 Remote Assist in action.

Introducing the general availability of Dynamics 365 Layout

Lastly, with today’s release of Dynamics 365 Layout we are now providing our customers and partners with a way to visualize room layouts in real-world scale. The ability to walk through proposed layouts in physical space or virtual reality. Review and make changes with stakeholders in real time, saving time and money. Move, resize and rotate 3-D models to edit layouts on the spot. Here is a peek at Dynamics 365 Layout in action.

Mixed reality business applications being deployed

About four months ago we released the public preview of our two mixed reality business applications. During that time, we had the chance to work with and learn from many great companies, and I wanted to take a moment to spotlight a couple of them here.

ZF Group

A great way to see Remote Assist and Layout in action is through the work of ZF Group, a German car-parts manufacturer headquartered in Friedrichshafen. An early partner on our journey, ZF Group has been working with us over the past few months to help ensure these apps, even in preview, are crafted with insights from those who will be using them daily to get their work done. In developing these apps with customers, we are not only looking to make great software — we’re looking to empower the Firstline Workers using the apps.

Learn more about how ZF Group uses Microsoft mixed reality tools.


Chevron is already achieving real, measurable results with its global HoloLens deployment. Previously it was required to fly in an inspector from Houston to a facility in Singapore once a month to inspect equipment. Now it has in-time inspection using Dynamics 365 Remote Assist and can identify issues or provide approvals immediately. In addition, remote collaboration and assistance have helped the company operate more safely in a better work environment, serving as a connection point between firstline workers and remote experts, as well as cutting down on travel and eliminating risks associated with employee travel. Learn how Chevron is deploying Microsoft Dynamics 365 Remote Assist and Microsoft HoloLens.

This is just the beginning! We will have a lot more to share in the months ahead, and I am looking forward to sharing more then.

As always, I’m available on Twitter (@lorrainebardeen) and eager to hear about what you’re doing with mixed reality.

Learn more

Posted on Leave a comment

What mixed reality’s amazing new health frontier means for you

In 1992, scientist and technologist Louis Rosenberg created Virtual Fixtures, one of the first augmented reality systems ever developed. By using a full upper-body exoskeleton, the wearer was able to control two physical robots, while innovative optics aligned the robot arms as an overlay of the user’s own arms.

In the nearly 30 years that followed, technology has advanced considerably, enabling fully immersive “virtual-reality” experiences, as well as “mixed reality”—the result of blending the physical world with the digital world.

While mixed reality is still a relatively new technology in health, it has the potential to make a significant impact on patient care. Its unique ability to project visualizations into physical space and its low barrier of entry is spurring health organizations to experiment in ways that are incredibly promising. Here are three examples of providers that are pioneering mixed reality in medical education, the perioperative pathway, and virtual care.

Enhancing medical education by helping students see the human body in three dimensions

While medical students have traditionally learned through textbooks and hands-on training, this approach has its disadvantages, such as a lack of real-world exposure to multiple anatomical variations. For practicing physicians, on-the-job training is often conducted via mannequins and simulators, which are an improvement over textbooks, but even these sophisticated tools have limitations. Each patient is different, and while mannequins are helpful, nothing beats actual patient care for learning.

Mixed reality enhances physician education by combining the anatomical and procedural to create a more robust education platform. Case Western Reserve University in Cleveland is utilizing mixed-reality devices to accelerate their medical students’ grasp of anatomy. With mixed reality, students can visualize the muscles on top of a skeleton and understand the body’s different layers. They can project any piece of anatomy digitally and examine it in three dimensions, move it around, or make it translucent to see through to what lies underneath.

As a result, students have more freedom to experiment and explore the ways anatomical systems work together, helping them build confidence and empowering them with stronger working knowledge of anatomy.

Expanding the opportunity to learn in a new dimension changes the way medical students and physicians see their patients and the world, opening new avenues to approach medicine from a “hands-on” perspective, not just a theoretical one.

Delivering new intelligence to the perioperative pathway by simulating the physical world

When approaching complicated surgeries, each scenario is unique. Some procedures are more complex than others due to the unique characteristics of the patient’s anatomy and the type of surrounding tissue and organs. A growing number of surgeons have already adopted innovative methods like three-dimensional (3D) printing to prepare for the intricacies of each surgery, but this approach is challenging to scale and hinders collaboration among surgical teams.

Mixed reality takes this innovation further, enabling surgeons to interact with an accurate digital representation of a patient’s unique organ structure, as well as collaborate with their teams to orchestrate and rehearse procedures. The University of Oslo is leveraging mixed reality to plan complex procedures, such as liver surgery. By creating a digital 3D model of the patient’s liver from a computed tomography (CT) scan, surgeons can move, scale, rotate and isolate different parts of the organ, as well as switch layers of the model off and on with simple hand gestures. Multiple surgeons can also share the same experience through separate devices.

The new technique enables doctors to navigate around the patient’s other organs and leave more of their healthy liver tissue undisturbed, improving their ability to withstand surgery during treatment. Researchers are looking for ways to apply this technology to patients undergoing other complicated procedures such as cardiac surgery.

Applying mixed reality to perioperative planning enables a customized approach for each patient, offering surgeons enhanced visibility into the patient’s unique anatomy. This approach can improve each individual surgery, helping surgical teams prepare more effectively to create the best possible outcome for each patient.

Bringing doctors and patients together virtually when they can’t be in the same place

Within certain patient populations, connecting patients with providers can be time-consuming and costly. For instance, getting to a provider’s office is a major challenge for elderly patients who lack transportation. Those in remote areas often face similar challenges in making the time to travel and finding transportation.

Through mixed-reality devices, doctors can provide high-quality care to patients in their own homes, saving time, money, and hassle while making them feel more comfortable. Silver Chain Group in Australia is doing this already by using Microsoft’s HoloLens to create an “Enhanced Medical Mixed Reality (EMMR) interface. By having a visiting nurse wear a mixed-reality headset at the patient’s bedside, a doctor can see the patient remotely as if they were in the same room. This extends doctors’ reach beyond the walls of the exam room and enables patients to receive care that would otherwise be difficult to obtain. As the world’s population continues to grow older, demand for community-based care will only increase, and innovative telehealth options will be even more critical.

Augmenting reality eliminates logistical challenges to providing care to remote and elderly populations, putting the patient and the problem in front of doctors’ eyes and removing the friction that can get in the way of patients receiving the right care at the right time.

Enable healthcare innovation with mixed reality

For years, technology adoption in medicine has been driven by doctors seeking ways to improve their techniques as well as by patients demanding high-quality care and convenience. Mixed reality is showing great promise in improving medical education, surgical planning, and even office visits, making healthcare delivery easier, faster, and cheaper while driving better results for patients. To learn more about what Microsoft and its partners are achieving with mixed reality, download our eBook on digital transformation in health.