Posted on Leave a comment

Seventeenth century French artifact arrives in Seattle for an immersive exhibition, powered by Microsoft

Visitors can explore the Mont-Saint-Michel through an AI and mixed-reality-powered experience at Seattle’s Museum of History & Industry

Museum visitors explore the Mont-Saint-Michel through an AI and mixed-reality-powered experie

SEATTLE — Nov. 21, 2019 Seattle’s Museum of History & Industry (MOHAI) and Microsoft Corp. on Thursday announced the opening of a new exhibit, “Mont-Saint-Michel: Digital Perspectives on the Model,” which features a unique blend of 17th and 21st century technology.

Powered by Microsoft AI and mixed-reality technology as well as the recently released HoloLens 2 device, the interactive exhibition transports visitors into a holographic tour of the picturesque Mont-Saint-Michel, a medieval monastery perched atop a remote tidal island off the coast of Normandy, France.

The virtual experience is complemented by a physical relief map of the Mont-Saint-Michel, an intricate, three-dimensional model of the landmark. Entirely crafted by hand in the 1600s by the resident Benedictine monks, the 1/144-scale model precisely depicts the monument in such intricate detail that maps like this were considered valuable strategic tools to leaders like Napoleon and King Louis XIV, who considered the maps military secrets and hid them from public view.

“The Museum of History & Industry is honored to share this icon of world history, enhanced by leading-edge technology, to create a unique experience born of innovations both past and present,” said Leonard Garfield, MOHAI’s executive director. “More than 300 years separate the remarkable relief map and today, but the persistent human drive toward invention and creativity bridges those years, reflecting the unbroken quest for greater understanding and appreciation of the world around us.”

The opening of the exhibit is timed with the 40th anniversary of the Mont-Saint-Michel being designated as a UNESCO World Heritage Site. This is the first time the relief map, as well as the mixed-reality experience, has been in North America.

“The relief maps were technological marvels of Louis XIV and Napoleon’s time. It’s exciting to see how we can blend old and new technology to unlock the hidden treasures of history, especially for younger generations,” said Brad Smith, president of Microsoft. “This exhibit provides a unique model for preserving cultural heritage around the world, something Microsoft is committed to through our AI for Good program.”

The Mont-Saint-Michel experience is an example of Microsoft’s AI for Cultural Heritage program, which aims to leverage the power of AI to empower people and organizations dedicated to the preservation and enrichment of cultural heritage. Microsoft is working with nonprofits, universities and governments around the world to use AI to help preserve the languages we speak, the places we live and the artifacts we treasure. For example, earlier today Microsoft announced it is working with experts in New Zealand to include te reo Māori in its Microsoft Translator application, which will enable instant translations of text from more than 60 languages into te reo Māori and vice versa. This will be one of the first indigenous languages to use the latest machine learning translation technology to help make the language accessible to as many people as possible. The AI for Cultural Heritage program is the fourth pillar of Microsoft’s AI for Good portfolio, a five-year commitment to use AI to tackle some of society’s biggest challenges.

The relief map is on loan to MOHAI from the Musée des Plans-Reliefs in Paris, which houses more than 100 historically significant and well-preserved relief maps. The relief map of Mont-Saint-Michel is considered the museum’s crown jewel.

“One of the challenges in the history of art is the relationship with the public. To gain the attention, to capture the view or the interest of the public, is not always evident,” said Emmanuel Starcky, director, Musée des Plans-Relief. “With the HoloLens technology, you have now the possibility to realize immersive experiences in art, where you still see the reality but have more information about it. It will be a unique experience for the American public to discover the relief map, its condition in the 17th century and its evolution through three centuries, as well as reflect on the purpose of those relief maps.”

Drawing from hundreds of thousands of detailed images, Iconem, a leader in the digital preservation of cultural heritage sites, used Microsoft AI to create a photorealistic 3D digital model of the historic structure. Then, French mixed-reality specialists at HoloForge Interactive developed a unique Microsoft HoloLens experience to draw people into the artifact like never before.

The “Mont-Saint-Michel: Digital Perspectives on the Model” exhibit, including both the original relief map and mixed-reality experience, will be on display at MOHAI Nov. 23, 2019 through Jan. 26, 2020.


MOHAI is dedicated to enriching lives through preserving, sharing, and teaching the diverse history of Seattle, the Puget Sound region, and the nation. As the largest private heritage organization in the State of Washington; the museum engages communities through interactive exhibits, online resources, and award-winning public and youth education programs.  For more information about MOHAI, please visit, or call (206) 324-1126. Facebook: Twitter: @MOHAI

About Microsoft

Microsoft (Nasdaq “MSFT” @microsoft) enables digital transformation for the era of an intelligent cloud and an intelligent edge. Its mission is to empower every person and every organization on the planet to achieve more.

For more information, press only:

Microsoft Media Relations, WE Communications, (425) 638-7777,

Museum of History & Industry PR, Wendy Malloy, (206) 324-1126, ext. 150,

Note to editors: For more information, news and perspectives from Microsoft, please visit the Microsoft News Center at Web links, telephone numbers and titles were correct at time of publication but may have changed. For additional assistance, journalists and analysts may contact Microsoft’s Rapid Response Team or other appropriate contacts listed at

Posted on Leave a comment

Smithsonian magazine: Inventor Alex Kipman’s grand vision for how holograms will change our lives

I’m in Redmond, Washington, in a room at Microsoft, pondering an all-terrain vehicle that has a busted engine. I have no idea how to fix it. I’ve never done engine repair before.

But I do have some help: On my head, I’m wearing the HoloLens 2, Microsoft’s “augmented reality” device. It has a see-through visor, almost like the one on a motorcycle helmet, and the HoloLens projects images onto the visor so they appear to float in the air before you.

When I look at the vehicle, the HoloLens flickers to life, and a guide to fixing the engine pops up in the air. A blue arrow points at a tableful of tools, and when I walk over to it, the arrow indicates that I should grab a torque wrench. Once I take that tool, a new arrow appears, beckoning me across the room to a case of bolts. I grab a bolt, and a third arrow shows me where on the engine to install and tighten it. In under two minutes I’ve completed the repair.

The sensation is bizarre, like living in a world of Harry Potter magic. I can even touch the holograms. While I’m doing another repair job, a virtual screen with the face of a remote mechanic materializes before me to talk me through the job. The screen is in my way, though—so I grab it by the corner with my fingers, right there in the air, and drag it off to the side.

It’s weird. It’s fun. And it is, argues Alex Kipman, the Microsoft engineer who invented the device, the future. “I have no doubt that devices like this are going to be the pervasive way of interacting with technology,” he tells me. Around Microsoft, Kipman is famous for pushing these sorts of oracular, sci-fi visions. “It’s kind of inevitable,” he shrugs. “It’s almost obvious.”

HoloLens 2
Built on the breakthrough technology of HoloLens 1, HoloLens 2 is more than twice as immersive and three times as comfortable as its predecessor. (Microsoft)

I met Kipman in his office, where he was wearing a gray sport coat over a T-shirt with an icon of pixellated sunglasses. He has long hair and a beard and when he talks he fixes you with an intense, Delphic gaze. The glass wall behind his chair was festooned with pink doodles made by his 9-year-old daughter, and the room was cluttered with relics of his work, including a square blue robot, a drone, and a gaming computer with a high-powered graphics card. He beckoned me to sit down at a glass coffee table that was also an art object: Inside was a huge pile of sand, on top of which a magnetic mechanism rolls a ball around, tracing hypnotically pretty patterns.

It’s a Kickstarter project he backed. The pattern it’s drawing is from software he wrote, he added. “I created a generative AI algorithm that overnight will scour the internet, and, like, dream the internet—and in the morning whatever the AI created, it puts it on the table.”

Kipman grew up in Brazil, got turned onto software by playing with his family’s Atari 2600 console, and after studying computer science at Rochester Institute of Technology, joined Microsoft in 2001 as a wunderkind. He toiled for years on Vista, Microsoft’s 2007 train wreck of an operating system. Then he shifted into hardware, leading a team to create the Kinect, a newfangled 3-D camera that plugged into Microsoft’s Xbox gaming system and tracked players’ body movements, allowing them to control a game by moving their limbs. It was a hit, selling 35 millions units, and it fired his enthusiasm for reimagining how we use computers.

He assembled another team to build the first HoloLens, which was released in 2016 to surprised enthusiasm. Surprised because augmented reality (or what Microsoft calls “mixed reality”) had recently seen a hostile reaction to Google Glass, a computer and camera mounted on an eyeglasses frame, which critics derided as too creepy and intrusive for everyday life. (People who wore the device were called “glassholes.”) To keep the HoloLens from falling into the creepiness pit, Kipman pitched it as a tool not for socializing but for working. He imagines an airplane mechanic in Japan using the HoloLens to summon a Rolls Royce engineer to help diagnose a busted engine, or a surgeon having hands-free, holographic access to a patient’s X-rays and medical history in the operating room. (Indeed, the recently reborn Google Glass also aims at industrial uses.)

“In our world, nothing really is impossible. Everything at best is improbable,” says Alex Kipman, as he discusses the holographic, augmented reality technology his team at Microsoft is pioneering.

Crafting the HoloLens required feats of miniaturization. One prototype “was like wearing a scuba thing,” laughs Ori Amiga, who develops software for HoloLens. It was shrunken down small enough to wear on your head, but people still complained that it was heavy, and the screen area where holograms appeared was narrow.

For HoloLens 2, Kipman and his team invented tiny mirrors that vibrate 12,000 times per second, generating holograms twice as wide as before. They upgraded to carbon fiber for the device’s body, which is half as heavy as aluminum and far more rigid. The carbon fiber also helps stabilize the delicate electronics in the headset, including dozens of sensors that track exactly where your head is turning or where your arms are. “And I’m talking like micron precision, right?” Kipman says. “Nanometer precision.”

Engineering on that vanishingly small scale is what allows Kipman to think big. His ultimate goal: Replace every screen, from smartphone to tablet to monitor, with the HoloLens or one of the next versions of it. “Why would I have my computer if I have infinite monitors in front of me?” he says. “Why would I have a phone?”

Granted, that vision is still years away. HoloLens 2 is a leap in technology from its predecessor, but “we’ve got a ways to go before we’ve got something that you can wear all day,” Kipman says. Eventually, he figures it’ll be as compact as a normal pair of horn-rimmed glasses. By then, perhaps its sheer ubiquity in the workplace will make it seem acceptable in social life. “You wear them all day,” he says.

When I said goodbye, Kipman argued that if he’s really successful, a reporter like me wouldn’t need to fly to Seattle to talk to him. We could use HoloLens to speak with the intimacy of being in the same room—a sort of supercharged version of Skyping. But why stop there? Maybe, he mused, artificial intelligence will advance to the point that neither space, nor time, nor anything else on this earthly plane can limit whom we speak to, as AI versions of people are preserved and available at a dial for chatting via hologram.

“Wouldn’t it be nice,” he laughed, “if you were in your home, and I had been dead a hundred years, and we were having this conversation?”

Posted on Leave a comment

The making of the HoloLens 2: How advanced AI built Microsoft’s vision for ubiquitous computing

Cloud collaboration

A key advantage of intelligent cloud-powered holographic computing is the ability to share information with others who have a HoloLens or another device with similar capabilities, said Marc Pollefeys, the director of Microsoft’s Mixed Reality and AI Zurich Lab in Switzerland.

Pollefeys is leading a team that develops core computer vision algorithms for a mixed reality cloud service called Azure Spatial Anchors that allows holograms to persist, locked in the real world, for anyone with the appropriate level of access to view.

For example, spatial anchor technology allows a manager in a factory to place holograms next to equipment on an assembly line that contain vital, real-time operating and maintenance information that any credentialed worker with a mixed reality capable device can access.

“If I can only place information that I will see back on my device, it’s probably never worth placing holograms in the world, but if I can annotate the world and afterward anyone else in the company that has the right access can see all of the information, it is suddenly much more valuable,” Pollefeys said.

To create this capability, Pollefeys and his team developed AI computer vision algorithms that process data from sensors to extract 3D geometric information about the environment and piece it together in the cloud to create a digital twin, or map, of the area of interest.

HoloLens has always built up a 3D or spatial understanding of its environment to function. Azure Spatial Anchors creates, refines and shares these maps across devices, Pollefeys noted. That’s why the maps from individual devices are pieced together and stored in the cloud.

“It doesn’t make sense to have that data only on an individual device,” he said. “It is one of those things where I have a little piece of the puzzle, and somebody else has a little piece of the puzzle, and all of the devices together have covered the whole space of interest.”

These maps get denser, more precise and robust over time as different mixed reality capable devices – HoloLenses as well as properly equipped phones, tablets and laptops – map their environment and share the data with the cloud.

For example, the map of the factory floor where the manager left holograms floating over pieces of equipment on the assembly line is steadily refined as more and more credentialed workers view the holograms with their devices.

This capability also enables scenarios such as a meeting between architects and clients to view and interact with a holographic 3D blueprint of a building, each of them with mixed reality capable devices looking at the blueprint from their own point of view as they sit around a table.

Azure contains pre-built services to write applications for these types of experiences on HoloLens and any other mixed reality device, including smartphones and tablets running the iOS and Android operating systems, noted White.

“That collaboration experience isn’t just locked to HoloLens,” she said. “And, the cost and complexity and skillset required to make an application that does something amazing is far down.”

The cross-device and platform capability, for example, enables experiences such as Minecraft Earth, which merges the popular video game with mixed reality in a way that players can build and place virtual structures in the real world that persist so that other players can interact with them on their devices.

“We all get to participate because it is based on using cloud technology that can be understood and interpreted by all different devices,” said White.

Technology that is designed for people

For HoloLens to work as envisioned, the technology that underpins the experience needs to understand the world in ways that are similar to the way people do, Kipman noted.

That’s why he and his collaborators across Microsoft have developed, deployed and leveraged AI solutions throughout the ubiquitous computing fabric, from the silicon in the headset of HoloLens 2 to Azure AI and mixed reality services.

Back at his digital whiteboard, Kipman has now sketched out a vision for ubiquitous computing that is rife with words, boxes, arrows – and a stick-figure picture of two people locked in conversation next to an intelligent device.

That, he says, is the ultimate goal of ubiquitous computing – to get people to interact with other people in natural ways.

To drive home the point, he establishes a moment of intense, deliberate eye contact and says, “Hopefully, you are getting more out of this conversation because you are physically present with me.”

“We could have done this over the phone,” he continues. “We could have done it over Skype. I could have recorded it and sent you a tape. You didn’t choose to do that. You chose to be physically present with me. Why? Because that’s how we do human things.”

“The con is you have to be here at the same time I am here, and we have to be in the same location. The power of this technology is it gives us the ability to displace space and time.”

Top image: Microsoft Technical Fellow Alex Kipman models the HoloLens 2, a sensor-packed holographic computing headset. Photo by Microsoft.


John Roach writes about Microsoft research and innovation. Follow him on Twitter.

Posted on Leave a comment

How REWIND is using HoloLens to take Red Bull Air Race fans closer to the action

Leila Martine, Director of Product Marketing at Microsoft, sees first-hand the excitement HoloLens is causing. “HoloLens is helping companies to work better by empowering staff. Every day we are seeing that workers from a range of sectors can easily collaborate to make complicated problems simple to solve. It really is taking human experiences to the next level.”

Virtual, augmented and mixed reality is becoming increasingly important to companies across the globe. According to market intelligence firm IDC, “worldwide revenues for the augmented reality and virtual reality market will grow … to more than $162 billion in 2020″.

REWIND is at the cutting-edge of that market. The company, which is based in St Albans (Rogers: “We’re only 2.5 miles outside the M25, so we’re London”), was only founded in 2011 but has grown quickly, boasting a team of more than 50 people. The group has already created a multi-award-winning virtual reality spacewalk for the BBC, as well as experiences with Jaguar, Lexus, Nissan, Rolls-Royce, Nike, Stella Artois, Savills and singer Bjork, among many others.

That level of technical experience led to REWIND being added to Microsoft’s HoloLens Agency Readiness Partner programme, which means the company will help other businesses use the mixed-reality headset to transform how they work. Rogers is excited by the possibilities.

“HoloLens is the first device humans have ever had that can augment human intelligence in real time. We have the world’s knowledge at our fingertips with one of these [he holds up his smartphone] but it’s a layer away, a search algorithm away. We have laptops, but what if the second screen is a HoloLens screen? If I can make this [he points to my laptop] as good as talking like we are now, as though I’m really here in the same space as you [when I’m really somewhere else], then why do we need to commute in the way we currently do, why do we all need to be compacted down this end of the country? What if you don’t like the weather so you change it to something else? That’s a little far away, but it’s not that big a leap. HoloLens has some amazing stuff, which is just the tip of the iceberg of what mixed reality can do.”

However, rather than see what he can do with HoloLens in the commercial sector, where the device has been predominantly used since its launch in 2016, Rogers wants members of the public to get their hands on the technology, too.