Augmented World Expo 2018 Recap: The 4 Most Compelling Applications for XR

Augmented World Expo (AWE), the largest conference dedicated to augmented and virtual reality, took place from May 30th to June 1st in Santa Clara, California. The event brings together developers and business leaders at all levels in the XR space for three days of talks and demonstrations showcasing new improvements in technological capabilities and discussing the business case for XR along with interesting applications.

After attending the conference, I was left with the sense that XR needs some strong applications to propel its advancement forward. Only when the business cases exist will the technology rapidly advance.

Here were the four areas in which I saw the most compelling Applications presented in the sense that they added value to a non-gaming application in a way that couldn’t be duplicated easily without using XR.

Military

In her talk, Hard Problems in AR at the US Navy’s Mixed Reality Lab, Heidi Buck, a Director at the lab, outlined a fascinating problem for the US Navy which is being solved through AR: a topside ship gunnery system relies on one officer shouting orders to the gunner on the deck of the ship. The problem is that the deck atmosphere, not to mention the gun itself, can be so loud that the gunner can have trouble hearing the instructions.

A Navy gunner brought this up during an internal innovation brainstorm and proposed a promising solution, now in development at the Mixed Reality Lab. It’s a heads up display for gunners, showing where and how many shots to fire overlaid on their field of view.

This example is a perfect use for heads-up displays as it’s difficult for the gunner to get the information they need otherwise. They can’t look back and forth from their target to a set of instructions and they sometimes can’t hear an officer over the noise level on the deck.

In addition to the use of AR for real time heads-up displays, VR training may be a useful tool for the military. Jay Fraser, a Marine Corps Infantry Officer, highlighted the tool in his talk, Transform Military & First Responder Training with VR. He outlined a case for using immersive VR training for active duty soldiers to learn procedural maintenance, dismounted troop operations, and cultural and soft skills.

One particular example of cultural skill training was a possibility for soldiers to learn the cultural customs of the people in the regions to which they’d be deployed. That way, they can rehearse how to respectfully interact with locals during their tour and practice ways to collaborate with local leaders to best protect the area and make an impact from the moment they arrive.

Sports

Cathy Craig, a Director at the Movement Innovation Lab at Queen’s University Belfast, showcased the use of VR for training and analysis in sports in her talk, Using VR to Study Decision-Making in Sport: What We Need to Consider. Decision-making in sports often happens quickly in the moment and athletes aren’t always able to articulate their thought processes afterwards. In Craig’s lab, they create realistic VR sport experiences and then study the athletes’ brains while they react to the situations to better understand the decision-making process. This research can be used to find trends and improve training techniques.

As Craig pointed out, however, in order to get meaningful results, there are many factors to consider while designing VR experiences for sports research. The virtual environment has to be accurate enough to evoke a realistic response in the user, but there cannot be only one situation presented.

If the users just learn how to respond to one situation, the researchers cannot evaluate trends in their reactions. There needs to be a variety of experiences. Additionally, Craig called for extended reality, including haptics, to be involved in order to realistically (but not tortuously) simulate pain in athletics— getting hit on the field, for example. She argued that without proper aversion to actions that are painful in real life, the athlete’s actions won’t be true to life.

With appropriately realistic and varied XR experiences, researchers can provide two main benefits to athletes. First, they can show them the trends in their behavior so that they can adapt training and improve their skills. Second, they can help athletes who fear a certain situation (such as one in which they were previously injured) to see the situation in a safe way and gain the confidence to return to play.

Marketing and Entertainment

There is an obvious novelty to AR and VR (especially with the onslaught of AR apps that have ensued as a result of the release of ARKit and ARCore), which lends itself to marketing and entertainment. Right now, it’s easy for companies to make a splash by doing something interesting in this space because it’s all exciting for consumers.

At my company, ISL, for example, we’re experimenting with bridging the gap between the digital and physical worlds by creating AR apps that interact with connected hardware. I presented the concept at AWE in my talk, AR-IRL: Augmenting Reality Beyond the Visual, as it could be an intriguing idea for a product activation.

Novelty aside, there is also a sense of storytelling and immersion as an extension of the screen which leads to an interesting Applications in marketing and entertainment. As Sandra Lopez of Intel said on a panel called Engaging Sports Fans Today with AR/VR/MR, “Storytelling is no longer linear.” Her fellow panelists, Mounir Zok, George Foster, and Louisa Spring of N3XT Sports, Stanford Graduate School, and Vari Parks, respectively, echoed that sentiment by acknowledging that fans want authenticity and behind the scenes access to players, so AR and VR experiences could give them more dimensions of the story.

As another example of an extended storytelling experience, Kalan Ray, a Development Supervisor at Magnopus, presented Bringing the Land of the Dead to Life: The Making of Coco VR. Magnopus worked with Disney-Pixar to create a VR world that was both a recreation and an extension of the world in the film, Coco. By letting users explore the world from the movie and take part in new activities and games within it, they drove user engagement and created a brand new immersive storytelling experience.

Conservation

The idea of XR as a conservation tool emerged in a few of the talks at Augmented World Expo. In a panel titled Preserving the Past to Benefit the Future, digital preservation experts Leslie Hartzell and Lon Addison discussed the role of XR in digitally conserving cultural heritage.

Specifically, they looked at how XR can expand access to historic sites by allowing anyone to “visit” without physically being there. There is also an opportunity to use XR to increase and diversify the types of interactions the visitors have at the site, promoting understanding and engagement.

Examples of these opportunities for conservation in XR came into focus in two other talks. In The Walt Disney Archives Goes Virtual, Steve Chapman, a Prototype Engineer at The Walt Disney Studios, talked about how he worked with Disney archivists to document and model everything in Walt Disney’s office and create a VR version for D23 convention attendees in Anaheim and Tokyo.

Chapman and his team modeled the office with extreme attention to detail, using many different methods of photogrammetry to capture all of the objects in the room. Where photogrammetry was insufficient- namely for highly reflective objects- they measured with a micrometer and developed 3D models from scratch. Chapman then took the modeled objects and room and added historical information and interactions to allow visitors to walk around and virtually pick up objects to learn about them. He made a couple of excellent points about the value of the digital preservation in this experience.

First, very few people actually get to go inside Walt Disney’s real office, so the VR version opened up access to it. Second, Burbank, CA, the site of the real office, lies on the San Andreas fault, so it’s very possible it could be destroyed someday by an earthquake. Now, with Chapman’s prototype, the office will still be preserved.

In Bringing the Terracotta Warriors to Life Through Augmented Reality, Brian Yetzer discussed an AR preservation application that was used at the Terracotta Warriors of the First Emperor exhibit at The Franklin Institute in Philadelphia. Using image recognition, visitors could use the exhibit itself to trigger AR interactions and then, in effect, bring the Terracotta Warriors home by placing a 3D model in their surroundings later.

This app allowed visitors to extend their interaction with the exhibit beyond their time at The Franklin Institute. To pull this off, Yetzer and his colleagues worked with subject matter experts to ensure their extension of the experience was historically accurate. Like Chapman’s work with the Disney Archivists, this shows how technologists’ collaboration with historians is essential to making these types of conservation efforts meaningful.

Check out the rest of AWE’s agenda for the breadth of technologies and applications that were on display at the conference. Youtube videos of all of the talks will also be available- some have already been uploaded to AWE’s channel.

See something there that you think should be on this list or have another compelling Applications for XR applications? I’d love to discuss- get in touch at [email protected] or @trish_o_connor.