A Deep Dive into Mixed Reality and Its Impact on Human Activities

• Environmental understanding: involving spatial mapping and anchors.

• Human understanding: encompassing hand-tracking, eye-tracking, and speech input.

• Spatial sound: enhancing auditory experiences.

• Locations and positioning: bridging physical and virtual spaces.

• Collaboration on 3D assets: facilitating collaborative work within mixed reality spaces.

These developments have propelled mixed reality into a realm that extends beyond visual displays, enriching the overall user experience with a comprehensive blend of physical and digital elements.

MR vs VR vs AR: Differences in Worlds and Hardware

• Environment: In Virtual Reality (VR), you are completely isolated from the real world. In AR vs MR, the real world can be integrated into the experience and either enhanced or blended with digital elements.

• Hardware: A device is essential for all three experiences. In VR, you wear a headset that isolates you from the real world. In AR, a device like a smartphone might be used to drive the experience. In MR, glasses or a headset, such as Microsoft HoloLens, HTC Vive, or Magic Leap, allow you to maintain awareness of the real world. For instance, Microsoft Corporation's Mixed Reality Portal app is a current, accessible example of MR. When you plug in or wear your headset, Windows can automatically launch the Mixed Reality Portal, starting whenever you open an application designed for mixed reality.

Exploring immersive technologies? Don't miss out on XR technology!

Photo credit: Microsoft

Use Cases of Mixed Reality

Entertainment

The advancement of AR and VR technology has caught the attention of the film and big-screen industries, sparking interest in incorporating these technologies into their movies. The concept of interactive storytelling, blending elements of a game with a movie, opens up new possibilities for mixed reality. A notable example is the Netflix series "You vs. Wild," featuring adventure survivalist Bear Grylls, allowing viewers to make decisions and influence the story's outcomes.

Mixed reality entertainment is already making strides, with companies like Magic Leap, Lucasfilm, and Industrial Light And Magic exploring its potential. Magic Leap, for mixed reality examples in the amusing industry, is venturing into mixed reality cinema using a technology they term Dynamic Digitised Lightfield Signal. This innovative tech directly projects images into the eye, bypassing the need to bounce off objects before reaching the eyes. This illusion tricks the brain into perceiving the presence of objects, even though they are, in reality, projections.

Construction

Managers can utilize MR to provide cues to workers regarding pending tasks. For instance, a supervisor could place a virtual pin to indicate that a particular piece of equipment requires attention. Workers would receive notifications and use their headsets to access schematic diagrams, facilitating the diagnosis of the issue and determining the necessary repairs.

Manufacturing

Inspectors can leverage mixed reality headsets during plant tours to observe real-time data on specific mechanical components. Details regarding the performance and energy consumption of the components can be transmitted using Near Field Communication (NFC) technology, automatically updating as the inspector shifts their gaze to different components.

Design

Designers can employ MR to engage with prototypes of new products before their physical construction. For instance, spacecraft engineers could utilize MR technology to visualize and interact with life-sized digital prototypes of a new Mars rover, enabling them to pinpoint potential design issues. This approach helps streamline the overall development process, minimizing both time and costs.

B2B communication

Global enterprises can employ MR to conduct virtual face-to-face meetings with product teams located in different geographic regions. Participants have the ability to observe a life-sized, 3D representation of the discussed product, and the integration of translation services facilitates seamless communication, overcoming language barriers.

Healthcare

Surgeons can use MR for simulated practice in inserting an exceptionally small medical implant into a specific patient's body. It is anticipated that, in the near future, medical students will also employ MR imaging instead of cadavers in anatomy classes.

Supply chain management

A supply chain manager can utilize his smartphone's camera or smart glasses to inspect a pallet, verbally inquire about the pallet's shipping history, and receive an immediate answer displayed on the screen.

Specific Types of Mixed Reality Apps

Enhanced environment apps

One of the most impactful ways mixed reality adds value is by allowing developers to integrate digital information or content into a user's current environment. This approach is particularly popular for apps where the contextual placement of digital content in the real world is crucial, emphasizing the importance of keeping the user's real-world surroundings present during their experience. Users can seamlessly transition between real-world and digital tasks, reinforcing the assurance that the mixed reality apps they encounter are genuinely integrated into their environment.

For examples:

• A mixed reality notepad-style app that enables users to create and place notes around their environment.

• A mixed reality television app positioned for comfortable viewing.

• A mixed reality cooking app placed above the kitchen island to assist with cooking tasks.

• A mixed reality app providing users with an "x-ray vision" effect, with holograms mimicking real-world objects and allowing users to see inside them holographically.

• Mixed reality annotations placed throughout a factory to provide workers with the necessary information.

• Mixed reality wayfinding in an office space.

• Mixed reality tabletop experiences akin to board game-style interactions.

• Mixed reality communication apps like Skype.

Blended environment apps

A blended environment app possesses the capability to recognize and map the user's surroundings, creating a digital layer that overlays the user's space. This thin layer respects the shape and boundaries of the user's environment, but the app may opt to transform certain elements to immerse the user more effectively. Unlike an enhanced environment app, blended environment apps focus on utilizing the environment to encourage specific user behavior or by replacing elements with alterations. This type of app may transform an element into an entirely different object while maintaining the rough dimensions of the base object.

Examples include:

• A mixed reality interior design application that enables the painting of walls, countertops, or floors with a variety of colors and patterns.

• A mixed reality app enabling an automotive designer to layer new design iterations for an upcoming car refresh over an existing car.

• A bed "covered" and replaced by a mixed reality fruit stand in a children's game.

• A hanging lantern "covered" and replaced with a signpost using roughly the same shape and dimensions.

• An app allowing users to blast holes in their real or immersive world walls, revealing a magical world.

Immersive environment apps

Immersive environment apps revolve around an environment that entirely transforms the user's world, transporting them to a different time and space. These environments aim to feel genuinely real, delivering immersive and thrilling experiences that are only bound by the imagination of the app creator. In contrast to blended environment apps, once Windows Mixed Reality identifies the user's space, an immersive environment app may completely disregard the user's current surroundings and replace them entirely with its own. These experiences may also disconnect time and space, allowing users to explore the streets of Rome in an immersive experience while remaining relatively stationary in their real-world space. The context of the real-world environment may not be crucial to an immersive environment app.

Examples include:

• An immersive app enabling users to tour a space separate from their own (e.g., walk through a famous building, museum, or popular city).

• An app orchestrating an event or scenario around the user (e.g., a battle or a performance).

Mixed Reality Portal