When the user of a Mixed and Augmented Reality (MAR) device is in an experience, the real world with the synchronized digital assets are perceived. The user’s senses continuously detect the visual and/or audio signals from the combination of the real world and digital assets in real time.
As Mixed and Augmented Reality experiences become more common there will be a natural need on the part of developers, users, employers and sponsors to capture and, in some circumstances to share or publish accurate archives of MAR experiences. For example:
- Workplace-based inspections, particularly those necessary to produce certification of a process being performed in compliance with corporate or regulatory policies, would be captured.
- A public safety officer using MAR to aid in posing questions when interviewing a suspect or witness may need to review the interview archive at a later time.
- Those seeking to capture tribal knowledge or best practices may also use an MAR experience capture system for sharing with novice users or students in the context of skills training or performance enhancement.
- The players of a MAR game may want to record, review or share their best moves.
The captured experience file will show the real world and digital content from the user’s point of view.
To support these capabilities, systems connected to the MAR display and sensors, and a variety of recording and file storage technologies are needed. Today, some preliminary tools and architectures are proposed.
The first available are designed to work with HoloLens, based on prior work published by Microsoft Research (see resources for developers). The field as a whole is very young and requires a great deal of further research.