Mixed Reality: the future is coming…

"Mixed Reality Drone Race @ voestalpine Linz" by Ars Electronica is licensed with CC BY-NC-ND 2.0.
"Mixed Reality Drone Race @ voestalpine Linz" by Ars Electronica is licensed with CC BY-NC-ND 2.0.



Soon, the much-anticipated game ‘Cyberpunk 2077’ will be released. The game depicts the future human world of cyberpunk style. Neon lights, computers and virtual information occupy every corner of the city. Between reality and fantasy, people are always immersed in the fantasies of the future in untouchable art. In fact, some of the features players expect in the game world are being quietly implemented in the real world. That is MR. Many people have heard about VR or AR in many ways, but never really understood them to the extent that MR seems to be a new term. Some users may even confuse them. No matter what kind of ‘R’ it is, they are already around us. From a macro point of view, they are the tip of the future information form. They will realistically change people’s lives and media usage.

This essay will introduce and elaborate on the place of MR in the real world and how we, as ordinary users, should perceive them. In what way they have entered history. What their respective uses are and what their prospects and applications are. Who has benefited and who has not?


VR, AR, MR? The artificial ‘Realities’.


To explain MRVR and AR cannot be ignored, as they are related.

The term “Virtual Reality” is credited to Jaron Lanier, founder of VPL Research (Machover and Tice, 1994) and its prototype was designed in the 1950s, and a machine called Sensorama, designed by Cinematographer Morton Heilig successfully used a variety of technologies to create an immersive, multi-sensory experience (BARNARD, 2020). Most of the VR products on the market now are for video games.

“Exploring the Universe in Virtual Reality” by NASA Goddard Photo and Video is licensed with CC BY 2.0.

Augmented Reality, AR, is closer to real-life. It is a technology based on Real Environment. AR is thought to have been invented in 1968 with the first head-mounted display system developed by Ivan Sutherland (Poetker, 2020). It is similar to MR, but it mainly focuses on visual information.

“Home Made Augmented Reality” by T. Geersing is licensed with CC BY 2.0.

MR is a relatively new technology. While VR and AR are not being used in the same direction, the fact remains that neither has a substantial effect on reality. Under such demands, the ensuing solution is MR, also known as Mixed Reality. MR will be used to describe the spectrum that exists between reality and the digital virtual, including the technologies of AR and VR (Frank Moore and Gheisari, 2019).


Simplified representation of a RV Continuum. (Milgram, Takemura, Utsumi and Kishino, 1995)

This figure shows the definition of MR. As shown in the figure, the applications of MR needs to adopt the functions of AR and VR.

Although MR and AR have many similarities, the concept of MR is broader than that of AR. The main reason is that it places more emphasis on the interaction between reality and virtual stuff. With MR’s technology, users can manipulate and interact with virtual objects rather than just seeing it. Besides, MR offers a dynamic experience; It allows the users to self-adjust the required balance between reality and the virtual world depends on the products, which is something that AR and VR cannot do.

Where devices exist on the Mixed Reality spectrum (Microsoft, 2020).


Who are the players in this market?

How MR will change us?


To this day, MR is still the new technology. Compared to VR and AR, two technologies that are already relatively mature, MR is still in the development stage. However, there are already several MR products on the market as well as a clearer and more clearly defined development direction.

With Microsoft at the forefront, companies in many countries have been developing and offering their MR products since the Hololens, which went on sale in 2016, began to introduce the concept of MR fully. From 2016 to 2019, up to $3500 down to $ 30. MR products have begun to have a development route to dive into the lower price. At the same time, the form of the product has also begun to diversify.

MR products on the market (Worldwide)

  •          Microsoft Hololens (1 & 2)                       $ 3,000/3,500                        2016/2019
  •           Occipital Bridge                                          $ 399                                     2016
  •   Dimension NXG AjnaLens                              $ 1,500                                     2018
  •            Magic Leap One                                        $ 2,295                                    2018
  •             Zappar Zapbox                                            $ 30                                       2018
  •                Nreal Light                                              $ 1000                                     2019

Data from The 9 best mixed reality headsets of 2019 (Noble, 2019)

So what is the future application of MR and how it will change our lives? One key answer is education and remote control.

An essential educational value of virtual labs is the reduction of the risks associated with operator error and the opportunity to experiment and practice without being exposed to hazards (Müller et al., 2007). In industrial experimentation and education, new trainees are often unfamiliar with machines. MR is the solution to this problem. Another example is the medical surgery. The MR Surgery Simulator gives novice surgeons access to more advanced technology. Watch and experience complex surgeries without having to step into the operating room (Hu et al., 2019).

Both of these examples show that, with this technology, trainees can use virtual machines to simulate the actual operating environment while keeping themselves safe. On the other hand, MR is more interactive than AR, which makes simulations feasible. This means that the trainees can not only see the simulation in action but also practice it at the same time. In addition, the visual peculiarities of MR will also drastically change the visual form of digital media.

Of course, MR technology does not benefit all industries. Whether it is AR or MR, the essence of the technology is to replace reality with the virtual. MR does not directly benefit the industries that are replaced. These include, for example, the furniture industry, the education industry and even the tourism industry. The user of MR may be able to replace some of his or her real expenses with virtual goods.

Narrow the focus from the industries to the individual. The impact of MR technology on the individual is equally huge. In the future world, as shown in the Youtube video ‘Hyper-Reality‘, the information will be much more open and transparent. It’s actually a double-edged sword. In the era of information transparency, MR technology does bring a lot of conveniences. For example, automatic translation, automatic product information search and advertising.



However, transparency also brings problems frequently. Highly popular types of AR applications in which they can collect intimate biometric data like facial expressions, speech data, and even retina patterns (Marr, 2020). In a highly connected world and the ease of information made possible by AR and MR, security and privacy breaches have become an inescapable problem. On the other hand, New techniques may exist for applications on AR systems to trick the user into interacting with elements. For example, AR applications may attempt to trick the user into interacting with physical objects (Roesner, Kohno and Molnar, 2014). It’s important to acknowledge that while MR is bringing about future changes, excessive convenience may lead to more deceptive, misleading information. At the same time, it will become more difficult to maintain network security under the guise of virtualization. More importantly, it is still a difficulty of how to achieve uniformity and legitimacy of the MR network under different governmental regimes in different countries, e.g. China and India with their highly regulated networks, and the USA and Australia with their emphasis on freedom of expression.





MR is not a stand-alone technology; it is not only related to VR and AR, but also the Internet and media. Throughout history, from the virtual visualization of AR to the virtual manipulation of MR, human society has become increasingly active and dependent on virtual information.

Admittedly, MR is not without its pitfalls. Most of its pitfalls are in the topic of cybersecurity. While this is not the intent of the technology, it is clear that MR amplifies these problems. Until cybersecurity technology and related legal policies are perfect, I do not think MR technology will spread to the consumer market.

At the same time, however, this essay argues that MR will still be the social and media form of the future and that there is no substitute for its convenience and innovation. The future of mixed reality can go anywhere that video and GPS can reach, which means that the breadth of human knowledge will no longer be limited to protected museums or dark computer labs (Stapleton, Smith and E. Hughes, 2016). On current trends, the future of MR is still very optimistic. For media, businesses and individuals alike, the combination of the virtual and the real could provide sufficiently robust benefits and legitimacy.


Reference List


Machover, C. and Tice, S., 1994. Virtual reality. IEEE Computer Graphics and Applications, 14(1), pp.15-16. Available at: <https://www.semanticscholar.org/paper/Virtual-reality-Machover-Tice/e0f9362543791a3fe9c745dc95f725caff0ea9a9> [Accessed 25 October 2020].

BARNARD, D., 2020. History Of VR – Timeline Of Events And Tech Development. [online] Virtualspeech.com. Available at: <https://virtualspeech.com/blog/history-of-vr> [Accessed 25 October 2020].

Poetker, B., 2020. A Brief History Of Augmented Reality (+Future Trends & Impact). [online] Learn.g2.com. Available at: <https://learn.g2.com/history-of-augmented-reality> [Accessed 25 October 2020].

Frank Moore, H. and Gheisari, M., 2019. [online] Mdpi.com. Available at: <https://www.mdpi.com/2313-576X/5/3/51/pdf> [Accessed 25 October 2020].

Microsoft, 2020. Where Devices Exist On The Mixed Reality Spectrum. [image] <https://docs.microsoft.com/en-us/windows/mixed-reality/discover/mixed-reality> [Accessed 25 October 2020].

Müller, D., Wilhelm Bruns, F., Erbe, H., Robben, B. and Yoo, Y., 2007. Mixed Reality Learning Spaces for Collaborative Experimentation. A Challenge for Engineering Education and Training, 3, p.15. Available at: <https://www.researchgate.net/publication/26488981_Mixed_reality_learning_spaces_for_collaborative_experimentation_A_challenge_for_engineering_education_and_training/fulltext/0e605516f0c46d4f0ab0ee66/Mixed-reality-learning-spaces-for-collaborative-experimentation-A-challenge-for-engineering-education-and-training.pdf> [Accessed 25 October 2020].

Marr, B., 2020. The Important Risks And Dangers Of Virtual And Augmented Reality. [online] Forbes. Available at: <https://www.forbes.com/sites/bernardmarr/2019/07/17/the-important-risks-and-dangers-of-virtual-and-augmented-reality/#132639853d50> [Accessed 27 October 2020].

Roesner, F., Kohno, T. and Molnar, D., 2014. Security and privacy for augmented reality systems. Communications of the ACM, 57(4), pp.88-96. Available at: <https://homes.cs.washington.edu/~yoshi/papers/arsec-cacm2014-preprint.pdf> [Accessed 25 October 2020].

Hu, H., Feng, X., Shao, Z., Xie, M., Xu, S., Wu, X. and Ye, Z., 2019. Application and Prospect of Mixed Reality Technology in Medical Field. Current Medical Science, 39(1), pp.1-6. Available at:<http://website60s.com/upload/files/1-application-and-prospect-of-mixed-reality.pdf>

Matsuda, K., 2016. HYPER-REALITY. Available at: <https://www.youtube.com/watch?v=YJg02ivYzSs&t=102s> [Accessed 27 October 2020].

Stapleton, C., Smith, E. and E. Hughes, C., 2016. The Art of Nurturing Citizen Scientists through Mixed Reality. [online] p.10. Available at: <https://www.researchgate.net/publication/221221113_The_Art_of_Nurturing_Citizen_Scientists_through_Mixed_Reality> [Accessed 28 October 2020].


Embedded content References


“Mixed Reality Drone Race @ voestalpine Linz” by Ars Electronica is licensed with CC BY-NC-ND 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc-nd/2.0/

Noble, S., 2020. The 9 Best Mixed Reality Headsets Of 2019 (March Update). [online] Aniwaa. Available at: <https://www.aniwaa.com/buyers-guide/vr-ar/best-mixed-reality-headsets/> [Accessed 27 October 2020].

Milgram, Paul & Takemura, Haruo & Utsumi, Akira & Kishino, Fumio. (1994). Augmented reality: A class of displays on the reality-virtuality continuum. Telemanipulator and Telepresence Technologies. 2351. 10.1117/12.197321. Available at: <https://www.researchgate.net/publication/228537162_Augmented_reality_A_class_of_displays_on_the_reality-virtuality_continuum/link/0c96052ade63de29c0000000/download>

“Exploring the Universe in Virtual Reality” by NASA Goddard Photo and Video is licensed with CC BY 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by/2.0/

“Home Made Augmented Reality” by T. Geersing is licensed with CC BY 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by/2.0/

About Yuanbin Chen 2 Articles
USYD Student. Majored in Japanese studies