Down the Rabbit Hole of Augmented/Mixed Reality

alice in wonderland at the criss museum
A virtual world depiction of "Alice in Wonderland at the Criss Museum" by rafeejewell is licensed under CC BY-ND 2.0

As remarked by David Kirby, technological innovation begins with a futuristic idea or “diegetic prototype” that aims to enhance our own existence (Jones, 2020, p. 100). Case in point, Star Trek’s first appearance of the holodeck in 1974 was an inspiration to the possibility of coexisting alternative realities (Zambetta, as cited in Farshid, Paschen, Eriksson & Kietzmann, 2018). Coined in 1994 by Milgram and Kishino (Park, Stangl, 2020), the term “mixed reality continuum” encompasses both augmented (AR) and virtual reality (VR). In the 21st century, the technologies have attracted great attention from the media and information technology industries. In comparison, the main difference between each reality is how immersive information can be ingrained in our actual world (table 1).

Difference in the actual/virtual reality continuum, partially extracted from Farshid et al (2018). All rights reserved.

Table 1: Difference in the actual/virtual reality continuum, partially extracted from Farshid et al (2018). All rights reserved.

History of Augmented/Mixed Reality

The origin of AR can be traced back to the 1960s, when Ivan Sutherland introduced a headworn device (HWD) – the “Sword of Damocles” through which users can view graphics overlaying a physical space (Liao, 2018). Sutherland’s prototype has defined and motivated subsequent and formative research in AR (National Research Council as cited in Liao, 2018). A few decades later, the application entered the arcade game market in 1991 with the first commercial and multiplayer virtual reality game – Dactyl Nightmare. The virtual adventure begins when a headset is donned by players in search for the prehistoric Pterodactyls roaming in the virtual scenery (Cowling, Birt, 2018). The HWD tradition continued to be widely adopted in the 2000s until the Google Glass debut at Augmented World Expo 2013, where the device was encountered with sceptical privacy concerns for the recorded individuals through the glasses’ recording function and built-in cameras that gravely hindered its commercial viability (Liao, 2018). Additionally, wearable tech has also experienced fierce competition against the evolving and successful mobile smartphone devices, penetrating the realm of mixed realities, namely Apple’s iPhone and Samsung Galaxy (figure 1).

Comparison of Virtual and Augmented Reality applications. Source: Rauschnabel (2018)

Figure 1: Comparison of Virtual and Augmented Reality applications. Source: Rauschnabel  (2018)

Contrary to Google Glass’s tragedy at AWE2013, Mobile’s AR adaptations have experienced commercial success with Pokémon GO, which transforms players’ physical environment into an interactive game space via their mobile devices, however, the AR app attracted both positive and negative media coverages upon its launch in 2016 (Jones, 2020, p. 102). Constant contestations between the two early AR models have sparked multiple discussions over which will become the industry flagship and standard application. And recently, Facebook announced its virtual reality research – Project Aria in 2020 (Carter, Egliston, 2020) – its mission:

Independence, mobility and social empowerment for the blind

~ Project Aria, (n.d.)

through the use of wearable prosthetic that enhances spatial perception mobility and object identification.

Power and Control

In addition to the visionary applications and prototype developments emerged throughout the genesis of augmented/mixed reality, mostly from American companies like Google, Amazon and Microsoft in the augmented reality smart glasses (ARSG) domain together with Asia Pacific enterprises like Samsung and Alibaba (Rauschnabel, 2018), AR researches in both software and hardware have been conducted worldwide with over 10,200 articles, still mainly contributed by the United States, China, England and Germany (figure 2), suggesting their dominance of the promising, disruptive application, trademark and patents of AR technology. Due to the perception that AR will be the next major computing platform/application, it has been reported that $800 million of venture capital have been invested into a single start-up company specializing in ARSG – Magic Leap, by Google and Alibaba (Schilz & Smith, 2016 cited in Rauschnabel, 2018). Thus, both financial and academic investments into the technology imply a race among tech companies to reign over the realm of mixed reality.

Network of countries that have been involved in VR research, the bigger the node and bolder the country name, the more research they have participated in Source: Rauschnable, (2018). (CC by 4.0)

Figure 2: Network of countries that have been involved in VR research, the bigger the node and bolder the country name, the more research they have participated in. Source: Cipresso, Giglioli, Raya & Riva (2018). (2018). (CC by 4.0)

As Cipresso, Giglioli, Raya and Riva (2018) have described, once a new technology is envisioned or created, a patent application is registered which then may or may not be introduced and made available for the market. Hence, whoever discovers the AR unicorn can hold immense power over the development and commercialisation of the technology, may it be commercial conglomerates or government agencies. In addition to the investments and amount of research conducted, the AR market size between the 3 major regions and rest of the world as shown on figure 3 suggests that countries such as the United States and China are in prime positions to win the AR race.

Augmented Reality Market, by region. Source: Markets and Markets’s Augmented Reality Market (2020), All rights reserved.

Figure 3: Augmented Reality Market, by region. Source: MarketsandMarkets – Augmented Reality Market (2020), All rights reserved.

The transformative effects of augmented/mixed reality

It would be short-sighted to consider AR exclusively within the technology, gaming, and media domains. As shown on figure 4, by bridging the digital and physical worlds, other industries such as retail, automotive, healthcare and hospitality can invest and utilize AR to enhance their operations (Strott, Wendin, Bissell, Oppen & Lasko, 2017). At first glance, with AR, companies can complement customer experience by fusing digital and physical shopping (ux_patricia, 2020), improve driving safety by displaying navigation information (NextReality, 2020) on the windshield, provide a more immersive experience in weather forecast (WAVE3Weather, 2020) or sensation-seeking in tourism such as the Van Gogh virtual reality exhibition (larrykim, 2020).

Investment in Virtual Reality and Augmented Reality by industry. Source: Strott, Wendin, Bissell, Oppen, Lasko (2017). All rights reserved.

Figure 4: Investment in Virtual Reality and Augmented Reality by industry. Source: Strott, Wendin, Bissell, Oppen, Lasko (2017). All rights reserved.

Continuing the positive influences of AR, education and training are promised with interactive technology that can increase students’ engagement with subjects such as medicine, physics, geography, and biology. The below video by the News Minute in Kerala, India demonstrates how AR technology can be implemented for online classes by recreating virtual animals, our solar system and simulating weather conditions with plausible graphics and sounds that can capture students’ attention more effectively.

Source: https://www.youtube.com/watch?v=05mWltJ1M8Q (The News Minute, 2020, Standard YouTube License)

However, for many emerging technologies such as AR, there are underlying issues that users should be aware of, namely the “Collingridge dilemma”:

“It is hard to predict the various impacts of a technology until it is extensively developed and widely used, but by then, it is almost impossible to control or change”

Carter, Egliston, (2020)

Social media and search engines are prime examples of technologies that have become so ubiquitous and enveloping that users can hardly relinquish these digital necessities.

Circling back to Facebook’s Project Aria, their novel mission to provide a virtual assistance for the blinds accentuates the social capital outcomes that AR can theoretically ameliorate the living conditions for the impaired population. However, privacy concerns and ethical issues persist when it comes to augmented reality technology. As pointed out, users’ physical movements are tracked 90 times per second for the scenery function to operate properly, and

“spending 20 minutes in a VR simulation leaves just under 2 million unique records of body language”.

VR Researcher Jeremy Bailenson (2018)

Ironically, whilst digital citizens are fighting against the data gathering practices and exposing the likes of Facebook and Google, recklessly trading our non-verbal data appears to be an undesirable option that many would shy away from embracing.

Conclusion

The potentials and benefits that augmented/mixed reality can embed in our real world are undoubtedly credible. However, at the same time, its controversial impacts are also crystal clear. Started in the early 2000s, but it was not until the mid-2010s when consumers started questioning Social Media platforms’ data-mining practices such as the Cambridge Analytical scandal and demanded for regulations (Cunningham, Craig, 2018). While AR technology has been experimenting for the more than half a decade, there are evidences that it can follow the footpath of Social Media platforms, and we as users can fall down the rabbit hole of mixed reality’s immersive experiences.

As our last days of reality approach, in concur with Mark Pesce (2017), I believe that it is imperative for early regulatory measures, standards and guidelines accentuating the protection against users’ data mining to be developed, building on the works of General Data Protection Regulations, National Institute of Standards and Technology as well as other evolving laws, which can potentially safeguard the playing field when the reality trap make its way to our digital life. Additionally, educational efforts such as the XR Safety Initiative that aims to inform users of the mixed reality’s pros and cons are highly advocated for, so consumers are better equipped to make the final decision of whether to become an early adopter or uphold their organic realities.

Reference List:

Ballenson, J. (2018). Protecting nonverbal data tracked in virtual reality. JAMA Pediatrics. doi: 10.1001/jamapediatrics.2018.1909

Carter, M., Egliston, B. (2020). Facebook’s virtual reality push is about data, not gaming. Retrieved from https://theconversation.com/facebooks-virtual-reality-push-is-about-data-not-gaming-145730

Cipresso, P., Giglioli, C. G. I., Raya, A. M., Riva, G. (2018). The past, present, and future of virtual and augmented reality research: a network and cluster analysis of the literature. Frontiers in Psychology, 9(2086). doi: 10.3389/fpsyg.2018.02086

Cunningham, S., Craig, D. (2018). We must not punish content creators in our rush to regulate social platforms. Retrieved from https://theconversation.com/we-must-not-punish-content-creators-in-our-rush-to-regulate-social-platforms-96270

Cowling, M., Birt, J. (2018). Designers of mixed reality experiences shouldn’t overlook the communal nature of video games. Retrieved from https://theconversation.com/designers-of-mixed-reality-experiences-shouldnt-overlook-the-communal-nature-of-video-games-90315

Farshid, M., Paschen, J., Eriksson, T., Kietzmann, J. (2018). Go bodly! Explore augmented reality (AR), virtual reality (VR), and mixed reality (MR) for business. Business Horizons, 61(5), 657-663. doi: 10.1016/j.bushor.2018.05.009

Jones, N. (2020). Immersion: entering the screen, In Spaces Mapped and Monstrous: Digital 3D Cinema and Visual Culture. New York: Columbia University Press.

larrykim. (2020). #TuesdayThoughts: Vincent van Gogh’s world in virtual reality [Twitter post]. Retrieved from https://twitter.com/larrykim/status/1318477054489157638

Liao, T. (2018). Mobile versus headworn augmented reality: how visions of the future shape, contest and stabilize and emerging technology. New Media & Society, 20(2), 796-814. doi: 10.1177/1461444816672019

Markets and Markets. (2020). Augmented reality market by offering (hardware (sensor, displays & projectors, cameras), software), device type (head-mounted, head-up), application (enterprise, consumer, commercial, healthcare, automotive), and Region – Global Forecast to 2024. Retrieved from https://www.marketsandmarkets.com/Market-Reports/augmented-reality-market-82758548.html#:~:text=%5B191%20Pages%20Report%5D%20The%20Augmented,46.6%25%20from%202019%20to%202024.&text=Investments%20in%20the%20AR%20market,over%20the%20past%20few%20years.

NextReality. (2020). Startup Phiar prepares the beta launch of its augmented reality driving navigation app for iPhone https://t.co/yXiiJDbgdo?amp=1 [Twitter post]. Retrieved from https://twitter.com/NextReality/status/1221872965610037248

Park, S., Stangle, B. (2020). Augmented reality experiences and sensation seeking. Tourism Management, 77. doi: 10.1016/j.tourman.2019.104023

Pesce, M. (2017). The last days of reality. Meanjin, Summer 2017. Retrieved from https://meanjin.com.au/essays/the-last-days-of-reality/

Project ARIA. (n.d.). Our mission. Retrieved from https://projectaria.com.au/our-mission

Rauschnabel, A. P. (2018). Virtually enhancing the real world with holograms: An exploration of expected gratifications of using augmented reality smart glasses. Psychology & Marketing, 35(8), 557-572. doi: 10.1002/mar.21106

Strott, E., Wendin, C., Bissell, K., Oppen, F., Lasko, R. (2017). The essential eight technologies board byte: augmented and virtual reality. PwC Governance Insights Center. Retrieved from https://www.pwc.com.au/pdf/essential-8-emerging-technologies-augmented-and-virtual-reality.pdf

The News Minute. (2020). Elephants, cows in class: Kerala school uses augmented reality in online lessons [Video file]. Retrieved from https://www.youtube.com/watch?v=05mWltJ1M8Q&ab_channel=TheNewsMinute

ux_patricia. (2020). Future augmented grocery shopping! I noticed that I am spending too much time in supermarkets checking tiny labels with hidden ingredients! Would love to see at the first glance what’s gluten-free, dairy-free etc. #ux #augmentedreality #futuregroceryshopping @AdobeXD [Twitter post]. Retrieved from https://twitter.com/ux_patricia/status/1253268677010624515

WAVE3Weather. (2020).  The Halloween forecast: With Halloween just 10 days away, we debuted two of our most immersive augmented reality weather graphics to date today. #WAVE3Weather app: http://wave3.com/apps [Twitter post]. Retrieved from https://twitter.com/WAVE3Weather/status/1319041372897370114

Olly Nguyen
About Olly Nguyen 4 Articles
USYD student studying Business Information System and Digital Cultures