My two cents on the metaverse: why it is important and how to build it using latest computational science-based tools
by George Papagiannakis
When the World Economic Forum brings together leading voices [1] from the private sector, civil society, academia and policy to define the parameters of an economically viable, interoperable, safe and inclusive metaverse, it means that its forthcoming economic and social value is imminent, unavoidable and even already been implemented [16]. Many might have objections with how the metaverse will be deployed, but I don’t think any organisations can afford to bypass or ignore its tremendous underlying potential and transformation on the way we work, play, learn and communicate.
In this short article, I will outline the latest science-based evidence of the importance of the metaverse (especially on education [18]) and my proposal on how we should go about building it using science-based tools, in the most human-centric way and for the widest possible social benefits.
First, from the hundreds of different definitions of the metaverse out there, I have chosen three, each one from a different social, economic and technology angle, on top of the generally accepted notion that the metaverse is the successor state of the internet:
The social angle of the metaverse
Anna Wiener’s in her recent article [2], is defining the term based on different perspectives of Key Opinion Leaders (KOLs):
“The metaverse will be a massively scaled and interoperable network of real-time rendered 3D virtual worlds which can be experienced synchronously and persistently by an effectively unlimited number of users” (the venture capitalist Matthew Ball). They offer that it might enable companies “to embed computing into the real world and to embed the real world into computing” (the Microsoft C.E.O., Satya Nadella) — and that it could make “the virtual world more real and the real world more rich with virtual experiences” (the Tencent C.E.O., Pony Ma). Tim Sweeney, the C.E.O. of Epic Games, has said that the metaverse may be “a multi-trillion-dollar part of the world economy”; Jensen Huang, the C.E.O. of Nvidia, thinks that it could create “a new economy that is larger than our current economy.” The over-all point is that it will be simultaneously a place for connection, community, and so on, and also a forum for transaction and extraction. For its makers, the metaverse will be stuffed with money — in every dimension, all the way down.”
The economic angle of the metaverse
The World Economic Forum is actively studying the imminent huge economic potential [3]:
“The metaverse is a future persistent and interconnected virtual environment, where social and economic elements mirror reality. Users can interact with it and each other simultaneously across devices and immersive technologies while engaging with digital assets and property.”
The technological angle of the metaverse
Caecilia, d’Anastasio provided a definition based on the major enabling technologies of the metaverse, namely virtual and augmented reality (VR/AR) [4]:
“Tomorrow’s cyberspace will be empyrean, transcendent, immersive, 3D, and all folded together, the disparate sites and services we live and die by gathered under one love. It will be a super-platform that convenes sub-platforms: social media, online video games, and ease-of-life apps, all accessible through the same digital space and sharing the same digital economy. Virtual reality companies say you’ll get there through VR headsets, while augmented reality companies say you’ll wear AR smart goggles. And with boyish enthusiasm for science fiction fueling their piety, these preachers are calling this vision the metaverse, after Neal Stephenson’s 1992 dystopian novel Snow Crash”.
An important contribution according to my opinion on the essential characteristics of the metaverse, are the “Seven rules of the Metaverse” [5], recently published by the co-creator of the original Virtual Reality Modelling Language (VRML), as an “attempt to avoid more confusion around the term, and with the sole intent of keeping all eyes on the prize, that is, building a metaverse intended for the greatest good and for the greatest number of people”:
Rule #1. There is only one Metaverse.
Rule #2: The Metaverse is for everyone.
Rule #3: Nobody controls the Metaverse.
Rule #4: The Metaverse is open.
Rule #5: The Metaverse is hardware-independent.
Rule #6: The Metaverse is a Network.
Rule #7: The Metaverse is the Internet.
Despite its undeniable economic worth, the metaverse won’t be built overnight. Many of its essential enabling technologies will be turned into products in the next 10–15 years. While that’s frustrating for those of us eager to dive right in, it gives us time to ask the difficult questions about “why” and “how” the metaverse “of the people, by the people, for the people” should be built.
Why the metaverse is more important than you can imagine?
An excellent recent publication [6] by my colleague Prof. Riva, describes with clear neuroscientific evidence, how the metaverse can essentially transform our autobiographical memories and effect our very own brain cognitive functions.
“We are workers because we go to the office, we are fans because we go to the stadium, we are students because we go to school or university, and so on. Inside our brain there are a series of specific neurons — the “place cells” and the “border cells” (also called “GPS neurons” since they work in a similar way to the GPS in our cars) that are activated when we occupy a position in the environment, allowing us to orient ourselves in space. These neurons play a key role in the construction of our autobiographical memory.”
This work [6] further provides all background evidence that for our brains, modern videoconferencing systems and other digital social platforms (that most of us ended up relying so much during and post-pandemic) are not places, and therefore do not directly connect the experiences we have within them with our autobiographical memory:
“On the other hand, the VR/AR technologies of the metaverse, are instead able to activate our GPS neurons and make us feel present in the digital places…. This ability makes the metaverse a significantly different technology from its predecessors. If television and social media are persuasive technologies, because of their ability to influence people’s attitudes and behaviors, the metaverse is instead a transformative technology”.
We were fortunate to recently collaborate with Prof. Riva’s Humane Technology Lab in this frontiers editorial and relevant special issue/research topic [7].
How the metaverse will be eventually built and sustained?
The cornerstone enabling technology as well as main interface for the metaverse to my opinion is virtual reality and its siblings (AR, MR etc.) along the Virtuality continuum by Milgram and Kishino [8] or what collectively the industry defined with the umbrella term XR, ‘extended reality’ by the XR Safety Initiative [9]. It is quite interesting they are also referencing one of our early published works on inside-out AR tracking as a means for 6 degrees-of-freedom head tracking in VR, well before this idea became standard in the field [10].
The matter of content creation for the metaverse will not be based on just few wall-garden proprietary technology stacks by few companies, but I believe on novel, interoperable authoring platforms, parallel to today’s wordpress or wix web-authoring platforms. For me the essential question of how to build the metaverse is not a matter of content but actually of tools as this comprehensive, latest survey by Coehlo et al highlights [11] as well as the latest excellent metaverse technologies survey by Lik-Hang Lee [17]. Our VR software design patterns method [12] is also reviewed in this work as one of the tools to build next-generation VR training applications for the metaverse.
At ORamaVR we are excited how computational medical science [13] will accelerate world’s transition to VR training in the metaverse and empower medical professionals to accelerate their proficiency and improve patient outcomes. Already several leading healthcare organisations worldwide are now employing the MAGES SDK metaverse authoring tool for state-of-the-art, high-fidelity VR training simulations. From hospitals (Inselspital Bern, Switzerland) to medical schools (New York University (NYU), University of Southern California (USC), Fayeteville State University (FSU) in United States) and to online university courses for training and assessment (Western Governors University (WGU), USA and University of Athens, Greece), and from medical device training organisations (Virtamed A.G, Switzerland) to medical VR content creators (Nonnocere, Germany and FORTH, Greece) employing a variety of different, metaverse-ready VR head mounted displays (HMDs): HTC Focus, Varjo XR, Meta Quest as well as AR/MR HMDs: Microsoft Hololens and Magic Leap:
https://www.youtube.com/watch?v=dqmFb7Ye0vs
with riveting testimonials:
https://youtu.be/rDXLDRDrjpo?t=1364
Like the internet and web development iteration cycles, the future of the metaverse is contingent on it being a democratic space. At ORamaVR we are looking already ahead and by providing authoring/creation metaverse tools to accelerate the transition: https://oramavr.com/products-2/#sdk
As a scientist, I’m humbled by how nascent the computational science behind VR, and by extension, the metaverse is. It is the science of only 50+ years [14], since Ivan Sutherland created in 1968 his first head mounted display [15]. Compared to classic sciences that are making human life better every day — physics, chemistry, biology etc. — there’s a long, long way to go for VR to realize its potential. As an entrepreneur I am amazed by the direct impact that VR, the metaverse and simulation-based learning can have to help and empower people with knowledge. With human-centric vision, outstanding research and socially responsible business execution, VR and the metaverse will not only transform and improve our quality of life but I believe it will contribute significantly towards extending it.
References
[2] https://www.newyorker.com/news/letter-from-silicon-valley/money-in-the-metaverse
[3] https://initiatives.weforum.org/defining-and-building-the-metaverse
[4] https://www.wired.com/story/big-tech-metaverse-internet-consolidation-business/
[5] https://medium.com/meta-verses/the-seven-rules-of-the-metaverse-7d4e06fa864c
[6] https://psyarxiv.com/dvcmw/
[7] https://www.frontiersin.org/articles/10.3389/frvir.2022.877494/full
[8] https://www.alice.id.tue.nl/references/milgram-kishino-1994.pdf
[9] https://xrsi.org/publication/the-xrsi-definitions-of-extended-reality-xr
[10] Zikas, P., Bachlitzanakis, V., Papaefthymiou, M., Papagiannakis, G. (2016). A Mobile, AR Inside-Out Positional Tracking Algorithm, (MARIOPOT), Suitable for Modern, Affordable Cardboard-Style VR HMDs. In: , et al. Digital Heritage. Progress in Cultural Heritage: Documentation, Preservation, and Protection. EuroMed 2016. Lecture Notes in Computer Science(), vol 10058. Springer, Cham. https://doi.org/10.1007/978-3-319-48496-9_21
[11] Coelho, H., Monteiro, P., Gonçalves, G. et al. Authoring tools for virtual reality experiences: a systematic review. Multimed Tools Appl (2022). https://doi.org/10.1007/s11042-022-12829-9
[12] Zikas P, Papagiannakis G, Lydatakis N, Kateros S, Ntoa S, Adami I, Stephanidis C (2020) Immersive visual scripting based on VR software design patterns for experiential training. Vis Comput 36(10–12):1965–1977. https://doi.org/10.1007/s00371-020-01919-0
[13] https://arxiv.org/abs/2108.04136
[14] https://blog.siggraph.org/2018/08/vr-at-50-celebrating-ivan-sutherland.html/
[15] Ivan E. Sutherland. 1968. A head-mounted three dimensional display. In Proceedings of the December 9–11, 1968, fall joint computer conference, part I (AFIPS ’68 (Fall, part I)). Association for Computing Machinery, New York, NY, USA, 757–764. https://doi.org/10.1145/1476589.1476686
[16] McKinsei & Company, “Value creation in the metaverse”, https://www.mckinsey.com/~/media/mckinsey/business%20functions/marketing%20and%20sales/our%20insights/value%20creation%20in%20the%20metaverse/Value-creation-in-the-metaverse.pdf
[17] https://arxiv.org/abs/2110.05352
[18] https://www.weforum.org/agenda/2022/05/the-future-of-education-is-in-experiential-learning-and-vr/
This is an article on a series of medical XR articles, ranging from computer graphics methodologies, to VR simulation algorithms, from research and innovation, to academic entrepreneurship. Parts of this article have been presented in the 3rd Annual Virtual Reality and Healthcare Europe Symposium 2021, Dublin, Ireland.
