Interactions in the Metaverse

There has been a lot of buzz around the term “metaverse” so today I would like to share our unique perspective on the metaverse and how we will interact with the metaverse. 

What is the metaverse?

     Well, that is a slightly complicated question as there are several large companies currently offering different visions of what the metaverse will look like. In broad terms, the metaverse is a concept from science fiction that describes a single fully virtual 3d world that users can connect to via the internet and interact with each other and the virtual world around them. Some of the most credited works on this concept are Neuromancer (1984), Snow Crash (1992), and of course Ready Player One (2011). Reading this broad description, you can easily classify many current video games as already checking the box of a metaverse (ie. Minecraft, Roblox, World of Warcraft, etc.). 

So why the hype? 

     The current excitement around the metaverse is that for the first time there exists hardware currently that is capable of fully immersing us within these digital worlds. Additionally, this has been fueled by every large tech company laying out its vision for the metaverse and how we will soon connect in these digital environments. 

Digital Interactions 

     
One aspect that I feel is often not talked about is how we will physically interact with these digital worlds. I believe this is a fundamental hurdle that must be overcome for the metaverse to happen.   Anytime you watch someone with new technology the second the interaction becomes difficult to the user and they feel like they are not in control they will very quickly stop using it. Especially when the world looks and sounds so real that you are fully immersed the second you can’t do something you could in the real world the user is frustrated and annoyed. At VRgluv we have a specific area of focus and expertise on this as we have been working with hands in VR and digital environments over the past five years.

     Let’s look at the current solutions. 
Controllers 

     Controllers are the reason we started VRgluv. Anyone who has ever given a demo to a first time VR users know controllers are the most difficult part to explain and get the user to use properly because it is a completely new interaction method. Sure, some gamers are quite adept at adapting to new control schemes because they have been doing it their whole lives. But most users of the metaverse will not necessarily be gamers, if we as an industry want the concept of the metaverse to succeed we must find ways to interact with the virtual environment more naturally. Using controllers is like cutting off your hands so that you must poke at the world around you.  

     Optical hand tracking
     There has been a lot of effort put into optical hand tracking but there are significant base-level problems.  The first is occlusion. This is the problem of the camera or sensor not always being able to see the entire hand. Make a fist and you will very quickly see the problem. Regardless of where you put the sensors your hands move in such complicated ways that it is nearly impossible to always see every finger to track it. You usually need an incredibly large array of cameras to fully capture the motions, or you have to build prediction algorithms to predict where the fingers are. The algorithm approach runs into the same problem of the second the user feels as though they are not actually in control of the hands, they will stop using it. 

     The other major problem of optical tracking is haptics. Use any hand tracking only solution and try and pick up an object. Very quickly you will see the haptics issue. You will see your virtual hand pick up the object and your brain will expect to feel it. This won't happen and your brain will immediately start to reject the virtual hand. In addition, you don’t have a good sense of did I pick up the object because you don’t have the haptic sensation to clue you in that you have picked up the object. Watching people use optical tracking you see them grab at the same object multiple times because you need that physical feedback. 

Gloves
 

     Gloves are the way we will use our hands in the metaverse. There have been promises of the future of haptics and how in ten years we will have gloves that are the size of surgical gloves that perfectly recreates all aspects of our sense of touch. While I am a fan of all of the research that is being done in the haptics space, we believe that we need gloves NOW because the metaverse is coming now and for the metaverse to work you need gloves that work today. When we think about gloves three critical aspects are necessary. 

Excellent finger and hand tracking 

     This is obvious because if the gloves don’t fully track your hands and fingers the user will walk away frustrated because they will not feel in control of their virtual hands.

 Force feedback 

     Force feedback is the only way to solve the haptics problem. Many companies have tried the vibration approach and universally they do not solve the problem of feeling like you are holding the object.   You get the vibration, but it is not enough to convince your brain that you are holding the object. Some approaches even give you a slight resistance even this is not enough to convince the user. You have to stop the fingers at the boundary of the virtual object. This is not easy. Our hands are incredibly strong and dexterous you need some real stopping power to give the illusion of holding the object and that the object is real and solid. 

Usability

     
This is one of the critical pieces of any pair of gloves. When entering the metaverse putting on a pair of gloves will be like putting on your sneakers to go for a walk. It needs to be seamless, and you need to be able to do it yourself.  In addition, once the gloves are on and the headset is in place the gloves need to disappear and get out of the way of the experience. They need to be wireless and lightweight enough that you can hold your hands up and interact with the world just as you would with the real world. 

     Thank you for taking the time to read our post. If you want to discuss how VRgluv can help you interact with the metaverse please reach out, we would be glad to speak with you! 

Chris Taylor
CEO
Contact Form
Text Link