Hand Tracking in Virtual Reality with Leap Motion

To find the seamless user experience for our Augmented Reality and Virtual Reality applications we re-tested Leap Motion and were thrilled with the software improvements.

The Leap Motion’s hand tracking prowess simplifies interaction, aids the sense of immersion, and allows for creative interaction concepts.

Read on to find out how we’re using Leap in our custom VR development.

 

WHAT IS LEAP MOTION

Leap motion is a peripheral mount to any VR headset that enables accurate hand tracking within VR. Instead of interacting with controllers and buttons, the leap motion allows you to use natural gestures such as pinching, grabbing, waving, and swiping without having to handle hardware.

The Leap Motion is like an expert at detecting hands— it was even designed to track how hands use tools--like a hand holding a stylus in virtual reality.
 

LEAP MOTION IN ENTERPRISE VR/AR APPLICATIONS

We like using Leap Motion whenever less equipment and less complexity is essential to the VR and AR experience. Controllers = Complexity.

When showing VR to clients, it’s often the first VR experience they’ve ever had. While frequent users and gamers will get used to Vive controllers easily, there is a learning curve for a novice. And when it comes to enterprise VR and AR experiences, you can’t rely on familiarity with VR hardware for interaction.

Leap Motion attaches to the headset and reconizes your hands as they type on virtual keyboards, push virtual buttons, or open virtual doors. Manipulating virtual objects with your hands reduces the barrier to interaction and simplifies the experience.

Consider the VR conference booth or an interactive kiosk—At a conference demonstration you only have a few seconds with each person or none at all. Leap Motion enables seamless user interaction with the virtual environment—without having to know that the trigger button grabs and the thumb pad slides.

Leap Motion is also is excellent whenever controllers would break complete immersion. Whether it’s a high-end immersive experience or you’re prototyping new processes in VR—Use hand tracking instead of controllers for a truer mockup and sense of immersion.

 

HOW TO USE LEAP MOTION

The Leap Motion hardware attachment runs around $70. All you need to try it is an application that’s programmed for it. Because Leap Motion requires an addition to a headset, you won’t find it in many off the shelf applications that consumers are using. But one way to try is in Alt Space.

If you’re building your own applications, Leap Motion is built into the Unreal engine, plugins, and input devices. Simply check the Leap Motion checkbox and set your pawn options. The Leap Motion is astoundingly programmable.

Its use in custom enterprise software makes a ton of sense. In enterprise, you’re not working with gamers. You’re much more likely to have a total VR novice. With a Leap Motion, VR novices can put on a headset and be proficient using their hands without fumbling around controllers.

A big hassle of VR is that you’re blind to the world. You have to find yourself in the virtual space, orient yourself to your controllers, and know how to interact with the controllers. Then there is additional interface confusion in dealing with hardware you can’t see.

Once Leap Motion is applied, it reduces the hardware complexity to just putting on a headset a going. With the attached Leap Motion, you can see your hands through the headset and immediately know how to interact with virtual space. Push buttons with your fingers, open doors with your hands.

 

CAVEATS - WHERE WE PREFER VR CONTROLLERS

There are some applications where we actually prefer controllers over the Leap Motion for interaction—and that’s when we need precision and fine grain interaction.

When you want precision and control, controllers win at this point. Controller tracking is precise enough to drop VR furniture in exactly the right place for example, and when drawing and modelling in VR, the controller is still essential. We anticipate the software will get better, but at the present, the action of releasing an object can be a struggle. The Leap Motion will occasionally rotate objects or drop objects as slightly incorrect angles.

That said, this software is improving all the time and it wouldn’t surprise us at all if within a few years it was a VR interaction standard.

 

INTERACTION CONCEPTS FOR LEAP MOTION

  • Multiple fingers for typing on virtual keyboard (chicken pecking). Even though there isn’t (currently) haptic feedback, typing in this way is much more comfortable than having to key in information on a keyboard.

  • VR calculator - same as virtual keyboard —working with a wand is obnoxious, but punching in the numbers with fingers is natural.

  • Any interacting with UI - things such as the TiltBrush UI would do well with the Leap Motion. You don’t have to deal with abstractions that motion controllers create. You can simply point and go.

  • Interface heavy applications - Virtual Showrooms where there are many different options for materials, furniture and schemes.

    • The frustration of all of the VR showrooms we try is finding the interaction buttons.

    • With the Leap Motion and clever spatial programming, you can change settings and parameters from one stop. Instead of a static UI, you can implement a context-aware interface.

  • Kiosks - The Leap makes forbetter for demonstration of an unsupervised Kiosk. If you want someone to walk up to an experience, put on a headset and go, Leap Motion is less setup.

  • Extended uses - because holding controllers for a long time can be a pain. The Leap Motion can provide extended VR users with better ergonomics

  • Finally, the Leap Motion enhancing immersion and presence in high end virtual experiences by forgoing extra handheld hardware and instead letting the user exit and interact naturally.