In fall 2011, Omek Interactive, a gesture-recognition and body-tracking technology company, approached Kicker Studio to create a gesture-controlled movie browser platform developed for Jinni, a movie genome company. Jinni’s software is an internet application that analyzes and makes movie recommendations based on what a user likes, not on the usual keywords and genres. Omek’s technology is able to detect gestures in X, Y and Z depth, and track the full body, allowing for a more natural gestural interaction with Jinni’s software.
Our challenge was to demonstrate Jinni’s technology using Omek’s software to create an easy and memorable movie browsing experience. We set out to create a super-powered browser that felt as natural as conversation by using profile recognition and “cursor tracking” along with gestures to maximize functionality while keeping the mental load low for users.
SPEAKING HUMAN TO A TV
Based on our experience designing the gestural interface for Canesta in 2008 and other projects, we came to this challenge with a solid footing in the gestural landscape, and especially for what works and doesn’t work for users gesturing to a TV from the comfort of their couches.
When we took a step back and looked at the current gesture-based UI, we noticed two opportunities. First, for us, the cursor-tracking metaphor most frequently found in gesture UIs poses a problem. It is easy for users to understand because it is very similar to the mouse/cursor interface but in practice, when the physical mouse is absent, the logical feedback loop is removed, and the unmoored action feels disjointed and clumsy. The majority of on-screen cues for these cursor-tracking interfaces feature large, seemingly touchable buttons, while in reality, the user is gesturing in free space and can’t actually touch anything. This confuses and frustrates the users in a fundamental way. The large size of the targets is necessary to be equally accessible for a 5-year-old 3’5” girl and a 38-year-old, 6’4” man. Each target, then, takes up a large part of the screen, limiting the amount of options available, making it tedious to sort through large amounts of information. Jinni’s system is full of links and settings so simple cursor-based interactions would make it too cumbersome to be usable. Adding modal gestures to the interface allows us to simplify the number of steps to get to the sort tags, creating an interface that is much faster and easier for the user to operate.
Second, and perhaps most importantly, relying on tracking means the user spends a good amount of time with an arm outstretched toward the TV at shoulder’s height, which quickly becomes strenuous. It is a rather awkward position to hold for any length of time, but especially in the comfort of one’s den. It feels uncomfortable and unnatural. Most people would start looking for the remote rather than feel the burn.
The benefit of Omek’s gesture recognition and tracking technology is its ability to detect gestures in X, Y and Z depth, and track the full body. It can detect when a new person comes into the room, and associate a profile with that user. This combination provides a baseline for natural interaction with Jinni’s software. Adding gestures to a cursor-based interface allows for more flexibility and greater access to interactive features. Gestures also feel more comfortable to users, provided they are easier to discover and remember. Rather than the monkey-arm feeling inherent in cursor tracking, instinctual gestures, like those used in normal body language, feel more natural, easy and comfortable. And that’s how we want to feel when sitting and watching videos.
OUTLINING THE IDEA
We explored a variety of concept metaphors that combined gesture and tracking user interaction, and that best represented Jinni’s experience, and its personality. The team decided on the following design principles to guide us:
Provide quick access to play
Do not require physical coordination
Don’t lose me (provide familiar foothold, exploration with grounding)
Reward discovery (provide results in real time)
Enable a conversational experience (the screen is never still)
Help the user to connect gesture to UI by emphasizing a sense of depth and space
COMFORTABLE AND NATURAL GESTURES
For the Jinni browser, Omek’s technology allowed us to create a simple set of two gestures based on body language that combined with the cursor-tracking needs of the interface to make them more efficient and less strenuous to users. To both coach the user through the necessary gesture and make the tracking feel more natural, we designed a series of on-screen visual cues that avoid desktop touchscreen “button” metaphors and instead rely on airy looking targets and a sense of depth. This airy interface helps to remove the mental disconnect of touchable targets that the user can’t touch. The sense of depth helps to anchor the actual gesture to real space.
The Omek/Jinni browser interface debuted at CES 2012.
To see how we can help you on your next project, contact us »