Ghost Fingers Typing for Tablet Devices
Tablet computers offer a unique opportunity to rethink how we type, mostly because traditional touch typing doesn’t work well on them unless the tablet is resting on a flat surface, like a desk or a lap. But most of the time, we’re holding tablets, which turns typing into a process of hunt-and-peck with one hand while the other hand tries to hold the tablet steady. Keyboard solutions from mobile phones don’t work well either; they just aren’t designed for the larger screen space, and spaced out keys make for a Fitts’ Law nightmare.
In an earlier post, I noted that typically, unless you are using a stylus, a tablet is held with two hands, with thumbs on the front or on the side of the device, and your four remaining fingers on the back. It’s a natural, grasping gesture for humans. This leads me to think there there could be two possible new typing methods: one for typing with just your thumbs using the screen on the front of the device (which I outlined as the 3x2x2 Method), or by typing with the remaining four fingers on the back of the device, using ghost fingers.
Ghost Fingers are a pattern I found when researching my book Designing Gestural Interfaces. Ghost Fingers are when a device becomes seemingly transparent so the users can “see” (onscreen) their fingers on the back of or inside of the device. (Some examples.) Ghost fingers could be employed with tablets by putting a touchscreen (it could even be just a wide capacitive strip) on the back of the tablet that would only turn on when the keyboard was deployed. It could even be a physical keyboard, although that could get annoying when simply holding the tablet. As the user pressed the (digital or analog) buttons on the back, the corresponding key on the front of the screen would be highlighted.
The fingers of your hand (aside from your thumb) have four positions: straight (extended), half curled, curled, and tucked under (into the palm). Tucked under doesn’t really help much for typing, so there are basically three positions that fingers can be used to press buttons/keys. This means at most, three columns of keys. Additionally, the index and pinky fingers can stretch away from the hand (the index finger more widely), which allowed for two extra rows of keys. Index fingers are far more flexible than the pinky, allowing for an extra row. However, stretching away from the hand limits flexibility, so the further the stretch, the less bend the finger has, and so less columns of keys. Those extra rows only gained eight keys, but it was enough to make the keyboard viable.
Depending on weight, tablets can actually be held without any fingers touching the back, with the device held between the two palms occasionally with a slight assist from the bottom of the index finger. This allows for easier finger movement and for less accidental touches of the keyboard.
With those ergonomics in mind, I designed the following keyboard:
The keyboard roughly maps the existing QWERTY keyboard to a new split configuration, much like ergonomic keyboards currently do. Stretching the index and pinky fingers allow access to more keys. Here’s what it would look like on an iPad:
I prototyped it with simple paper taped to the back of an iPad. I had to adjust the keyboard layout and key size several times until it felt right. Done incorrectly, it felt like too much of a stretch to reach keys, or the curl of the finger was too much. Note that the “keys” on the back would likely not need visuals like I’ve shown here. After all, they are on the back of the device, unseen by the user.
There are some limitations to Ghost Fingers typing: namely that you could never lay the device down flat to type; unless you have a USB/bluetooth keyboard or a more traditional touchscreen keyboard, you’d always have to hold the device to type. There would obviously be a learning curve, and in all honesty, I wonder how long it would take to become comfortable not seeing your fingers type. Then again, people text message with their mobile phones are in their pockets, and aren’t focusing on their fingers as much as the screen with regular typing anyway, so I suspect it’s not a huge barrier to overcome.
Like the 3x2x2 method, the next step for this design would be to prototype it working on the iPad (or similar device), check its time against regular typing, and perhaps adjust the alphabet to key matchup. I don’t have any idea how this would translate for other, larger, alphabets.
Related Post: 3x2x2: A new method of thumb typing for tablet computers
ABOUT KICKER STUDIOCASE STUDIES
SERVICES
TEAM
BLOG
CONTACT US
5 Comments
Hi Dan –
Cool concept. You could take the Ghost Finger pattern a step further in your concept and consider using a translucent overlay on the screen that shows the virtual keyboard on the screen as it is laid out on the back (the mirrored image of the iPad mockup). It would allow users to essentially punch the keys “through” the screen. This would likely reduce the time to learn the keyboard layout (recognition instead of recall). It could be configurable, so expert users can turn it off.
Looking forward to reading about the next steps.
-Kris
Good catch, Kris. It’s supposed to be a 1:1 mapping, laid out as it appears on the screen. So actually my paper prototype on the back should be reversed. I used it more for key spacing than anything. I didn’t notice that the paper keys should have been reversed because I couldn’t really see them.
Very cool to see gaming conventions applied to the keyboard problem! Taking Kris’s idea a little further I was wondering if you could have a translucent, onscreen highlight for the keys that are currently being touched. Less of the screen would be obscured, you’re instantly shown if your fingers are in the correct position and the shape of the overlay would change (pattern & motion recognition) as you naturally move to type.
It is a really inspiring post Dan!
“Less of the screen would be obscured, you’re instantly shown if your fingers are in the correct position and the shape of the overlay would change (pattern & motion recognition) as you naturally move to type.”
I vote for that
an on screen translucent representation of the back keys. By the way it needn’t to be 1:1 scale representation.
Might be a 50% scale representation (like in your example) with hovered keys being emphasized by a bigger size?!?
This would allow also to match user experience on using this back keyboard. The more the user is used to it, the smaller the transparent representation of back keyboard might be and the less obstructive it is to displayed content. It might be a great “adaptability to user knowledge” pattern…
By the way the upcoming Notion Ink Adam tablet has a back track pad panel which might mean that we aren’t that far from your suggestion.
http://www.youtube.com/watch?v=eQvefuCHZds
In addition I read that Apple registered a patent for a “back-side touch”.
http://bit.ly/bO1Axf
An some rumors:
“Next-Generation iPhone to Offer Touch-Sensitive Casing?”
http://bit.ly/bO1Axf
Best regards.
Cedric Madelaine (aka maddec)
analogd interactive
http://analogd.com
Twitter: maddec303
Dan,
Just a second thought on your post:
“There are some limitations to Ghost Fingers typing: namely that you could never lay the device down flat to type; unless you have a USB/bluetooth keyboard or a more traditional touchscreen keyboard, you’d ”
Off course you can use a regular touch keyboard instead, but it might be a bit confusing to user to switch from one layout to another….
You might introduce another pattern a new?!? pattern I call the “Permeating Fingers Pattern” to avoid this drawback:
1: your Tablet is put flat on a table
2: on the screen is displayed your overlayed keyboard left and right
3: On the bottom of the touch screen you have a special zone which transform your touches in a second layer of interactivity (a bit like control/command key pressed + mouse click).
When you you click the zone –> activate the feature.
Activating the feature might be to hold button pressed as Shift key, but I feel it would be better to have a “Caps Lock” approach to allow both hands to be free.
Once the “Permeating” feature is activated,
it allows to directly interact with the backscreen representation of the keyboard. A bit as if physically your fingers can go through the screen and tap the back of the case.
This “Permeating” feature might be activated also automatically through gyroscope or special routines in the back case sensitive zone.
Your back keyboard idea brings also new problematics to address, like how do you communicate in the interface representation that it is not a regular touch interface but one that is managed by interacting with the back of the case of the device?
A way might be to use a “revert colors filter” + blur approach?!?
I could have some time to test some mockups, do you mind to send me a vectorial version of your keys design?
Thank you in advance
One Trackback/Pingback
[...] and another typing with just your thumbs using the screen on the front of the device. I address Ghost Finger Typing in a separate post, but for now, let’s focus on thumb [...]