Touchscreen Touch Feedback

March 16th, 2012
ideas, tech
The tablet and smart-phone are incredibly flexible because they are just rectangles of multitouch display-glass. By getting rid of the buttons of earlier designs they give the application designer full control over the interface. We made the same jump with the switch to mouse and GUI: instead of simply repurposing the buttons on the keyboard we could place labeled buttons in any arrangement we wanted.

When we added the mouse we kept the keyboard for its typing speed, but with touchscreens eliminating the keyboard is now a serious option: phone keyboards have never been that good and multitouch screens aren't that bad for typing. Still, the pictures under glass model suffers from a lack of feedback. When you type on a keyboard you can feel the edges of the keys and your body learns calibration until it becomes an automatic extension of your body. This doesn't happen on a screen.

It's especially apparent with digital instruments: while there are many great musical iPhone apps [1], and the touchpad on a MacBook makes a nice MIDI controller, you need to be constantly looking at the screen not to get lost. With a guitar or piano your fingers quickly learn to do most of the fine tuning of positioning on their own and allow you to play much more quickly than if you had to keep your eyes in the loop.

How do we get the eyes out of the loop? Especially, how do you give better feedback than a simple right or wrong, where as the fingers start to stray they can adjust back unconsciously? The best we have so far is a click sound or short vibration after a successful button press, but that can't hint what you need to fix or where you're in danger of going wrong. Instead we fall back on visual feedback, but the fingers get in the way, and it doesn't combine well with muscle memory.

I want something better: a precise way to get a feeling into the fingers from the surface of the device. It should be immediate, but it's probably fine if it doesn't feel much like anything we're used to: we don't need to imitate the feel of a button, just get some of its advantages. Either we need to trigger the finger's natural sense of touch by deforming the surface, or we need some way to act on the finger at a distance. An array of raisable pins like in a Braille reader? Strong charges of static electricity just below the screen tingling in the fingers? [2] Direct electrical stimulation with tiny contacts? A major trade-off here is whether you get to keep the surface as a simple pane of sturdy glass. Tiny dots probably make for a much worse image. On the other hand I don't know the physics to say whether a finger-tingling voltage in close proximity to the rest of a touch screen is practical. Are there other ways to trigger the nerves of the finger remotely?

One possibility is to not worry about any of this and just write smart software that takes a noisy imprecise input stream and sorts it out. Siri and Swype, gestures with the mouse or finger. Either we give people better feedback so we can be more precise, or we teach devices enough that we don't have to.


[1] I say iPhone because you can't make them for Android.

[2] You could put magnets in the fingers and then have the device wiggle them, but we'd like to just work for everyone without requiring biomods.

Comment via: google plus, facebook

Recent posts on blogs I like:

What are the results of more parental supervision and less outdoor play?

Ups and downs for mental health and injury rates The post What are the results of more parental supervision and less outdoor play? appeared first on Otherwise.

via Otherwise November 24, 2023

My startup advice

I sat down for a conversation with Alex Long. He took notes and sent them to me, and it seemed worth lightly-editing the notes and posting. I’ve left it quite raw, more like a tweet thread than a proper blog post.

via Home October 23, 2023

A Big Problem With The Going To Bed Book

One day my dad was reading this book called the "Going to Bed Book" to my sister Nora. The book is basically about a bunch of animals who are getting ready for bed on a boat. They go down the stairs, take a bath, hang their towels on the wall, find…

via Lily Wise's Blog Posts September 18, 2023

more     (via openring)