::  Posts  ::  RSS  ::  ◂◂RSS  ::  Contact

Touchscreen Touch Feedback

March 16th, 2012
tech, ideas  [html]
The tablet and smart-phone are incredibly flexible because they are just rectangles of multitouch display-glass. By getting rid of the buttons of earlier designs they give the application designer full control over the interface. We made the same jump with the switch to mouse and GUI: instead of simply repurposing the buttons on the keyboard we could place labeled buttons in any arrangement we wanted.

When we added the mouse we kept the keyboard for its typing speed, but with touchscreens eliminating the keyboard is now a serious option: phone keyboards have never been that good and multitouch screens aren't that bad for typing. Still, the pictures under glass model suffers from a lack of feedback. When you type on a keyboard you can feel the edges of the keys and your body learns calibration until it becomes an automatic extension of your body. This doesn't happen on a screen.

It's especially apparent with digital instruments: while there are many great musical iPhone apps [1], and the touchpad on a MacBook makes a nice MIDI controller, you need to be constantly looking at the screen not to get lost. With a guitar or piano your fingers quickly learn to do most of the fine tuning of positioning on their own and allow you to play much more quickly than if you had to keep your eyes in the loop.

How do we get the eyes out of the loop? Especially, how do you give better feedback than a simple right or wrong, where as the fingers start to stray they can adjust back unconsciously? The best we have so far is a click sound or short vibration after a successful button press, but that can't hint what you need to fix or where you're in danger of going wrong. Instead we fall back on visual feedback, but the fingers get in the way, and it doesn't combine well with muscle memory.

I want something better: a precise way to get a feeling into the fingers from the surface of the device. It should be immediate, but it's probably fine if it doesn't feel much like anything we're used to: we don't need to imitate the feel of a button, just get some of its advantages. Either we need to trigger the finger's natural sense of touch by deforming the surface, or we need some way to act on the finger at a distance. An array of raisable pins like in a Braille reader? Strong charges of static electricity just below the screen tingling in the fingers? [2] Direct electrical stimulation with tiny contacts? A major trade-off here is whether you get to keep the surface as a simple pane of sturdy glass. Tiny dots probably make for a much worse image. On the other hand I don't know the physics to say whether a finger-tingling voltage in close proximity to the rest of a touch screen is practical. Are there other ways to trigger the nerves of the finger remotely?

One possibility is to not worry about any of this and just write smart software that takes a noisy imprecise input stream and sorts it out. Siri and Swype, gestures with the mouse or finger. Either we give people better feedback so we can be more precise, or we teach devices enough that we don't have to.


[1] I say iPhone because you can't make them for Android.

[2] You could put magnets in the fingers and then have the device wiggle them, but we'd like to just work for everyone without requiring biomods.

Comment via: google plus, facebook

Recent posts on blogs I like:

I’m Giving a Talk About Construction Costs Tomorrow

By popular demand, I’m giving the talk I gave 2 weeks ago at NYU, again. The database will be revised slightly to include more examples (like Ukraine, which I added between when I gave the talk and when I blogged about it), and I may switch around a few t…

via Pedestrian Observations December 2, 2019

Your room can be as bright as the outdoors

The effect was huge: I became dramatically more productive between 3:30pm and whenever I turned off the light. I estimate the lamp bought me between half an hour and two hours a day, depending on how overcast it was.

via benkuhn.net November 26, 2019

git-subtrac: all your git submodules in one place

Long ago, I wrote git-subtree to work around some of my annoyances with git submodules. I've learned a lot since then, and the development ecosystem has improved a lot (shell scripts are no longer the best way to manipulate git repos? Whoa!). Thus, I …

via apenwarr November 24, 2019

more     (via openring)

More Posts:


  ::  Posts  ::  RSS  ::  ◂◂RSS  ::  Contact