2009 Natural User Interface Group Summer of Code Application
Tiago Serra / Tony Gonçalves
Abstract
Multitouch interfaces provide a great deal of benefits for integrating new interaction modes within applications. These surfaces allow for a variety of gestures beyond point and click that create more efficient application navigation and/or interactions.
Supporting orientation-less gestures that can be composed of several strokes in pyMT would mean we can build a high level multitouch gesture library like zooming, pinching, tilting, etc and also pave the way for creation of simple OCR software like a Palm Grafitti emulator for instance.
Multi-touch gestures is an area that will definitely expand in the future, thus it’s important to create tools that allow developers to easily integrate this technology into their applications.
Features
Orientation-less gesture recognition
Gestures can be composed of several strokes
Gesture recording widget that saves to a gesture library
Connect gestures with application events
Detailed description
In order to achieve intuitive interaction there is a need to define a library of gestures inside pyMT that can be translated to events, thus simplifying the interaction model. This gestural support is in no way limited to any pre-defined patterns, but to potentiate exactly the creation and brainstorming of natural interactions. These gesture movement definitions will become very important in the future development of multitouch screens. Because of this we need to facilitate the work of newcomers and interaction designers to pyMT, giving them a standard library of gestures as well as a way to simply define new ones.
Orientation-less gestures means we can match gestures independent of their orientation or angle especially when dealing with table-top surfaces.
Gestures can be composed of several strokes by establishing a small time frame between each individual stroke. With this feature we can compose gestures with more than one finger or stroke and pave the way for a primitive letter recognition library.
A standard gesture library is important for creating higher level gestures like zoom and rotate. Although pyMT supports this gestures in the ScatterWidget we need a general way to map these higher-level events to other widgets or applications.
A way of connecting drawn gestures to events inside pyMT would make the whole process of designing interactions simpler, more efficient and rapidly evolving the whole framework.
Justification and why me:
This functionality is crux to efficiently recognize user gestures in table-top surfaces where the position of the user is unknown. Interpreting the gestures will tell us about the user position thus allowing us to appropriately position the interface feedback.
pyMT already has gestural support although not orientation-less and not composed of several strokes. By implementing the described functionality we can truly make it flexible to new and interesting interaction design work.
I’m interested in exploring the interaction possibilities of surface computers and I believe this is a crucial feature to enter this endeavor.
