This is a draft proposal, feel free to contribute to it!
With the increasing popularity of touch-enabled devices nowadays, it is getting more common to insert information directly on the screen, using the fingers or a pen, instead of using a mouse. This project aims to enrich user's interaction with GEF/GMF editors.
Sketch provides an API to allow users to insert elements (EditParts) on GEF/GMF editors by capturing its gestures, typically using a pen on a tablet. The idea is to allow user might freely draw the representation of an element on the editor's area. The API would then transform this rough user's representation on something processable, interpreting what element the user meant to insert.
The API might learn from user's input, recognizing each sketch in the way it is drawn by different users. In cases when it is impossible to determine what element is meant by a sketch, the API 'asks' the user, through a small dialog, what he/she meant by that sketch. Thus the API is able to learn for the next times.
Sketch is proposed as an open source project under the Eclipse Technology Project (http://www.eclipse.org/technology/). This proposal is still in the Project Proposal Phase, and is being made in order to call for more community participation. You are invited to comment on and/or join the project. Please send all feedback to the http://www.eclipse.org/newsportal/thread.php?group=eclipse.technology.sketch newsgroup.
Background and Goal
The Sketch API is target primarily at editors with a large set of elements, in which the user have to constantly search for elements to add on the GEF Palette.
The recognition is being made primarily with Levenshtein's algorithm for string distance. The user's sketch is transformed in a set of points placed on a grid, then each point is transformed to a number according to the next point's position. This approach is described in detail here (http://www.springerlink.com/content/u0r7261w01248q1t/).
Each element has a bank of "words" that describes its many forms, once drawn by the user. The recognizer compares each new sketch with the words of each element, calculating the distance between them. If the distance is smaller for a square than for a triangle, for example, then the element is probably a square.
Architecture and Extensibility
This API is composed of three parts: a sketch Tool, a Recognizer and a sketch Bank. The figure below shows how the API integrates with GEF, through AbstractTool and GraphicalEditor.
- Add more and more robust interpreters at the recognition chain
- Support for GEF
- Support for GMF through sketch models related to .gmfgraph (one group of primary sketches for each graphical node)
Packaging and Deployment
Sketch will ship in the form of an Eclipse update site...
- Chris Aniszczyk
- Ugo Sangiorgi (ugo.sangiorgi at gmail.com)
- Chris Aniszczyk
- Mariot Chauvin