Jump to: navigation, search

Difference between revisions of "Sketch/Proposal"

m
m
Line 16: Line 16:
 
The Sketch API is target primarily at editors with a large set of elements, in which the user have to constantly search for elements to add on the GEF Palette.  
 
The Sketch API is target primarily at editors with a large set of elements, in which the user have to constantly search for elements to add on the GEF Palette.  
  
[[Image:SketchAPIExample.png |400px| Application prototype for Sketch API‎ ]]
+
[[Image:SketchAPIExample.png |center|400px| Application prototype for Sketch API‎ ]]
  
 
=== Sketch Recognition ===
 
=== Sketch Recognition ===
 
The recognition is being made primarily with Levenshtein's algorithm for string distance. The user's sketch is transformed in a set of points placed on a grid, then each point is transformed to a number according to the next point's position. This approach is described in detail here (http://www.springerlink.com/content/u0r7261w01248q1t/).  
 
The recognition is being made primarily with Levenshtein's algorithm for string distance. The user's sketch is transformed in a set of points placed on a grid, then each point is transformed to a number according to the next point's position. This approach is described in detail here (http://www.springerlink.com/content/u0r7261w01248q1t/).  
[[Image:Levenshtein.png‎ |300px| Levenshtein applied to sketch Recognition‎ ]]
+
[[Image:Levenshtein.png‎ |left|300px| Levenshtein applied to sketch Recognition‎ ]]
  
 
Each element has a bank of "words" that describes its many forms, once drawn by the user. The recognizer compares each new sketch with the words of each element, calculating the distance between them. If the distance is smaller for a square than for a triangle, for example, then the element is probably a square.
 
Each element has a bank of "words" that describes its many forms, once drawn by the user. The recognizer compares each new sketch with the words of each element, calculating the distance between them. If the distance is smaller for a square than for a triangle, for example, then the element is probably a square.
Line 28: Line 28:
 
This API is composed of three parts: a sketch '''Tool''', a '''Recognizer''' and a sketch '''Bank'''.  The figure below shows how the API integrates with GEF, through AbstractTool and GraphicalEditor.
 
This API is composed of three parts: a sketch '''Tool''', a '''Recognizer''' and a sketch '''Bank'''.  The figure below shows how the API integrates with GEF, through AbstractTool and GraphicalEditor.
  
[[Image:Sketch_Class_Diagram.png |400px| Class diagram for Sketch API‎ ]]
+
[[Image:Sketch_Class_Diagram.png |center|400px| Class diagram for Sketch API‎ ]]
  
  

Revision as of 15:52, 1 February 2010

Draft proposal

This is a draft proposal, feel free to contribute to it!

Introduction

With the increasing popularity of touch-enabled devices nowadays, it is getting more common to insert information directly on the screen, using the fingers or a pen, instead of using a mouse. This project aims to enrich user's interaction with GEF/GMF editors.

Sketch provides an API to allow users to insert elements (EditParts) on GEF/GMF editors by capturing its gestures, typically using a pen on a tablet. The idea is to allow user might freely draw the representation of an element on the editor's area. The API would then transform this rough user's representation on something processable, interpreting what element the user meant to insert.

The API might learn from user's input, recognizing each sketch in the way it is drawn by different users. In cases when it is impossible to determine what element is meant by a sketch, the API 'asks' the user, through a small dialog, what he/she meant by that sketch. Thus the API is able to learn for the next times.

Sketch is proposed as an open source project under the Eclipse Technology Project (http://www.eclipse.org/technology/). This proposal is still in the Project Proposal Phase, and is being made in order to call for more community participation. You are invited to comment on and/or join the project. Please send all feedback to the http://www.eclipse.org/newsportal/thread.php?group=eclipse.technology.sketch newsgroup.

Background and Goal

The Sketch API is target primarily at editors with a large set of elements, in which the user have to constantly search for elements to add on the GEF Palette.

Application prototype for Sketch API‎

Sketch Recognition

The recognition is being made primarily with Levenshtein's algorithm for string distance. The user's sketch is transformed in a set of points placed on a grid, then each point is transformed to a number according to the next point's position. This approach is described in detail here (http://www.springerlink.com/content/u0r7261w01248q1t/).

Levenshtein applied to sketch Recognition‎

Each element has a bank of "words" that describes its many forms, once drawn by the user. The recognizer compares each new sketch with the words of each element, calculating the distance between them. If the distance is smaller for a square than for a triangle, for example, then the element is probably a square.

Architecture and Extensibility

This API is composed of three parts: a sketch Tool, a Recognizer and a sketch Bank. The figure below shows how the API integrates with GEF, through AbstractTool and GraphicalEditor.

Class diagram for Sketch API‎


Project Scope

TODO

Future Roadmap

  • Add more and more robust interpreters at the recognition chain
  • Support for GEF
  • Support for GMF through sketch models related to .gmfgraph (one group of primary sketches for each graphical node)

Concerns

TODO

Packaging and Deployment

Sketch will ship in the form of an Eclipse update site...

Organization

Mentors

  • Chris Aniszczyk
  •  ??

Initial committers

  • Ugo Sangiorgi (ugo.sangiorgi at gmail.com)

Interested parties

  • Chris Aniszczyk
  • Mariot Chauvin