Diva Sketch Overview

Introduction

The sketch library provides a set of classes for constructing a sketch-based user interface. A sketch system supports freeform drawing and can be enhanced with a sketch interpreter that interprets the sketch. This document describes the basic sketch architecture and how to use this package to create your own sketch-based applications.

Architecture Overview

The following is a UML diagram illustrating the sketch architecture including the recognition architecture enclosed in the dashed box. The basic idea is as follows:

The sketch system uses call semantics. Control flows down the tree and information propagates back up the tree. More specifically, JSketch contains a SketchPane which receives sketch input in the form of mouse events. These events are passed to the sketch controller inside the pane and the controller asks its sketch interpreter to process these events. The interpreter in turn asks a gesture recognizer to recognize the events. When the recognizer comes back with an answer, the interpreter would request the controller to perform some action which may involve updating the application model and/or display.

The recognition system is designed to be application independent and is highly configurable. A recognizer processes a TimedStroke which is a collection of points taken from from pen down to pen up and generates RecognitionSet objects. A RecognitionSet object contains multiple Recognition object each of which is an analysis/prediction of the stroke.

For a more detailed UML diagram, please click on the image below.

Creating a simple sketch-based editor

Using the sketch package, it is very simple to create a sketch-based user interface. The following three lines are all you need:

JFrame f = new JFrame();
JSketch sketch = new JSketch();
f.getContentPane().add("Center", sketch);

This instantiates a canvas to sketch on. It does not provide gesture recognition however. What this piece of code does is that it creates a JSketch object and adds it to the center of a frame. A JSketch when instantiated sets up a sketch system which is depicted by the upper part of the UML diagram. When the user sketch on the editor, the mouse events are passed from JSketch all the way down to a sketch interpreter which by default is a BasicInterpreter. The job of a sketch interpreter is to map a gesture to what it means in an application. So it would use a gesture recognizer to determine what an input gesture is and tells the application to do something accordingly. In the default case, BasicInterpreter doesn't perform any interpretation. It simply updates the current figure being drawn. Referring to the UML diagram above, this simple example constructs a sketch system without the gesture recognition part (in the dashed box).

Creating an application-specific sketch-based editor

To construct an application-specific sketch-based editor, one would need to replace BasicSketchController and BasicInterpreter with application-specific controller and sketch interpreter. A sketch interpreter serves as a bridge between the application and the underlying recognition system. The idea is that the sketch interpreter receives mouse events from the application, asks the gesture recognizer to perform recognition on these events, and depending on what the gesture recognizer says, the sketch interpreter calls the controller to perform certain actions. For example, a user draws a circular shape in a sketch-based graph editor, and the sketch interpreter calls its recognizer to recognize the stroke. The recognizer may come back with a list of answers among which "circle" receives the highest score meaning it is most likely that the stroke is a circle. The interpreter may then call controller's "add node" function to add a node to the graph.

The following piece of code shows how to set up JSketch to use the application-specific controller (the interpreter is instantiated inside the controller):

AppController controller = new AppController();
SketchPane pane = new SketchPane(controller);
JSketch sketch = new JSketch(pane);