Fluent visual language editing: CS260 project proposal

This project implements and evaluates conventional and novel user interface techniques to improve editing fluency in a box-and-arrow visual language. User evaluation will be used to demonstrate the performance improvement (or otherwise) of the novel techniques, and to assess user satisfaction with the interface.

Problem

I have partially implemented a prototype of the user interface for the new version of Ptolemy, a signal processing simulation package. Although I have used relatively conventional drawing techniques in this prototype, I would like to evaluate more novel techniques. My goal is to improve the speed with which programs can be constructed and modified, where programs can contain up to several hundred graphical elements. This project considers only drawing, and not other parts of the interface.

Analysis

A screen shot of the current prototype is shown below. Editing is modeless, at least for the interaction completed so far: a connection is made by dragging from one terminal to another; a connection is modified by selecting an edge and dragging a grab-handle (shown in blue); moving a vertex reshapes all connected lines. This is the primary interaction mode. Some elements have a secondary mode, selected by control-clicking: an edge, for example, splits off a junction (solid diamond) and a new edge.

I am expecting the prototype to run into two problems. Firstly, it does not scale: with tens or hundreds of elements, drawing and modification will become tedious. Constraints or higher-order drawing operations are two possible approaches to increasing fluency for these situations. Secondly, additional modes of interaction will arise; for example, I will probably need to make it possible to add, move, and delete the terminals attached to icons. I would like to explore the use of gestures as a basis for providing a wider range of editing operations.

Design

The design of the interface is constrained by two main factors: it needs to use our existing simulation and UI infrastructure, which means writing it in Tcl/Tk; and it needs to be usable by both our development team and Ptolemy users, meaning, among other things, that any truly novel features I introduce need empirical justification.

The toolkit upon which the editor is built uses the concept of interactors from Garnet to handle input events. Addition of novel interaction techniques should, in theory, consist mainly of discovering and implementing new interactors and making them co-operate with other interactors.

The project has four phases:

  1. Completion of the initial prototype. For the purposes of this project, this requires adding shape editing to the edges. This is complicated by the fact that edges reshape themselves according to the conditions of their end-points.
  2. Initial user evaluation. The initial prototype will be measured for performance in a) creating and b) modifying a given program. The results of this evaluation will be used to refine the prototype and identify problem areas that need to be addressed by new techniques.
  3. Implementation of two new prototypes, one using conventional (select, drag) techniques, and one using constraints, higher-order interaction, or gestures.
  4. Final user evaluation. The two prototypes will be evaluated for user performance and satisfaction.

Experiment

Independent variables

The first user evaluation aims to obtain a benchmark performance, and there are thus no independent variables. The second user evaluation compares the two final prototypes. The conventional prototype will be measured with two sets of users: those from the first evaluation, and a new set.

Dependent variables

Each evaluation will gauge user satisfaction and will measure the time to perform a set series of tasks.

Participants

Three groups of users are required. Assuming four users per group, perhaps half to three-fourths will be graduate students from the Ptolemy group, and the remainder CS or EECS grad students. Participants ideally follow the profile of Ptolemy users, which (I am assuming) is highly computer-skilled but not familiar with user interface design.

Method

Users will be given a half-hour of training on one of the prototypes. Each user will then be asked to perform the following set series of tasks given a starting point. Successful completion means producing a program that is semantically identical and syntactically "close enough":
  1. Construct a small program
  2. Modify a small program
  3. Construct a large program
  4. Modify a large program

Results and discussion

If all goes as expected, user performance and satisfaction will improve in the revised conventional prototype, over the first one. I cannot predict whether the same will be true for the unconventional prototype, although the techniques adopted will obviously be chosen to attempt to produce this effect. If performance does not improve, then there may be problems with unfamiliarity, implementation (for example, using a mouse to gesture), or the way in which novel techniques were combined with conventional ones.

Grading contract

Implementation 30%
User evaluation 40%
Final report 30%


John Reekie, johnr@eecs.berkeley.edu