GTEDIT: A GESTURE-BASED VISUAL PROGRAMMING

ENVIRONMENT FOR TEACHING LISP

ジェスチャを用いた視覚的なLisp教育環境

by

Kenji Hara

原 謙治

A Senior Thesis

卒業論文

Submitted to

the Department of Information Science

on February 12, 2003

in partial fulfilment of the requirements

for the Degree of Bachelor of Science

Thesis Supervisor: Takeo Igarashi 五十嵐 健夫

Title: Assistant Professor of Information Science

ii

ABSTRACT

In conventional text editors, a Lisp program is written with many parentheses and it makes it

difficult to understand the tree structure of the program. Additionally, it is difficult for a student to

understand the teacher's intention by watching her editing operation in programming classes.

To solve these problems, we developed GTEdit (Gesture-based Tree Editor), a visual programming

environment for teaching Lisp. In this system, the user can intuitively add, remove, open, and close any

node in visualized tree structure using gestures. The gesture-based operation makes it easier for a

student to understand the teacher's intention by watching her operation.

We performed a user study to clarify the characteristics of GTEdit compared with a conventional

text editor. The result of the user study indicates that our method can facilitate better understanding of

programming compared with the conventional method.

論文要旨

従来のテキストエディタにおけるLispプログラムは多くの括弧を用いて書かれており、プ

ログラムの木構造が見えにくいものとなっている。また、テキストエディタで編集している様

子を見ても何をしようとしているのかがわかりにくいという問題も存在する。我々はそうした

問題を解決するためにジェスチャを用いた視覚的なLispプログラミング環境GTEdit

(Gesture-based Tree Editor)を開発した。このシステムにおいては、ユーザはジェスチャを用い

て、可視化された木構造のノードに対して追加、削除、開閉操作を直感的に行うことができ、

編集している人の意図の推測もしやすい。我々はテキストエディタと比較した場合の本手法の

特性を明らかにするために評価実験を行った。評価実験の結果からは我々の手法を利用するこ

とにより、従来の方法に比べてよりプログラミングの理解が深まる可能性のあることが示され

た。

iii

Acknowledgments I would like to thank Doctor Takeo Igarashi for his supervision and guidance, and for his

unflagging patience. I wish to thank the subjects for their time and thoughtful comments. Moreover, I

am grateful to all the members of Igarashi Laboratory for their useful discussions and suggestions.

iv

Contents

1. Introduction ..............................................................................................................1

2. Related work ............................................................................................................5

2.1. Visual languages................................................................................................5

2.2. Pen-based visual programming environments...................................................6

2.3. Empirical studies related to visual programming..............................................7

2.4. Graffiti usability.................................................................................................7

3. Interface....................................................................................................................8

3.1. Gestures .............................................................................................................8

3.1.1. Overview .....................................................................................................9

3.1.2. Node addition ............................................................................................11

3.1.3. Text entry ..................................................................................................12

3.1.4. Program execution.....................................................................................12

3.1.5. Node removal ............................................................................................12

3.1.6. Keyword completion .................................................................................13

3.1.7. Copy-and-Paste .........................................................................................13

3.1.8. Node insertion ...........................................................................................13

3.1.9. Node elevation...........................................................................................13

3.1.10. Open-and-Close.......................................................................................13

3.1.11. Free-hand comment.................................................................................14

3.2. Keyboard typing ..............................................................................................14

4. Implimentation .......................................................................................................15

v

5. Empirical study ......................................................................................................16

5.1. Subjects............................................................................................................16

5.2. Procedure .........................................................................................................16

6. Result......................................................................................................................19

7. Discussion ..............................................................................................................22

8. Conclusions and future work .................................................................................23

References ..................................................................................................................24

Appendix A ................................................................................................................27

Stroke Recognition Algorithm................................................................................27

Appendix B ................................................................................................................29

Content of the lecture conducted in the user study.................................................29

1. Expressions in Scheme....................................................................................29

2. Variable definition...........................................................................................30

3. Function definition ..........................................................................................30

4. If-expression....................................................................................................31

vi

List of Figures Figure 1. Screen shots of Tinkertoy.................................................................................................... 2

Figure 2. Node disconnection gesture in Tinkertoy............................................................................ 3

Figure 3. Node connection gesture in Tinkertoy ................................................................................ 3

Figure 4. Factorial program in Hyperflow.......................................................................................... 6

Figure 5. Screen shot of GTEdit ......................................................................................................... 8

Figure 6. Basic operations .................................................................................................................. 9

Figure 7. Advanced operations for efficient construction of programs............................................ 10

Figure 8. Advanced operations for reader of programs .................................................................... 11

Figure 9. Examples of node addition ................................................................................................ 12

Figure 10. Distribution of correctness of the answers to exercises .................................................. 20

Figure 11. Means and SDs(Standard Deviations) of the time for working out exercises A(A1, A2,

A3, A4) and B(B1, B2, B3, B4)................................................................................................ 20

Figure 12. Process of recognizing stroke.......................................................................................... 27

vii

List of Tables Table 1. Result of pre-test questionnaire ......................................................................................... 17

Table 2. Distribution of selected input device .................................................................................. 17

Table 3. Content of post-test questionnaire ...................................................................................... 17

Table 4. Distribution of the answers to the question “Which one is the better?”............................. 21

Table 5. Advantages of each editor that were answered by subjects................................................ 21

1

1. Introduction It is difficult to learn programming. One must codify their notions of specification and process, to

acquire an understanding of abstractions for which they may have no prior referents, and to express

these concepts in a formal style of language they have never previously encountered. In other words, it

is like trying to alter their belief system by quoting theoretical philosophy to them in hieroglyphics.

Therefore, it is important to teach programming to novices simply and intuitively. Teachers should not

select the programming language and environment that make it difficult for novices to understand the

essence of programming. We use Scheme[1], a lexically scoped dialect of Lisp, to teach programming

because its syntax is simple compared with other programming languages.

However, current Scheme programming education is cunducted textually, and it has following

problems. First, the tree structure of Scheme programs contains too many parentheses in text form. This

makes it difficult for novices to understand the structure. Second, it is not intuitive for novices to edit

trees by typing and deleting parentheses. Third, it is difficult for novices to understand the teacher’s

intention by watching her operation. It is because the teacher constructs trees by typing and deleting

parenthesis textually rather than visually adding and removing nodes. Last, users need to learn keyboard

typing to construct programs. If the user is not familiar with keyboard, she must learn not only

programming but also keyboard typing.

To solve these four problems, we propose GTEdit, a graphical tree editor for teaching Lisp. We

conducted a user study to clarify the advantages of the editor. GTEdit has three main features. First,

trees are represented visually. This makes the tree structure of programs clear. Second, users can

intuitively edit trees by using gestures. This help students understand the tearcher’s intention by

watching her gestures. Last, users can use this editor by pen. We adopted Graffiti[3] for text entry not

keyboard typing.

One of the reasons for using pen is that pen is faster than the keyboard for many tasks that involve

intermittent data input. This is true for entering and changing data in spreadsheets, editing documents,

creating presentations, and diagrams. Apte and Kimura[20] found that the pen was twice as fast as the

mouse for creating diagrams and charts.

Another aspect of the pen’s value stems from the particular needs of education. With the pen, the

child is touching the face of the computer and not a detached peripheral of it. She is using a motor skill

deeply associated with the learning process [4]. Learning takes place as the pen moves. In contrast with

2

keyboards and mice, learners comment that the pen feels more natural for “doing” math. This is

reminiscent of Hall’s [5] study of the critical role played by drawing — even doodling — to activate a

successful strategy for the solution of algebraic word problems. As Newman observes, “The pen is the

most common communications device in the world. . . . The pen is not a computer input device. It’s a

person output device.” [6,7]

There is another research for a visual programming environment for Lisp. The “Tinkertoy”[8]

programming system provides a graphical development environment for Lisp. Figure 1 shows the screen

shots of Tinkertoy. The visual representation used has an exact correspondence to Lisp S-expression

syntax. The head of each textual list is shown as a tree node, while each item from the remainder of the

list becomes a branch of the node. Branches can be Lisp atoms (numbers and symbols), other S-

expressions, or can be an “empty” branch, indicating that another expressions needs to be attached in

order to complete the expression. Some Lisp special forms, such as lambda-expressions and built in

arithmetic functions have special iconic nodes, while other functions and special forms simply use text

to label the tree nodes.

Figure 1. Screen shots of Tinkertoy

3

a)pick up b)pull away c)result of node

desconnection

Figure 2. Node disconnection gesture in Tinkertoy

a)drag start b)drag

continue

c)result of node

connection

Figure 3. Node connection gesture in Tinkertoy

Since there is a one-to-one equivalence between textual Lisp expressions and visual expressions,

Tinkertoy expressions and programs can be translated to and from Lisp code. This allows existing

textual code to be imported and manipulated visually, and allows visual code to be exported, executed,

and imported back into the system for inspection.

Lisp’s S-expression syntax is both loved and hated by programmers. On one hand it is elegantly

simple, and on the other it requires copious use of parentheses, and needs careful indentation to retain

legibility. Tinkertoy’s visual tree syntax keeps the elegance, while making plain the structure of

expressions without the parentheses.

Tinkertoy code is constructed by instantiating skeleton expressions in a workspace, and then

connecting heads of expressions in the workspace to empty argument branches. Figure 2 shows the node

disconnect operation and Figure 3 shows the node connect operation. New expressions are instantiated

by entering small textual Lisp expressions, which is something of a impediment to the visual editing

4

process. There is another problem in Tinkertoy. In the system, much of the space are wasted by

interconnections(which do not exist in text language) and oddly sheped stuctures.

GTEdit has several features that are not provided by Tinkertoy. First, programs in GTEdit can be

constructed without entering textual Lisp expressions. This makes it easy for novices to construct

programs. Second, each node in GTEdit is a descendant of the root node. This makes it possible to lay

out nodes orderly and saves spaces compared with oddly shaped layouts. Third, users can translate a tree

into a node by “Open-and-Close” operation. This operation can also save spaces and help users to read

programs efficiently. Last, GTEdit provides more gesture-based operations than Tinkertoy. Tinkertoy

provides only two gesture-based operations (node connect and node disconnect). However, our system

provide four basic gestures (node addition, node removal, text entry, program execution) and six

advanced gestures (keyword completion, Copy-and-Paste, node insertion, node elevation, Open-and-

Close, and free-hand comment). The gestures help users to edit trees intuitively.

In addition, the availability of Tinkertoy have not been studied empirically. We conducted a user

study to clarify the characteristics of GTEdit compared with a conventional text editor. The result of the

user study suggests that GTEdit has the potential of making it easy for novices to learn Scheme

programming.

We have organized the rest of this paper as follows; In section 2, we review related work on visual

programming and pen-based interfaces. In section 3, we describe the interface design of GTEdit. In

section 4, we explain the implementation. In section 5, we describe how we conducted the user study. In

section 6, we report the result of the user study. In section 7, we discuss about the availability of GTEdit

on the basis of the result. In the fnal section, we describe the conclusions and future work.

5

2. Related work This section describes previous work about visual languages, pen interface, empirical studies

related with visual programming, and Graffiti usability.

2.1. Visual languages A number of visual languages has been proposed for several purposes. We introduce the brief

history of visual languages and several systems. Sketchpad[10], designed in 1963 on the TX-2 computer

at MIT, has been called the first computer graphics application. The system allowed users to work with

a lightpen to create 2D graphics by creating simple primitives, like lines and circles, and then applying

operations, such as copy, and constraints on the geometry of the shapes. Its graphical interface and

support for user-specifiable constraints stand out as Sketchpad's most important contributions to

VPLs(visual programming languages). By defining appropriate constraints, users could develop

structures such as complicated mechanical linkages and then move them about in real time. We will see

the idea of visually specified constraints and constraint-oriented programming resurface in a number of

later VPLs. Ivan Sutherland's brother, William, also made an important early contribution to visual

programming in 1965, when he used the TX-2 to develop a simple visual dataflow language. The system

allowed users to create, debug, and execute dataflow diagrams in a unified visual environment.

The next major milestone in the genesis of VPLs came in 1975 with the publication of David

Canfield Smith's PhD dissertation entitled ``Pygmalion: A Creative Programming Environment'' [11].

Smith's work marks the starting point for a number of threads of research in the field which continue to

this day. For example, Pygmalion embodied an icon-based programming paradigm in which the user

created, modified, and linked together small pictorial objects, called icons, with defined properties to

perform computations. Much work has since gone into formalizing icon theory, as will be discussed

below, and many modern VPLs employ an icon-based approach. Pygmalion also made use of the

concept of programming-by-example wherein the user shows the system how to perform a task in a

specific case and the system uses this information to generate a program which performs the task in

general cases. In Smith's system, the user sets the environment to ``remember'' mode, performs the

computation of interest, turns off ``remember'' mode, and receives as output a program, in a simple

assembly-like subset of Smalltalk, which performs the computation on an arbitrary input.

There are several categories of visual languages. VIPR[12], Prograph[13], PICT/D[14], and

Cube[15] are classified into purely visual systems. In the systems, icons or other graphical

6

representations are manipulated to create a program which is then debugged and executed in the same

visual environment. Rehearsal word World[16] the hybrid of textual and visual languages. In the system,

users creates visual programs and then translated into an underlying textual language, or creates textual

programs by using graphical elements. Form/3[17] is another visual language that uses visualizations

descended from a spreadsheet metaphor.

2.2. Pen-based visual programming environments Very little academic research has been done redarding pen-based visual programming system.

Hyperflow[18,19] is a visual programming language on a pen-based computer system designed for

use by children. Hyperflow is a dataflow-based visual language intended for a range of software

development. Figure 4 shows the screen shot of the system. Hyperflow was implemented within the

PenPoint Operating System [21].

Figure 4. Factorial program in Hyperflow

PDAGraph[22] is an event-driven, component-based visual programming language for power users

of Handheld Computers or Personal Digital Assistants (PDAs) who are not necessarily professional

programmers. It is intended to run on a PDA and give power users the ability to create customized

applications. The language ensures that the user can create only syntactically correct programs by

placing and linking components or component items.

7

2.3. Empirical studies related to visual programming Several experiments have been done in visual programming research. Wayne V. Citrin[23]

presented an analysis of visual languages influencing diagram entry, and also the results of an

experiment comparing users’ performance on differnent types of diagram editors. The result shows the

effectiveness of pop-up palette and gesture-based approach, and the importance of the copy-and-paste

operation. In addition, they conjectured that the act of repeatedly setting down and taking up the pen

would meet with less user approval than the equivalent operation of letting go of, and taking hold of, the

mouse.

Michael D. Byrne, Richard Catrambone, and John T. Stasko[24] conducted two experiments that

examined the general claim that animations can help students learn algorithms more effectively. They

used animations and instructions that explicitly required learners to predict the behavior of an algorithm

during training. Post-test problems were designed to measure how well learners could predict algorithm

behavior in new situations as well as measure learners’ conceptual understanding of algorithms. In

Experiment 1, they found that when learners both viewed an animation and made predictions their

performance on novel problems improved compared to a control group’s, but the effect of the two

manipulations were not distinguishable. In Experiment 2, no effect was found for conceptual measures

of learning, but a marginally reliable effect similar to the one seen in Experiment 1 was found for

procedural problems. They concluded that the results from the two experiments suggests that the

benefits of animations are not obvious.

2.4. Graffiti usability I.Scott MacKenzie and S.Zhang[2] measured the immediate usability of Graffiti. They presented

four empirical measures of the immediate usability of Graffiti. They measured accuracy attainable after

minimal exposure. The first measure, 79%, was the inherent accuracy, or the extent to which Graffiti

strokes match letters in the Roman alphabet. In other three measures, they asked 25 subjects to enter the

alphabet five times into a pen-based computer under three conditions: (a) following minute studying the

Graffiti reference chart, (b) following five minutes of practicing with Graffiti, and (c) following a one

week lapse with no intervening practice. The accuracy was 86%, 97%, and 97%, respactively.

8

3. Interface This section describes GTEdit’s editing operations from the user’s point of view. Our tree editor

can be used with gestures or keyboard typing or both of them. Figure 5 shows a screen shot of GTEdit.

Program

execution area

Tree edit area

Control area

Figure 5. Screen shot of GTEdit

3.1. Gestures GTEdit provides several gesture-based operations. We first give an overview of the tree

construction process, and then explain each operation in detail. The editing gestures are carefully

designed to allow incremental learning by novice users. Users can create a variety of trees and execute

programs by learning four basic operations (node addition, node removal, text entry, and program

execution), and can incrementally increase the efficiency of construction by learning other operations as

necessary. We have found it helpful to restrict first-time users to the four basic operations, and then to

introduce other advanced operations after these basic opearations are mastered.

9

a)initial

state

b)start

drag

c)drag into

the parent

node

d)result of

node

addition

e)tap

node

f)transluent

rectangle

appears

g)input

stroke

h)result of

text entry

i)drag

from node

j)continue

drag

k)result of

program

execution

l)start drag m)cut

branch

n)result of

node

removal

Figure 6. Basic operations

3.1.1. Overview

Figure 6, Figure 7 and Figure 8 introduces our tree editor’s general tree construction process.

Figure 6, shows the basic operations, Figure 7 shows the advanced operations for efficient construction

of programs, and Figure 8 shows the advanced operations for a reader of programs. The user begins by

dragging into the root node. When the user drags into the root node, a new node is added (Figure 6a-d).

Once a node is created, the user can entry text in the node by using Graffiti or keyboard (e-h). A variety

of programs can be constructed by using operations above. The constructed programs can be executed

by dragging into the upper text area (i-k). In addition, unnecessary nodes can be removed by cutting

branches (l-n). These are the basic operations.

There are various advanced operations provided for efficient construction of programs. Keyword

completion is one of the most convenient operations for fast edit of programs. It can be performed by

drawing a given stroke as a Graffiti stroke (Figure 7a-c). Copy-and-Paste, which is also important

operation for efficient edit, is performed when a user drag a node into an other node (d-f). The user can

10

insert a node by dragging a node into the same node, and the contrary operation, node elevation, can be

performed by dragging from a node into its header (the blue rectangle on the left of the node) (g-j).

a)before

keyword

completion

b)input a

stroke from the

lower left to

the upper right

c)result of

keyword

completion

d)drag from

node

e)drag into

other node

f)result of Copy-

and-Paste

g)drag from node h)continue drag

i)drag into the

same node

j)result of node insertion

k)drag from

node

l)drag into the

header (the blue

rectangle on the left

of the node)

m)result of

node

elevation

Figure 7. Advanced operations for efficient construction of programs

Following two operations are desigined for the a reader of programs. Open-and-Close operation,

which changes the expressing way of trees, can be performed by dragging horizontally from a header

(Figure 8a-e), and a user can draw free-hand comment to help students understand the structure of

programs (f).

11

a)drag from

header

b)continue

drag

c)continue

drag

d)continue

drag

e)result of

Open-and-

Close

f)example of free-

hand comment

Figure 8. Advanced operations for reader of programs

3.1.2. Node addition

A user can add a new node by dragging into its parent node. The new node is added near by the

origin point of the drag. When the user drags into a node, a new node is added and highlighted in dark

blue (Figure 6a-c). Finishing the drag completes this operation (d). Figure 9 shows the examples of the

gesture. The operation fails if the arrow intersects existing branches.

12

a) b)

c) d)

e)

f)

g)

h)

Figure 9. Examples of node addition

(a,c,e,g: start drag, b,d,f,h: drag into the parent node)

3.1.3. Text entry

Users can entry text by using Graffiti. A translucent box appears when a user taps a node (Figure

6e-f). Users can entry text at the cursor position by drawing Graffiti strokes inside the box (g-h). The

algorithm used for stroke recognition is explained in the appendix.

3.1.4. Program execution

Programs in tree form can be executed by dragging them into the upper text area (Figure 6i-k). An

external interpreter executes the programs in actually. The output of the interpreter is written in the text

area.

3.1.5. Node removal

Removing node operation is performed by dragging across branches. When the user drags across

branches, the nodes connected by the branches are highlighted in dark blue (Figure 6l-m). The

highlighted nodes are removed when the user finishes the drag (n). This gesture gives a user the feeling

of cutting the branches that connect the nodes to remove.

13

3.1.6. Keyword completion

When a user want to input a keyword that is decidable from the first characters, keyword

completion is available. Keyword completion can be performed by drawing a stroke from the lower left

to the upper right in the transient box for Graffiti strokes (Figure 7a-c). This operation not only

completes the keyword but also adds appropriate numbers of children to the edited node. Users can

construct programs easily and quickly by using this facility.

3.1.7. Copy-and-Paste

Users can copy the existing node by Copy-and-Paste operation. The operation is performed by

dragging a node into another node (Figure 7d-f). This Drag-and-Drop operation replaces the content of

the drop-target node with the content of the drag-source node.

3.1.8. Node insertion

A user can insert a new node between a node and its parent node by dragging from a node into the

same node (Figure 7g-j). When the user drags into the node, the cursor image is changed to a right-

arrow. The node insertion is performed by finishing the drag at this time. This operation helps the user’s

incremental construction of trees.

3.1.9. Node elevation

There is a operation that is contrary to node insertion. This operation replaces a node with the child

node. When a user drags from a node to the header (the blue box on the left of a node) of the node, the

cursor image is changed to a left-arrow (Figure 7k-l). Finishing the drag at this time replaces the parent

of dragged node with the dragged node (m).

3.1.10. Open-and-Close

Users can transform a tree into a node by changing the expression. Programs are normally

expressed in tree form without parentheses, but it is possible to express them in text form with

parentheses. Users can change the expressing way by dragging horizontally from a header. When user

starts the drag, a vertical bar appears (Figure 8a-b). The nodes on right of the bar are expressed in text

14

form (closed), and the left nodes in tree form (open) (c-e). We call this operation “Open-and-Close”.

There are following advantages of Open-and-Close operation. First, users can read programs efficiently

by close the trees that are to be payed attention to. Second, users can not only construct programs

visually but also write programs textually.

3.1.11. Free-hand comment

Users can draw free-hand comment in background (Figure 8f). Effective free-hand comment can

make it easy for students to understand the structure of programs.

3.2. Keyboard typing A User can use GTEdit by keyboard. Many operations are available for efficient construction of

trees. Keyboard shortcut operations are so carefully designed that the user need not take her hands off

the keyboard.

15

4. Implimentation GTEdit is implimented as a Java program. The implimentation contains the execution of Scheme

programs. The inputted programs are executed by not GTEdit but an external interpreter. GTEdit only

communicate with the interpreter process. When a user input a program, GTEdit write the program to

the standard input of the interpreter. The standard output of the interpreter is read by GTEdit and

displayed in it. This makes it possible for a user to select any Scheme interpreter that she wants to use.

16

5. Empirical study We conducted a user study to clarify the characteristics of GTEdit compared with a conventional

text editor in programming education for novices.

5.1. Subjects Subjects were 6 undergraduate students, non-computer science majors. They had no previous

experience of programming in Lisp. Three subjects were in tree-text condition, and the other three were

in text-tree condition.

5.2. Procedure The subjects first filled out a pre-test questionnaires that include how they use computers. Table 1

shows the details of the result. In addition, the two who don’t use computer almost everyday had a little

previous experience of programming in Java and Fortran.

Next, subjects in tree-text condition were instructed in the use of GTEdit. After that, they were

taught the use of STk[], a Scheme interpreter. Subjects in text-tree condition were taught in the inverse

order.

The instruction in the use of GTEdit consists of four operations: node addition, node removal, text

entry, and program execution. After the instruction on GTEdit, we let them select the favorite input

device from a menu of choices: pen, mosue & keyboard, or keyboard. Table 2 shows the result of the

selection.

After the instructions, all subjects received four sections of a programming lecture. The content of

the lecture is showed in the appendix. The sections are 1) simple expressions in Scheme, 2) defining

variable, 3) defining function, and 4) if-expression. Each section consists of two subsections, A and B.

Subsection A is for an editor and B for the other editor. In the first and second sections, the subjects first

received subsection A, and then received subsection B. In the third and last sections, the subjects

received the two subsections in the inverse order. Each subsection consists of an explanation of the

content of the section and an exercise. We measured the correctness of the answer to the exercise and

the time for working out it.

17

In addition, Subjects in tree-text condition received each subsection A by using the tree editor and

each subsection B by using the text editor. Subjects in text-tree condition use editors in the inverse

order.

After the lecture, we asked them to fill out a post-test questionaires. Table 3 shows the content.

Finally, we discussed the advantages and shortcomings of each editor with the subjects on the basis of

the answers to the questionaires.

Yes No

Use computer almost everyday 4/6 2/6

Use keyboard almost everyday 4/6 2/6

Use text editor almost everyday 1/6 5/6

Interested in programming 0/6 4/6

Have previous experience of

programming

4/6 2/6

Table 1. Result of pre-test questionnaire Pen Mouse&Keyboar

d

Keyboard

0/6 5/6 1/6

Table 2. Distribution of selected input device

1. Which one is the better? (tree editor / same

/ text editor)

2. Please write the advantages of the tree

editor.

3. Please write the advantages of the text

editor.

Table 3. Content of post-test

18

questionnaire

19

6. Result This section describes how they used GTEdit, how they worked out exercises A(A1, A2, A3, A4)

and B(B1, B2, B3, B4) by using each editor, and what advantages of each editor the subjects answered.

In the user study, no subjects selected pen for using GTEdit. They answered that it’s because they

had little experience of using a digital pen and Graffiti compared with mouse and keybaord.

Figure 10 shows the distribution of the correctness of the answers to the eight

exercises(A1,A2,A3,A4,B1,B2,B3,B4) by using each editor, and Figure 11 shows the means and

Standard Deviations of the time for working out the exercises. The two results also indicate that the

subjects had less difficulty in working out exercises by using our text editor than conventional text

editor.

Table 4 and Table 5 shows the answers to the post-test questionnaire. According to the result, the

subjects found more advantages in our tree editor than the text editor.

20

0 1 2 3 4 5 6 7 8

Answeredcorrectly allby herselfAnsweredcorrectly withhelpAnsweredwrongly

Figure 10. Distribution of correctness of the answers to exercises

0

2

4

6

8

10

12

14

16

18

20

min Text Editor

Tree Editor

Figure 11. Means and SDs(Standard Deviations) of the time for working out

exercises A(A1, A2, A3, A4) and B(B1, B2, B3, B4)

Text Editor

Tree Editor

A B

21

Tree

editor

Same Text

editor

5/6 1/6 0/6

Table 4. Distribution of the answers to the question “Which one is the better?”

Edito

r

Content of advantages Frequenc

y

It is easier to understand the

structure of programs.

6/6

The tree editor prevents

users from constructing

programs that contain

syntax errors.

5/6

Node removal gesture

(cutting branches) is

intuitive.

4/6

This tree editor is fun 2/6

Tree editor is better for

constructing large programs

that are not fixed yet.

2/6

Tree

There is no need for flow

charts.

1/6

Text Text editor is better for

writing fixed small

programs.

2/6

Table 5. Advantages of each editor that were answered by subjects

22

7. Discussion The result of the user study indicates several advantages of our tree editor. The main advantages

are as follows. First, it is easier for novices to understand the tree structure of programs. Second, GTEdit

prevents users from constructing programs that contains syntax errors. Last, the node removal gesture is

intuitive. These advantages are answered by more than sixty five percents of the subjects. In contrast,

the subjects put forward only one advantage of the text editor. Less than thirty five percents of subjects

answered that they can write fixed small programs faster by using the text editor.

It is important to make it easy for novices to understand and edit programs from the point of view

of programming education. The first advantage of our tree editor helps novices understand programs,

and the second and third advantages help them edit programs easily. The only advantage of the text

editor seems to be effective for not novices but experts.

In addition, using GTEdit makes it possible to conduct common one-on-many classes by using

large touch panel displays as blackboards, tablet PCs as notebooks, and digital pens as chalk and

pencils. It seems feasible because the visual representation of programs makes it possible for students to

identify the tree structure of programs from a long distance. The identifiability also makes it easier for

students to copy the programs. In the classes, teachers can help the students understand programming by

drawing effective free-hand comment.

23

8. Conclusions and future work We have proposed a tree editor to solve the inherent problems in text-based Lisp programming and

conducted a user study to examine the tree editor’s characteristics. The result of the user study suggests

that GTEdit has the potential of making it easy for novices to learn Scheme programming.

However, there are still several aspects to clafiry. First, the user study donsn’t clarify the effect of

using pen. In the user study, we let the subjects select the favorite input device, and no subjects selected

pen. They answered that it’s because they had little experience of using a digital pen and Graffiti

compared with mouse and keybaord. Therefore, we must conduct a user study on those who have little

experience of using mouse and keybaord. Second, the programming lecture conducted in the user study

was not to multiple subjects but to single subject. However, a general lecture is conducted to multiple

students. Therefore, in the next user study, the lectures of programming must be conducted to multiple

subjects.

Another important research direction is to support a wider variety of use. Possible extensions are

parallel textual editing of programs by mode selection and extending to other languages (XML, ML,

etc).

24

References 1.R. Kelsey, W. Clinger, AND J. Rees. “Revised 5 report on the algorithmic language scheme”. 2003.

Available at http://www.swiss.ai.mit.edu/~jaffer/r5rs_doc.html

2.I. S. MacKenzie and S. Zhang. “The Immediate Usability of Graffiti”. Proceedings of Graphis

Interface '97, Canadian Information Processing Society. 1997. pp. 129-137

3.Palm Products - How to Enter Data into a Palm Handheld. 2003. Available at

http://www.palm.com/products/input/

4.“Silicon Paper and a Visual Interface for Neural Networks”. Proceedings of IEEE 1990 Workshop

on Visual Languages, Chicago. New York: IEEE, 1990. 241-246.

5.R. P. Hall. “Making Mathematics on Paper: Constructing Representations of Stories about Related

Linear Functions”. Doctoral diss., University of California, Irvine, 1990.

6.Newman, Jeremy et al. Position on Pens. Personal Electronic News. May/June, 1994, 44-62.

7.R. Grothe. Interview by author. October, 1991.

8.M. Edel. "The Tinkertoy Graphical Programming Environment," IEEE Transactions on Software

Engineering, no. 8, pp. 1110-1115, 1988.

9.M. Burnett. “Visual language research bibliography”. Technical report, Oregon State University,

2002. Available from http://www.cs.orst.edu/~burnett/vpl.html.

10.I. E. Sutherland. "SketchPad: A Man-Machine Graphical Communication System" in AFIPS

Spring Joint Computer Conference. 1963. 23. pp. 329-346.

11.D. C. Smith. “PYGMALION: A Creative Programming Environment”. PhD desertation, Standford

University, 1975.

12.W. Citrin, M. Doherty, and B. Zorn, "Design of a Completely Visual Object-Oriented

Programming Language," in Visual Object-Oriented Programming, ed. by M. Burnett, A. Goldberg, and

T. Lewis, Prentice-Hall, New York, 1995.

13.P. T. Cox, F. R. Giles & T. Pietrzykowski. “Prograph: a step towards liberating programming from

textual conditioning”. Proceedings of 1989 IEEE Work shop on Visual Lunguages,Rome, Italy, pp. 150-

56.

25

14.E. P. Gilnert, S. L. Tanimoto; "Pict: An Interactive Graphical Programming Environment". IEEE

Computer, Nov. 1984, pp 7-25.

15.M. A. Najork. “Programming in Three Dimensions”. PhD thesis, Universiti of Illinois at Urbana-

Chamaign, Urbana, Illinois, 1994.

16.M. Erwig and B. Meyer. “Heterogeneous visual languages - integrating visual and textual

programming”. Proceedings of 1995 IEEE Symposium on Visual Lunguages, Darmstadt, Germany,

1995, pp 318-325.

17.M. Burnett and H. Gottfired. “Graphical definitions: Expanding spreadsheet languages through

direct manipulation and gestures”. ACM Transactions on Computer-Human Interaction, 5(1):1-- 33,

March 1998.

18.T. D. Kimura. “Hyperflow: A visual Programming Language for Pen Computers”. Proceedings of

1992 IEEE Work shop on Visual Lunguages, Seattle, Washington, September 1992, pp 125-132.

19.T. D. Kimura. “Hyperflow: a uniform visual language for different levels of programming”.

Proceedings of the 1993 ACM conference on Computer science 1993, indianapolis, Indiana, United

States, pp 209-214.

20.A. Apte and T. D. Kimura. “A Comparison Study of the Pen and the Mouse in Editing Graphic

Diagrams”. Proceedings of Visual Languages 1993 Conference, Bergen, Norway, 1993.

21.R. Carr and D. Shafer. “The Power of PenPoint”, Addison-Wesley, 1991.

22.S. Armstrong, Y. Kollet, and T. J. Smedley. Visual Scripting for Handheld Computers.

Proceedings of IEEE 2002 Symposia on Human Centric Computing Languages and Environments,

Arlington, Virginia, USA, pp 68-70.

23.W. Citrin. "Requirements for Graphical Front Ends for Visual Languages", Proceedings of IEEE

Symposiumon Visual Languages, pp. 142-150, Bergen, Norway,August 1993.

24.M. D. Byrne, R. Catrambone, and J. T. Stasko. “Do algorithm animations aid learning?”.

Technical Report GIT-GVU-96-18, GVU Cener, Georgia Institute of Technology, Atlanta, GA, August

1996.

25.M. D. Gross and E. Y. Do. “Ambiguous intentions: A paper-like interface for creative design”.

Proceedings of the 9th Annual ACM Symposium on User Interface Software and Technlogy, pages 183--

192, Seattle, Washington, 1996.

26

27

Appendix A Stroke Recognition Algorithm

We next describe how the system recognizes the user’s Graffiti strokes. The recognition draws

upon the algorithm that Mark D Gross and Ellen Yi-Luen Do used in Cocktail Napkin[25]. The

recognizer’s job is to identify the best match in a given context for an input stroke along with a certainty

value for the match (0.0-∞ where 0.0 is most certain). Recognition of each stroke is performed

immediately after input. Raw data from the stroke is stored as a sequence of x,y points. In the first

processing step, the stroke’s bounding box is identified, and a new 5x5 square grid is identified by using

the information of the bounding box. The grid’s center of gravity is same as the bounding box’s and its

edge is five-fourths times as long as the longer of the bounding box’s edges. The stored point

coordinates are rectified to the 5x5 grid. The rectified coordinates are coded to a new sequence of

coordinates ((0,0)-(4,4)) that indicate the grid squares of the pen path. (Figure 12)

Figure 12. Process of recognizing stroke

(left to right)

After analyzing the input stroke, the recognizer compares the sequence against a library of

previously stored sequence templates, each of which records a scale factor and a range of allowable

values.

The match proceeds as follows: The recognizer calculates the distance between the stroke and the

template sequence. The distance is the value for match. It is returned by the following function

Distance(S, T).

28

S(N) is the function that returns the Nth point coordinate of the stroke

sequence, and |S| is the size of the sequence.

T(N) is the function that returns the Nth point coordinate of the template

sequence, and |T| is the size of the sequence.

SF is the scale factor of the template.

RoundUp is the function that rounds a given number

M = max(|S|, |T|)

D = { ((sx – tx)2 + (sy – ty) 2) 2 | I ∈ {1,…,M}, (sx, sy) = S(RoundUp(I * |S| /

M)),

(tx, ty) = T(RoundUp(I * |T| / M)) }

Distance(S, T) = ((ΣD) / M) * SF

If the value is in the range of allowable values, the template is selected as candidate.If there was no

candidates, the recognition fails.

Dots are treated as special cases. Any very small stroke is a dot.

29

Appendix B Content of the lecture conducted in the user study 1. Expressions in Scheme

Lecture A1 & B1. Input following expressions

1 => 1

(+ 1 2) => 3

(- 3 4) => -1

(* 5 6) => 30

(+ 1 2 3) =>

6

(+ 1 2 (* 3

4)) => 15

(= 0 0) => #t

(> 0 1) => #f

Exercise1. Calculate the following expression in Scheme

[A1] (23 + 49) * (511 – 43)

[B1] 12 * (2 - 13) + 87

30

2. Variable definition

Lecture A2 & B2. Variable definition is conducted as follows.

(define x 1)

=> x

x => 1

(+ x 1) => 2

Exercise 2. Define y as 3, and then calculate the following expressions.

[A2] 3 * (y + 5)

[B2] y * 5 – (7 – y)

3. Function definition

Lecture A3 & B3. Function definition is conducted as follows.

(define (plus x y)

(+ x y)) =>

plus

(plus 1 2) => 3

(plus 3 4) => 7

Exercise 3.

[B3] Define (square x) as the function that calculates the square of x, and then evaluate (square 29).

[A3] Define (f x y) as the function which calculates “(x + 3) * y”, and then evaluate (f 3 5).

31

4. If-expression

Lecture A4 & B4. If-expression takes three arguments (arg1, arg2, and arg3). The expression

returns arg2 when arg1 is #t, and returns arg3 when arg1 is #f.

(if #t 1 2) => 1

(if (= 0 1)

(+ 2 3)

(- 4 5)) => -1

(if (> 0 1)

1

(if (< 0 2)

(+ 3 4)

1)) => 7

(define (abs x)

(if (< x 0)

(- 0 x)

x)) => abs

(abs -4) => 4

(abs 4) => 4

Exercise 4.

[B4] Define the function (g2 x y) that returns 1 when y equals 0, returns x when y equals 1, and

returns x * x when y equals 2.

[A4] Define the function (max x y) that returns the larger number of x and y.

(the function returns x when x is larger than y, and returns y when y is larger than x)