Wayne Citrin
Department of Electrical and Computer Engineering
Campus Box 425
University of Colorado
Boulder, CO, USA 80309-0425
Tel: +1 303 492-1688
E-mail: citrin@cs.colorado.edu
Mark D. Gross
College of Architecture and Planning
Campus Box 314
University of Colorado
Boulder, Colorado, USA 80309-0314
Tel: +1 303 492-6916
E-mail: mdg@cs.colorado.edu
Adrienne Warmack
College of Architecture and Planning
Campus Box 314
University of Colorado
Boulder, Colorado, USA 80309-0314
Tel: +1 303 492-4897
E-mail: warmack@stripe.colorado.edu
ABSTRACT
As part of an ongoing project to develop drawing and diagram recognition systems whose functionality is distributed between personal digital assistant (PDA) front ends and host computer back ends, we have designed a collaborative drawing system that allows multiple users, at different locations, to collaborate in the construction of diagrams. The system has a number of advantages over previously proposed collaborative drawing systems. It employs inexpensive hardware supporting both pen-based entry and communications, it allows the easy integration of shape and handwriting recognition, it supports off-line drawing of diagrams which can subsequently be distributed and collaborated on, and it allows mediation of the process by a centralized diagram-recognition back end.
Keywords
Pen-based input, collaborative work, diagram recognition, personal digital assistants.
INTRODUCTION
In order to support collaboration over large distances, a number of collaborative drawing systems have been proposed and developed [2]. These systems allow users to communicate with written input in real time. We have developed a system supporting telewriting and collaborative drawing that employs inexpensive PDA (personal digital assistant) hardware. Using PDAs has a number of advantages. The systems are light and portable, and they support both wired and wireless communication, as well as built-in shape and handwriting recognition and graphical editing capabilities. The devices, though small, contain sufficient storage capacity that diagrams may be drawn and stored while the device is off-line (for example, while the user is working in the field, away from a computer network), and the diagrams may be shared later with collaborators.
Figure 1 shows a schematic of the collaborative drawing environment.
SYSTEM ORGANIZATION
We have created a system called SmartPad, that runs on an Apple Newton MessagePad. SmartPad employs facilities provided by the Newton MessagePad to provide all of the services offered by the drawing environment. It handles pen-based input and manages ink-gathering and storage of the recognized strokes. If so configured, it performs shape and handwriting recognition, using the Newton's built-in recognition facilities, and stores the recognized information along with the ink. It handles communications with other PDAs, allowing strokes to be transmitted as soon as they are entered, or allowing an entire diagram to be transmitted (for example, if it was entered in the field). It supports editing capability, allowing recognition of users' gestures (select, erase, move, resize), and it transmits that information to the other PDAs. Finally, it handles the display of graphical information, either entered with the pen, or through the communications port. The fact that both types of graphical information are displayed in the same space allows the system to be used for collaborative drawing. Details of the implementation of SmartPad may be found in [1].
Users may simultaneously draw on two connected PDAs without having to coordinate their actions. In order to avoid synchronization problems, we have made the objects immutable (as was done by Moran in the Tivoli system [4]), so that editing an object creates a new object. Editing conflicts, therefore, result in multiple objects being created, thereby providing immediate feedback and allowing users to back out. We have currently only tested the system on Newtons connected via a serial cable, although the hardware supports AppleTalk networks, modem-based communication, and wireless communication when the appropriate communications hardware is installed. We intend to investigate all of these configurations in the near future, as well as communication over the Internet, as soon as Newtons supporting TCP/IP become available.
HOST-BASED MEDIATION
We have designed the protocol so that other devices besides PDAs can receive the messages and participate in the conversations. In particular, we have configured the system so that a high-level diagram recognition system, the Electronic Cocktail Napkin [ECN] [3], can mediate the conversation. The ECN eavesdrops on the conversation between the PDAs and maintains its own version of the diagram being drawn. It analyzes the developing diagram to see if elements possess certain spatial relations it has been configured to recognize. If it finds that such relations exist, it notifies the PDAs which may display the information.
EXAMPLE SCENARIO
The following simple scenario suggests ways in which the collaborative drawing system may be used. Figure 2a shows a sketch of a building site made by an architect in the field. Note the mix of unrecognized ink and recognized shapes and text. (Recognition is turned on and off by the toggles in the lower lefthand corner of the display.) The architect brings the PDA back to the office, connects it to a network, contacts his colleague (by e-mail or telephone) in another office or another part of town, and transmits the diagram to the second architect. The first architect then begins to lay out the site (2b). Shape and text recognition are turned on where appropriate, and the graphical elements are transmitted to the colleague in real time. His colleague decides that the garage is poorly positioned, so he selects the garage and its label (2c) and moves it to a new site. He then redraws the path and driveway (2d). These changes are transmitted to the first architect's PDA and rendered as they are drawn.
FUTURE WORK
This is part of an ongoing project to develop diagram recognition systems whose functionality is distributed between front-end PDAs and back-end host computers [1]. Future work include adding multiple paging to the collaborative drawing system. SmartPad currently supports multiple pages, but we have yet to design a scheme to allow the pages in several sketchbooks to be synchronized. We are currently applying this technology to develop a system to aid field service workers. The workers will be able to download diagrams or photographs from a host computer database, containing up-to-date information on the hardware they are servicing, onto PDAs. The workers will be able to annotate the diagrams in the field to reflect the changes they have made, and later upload the annotated diagrams to the host computer, where the annotations will be interpreted and integrated into the database, using a system similar to ECN. A prototype of this system will be completed by Fall 1997.
ACKNOWLEDGEMENTS
This work was supported by the Colorado Advanced Software Institute, US West Advanced Technologies, and the NEC Corporation.
REFERENCES
[1] Citrin, W. and M. Gross, "Distributed Architectures for Pen-Based Input and Diagram Recognition," in ACM Workshop on Advanced Visual Interfaces (AVI '96). 1996. Gubbio, Italy, 132-140.
[2] Greenberg, S., S. Hayne, and R. Rada, ed. Groupware for Real-Time Drawing: A Designer's Guide. 1995, McGraw-Hill: London. 248.
[3] Gross, M. D., "The Electronic Cocktail Napkin - computer support for working with diagrams." Design Studies, 1996. 17(1): 53-69.
[4] Moran, T. P., K. McCall, B. van Melle, et al., "Some Design Principles for Sharing in Tivoli, a Whiteboard Meeting-Support Tool," in Groupware for Real-Time Drawing: A Designer's Guide, Greenberg, S., S. Hayne, and R. Rada, eds. 1995, McGraw-Hill: London. 24-36.