Dr. Ben Bederson, Assistant Professor, Human-Computer Interaction Lab,
University of Maryland
interviewed by Jon Byous
The audience "oohs" and "aahs" as four people move paint cursors
independently across a single computer screen, each drawing their part
of a child's school house. These four people each have a mouse, and
they are driving four simultaneous inputs into one system. Everyone is
impressed, even the nine-year-old.
Is there a way that computers can better support people working
together, really together? That is the question that Dr. Ben Bederson
wrestles with day and night.
His conclusion is that, "people work together best when they are
together in the same room, using the same computer."
So Dr. Bederson has devised applications that demonstrate the value of
a "zoomable interactive space with information visualization for
multiple simultaneous interactive users."
How did this idea for
your research and development come about?
People just seem to
collaborate, or work together, better when they are in close
proximity, seeing each other in person.
Human bodies are extremely powerful input devices. We're more
than eyes and ears. We sense the presence of someone, even if we
don't see them. We receive a lot of information from small
things, like the angle of the listener's neck and position of
Humans are very perceptive at minute, subtle differences,
especially in face-to-face communication with another person, or
But computers don't
support that environment?
Not yet. But, I can't
imagine that ten years from now, computers will be used by only one
person at a time. It almost has to be that computers will support
multiple simultaneous users. The only question for me is not 'if' but
'when' and 'how.'
We're taking the first steps in answering that.
How are you approaching
My research mission is to
question the assumptions that computer designers have made implicitly
through history in the design of computers. There are two basic
assumptions that I am exploring.
The first is the assumption that the correct interface for a
computer is a "Windows/Office" style interface. This WIMP
(windows/icons/menus/pointers) interface came out of Xerox PARC
in the late 70s, it worked, it was successful, and it stuck. But
does an interface have to be that way?
The second assumption is that a computer is designed for one
person sitting in front of it. In practice, people commonly work
together with more than one person in front of a computer at a
time. There might be two people debugging, or two people writing
something, or one person showing someone else how to use
something, or several people being creative together,
brainstorming, searching. That's how people work together. And
computers don't support that type of collaboration with more
than one user.
Of course, there are significant challenges to be solved with
two people sitting in front of a single computer. To build a
system that supports multiple simultaneous users means solving
ergonomic design problems and physical space issues. There's not
enough display space. It puts more demands on the computer, and
there are command conflicts with multiple simultaneous inputs.
The first way I have addressed these assumptions and problems is
to break out of the box. The computer screen is a terrible
display device compared to the real world, but we can take
advantage of it as a medium by remembering that it has a
powerful computer attached to it. So instead of having documents
in windows that mimic paper as a direct metaphor, I've taken
elements of the user data and laid them out spatially on the
screen. This way, we can take advantage of the fact that humans
are highly tuned to recognizing motion and understanding visual
The result of this work so far is three related technologies:
- Jazz is a platform for 2D+zooming object-oriented graphics.
- MID (Multiple Input Devices) is a toolkit that supports multiple input devices.
- KidPad is a multi-user storytelling tool for children that builds
on Jazz and MID
The ability to zoom in
and out is an essential feature of your work. Do you use 3D also?
I opted not to use
3D. For information that is more abstract and semantic, 2D plus
zooming may actually be better. And it is definitely less complicated
-- both from the implementation side, and from the user's point of
With 2D zooming, you can "move" out for an overview of the data
and move in for more detail. You still have the simplicity of a
WIMP interface, but it doesn't have to be more complex than just
clicking on something to go to it.
Of course, there is a place in many applications for WIMP
interfaces. But not all of them. The KidPad application doesn't
have a single button, pull-down menu, or tool palette. It's all
just very smooth interactive graphics in an immersive space.
And for the times you do want traditional interfaces, Jazz
supports embedding Swing widgets within the zoomable space. So,
our goal is to support the best of both worlds, mixing
interactive quality graphics with zooming, and traditional
What is a zoomable user
interface best suited for?
A zoomable interface is
very good for the process of storytelling, and making presentations.
A key element of storytelling is spinning a web that shows
relationships - being able to relate parts of your content to other
parts. So an interface is needed where the contextual relationship
between your information parts is an implicit part of the interface
itself. And also, storytelling and presenting are tasks where visual
aesthetic and entertainment are important components to getting the
job done. If you are not engaging your audience, then you are not
communicating with your audience. So having an interface mechanism
that is engaging is not irrelevant, it's part of the task. It's not a
bonus, it is the task.
So all of those requirements have to come together in an interface
designed for storytelling and giving presentations, and that's why I
have focused on zoomable interfaces.
How do you set about
building a zoomable interface?
In a way, a zoomable
interface is 2D graphics with this extra parameter of scale. By far,
the easiest way to do this is in an object-oriented way.
First, you have one object that corresponds to each graphic
element. Then I use 2D affine transforms, already built into the
Java 2 platform, as mechanisms to
translate, scale, rotate, or shear any object in the space. These
objects -- text, images, graphics -- can then be grouped in space in a
way you might think of as slides in a presentation.
Then you need a "camera" to move around and look at different
places in this space -- to do sophisticated information
visualization. And you want to be able to change the "view" of
the camera, for example, with zooming lenses, fisheye views, and
So, we built a platform to do this called Jazz. It is an
object-oriented structure -- built entirely on the Java 2 platform --
for building 2D objects with transforms and camera views.
Jazz is the implementation of the Java platform that supports 2D
object-oriented zoomable graphics, multiple views, and animation. It
is analogous to the Java 3D API,
but for 2D graphics. In fact, it uses the Java 2DTM API-based renderer as its functionality
How does it work?
The Java 2 platform
provides a wonderful renderer and GUI tool kit. But it lacks the data
structure for building 2D object-oriented graphics. Jazz is the
platform that makes that easy for developers. It is a 2D scenegraph,
or hierarchical structure, that sits on top of the Java 2D
Using a scenegraph is not the standard way of thinking about 2D
graphics, but I think it is important. It is the structure that
manages hierarchical relationships among nodes.
Different node types provide functionality, such as grouping,
fading in and out, transforming object stickiness, and
animation, among others. Then there are visual components that
plug into these nodes. They're a very simple class that define a
paint method, a bounds method, and a pick method, allowing the
object to be plugged into the scenegraph and rendered with all
of the power of the renderer.
With this structure of basic visual components, Jazz is very
extensible for developers to define their own visual components
that will then plug into the scenegraph. In addition, if
developers want to add new interactive functionality, developers
can extend the node class to define a new node type that will
fit into the scenegraph structure.
It sounds wildly
That's an important
goal. Jazz is primarily funded by DARPA for the Command Post of the
Future project. We are building a platform that will be used in many
different ways. We know from our experience of thinking about these
interfaces for a number of years, that extensibility is key. Even
though we have ideas about what's interesting, we can't possibly add
all of the features that could be used. The best thing that we can do
is to provide a structure which makes it easy for other developers to
add their own features.
And because the Java 2 platform is so flexible, and allows dynamic loading,
you can do this very dynamically. You can load in a module that
defines a new node type or a completely different layout manager
developed by someone else, or from the Internet.
What drove your decision
to choose Java 2 technology?
I started this research
at the University of New Mexico with my collaborators several years
ago. At that time, I built the initial program in C++, tightly tied it
to UNIX® and X, and hooked it up
to the Tcl/Tk scripting language.
What we found was that it was very simple to use for mocking
things up, or prototyping. But it was very difficult to extend
it in any meaningful way. And it was tied to UNIX, which most of
our target users didn't have.
When Java 2 technology with the Java 2D API came along, it gave us everything we
needed: a clean language, good object oriented design, freedom
from low-level details, such as memory management, and access to
substantial utility libraries. It also had a high-quality 2D
renderer that supported rendering anything in any scale.
Finally, it ran under both Microsoft Windows and UNIX.
How big is your
development team and how long did it take to develop Jazz?
There are five of us: one
staff programmer, three students, and me. We did the redesign and
coding of Jazz from its original C++ version using Java 2 technology in just
under a year. And that includes the development of MID and KidPad.
Is Jazz available to
Yes. Jazz is entirely
open source so it is freely available on the web site, and we
encourage people to use it and tell us what you think, and make
contributions to it.
We want people to download the platform, use it, and give us
feedback, preferably with code additions! We will continue to
offer new releases of Jazz on the web. While Jazz is open
source, we defined it in such a way that people are able to
develop proprietary applications based on it.
I'm strongly motivated by the Java technology community source
model. I wouldn't be giving away open source code if it didn't work
with the Java technology community. But it does clearly work. In fact,
I'm able to do all kinds of things in building these applications that
I couldn't do if I didn't have the Java 2 source code.
What is the Single
Display Groupware application you are working on?
At the University of
Maryland, we have been working in collaboration with researchers in
Sweden and England, funded by the European Union, on a project called
"KidStory". Our goal is to support kids with new collaborative
storytelling tools. By doing so, hopefully kids can learn
communication, collaboration, and literacy skills. We work with
children as our design partners which also leads to some fascinating
challenges in our research process.
Through this, we've developed an application using Jazz called
KidPad, which gives children as young as five years old the
ability to have fairly advanced computer skills. They have the
ability to do creative authoring, zooming, hyperlinks, scene
manipulation, and animation. This is all done in the application
in an entirely graphical way with direct manipulation without
any pull-down menus or tool palettes.
But the most interesting part is that we do this with several children
simultaneously working on one computer. Each computer has several mice
plugged into a Universal Serial Bus port, and the software enables the
mice to work independently.
When one child joins another, they collaborate, say, by drawing an
object together or by drawing separate objects on the same screen
simultaneously. When their drawing tools are in close proximity,
special properties take over to enhance the collaboration, allowing
them to, for example, fill in the space between their "crayon" tools
as one fat crayon or squash and stretch objects by working
together. Various tools have different properties when they are
working with another user's tool.
We had to develop the Multiple Input Device (MID) software ourselves
to do this, because most operating systems are based on the assumption
that a computer has only one mouse and one keyboard.
MID is a Java technology-based toolkit that parallels the traditional
Java technology-based event mechanism but provides access to
independent sources. It gives you the option of creating different
Java platform modules for different event sources. We have one so far. It
supports multiple USB mice on Windows 98.
So if you run your application to use MID events, rather than regular
events, every event has an extra method that returns the device
that generated each event. Or, you can add a MID event listener that
only fires when events come from a specified device. It's very
clean. MID is designed to support many different types of event
devices so other people can write their own event sources, such as
gestures for pen input.
Here again, Java 2 technology was our choice. It has a very nice event listener
architecture that we were able to extend to support other event
types. And because Jazz is written for the Java 2 platform, we were able to write an
application such as KidPad that uses both Jazz and MID.
It's all very flexible.
To contact Dr. Bederson, email him at: firstname.lastname@example.org.
The Jazz Home Page
The MID Home Page
Dr. Bederson's Home
About the Author
Jon Byous has been writing about technology -- from chips to
mainframes -- for twenty years, working for Sun, Cisco, Oracle,
Netscape, Quantum, and dozens of smaller tech firms. Jon has written
over 25 articles about Java technology, a passion of his, for Sun's
Java technology web site.