Less gear, better interactions: A UW talk on augmented reality

Keypad in hand

Image via seriousgamesmarket.blogspot.com

After attending the University of Washington Dub Talk “High-Fidelity Augmented Reality Interactions Without Goggles, Gloves or 3D Trackers” by Hrvoje Benko, researcher from Natural Interaction Research group at Microsoft Research, I decided to share some things I learned and some things the talk made me think about.

As the title suggests, the talk focused on discussing how technologies such as projectors and depth-sensing cameras can provide a rich augmented reality experiences without the need of using extensive gear.

Compared to the developments in computer graphic, which are allowing people to have increasingly immersive experiences, the primary ways of interacting with a computer have stayed relatively the same.

Benko compared the richness and diversity of interactions humans have in the real world to the limited “See, Hear, Touch” we generally have in the digital world. He believes that in order to change that, we should look away from the commonly employed method of trying to augment the person, and augment the space instead. By demonstrating examples of his research projects, Benko outlined two primary principles of augmenting the space.

Augmentation of space is very much about detection of surfaces that are being physically interacted with, and then consequently augmenting it through projection.

Since we don’t have much computational control of the physical environments, detection of surfaces becomes a difficult task. The are two primary ways to detect a surface: analytically and empirically.

Analytically detecting interactions presumes knowing the geometry of the space and then telling the computer where it is. Interactions with projection-mapped artifacts is an example of this approach.

Empirical detection of interactions involves detection of some change in either in the surface or the user itself. This approach would involve, for example, tracking a tip of a person’s finder and detecting when a surface is touched by means of a thermal camera.

Detection of surfaces becomes really problematic when the surface is not static, but is dynamically changing as a result of person moving in space. For example, interaction with a virtual keypad projected on your palm by a mini shoulder projector is possible, but difficult due to the complexity of tracking not only the pointer, but the target as well. Benko discussed how his research addressed these technical challenges and shared examples of two of his projects, LightSpace and Mirage Blocks.

The idea and application potential of augmented reality is exciting, yet still only real to many of us in science fiction films such as Minority Report, among others.

It was particularly intriguing to see such substantial technological progress, achieved by Benko and others in this field, towards making augmented interactions increasingly real.

Being an Interaction Design student myself, the talk left me wondering about the introduction of these augmented reality technologies in our everyday lives, and more specifically about the ways of applying these technologies to support the demands of everyday interaction in the physical world.

How can we design interactions that do not simply extend existing technologies into more personal domains, but actually respond to the demands of that domain, such as safety, privacy, social dynamics, etc.?

Here I am concerned with avoiding creating interactions that perpetuate unfortunate consequences such as, for example, texting while driving, or disrupting social dynamics by using a mobile phone. Those are all side-effects that could be avoided if only we anticipated them.

Anticipating the consequences and adopting technology, rather then just extending it “as is” to new realms, requires redefinition of what a particular technology is and and how it works.

The way we define a computer has not really changed much since the introduction of the first personal computer in the 1970 by the Xerox Corporation. The definition of a computer still remains primarily as an computational general-use device consisting of a box with a screen. And that is what we try to extend to our everyday activities. But since the real physical world interactions are different from the those supported by the “desktop” metaphor (storing and retrieving files) of the early computer, thus the role of the computer and how we interact with it should be redefined accordingly.

Benko and others in his field are certainly contributing to the redefinition of how humans communicate with a computer, but I think it is just as important to redefine computer’s role itself as it is being extended into the everyday activities of our lives. For example, what does a computer help you accomplish by being in your pocket no matter where you are and what you do? Is it merely a window though which we connect to the digital universe that often has nothing to do with our immediate context? Or is it instead an unobtrusive and useful companion that is sensitive to our context?

Although augmenting the space with immersive gear-free experiences is valuable and exciting progress, I am confident that such developments should be accompanied by studies of social, cultural, and psychological demands of everyday interactions and the way computers can be adopted to meet those demands.