Maze Experiment #4
Maze Experiment #4
Factors affecting representations
The results of experiment III suggested that VE’s
allow accurate
and useful survey knowledge of large-scale real world environments.
However, the VE used in experiment III had features that were designed
specifically to assist the formation of survey knowledge. Experiment IV
was conducted to determine what minimal features of a VE are necessary to
allow survey knowledge. We compared the representations of people after
experience with a wireframe VE, a fully rendered VE maze, and a real-world
walk-through of a maze.
A partial account of this experiment was presented at the 39th annual
meeting of the Psychonomics Society, Dallas TX, 21 Nov. 1998.
A postscript file of this paper can be downloaded here.
Or click the Acrobat icon to download a .pdf file of the paper 
ABSTRACT
Spatial knowledge in virtual environments (VE’s) is often evaluated using
performance measures acquired in the VE. We show that pointing errors
measured in a VE are highly predictive of pointing errors in the real
world; however, errors in distance estimations made in a VE are not as
predictive of distance errors in the real world. We examine factors that
affect bearing and distance estimations made in real vs. virtual
environments and find that gender is highly influential.
INTRODUCTION
In the last decade, there has been considerable interest in using
computer-generated environments (virtual environments – or VE’s) for
training spatial knowledge. Because VE’s are able to depict
three-dimensional spaces interactively, they offer a promising medium for
training people about the spatial characteristics of places and situations
that are rare, remote, or dangerous. For example, VE’s can be used to
train firefighters about a building’s layout before they must enter it to
put out a fire (see Bliss, Tidwell, & Guest, 1997). Another promising
application of VE technology is as a research tool for understanding human
spatial cognition. In addition to enabling participants to explore large
spaces within the confines of the laboratory, VE’s also allow
experimenters more control over stimulus characteristics than they
typically have in the real world. Indeed, several recent
neurophysiological studies of spatial cognition have used VE’s as stimuli
to draw conclusions about real-world spatial cognition (Aguirre &
D’Esposito, 1997; Maguire et al., 1998).
Training and research applications that use VE’s as substitutes for
real-world spaces raise two important questions. First, when spatial
knowledge is acquired in a VE, what is the degree to which measurements
taken in the VE can substitute for similar ones in the real-world? This
question is especially important for training applications in which
assessment of the trainee’s capabilities must be made before transfer.
Second, before generalizing results found in a VE to real-world cognition,
it is important to know the degree to which spatial knowledge acquired in
a VE is comparable to that acquired in the real-world. A recent study by
Ruddle, Payne, and Jones (1997) focused our attention on these questions.
Ruddle et al. (1997) trained people to learn the spatial layout of a
virtual office building and later measured their knowledge of it while
they were in the VE. They then compared these results directly with those
from a widely-cited study of spatial cognition in a similar but real
office building (Thorndyke & Hayes-Roth, 1982). Before making such
comparisons between measurements acquired in a VE and those acquired in
the real-world, we feel that it is necessary to assess the degree to which
the understanding of a virtual space predicts that of the real world space
on which it was modeled. It is also important to understand the degree to
which spatial knowledge acquired in a VE is systematically different than
that acquired in the real world.
We addressed these issues by exposing people to two maze environments—one
virtual, the other real. We then tested participants’ knowledge of
distances and directions between objects in these mazes. To examine the
transfer of spatial knowledge, participants were also tested in a
real-world maze after learning in a virtual one.
METHOD
Participants
The participants were 27 students (12 men) enrolled in an introductory
Psychology course at the University of Washington. Twenty-one of the
participants received extra credit for their participation. The remaining
six were paid $10 per hour.
Materials
The real-world environments were two 4.88 m x 4.88 m mazes constructed
from 2.13 m black curtains hanging from an overhead grid of cables. The
system of cables and curtains allowed the experimenter to reconfigure the
mazes rapidly between conditions of the experiment. Mazes were covered
with white fabric to reduce the amount of directed light entering them.
Four prominent landmarks (in the real world : three large cardboard
letters and a cardboard box; in the VE : a ball, a violin, a sword, and a
gun) were placed at fixed locations in either maze. Figure 1 illustrates
the configuration of the mazes.
Figure 1. Schematic maps of the mazes used in the experiment. T’s denote
the location of the targets to which participants pointed and estimated
distances.
For the virtual condition of the experiment, these mazes were modeled in
World Up by SENSE8 and run on a Pentium Pro 200 using a Diamond FireGL
3000 graphics accelerator board. We used color and fog effects (but no
texture mapping) in the VE maze to enhance its interpretability.
Participants viewed the VE while sitting 38 cm from a 35 cm x 26.5 cm
monitor with a resolution of 1152 x 900 (32K colors, 76 Hz refresh). The
speed of this system was approximately 12.64 frames per second.
Navigation in the VE was controlled with a Thrustmaster PFCS joystick,
providing three degrees of freedom of movement: translation in either
dimension along the ground as well as the ability to pan the viewpoint
around the vertical axis (yaw).
Procedure
Participants were first trained to navigate in a VE with a joystick until
they could complete a ‘virtual obstacle course’ in under five minutes.
They then practiced pointing and estimating distances in feet and the
experimenter thoroughly explained the nature of their tasks to them.
Participants then learned both a real-world and a VE maze. The order of
presentation and the maze configuration were counterbalanced across
participants.
In the virtual maze, participants were given as much time as they wanted
(minimum of seven minutes) to explore and learn the relative locations of
objects. When the participant indicated that he or she had adequately
learned the virtual maze, we tested his or her knowledge of it : first in
the VE (virtual measurements), and then in the real world (transfer
measurements).
Each participant provided nine bearing and six distance estimations while
in the VE. For bearing estimations, participants were placed directly in
front of one of the maze objects and instructed to rotate their viewpoint
to the direction of one of the three other targets. A set of cross-hairs
superimposed on the monitor screen helped participants align the direction
of their viewpoint on the target. After each bearing estimation,
participants estimated the distance to the target in feet. The three
symmetric distance estimations (e.g. A to B after having already estimated
the distance between B and A) were not used in subsequent analyses.
For the transfer phase of the experiment, the experimenter escorted the
participant into the real world version of the (virtual) maze that had
just been learned. Participants made eight bearing and distance estimates
from four fixed locations to various unseen (and un-passed) objects in the
maze. Bearing estimations in the real world were measured with a dial
mounted on a 1.04 m stand. Participants rotated the dial to point in the
direction of each target, and the direction was recorded to the nearest
degree. After each bearing estimation, participants estimated the
distance to the target in feet.
Each participant was also given as much time as they wanted (minimum of
four minutes) to explore a real-world maze (see Figure 1). After learning
the real-world maze, participants made eight bearing and distance
estimations within it using the dial described above. Table 1 summarizes
the three main types of measurements that we obtained.
Table 1. The three measurement types taken repeatedly from each
participant.
RESULTS
Mean signed distance and bearing errors for the three types of
measurements (virtual, transfer, and real) were computed for each
participant by averaging the differences between estimations and actual
quantities. Averaged over all participants, mean signed bearing errors
(Mreal = -0.45°, SD = 14.06; Mtranfer = -0.59°, SD = 23.84; Mvirtual =
-5.48°, SD = 20.97) were not significantly different from zero, nor were
they significantly different between measurement types (F(2, 52) = 0.52, p
= .60). Figure 2 illustrates participants’ signed bearing errors for each
of the measurement types.
Distances were generally overestimated. This overestimation was
significant for the real world (M = 0.66 feet, SD = 1.02)(t(26) = 3.40, p
= .002) and transfer measures (M = 0.51 feet, SD = 1.03)(t(26) = 2.57, p =
.02) and approached significance for the virtual measure (M = 0.53 feet,
SD = 1.50) (t(26) = 1.85, p=.08). However, there was no significant
difference in signed distance errors between the three measurement types
(F(2, 52) = 0.18, p = .84). Because there were no significant differences
between virtual, transfer, and real signed errors for both distances and
bearings, the analyses that follow use only unsigned errors.
Relationships between measurement types
In general, correlations between unsigned bearing errors were higher than
those for distance errors. The correlation between real and virtual
bearing errors (r(27) = .63, p < .001) was higher than that between real
and virtual distance errors (r(27) = .46, p = .02), however, this was not
a statistically significant difference. Similarly, the correlation
between transfer task and virtual bearing errors (r(27) = .72, p < .001)
was higher (though not significantly higher) than that between transfer
task and virtual distance errors (r(27) = .43, p = .03). Table 2 presents
the intercorrelations, means, standard deviations, and estimated
reliabilities of the bearing and distance errors for the three measurement
types.
Figure 2. Bearing estimations for men and women relative to the true
bearing in the three phases of the experiment.
The centroid of the
estimations is indicated by the arrow head.
Table 2. Means, standard deviations, reliabilities, and Pearson
intercorrelations between distance and bearing errors for the three
measurement types.
Factors affecting bearing and distance errors
Figure 3 illustrates the relationship between the amount of time
participants spent learning each maze and the degree to which they made
errors in bearing estimations. In general, people who were more
disoriented were those who had spent more time learning the maze.
Jackknifed correlations between the time spent learning the maze and the
error in bearing estimations were greatest for the transfer (r(27) = .515,
p = .01) measure, and were similar for the real (r(27) = .386, p = .05)
and virtual (r(27) = .386, p = .05) measures. Because learning time
exerted a significant effect on bearing errors, we used it as a covariate
in the analyses that follow.
Gender significantly influenced the accuracy with which participants made
bearing judgments. In general, men’s errors in bearing estimation
(Mreal = 16.70°, SD = 6.75; Mtransfer = 24.10°, SD = 7.21; Mvirtual =
17.46°, SD = 11.10) were smaller than those for women (Mreal = 20.23°, SD
= 6.46; Mtransfer = 39.60°, SD = 20.41). Figure 2 shows that the error in
women’s bearing estimations was particularly high in the VE (Mvirtual =
52.15°, SD = 29.31). A 2 (gender [between] ) by 3 (measurement type –
real/transfer/VE [within] ) mixed effects ANCOVA on unsigned bearing
errors (treating the average time spent learning the mazes as a covariate)
revealed a significant interaction between gender and measurement type
(F(2,48) = 6.62, p = .003) and a significant main effect of gender
(F(1,24) = 6.96, p = .01). These effects are illustrated in Figure 4.
Figure 3. Bearing error as a function of time spent learning the maze.
Figure 4. The effects of gender and measurement type on errors in bearing
estimation.
Very different effects were found for errors in distance estimations.
Distance errors were more affected by measurement type (see table 2) than
by gender or the interaction of gender and measurement type. A 2 (gender)
x 3(measurement type : real/transfer/VE) ANCOVA conducted on unsigned
distance errors (and using the mean time spent learning the mazes as a
covariate) revealed no significant main effects or interactions, although
the effect of environment approached significance (F(2,48) = 3.14, p =
.052). These data are illustrated in Figure 5.
Figure 5. The effect of gender and measurement type on distance errors.
A common scale for angles and distances
In order to compare the relative accuracy of distance estimations (which
are measured in feet) and bearing estimations (which are measured in
degrees), we correlated judged interpoint bearings with actual bearings
(and judged with actual interpoint distances) for each participant. After
standardizing these correlations (with Fisher’s r-to-z transformation), we
performed a 2 (gender) x 3 (measurement type : real/transfer/virtual) x 2
(data source : bearings/distances) ANOVA to examine the factors
influencing relative accuracy of bearings and distances. The most
significant effects were those of the data source (Mbearings = 2.10, SD =
0.57; Mdistances = 1.11, SD = 0.48) (F(1,25) = 242.2, p < .001) and
measurement type (Mreal = 1.96, SD = 0.45; Mtransfer = 1.47, SD = 0.63;
Mvirtual = 1.39, SD = 0.65) (F(2,50) = 19.05, p <.001) however, gender and
several interactions were also prominent. Figure 6 illustrates the
effects of gender, data source, and measurement type on relative accuracy
of bearing and distance judgments.
Figure 6. Effects on the correlations between judged and actual distances
and judged and actual bearings.
DISCUSSION
One of the aims of this experiment was to address the degree to which
measuring spatial knowledge in a VE adequately predicts subsequent
performance in the real-world. Our results suggest that a person’s
ability to point to unseen objects in a VE is quite predictive (r » .60)
of their ability to do so in the real world. On the other hand, the
accuracy of distance judgments in a VE does not transfer as well to the
real world (r » .44). The lower correlations between virtual and real
distance judgments are probably due in part to the lower reliability of
errors in distance judgments. Much of the difference between the
predictive capability of bearing and distance errors may also be due to
range effects, and future work should aim at determining the degree to
which these findings generalize to larger environments. It is clear that
in our room-sized maze environment, it was easier for participants to
become severely disoriented than to make grossly inaccurate distance
estimates. The extent to which this trend also occurs in larger spaces is
an open question. Despite these caveats, the finding that errors in
distances and errors in bearings are differentially affected by factors
such as gender and measurement type provides some support for the notion
that bearings and distances are generated from distinct psychological
processes, rather than both deriving from a common conception.
It is important to note that disorientation in virtual mazes was
particularly severe for women. Several (7) women – and no men – had
average bearing errors in excess of 40° for both the virtual and transfer
phases of the experiment. However, the gender difference between real
world errors was much smaller. These results corroborate earlier findings
that understanding the spatial characteristics of VE’s may be more
challenging for women than for men (Waller, Hunt, & Knapp, 1998; Astur,
Ortiz, & Sutherland, 1998). Gender differences in the ability to use and
understand VE’s is an important issue for future research.
ACKNOWLEDGEMENTS
This research was supported by the Office of Naval Research grant
N00014-96-0380, Earl Hunt, Principal Investigator. Correspondence
concerning this article should be addressed to David Waller, Department of
Psychology Box 351525, University of Washington, Seattle, WA 98195.
Electronic mail may be sent to dwaller@u.washington.edu.
REFERENCES
Aguirre, G. K., & Desposito, M. (1997). Environmental knowledge is
subserved by separable dorsal/ventral neural areas. Journal of
Neuroscience, 17(7), 2512-2518.
Astur, R. S., Ortiz, M. L., & Sutherland, R. J. (1998). A characterization
of performance by men and women in a virtual Morris water task: A large
and reliable sex difference. Behavioural Brain Research, 93(1-2),
185-190.
Bliss, J. P., Tidwell, P. D., & Guest, M. A. (1997). The effectiveness of
virtual reality for administering spatial navigation training to
firefighters. Presence: Teleoperators and Virtual Environments, 6, 73-86.
Maguire, E. A., Burgess, N., Donnett, J. G., Frackowiak, R. S. J., Frith,
C. D., & Okeefe, J. (1998). Knowing where and getting there: A human
navigation network. Science, 280(5365), 921-924.
Ruddle, R. A., Payne, S. J., & Jones, D. M. (1997). Navigating buildings
in "desk-top" virtual environments: Experimental investigations using
extended navigational experience. Journal of Experimental Psychology:
Applied, 3, 143-159.
Thorndyke, P. W., & Hayes-Roth, B. (1982). Differences in spatial
knowledge acquired from maps and navigation. Cognitive Psychology, 14,
560-589.
Waller, D., Hunt, E., & Knapp, D. (1998). The transfer of spatial
knowledge in virtual environment training. Presence: Teleoperators and
Virtual environments, 7(2), 129-143.
[ return to top ]