Current Trends in Educational Technology Research:

The Study of Learning Environments.

William Winn,

College of Education,

University of Washington.

September, 2001.

William Winn,

412 Miller, Box 353600,

University of Washington,

Seattle, WA 98195.

Phone (206) 685-1185.

FAX (206) 543- 8439.

billwinn@u.washington.edu

RUNNING HEAD: Educational Technology Research.

Abstract

Educational Technology research has passed through a number of stages, focusing, in turn, on the content to be learned, the format of instructional messages, and the interaction between computers and students. The field is now concerned with the study of learning in complete, complex, interactive learning environments. These allow both the simulation of experiences that students might have in the real world, and also the creation of compelling experiences of phenomena that cannot normally be experienced directly. Learning environments also often allow students to communicate their own ideas using a variety of symbol systems. These environments are also frequently inhabited by more than one person, making learning within them a social activity, and an activity where learning is distributed among both people and artifacts. Finally, these learning environments are complex. Studying how they contribute to learning therefore requires research methods other than controlled experiments.

This article reviews research on learning environments to give both an historical perspective on Educational Technology research as well as a selective view of the current state of the discipline. It concludes by identifying implications for both practice and future research.

Keywords: Educational technology, Learning research, Learning environments, Virtual reality, Computer-based learning.

Introduction

Technology is having more impact on education today than it has ever had. The cost of powerful desktop computers has come down to the point where most schools can afford them. The Internet has pervaded many activities and most places, including schools and colleges. Computers and communication technologies are found in more homes every year; children are growing up with technology. And research into the educational effectiveness of technology is finally showing that, under the right circumstances, it can bring benefits to students of all ages, studying all manner of subjects.

The purpose of this article is to look at a selection of this research in order to get a sense of what work is being done. However, to set today's research in context, I begin with a review of what research in Educational Technology has already accomplished. The rest of the article focuses on current research and the results it is producing. I conclude with some implications for practice and with suggestions for where it would be useful for research to go next.

What has research accomplished so far?

I find it useful to think of the discipline of educational technology, and the work researchers have done within the discipline, as having moved through three "ages", each building on the previous one, and each characterized by fairly unique assumptions and activities. We are now at the start of a fourth age, towards which I direct my attention later in the article.

1. The Age of Instructional Design: A Focus on Content.

The Educational Technology field found its feet when researchers and practitioners discovered that instruction was something that could be planned, designed, evaluated, and revised before it was ever used with students. Moreover, researchers found that instruction designed in this way brought two major benefits. First, it was usually effective at helping students learn. Second, within limits, it enabled students to study on their own, at their own pace, even away from school or on the job. The seminal work of Gagné (Gagné, Briggs & Wager, 1988) is typical of this age. Gagné assumed that different types of learning, such as learning facts or concepts or procedures, required different instructional strategies, which were primarily distinguished by how content was organized and presented to students. Task analysis, whether behavioral or cognitive (Greeno, 1980; Resnick, 1976), became the preeminent tool for determining content organization. The products of task analysis, for example learning hierarchies or sequences, led directly to the specification of instructional objectives, thus prescribing the backbone of instruction. Gagné's "events of instruction" (Gagné et al., 1988) completed the picture. They pointed to which instructional strategies to use, in order for learning to occur most efficiently and most effectively, once content had been organized, the type of learning identified, and objectives written.

The main goal of educational technology at this time was relatively modest, namely to make stand-alone instruction as good as instruction delivered by a teacher. Much of the early instructional computer software attempted to teach what teachers teach and, in many cases, to teach it in the same way. Gagné's events of instruction, after all, describe what a good teacher should always do. So the standard for comparison in much of the research on computer-assisted instruction, examined, for example, in Kulik's (1983, 1985) meta-analyses from the 'eighties, was teacher-led instruction. If studies showed that technology-based instruction had done as well as a good teacher, we concluded that technology had done its job.

2. The Age of Message Design: A Focus on Format

If instructional design is one of the disciplines on which Educational Technology is built, a second, of equal importance, is mediated instruction. From its very beginning, Educational Technology has been concerned with educational applications of communications media. Early research on messages, such as Levie's (1978) work on persuasion and attitude change, revolved around the effects of the mass media, particularly television. However, two circumstances changed all that. The first, purely technical, circumstance was the development of computer hardware that could show graphics and produce sounds, thus increasing the control designers and students had over the material. The second was the body of research that showed that students with different skills and abilities learned differently from different instructional treatments. Cronbach and Snow (1977) summarized much of this research. Although Cronbach and Snow were somewhat pessimistic about the quality and usefulness of research on the interaction of aptitude and treatment ("ATI"), where the treatment variable was presentation format, educational technology researchers continued (and continue to this day) to study how the format in which content is presented to students interacts with student characteristics to produce learning of varying quality and permanence.

Message design continues to be an important part of the work educational technologists do, and, in some form or other, remains a resilient piece of the Educational Technology curriculum. One reason is the continuing importance, in research and development, of determining how to cater to individual differences among students, whether in aptitude or on some other trait. Another reason is the flexible evolution of what is meant by the term, "message design". A comparison of the first and second editions of Fleming and Levie's important book (Fleming & Levie, 1977, 1993) shows that, not only has our ability to deliver instruction in different formats changed dramatically, but also that the underpinning theory of message design has kept up with (is indeed well grounded in) developments in more basic research on learning and instruction.

3. The Age of Simulation: A Focus on Interaction

I have already hinted, in the preceding comments on instructional design and message design, that technology-based instruction can bring to students information in formats that teachers can not. Yet Clark (1983) stated that instructional methods make the difference in how well students learn, not the format of the message or the delivery technology. This is indeed true. For example, well-designed instructional strategies, intended to bring about mastery, can result in significant improvements in student performance without using media or technology (Bloom, 1984; Lysakowski & Walberg, 1982). However, as Kozma (1991) implied in his reply to Clark, significant advances in media technology, which occurred between the appearance of Clark's article and Kozma's own, provided affordances that could support instructional strategies that would not be possible without the technology. A number of these have to do with what can now be done with graphics (Winn, 1994) and sound (Hereford & Winn, 1994). But the most important is the ability of students to take control of the material, from which they are learning, in simulations.

The ability of computers to simulate phenomena, whether natural or imagined, has two advantages. The first is the full implementation of learner control. Studies from the 'eighties (Carrier, Williams & Davidson, 1985; Dwyer, 1985) showed that allowing students a measure of control, with guidance (Johansen & Tennyson, 1983), over how they studied could bring about significant learning gains. The second advantage of simulation is the ability of students to do things that they could never do in the real world. In the sciences, this could mean conducting experiments that would be impossible otherwise, such as changing the Earth's climate to see what would happen if greenhouse gases were reduced (Jackson, Taylor & Winn, 1999). In social studies, this could mean making decisions about what to do in order to survive the trek west in 19^th century America (Minnesota Educational Computing Consortium, 1995).

The critical feature of simulations, for learning, is the student's ability to act on the environment experimentally, not simply to observe it. The key theoretical assumption of learning from simulations is that students construct understanding for themselves by interacting with information and materials, an orientation to learning that has acquired the name "Constructivism" (Duffy & Jonassen, 1992). Applied with care, and with special attention to "scaffolding" students' learning as they interact with simulations (Linn, 1995), constructivist approaches to teaching and learning can be motivating and effective (see relevant sections of Brown, Bransford & Cocking, [2000]).

The "New Age" of Research in Educational Technology:

A Focus on Learning Environments

The ideas of carefully designing instruction, varying the formats in which information is presented to students, and building interactive simulations lead naturally to the idea of constructing entire "learning environments", in which the student has unprecedented freedom to act. Increasingly today, researchers in Educational Technology study students working in complete learning environments. Learning environments can either be entirely natural, or they can be artificial, existing only through the agency of technology.

Of course, learning always takes place in an environment. In the 18^th century, Rousseau (1762/1933) argued that the most effective and socially appropriate education arose from a student's interaction with Nature -- the natural environment. More recently, educators have proposed that a modified natural environment can serve to meet more specific learning objectives, through cognitive (Brown, Collins & Duguid, 1989) or professional and vocational (Lave & Wenger, 1991) apprenticeships. Apprentices work in a natural environment that has either been modified for pedagogical purposes, or within which the apprentice's freedom to act is limited, so that, in accordance with good instructional design, apprentices' learning can be guided to bring about knowledge construction. Schools are also learning environments. However, they are recent on the scale of human history and quite artificial (Winn & Windschitl, in press). More often than not, all that students learn in them is how to be students (Brown, 1997).

Many technology-supported learning environments simulate some aspect of the natural environment. This allows learning to be "authentic", engaging students in projects that have some meaningful connection with problems that exist in the real, non-simulated world. Because most authentic real-world activities involve more than one person, it follows that most technology-supported learning environments include people in addition to the student, confirming that learning occurs socially. Current theories of learning and instruction acknowledge the social nature of learning (Vygotsky, 1978). Many current educational research projects study social interaction as a promoter of learning, often through the agency of learning communities created with the Internet (Gordin, Edelson & Pea, 1996; Malarney, 2000).

In this section, I examine current research on learning in technology-supported learning environments and related phenomena. Unavoidably, my account is selective. However, the topics and research projects described below capture critical and typical characteristics of current research in educational technology.

Research on artificial learning environments

Why would anyone want to learn in an artificial environment when there is plenty to learn in the natural world? There are two reasons. First, it may not be possible to learn in the natural environment because it is too dangerous. Flight simulators, other industrial simulators, even surgery simulators, allow students to practice procedures and gain understanding without risk to themselves or to other people. Second, the concepts and principles that govern the behavior of the natural world are often hidden and counter-intuitive. A child who watches a ball roll across the floor and slow down might conclude that the ball runs out of energy, not that the force of friction acts on it. A demonstration of Newton's laws of motion, that did not induce this misconception that arises from observation alone, would have to take place in a frictionless environment (perhaps in outer space) or in a virtual learning environment where friction could be turned off. The only practical strategy in this case is to build an artificial environment within which a simulation can be run of phenomena that are impossible to observe directly in the real world. Dede and his colleagues (Dede, et al., 1997) have created such a virtual world to teach Newton's laws.

This leads me to make a distinction between simulation and reification (Winn, 1993a). The purpose of a simulation is to create as accurate a facsimile of real objects or events as possible. Yet, as the example with the rolling ball suggested, sometimes fidelity to the real world can result in misconceptions. What is more, high realism in simulations and virtual environments can militate against transfer (Caird, 1996) and generalization (Jackson et al., 1999; Osberg et al., 1997). This is because the more precise and constrained the representations and modes of interaction with simulations are, the more precise and inflexible a student's performance becomes, whether motor or cognitive. Merrill (1992) made the same point in his case for direct instruction of generalities and against situating all learning in well-defined contexts.

On the other hand, reification is the process whereby phenomena that cannot be directly perceived and experienced in the real world are given the qualities of concrete objects that can be perceived and interacted with in a virtual learning environment. This is necessary when the real phenomena or objects are too small to see, like atoms (Byrne, 1996), or too large to work with manageably, like the solar system (Barab et al., 2000). At other times, the phenomena have no physical form. Typical are dynamic natural processes, like evaporation or Nitrogen fixation (Osberg, 1998). At yet other times, the phenomena are simply inaccessible, like events that take place beneath the surface of the ocean (Windschitl & Winn, 2000). Reification therefore allows students to experience in computer-created virtual learning environments what they cannot experience in the real world, which is the most important contribution they make to learning.

Reification relies to a large extent on metaphor. If something to learn about has no perceptible form, the designer has to create a metaphor for it that can be rendered, visually or audibly, by the computer. A strength of metaphors in this context arises from the way computers build virtual learning environments. Everything that the computer creates is built from data. Real objects, like chairs and kangaroos, are modeled by designers and stored in a database, from which the computer draws when the time comes for the virtual form of the object to appear. Similarly, reified objects, like the sphere that represents a neutron (Byrne, 1996) or the arrows that represent the speed and direction of ocean currents (Winn et al., 2001), are drawn by the designer and stored in a database, and recalled when they are needed. The computer cannot distinguish between virtual objects that stand for real objects, or those that stand for reified abstractions. This means that both real and reified objects have equal status in virtual environments, allowing students to view and interact with reifications in exactly the same way that they do with representations of real objects.

Recent research has shown that, overall, visiting artificial learning environments that have the characteristics I have just described, helps students understand concepts and processes that the environments represent. This is true for astronomy (Barab et al., 2000), meteorology (Hay, 1999), physical oceanography (Winn & Windschitl, 2001), maintenance of nuclear reactors (Kashiwa et al., 1995), sub-atomic chemistry (Byrne, 1996), global warming (Jackson, 2000), and other content areas. However, reifying phenomena with metaphors is not without danger. Here are two examples from our own work. In Jackson's simulation of global warming (Jackson, 2000), students can vary the amount of greenhouse gases coming from vehicles and from factories. They can also manipulate the amount of green plant matter available to absorb carbon dioxide. Jackson's metaphor for this second manipulation is adding or removing trees from the environment. (His motivation for choosing this metaphor was to connect global warming to the destruction of the rain forests.) As students add trees to the environment, global warming becomes less of a problem. As they remove trees, it gets worse. Several upper elementary and middle school students concluded that global warming is not a problem after all: If it gets bad, all we have to do is plant more trees! The tree metaphor oversimplified the complex interactions that affect climate change and induced a misconception.

The second example comes from our simulation of physical oceanography (Windschitl & Winn, 2000). Current speed and direction are reified using arrows whose length represents current speed and whose orientation shows current direction. Because the arrows are longest when the current is fastest, narrow passages look "clogged" when the current through them is fast. This has led some students to conclude that water slows down when it moves through narrow passages, instead of speeding up in accordance with principles of fluid dynamics. We are now working with a particle advection metaphor to get around this problem.

Research on inscriptional systems.

The technologies that support learning environments continue the message design tradition by presenting information in a variety of formats. Learning environments use a variety of realistic and metaphorical representations of data in graphic, auditory and, increasingly, haptic modalities. The advantages of seeing, hearing, and feeling representations that vary in pedagogically appropriate ways has been frequently documented in the research. However, representations are used in new ways today by researchers and students, in addition to providing information and affording interaction.

The phrase "inscription" was suggested by Pea (1994), as an alternative to "representation", to refer to external representations, rather than internal representations such as images and mental models. Inscriptions are created by students, as well as by scientists and learning environment designers, to externalize understanding and to serve as points of reference during discussions (Gordin, Edelson & Pea, 1996). Inscriptions can be made by students using paper and pencil or complex animated 3D authoring tools.

The CoVis project (Gordin, Edelson & Pea, 1996) is one of the best known projects that uses inscriptions both to show information to students and to allow students to express and discuss their own ideas on a topic. There are two basic principles behind this project. The first is that the datasets and models used by scientists to represent and study the world have enormous potential for helping students understand natural phenomena by using the same technical and intellectual tools that scientists use. This allows students to do "authentic" science, in this case meteorology. The second principle is that by providing the tools to let students create their own visualizations, students are required, first, to master the content before committing it to being displayed, and then to use it as a point of reference during discussions with other students, teachers and scientists. It is in the course of such conversations that a lot of learning takes place.

Of course, the actual tools scientists use, and the symbol systems they use to represent their data, are sometimes difficult (and unnecessary) for students to master. For this reason, the CoVis team has developed a tool for students to use, "Weather Watcher", as they go through problem-solving exercises, to help them understand global warming, for example. This tool provides a simpler interface than meteorologists use, a less technical symbol system, and a number of ways for students to complete their tasks. The students are connected with an entire community of learners over the Internet that includes professional meteorologists, who advise the students, when invited, on their projects. Data from the project demonstrate the effectiveness of Weather Watcher for learning.

Other research that combines the use of inscriptions for studying natural phenomena, as well as the creation of inscriptions by students as part of the learning process, involves work with virtual environments. Barab and his colleagues (Barab et al., 2000) have studied astronomy students' construction of virtual worlds as a means to building an understanding of aspects of the Solar System. Using commercially-available software for creating and showing virtual worlds, undergraduates have built computer-based animated visualizations that illustrate, for example, how eclipses occur. Data from Barab's research show that the actions of designing and building their own materials, the discussion involved in those actions, and the discussion when other students use these materials to study eclipses, all contribute to helping students understand the often counter-intuitive mechanisms that cause eclipses.

Our own work has arrived at similar conclusions (Osberg, 1997; Winn et al., 1999). Our study of students building their own inscriptions -- virtual worlds -- has been less formal than Barab et al's research. Nonetheless, we have found that the act of designing and creating environments that embody concepts and principles governing phenomena as diverse as wetlands ecology and medieval castles helps students to master these topics with depth and clarity (Winn et al., 1999). What is more, having students build their own virtual worlds works best with students who tend not to do well in school. One problem has been that world-building, for younger students who have not yet learned to reason abstractly, tends to limit their ability to transfer what they have learned to other domains. They think of the content almost exclusively in terms of how they chose to represent it in their virtual world. However, knowing this, we can now use additional strategies deliberately aimed at helping them achieve transfer.

Research on social aspects of learning

A characteristic of most learning environments is that they contain other people. This means that learning environments created or supported by technology, like other learning environments, are places where learning takes place, in part, through social interaction. Researchers in Educational Technology have long acknowledged the importance of the social aspect of learning, being influenced in large part by the writing of Vygotsky (1978).

One advantage for researchers of looking at the social interactions that take place as students learn is that the conversations among students are themselves useful sources of data. Thus, we find that discourse analysis can shed light on the processes and products of learning (Herrenkohl & Guerra, 1998; Herrenkohl et al., 1999). A good example of this is the Knowledge Integration Environment (KIE) project (Linn, Bell & Hsi, 1998). KIE is a web-based environment that allows students post evidence and arguments for competing scientific theories. It consists of a number of tools that guide students to information that can be construed as evidence, assembled and made public. Used with appropriate classroom activities, these tools can generate discourse that not only involves students in serious debate about the content and nature of science, but also reveals a lot about how students' concepts and beliefs about science change. Bell (in press) shows how this approach can be used effectively. His study had students find evidence for and against two explanations about what happens to light -- it goes on for ever versus it dies out. Working in pairs, the students assembled their evidence for and against each theory, using the tools. They then participated in a class debate with their peers during which they shared their evidence and their accounts of what happens to light. Interestingly, what some students considered to be evidence for one theory was given as evidence against the same theory by other students. The conversations that occurred during the debate about why the students used their evidence the ways they did was a good way for students to learn and for the research group to understand what the students were thinking.

Bell has built on this work in the SCOPE ("Science controversies online partnerships in education") project (Bell & Slotta, 2001). Here, students, teachers and scientists can share information at a web site about issues on which even scientists do not agree and which are also visibly in the forum of public debate. So far, SCOPE has dealt with the debate about why amphibians are disappearing from their natural habitats, whether DDT should be used to combat malaria, and whether genetically-modified foods are harmful to people and the environment. Again, by engaging groups of people with very different backgrounds and points of view in a learning community, the public debate becomes both a way for students to learn and also a source of data for researchers.

A final comment: I believe there is a danger if researchers focus on discourse at the expense of other data sources. Guddemi (2000) made the point that, although it is useful to study how people talk about things, it is also important to continue to look at what they learn, using methods other than discourse analysis. This comment is not a direct criticism of any of the work I have just described. In all cases, this research does not fall into this trap. However, the potential for problems remains.

Research on distributed cognition in learning communities

Once we acknowledge the social nature of learning, and understand the ways in which technology can support interaction among students, teachers, and experts, we can go the additional step and ask whether cognition, generally, is distributed over entire communities linked by technology. (Chapters in Salomon's [1995] edited volume present various points of view about this issue.) Hutchins (1995) suggests that it might be. The opening vignette in his book on distributed cognition is a detailed account of how the crew of a Navy ship pilots it safely into San Diego harbor. The account contains many examples of people working with people, people working with devices (gyro-compass repeaters, radar), even devices working with devices, to get the job done. Perhaps the most important point, though, is that at any time no one person or device is in possession of all of the information necessary to pilot the ship. The knowledge required to get the job done is in the truest sense distributed among a community of people and devices.

The seeds of the idea of technology helping to distribute cognition, sharing problem solving among people and devices, goes back more than twenty years. Pask (1975) suggested that learning to solve problems was like a conversation, whose goal was to "arrive at an agreement over an understanding". Pask noted that the participants in the conversation could be people, people and machines, or machines and other machines. For Pask, the necessary condition for cognition was the organization of concepts within a coherent system, not the possession of a human brain. Today, we take everything from pedagogical agents, computer-based learning programs, and using calculators to solve math problems pretty much for granted.

The study of distributed learning mostly involves research on three things, each of which is sometimes considered to be a research domain in its own right. These are: Learning communities comprising people with varying backgrounds and levels of expertise; a technology that supports communication and productive activity within the community (almost always the world wide web); and engagement in authentic activity. This last feature is by no means required for distributed learning to occur. However, in most projects, one group of participants in the learning community consists of experts in the domain, one of whose roles is to connect the learning experience to the world of practice.

Malarney's (2000) research serves as a useful case study of distributed learning. Her project created a learning community that consisted of teachers and students in a grade ten classroom and the crew and scientists on board a NOAA ship working in the tropical Pacific. "Classroom at Sea" had as its objective to help students understand ocean science by collaborating, albeit vicariously, with the people who do it. The project used the internet for email, and provided a web site for collecting and displaying material about ocean science and students' research (http://classroomatsea.noaa.gov). Audio and video communications, carried live by satellite, supported two-way interaction between the students and the ship at sea.

The project was a mixed success. On the positive side, the crew helped students carry out an experiment. Each student wrote a question on a styrofoam cup. The cups were delivered to the ship prior to sailing. Once on station, the scientists put the cups in a bag and attached them to a sampling device that they lowered into the ocean. The students had been asked to predict what would happen to the cups. (They are compressed to about half their size by the pressure of the ocean at depth, but retain their shape perfectly.) During the first live broadcast, the captain showed the students what had happened to their cups, and also answered the questions written on them, still perfectly legible, though smaller. This activity tied the ship and the classroom together. The experiment with the cups could only have been done by distributing the activity between the students and the shipboard scientists.

In spite of this and other successes, there were problems that prevented the complete cohesion of the classroom and the ship. Inevitably, both the crew and the students were busy with other work which distracted them from the project. More interesting was a problem arising from a lack of understanding on the part of the scientists about how to communicate with non-specialists. For example, one email question from a student about the buoys the ship uses to gather data elicited a very lengthy and technical response from the ship. The student was completely turned off and never asked a question again.

Malarney's study shows that technology alone is not sufficient to create a successful learning community. At least two more components must be present. The first is a carefully structured set of protocols and activities that set out what will happen and how communication about these events should take place. The second component is not necessary but desirable. This is at least one face-to-face meeting among the members of the learning community. This did occur in a pilot project, where the students were able to spend a day at sea on the ship. More generally, educators make a lot of unfounded assumptions about how the web can support learning without realizing that it was not designed to teach anything. It only provides access to information. Other features of learning communities, where the responsibility for helping students is widely distributed, need to be developed if learning is to occur.

Some of the work completed by Malarney's students was authentic science. They collaborated with real scientists at sea as they developed their projects. Such an approach has a number of advantages. First, it is motivating. Students see that what they are learning is useful. Second, it does not seriously oversimplify the world with which students interact in order to make it easier to learn. (If the truth be known, content is often simplified to make it easier to teach, rather than for the benefit of the students.) Spiro (1992) has decried this practice, dubbing it "reductive bias", and arguing that oversimplification of processes that are naturally complex and difficult to describe leads to misconceptions. Third, sometimes the work students do as they learn science can make a valuable contribution to a real scientific research project. This, too, enhances motivation and authenticity. For example, Teasley et al. (2000) describe a project that has equipped many schools in Nebraska with cosmic ray detectors so that students may contribute to a national study of cosmic ray activity. By distributing the instruments over a wide geographical area, more accurate data can be gathered. And the only way the entire network of instruments can be operated is by having the students do the data-gathering.

Research on complete systems of variables in "real-world" contexts

One of the problems that the increasing emphasis on learning environments brings is the complexity of the interactions that occur while the students are learning. Traditional methods for observing, recording, and analyzing learning are inadequate for dealing with learning that results from interactions of several learners with complex learning environments. However, researchers should regard dealing with this not as a problem but as a necessity. Salomon (1991) has made the case that educational researchers should indeed study systems of variables, both in the student and in the environment, that bring about learning, not one or two variables in isolation. Unfortunately, some researchers have taken this to mean that learning can no longer be studied quantitatively, or even with the rigor required when drawing conclusions from evidence. However, conceptual and methodological frameworks exist that allow researchers to bring rigor to their work with students learning in complete environments.

Much of the research I have cited was conducted through "design experiments" (Brown, 1992). Unlike traditional experiments, where a technology or strategy is used with groups of students, who are assessed in some way at the end, design experiments are iterative. A learning tool is built in the research laboratory. The research team then takes it into the field where it is used with students. The data that come from the study of what and how students learn using the tool serve two purposes. They serve, obviously, to guide revisions to the tool itself. The data also serve to help researchers understand the learning process and how it is affected by the tool. In effect, research and development occur simultaneously. This stands in contrast to traditional instructional design and development, where the design and implementation phases are kept separate on the assumption that the design procedures are sufficiently reliable to allow the development of effective instruction before that instruction is used in the classroom (Winn, 1993b).

It is clear that design experiments address the perpetual problem in educational research of connecting research and theory to practice. Research on the same question takes place both in the laboratory and in the field, with the outcomes of one informing the activities of the other in a reciprocal manner. Design experiments also address the issue of the complexity of learning in technology-based learning environments. In a traditional experiment, designed to test the effectiveness of a learning tool, as many variables as possible that are not of interest to the research are controlled. This gives a distorted view of what might happen when the tool or strategy is used in a setting, like a classroom, where these variables cannot be controlled. The design experiment can get around this by examining, eventually, all factors expected to affect learning. Data gathering is not the end of the matter, rather the beginning of the next go-round, during which other factors can be observed, measured, and evaluated. Experimental control is less, if at all, necessary in the case of design experiments. Over time, the impact of a whole host of factors on learning can be assessed.

Implications

Implications for practice

In the past, Educational Technology research has often been disconnected from practice. On the one hand, the research has been conducted in laboratories, isolated from the many factors that support or impede the practical implementation of research findings. On the other hand, it has been hard for practitioners to find and use information, materials and programs of activities that the research has created. Fortunately, these two impediments to putting research findings and products into the hands of practitioners are not as powerful as they once were.

There are two reasons for this. The first is that much of the current research on learning environments is done in schools. The iterative nature of design experiments, as we have seen, requires the frequent assessment of learning tools in real settings, with real students and teachers, while the research that leads to their development is underway. A significant feature of this approach is that the users of the tool are involved in its development from the very start and are partners in the research on its effectiveness. Second, the materials that researchers have designed, studied and found to be effective are, in many cases, available to practitioners. Following the references to projects cited in this article, or searching for the projects on the Web, the reader will more often than not arrive at a web site that has useable materials to download or offers free membership in a learning community. For example, the CoVIS project, described earlier, has become part of the Center for Learning Technologies in Urban Schools (LeTUS). Their website, http://www.letus.org, contains an open invitation for educators to join the project, a large amount of download-able curriculum material and software that includes "World Watcher", the newest version of the "Weather Watcher" software, described above, for studying global warming. The SCOPE site http://scope.educ.washington.edu likewise contains an invitation to join the project and provides curriculum materials related to scientific controversies.

That said, a lot of the research mentioned in this article has not yet produced useable materials and strategies. One reason for this is that the more complex learning environments still require more powerful equipment to run them than is commonly available in schools. This is certainly true, for now, of most of the material produced in our laboratory (Jackson et al., 1999; Osberg et al., 1997; Winn et al., 2001), as well as some of Barab's (Barab et al., 2000) material, although the latter has been used effectively in university courses in Astronomy at Indiana University. Dede's materials (Dede et al., 1997) were also originally confined to the laboratory. However, with adaptation to less expensive hardware, some of his materials are currently being used in schools in Virginia. The point for the practitioner is that even the more esoteric hardware and software are being used in classes, not just laboratories. As the technology that is available to schools becomes more powerful, without increases in cost, any learning environment will be useable in a school.

The findings of the research I have described also have important things to say to practitioners. Here are a few practical suggestions:

1. Learning environments that reify abstractions use metaphors to communicate ideas. Even the best-intentioned designers cannot avoid a measure of idiosyncrasy when they select metaphors. Teachers must take care to anticipate metaphors that are likely to be difficult for their students to understand, and be prepared to teach what they mean.

2. Computer-supported learning environments that are built around simulations work best with a problem-solving or constructivist approach to learning. Teachers must allow students to experiment in them and above all to learn from making mistakes. It would not be a good idea to use a virtual world to learn basic facts.

3. Technology may sometimes be a necessary condition for the creation of learning communities, but it is never a sufficient condition. Simply creating a web site and signing people up to use it will not create a learning community. The community needs a purpose, activities to pursue off-line, and possibly even other modes of communication, like face-to-face meetings, in order to be successful.

4. Likewise, simply creating an interactive learning environment is not sufficient to bring about learning. Students using any kind of simulation, whether a self-contained learning environment or one that is a part of some broader activity, need to understand clearly what they are supposed to accomplish. They will require careful, though not intrusive, scaffolding to help them achieve their goal.

5. By default, technology tends to isolate students from one another. This means that practitioners must be deliberate in their attempts to create a social context for learning in a technology-based learning environment. What is more, teachers need to give students credit for collaboration and put to rest the traditional view that sharing work is cheating.

6. Effective learning communities often include experts from outside education. Involving experts from the local community, or from a wider community via the Internet, is a good strategy to pursue. Provided time commitments are not too onerous, practitioners will often be pleasantly surprised by the willingness of experts to work with students.

7. Students should be encouraged, when appropriate, to create or modify the learning environments they work in. Inscription is a successful learning strategy that not only helps students by externalizing and sharing their thoughts, but provides a source of information to teachers about what students understand.

8. Practitioners and students make significant contributions to the research that leads to innovative software, hardware and learning activities. Partnerships between students, teachers and researchers must be encouraged.

Implications for research

Although the research I have described is selective, I believe the categories into which I have placed it are useful. Instructional developers, in both the public and private sectors, are already using technology to create more complete and more complex learning environments. Educational researchers need to study which characteristics of these environments help or hinder learning. The environments themselves present information in many ways and often allow students to add their own inscriptions. Researchers need to study how sharing information in these ways encourages useful discourse about the environment that, in turn, supports learning. Now that educators have acknowledged the social nature of learning, researchers need to study how technology can be used to support interaction among students and teachers, and how the responsibility for learning can be fairly and meaningfully distributed within learning communities. Finally, research methodologies need to adjust to the demands of studying increasingly more complex interactions among students and their environments. Design experiments are a good start.

Looking ahead, I am tempted just to say that researchers should do more of the same. However, there are activities on, or just over the horizon that do not yet qualify as trends in educational technology research, but which nonetheless require attention. First is the increasing viability of studying learning from the perspective of the neurosciences. Educational Technology researchers have been content to build conceptual frameworks within which to conduct research from the metaphors of psychology rather than from the mechanisms of neurobiology. However, neuroscientists may be close to understanding learning in terms of neural activity (see sections of Pinker, 1997; Dennett, 1995; also Bruer, 1999). Should such an approach to the study of learning prove viable and useful, it will provide a fundamentally different view of how students learn. Only time will tell where this view will lead research on learning and educational practice.

Another thing to watch for is the fundamental transformation, or even possible demise, of schooling as we know it. Such a concern is based on current forces that are both pushing and pulling schools to change. On the "push" side is the concern that our schools are not doing as well as they could and must be reformed. On the "pull" side is the demonstrated effectiveness of other ways to educate people, ranging from home schooling to on-line classes and programs. In the early days of Educational Technology, people (for example, Heinich, 1970) predicted that technology would radically alter what happens in schools, would, in fact, replace a lot of what teachers do in their classrooms. These predictions have not yet turned out to be correct, probably because educational technologists have an exaggerated expectation of the power of technology "pull". However, technology will inevitably play a role in the reform of public education, however radical that turns out to be. Educational Technology researchers therefore have as much responsibility in studying the means and ends of school reform as anyone else.

Finally, there is a trend in the training community towards supplanting traditional training with just-in-time training, or of replacing training altogether with personal support systems. As our technologies become more able to bring information, learning materials, even learning environments to wherever people happen to be, the argument can be made that we no longer need to remember what we need to know; we can simply call it up and display it when it is needed. Whether this trend will spill over into the world of Education to any great extent is unclear. If it does, then the impact on traditional curricula will be tremendous. Teachers, parents and politicians will expect students to master skills in information science and interpretation, maybe even at the expense of science or math or social studies. As a result, research will switch its current focus on learning with technology to a new focus on how students interpret and use what technology presents to them in real time.

These last points may seem a bit "far out". However, it is only by anticipating change that we can prepare to deal with what it brings. Technology is now firmly entrenched in all manner of educational organizations and supports a wide range of learning activities. And technology is extremely volatile and unpredictable. Trying to keep up with how it changes and how it will change education is difficult. But it is necessary for those of us who study how technology can be used effectively to try.

References

Barab, S.A., Hay, K.E., Squire, K., Barnett, M., Schmidt, R., Karrigan, K., Yamagata-Lynch, L., & Johnson, C. (2000). The virtual solar system: Learning through a technology-rich, inquiry-based, participatory learning environment. Journal of Science Education and Technology, 9(1), 7-25.

Bell, P. (in press). Using argument maps to make thinking visible for individuals and groups. To appear in N. Miyake, R. Hall, & T. Koschmann (Eds.), Computer supported collaborative learning. Mawah, NJ: Erlbaum.

Bell, P., & Slotta, J.J. (2001, February). Technology as controversy teaching partner: Genetically modified foods. Paper presented at the Annual Meeting of the American Association for the Advancement of Science, San Francisco, CA.

Bloom, B. (1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13 (6), 4-16.

Brown, A. L. (1992). Design Experiments: Theoretical and Methodological Challenges in Creating Complex Interventions in Classroom Settings. The Journal of the Learning Sciences, 2(2), 141-178.

Brown, A.L. (1997). Transforming schools into communities of thinking and learning about serious matters. American Psychologist, 52(4), 399-413.

Brown, A., Bransford, J., & Cocking, R. (1999). How people learn: Brain, mind experience and school. Washington DC: National Academy Press.

Brown, J.S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32-43.

Bruer, J.T. (1999). Neural connections: Some you use, some you lose. Phi Delta Kappan, 81(4), 264-277.

Byrne, C. (1996). Water on tap: The use of virtual reality as an educational tool. Ph.D. Dissertation, Department of Industrial Engineering, University of Washington, Seattle, WA.

Caird, J.K. (1996). Persistent issues in the application of virtual environment systems to training. IEEE Computer Society: Human Interaction with Complex Systems, 3, 124-132.

Carrier, C., Williams, M., & Davidson, G. (1985). Selection of instructional options in a computer-based coordinate concept lesson. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL.

Clark, R.E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53, 445-460.

Cronbach, L., & Snow, R. (1977). Aptitude and instructional methods. NY: Irvington.

Dede, C., Salzman, M., Loftin, R.B., & Ash, K. (1997). Using virtual reality technology to convey abstract concepts. In M. Jacobson & R. Kozma (Eds.), Learning sciences for the 21^st century: Research, design and implementing advanced technology learning environments. Mawah, NJ: Erlbaum.

Dennett, D. (1995). Darwin's dangerous idea. New York: Simon and Schuster.

Duffy, T.M., & Jonassen, D.H. (1992). Constructivism: New implications for educational technology. In T. Duffy & D. Jonassen (Eds.), Constructivism and the technology of instruction: A conversation. Hillsdale, NJ: Erlbaum.

Dwyer, F.M. (1985, January). Varied Self-Paced Micro-Computer Based Instructional Programs for Addressing Individual Differences when Acquiring Different Levels of Instructional Objectives. Presented at the annual convention of the Association for Educational Communication and Technology, Anaheim, CA.

Fleming, M.L., & Levie, W.H. (1978). Instructional message design. Principles from the behavioral sciences. Englewood Cliffs, NJ: Educational Technology Publications.

Fleming, M.L., & Levie, W.H. (1993) Instructional message design: Principles from the behavioral and cognitive sciences. Englewood Cliffs, NJ: Educational Technology Publications.

Gagné, R.M., Briggs, L., & Wager, W. (1988). Principles of Instructional Design. New York: Holt, Rinehart & Winston.

Gordin, D.N., Edelson, D.C., & Pea, R. (1996, April) Supporting students' science inquiry through scientific visualization activities. Paper presented at the annual meeting of the American Educational Research Association, New York.

Greeno, J. (1980). Some examples of cognitive task analysis with instructional implications. In R.E. Snow, P-A Federico, & W.E. Montague (Eds.), Aptitude, learning and instruction, Volume 2. Hillsdale, NJ: Erlbaum.

Guddemi, P. (2000). Autopoeisis, semeiosis and co-coupling: A relational language for describing communication and adaptation. Cybernetics and Human Knowing, 7, 127-145.

Hay, K. (1999, May). The digital weather station: A study of learning with 5-D visualization. Presented at the meeting of the American Educational Research Association, Montreal, Canada.

Heinich, R. (1970). Technology and the management of instruction. Washington, DC: Association for Educational Communication and Technology.

Hereford, J., & Winn, W.D. (1994). Non-speech sound in the human-computer interaction: A review and design guidelines. Journal of Educational Computing Research, 11, 209-231.

Herrenkohl, L., & Guerra, M. (1998). Participant structures, scientific discourse, and student engagement in fourth grade. Cognition and Instruction, 16, 431-473.

Herrenkohl, L., Palincsar, A., DeWater, L., & Kawasaki, K. (1999). Developing scientific communities in classrooms: A socio-cognitive approach. Journal of the Learning Sciences, 8, 451-493.

Hutchins, E. (1995). Cognition in the wild. Cambridge, MA: MIT Press.

Jackson, R. (2000). Collaboration and learning within tele-immersive virtual environments. PhD. Dissertation, College of Education, University of Washington, Seattle.

Jackson, R., Taylor, W., & Winn, W.D. (1999, April). Peer collaboration in virtual environments: An investigation of multi-participant virtual reality applied in primary science education. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada.

Johansen, K., & Tennyson, R. (1983). Effect of adaptive advisement on perception in learner-controlled, computer-based instruction using a rule-learning task. Educational Communication and Technology Journal, 31, 226-236.

Kashiwa, K.I., Mitani, T., Tezuka, T., & Yoshikawa, H. (1995). Development of machine maintenance training system in virtual environment. Proceedings of the 4th IEEE International Workshop on Robot and Human Communication, RO-MAN '95. Tokyo, Japan. New York: IEEE., 295-300.

Kozma, R. (1991). Learning with media. Review of Educational Research, 61, 179-211.

Kulik, J.A., (1983). Effects of computer-based teaching on secondary school students. Journal of Educational Psychology; 75 (1), 19-26.

Kulik, J.A., (1985). Effectiveness of computer-based education in elementary schools. Computers in Human Behavior; 1 (1), 59-74.

Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: Cambridge University Press.

Levie, W.H. (1978). Principles for attitude change. In M.L. Fleming & W.H. Levie, (Eds.), Instructional message design. Principles from the behavioral sciences. Englewood Cliffs, NJ: Educational Technology Publications.

Linn, M. (1995). Designing computer learning environments for engineering and computer science: The scaffolded knowledge integration framework. Journal of Science Education and Technology, 4(2), 103-126.

Linn, M., Bell, P., and Hsi, S. (1998). Using the Internet to enhance student understanding of science: The knowledge Integration Environment. Interactive Learning Environments, 6, 4-38.

Lysakowski, R.S., & Walberg, H.J. (1982). Instructional effects of cues, participation and corrective feedback: A quantitative synthesis. American Educational Research Journal, 19, 559-578.

Malarney, M. (2000). Learning communities and on-line technologies : the classroom@sea experience. Ph.D. Dissertation, College of Education, University of Washington.

Merrill, M.D. (1992). Constructivism and instructional design. In T. Duffy & D. Jonassen (Eds.), Constructivism and the technology of instruction: A conversation. Hillsdale, NJ: Erlbaum.

Minnesota Educational Computing Consortium (1995). Oregon Trail II. (CD ROM). Minneapolis: Minnesota Educational Computing Consortium.

Osberg, K.M. (1997). Constructivism in Practice: The Case for Meaning-Making in the Virtual World. PhD. Dissertation, College of Education, University of Washington, Seattle, WA.

Osberg, K. M., Winn, W., Rose, H., Hollander, A., & Hoffman, H. (1997, May). The effect of having grade seven students construct virtual environments on their comprehension of science. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, IL.

Pask, G. (1975). Conversation, cognition and learning. Amsterdam: Elsevier.

Pea, R. D. (1994). Seeing what we build together: Distributed multimedia learning environments for transformative communications. Journal of the Learning Sciences, 3(3), 285-299.

Pinker, S. (1997). How the mind works. New York: Norton.

Resnick, L. (1976). Task analysis in instructional design: Some cases from Mathematics. In D. Klahr (Ed.), Cognition and instruction. Hillsdale, NJ: Erlbaum.

Rousseau, J-J. (1762/1933). Emile. New York: E.P. Dutton. [Translated by B. Foxley].

Salomon, G. (1995). (Ed.), Distributed cognitions: Psychological and educational considerations. Cambridge: Cambridge University Press.

Salomon, G. (1991). Transcending the qualitative-quantitative debate: The analytic and systemic approaches to educational research. Educational Researcher, 20(6), 10-18.

Spiro, R., Feltovitch, P.J., Jacobson, M.J., & Coulson, R.L. (1992). Cognitive flexibility, constructivism and hypertext: Random access instruction for advanced knowledge acquisition in ill-structured domains. In T. Duffy & D. Jonassen (Eds.), Constructivism and the technology of instruction: A conversation. Hillsdale, NJ: Erlbaum.

Teasley, S.D., Finholt, T.A., Potter, C.S., Carragher, B., Carroll, L., Grosser, B., Hanlon, J., Kisseberth, N., Robinson, S., Weber, D., Snow, G., Claes, D.R., & Myers, J.D. (2000). Participatory science via the Internet. In B. Fishman & S. O'Connor-Divelbiss (Eds.), Proceedings of the Fourth International Conference of the Learning Societies, 376-383. Mahwah, NJ: Erlbaum.

Vygotsky, L.S. (1978). Mind in society. Cambridge, MA: Harvard University Press.

Windschitl, M., & Winn, W.D. (2000). A virtual environment designed to help students understand science. In B. Fishman & S. O'Connor-Divelbiss (Eds.), Proceedings of the Fourth International Conference of the Learning Societies, 290-296. Mahwah, NJ: Erlbaum.

Winn, W.D. (1993a). A conceptual basis for educational applications of virtual reality. HITL Technical Report R-93-9. Seattle WA: Human Interface Technology Laboratory, University of Washington.

Winn, W.D. (1993b). A constructivist critique of the assumptions of instructional design. T. Duffy, J. Lowyck, & D. Jonassen (Eds.), Designing environments for constructive learning. New York: Spinger, 189-212

Winn, W.D. (1994). Contributions of perceptual and cognitive processes to the comprehension of graphics. In W. Schnotz & R. Kulhavy (Eds.), Comprehension of Graphics. Amsterdam: Elsevier. 3-27.

Winn, W.D., Hoffman, H., Hollander, A., Osberg, K., Rose, H., & Char, P. (1999). Student-built virtual environments. Presence, 8(3), 283-292.

Winn, W.D., Windschitl, M., Fruland, R., Hedley., N., & Postner, L. (2001, April). Learning Science in an immersive virtual environment. Paper presented at the annual meeting of the American Educational Research Association, Seattle.

Winn, W.D., & Windschitl, M. (in press). Towards an explanatory framework for learning in artificial environments. Cybernetics and Human Knowing.