Research in Engineering Student Learning
Description of project strand. In this research area we seek to characterize how engineering students solve engineering design problems, to understand the effectiveness of current approaches to engineering design instruction, and to ultimately develop and evaluate instruction to enhance the learning of engineering design. (For examples of work in this area, see our NSF conference, ICREE, and Research & Application handouts, as well as a CELT tech report on representations of design process, CELT-11-03.)
Relevance of project strand. Design is central to engineering and thus a central area of CELT research. Although increasing amounts of research exist concerning engineering design processes, few studies focus on the design processes of engineering students and the capabilities they must acquire to be effective engineering practitioners. As such, a broad goal is to more closely align the outcomes of engineering education with the needs of engineering practice. This would include an ability to consider global issues in engineering problem solving such as economic, environment, and social factors.
Significant findings. For more than a decade we have conducted empirical studies building on multiple datasets to comprehensively document and understand engineering student design processes. One of these studies includes longitudinal data where undergraduate engineering students participated as entering freshmen and later as graduating seniors. We have used these studies collectively to (a) describe how freshmen and senior engineering students differ in their approaches to design problem solving, (b) measure differences in design activity across levels of design experience and final solution quality, (c) correlate design performance with various design behaviors (e.g., information gathering behaviors, iteration, generating solutions, and evaluating solutions), (d) explore the effects of interventions on student design processes, and (e) develop a set of metrics that can be used to document and assess design processes. A review of some of these findings is described below.Student Design Processes:
We used verbal protocol analysis, a research method firmly grounded in cognitive psychology to capture the design process of 50 students (26 freshmen, 26 seniors) as they designed a playground for a fictitious neighborhood. The students solved the problem out loud. A written transcript is made of the verbal record Ð and this forms the data for further analysis. The transcripts are coded in a two-step process. First, they are segregated into idea units. Second, a coding scheme derived from prescriptive models of the design process is applied to these segments. Each of these two steps is applied to the data independently by two analysts. Their results are compared, and any discrepancies are argued to a consensus judgment. A minimum reliability is expected to ensure replicability of results. The data show that the students' processes are substantially different from the prescriptive models. In addition, the freshman and senior student processes are significantly different on several key variables (e.g., quality of solutions, number of transitions, and progress to latter stages in the design process) (Atman et al., 1999).
In Bursic and Atman (1997), we explore how students bring information into the design process. This work used the same data described above, but focused on characterizing the information students requested in the design process. As such it provides a complimentary image of the design process.
We explored the design processes of 93 freshmen and senior engineering students using a within-subject verbal protocol analysis experimental design. We are able to replicate of our main findings from the playground study described above (Turns et al., 2002, Atman et al., in press).
Expert Design Processes:
In Atman and Bursic (1996) we sought to determine the smallest intervention that may result in different in student design processes (characterized with the same verbal protocol methodology described above). We found that an intervention as small as reading from a design text had a significant impact on student design processes.
We are working with industry partners to extend our work in engineering student design behavior with studies of practicing engineering design experts. These experts include professionals in such engineering fields as mechanical, electrical, civil, industrial engineering, and materials science and engineering. This will provide opportunities to compare experts to the students in our earlier studies to :
(1) identify appropriate learning targets and educational environments in the teaching of design and
(2) develop a continuum for describing the learning of design (Adams et al, 2003).
In a preliminary study of four engineering faculty members' design processes we observed that engineering educators approach design with a varied toolbox of strategies (Atman et al, 2003). Two focused heavily on gathering information about the problem while the other two focused more on generating design ideas. Two also displayed considerable awareness of the decisions they made throughout their design process. All iterated frequently - continuously revisiting their understanding of the design problem to produce better solutions. Despite their use of varying strategies, the instructors have a keen awareness of what they are doing as they design and why they are employing that particular strategy. Engineering students can benefit from this combination of varied strategies and awareness of design decisions.
Problem scoping is central to design activity and refers to the portion of the design process where designers define the nature of the design problem and the space in which they will search for design solutions. This often involves gathering information from broad sources, framing the requirements of design solutions, clarifying and prioritizing these requirements, and determining the needs of the intended user. We have been developing and utilizing strategies for documenting problem scoping behaviors across academic levels and engineering disciplines (see Bogusch et al, 2000; Rhone et al, 2001). We have found that as designers gain more experience, they ask for more information of a broader scope such as maintenance and safety issues (Bursic and Atman, 1997).
Representations play a multifunctional role in design activity. For example, they can capture a designer's current understanding of the problem, provide feedback about strengths and weaknesses of a particular design solution, and communicate ideas and final designs to broad groups of people. We have analyzed the ways in which designers' create and manipulate representations to support their design activity (Cardella et al, 2002). One finding is that sketches support all aspects of design activity including evaluating and making decisions, two activities that from our previous work we have observed that students tend to underemphasize.
We have recently been conducting research on the effectiveness of our design representations. Click the image below to see and hear our representations of design process data.
Conceptions of Design:
Beliefs and attitudes about the nature of design can shape a designer's perspective in how they solve design problems. For example, we have found that engineering seniors who consider iteration as central to design are more likely to spend time iterating and use a wider variety of iterative strategies to work towards a high quality solution (Adams, 2001).
Through our research we have developed and validated measures for capturing various aspects of design knowledge and skills. For example, we have identified measures of design processes and how they correlate with performance or experience (e.g., Atman et al 1997; in press). We have developed frameworks to support design assessment in design courses (Safoutin et al, 2000). We have also developed representations for illustrating and analyzing design behavior such as process timelines and problem space grids (e.g., Rhone et al 2001). Some of these we have transformed into instructional and assessment tools for improving design teaching and learning.
We are currently exploring different strategies for synthesizing across all of our studies. For example, in Adams (2003) we discuss how our findings can be used to characterize students as reflective practitioners. In Adams et al (2003) we discuss our efforts in developing a continuum to describe the acquisition of design expertise. See our publications section to view our current publications. Additionally, see our Instructional Services web pages to access materials from some of CELTís faculty workshops and classroom presentations that are based upon our design research.
Our research was funded by an NSF ROLE grant to study design expertise and a Center for Teaching and Learning grant (CAEE) to develop capacity in the scholarship of engineering education. A portion of our work is also funded through our development efforts at the Center for Engineering Learning and Teaching which includes public and private donations. This would include grants from industry partners such as Ford, the GE Fund, HP, and Boeing. Past research has been funded through a variety of NSF awards, particularly a Young Investigator Award (Atman), and grants from industry partners.