UNIVERSITY OF WASHINGTON>
Office of Educational Assessment
Program Evaluation Division
Updated 07/16/2002

Hands-On Laboratory-Driven EE Distance Curriculum (PANDORA)
 
EVALUATION SUMMARY

This site is intended to facilitate communication among members of the PANDORA Leadership Team and to serve as a readily accessible archive of evaluation documents to assist PANDORA leadership in reporting to external stakeholders.


PROGRAM DESCRIPTION
 

The Hands-on Laboratory-driven Electrical Engineering Curriculum project (Pandora) seeks to address the need for skilled workers in electrical and computer engineering. Funded by FIPSE, the project creates entry-level electrical engineering courses that combine project-driven curriculum and low-cost instrumentation tool kits, and offers these courses to distance learning students at other universities, at community colleges, and at home. The innovative curriculum development philosophy, which is guided by the ABET learning outcomes, positions hands-on experiments at the beginning of each topic to motivate student learning, then adds instruction to clarify and support student understanding.

Project Goals:

  • Develop a laboratory-driven curriculum for two-year and four-year postsecondary institutions to include hands-on hardware-based laboratory experience. The curriculum may be delivered on-site or synchronously on-line.
  • Adapt and enhance this laboratory-driven curriculum to target students in geographically remote communities without convenient physical access to nearby post-secondary educational institutions. This curriculum is delivered asynchronously on-line.
  • Establish a laboratory-driven curriculum development methodology common to two-year and four-year institutions, to provide hands-on laboratory experience at a reasonable cost.

EVALUATION PLAN
  The evaluation plan includes both formative measures to provide feedback to project developers, and summative assessment to address project effectiveness as shown in Table 1, below.
 

Table 1. Project Timeline and Evaluation Activities
 
Oct Nov Dec   Jan Feb Mar   Apr May Jun   Jul Aug Sep

First Year: 2000-2001
Lab Kit Development
Curriculum Development EE 215, 233, 235, 341
Online Programming
(a)   (b)   (b)
 
Second Year: 2001-2002
EE 215 Pilot   EE 233 & 235 Pilot   EE 341 Pilot
(c), (d)   (c), (d)   (c), (d)
 
Third Year: 2002-2003
CC Extension   CC Extension   CC Extension   CC Extension
(b), (c), (d)   (b), (c), (d)   (b), (c), (d)   (b), (c), (d)
 
(a) Lab kit usability testing (b) Faculty interviews (c) Student surveys (d) Student grades

Formative evaluation strategies contribute to the development and implementation of course materials. Evaluation questions include:

  • How well do curricula map to ABET learning outcomes?
  • How do students rate the quality of and their satisfaction with various instructional components?
  • How do instructors rate the quality and effectiveness of the synchronous on-line format?

The summative evaluation will address the quality and usefulness of the completed courses. Evaluation questions include:

  • What is the overall quality and effectiveness of the on-site course based on student ratings and review of course documents? How does the on-site course with motivating experiments compare to similar courses without the motivating experiments? Is there evidence of differences in student performance?
  • What is the overall quality and effectiveness of the synchronous on-line and at-home versions of the course? How do these courses compare to each other and to the on-site version in terms of student satisfaction and quality ratings? Are the printed materials well laid-out and useful? Are the instrumentation tool kits useful, easy to sue, reliable, and inexpensive? What special problems do students have taking these courses?
  • How do participating instructors rate the quality and effectiveness of the courses? What do they think of the effectiveness of the motivating experiments and the underlying instructional philosophy? Are faculty handbooks well laid out and useful?
  • What are the demographic characteristics of students enrolled in each type of course?

FINDINGS
 

First Year Progress Report

The first year of the Pandora Project has focused on curriculum development and production of the lab kit. Monthly meetings of the project team track progress and provide a venue for communicating developments and clarifying strategies.

Lab Kit
A pilot test of the lab kit was conducted during spring quarter 2001, using four student volunteers. The project director made observational notes as each student used the lab kit, and students were asked to complete a short questionnaire. Student responses were enthusiastic and suggested that the lab kit was much easier to use than current instrumentation in the lab. They also made several suggestions for improvement of the lab kit that will be incorporated in its ongoing development.

Second Year Progress Report

The second year of the Pandora project has continued to include monthly meetings of the project team to track progress and to provide a venue for communicating developments and clarifying strategies. The team has been developing course materials, reviewing and revising the online curriculum, and continuing the process of creating a lab kit at a reasonable cost. The second year was also the pilot year for the first course, EE 215.

Curriculum Development
As part of Pandora's formative program evaluation process, the Office of Educational Assessment (OEA) coordinates reviews of the online course materials. During autumn quarter 2001, OEA carried out a focus group review session session for the EE 215 online course. Five undergraduate engineering students who were concurrently enrolled in the EE 215 on-campus course participated in the review. The students viewed the curriculum and made a number of suggestions for improving the EE 215 online materials.

Student Handbook
The Student Handbook was developed to assist students in understanding electrical engineering concepts. During summer 2001, students enrolled in EE 233 were informed of resources developed for their use, including the handbook. As part of the handbook revision process, these students were asked to complete a questionnaire asking for their perceptions of the handbook as a guide to their learning. Student responses indicated that only five out of twenty-seven students used the handbook. More often, students used the textbook as their primary resource, even though it had many errors. Students indicated that the handbook was fairly well written, but gave it lower ratings with respect to usefulness of the content. In response to open-ended questions, students stated that they had the most difficulty learning math concepts and completing the problems for the course. They suggested including more sample, real-world problems in the handbook and less text. Two other suggestions provided were referring to the handbook as a workbook, and making the internet link to it more obvious.

Interviews with Online EE 215 Students
The OEA conducted student interviews of those enrolled in the online EE 215 during spring of 2002. Students were interviewed two weeks into the course and again during the final week of the quarter. The purpose of the student interviews was to gather feedback on course progress and development, as well as to reflect on the overall EE 215 experience. Student responses indicated that while there are changes that need to made for smoother course implementation, most students in hindsight would still enroll in the course. This fact attests to the importance of providing EE distance learning opportunities for students who have neither the time nor the opportunity to take an on-campus course.

Macromedia Flash Animation Use in EE 235
Flash animations are being used successfully in EE 235 to help students visualize and understand difficult concepts through active learning that enhances student problem solving. This technology consistently presents accurate information in a self-paced environment permitting students to avoid boredom or confusion by "clicking ahead" or returning to review.

EE 233 Student Focus Group: Lab Videos and Online Materials
Students in the EE233 student focus group did not consistently utilize the lab videos, but felt that they would use the videos if they were improved. Students suggested including more explanation of how to set up the labs and more examples of how to approach challenging problems. Suggestions for improving online materials included providing more practice problems; having instructors give incentives for students to access online materials, making a clear web link to online materials; and fixing answer formatting and scrolling problems.


Copyright© 2001-2003 UNIVERSITY OF WASHINGTON Office of Educational Assessment