Prior Projects

Undergraduate-focused Evaluation

Diversity Action Plans

Supported by the UW College of Engineering and several engineering departments, the purpose of the UW Engineering diversity action plans was to facilitate the departmental ability to make data-driven decisions that will enable UW departments to attract increasing numbers of highly sought after diverse students.  Using a mixed methods approach (surveys, demographic data, bench-marking, and focus groups), CERSE synthesized data to create a diversity profile for each department and made targeted, actionable recommendations to department chairs.

ENGAGE

Funded by the National Science Foundation, the goal of ENGAGE was to increase the capacity of engineering schools to retain undergraduate students by facilitating the implementation of three research-based strategies to improve student day-to-day classroom and educational experience.  As the evaluator of ENGAGE, CERSE interviewed and surveyed implementation teams on each campus to determine the extent to which students were being impacted by the ENGAGE strategies, faculty perceptions of changes in student engagement, and changes in student performance in spatial visualization.

Oregon State University Improving Undergraduate STEM Education

The Oregon State University (OSU) STEM Leaders Program (STEM-LP): Improving Undergraduate STEM Education NSF grant, was designed to increase the number and diversity of STEM graduates at OSU. The program offered a cohort-based STEM orientation seminar, upper-class peer mentoring, faculty-mentored research, and cohort-based workshops. CERSE served as evaluator for the STEM-LP program, quantitatively tracking both institutional and student-level data and conducting comparative student focus groups to measure the program’s impact.

Northwest Engineering Talent Expansion Partnership (NW-ETEP)/MESA Community College Partnership (MCCP)

CERSE participated in the NSF-funded MESA Community College Partnership (formerly Northwest Engineering Talent Expansion Partnership), a joint effort to increase the number of underrepresented minorities and women in engineering. MCCP is composed of Washington State community colleges and universities offering engineering and preparatory degrees, as well as organizations within these institutions that are concerned with diversifying the STEM workforce. The primary goal of MCCP is to promote the participation and retention of underrepresented groups in engineering programs in Washington State. CERSE was the internal evaluator of the MCCP grant, assessing MCCP activities at eight post-secondary institutions.

Transforming Engineering through PEERs: Building a Better Experience for Underrepresented Students

PEERS was an institutional transformation project funded by NSF to positively impact the climate of engineering through a cadre of change agents who create and encourage improved and more equitable relationships at the University of Washington. This project worked to “fix the system” rather than the typical retention approach of “fixing the student.” In its assessment, CERSE measured the outcomes of this project on the institutional climate of engineering at the UW through undergraduate climate surveys; focus groups with students, faculty, and staff; and analysis of changing demographics in engineering enrollments and degrees.

Replicable Model for University Student-Run Outreach Programs

Funded by the Engineering Information Foundation, University of Washington Women’s Initiative (UWWI)  provided tools and information to establish student-led outreach programs in order to recruit more students into engineering fields. As the evaluator, CERSE conducted a short-term outcomes and formative assessment to measure preliminary impact of the program on middle school students. Outcomes examined included changing stereotypes of engineers, new perspectives on the social value of engineering, and increased interest in the engineering profession. In addition, CERSE assessed the effectiveness of the web materials.

Pacific Northwest Louis Stokes Alliance for Minority Participation (NSF PNW LSAMP)

The Pacific Northwest Louis Stokes Alliance for Minority Participation (PNW LSAMP) is an NSF-funded project that aims to broaden the participation of underrepresented minorities (URM) in science, technology, engineering and mathematics (STEM). The Alliance is comprised of five universities (University of Washington, Washington State University, Boise State University, Portland State University and Oregon State University) and partnering two-year colleges in Washington, Oregon, and Idaho. PNW LSAMP provides academic advising, orientation programming, undergraduate research resources, professional development activities, study groups and tutoring services, and various other skills building workshops and seminars to URM STEM students. CERSE served as the internal evaluator for PNW LSAMP from 2009-2019. CERSE’s PNW LSAMP evaluation activities included: Collection, management, analysis and submission of each institution’s student-level and institutional data to NSF; Analysis of programmatic activities, accomplishments, and obstacles; PNW LSAMP participant survey; PNW LSAMP conference survey. With renewed NSF funding, CERSE will continue their partnership with PNW LSAMP in a research rather than an evaluation capacity. See the Research page for more information.

Washington State Academic RedShirt (STARS)

University of Washington and Washington State University were awarded a collaborative grant by the National Science Foundation to establish the Washington State Academic RedShirt (UW STARS and WSU STARS) program. STARS provides highly motivated students who are eligible for financial aid with a specialized first-year curriculum designed to build learning skills and academic preparation. The goal of STARS is to increase the number of undergraduate engineering degrees awarded to underrepresented students. CERSE served as the evaluator of STARS. Evaluation activities included using comparison groups to monitor the impact of the program on student perceptions of academic and social supports as well as academic indicators of success such as difference in retention, grades in math courses, and grade point average.

Faculty-focused evaluations

The Engineering Deans Equity (EDGE) Initiative

The American Society for Engineering Education (ASEE)’s Engineering Deans Gender Equity (EDGE) Initiative is a 3-year program funded through a 2018 National Science Foundation ADVANCE Adaptation award. The EDGE Initiative provides tools, strategies, and resources to support academic engineering deans in identifying and removing barriers impeding recruitment, retention and advancement of women faculty from all backgrounds in engineering higher education (https://edge.asee.org/).

CERSE’s evaluation of the EDGE Initiative focuses on impacts at the individual, college, and ASEE levels. At the individual level, to what extent does participation in EDGE efforts increase deans’ understanding of the state of equity within their colleges, knowledge of strategies to improve equity, and development of action plans to address gender equity issues? At the college level, to what extent are data from the EDGE tools used to inform strategies and equity decisions? At the ASEE level, how are EDGE initiatives influencing access to gender equity resources, and to what extent do EDGE efforts result in more deans signing the ASEE diversity and equity pledge?

Launching Academics on the Tenure-Track: an Intentional Community in Engineering (LATTICE)

LATTICE, a joint effort between the University of Washington, North Carolina State University, and California Polytechnic State Institute, is a national program designed to advance faculty diversity in engineering. LATTICE seeks to positively impact early-career women in Electrical Engineering and Computer Science and early-career underrepresented minority women in Engineering who are interested in faculty careers. This program includes a professional development intervention, adapted from the BRAINS program, along with a research study to understand how and why the intervention works in order to develop a model framework to allow others to develop such programs for a wide variety of fields and populations.

CERSE is utilizing a mixed-methods approach for the formative and summative evaluation of this project. Deliverables include results of a longitudinal survey that assesses individual-level impacts and qualitative interviews to contribute to formative feedback on the effectiveness of program adaptations by discipline and target population. Evaluation seeks to answer the following summative evaluation question: Are the professional development programs having a positive impact on participants at the individual level in terms of professional skills development, effective use of mentoring, and access to professional networks?

Postdoc Best Practices

UW Computer and Science Engineering (PI: Ed Lazowska and Brian Curless) is leading a project entitled “Taking Collective Responsibility for the Postdoc Experience at the University of Washington” with goals to: bring UW CSE support for postdocs in line with best practices and experiment with new practices; disseminate standardized national resources to be used by other CS departments; demonstrate the effectiveness of their proposed “best practices” as well as their adoption and effect at other programs around the country.

The evaluation seeks to answer questions such as: To what extent do the best practices (around position and benefits, goals and expectations of the appointment, periodic evaluation, and career development and job placement) improve the experiences and outcomes of UW CSE post-docs? If so, how? Which best practices have the most evidence of impact?

Active Societal Participation In Research and Education (ASPIRE)

The NSF-funded ASPIRE project aims to broaden participation in the geosciences by developing a model for Mobile Working Groups (MWGs)–a place-based participatory research strategy intended to bridge divides between marginalized communities and the academy. In doing so, ASPIRE hopes to cultivate a culture of high-quality, community-centered research that will attract a new and more diverse generation of geoscientists.

CERSE’s evaluation of ASPIRE is focused on assessing the development of the MWG model with regard to the intent of the project, the support and guidance provided to MWG leaders, and the change in mindset/behavior among scientists and community leaders engaged in the project. Evaluation activities include observation of team meetings and workshops, ongoing discussions with the team regarding “lessons learned” from the pilot phase, and surveys and interviews with MWG participants.

Transforming Engineering Culture to Advance Inclusion and Diversity (TECAID)

TECAID, funded by the NSF, supports five U.S. Mechanical Engineering (ME) departments to advance diversity and build inclusive environments for students, faculty, and staff. Over an 18-month period, TECAID has provided support for ME departments to create and sustain inclusive interactions and cultures that benefit all participants–in classrooms and labs, in student design groups, in faculty meetings and hallway interactions, and in the underlying department dynamics.

CERSE is using a longitudinal design to monitor the extent and nature of departmental transformation (in terms of awareness, understanding, and behavior) over time by examining the impact of TECAID implementation at departmental unit and department chairs/leader levels.

Advancement, Recruitment, and Retention of Women in Science (ARROWS) with Medical University of South Carolina

ARROWS was designed to foster a culture in the medical school that promotes greater participation and advancement of basic scientist women in a way that is both sustainable and serves as an easily portable and adoptable model for peer institutions. This project included the development and implementation of a package of practices and policies to specifically benefit women in basic science in a single medical school, as well as usher a cultural change across other medical schools nationwide by helping them to recruit, retain, and advance women scientists. CERSE served as the external evaluator for this grant. The process evaluation identified progress ARROWS made in the implementation of its activities to advance its two outcomes of improving recruitment as well as retention and advancement of women and minority basic scientists. To complement the summative assessment conducted by the internal evaluation team, the external evaluators examined the depth and pervasiveness of changes in basic science faculty recruitment, retention, promotion and tenure, and advancement of women into leadership positions within departments.

Center for Institutional Change, ADVANCE Institutional Transformation

With a National Science Foundation ADVANCE Institutional Transformation Award, the University of Washington created the Center for Institutional Change (CIC) to help transform the culture at the UW. The aim of ADVANCE was to increase the participation of women in science and engineering.

The UW ADVANCE project was designed to build upon existing strengths at the university while serving as the catalyst for institutional transformation. The CIC focused on the implementation of programs designed to eliminate existing barriers to women and to precipitate cultural change at both the departmental and the institutional level. The UW ADVANCE Center for Institutional Change had six major components:

• Leadership development for science, technology, engineering, and mathematics (STEM) chairs and deans.
• Department cultural change.
• Examination of UW policies for equity and policy transformation.
• Mentoring women in STEM for leadership.
• Visiting scholars who can contribute to the goals of the project.
• Transitional support for female faculty members in STEM departments.

CERSE served as the internal evaluator for the ADVANCE grant and managed a faculty and graduate student mentoring program for the CIC. CERSE also coordinated the external evaluation process led by the American Association for the Advancement of Science (AAAS). The NSF ADVANCE grant has ended, but CIC has continued a subset of the activities and CERSE has continued to serve as internal evaluator, albeit in a more limited capacity.

Center for Institutional Change, Chair Leadership Workshops

Leadership Excellence for Academic Diversity (LEAD) was a series of two-day national leadership workshops over a three-year period for department chairs, deans, and emerging leaders in STEM offered by the ADVANCE Center for Institutional Change at the University of Washington and funded by the National Science Foundation. The goals of LEAD included:

• Learn practical strategies and tools to improve department culture and climate.
• Explore unintended and unconscious biases that disproportionately impact women and under-represented faculty.
• Address important issues such as the recruitment and selection of faculty, the evaluation of faculty performance, conflict resolution, and leadership.
• Discuss case studies on faculty and departmental life.
• CERSE served as the evaluator of LEAD to measure the usefulness, relevance, and applicability of the workshop content.

Center for Institutional Change, Lead-it-Yourself! (LiY!)

From 2007 to 2009, the University of Washington’s ADVANCE Center for Institutional Change (UW ADVANCE) promoted the advancement of women and other underrepresented faculty in STEM departments across the nation by annually creating and delivering in-person Leadership Excellence for Academic Diversity (LEAD) workshops. To meet the national demand for LEAD workshops, UW ADVANCE created LEAD-it-Yourself! (LiY!), an online open-source toolkit of planning and instructional materials to enable institutions to run their own local or regional LEAD-inspired workshops. Several institutions were engaged in iterative evaluations instituting their own LEAD-inspired workshops. CERSE served as the assessment and evaluation consultant and conducted both process and outcomes evaluation of LiY!. CERSE tracked both implementation fidelity, monitoring whether LiY! workshops are delivered as the project team intends, and outcomes of LiY! workshops via impact of participants.

Center for Institutional Change, On-Ramps into Academia Workshops

Funded by NSF, On Ramps into Academia was a series of two-day workshops over a three-year period to provide practical tools and support to women who are interested in making the transition to academia. The goal of this project was to increase the pool of women faculty available to all universities by providing professional development to Ph.D.-level women in industry or research laboratories. It was offered by the ADVANCE Center for Institutional Change at the University of Washington.

CERSE served as the evaluator of On-Ramps. The assessment plan was strongly directed at outcomes evaluation and helped identify concrete best practices to encourage women into faculty pathways.

Council of Colleges of Arts & Sciences, ADVANCE Institutional Transformation

The Council of Colleges of Arts and Sciences (CCAS) was the focus of a collaborative effort funded through the National Science Foundation’s (NSF) ADVANCE program. The overarching aim of this grant was to cultivate academic leaders who are more knowledgeable about STEM gender equity issues, more able and motivated to address those issues, and thus better positioned to effect positive and transformational change in their own colleges and departments. CERSE used several assessment strategies including longitudinal survey analysis, document analysis, and observations to conduct summative, formative, and process evaluations for this $1.2 million project.

NSF CAREER Award Assessment of Educational Outreach Plans

Funded by the NSF CAREER program, Dr. Sapna Cheryan received a grant to fund her research and education activities related to stereotypes in computer science (CS). Part of her educational plan is to educate teachers, professors, advisors, and students about how stereotypes can limit the diversity in the CS field and how even minor adjustments to environments can have a positive impact on student perceptions of CS. CERSE evaluated progress towards Dr. Cheryan’s educational goals through workshop feedback and portfolio reviews.

Women Evolving Biological Sciences Symposia

Funded by NSF, Women Evolving Biological Science (WEBS) was a  three-day symposium aimed at addressing the retention of female scientists and issues related to the transition of women from early career stages to tenure track positions and leadership roles in academic and research settings that ran from 2007 through 2013. The goal of WEBS was to significantly increase the retention and promotion of women in academia in the biological sciences in order to create greater diversity in academic and scientific leadership.

As the evaluator, CERSE provided an outcomes evaluation and formative evaluation. Participants were tracked longitudinally to monitor changes in their career paths and the role of WEBS in providing guidance and support in their careers.

K-12 / MESA Evaluations

First Nations MESA

CERSE worked with Washington Math, Engineering, and Science Achievement (MESA) to conduct an evaluation for a capacity-building grant for First Nations MESA in order to: (1) assist First Nations MESA in developing a best practices model that can be exported to other Native communities across the state of Washington; (2) assess the need for culture-based science, technology, engineering, and mathematics (STEM) curriculum in the local schools; and (3) provide recommendations for local expansion. Evaluation activities included conducting a) pre-evaluation focus groups with stakeholders (completed Fall 2008), b) observations of selected First Nations MESA activities (2008-2009), and c) a second stakeholder analysis.

Strategic Education Centers (SEC)

Funded by the Gates Foundation, Strategic Education Centers aim to empower youth in Swaziland by providing health education, student exchange opportunities, and technology and academic training. In addition to the prevention of HIV/AIDS, which is widespread in Swaziland, SEC aims to give adolescent boys and girls the knowledge to pursue higher education in technological fields, and subsequent employment in thriving fields. Originally intended for girls, the program will soon be expanded to include boys.

CERSE, in conjunction with the UW Center for Health Research and Education (CHER), monitored and evaluated the success of each cohort of students participating in the SEC program. CERSE’s formative evaluation constituted an improvement of SEC’s previous monitoring and evaluation techniques, and allowed SEC to document the changes in student behaviors, attitudes, and knowledge of HIV/AIDS prevention.

Nanotechnology-related Evaluations

Establishing a Minor in Nanoscale and Molecular Biology (NSF NUE)

In 2009, the National Science Foundation Nanotechnology Undergraduate Education (NUE) program awarded the University of Washington a grant to support the project NUE NME Minor: Minor in Nanoscience and Molecular Engineering. The main purpose of this grant was to support development of an undergraduate minor in Nanoscience and Molecular Engineering. To measure the impact of lab modules and new course development on students, formative and summative (outcomes) evaluations were conducted to assess and improve the effectiveness of these components of the new NME minor curriculum. Strategies employed included course and lab module assessments and tracking of student interest and intent to minor in NME.

Nanotechnology Ph.D. IGERT Program

Funded by the National Science Foundation IGERT program, CERSE was contracted by the UW Center for Nanotechnology to study outcomes of students participating in its Ph.D. program. The goals of the evaluation were to:

• Monitor the progress of students affiliated with the Nanotechnology Program.
• Give students a confidential mode of providing feedback to the Nanotechnology Program.
• Address specific topics that were designated by the Nanotechnology Program.
• Assess the strengths and weaknesses of the program from a student perspective.

CERSE developed a tracking system that monitored student progress, recorded student perceptions about the field, and evaluated the role of an integrative educational program in the lives of the students. In addition, the questionnaire was designed to assess career preparation, participation in industry internships, interdisciplinary collaboration and student’s publication efforts.