Prior Projects

Undergraduate-focused Evaluation

Future Rivers NSF Research Traineeship (NRT), University of Washington

The University of Washington Future Rivers NRT program intends to develop innovative, culturally-aware STEM workforce and produce increasingly diverse cohorts of interdisciplinary freshwater scientists. To help achieve these goals, CERSE provides STEM equity and inclusion training through workshops to provide program mentors and Summer Institute participants with the tools to do inclusive science that is attractive to a diverse set of program participants. CERSE representatives also consult with faculty and students to provide advice and feedback on equitable pedagogy and inclusion practices for the classroom.

NCWIT Extension Services for Undergraduate Programs

The evaluation of NCWIT Extension Services incorporates both summative and formative evaluation including data collection and analysis for the NCWIT Extension Services metrics. Metrics are collected around awareness of the gender related issues in participating departments, understanding of organizational change, progress in implementing interventions, practices and strategic plans; growth of a social science resource pool in the form of the Extension Service Consultants, and the ultimate goals of increased recruitment and retention of women in computing.

Redshirt in Engineering Consortium

The NSF-funded Redshirt in Engineering Consortium was formed in 2016 with the goal of enhancing the ability of academically talented but underprepared students coming from low-income backgrounds to successfully graduate with engineering degrees. The academic Redshirt model developed at the University of Colorado Boulder (CU-B) and extended to the University of Washington (UW) and Washington State University (WSU) was inspired by athletic Redshirt programs, which provide college athletes a year to prepare to compete at the university level. The Consortium builds off the success of existing academic Redshirt programs at CU-B, UW, and WSU by expanding the model to Boise State University, University of California at San Diego, and University of Illinois at Urbana-Champaign. The Existing Redshirt Institutions (ERIs) help mentor and train the new Student Success Partners (SSP), and SSPs contribute their unique expertise to help ERIs improve existing Redshirt programs.

CERSE’s evaluation of the Redshirt in Engineering Consortium involves observation of consortium-level activity, collection and analysis of university/college-level degree and enrollment data, and bi-annual student surveys. The evaluation assesses the extent to which programs have an impact on student success and broadening participation in engineering, and whether/how the consortium collaborates to exchange best practices and refine the Redshirt model for broader dissemination.

Selected Publications

Knaphus-Soran, E., Delaney, A., Tetrick, K. C., Cunningham, S., Cosman, P., Ennis, T. D., Myers, B. A.,…Ferrez,M. (2018). Institutional Context and the Implementation of the Redshirt in Engineering Model at Six Universities. Proceedings of the American Society for Engineering Education, Salt Lake City, UT, June 2018.

Myers, B., Knaphus-Soran, E., Llewellyn, D. C., Delaney, A., Cunningham, S., Cosman, P., Pitts, K. (2018). Redshirt in Engineering: A model for improving equity and inclusion. Proceedings of the Collaborative Network for Engineering and Computing Diversity (CoNECD) Conference, Crystal City, VA, May 2018.

Spokane Falls Community College ATE in Cybersecurity

Spokane Falls Community College launched the Advanced Technological Education (ATE) program focuses on the education of technicians for the high-technology fields and partners with academic institutions and promote improvement in the education of science and engineering technicians at the undergraduate and secondary institution school levels with an emphasis on two-year institutions. CERSE conducts focus groups with participating students, interviews with employers, and observes team meetings in order to formatively and summatively assess the program implementation.

University of Illinois IUSE/EHR Project

CERSE is the evaluator for the University of Illinois IUSE/EHR grant “Gendered Career Paths in Engineering Undergraduate Education: Antecedents and Implications” which is studying how and why elective track choices become gendered during undergraduate engineering education. Our evaluation scope of work includes monthly formative feedback on processes and products, characterization of the case analysis and emergent themes, and describing potential impact of the project outcomes.

Western Washington University Computer Science and Math Scholars S-STEM

Western Washington University’s Computer Science and Math Departments are implementing an S-STEM program to support low-income but high achieving undergraduate students in their departments. The project is also researching how curricular activities influence student self-efficacy, identity or sense of belonging, and how early exposure to computer science affects personal attributes and professional outcomes. CERSE is evaluating the CS/M Scholars goals using student surveys, focus groups, observations, and review of the research to ensure that the program will be effective and efficient in achieving their goals.

Society of Women Engineers (SWE) Networking Events for Community College Students

The Society of Women Engineers (SWE) has organized two networking series. SWE’s long-term goal for this project is to expand the representation of women in engineering and computer by supporting women community college students who may pursue this professional pathway. This project constitutes the third phase of an ongoing research project that investigates persistence in engineering and computer science for women at community colleges. Phase III builds on findings from previous work that CERSE conducted in collaboration with SWE that suggested community college women in engineering could benefit from more information about engineering and computer science career pathways, increased focus on mentorship, strengthening relationships with engineering professionals, and improving self-confidence. CERSE is evaluating the series of networking events to assess the extent to which participating in the events successfully facilitates community college women’s professional knowledge, relationships, and self-confidence.

 

Diversity Action Plans

Supported by the UW College of Engineering and several engineering departments, the purpose of the UW Engineering diversity action plans was to facilitate the departmental ability to make data-driven decisions that will enable UW departments to attract increasing numbers of highly sought after diverse students.  Using a mixed methods approach (surveys, demographic data, bench-marking, and focus groups), CERSE synthesized data to create a diversity profile for each department and made targeted, actionable recommendations to department chairs.

ENGAGE

Funded by the National Science Foundation, the goal of ENGAGE was to increase the capacity of engineering schools to retain undergraduate students by facilitating the implementation of three research-based strategies to improve student day-to-day classroom and educational experience.  As the evaluator of ENGAGE, CERSE interviewed and surveyed implementation teams on each campus to determine the extent to which students were being impacted by the ENGAGE strategies, faculty perceptions of changes in student engagement, and changes in student performance in spatial visualization.

Oregon State University Improving Undergraduate STEM Education

The Oregon State University (OSU) STEM Leaders Program (STEM-LP): Improving Undergraduate STEM Education NSF grant, was designed to increase the number and diversity of STEM graduates at OSU. The program offered a cohort-based STEM orientation seminar, upper-class peer mentoring, faculty-mentored research, and cohort-based workshops. CERSE served as evaluator for the STEM-LP program, quantitatively tracking both institutional and student-level data and conducting comparative student focus groups to measure the program’s impact.

Northwest Engineering Talent Expansion Partnership (NW-ETEP)/MESA Community College Partnership (MCCP)

CERSE participated in the NSF-funded MESA Community College Partnership (formerly Northwest Engineering Talent Expansion Partnership), a joint effort to increase the number of underrepresented minorities and women in engineering. MCCP is composed of Washington State community colleges and universities offering engineering and preparatory degrees, as well as organizations within these institutions that are concerned with diversifying the STEM workforce. The primary goal of MCCP is to promote the participation and retention of underrepresented groups in engineering programs in Washington State. CERSE was the internal evaluator of the MCCP grant, assessing MCCP activities at eight post-secondary institutions.

Transforming Engineering through PEERs: Building a Better Experience for Underrepresented Students

PEERS was an institutional transformation project funded by NSF to positively impact the climate of engineering through a cadre of change agents who create and encourage improved and more equitable relationships at the University of Washington. This project worked to “fix the system” rather than the typical retention approach of “fixing the student.” In its assessment, CERSE measured the outcomes of this project on the institutional climate of engineering at the UW through undergraduate climate surveys; focus groups with students, faculty, and staff; and analysis of changing demographics in engineering enrollments and degrees.

Replicable Model for University Student-Run Outreach Programs

Funded by the Engineering Information Foundation, University of Washington Women’s Initiative (UWWI)  provided tools and information to establish student-led outreach programs in order to recruit more students into engineering fields. As the evaluator, CERSE conducted a short-term outcomes and formative assessment to measure preliminary impact of the program on middle school students. Outcomes examined included changing stereotypes of engineers, new perspectives on the social value of engineering, and increased interest in the engineering profession. In addition, CERSE assessed the effectiveness of the web materials.

Pacific Northwest Louis Stokes Alliance for Minority Participation (NSF PNW LSAMP)

The Pacific Northwest Louis Stokes Alliance for Minority Participation (PNW LSAMP) is an NSF-funded project that aims to broaden the participation of underrepresented minorities (URM) in science, technology, engineering and mathematics (STEM). The Alliance is comprised of five universities (University of Washington, Washington State University, Boise State University, Portland State University and Oregon State University) and partnering two-year colleges in Washington, Oregon, and Idaho. PNW LSAMP provides academic advising, orientation programming, undergraduate research resources, professional development activities, study groups and tutoring services, and various other skills building workshops and seminars to URM STEM students. CERSE served as the internal evaluator for PNW LSAMP from 2009-2019. CERSE’s PNW LSAMP evaluation activities included: Collection, management, analysis and submission of each institution’s student-level and institutional data to NSF; Analysis of programmatic activities, accomplishments, and obstacles; PNW LSAMP participant survey; PNW LSAMP conference survey. With renewed NSF funding, CERSE will continue their partnership with PNW LSAMP in a research rather than an evaluation capacity. See the Research page for more information.

Washington State Academic RedShirt (STARS)

University of Washington and Washington State University were awarded a collaborative grant by the National Science Foundation to establish the Washington State Academic RedShirt (UW STARS and WSU STARS) program. STARS provides highly motivated students who are eligible for financial aid with a specialized first-year curriculum designed to build learning skills and academic preparation. The goal of STARS is to increase the number of undergraduate engineering degrees awarded to underrepresented students. CERSE served as the evaluator of STARS. Evaluation activities included using comparison groups to monitor the impact of the program on student perceptions of academic and social supports as well as academic indicators of success such as difference in retention, grades in math courses, and grade point average.

Faculty-focused evaluations

AdvanceRIT

We conduct evaluation of AdvanceRIT programs, which aim to improve the representation, retention, and career advancement of women faculty in STEM by removing barriers to important resources and providing new resources through strategic interventions. We evaluate AdvanceRIT programming, including the Connectivity Series Events and Connect Grants.

ASME Women in Mechanical Engineering

The American Society of Mechanical Engineering (ASME) promotes the art, science & practice of multidisciplinary engineering and allied sciences around the globe. ASME Women in Mechanical Engineering (WiME) aims to increase the diversity and competitiveness of the overall engineering workforce by hosting a national conference to discuss engineering education and culture as a key driver in increasing the number of women in mechanical engineering. CERSE is providing both formative and summative evaluation, observing monthly calls and the conference. Attendee and outcome surveys assess how well the conference helped build networks, collaborations, community, and knowledge.

Council of Public Liberal Arts Colleges ADVANCE Partnering and Liaising Across the Nation (COPLAC ADVANCE PLAN)

Funded by an NSF ADVANCE Partnership grant, this project brings research research-based training in gender equity issues to administrators and provides networking opportunities for women faculty and faculty from underrepresented groups. The COPLAC ADVANCE PLAN is designed to create institutional change across COPLAC by training both administrators and department chairs to recognize bias, identify intersectional policies, and solve issues that contribute to gender inequities in STEM. The project also provides professional support and networking to women STEM faculty through interest-based affinity groups within COPLAC to foster community, empowerment, research collaboration, mentoring, and solutions to overcoming gender-based barriers to career advancement.

LGBTQ+ Advocacy in STEM 

This project aims to increase the inclusion of LGBTQ+ students and professionals in engineering by providing support for new approaches to foster inclusion, through both a Virtual Community of Practice and the refinement and dissemination of Safe Zone Workshops. CERSE provides both formative and summative evaluation via annual surveys and participant observation.

The Engineering Deans Equity (EDGE) Initiative

The American Society for Engineering Education (ASEE)’s Engineering Deans Gender Equity (EDGE) Initiative is a 3-year program funded through a 2018 National Science Foundation ADVANCE Adaptation award. The EDGE Initiative provides tools, strategies, and resources to support academic engineering deans in identifying and removing barriers impeding recruitment, retention and advancement of women faculty from all backgrounds in engineering higher education (https://edge.asee.org/).

CERSE’s evaluation of the EDGE Initiative focuses on impacts at the individual, college, and ASEE levels. At the individual level, to what extent does participation in EDGE efforts increase deans’ understanding of the state of equity within their colleges, knowledge of strategies to improve equity, and development of action plans to address gender equity issues? At the college level, to what extent are data from the EDGE tools used to inform strategies and equity decisions? At the ASEE level, how are EDGE initiatives influencing access to gender equity resources, and to what extent do EDGE efforts result in more deans signing the ASEE diversity and equity pledge?

Launching Academics on the Tenure-Track: an Intentional Community in Engineering (LATTICE)

LATTICE, a joint effort between the University of Washington, North Carolina State University, and California Polytechnic State Institute, is a national program designed to advance faculty diversity in engineering. LATTICE seeks to positively impact early-career women in Electrical Engineering and Computer Science and early-career underrepresented minority women in Engineering who are interested in faculty careers. This program includes a professional development intervention, adapted from the BRAINS program, along with a research study to understand how and why the intervention works in order to develop a model framework to allow others to develop such programs for a wide variety of fields and populations.

CERSE is utilizing a mixed-methods approach for the formative and summative evaluation of this project. Deliverables include results of a longitudinal survey that assesses individual-level impacts and qualitative interviews to contribute to formative feedback on the effectiveness of program adaptations by discipline and target population. Evaluation seeks to answer the following summative evaluation question: Are the professional development programs having a positive impact on participants at the individual level in terms of professional skills development, effective use of mentoring, and access to professional networks?

Postdoc Best Practices

UW Computer and Science Engineering (PI: Ed Lazowska and Brian Curless) is leading a project entitled “Taking Collective Responsibility for the Postdoc Experience at the University of Washington” with goals to: bring UW CSE support for postdocs in line with best practices and experiment with new practices; disseminate standardized national resources to be used by other CS departments; demonstrate the effectiveness of their proposed “best practices” as well as their adoption and effect at other programs around the country.

The evaluation seeks to answer questions such as: To what extent do the best practices (around position and benefits, goals and expectations of the appointment, periodic evaluation, and career development and job placement) improve the experiences and outcomes of UW CSE post-docs? If so, how? Which best practices have the most evidence of impact?

Active Societal Participation In Research and Education (ASPIRE)

The NSF-funded ASPIRE project aims to broaden participation in the geosciences by developing a model for Mobile Working Groups (MWGs)–a place-based participatory research strategy intended to bridge divides between marginalized communities and the academy. In doing so, ASPIRE hopes to cultivate a culture of high-quality, community-centered research that will attract a new and more diverse generation of geoscientists.

CERSE’s evaluation of ASPIRE is focused on assessing the development of the MWG model with regard to the intent of the project, the support and guidance provided to MWG leaders, and the change in mindset/behavior among scientists and community leaders engaged in the project. Evaluation activities include observation of team meetings and workshops, ongoing discussions with the team regarding “lessons learned” from the pilot phase, and surveys and interviews with MWG participants.

Transforming Engineering Culture to Advance Inclusion and Diversity (TECAID)

TECAID, funded by the NSF, supports five U.S. Mechanical Engineering (ME) departments to advance diversity and build inclusive environments for students, faculty, and staff. Over an 18-month period, TECAID has provided support for ME departments to create and sustain inclusive interactions and cultures that benefit all participants–in classrooms and labs, in student design groups, in faculty meetings and hallway interactions, and in the underlying department dynamics.

CERSE is using a longitudinal design to monitor the extent and nature of departmental transformation (in terms of awareness, understanding, and behavior) over time by examining the impact of TECAID implementation at departmental unit and department chairs/leader levels.

Advancement, Recruitment, and Retention of Women in Science (ARROWS) with Medical University of South Carolina

ARROWS was designed to foster a culture in the medical school that promotes greater participation and advancement of basic scientist women in a way that is both sustainable and serves as an easily portable and adoptable model for peer institutions. This project included the development and implementation of a package of practices and policies to specifically benefit women in basic science in a single medical school, as well as usher a cultural change across other medical schools nationwide by helping them to recruit, retain, and advance women scientists. CERSE served as the external evaluator for this grant. The process evaluation identified progress ARROWS made in the implementation of its activities to advance its two outcomes of improving recruitment as well as retention and advancement of women and minority basic scientists. To complement the summative assessment conducted by the internal evaluation team, the external evaluators examined the depth and pervasiveness of changes in basic science faculty recruitment, retention, promotion and tenure, and advancement of women into leadership positions within departments.

Center for Institutional Change, ADVANCE Institutional Transformation

With a National Science Foundation ADVANCE Institutional Transformation Award, the University of Washington created the Center for Institutional Change (CIC) to help transform the culture at the UW. The aim of ADVANCE was to increase the participation of women in science and engineering.

The UW ADVANCE project was designed to build upon existing strengths at the university while serving as the catalyst for institutional transformation. The CIC focused on the implementation of programs designed to eliminate existing barriers to women and to precipitate cultural change at both the departmental and the institutional level. The UW ADVANCE Center for Institutional Change had six major components:

• Leadership development for science, technology, engineering, and mathematics (STEM) chairs and deans.
• Department cultural change.
• Examination of UW policies for equity and policy transformation.
• Mentoring women in STEM for leadership.
• Visiting scholars who can contribute to the goals of the project.
• Transitional support for female faculty members in STEM departments.

CERSE served as the internal evaluator for the ADVANCE grant and managed a faculty and graduate student mentoring program for the CIC. CERSE also coordinated the external evaluation process led by the American Association for the Advancement of Science (AAAS). The NSF ADVANCE grant has ended, but CIC has continued a subset of the activities and CERSE has continued to serve as internal evaluator, albeit in a more limited capacity.

Center for Institutional Change, Chair Leadership Workshops

Leadership Excellence for Academic Diversity (LEAD) was a series of two-day national leadership workshops over a three-year period for department chairs, deans, and emerging leaders in STEM offered by the ADVANCE Center for Institutional Change at the University of Washington and funded by the National Science Foundation. The goals of LEAD included:

• Learn practical strategies and tools to improve department culture and climate.
• Explore unintended and unconscious biases that disproportionately impact women and under-represented faculty.
• Address important issues such as the recruitment and selection of faculty, the evaluation of faculty performance, conflict resolution, and leadership.
• Discuss case studies on faculty and departmental life.
• CERSE served as the evaluator of LEAD to measure the usefulness, relevance, and applicability of the workshop content.

Center for Institutional Change, Lead-it-Yourself! (LiY!)

From 2007 to 2009, the University of Washington’s ADVANCE Center for Institutional Change (UW ADVANCE) promoted the advancement of women and other underrepresented faculty in STEM departments across the nation by annually creating and delivering in-person Leadership Excellence for Academic Diversity (LEAD) workshops. To meet the national demand for LEAD workshops, UW ADVANCE created LEAD-it-Yourself! (LiY!), an online open-source toolkit of planning and instructional materials to enable institutions to run their own local or regional LEAD-inspired workshops. Several institutions were engaged in iterative evaluations instituting their own LEAD-inspired workshops. CERSE served as the assessment and evaluation consultant and conducted both process and outcomes evaluation of LiY!. CERSE tracked both implementation fidelity, monitoring whether LiY! workshops are delivered as the project team intends, and outcomes of LiY! workshops via impact of participants.

Center for Institutional Change, On-Ramps into Academia Workshops

Funded by NSF, On Ramps into Academia was a series of two-day workshops over a three-year period to provide practical tools and support to women who are interested in making the transition to academia. The goal of this project was to increase the pool of women faculty available to all universities by providing professional development to Ph.D.-level women in industry or research laboratories. It was offered by the ADVANCE Center for Institutional Change at the University of Washington.

CERSE served as the evaluator of On-Ramps. The assessment plan was strongly directed at outcomes evaluation and helped identify concrete best practices to encourage women into faculty pathways.

Council of Colleges of Arts & Sciences, ADVANCE Institutional Transformation

The Council of Colleges of Arts and Sciences (CCAS) was the focus of a collaborative effort funded through the National Science Foundation’s (NSF) ADVANCE program. The overarching aim of this grant was to cultivate academic leaders who are more knowledgeable about STEM gender equity issues, more able and motivated to address those issues, and thus better positioned to effect positive and transformational change in their own colleges and departments. CERSE used several assessment strategies including longitudinal survey analysis, document analysis, and observations to conduct summative, formative, and process evaluations for this $1.2 million project.

NSF CAREER Award Assessment of Educational Outreach Plans

Funded by the NSF CAREER program, Dr. Sapna Cheryan received a grant to fund her research and education activities related to stereotypes in computer science (CS). Part of her educational plan is to educate teachers, professors, advisors, and students about how stereotypes can limit the diversity in the CS field and how even minor adjustments to environments can have a positive impact on student perceptions of CS. CERSE evaluated progress towards Dr. Cheryan’s educational goals through workshop feedback and portfolio reviews.

Women Evolving Biological Sciences Symposia

Funded by NSF, Women Evolving Biological Science (WEBS) was a  three-day symposium aimed at addressing the retention of female scientists and issues related to the transition of women from early career stages to tenure track positions and leadership roles in academic and research settings that ran from 2007 through 2013. The goal of WEBS was to significantly increase the retention and promotion of women in academia in the biological sciences in order to create greater diversity in academic and scientific leadership.

As the evaluator, CERSE provided an outcomes evaluation and formative evaluation. Participants were tracked longitudinally to monitor changes in their career paths and the role of WEBS in providing guidance and support in their careers.

K-12 / MESA Evaluations

First Nations MESA

CERSE worked with Washington Math, Engineering, and Science Achievement (MESA) to conduct an evaluation for a capacity-building grant for First Nations MESA in order to: (1) assist First Nations MESA in developing a best practices model that can be exported to other Native communities across the state of Washington; (2) assess the need for culture-based science, technology, engineering, and mathematics (STEM) curriculum in the local schools; and (3) provide recommendations for local expansion. Evaluation activities included conducting a) pre-evaluation focus groups with stakeholders (completed Fall 2008), b) observations of selected First Nations MESA activities (2008-2009), and c) a second stakeholder analysis.

Strategic Education Centers (SEC)

Funded by the Gates Foundation, Strategic Education Centers aim to empower youth in Swaziland by providing health education, student exchange opportunities, and technology and academic training. In addition to the prevention of HIV/AIDS, which is widespread in Swaziland, SEC aims to give adolescent boys and girls the knowledge to pursue higher education in technological fields, and subsequent employment in thriving fields. Originally intended for girls, the program will soon be expanded to include boys.

CERSE, in conjunction with the UW Center for Health Research and Education (CHER), monitored and evaluated the success of each cohort of students participating in the SEC program. CERSE’s formative evaluation constituted an improvement of SEC’s previous monitoring and evaluation techniques, and allowed SEC to document the changes in student behaviors, attitudes, and knowledge of HIV/AIDS prevention.

Nanotechnology-related Evaluations

Establishing a Minor in Nanoscale and Molecular Biology (NSF NUE)

In 2009, the National Science Foundation Nanotechnology Undergraduate Education (NUE) program awarded the University of Washington a grant to support the project NUE NME Minor: Minor in Nanoscience and Molecular Engineering. The main purpose of this grant was to support development of an undergraduate minor in Nanoscience and Molecular Engineering. To measure the impact of lab modules and new course development on students, formative and summative (outcomes) evaluations were conducted to assess and improve the effectiveness of these components of the new NME minor curriculum. Strategies employed included course and lab module assessments and tracking of student interest and intent to minor in NME.

Nanotechnology Ph.D. IGERT Program

Funded by the National Science Foundation IGERT program, CERSE was contracted by the UW Center for Nanotechnology to study outcomes of students participating in its Ph.D. program. The goals of the evaluation were to:

• Monitor the progress of students affiliated with the Nanotechnology Program.
• Give students a confidential mode of providing feedback to the Nanotechnology Program.
• Address specific topics that were designated by the Nanotechnology Program.
• Assess the strengths and weaknesses of the program from a student perspective.

CERSE developed a tracking system that monitored student progress, recorded student perceptions about the field, and evaluated the role of an integrative educational program in the lives of the students. In addition, the questionnaire was designed to assess career preparation, participation in industry internships, interdisciplinary collaboration and student’s publication efforts.