METHODS IN ACTION

Research-Practice Partnership to Support Social Emotional Learning in Schools

Contributor Biographies

Todd I. Herrenkohl, Ph.D., is Professor in the School of Social Work, University of Washington and Co-Director of the newly formed 3DL Partnership, an interdisciplinary center at the University of Washington focused on social, emotional, and academic learning in children and youth. Dr. Herrenkohl’s work focuses on the study and promotion of positive youth development and the amelioration of risk factors related to interpersonal violence. His funded projects and numerous publications examine various health-risk behaviors in children exposed to violence, resilience and protective factors that buffer against early risk exposure in children, and methods and approaches to promoting wellness by investing in the whole child.

Leslie Rupert Herrenkohl, Ph.D. is a Professor in the Learning Sciences and Human Development Program at the University of Washington and Co-Director of the 3DL Partnership. She is a developmental psychologist and learning scientist who brings a holistic, socio-cultural approach to examine how people learn concepts, develop skills, and shift their participation to become new people through their experiences. She considers how social and emotional dimensions intersect with the traditional intellectual and academic perspectives in learning sciences research. She has received funding from the National Science Foundation, the Spencer Foundation, and the James S. McDonnell Foundation. Her research has been featured in National Academy of Sciences summary volumes on science learning and 21^st century skills. Her latest book is How Students Come to Be, Know, and Do: A Case for a Broad View of Learning, published by Cambridge University Press.

John Benner, M.A. is a PhD. candidate in the College of Education at the University of Washington. He is a research assistant with the 3DL Partnership, an interdisciplinary center at the University of Washington focused on social, emotional and academic learning in youth. His research interests include social and emotional learning in informal settings and inclusive parent engagement practices in diverse urban schools. He brings over 20 years of experience as an early child educator and administrator and has worked on projects ranging from evaluating and selecting a school based SEL curriculum, grant writing for a STEM after school program, and a research-practice partnership facilitating the creation of a full-service community school.

Michelle A. Proulx, M.Ed., Social and Emotional Curriculum Developer in the Bellevue School District, is pursuing a Ph.D. in Learning Sciences at the University of Washington. Ms. Proulx’s research in social and emotional learning is influenced by her training as a developmental psychologist and her experiences working as a counselor and psychologist in a diverse urban school district. Ms. Proulx is particularly interested in the implementation and evaluation of preventative social and emotional instruction and the impact of explicit skill development on teacher practices and student outcomes. Central to her work are issues of measurement and the pursuit of efficient and authentic methods for assessing skill development among culturally diverse groups.

Naomi Calvo, Ph.D. is the Director of Research and Accountability for the Bellevue School District. She oversees the assessment department, is responsible for developing the district-wide accountability system, manages research and program evaluation, and provides analytic tools for schools and central office staff. Naomi brings over 15 years of experience in the field of educational research, policy, and analysis to the district. Prior to Bellevue, Naomi was a Manager for Education Resource Strategies in Massachusetts, specializing in school design practice and working with districts such as Atlanta, Baltimore, Boston, Charlotte, and Denver on school effectiveness and resource use.

Relevant Disciplines

Education, Psychology, Social Work, Learning Sciences, Human Development, Developmental Science

Academic Levels

Intermediate Undergraduate, Advanced Undergraduate, Graduate, Postgraduate

Methods Used

Application of Design-Based Implementation Research to Support a District-Wide Rollout of a Universal Social Emotional Learning Curriculum

Keywords

Social Emotional Learning, Evidence-Based Practices and Programs, Prevention, Youth Development, Schools and Schooling

Abstract

This case describes a body of work structured around a research-practice partnership that began in 2013 to support the implementation and evaluation of a universal social-emotional learning (SEL) curriculum in the Bellevue (Washington) School District (BSD). Research activities are organized around a Design-Based Implementation Research (DBIR) framework, which focuses on developing and testing strategies to enhance teaching and learning. DBIR is different from traditional evaluation research in that it takes an active role in the actual engineering of the implementation process by collecting and using data to drive refinements and innovation.

The partnership between the BSD and the University of Washington 3-Dimensional Learning (3DL) Partnership, a joint initiative of the University of Washington School of Social Work and UW College of Education, focuses on a district-wide implementation and evaluation of RULER, an evidence-based SEL program (Brackett, Rivers, Reyes, & Salovey, 2012; Rivers & Brackett, 2011). RULER is one of several evidence-based programs highlighted by CASEL, the Collaborative for Academic, Social, and Emotional Learning (http://www.casel.org/guide), a national resource on SEL. RULER focuses on helping teachers and children recognize, understand, label, express, and regulate emotions in the classroom with the goal of increasing student attentiveness, lessening conduct problems, and improving academic outcomes.

Our work with the Bellevue schools draws on the principles of design based implementation research (DBIR). DBIR is suited to developing and testing innovative strategies to enhance teaching and learning within school and agency contexts and is an extension of classroom-based design research that emphasizes collaborative and sustained engagement to enhance systematic inquiry and refinement of practices related to learning and instruction in schools (Coburn, Penuel, & Geil, 2013; Penuel, Fishman, Cheng, & Sabelli, 2011).

Learning Objectives

By the end of this case, students should:

Understand the idea and defining characteristics of a research-practice partnership
Understand and be able to explain the characteristics, goals, and objectives of Design-Based Implementation Research (DBIR), as well as to distinguish DBIR from other models of evaluation research.
Understand the goals and contributions of universal social emotional learning (SEL) curricula in schools.

Project Overview and Context: Building Capacity for a Research-Practice Partnership to Support Evidence-Based Programming in Social Emotional Learning

Research Challenges

For more than a year, researchers from the University of Washington 3DL Partnership have been working in collaboration with staff at BSD to support the district’s implementation and early-stage evaluation of RULER, a universal SEL curriculum described in more detail in the sections below. In 2013-2014, RULER was implemented in all 3-5 grade classrooms of the district. These classrooms contain approximately 4300 students and 173 teachers.

Our work on the implementation and evaluation of RULER within BSD has focused to this point on developing and piloting several measurement and data collection tools to examine teachers’ and administrators’ support of the new initiative and to study use of the various “anchor tools” that are part of the RULER approach. We eventually will study student outcomes (e.g., social emotional skills, academic achievement) to assist in determining whether the program has the impacts in Bellevue as it has in other districts. Throughout this initial phase of work, we have had ongoing contact with the program developers at the Yale Center for Emotional Intelligence (http://ei.yale.edu/) to align our efforts locally with those elsewhere in the country and internationally. Because our project is not currently funded by research grants, we started with very delimited tasks of high priority for the districts. Our goal moving forward is to generate grant funding to support an expansion of our work and to sustain our partnership over several additional years.

From the beginning, we have thought of this effort as a “research-practice partnership,” which emphasizes a highly collaborative, sustained engagement and shared ownership of project goals, processes, and outcomes on the part of researchers and practitioners working closely together (Coburn et al., 2013). In keeping with the model, we continue to look at ways to have each member of the project team of district staff, university faculty, and students actively contributing to the work by helping to plan and execute the research goals, and by sharing their expertise to help stimulate new ideas and refine existing strategies.

In a white paper prepared for the W.T. Grant Foundation, Coburn et al. (2013) define several different kinds of research-practice partnerships: (1) Research Alliances; (2) Design Research; (3) Networked Improvement Communities. Our partnership falls primarily into the second category, Design Research, where the goal is to “build and study solutions at the same time in real world contexts” (Coburn et al., p.8). Although our partnership did not design the RULER curriculum, we are working with the program developers at Yale to study and support the implementation of the program “at scale” in the Bellevue district. Our work as a partnership reflects a commitment to advancing SEL theory and to extending the district’s goal of fully integrating SEL into its core programming goals. Research questions discussed in the case relate directly to those goals.

Our project began by engaging in the usual tasks of signing letters of agreement and submitting human subjects clearances. From that point, we started the process of negotiating what it means to “be” a team, which, in our case, included deciding on roles and responsibilities, developing and revising research questions, and constructing a project timeline. In addition, the team has spent many hours brainstorming ideas for a next phase of our work together. In the project so far, we have prioritized data collection strategies that lend themselves to ongoing professional development with teachers and staff within the district, although it is understood that attention must also be paid to program impacts that will ultimately help the district determine whether its investment in RULER is financially and programmatically sound.

Social-Emotional Learning and RULER

Social-emotional learning (SEL) is increasingly recognized as a critical component of academic and life success (Brackett, Rivers, & Salovey, 2011; Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011; Lee & Shute, 2010; Payton et al., 2008). There are many examples of promising SEL programs for schools, several of which are included on the Collaborative for Academic, Social, and Emotional Learning (CASEL) website (http://www.casel.org/). CASEL is a national organization that focuses on the advancement of SEL competencies by conducting and disseminating research and supporting practice and policy change efforts at the national, state, and local levels. Skills taught in SEL programs support emotion regulation, social-awareness, conflict resolution, and responsible decision making (Durlak et al., 2011). Emphasis is also placed on community-building and school climate, which influence interactions among students and teachers (Durlak et al., 2011). Reviews of research have shown that even in programs where SEL is but one of several core content areas of curricular innovation, measurable reductions in student conduct problems can be achieved (http://www.casel.org/). Increasingly, studies of universal, school-wide SEL programs, like RULER, show, additionally, that students’ academic performance can also be improved, particularly when programs are delivered appropriately (Durlak et al., 2011). In fact, Durlak et al.’s (2011) meta-analytic review of 213 school-based SEL programs found consistent gains in students’ academic performance, with an average effect size above .20 across intervention formats.

In March 2010, Washington State passed Substitute House Bill 2801 to amend state law governing harassment, intimidation, and bullying (HIB) in schools. In response to state policy requirements, BSD implemented a universal bullying prevention curriculum in Fall, 2011. After the program was implemented, teachers and school leaders were surveyed to gather feedback on the curriculum. Survey responses suggested that teachers in grades-3-5 perceived the program as too heavily focused on problem behaviors and not enough on skill development for social competency and positive youth development.

In response, BSD convened a committee of parents, administrators, teachers, school counselors/psychologists, instructional technology leaders, and the district curriculum developer to review and make recommendations on alternative approaches. The committee reviewed the CASEL Guide for Effective Social and Emotional Learning Programs (http://www.casel.org/guide), which provides a rating framework for evaluating and selecting SEL programs. From its review, the committee selected RULER, a CASEL recommended program the committee deemed well aligned with the needs of the district.

RULER was initially piloted in 15 of the district’s 16 elementary schools. In the pilot, 27 classrooms were matched for student demographics and grade level. Teachers were randomly assigned to teach RULER or to use Second Step, another SEL program, as an alternative. The classrooms represented a wide cross-section of students that represent different populations served by Bellevue schools (e.g. Title 1, highly gifted, special education, English language learners). Data collected during the pilot project included classroom observations, focus groups, surveys, and student assessments. From the data, the committee recommended the full adoption of RULER for all grade 3-5 classrooms in the district.

RULER, a universal SEL program, was developed by researchers at the Yale Center for Emotional Intelligence (Yale Center for Emotional Intelligence, 2013) (http://therulerapproach.org/index.php/programs-and-services/main/). It is based on emotional intelligence theory, which proposes that the skilled processing of emotions enhances students’ cognitive and social functioning (Denham, 1998; Eisenberg, 2000; Izard, 2002; Mayer & Salovey, 1997; Nowicki & Dukein, 2001; Saarni, 1999; Salovey & Mayer, 1990). The program focuses on helping children recognize, understand, label, express, and regulate emotions by making SEL a central part of classroom routines and core instruction.

RULER has been used in thousands classrooms in schools across the nation. Feedback from students, teachers, parents, and administrators has been, according to the program developers, overwhelmingly positive (Brackett, personal communication). Teachers using RULER report that the program is easy to integrate into classroom practices and routines and most believe it benefits students socially and academically. According to one study of RULER, 5^th and 6^th grade students (n=155) who participated in the program had higher grades in English Language Arts and better work habits than did those in comparison classrooms. Students in RULER classrooms also received higher teacher ratings of social and emotional competence, as evidenced by stronger leadership, social, and study skills, compared to students in comparison classrooms (Brackett et al., 2012). A randomized control trial of 62 schools also found evidence of a more welcoming (warm) environment after RULER was implemented. Teachers and students in RULER schools also were more “connected” and there was less bullying among students in these schools (Brackett & Rivers, in press).

Limitations and Complexities in Research on SEL Programs

While a growing body of research shows that RULER and other universal SEL programs can improve outcomes for students who use them, there is considerably more that needs to be done to strengthen theory and research and to show where further refinement of specific program goals and methods is needed (Durlak et al., 2011). Additionally, there are few, if any, standardized approaches to assessing student outcomes and it is yet unknown how student characteristics (including race/ethnicity, cultural background, socioeconomic status, gender) influence program outcomes. Further, the field has lacked consistent measurement and data collection routines that can allow teachers and district administrators to reflect on, revise, and readily use data to course-correct and refine programs so that resources are used efficiently. Family engagement, including the degree to which parents understand, use, and reinforce SEL norms and practices that are taught in school, is also an important, yet understudied topic.

Design-Based Implementation Research and SEL Research

Our research is informed by the concepts and ideas of Design-Based Implementation Research (DBIR). DBIR is not so much a specific methodology, like ethnography or survey research, as it is an approach to understanding and guiding the implementation of a program or policy in a particular setting. Design Based Research (DBR), a closely related framework, is concerned with designing new artifacts such as new programs or new learning environments in a collaborative and iterative process between practitioners and researchers (Design-Based Research Collective, 2003). While emerging from the same research tradition and retaining some of the same principles of DBR, DBIR focuses somewhat more so on generating new implementation strategies for existing interventions or to leverage local contextual realities and maximize the effectiveness of a particular program or intervention (Penuel & Fishman, 2012). Like DBR, DBIR is also an iterative approach to developing innovations (Penuel, et al 2011). Rather than focusing on an intervention at one level (i.e., individual classroom, or grade level in a school district), DBIR designs and tests interventions across levels and settings to investigate and improve the implementation of interventions (Penuel & Fishman, 2012). It is different than traditional evaluation research in that it takes an active role in the actual engineering of the implementation process by collecting and using data to drive refinements and innovation. While traditional efficacy or effectiveness studies attempt to measure program impacts in highly controlled settings in order to isolate and control for contextual influences, DBIR incorporates and emphasizes those contextual influences as part of the implementation and evaluation processes.

In DBIR, the relationship between researcher and participant takes on a form that also differs from a more traditional model. Rather than view practitioners as “subjects” in a study, they are viewed as research “partners” whose knowledge and expertise are valued and deemed essential to planning and executing a research project. In our case, this approach has helped us to look closely at the relationships between the content of an evidence-based SEL program and the context factors (e.g., setting, professional roles, supports and barriers) that facilitate and disrupt its implementation. Elias, Zins, Graczyk, & Weissburg, (2003) note that efforts to implement SEL programs are contingent upon many factors that are so highly context-dependent that a flexible, theoretically informed action research approach to implementation is critical.

Research Findings

Under a DBIR umbrella, we have accomplished a number of tasks in our work to date. For example, we administered surveys to principals and instructional leaders and to teachers involved in RULER trainings to assess how they perceived the curriculum and whether they were generally supportive of its being adopted by the district. Results of both surveys showed broad support for the program. Over three quarters (76%) of the school leaders we surveyed stated that they favored the district’s adoption decision. During this first phase of work, we also began the task of researching and vetting previously validated measures to study implementation and outcomes. Along the way, we have learned about challenges of implementing RULER. Specifically, we have found that how RULER is being implemented varies from school to school and sometimes across classrooms within a school, due to teacher expertise and comfort with the curriculum, as well as characteristics of the classroom. Higher-level challenges appear to be centering on the organizational capacity of schools and the district to provide adequate training and support for teachers so they understand how to integrate RULER into their teaching. The challenges we have seen are especially difficult in the context of other district-wide initiatives involving implementing common core standards and new academic assessment tools, and new academic grading and management software. Our initial understanding of these challenges comes from surveys we conducted earlier in the school year.

One emerging strategy to help address at least a couple of these challenges is to create lunch-time professional development conversations about RULER, where teachers can talk with trainers and one another about their experiences with the curriculum. In addition to supporting teacher learning through informal peer-interaction, these sessions also provide opportunities for our team to gather data relevant to our research questions.

A facet of our work going forward is to better understand the role of parent involvement in school-based social and emotional learning. A first step to this process is to better understand how parents in the district understand SEL and perceive the new initiative. We are also interested to learn if parents feel adequately informed about the initiative and if there are ways that they would like to be more involved. The next phases of our work will be guided by several research questions that relate to several of the issues and challenges summarized in the paragraphs above. These include:

Are the units and lessons of RULER implemented with fidelity across schools in the district?

Which strategies are best suited to supporting district and school level staff to collect and use data to improve program quality?

What are the best measures to assess the impacts of RULER in the local context and what do these reveal about student outcomes?

Do outcomes vary across schools within the district that house different groups of students and programs served by BSD (Title 1, highly gifted, special education, English language learners)?

How do parents perceive the initiative and do they feel adequately informed? Are there ways they can assist parents to use of SEL practices outside the school context?

Data to study these questions are or will be collected using a variety of methods and approaches, including surveys, interviews, focus groups, and classroom observations. The plan is also to use “walkthrough checklists” to guide classroom observations of RULER implementation (e.g., assess whether the tools and exercises of the program are being used consistently and whether students and teachers appear to respond uniformly).

We also plan to study program impacts on student outcomes by assessing SEL competencies, behaviors, and academic outcomes using standardized test scores, disciplinary reports, and attendance records. Data collection may also include an assessment of students’ emotional intelligence and students’ perceptions of teaching effectiveness, student engagement, and school climate.

Conclusions and Implications

As is reflected in the various sections of this case, our intention is to step outside traditional ways of thinking about implementation and program evaluation research —and to do so in a way that values the expertise and contributions of the various partners involved. Our intent is to use the work we do collaboratively around RULER, as one exemplar of evidence-based programs in SEL, to raise the profile of SEL in school contexts and to look at ways of integrating SEL content and academic programs. We go about this work mindful of what Gutierrez and Penuel (2014) argue is a new conceptualization of scientific rigor. To them, rigor involves the “sustained, direct, and systematic documentation of what takes place inside programs” so that findings are not so much directly replicable or “generalizable” to other contexts as they are highly relevant to practice and “transferrable” to other settings in which similar work is being done. Combining practice and research expertise in the way we have in this partnership maximizes opportunities to learn from one another and to build capacity for data-driven decision-making around the most pressing questions and needs at the time.

Discussion Questions

What is a research-practice partnership?

What are the challenges that can be encountered when engaging in a research-practice partnership?

How is the researcher’s relationship to a study different in DBIR than in more other forms of implementation and evaluation research?

How does the notion of scientific “rigor” mentioned in the conclusion differ from that of research operating from another paradigm?

References

Brackett, M. A., & Rivers, S. E. (in press). Transforming students’ lives with social and emotional learning Handbook of Emotions in Education.

Brackett, M. A., Rivers, S. E., & Salovey, P. (2011). Emotional intelligence: Implications for personal, social, academic, and workplace success. Social and Personality Psychology Compass, 5(1), 88–103.

Committee for Children. (2013). Second Step | Skills for Social and Academic Success. Retrieved December 13, 2013, from http://www.cfchildren.org/second-step.aspx

Denham, S. A. (1998). Emotional development in young children. New York: Guilford Press.

Design-Based Research Collective. (2003). Design-Based Research: An Emerging Paradigm for Educational Inquiry. Educational Researcher, 32(1), 5–8. doi:10.3102/0013189X032001005

Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82(1), 405–432.

Eisenberg, N. (2000). Dispositional emotionality and regulation: Their role in predicting quality of social functioning. Journal of Personality and Social Psychology, 78, 136-157.

Elias, M. J., Zins, J. E., Graczyk, P. A., & Weissburg, R. P. (2003). Implementation, Sustainability, and Scaling Up of Social-Emotional and Academic Innovations in Public Schools. School Psychology Review, 32(3), 303–319.

Izard, C. E. (2002). Emotion processes in normal and abnormal development and preventive intervention. Development & Psychopathology, 14(4), 761-787.

Lee, J., & Shute, V. J. (2010). Personal and social-contextual factors in K-12 academic performance: An integrative perspective on student learning. Educational Psychologist, 45(3), 185–202.

Mayer, J. D., & Salovey, P. (1997). What is emotional intelligence? In P. Salovey & D. J. Sluyter (Eds.), Emotional development and emotional intelligence: Educational implications. New York, NY. p. 3-34.: Basic Books.

Nowicki, S., & Dukein, M. P. (2001). Nonverbal receptivity: The Diagnostic Analysis of Nonverbal Accuracy (DANVA). In J. A. Hall & F. J. Bernieri (Eds.), Interpersonal sensitivity: Theory and measurement (pp. 183-198). Mahwah, N.J.: Lawrence Erlbaum Associates

Payton, J., Weissberg, R. P., Durlak, J. A., Dymnicki, A. B., Taylor, R. D., Schellinger, K. B., & Pachan, M. (2008). The positive impact of social and emotional learning for kindergarten to eighth-grade students. Chicago, IL: Collaborative for Academic, Social, and Emotional Learning.

Penuel, W. R., & Fishman, B. J. (2012). Large-scale science education intervention research we can use. Journal of Research in Science Teaching, 49(3), 281–304. doi:10.1002/tea.21001

Rivers, S. E., & Brackett, M. A. (2011). Achieving standards in the English Language Arts (and more) using The RULER Approach to social and emotional learning. Reading & Writing Quarterly, 27, 75–100.

Saarni, C. (1999). The development of emotional competence. New York: Guilford Press.

Salovey, P., & Mayer, J. D. (1990). Emotional intelligence. Imagination, Cognition & Personality, 9(3), 185-211.

Yale Center for Emotional Intelligence. (2013). RULER Overview. Yale Center for Emotional Intelligence. Retrieved December 13, 2013, from http://ei.yale.edu/ruler/ruler-overview/