Background
Wraparound programs frequently rely on outside experts for the training and technical assistance they need to support implementation. But, how satisfied are stakeholders with the training and TA they receive, and what is the longer-term impact on workforce skill development, fidelity, and youth/family outcomes? To support such evaluation, the National Wraparound Implementation Center (NWIC) has developed and tested (1) an assessment of training and TA impact called the Impact of Training and Technical Assistance (IOTTA), and (2) a tool to support skills-based coaching and evaluation of workforce progress called the Coaching Observation Measure for Effective Teams (COMET).
The Impact of Training & Technical Assistance (IOTTA)
The IOTTA measures stakeholder perceptions of immediate and long-term benefits in areas that research has linked to effective implementation and practice change. The tool consists of two surveys, each of which take approximately 10-15 minutes to complete.
The first survey is taken immediately post-training and asks participants to rate their satisfaction with various aspects of the training, the importance of the training goals, the anticipated level and type of impact that the training will have, their current mastery of the training information and skills, and their anticipated mastery of the information and skills at 6-8 weeks in the future.
The second survey is taken 8 weeks post-training and asks participants to rate their current mastery of the information and skills from the training, the amount and type of practice-level change that resulted from the training (i.e., how they interact with families, how they collaborate with colleagues, etc.), drivers and barriers to integrating the information and skills into their practice (i.e., didn't have enough time, families reacted positively, etc.), and the overall "worthwhile-ness" of the training.
For more information on using the IOTTA, please contact WERT.
The Coaching Observation Measure for Effective Teams (COMET)
The COMET is a practice level implementation tool utilized by supervisors and coaches to assess Wraparound practitioners' mastery of the skills necessary to ensure a high fidelity and quality Wraparound process. In partnership with the University of Maryland Institutes for Innovation and Implementation, and as part of the NWIC's suite of Wraparound process improvement tools, WERT uses data collected from the COMET to produce comprehensive and targeted reports based on the specific skills noted in the COMET. Scores are used by NWIC trainers to inform coaching and skill development in local staff. National scores are used as comparison on both the NWIC and local level. The COMET can only be utilized by NWIC collaborating states and entities.