Pilot Study of a Text-Based Outcomes Monitoring System for Wraparound

Information for Potential Study Sites

With funding from the National Institute for Mental Health, the National Wraparound Initiative (NWI) and University of Washington Wraparound Evaluation and Research Team (UW WERT) have partnered with 3C Institute to develop a text-based outcomes monitoring system for caregivers and youth participating in Wraparound.

SMART-Wrap (Short Message Assisted Responsive Treatment for Wraparound) aims to provide a low-burden, reliable approach for obtaining feedback from caregivers and youth enrolled in Wraparound and feed those data back to care coordinators, supervisors, and managers. Based on feedback from Wraparound experts, providers, parents, and youth, SMART-Wrap uses a secure, HIPAA-compliant interface to send brief (3—4 item) surveys focused on engagement, fidelity, satisfaction, and outcomes.

Pilot testing and user input to date shows that more than 80% of texts are opened and surveys completed and that the surveys take only 30-60 seconds to complete. 3CI is currently developing a back end to the SMART-Wrap system will translate survey responses into reports and a dashboard that can be used by care coordinators to monitor progress and engagement of families they work with, by supervisors to help direct supervision and coaching, and by evaluators and managers to inform accountability and CQI.

We Will Have a Second Round of SMART-Wrap Testing from February-June 2023.

To participate in the SMART-Wrap pilot, we are seeking 10 Wraparound provider organizations across the U.S. Each pilot site will agree to the following:

• Identify 3 Wraparound care coordinators (CC) to participate in the pilot, and up to 3 (but at least 1) supervisor and up to 3 (but at least 1) manager/administrator who oversees the CCs.
• Each CC will then obtain consent for one family to participate in the study.
  
  • Parents/caregivers (CGs) and youths (13 and over, with mobile devices), will use the SMART-Wrap prototype as part of their Wraparound process for two months.
  • CCs and managers/supervisors will use a web-based SMART-Wrap interface, including a dashboard and more detailed reports, to monitor engagement, satisfaction, fidelity, and outcomes and use the information for supervision and CQI.
  • CGs, youth, CCs, and managers/supervisors will complete a post-test online evaluation survey after 2 months.
  • Selected CGs, youth, CCs, and managers/supervisors will participate in focus groups.

Software Data Collection: For family participants, responses will be collected via SMS messaging. Rigorous security measures will be employed for data transfer and storage to protect privacy and confidentiality of client data. The research team will track use of the system by all users during the test period, including logins, amount of time spent, number/type of technical assistance requests, etc.

Rating Usability of the System: After the 2-month pilot period, all participants will evaluate their experience, specific to the components they used. For example, youth and caregivers will rate the SMS system and outreach features whereas CCs and supervisors will rate message deployment, tracking features, and online reporting functions[1]. Examples of ratings to be obtained include:

• SMART-Wrap usability e.g., easy to use, clear navigation, few technical problems;
• Acceptability e.g., satisfaction, motivation to use, worthwhile for intended purpose;
• Feasibility e.g., time efficient, practical to use, doable in everyday life/typical care;
• Value e.g., valuable for improving care/outcomes, motivation to use after pilot; and
• Compatibility of the SMART-Wrap product for integration with existing Wraparound systems (e.g., existing EMR/data system(s), existing evaluation methods).

Compensation for participants:

• During testing, each participating CG and youth will receive $60 in incentives for (1) testing the system by receiving and responding to brief surveys for 2 weeks ($30) and then (2) completing usability measures via online survey after pilot testing ($30).
• Each participating CC will receive $100 to help identify and refer families to the research team and to complete a summary survey of their experiences at the end of the two months.
• Each participating supervisor and manager will receive a small “thank you” gift card of $100 to help encourage the participation of their staff and complete a summary survey of their experiences at the end of the two months.
• Selected CGs/youths, CCs, and supervisors/managers who participate in focus groups at the end of testing will receive an additional $100.

Other benefits to pilot test organizations: Wraparound provider organizations and initiatives that participate in testing will also be afforded opportunities for more rapid onboarding to the system, and reduced fees for use of SMART-Wrap once it is available for widespread use.

For more information, please contact wrapeval@uw.edu and/or ebruns@uw.edu.

UW WERT and 3CI also plan to provide an overview of the system and how it works, present initial results, and details on how your Wraparound program or initiative can participate in a new round of testing. This webinar is tentatively scheduled for Tuesday January 17 at 2pm ET / 11am PT.