The Measurement Guidance Toolkit was developed by the Office of Juvenile Justice and Delinquency Prevention’s National Mentoring Resource Center (NMRC), through the work of the NMRC Research Board. A full list of contributors to the Toolkit appears below.
Youth mentoring programs are increasingly looking to make informed decisions about strengthening their programs and convincingly demonstrate their impact to stakeholders. These needs place a premium on the availability of practical guidance for how to evaluate the impact mentoring relationships are having on youth within their unique contexts. Most programs recognize that valid and reliable measurement tools are an essential component of any high-quality evaluation.
But program staff are often challenged to find accessible surveys, scales, and other data collection instruments they can use in their work. It can be daunting to wade through the many instruments available for measuring even a single outcome and select the one that is “best” (i.e., provides a brief, yet accurate assessment of the outcome).
The goal of this Toolkit is to provide well-vetted recommendations for instruments that are suitable for use by mentoring programs and their evaluators. This should allow programs to more accurately capture their impacts, thus setting the stage for both improvements in program quality over time and, in the case of programs that show promise for effectiveness, generating a stronger and more targeted argument for programs. A further goal of the Toolkit is to create greater consistency in how youth outcomes are measured across programs. In this way, more meaningful comparisons can be made across programs. For example, does one program really improve school connectedness to a greater extent than others or is the difference due simply to how this outcome is being measured? Likewise, increased uniformity in how outcomes are assessed will provide researchers with the ability to aggregate evidence on impact across programs using common outcome metrics, thus offering an unprecedented opportunity to track and document trends in effectiveness for the youth mentoring field as a whole.
Toolkit Structure and Instrument Profiles
The Toolkit is built around six domains of youth outcomes that the NMRC Research Board identified as the most common areas in which mentoring programs could expect to have an impact: mental and emotional health, social emotional skills, healthy and prosocial behavior, problem behavior, interpersonal relationships, and academics. The Toolkit also includes recommendations for assessing different types of risk and protective factors, which can be used to capture the challenges and needs that youth bring to mentoring programs, providing valuable context for understanding other outcomes. In 2018, we added measures of relationship quality and characteristics, as assessing these factors can help ensure that youth are receiving appropriate support and that their mentoring relationships are progressing as intended and are likely to result in positive outcomes in relevant domains. Additional domains, and outcomes within domains, will be added to the Toolkit over time.
Within each domain, the Research Board identified several specific outcomes of likely interest to mentoring programs. For example, Aggression, Delinquent Behavior, and School Misbehavior are addressed as outcomes within the domain of Problem Behavior. For each outcome, the Research Board selected a recommended measure. A brief profile of the measure explains exactly what it is designed to measure, why it is recommended for use by mentoring programs, practical guidance for using the tool in real-life contexts, and information about how to access the measure. Alternative measures are also described that may be more suitable for a given mentoring program depending on its goals, the age range of the youth it serves, or other considerations.
For the Mentoring Relationship Quality and Characteristics measures, the Research Board engaged in a similar process of identifying measures that had been used previously in mentoring evaluations and had validity evidence in that context. The Board conducted a thorough review of 17 multi-faceted measures (several of which were represented by both youth- and mentor-report measures); 13 unidimensional measures; and 9 more tailored measures that had been used in youth mentoring studies, reviewing information on the measure’s characteristics, strength (e.g., how well the scale(s) holds together, measures what it intends to measure, and relates to other youth outcomes) and usage. The search yielded mostly measures that capture internal match quality, particularly the relational aspect of this component. Most reviewed measures were also from the youth’s perspective, as opposed to mentor-reported measures. Thus, as part of the selection process, the Board also considered which aspects of relationship quality were covered and strived to include at least a handful of mentor-reported measures. You can learn more about the selection criteria for these measures in the introduction to that section of the Toolkit.
You can use the navigation to the left to toggle through the various domains and measures within each.
The Toolkit also includes a section that features tips for administering instruments as well as advice for incorporating the recommended measures effectively into broader evaluation designs.
Toolkit Contributors Research Board members:
- Ed Bowers, Ph.D. – Clemson University
- David DuBois, Ph.D. – University of Illinois at Chicago (Chair)
- Chris Elledge, Ph.D. – University of Tennessee at Knoxville
- Stephanie Hawkins, Ph.D. – RTI International
- Carla Herrera, Ph.D. – Independent Researcher (Associate Chair)
- Enrique Neblett, Ph.D. – University of North Carolina at Chapel Hill
- Naida Silverthorn, Ph.D. – University of Illinois at Chicago (Senior Research Specialist)
- Fasika Alem – University of Illinois at Chicago (Post-Doctoral Research Associate)
Additional Toolkit content and technical support by:
- Michael Garringer – MENTOR: The National Mentoring Partnership
- Mandy Howard – FirstPic, Inc.