Toolkit

Toolkit

Brief Description: This is a measure of juvenile offending based on information gathered from juvenile justice agencies on arrests, offenses, and sentencing.

Rationale: Evaluators frequently rely on youth self-reports of delinquent behavior and arrests. However, some youth under-report delinquent behavior or misreport the nature of their offense (Thornberry & Khron, 2000), perhaps due to consequences that could arise from their disclosure or a failure to recall these behaviors. In addition, some groups of youth tend to over-report their arrests, whereas others (i.e., those with more arrests) tend to under-report arrests (Kirk, 2006; Krohn, Lizotte, Phillips, Thornberry, & Bell, 2013). For this reason, collecting records of offending is desirable when feasible.  

Cautions: Be sure to set aside adequate time and staff resources to collect records on juvenile offending. Accessing juvenile justice records often requires extensive planning and negotiation with the state (e.g., Office of Juvenile Justice, Department of Corrections) and/or reporting agency or agencies (e.g., juvenile court, family court), particularly around issues of data privacy and confidentiality. The juvenile justice system is complex in structure and process, including jurisdictional variations and ongoing reforms and other changes over time. See flow chart here for an overview. It is also important to note that an arrest should not be interpreted as an indication that a youth necessarily committed the offense with which they were charged and that the behavior of certain groups is more likely to result in an arrest, charge, and conviction than the behavior of individuals from other groups (Developmental Services Group, 2014; Huizinga et al. 2007, Kakade et al. 2012). 

Access and permissions: Access to juvenile offending records typically involves strict confidentiality conditions. Organizations wanting access to these records typically will be required to complete a formal written application, part of which will be detailing the specific information being requested. If the request is approved, it should be expected that written parent permission and potentially also youth consent or assent (depending on the age of the youth) will be required prior to release of any data that identifies individual youth (i.e., is not deidentified). Consideration also should be given to collecting the youth’s social security number with parent permission because agencies granting access to juvenile offending records often require it to facilitate identification of youth within available records. Also, consider budgeting funds to reimburse time for agency staff to gather these data. 

What to Collect: A formatted data collection guide of the types of data that programs might consider requesting directly from an agency can be found here. 

How to Collect:

Sources: It is possible that the agency providing juvenile justice records could vary across different jurisdictions or that these records reside in multiple agencies in the same jurisdiction. Programs that serve youth in more than one geographical area should anticipate potentially needing to work with a range of differing types of agencies (e.g., office of juvenile justice, juvenile court, Department of Corrections) or with agencies that have different names in different jurisdictions but serve the same function (e.g., juvenile court, family court), and each of these agencies may have slightly different requirements for access. Establishing contact with administrators at the agencies from which there is a plan to collect juvenile justice records data prior to an evaluation is critical. This will clarify the documentation needed (e.g., it may be necessary to develop permission/consent forms that include very specific information) and potential barriers to collecting the data. Depending on the nature of the evaluation and the jurisdiction’s policies, the agency may agree to provide only “deidentified” data (i.e., data that does not include youth names or other identifying information). If so, it is advisable in the data request to attach information to each youth’s name, such as basic demographics (gender or race/ethnicity) or program participation status so that the data once obtained (with this information attached to each line of data, but with the youth’s name removed) will allow you to use this information in analyses. Care must be taken, however, to ensure this type of attached information does not allow a youth to be inadvertently identified; a general rule of thumb is to ensure that the data once obtained do not include subgroups (e.g., male Native American youth) of fewer than 10 youth.

Additional Considerations: If there is  interest in assessing change over time in outcomes, the time period for which data are requested should be specified accordingly along with a request for information (e.g., dates of arrest) that will allow for the desired type of analysis (e.g., before and after program participation). It is also advisable to consider requesting data not only for the entire period of a youth’s program participation, but also a period of time after participation has ended as this information can be helpful for evaluating possible longer-term effects of program involvement. Care also should be taken to account for possible variation in the time period for which data should be requested for different youth, such as in cases in which youth are enrolling in a program at different points in time. If possible, consideration also should be given to collecting similar records for a comparable group of youth not participating in the mentoring program. These data can be used to compare outcomes for program and non-program participants, which is a more robust evaluation design than simply looking at changes over the course of program involvement for program participants (for further discussion of evaluation design considerations, see the Evaluation Guidance and Resources section of this Toolkit).

How to Analyze:

Format: It is important to work closely with reporting agencies to interpret differences in juvenile justice terminology and offense classifications across agencies. It may also be necessary to review key terms (e.g., informed by an interpretation guide if available) prior to collecting or “coding” this information (i.e., translating values or categories into numbers that can be analyzed). A list of some of these terms can be found here. Additionally, records data on arrests, juvenile offenses, and sentencing may be presented in many different formats across agencies. For example, some agencies may provide an Excel table with columns containing each piece of information requested, whereas others may provide photocopied case files with the information in a narrative form across multiple documents, in which case, it will be necessary to read through these descriptions to pull out the information needed.

Scoring: How the collected information is coded or “scored” will depend on your specific aims. For example, a program might be interested in a broad count of arrests. In this case, it may be sufficient to simply add up the total number of arrests reported. Similarly, if there is interest only in whether the youth was arrested, a simple “yes/no” indicator in which a score of 1 reflects at least one arrest and a score of 0 reflects no arrest can be used. However, not all arrests involve offenses of equal severity. Distinctions that may be useful to consider include whether an offense involves violence and whether an offense is person-related (e.g., robbery) or property-related (e.g., vandalism). As detailed in the data collection guide (here), for example, the Uniform Crime Reporting Program of the Federal Bureau of Investigation distinguishes between a set of "index" crimes (e.g., aggravated assault, burglary) and other types of offenses (e.g., disorderly conduct), the former being further subdivided into violent and property crimes, respectively. Considering such distinctions (e.g., total number of violence-related arrests or an indication of whether a youth was arrested for any person-related offense) may provide a more nuanced understanding of a program’s effects or the needs of the youth that it serves.

How to interpret findings: When using frequency counts or “presence or absence” of arrests or offenses in evaluating a program’s possible effects, results for a group of youth (e.g., those participating in a program) can be expressed as the average number of arrests per youth or as the percentage of youth with one or more arrests in that group within a given time period.  Programs interested in gauging their effectiveness will generally be interested in seeing declines in these numbers during or after youth’s participation in the program. However, such change could occur for reasons unrelated to program participation (e.g., initiation of local reforms such as pre-arrest diversion programs); conversely, an absence of change, and even an increase in arrests might be observed due to non-program factors such as a developmental trend toward greater involvement in delinquent behavior with age. In the absence of an appropriate comparison group, findings should never be interpreted as being indicative of program effectiveness or lack thereof (for further discussion, see the Evaluation Guidance and Resources section of this Toolkit).

Alternatives: Although youth self-reports of arrests may not be as accurate or precise as data collected from official records, youth often recall arrest information with good accuracy, particularly when arrests are few in number (Thornberry & Krohn, 2000). In addition, although some youth may be reluctant to report delinquent behavior, official arrest records are also limited in that they do not capture delinquent acts for which youth are not caught and arrested. For this reason, youth self-reports of arrests and/or delinquent activity could be a good alternative. 


Citations:

Development Services Group, Inc. (2014). Disproportionate minority contact. Washington, DC: Office of Juvenile Justice and Delinquency Prevention. Prepared by Development Services Group, Inc., under cooperative agreement number 2013–JF–FX–K002. Points of view or opinions expressed in this document are those of the author and do not necessarily represent the official position or policies of OJJDP or the U.S. Department of Justice. Available at: https://www.ojjdp.gov/mpg/litreviews/Disproportionate_Minority_contact.pdf

Huizinga, D, Thornberry, T. P., Knight, K. E., Lovegrove, R. L., Hill, K., Farrington, D. P. (2007). Disproportionate Minority Contact in the Juvenile Justice System: A Study of Differential Minority Arrest/Referral to Court in Three Cities. Washington, DC: Office of Juvenile Justice and Delinquency Prevention.

Kakade, M., Duarte, C. S., Liu, X., Fuller, C. J., Drucker, E., Hoven, C. W., . . . Wu, P. (2012). Adolescent substance use and other illegal behaviors and racial disparities in criminal justice system involvement: Findings from a US national survey. American Journal of Public Health, 102, 1307–1310. https://doi.org/10.2105/AJPH.2012.300699

Kirk, D. S. (2006). Examining the divergence across self-report and official data sources of inferences about the adolescent life-course of crime. Journal of Quantitative Criminology, 22, 107-129. https://doi.org/10.1007/s10940-006-9004-0

Krohn, M. D., Lizotte, A. J., Phillips, M. D., Thornberry, T. P., & Bell, K. A. (2013). Explaining systematic bias in self-reported measures: Factors that affect the under- and over-reporting of self-reported arrests. Justice Quarterly, 30, 501-528. https://doi.org/10.1080/07418825.2011.606226

Thornberry, T. P., & Krohn, M. D. (2000). The self-report method of measuring delinquency and crime. In D. Duffee (Ed.), Criminal justice 2000 (pp. 33-84). Washington, DC; U.S. Department of Justice, the National Institute of Justice. National Institute of Justice.

Brief Description: This measure consists of school records of student discipline. These include both descriptions of the disciplinary incident or event (e.g., fighting, disruptive behavior) and corrective action taken by the school (e.g., detention, in-school suspension, out-of-school suspension, expulsions).

Rationale: Evaluators and researchers frequently rely on students to self-report school misbehavior and discipline. However, the accuracy of such reports can be influenced by a number of factors, including student age, cognitive ability, and actual behavior. For these reasons, collecting records of school discipline is desirable where feasible.  

Cautions: When planning to collect school disciplinary records be sure to set aside adequate time and staff resources. Accessing school records can be complex, particularly when working with multiple schools and grade levels. Schools and districts may use different terminology and classifications of student behavior. Additionally, the frequency of office disciplinary referrals and formal disciplinary actions may differ across school districts or even within schools, which is critical to consider when using disciplinary data for program evaluation or research purposes. Interpreting school disciplinary records requires the ability to collate and analyze data. Programs without on-staff expertise may want to work with an external program evaluator.

Access and permissions: When working with an outside agency (e.g., a school or district) to collect school records, access to their data typically involves strict confidentiality conditions (see FERPA guidelines). You may be required to provide written parent permission with very specific information included (that can vary across schools or districts). Standard permission/consent language can be incorporated into program enrollment forms (see sample). Also, consider budgeting funds to reimburse time for school officials to gather needed data. More extensive guides on federal privacy guidelines and how to establish data sharing partnerships with school districts can be found in the Evaluation Guidance and Resources section of this Toolkit.

What to Collect: A formatted data collection guide of variables you might consider requesting directly from schools can be found here. If you are collecting data directly from parents/youth, this guide can be used to help structure your database for storage and analysis. 

How to Collect:

Sources: Programs can request student disciplinary records directly from schools or school district offices, in which case, a formal MOU will typically be required. Schools or districts may agree only to provide “deidentified” data (i.e., data that do not include student names or other identifying information).  If so, it is advisable in the data request to attach information to each youth’s name, such as basic demographics (gender or race/ethnicity) or program participation status so that the data once obtained (with this information attached to each line of data, but with the youth’s name removed) will allow you to use this information in analyses. Care must be taken, however, to ensure this type of attached information does not allow a youth to be inadvertently identified; a general rule of thumb is to ensure that the data once obtained do not include subgroups (e.g., male Native American youth) of fewer than 10 youth.

Additional Considerations: If you are interested in assessing changes over time, make sure to collect a "baseline" in the period before the student began program involvement. In addition, request disciplinary records for the entire time period of the student’s program participation and after the end of the program, as these data can help to assess longer-term program effects. And be sure to align data requests with the specific timeframe of program enrollment for each student (e.g., one student may need disciplinary records starting in the spring quarter of one school year through the fall of the next school year, whereas another student may have a very different time frame of participation). If possible, you may also want to consider collecting school discipline data for a comparable group of students not participating in the mentoring program. These data can be used to compare outcomes for program and non-program participants, which is a more robust evaluation design than simply looking at changes over the course of program involvement for program participants.

How to Analyze:

Scoring: It is important to work closely with school or district officials to interpret scoring differences across years, grade levels, and types of disciplinary incidents. If available, an annual district interpretation guide can be useful.

Disciplinary Action: Schools commonly report the number and type of formally recorded decisions that result from student behavior within a given term (quarter or semester). Here, it is helpful to distinguish between exclusionary or non-exclusionary actions (i.e., those that exclude or do not exclude the student from class or school). When comparing disciplinary action across groups, such as the groups receiving or not receiving your mentoring program, you may want to analyze differences in: (1) the percentage of students in each group receiving one or more disciplinary actions; (2) the percentage receiving each type of action; and (3) the average number of disciplinary actions received. Other disciplinary consequences, such as reports made to law enforcement, are important to include as well.

Type of Behavior: Student behaviors should be analyzed carefully, as they vary in severity and are associated in different ways with student risk. For example, class disruption leading to an in-school suspension is very different from aggravated assault that leads to an out-of-school suspension.

Subgroup Differences: Research has shown that some racial and ethnic minority groups, males, low-achieving students, special education students, and students from low socioeconomic backgrounds experience disproportionate rates of school suspensions and expulsions, which is likely the result of decisions made by school administrators rather than actual differences in student behavior. Therefore, programs may want to analyze disciplinary actions separately for relevant subgroups and include findings from a similar group of youth that does not receive program services. 

How to interpret findings: A reduction in the number of disciplinary actions or in severity of behavior indicates better student behavior at school. Keep in mind, however, that serious misbehavior and more severe exclusionary disciplinary actions may occur infrequently, especially at lower grade levels. Moreover, disciplinary rates vary widely across schools, which could be due to a variety of factors in addition to student behavior, including referral processes and teacher tolerance for disruptive behavior. Detailed district-level data on discipline is available through the U.S. Department of Education Office for Civil Rights. 

Alternatives: School disciplinary records are limited to incidences where a formal office referral was made. As such, they provide a good indicator of school discipline but may miss many instances of school misbehavior or problem behavior more generally. Office disciplinary referrals often focus on serious infractions and may undercount less serious behavior, such as classroom disruptions. Therefore, programs that want to assess a wider range of youth behavior may want to collect self-reports of misbehavior (a measure of self-reported school misbehavior can be found here).


Citations:

Cholewa, B., Hull, M. F., Babcock, C. R., & Smith, A. D. (2018). Predictors and academic outcomes associated with in-school suspension. School Psychology Quarterly, 33 (2), 191.

DuBois, D. L., Portillo, N., Rhodes, J. E., Silverthorn, N., & Valentine, J. C. (2011). How effective are mentoring programs for youth? A systematic assessment of the evidence. Psychological Science in the Public Interest, 12 (2), 57-91.

Fabelo, T., Thompson, M. D., Plotkin, M., Carmichael, D., Marchbanks, M. P., & Booth, E. A. (2011). Breaking schools’ rules: A statewide study of how school discipline relates to students’ success and juvenile justice involvement. New York: Council of State Governments Justice Center.

Huang, F. L., & Cornell, D. G. (2017). Student attitudes and behaviors as explanations for the Black-White suspension gap. Children and youth services review, 73, 298-308. 

Morrison, G. M., Peterson, R., O’Farrell, S., & Redding, M. (2004). Using office referral records in school violence research: Possibilities and limitations. Journal of School Violence, 3(2/3), 39-61.

Nishioka, V. (2017). School Discipline Data Indicators: A Guide for Districts and Schools. REL 2017-240. Regional Educational Laboratory Northwest.

U.S. Department of Education Office for Civil Rights. (n.d.). Civil Rights Data Collection. Available at https://ocrdata.ed.gov/Home.

Brief Description: This measure consists of school records of student absences (both excused and unexcused).

Rationale: Evaluators and researchers often rely on students to report the number of days they miss school. However, research has shown that there is a weak association between self-reported absenteeism and absences reported in administrative data, with students frequently underestimating days absent, particularly when absences are unexcused.

Cautions: When planning to collect student absences directly from schools or school districts, be sure to set aside adequate time and staff resources to work with school personnel. Interpreting school records requires the ability to collate and analyze data. Programs without on-staff expertise may want to work with an external program evaluator.

Access and permissions: When working with an outside agency (e.g., a school or district) to collect school records, access to their data typically involves strict confidentiality conditions (see FERPA guidelines). You may be required to provide written parent permission with very specific information included (that can vary across schools or districts). Standard permission/consent language can be incorporated into program enrollment forms (see sample). Also, consider budgeting funds to reimburse time for school officials to gather needed data. More extensive guides on federal privacy guidelines and how to establish data sharing partnerships with school districts can be found in the Evaluation Guidance and Resources section of this Toolkit.

What to Collect: Suggestions for variables to request from schools or school districts can be found in a formatted data collection guide, here. If you are collecting report cards from parents or youth, this guide can be used to help structure your database for storage and analysis. 

How to Collect:

Sources: One option for collecting school absence records is to get them directly from parents or youth (e.g., copies of the students’ report card). A small incentive for providing this information, when possible, may be helpful. Another option is to get administrative data directly from schools or school district offices, in which case, a formal MOU will typically be required. Schools or districts may agree only to provide “deidentified” data (i.e., data that do not include student names or other identifying information).  If so, it is advisable in the data request to attach information to each youth’s name, such as basic demographics (gender or race/ethnicity) or program participation status so that the data once obtained (with this information attached to each line of data, but with the youth’s name removed) will allow you to use this information in analyses. Care must be taken, however, to ensure this type of attached information does not allow a youth to be inadvertently identified; a general rule of thumb is to ensure that the data once obtained do not include subgroups (e.g., male Native American youth) of fewer than 10 youth.

Additional Considerations: If you are interested in assessing changes over time, make sure to collect a "baseline" in the period before the student began program involvement. If mentoring program participation is less than the full school year, be sure to collect a similar time period for comparison to account for seasonal variations in absence rates (e.g., fall semester to fall semester). If possible, you may also want to consider collecting absence data for a comparable group of students not participating in the mentoring program. These data can be used to compare outcomes for program and non-program participants, which is a more robust evaluation design than simply looking at changes over the course of program involvement for program participants.

How to Analyze: It is important to work closely with school or district officials to understand how they record absences/non-attending. For example, there may be variations in whether absences include when students are late or leave school early, how staff determines what counts as an excused absence, and whether suspensions are included in the number of days absent. Attendance records may also vary across grade levels, dual enrollment programs, and virtual learning environments. Additionally, school records may or may not flag chronic absenteeism, defined by the U.S. Department of Education as missing 10% of the academic year (e.g., 18 days in a 180-day school year). If the schools you are working with do not track chronic absenteeism, ask for the total number of days in the academic year and divide the number of days a student has been absent (both excused and unexcused) by the total number of school days. Finally, excessive unexcused absences can trigger legal action under a state or locality’s truancy statute. School records typically indicate whether a student is considered truant, however, the number of allowable absences before the threshold for truancy is met may vary across schools or districts. 

How to interpret findings: Programs can report changes in total days missed, with a reduction in this number indicating student improvement. Here, it is important to distinguish between excused and unexcused absences, as the latter is more strongly associated with poor academic outcomes. Changes from chronically absent or truant during the baseline period to no chronic absenteeism or truancy in the follow-up period also indicates improvement in school attendance over time.


Citations:

Attendance Works. (2014). The Attendance Imperative: How States Can Advance Achievement by Reducing Chronic Absence. Retrieved from http://www.attendanceworks.org/state-policy-brief-attendance-imperative

Hancock, K. J., Gottfried, M. A., & Zubrick, S. R. (2018). Does the reason matter? How student‐reported reasons for school absence contribute to differences in achievement outcomes among 14–15 year olds. British Educational Research Journal, 44 (1), 141-174.

Keppens, G., Spruyt, B., & Dockx, J. (2019). Measuring school absenteeism: Administrative attendance data collected by schools differ from self-reports in systematic ways. Frontiers in Psychology10, 2623.

National Forum on Education Statistics. (2018). Forum Guide to Collecting and Using Attendance Data (NFES 2017-007). U.S. Department of Education. Washington, DC: National Center for Education Statistics. https://nces.ed.gov/pubs2017/NFES2017007.pdf

Teye, A. C., & Peaslee, L. (2015, December). Measuring educational outcomes for at-risk children and youth: Issues with the validity of self-reported data. In Child & Youth Care Forum (Vol. 44, No. 6, pp. 853-873). Springer US.

Brief Description: This measure consists of school records of student grades (e.g., letter grades).

Rationale: Evaluators and researchers frequently rely on student reports of their own grades. However, the accuracy of such reports can be influenced by a number of factors, including student age, cognitive ability, and actual school performance. For example, lower performing students tend to overestimate their grades, and younger students often have difficulty recalling them accurately. For these reasons, collecting records of grades is desirable when feasible. 

Cautions: When planning to collect grades, be sure to set aside adequate time and staff resources. Accessing school records can be complex, particularly when working with multiple schools, grade levels, and academic subjects. Additionally, the format of grades may differ across or even within schools, which is critical to consider when using grade data for program evaluation or research purposes.

Access and permissions:  When working with an outside agency (e.g., a school or district) to collect school records, access to their data typically involves strict confidentiality conditions (see FERPA guidelines). You may be required to provide written parent permission with very specific information included (that can vary across schools or districts). Standard permission/consent language can be incorporated into program enrollment forms (see sample). Also, consider budgeting funds to reimburse time for school officials to gather needed data.

What to Collect: Suggestions for variables to request from schools or school districts can be found in a formatted data collection guide, here. If you are collecting report cards from parents/youth, this template also can be used to help structure your database for storage and analysis.

How to Collect:

Sources: One option for collecting grades is to get them directly from parents/youth (e.g., copies of the students’ report card). A small incentive for providing this information, when possible, may be helpful. Another option is to get grade records directly from schools or school district offices, in which case, a formal MOU will typically be required. Schools or districts may agree only to provide “deidentified” data (i.e., data that does not include student names or other identifying information).  If so, it is advisable in the data request to attach information to each youth’s name, such as basic demographics (gender or race/ethnicity) or program participation status so that the data once obtained (with this information attached to each line of data, but with the youth’s name removed) will allow you to use this information in analyses. Care must be taken, however, to ensure this type of attached information does not allow a youth to be inadvertently identified; a general rule of thumb is to ensure that the data once obtained do not include subgroups (e.g., male Native American youth) of fewer than 10 youth.

Weighting: Be sure to consider how you will “weight” advanced, honors, and AP courses (e.g., a B in science = 4; a B in AP science = 5).

Additional Considerations: If you are interested in assessing changes over time, make sure to collect a "baseline" in the period before the student began program involvement. In addition, request grades for the entire time period of the student’s program participation and after the end of program involvement, as these data can help to assess longer-term program effects. And be sure to align grade requests with the specific timeframe of program enrollment for each student (e.g., one student may need grade information starting in the spring quarter of one school year through the fall of the next school year, whereas another student may have a very different time frame of participation). Typically, student report cards are issued quarterly but these records may vary in their meaning or purpose. For example, in some cases, quarterly grades may simply reflect student progress, while semester grades serve as the formal record of performance. If possible, you may also want to consider collecting grade information for a comparable group of students not participating in the mentoring program. These data can be used to compare outcomes for program and non-program participants, which is a more robust evaluation design than simply looking at changes over the course of program involvement for program participants.

How to Analyze:

Scoring: It is important to work closely with school or district officials to interpret scoring differences across years, grade levels, and subjects. If available, an annual district interpretation guide can be useful.

Grades and Subjects: Grades are commonly reported as letters which need to be converted to numeric values (e.g., F (Failing)=1, A (Excellent)=5). Additionally, schools may use different scales which need to be converted to one system to allow you to compare youth across these schools (e.g., O, S, N, & U; Pass/Fail; A, B, C, D & F). Depending on program goals, you may want to combine related subjects into a broader field (e.g., Biology and Chemistry can both be designated as Science) or to combine all course grades into a single GPA measure (which can be helpful when students are missing grades in particular subjects or their progress is being compared across multiple grades or scoring systems). Failing grades also can be used as a single indicator of performance (e.g., the number of failing grades a student has or whether the student has any failing grades in a given time period). 

How to interpret findings: When using GPA or individual course grades, higher scores indicate better academic performance. When using a count, or “presence or absence” of failing grades, a percent reduction can indicate improved academic performance. Remember that grade maintenance may be a positive outcome for some groups of youth (e.g., students sustaining the same level of grades before and after program participation during a period when grades often decline, such as the transition to junior high school).

 Alternatives: While useful, grades are not the only formal measure of student academic performance. Standardized test scores, grade retention, descriptive (“open-ended”) classroom-based assessments, portfolios or individual projects also can be collected. Depending on your program goals, these measures of performance can provide depth, richness, and perspective and increase your ability to detect program effectiveness.


Citations:

Karcher, M. J. (2008). The study of mentoring in the learning environment (SMILE): A randomized evaluation of the effectiveness of school-based mentoring. Prevention Science, 9(2), 99.

Kuncel, N. R., Credé, M., & Thomas, L. L. (2005). The validity of self-reported grade point averages, class ranks, and test scores: A meta-analysis and review of the literature. Review of educational research, 75(1), 63-82.

Teye, A. C., & Peaslee, L. (2015, December). Measuring educational outcomes for at-risk children and youth: Issues with the validity of self-reported data. In Child & Youth Care Forum (Vol. 44, No. 6, pp. 853-873). Springer US.

Scale: Youth Risk Behavior Survey Questionnaire (YRBS) from the Youth Risk Behavior Surveillance System - Opioid misuse items

What it measures:

  • Lifetime and 30-day misuse of prescription pain medication and heroin.

Intended age range: The items are intended for use with middle and high school students.

Brief description: The YRBS includes three questions on opioid use: lifetime misuse of prescription pain medication (i.e., “use of prescription pain medicine without a doctor’s prescription or differently than how a doctor told you to use it”); 30-day misuse of prescription pain medication; and lifetime heroin use. Each item is rated on a 6-point scale: 0 times; 1 or 2 times; 3 to 9 times; 10 to 19 times; 20 to 39 times; 40 or more times. 

Rationale: This measure was selected based on its brevity, appropriateness for use with youth, coverage of use of different types of opioids, availability of national norms, and evidence of validity. 

Cautions: This measure does not include pictures of different pain medications, which could help youth distinguish among types of prescription medication (e.g., opioids versus methamphetamines). It also does not assess accidental exposure to opioid drugs, such as might occur through use of a substance that is contaminated with an opioid drug.

Special administration information: The items could be adapted to assess use of specific opioids, such as by replacing “prescription pain medicine” with the name of specific drugs of interest.  However, such items are not included on the YRBS and thus validity data and national norms would not be available.

How to score: Each item is scored from 1 (0 times) to 6 (40 or more times) to yield three descriptive measures of opioid misuse.

How to interpret findings: Higher scores indicate more frequent opioid misuse.

Access and permissions: The YRBS is available for use with no charge and is available here. The specific questions on opioid misuse are available here

Alternatives: The Monitoring the Future survey includes questions asking about 30-day and/or 12-month use of specific opioids, including OxyContin, Vicodin, non-prescription cough or cold medicine, and heroin. The full survey codebook for the 8th and 10th grade surveys can be found here.  The National Institute on Drug Abuse has also created a screener to assist clinicians serving adult patients in screening for drug use, that appears potentially appropriate for use with adolescents. The screener asks about frequency of opioid (and other drug) use and follows with questions designed to assess risk for having or developing a substance use disorder and suggested next steps. Information on the screener can be found here.


Citation: Centers for Disease Control and Prevention (2018). 2019 Youth Risk Behavior Survey Questionnaire. Retrieved from www.cdc.gov/yrbs.

 

Scale: Goal Based Outcomes tool

What it measures:

  • A youth’s reported progress toward their chosen goals.

Intended age range: 10- to 18-year-olds and also has been used with adults.

Brief description: On the Goal-Based Outcomes tool, youth identify up to three goals for themselves. The tool was developed originally for use in clinical services context. In this context, developers recommend that goals be established collaboratively with a service provider, but that they must be agreed upon and owned by the person seeking help. In the context of a mentoring program, goals could be set with the assistance of program staff, mentors, and/or parents.  After youth goals are set and recorded, progress toward each goal is rated on a scale from 0 = Goal not at all met to 10 = Goal fully met/reached, with a midway anchor point of 5. The tool with the same goals listed can then be completed again at later points in time to assess progress toward those goals. Ratings on progress toward each goal can be completed independently by youth as well as with the assistance of program staff, mentors, and/or parents.

Rationale: This measure was chosen based on its applicability across diverse populations and settings, demonstrated ability to detect change in youth, evidence of reliability and validity, and brevity. Reported progress on goals as assessed on the Goal Based Outcomes tool has been positively associated with improvements in emotional symptoms and functioning and negatively associated with psychosocial difficulties in youth.

Cautions: The simplicity of the goal-generation procedure may lead youth and others involved in helping them to set goals to overlook underlying or less conscious concerns of youth. Most evidence for the use of the Goal Based Outcomes tool is based on samples of youth in clinical settings. It also should be taken into account that ratings on the Goal Based Outcomes tool (and other similar measures) involve a subjective component and may be subject to social desirability bias (i.e., a motivational tendency or investment on the part of raters, such as youth or mentors, to report positive progress). Related to this, it is important to remember that youth should not be expected to fully meet their goals, that youth may only make limited progress on their goals in a given period of time, and that a youth's goals may change. In view of these considerations, it may be useful to have a structure in place to review and discuss goal progress ratings as well as to collect data on goal progress from other sources.

Special administration information: The developers of the Goal Based Outcomes tool note that it may be useful to periodically provide opportunities for goals to be reset.

How to score: Each goal is scored from 0 (Goal not at all met) to 10 (Goal reached). Progress on up to three goals could be assessed at each meeting with youth, the beginning and end of an intervention, or at set time points. Goal progress for up to 12 meetings can be monitored using the goal progress chart.  Reported progress on up to three goals are averaged at each meeting. Change is calculated by averaging the differences between scores on each goal. Although not discussed by developers, the measure also appears potentially appropriate for use with youth who are not participating in an intervention (for example, youth in a non-mentoring comparison group in an evaluation study).

How to interpret findings: Higher scores reflect greater progress toward the goals set.

Access and permissions: The scale is available for non-commercial use with no charge and is available here.

Alternatives: Goal Attainment Scaling (GAS) is a widely used process that provides greater depth and nuance in assessment of goal attainment; however, GAS is more complex and time intensive and requires training.  More information on this measure and its application within mentoring is available here.


Citation: Law, D., & Jacob, J. (2015). Goals and Goal Based Outcomes (GBOs): Some useful information (3rd Ed.). London: CAMHS Press. Retrieved from https://www.annafreud.org/media/3189/goals-booklet-3rd-edition220715.pdf

 

Scale: Risk Factors for Opioid Misuse and the Negative Impact of Others' Opioid Misuse

What it measures:

  • Items address a range of potential risk factors for opioid use among youth and negative impact on youth of others' opioid misuse.

Intended age range: Youth aged 12 and older.

Brief description: Survey items include doctor prescribed opioid use (1 item) and, if yes, age of first opioid prescription (1 item), perceived availability of opioids (2 items), friends’ substance use, including opioids (1 item), perceived risk of harm from the use of opioids (2 items), awareness of opioid misuse by others (1 item), and negative impact of others' opioid misuse (1 item). Sample items include “How difficult or easy would it be for you to get some heroin, if you wanted some?” and “How much do you think people risk harming themselves (physically or in other ways), if they try opioid drugs other than heroin once or twice?” Response scales vary across items (see measure).

Rationale: These items were selected due to their relative simplicity and brevity, use in large-scale surveys of drug use (e.g., Monitoring The Future, National Survey on Drug Use and Health), and appropriateness for use with adolescents.

Cautions: This survey is not an exhaustive list of factors that may influence risk that a youth will misuse opioids. Youth mentoring programs specifically targeting opioid risk or opioid misuse should consider conducting a more comprehensive assessment of risk (e.g., psychological distress, other substance use).  Items addressing opioid misuse by others and its potential impact on youth were developed for purposes of this measure and thus have not been field tested or assessed for reliability or validity.

Special administration information: None.

How to score: There is no total or aggregate score created. Users of the measure should examine responses to individual items to facilitate understanding of risk reported by individual youth or groups of youth.

How to interpret findings: Responses indicating doctor prescription of opioids at an earlier age, greater access to opioids, less perceived risk of harm associated with opioid use, and opioid use by others (e.g., friends, parents) may indicate greater risk for opioid misuse by the youth. Reports of being negatively impacted by others' opioid use, furthermore, indicate a potential for youth to be harmed by opioid use occurring in their family, peer, and community environments.

Access and permissions: A copy of the measure can be found here. The measure is available for non-commercial use with no charge.

Alternatives: The Screener Opioid Assessment for Patients with Pain-Revised (SOAPP-R) is a measure of risk for opioid medication misuse (Butler, Fernandez, Benoit, et al., 2008) and may be a good alternative to the current measure for use with young adults or adults. The 24 items are rated from 0 (“never”) to 4 (“very often”) by respondents based on frequency of occurrence. Total scores range from 0 to 96. Scores ≥ 18 indicate high risk for opioid misuse. It should be kept in mind that this measure was validated with pain patients and thus broader applicability to other populations remains to be established. A copy of the measure can be found here.


Citations:

Arthur, M. W., Hawkins, J. D., Pollard, J. A., Catalano, R. F., and Baglioni Jr., A. J. (2002). Measuring risk and protective factors for substance use, delinquency, and other adolescent problem behaviors: The Communities That Care Youth Survey. Evaluation Review, 26, 575–601. doi: 10.1177/0193841X0202600601 

Center for Behavioral Health Statistics and Quality. (2018). 2019 National Survey on Drug Use and Health (NSDUH): CAI Specifications for Programming (English Version). Substance Abuse and Mental Health Services Administration, Rockville, MD. Retrieved from https://www.samhsa.gov/data/sites/default/files/cbhsq-reports/NSDUHmrbCAISpecs2019.pdf

Glaser, R. R., Van Horn, M. L., Arthur, M. W., Hawkins, J. D., & Catalano, R. F. (2005). Measurement properties of the Communities That Care® youth survey across demographic groups. Journal of Quantitative Criminology, 21, 73-102. doi: 10.1007/s10940-004-1788-1

Kecojevic, A., Wong, C. F., Schrager, S. M., Silva, K., Bloom, J. J., Iverson, E., & Lankenau, S. E. (2012). Initiation into prescription drug misuse: Differences between lesbian, gay, bisexual, transgender (LGBT) and heterosexual high-risk young adults in Los Angeles and New York. Addictive Behavior, 37, 1289-1293. doi: 10.1016/j.addbeh.2012.06.006

 Miech, R. A., Johnston, L. D., O’Malley, P. M., Bachman, J. G., Schulenberg, J. E., & Patrick, M. E. (2019). Monitoring the Future national survey results on drug use, 1975–2018: Volume I, Secondary school students. Ann Arbor: Institute for Social Research, The University of Michigan. Available at http://monitoringthefuture.org/pubs.html#monographs

 

The relationships that develop between youth and their mentors are thought to be the central route through which mentoring can benefit (or, inadvertently, harm) youth (Rhodes, 2005; Karcher & Nakkula, 2010a). Thus, it is important to be able to assess the salient characteristics, or “quality,” of youth’s mentoring relationships.

Mentoring programs generally understand the importance of relationship quality and its potential role in fostering program benefits for youth. In fact, many programs have made relationship quality a central component of their internal evaluation activities. But programs often struggle to determine which components of relationship quality are most essential to measure and often use “homegrown” tools or limited measures of relationship satisfaction, rather than digging deeper into how the relationship is experienced across many dimensions or from multiple viewpoints. And when programs do try to use research-backed measures, they are faced with a dizzying array of options to choose from and it is difficult for them to gauge their relative merits and potential fit with their programs.

This section of the Measurement Guidance Toolkit is intended to help programs with this process of selecting reliable and valid tools for assessing the quality of the mentoring relationships that they are cultivating through their efforts.

A Framework for Understanding Relationship Dimensions

To guide our selection of measures to include in the toolkit, we followed a framework developed by Nakkula and Harris (2014). This framework highlights the following aspects of relationship quality: Internal match quality (consisting of relational and instrumental components), match structure, and external match quality.

Internal Match Quality encompasses how the mentor and youth feel about their relationship and each other as well as more objective indicators of quality:

  • Relational aspects of this dimension reflect how the youth and mentor feel about each other and the way they relate to each other, including their perceptions of compatibility as well as feelings of mutual closeness, trust, and overall satisfaction with the relationship. Objective indicators in this category include the frequency and duration of meetings and the longevity of the mentoring relationship;

  • Instrumental aspects of internal match quality reflect the degree of growth orientation in the relationship. Specific indicators include the extent to which the mentor and youth focus on achieving goals together and the youth’s satisfaction with the support received. More objective indicators include the types and frequency of support received.

Match structure includes what the mentee/mentor want to do together, how they decide what to do, and objective measures of the types of activities in which they ultimately engage.

Finally, External Match Quality includes elements outside of the mentoring relationship that can affect its development, such as perceived program support and the degree of parent engagement in the match. We did not focus on this last component this year, limiting our review to the measurement of relationship components that occur within the relationship. Exploring measures that assess these external influences on mentoring relationships may be a priority in future additions to the Toolkit.

We selected the Nakkula and Harris (2014) framework from several strong and influential theoretical frameworks in the field (see Karcher & Nakkula, 2010a), mainly because it is extremely comprehensive in the facets of relationship quality that it includes and thus would support our goal of considering measures of a broad range of different aspects of mentoring relationships. The framework also reflects, or accounts for, elements emphasized in other important frameworks in the field. For example, early seminal work by Morrow and Styles (1995) emphasized the importance of mentor approach, with findings suggesting that a developmental approach (focusing on youth’s voice in the relationship) is most conducive to relationship success, relative to a prescriptive approach (letting the adult’s goals for youth guide the relationship). Hamilton and Hamilton’s (1992) conceptualization of mentoring focuses on the instrumental roles that mentors take on when helping youth achieve different goals (see also Hamilton, Hamilton, DuBois, & Sellers, 2016). The TEAM framework (described in Karcher & Nakkula, 2010b) emphasizes the focus, purpose and authorship of the mentoring relationship and how these factors can interact in shaping its tenor. Keller and Pryce’s (2010) framework conceptualizes all relationships in terms of power (i.e., whether the relationship is vertical as in a parent-child relationship or horizontal as in friendships) and permanence (the degree to which the relationship is obligated or voluntary). Mentoring relationships are a unique combination of these elements (i.e., both unequal in power and voluntary). A mentor’s ability to maintain this hybrid role in her or his approach is posited to be key to relationship success (Keller & Pryce, 2012). All of these conceptually rich frameworks provide some guidance in which aspects of the relationship may be important to measure and are highly recommended for review as programs consider which aspects of relationship quality may be particularly telling for their specific program.

Selected measures.

A total of nine measures of mentoring relationship quality are included in this section of the Toolkit (an overview of how the measures were selected can be found on the About This Toolkit page). The recommended instruments are organized into three groups.

The first group, multi-faceted measures, consists of three measures. These measures are designed to provide insight into multiple aspects of relationship quality or, more specifically, at least two of the aspects of relationship quality outlined in the Nakkula and Harris framework (2014). These measures consist of multiple scales and thus are relatively lengthy but compensate for that length in richness and scope. Two of the measures include both a mentor and a youth version.

The second category consists of unidimensional measures. These instruments assess one dimension of relationship quality, in most cases, using only one scale. The two selected measures may be particularly attractive options for programs that can ask their participants only a limited number of questions. Nakkula and Harris (2014) propose that if only one aspect of relationship quality can be assessed, the relational aspect (as described above) is most central. In fact, almost all of the instruments we reviewed focused, at least in part, on relational aspects of relationship quality—and both of the unidimensional measures cover this feature of relationships—suggesting general agreement in the field that this is one of the more telling aspects of relationship quality.

The final category consists of measures of specific facets of relationships. These measures are oriented toward assessing aspects of mentoring relationships that, although likely to be relevant to many programs, are not routinely captured by more general-purpose measures of mentoring relationship quality. Illustratively, recent survey data indicate that as many as 1 in 3 youth served by mentoring programs meet with their mentor in a group context (Garringer, McQuillin, & McDaniel, 2017). Yet, none of the recommended measures in the other two categories described above are geared toward capturing the nature and quality of the various types of interactions and group dynamics that may take place within these types of programs. Thus, one of our recommended measures in this category covers this important area.

The four facets of mentoring relationships assessed by the measures in this last category are:

  1. Youth-centeredness: Youth’s “voice” in the relationship--that is, the extent to which youth feel that the activities and direction of the relationship reflect their own interests and needs. Morrow and Styles (1995) in their qualitative study of Big Brothers Big Sisters community-based mentoring relationships provided important early evidence of the importance of youth voice in contributing to successful relationships. More recent work (Herrera, DuBois & Grossman, 2013) has linked youth reports of youth-centeredness to youth reports of a stronger growth/goal focus in the relationship (see “Growth focus” below) and to program supports (i.e., those mentors who are trained and better supported have mentees who report higher levels of youth centeredness in their relationship).

  2. Mentor cultural sensitivity: The mentor’s attention to supporting her or his mentee’s cultural identity (Sánchez, Pryce, Silverthorn, Deane, & DuBois, 2018; Spencer, 2007). Studies suggest that improving mentor’s attention to this important component of youth identity can foster higher-quality relationships (Sánchez, Pryce, Silverthorn, Deane, & DuBois, 2018; Spencer, 2007). In fact, mentors’ reports of multicultural competence have been found to be correlated with their reported levels of satisfaction with their relationships with both the mentee and the mentoring organization as well as the quality of their relationship with the mentee’s family (Suffrin, Todd, & Sanchez, 2016). Reports by youth of color of receiving mentor support in this area were also found to be positively associated with the youth’s own reports of satisfaction with relational and instrumental aspects of the mentoring relationship (Sanchez et al., in press).

  3. Growth focus: The extent to which the relationship includes a focus on growth or goal achievement (Karcher & Nakkula, 2010b). Youth reports of growth focus in their mentoring relationship have been linked positively with the mentor’s receipt of training (both early on in the match and ongoing training) and receipt of higher quality support from program staff (Herrera et al., 2013). In the same study, youth who rated their mentors as higher in youth centeredness also tended to report a stronger growth/goal focus in their relationships.

  4. Group mentoring processes: The various interactional processes that occur when mentors meet with youth in a group context (Kuperminc & Thomason, 2014). Relatively few studies have been conducted that explore this important area. One recent study, however, reported that youth reports of their experiences within their mentoring group predicted several key youth outcomes including self-efficacy, school belonging, and school participation (Kuperminc, Sanford, & Chan, 2017).

We hope this section of the toolkit helps programs strategize more thoughtfully about the aspects of mentoring relationship quality that help both the relationships and youth they support to thrive and how they might go about measuring these qualities at various points in the mentoring relationship. Please also remember that any mentoring program can get free technical assistance to help think through how best to assess mentoring relationship quality in their program by requesting assistance through this website.


Cited Literature

Bayer, A., Grossman, J. B., & Dubois, D. L. (2015). Using volunteer mentors to improve the academic outcomes of underserved students: The role of relationships. Journal of Community Psychology, 43(4), 408-429. doi: 10.1002/jcop.21693

DuBois, D. L. and Neville, H. A. (1997). Youth mentoring: Investigation of relationship characteristics and perceived benefits. Journal of Community Psychology, 25, 227-234. doi:10.1002/(SICI)1520-6629(199705)25:3<227::AID-JCOP1>3.0.CO;2-T

DuBois, D. L., & Neville, H. A. (1997). Youth mentoring: Investigation of relationship characteristics and perceived benefits. Journal of Community Psychology, 25, 227–234.

Garringer, M., McQuillin, S., & McDaniel, H. (2017). Examining youth mentoring services across America: Findings from the 2016 National Mentoring Program Survey. Boston, MA: MENTOR: The National Mentoring Partnership.

Hamilton, S. F., & Hamilton, M. A. (1992). Mentoring programs: Promise and paradox. Phi Delta Kappan, 73(7), 546. 

Hamilton, M. A., Hamilton, S. F., DuBois, D. L., & Sellers, D. E. (2016). Functional roles of important nonfamily adults for youth. Journal of Community Psychology, 44(6), 799-806.

Herrera, Carla, David L. DuBois and Jean Baldwin Grossman. (2013). The Role of Risk: Mentoring Experiences and Outcomes for Youth with Varying Risk Profiles. New York, NY: A Public/Private Ventures project distributed by MDRC.

Karcher, M. J., & Nakkula, M. J. (2010a). New Directions for Youth Development, 2010.

Karcher, M. J., & Nakkula, M. J. (2010b). Youth mentoring with a balanced focus, shared purpose, and collaborative interactions. New Directions for Youth Development, 2010(126), 13-32.

Keller, T. E., & Pryce, J. M. (2010). Mutual but unequal: Mentoring as a hybrid of familiar relationship roles. New Directions for Youth Development, 2010(126), 33-50.

Keller, T. E., & Pryce, J. M. (2012). Different roles and different results: How activity orientations correspond to relationship quality and student outcomes in school-based mentoring. The Journal of Primary Prevention, 33(1), 47-64.

Kuperminc, G., Sanford, V., & Chan, W. Y. (2017, February). Building effective group mentoring programs: Lessons from research and practice on Project Arrive. Workshop presented at the National Mentoring Summit, Washington, DC.

Kuperminc, G. P., & Thomason, J. D. (2014). Group mentoring. In D. Dubois & M. Karcher (Eds.), Handbook of youth mentoring (2nd ed., pp. 273-289). Thousand Oaks, CA: Sage Publications.

Morrow, K. V., & Styles, M. B. (1995). Building relationships with youth in program settings: A study of Big Brothers/Big Sisters. Philadelphia, PA: Public/Private Ventures.

Nakkula, M., & Harris, J. (2014). Assessing mentoring relationships. In D. Dubois & M. Karcher (Eds.), Handbook of youth mentoring (2nd ed., pp. 45-62). Thousand Oaks, CA: Sage Publications.

Parra, G. R., DuBois, D. L., Neville, H. A., Pugh‐Lilly, A. O. and Povinelli, N. (2002), Mentoring relationships for youth: Investigation of a process‐oriented model. Journal of Community Psychology, 30, 367-388. doi:10.1002/jcop.10016

Rhodes, J. E. (2005). A model of youth mentoring. In D. L. DuBois & M. J. Karcher (Eds.) Handbook of youth mentoring (pp. 30–43). Thousand Oaks, CA: SAGE.

Rhodes, J. E., Schwartz, S. E., Willis, M. M., & Wu, M. B. (2017). Validating a mentoring relationship quality scale: Does match strength predict match length?. Youth & Society, 49(4), 415-437.

Sánchez, B., Pryce, J., Silverthorn, N., Deane, K., & DuBois, D. L. (in press). Do mentor support for racial/ethnic identity and cultural mistrust matter for girls of color? A preliminary investigation. Cultural Diversity and Ethnic Minority Psychology

Spencer, R. (2007). “It's Not What I Expected” A Qualitative Study of Youth Mentoring Relationship Failures. Journal of Adolescent Research, 22(4), 331-354.

Suffrin, R. L., Todd, N. R., & Sánchez, B. (2016). An ecological perspective of mentor satisfaction with their youth mentoring relationships. Journal of Community Psychology, 44(5), 553-568.

Thomson, N., & Zand, D. (2010). Mentees’ perceptions of their interpersonal relationships: The role of the mentor-youth bond. Youth & Society, 41, 434-447.

Zand, D. H., Thomson, N., Cerventes, R., Espiritu, R., Klagholz, D., LaBlanc, L., & Taylor, A. (2009). The mentor–youth alliance: The role of mentoring relationships in promoting youth competence. Journal of Adolescence, 1–17. doi:10.1016/j.adolescence.2007.12.006

 

 

Scale: Youth-Centered Relationship

What it measures:

  • Youth's perceptions of the extent to which the activities engaged in with the mentor are centered on the youth's interests.

Intended age range: 10- to 18-year-olds.

Brief description: This scale consists of 5 items assessing the extent to which the youth feels the mentor considers their preferences and interests when selecting activities. Sample items include: “My mentor almost always asks me what I want to do” and “My mentor and I like to do a lot of the same things.” Youth respond on a 4-point scale: Not true at all, Not very true, Sort of true, or Very true.

Rationale: This measure was chosen because of its brevity and evidence of validity and reliability across youth of differing gender, race/ethnicity and risk profiles.

Cautions: None.

Special administration information: None.

How to score: Each item is scored from 1 (Not true at all) to 4 (Very true). The overall perception of youth-centeredness is created by averaging across all five items.

How to interpret findings: Higher scores on the scale reflect higher levels of youth-centeredness in the mentoring relationship.

Access and permissions: The scale is available for non-commercial use with no charge and is made available here.

Alternatives: None recommended.


Citation: Jucovy, L. (2002). Measuring the quality of mentor-youth relationships. Philadelphia, PA: Public/Private Ventures. Available at http://educationnorthwest.org/sites/default/files/packeight.pdf.

Grossman, J. B., & Johnson, A. (1999). Judging the effectiveness of mentoring programs. In J. B. Grossman (Ed.), Contemporary Issues in Mentoring (pp.24-47). Philadelphia, PA: Public/Private Ventures. Available at https://www.issuelab.org/resources/11829/11829.pdf.

Scale: Youth Strength of Relationship (YSoR) and Mentor Strength of Relationship (MSoR)

What it measures:

  • A youth or mentor’s perceptions of, and experiences in, the mentoring relationship.

Applicable age range: 5- to 21-year-old mentees; Mentors 17 and over (though the items also appear relevant for slightly younger mentors).

Brief description: The youth version of this scale consists of 10 items assessing both positive (6 items, e.g., “My Big has lots of good ideas about how to solve a problem”) and negative (4 items, e.g., “When I am with my Big, I feel ignored”) perceptions of the relationship with their mentor. Youth respond on a 5-point scale: Never true, Hardly ever true, Sometimes true, Most of the time true, or Always true. The mentor version consists of 14 items assessing both positive and negative perceptions of the relationship using two subscales: Affective (10 items, e.g., “I enjoyed the experience of being a Big,” “Sometimes I feel frustrated with how few things have changed with my Little”) and Logistical (2 items, e.g., “It is hard for me to find the time to be with my Little”). Mentors respond on a 5-point scale: Strongly disagree, Disagree, Neutral, Agree, or Strongly agree.

Rationale: The YSoR and MSoR scales were selected because of their brevity and the fact that they capture both negative and positive experiences within the mentoring relationship. Both mentor and youth versions also have demonstrated good reliability for the total scores and associations with match length in a sample of BBBS community-based matches. 

Cautions: Although promising, evidence of reliability and validity is limited to one study.

Special administration information: When administering, references to “Big” can be substituted with “mentor,” and “Little” can be substituted with “mentee.”

How to score: The mentor version is scored on a 5-point scale from 1 (Strongly disagree) to 5 (Strongly agree). The youth version is scored on a 5-point scale from 1 (Not true at all) to 5 (Always true). Prior to scoring, negatively worded items are reverse scored (items 3, 4, 6, & 8 on the YSoR and items 2, 5, 6, 8, 9, 10, 12, & 13 on the MSoR). The total score is the average of all 10 YSoR items or 14 MSoR items. For the YSoR, subscale scores are computed as the average for the Positive (items 1, 2, 5, 7, 9, & 10) and Negative subscales (items 3, 4, 6, & 8). For the MSoR, subscale scores are computed as the average for the Affective (items 1-4, 6-9, & 11-14) and Logistical (items 5 & 10) subscales.

How to interpret findings: Higher scores reflect more positive perceptions of the mentoring relationship.

Alternatives: The Relationship Quality Scale (Rhodes et al., 2005) is an earlier version of the scale that was revised to become the YSoR (see link in Citations below). 

Access and permissions: Both the youth-report and the mentor-report measures are available for non-commercial use with no charge and are made available here.


Citations: Rhodes, J. E., Reddy, R., Roffman, & Grossman, J. B. (2005). Promoting successful youth mentoring relationships: A preliminary questionnaire. The Journal of Primary Prevention, 26, 147-167. https://doi.org/10.1007/s10935-005-1849-8

Rhodes, J. E., Schwartz, S. E. O., Willis, M. M., & Wu, M. B. (2017). Validating a mentoring relationship quality scale: Does match strength predict match length? Youth & Society, 49, 415-437. https://doi.org/10.1177/0044118X14531604

Page 1 of 6

Request no-cost help for your program

Advanced Search