Brief Description: This measure consists of school records of student grades (e.g., letter grades).
Rationale: Evaluators and researchers frequently rely on student reports of their own grades. However, the accuracy of such reports can be influenced by a number of factors, including student age, cognitive ability, and actual school performance. For example, lower performing students tend to overestimate their grades, and younger students often have difficulty recalling them accurately. For these reasons, collecting records of grades is desirable when feasible.
Cautions: When planning to collect grades, be sure to set aside adequate time and staff resources. Accessing school records can be complex, particularly when working with multiple schools, grade levels, and academic subjects. Additionally, the format of grades may differ across or even within schools, which is critical to consider when using grade data for program evaluation or research purposes.
Access and permissions: When working with an outside agency (e.g., a school or district) to collect school records, access to their data typically involves strict confidentiality conditions (see FERPA guidelines). You may be required to provide written parent permission with very specific information included (that can vary across schools or districts). Standard permission/consent language can be incorporated into program enrollment forms (see sample). Also, consider budgeting funds to reimburse time for school officials to gather needed data.
What to Collect: Suggestions for variables to request from schools or school districts can be found in a formatted data collection guide, here. If you are collecting report cards from parents/youth, this template also can be used to help structure your database for storage and analysis.
How to Collect:
Sources: One option for collecting grades is to get them directly from parents/youth (e.g., copies of the students’ report card). A small incentive for providing this information, when possible, may be helpful. Another option is to get grade records directly from schools or school district offices, in which case, a formal MOU will typically be required. Schools or districts may agree only to provide “deidentified” data (i.e., data that does not include student names or other identifying information). If so, it is advisable in the data request to attach information to each youth’s name, such as basic demographics (gender or race/ethnicity) or program participation status so that the data once obtained (with this information attached to each line of data, but with the youth’s name removed) will allow you to use this information in analyses. Care must be taken, however, to ensure this type of attached information does not allow a youth to be inadvertently identified; a general rule of thumb is to ensure that the data once obtained do not include subgroups (e.g., male Native American youth) of fewer than 10 youth.
Weighting: Be sure to consider how you will “weight” advanced, honors, and AP courses (e.g., a B in science = 4; a B in AP science = 5).
Additional Considerations: If you are interested in assessing changes over time, make sure to collect a "baseline" in the period before the student began program involvement. In addition, request grades for the entire time period of the student’s program participation and after the end of program involvement, as these data can help to assess longer-term program effects. And be sure to align grade requests with the specific timeframe of program enrollment for each student (e.g., one student may need grade information starting in the spring quarter of one school year through the fall of the next school year, whereas another student may have a very different time frame of participation). Typically, student report cards are issued quarterly but these records may vary in their meaning or purpose. For example, in some cases, quarterly grades may simply reflect student progress, while semester grades serve as the formal record of performance. If possible, you may also want to consider collecting grade information for a comparable group of students not participating in the mentoring program. These data can be used to compare outcomes for program and non-program participants, which is a more robust evaluation design than simply looking at changes over the course of program involvement for program participants.
How to Analyze:
Scoring: It is important to work closely with school or district officials to interpret scoring differences across years, grade levels, and subjects. If available, an annual district interpretation guide can be useful.
Grades and Subjects: Grades are commonly reported as letters which need to be converted to numeric values (e.g., F (Failing)=1, A (Excellent)=5). Additionally, schools may use different scales which need to be converted to one system to allow you to compare youth across these schools (e.g., O, S, N, & U; Pass/Fail; A, B, C, D & F). Depending on program goals, you may want to combine related subjects into a broader field (e.g., Biology and Chemistry can both be designated as Science) or to combine all course grades into a single GPA measure (which can be helpful when students are missing grades in particular subjects or their progress is being compared across multiple grades or scoring systems). Failing grades also can be used as a single indicator of performance (e.g., the number of failing grades a student has or whether the student has any failing grades in a given time period).
How to interpret findings: When using GPA or individual course grades, higher scores indicate better academic performance. When using a count, or “presence or absence” of failing grades, a percent reduction can indicate improved academic performance. Remember that grade maintenance may be a positive outcome for some groups of youth (e.g., students sustaining the same level of grades before and after program participation during a period when grades often decline, such as the transition to junior high school).
Alternatives: While useful, grades are not the only formal measure of student academic performance. Standardized test scores, grade retention, descriptive (“open-ended”) classroom-based assessments, portfolios or individual projects also can be collected. Depending on your program goals, these measures of performance can provide depth, richness, and perspective and increase your ability to detect program effectiveness.
Karcher, M. J. (2008). The study of mentoring in the learning environment (SMILE): A randomized evaluation of the effectiveness of school-based mentoring. Prevention Science, 9(2), 99.
Kuncel, N. R., Credé, M., & Thomas, L. L. (2005). The validity of self-reported grade point averages, class ranks, and test scores: A meta-analysis and review of the literature. Review of educational research, 75(1), 63-82.
Teye, A. C., & Peaslee, L. (2015, December). Measuring educational outcomes for at-risk children and youth: Issues with the validity of self-reported data. In Child & Youth Care Forum (Vol. 44, No. 6, pp. 853-873). Springer US.