Peraj Mentoring Program

In considering the key takeaways from the research on this program that other mentoring programs can apply to their work, it’s useful to reflect on the features and practices that might have influenced its rating as “Promising” (that is, a program that shows some evidence that it achieves justice-related goals when implemented with fidelity).

1. Peraj is a good example of replicating a program that has been implemented successfully in other environments.

As noted in the journal article that describes the Peraj evaluation, this national program in Mexico is actually an adaptation of the Perach program that originated in Israel in the 1970s. While the replication of the program model in Mexico required some adjustments to account for the differences in the educational systems between the two countries, one can see the benefits of not starting from scratch in the section that details the structure of Peraj and how the program functions on each college campus. The description of the central office and the duties they perform—recruitment of participants, training staff and mentors, conducting evaluation activities, securing funding, developing mentoring activities and curricula, etc.—make it very clear that roles and responsibilities in the program have been well thought and that there is a robust infrastructure designed to work well with the structure of higher education institutions across the country.

The program also offers very clear descriptions of the mentors’ role and their eligibility criteria, including how their time as mentors fits into their overall educational progression. The program, furthermore, maximizes the campus setting, by connecting mentees to other mentors who might have subject matter expertise in school subjects of interest to the mentee—essentially offering access to team mentoring within the context of a one-to-one match.

While the program may have come up with this comprehensive structure and clear roles and responsibilities on their own, building off of a successful mentoring model working in higher education in another country obviously helped Peraj find their footing more easily and set them up for the successful outcomes we see described here.

2. To achieve outcomes, you have to know what’s causing the thing you want to change.

There is a lengthy description in the evaluation article articulating exactly why Mexican youth drop out of their educational system at various points. And to be clear, the dropout issue in Mexico is a serious one: according to one study, only 36% of Mexican youth graduate high school on timeⁱ and only 52% of the 15-17 year olds in the country were even enrolled in high schoolⁱⁱ. Data also show that young people tend to drop out all throughout their educational progression, unlike in the United States where dropping out before the high school years is extremely rare.

Unfortunately, the authors admit that tracking youth through the Mexican educational system is extremely challenging because of the lack of standardized data systems across the nation, making it very hard to determine if children who drop out ever come back and complete school, or even determining if those who move with their families enroll in their new locations. The developers of Peraj, however, correctly recognize that leaving school early is often not caused by one factor but rather by a process that involves multiple factors over time leading up to the decision to leave school. This mirrors research from the U.S. showing that leaving school early is a lengthy process that involves many competing resilience and risk factorsⁱⁱⁱ. In fact, the little research that is available to programs like Peraj that want to address this issue in Mexico points to a number of individual, social, and educational system factors that predict leaving school early—everything from pregnancy and marriage, to joining the workforce, to struggling in school and not receiving enough academic help.

To combat this, Peraj takes a multi-pronged approach, using mentors to support the 5^th and 6^th graders served by the program in the areas of self-esteem, social skills, motivation, and study skills. And while these all seem helpful, and the evaluation did find evidence that the program lead to a reduction in a composite measure of dropout risk, one wonders if the intervention matches the reasons for leaving school as well as it could. First of all, the program is working with 5^th and 6^th graders, and while their national data shows that many of these young people are leaving school around those ages, many more will leave toward the end of middle school and throughout high school. It is unclear as to whether a program like Peraj would have lingering effects over those years as well. One can assume that if the program can get youth feeling better about school and finding some success academically, that those gains should provide some resilience against dropping out later on. But that is very unclear and one wonders if the program would even be able to track mentees over time to find out given the lack of structured data collection in the Mexican educational system.

Unfortunately, while the article does mention several factors that influence dropping out in Mexico, it does not ascribe a percentage or volume to each of these factors. In other words, it is unclear if Peraj is addressing factors that account for 25% or 75% of dropouts. It may be that other services and supports are needed to account for factors not directly addressed by the program. And it’s hard to know whether the mentoring that happens when a child is 10 or 11 years old will help them avoid reasons that clearly drive some percentage of older dropouts in Mexico, namely joining the workforce or starting one’s own family at a young age.

So while the program gets a lot of credit for trying to unpack the reasons behind the dropout crisis and counter them with a multi-faceted mentoring approach, one wonders if they might have more impact on their national issue by serving older youth or partnering with other providers to address some of those “down the road” reasons for not completing school. After all, 80% of Mexican students finish elementary school, but that drops to 60% at the end of middle school and 46% by 11th grade, well after the mentoring experience currently provided by Peraj has ended.

3. If you are going to develop a home-grown evaluation tool, Peraj offers a nice example of doing it the right way.

To conduct the evaluation described in the article, Peraj wanted to develop a composite measure of dropout risk that accounted for a variety of factors and boiled it down to one number that predicted risk. The article describes the development of the tool and how it measures eight key dimensions of dropout factors, ranging from self-control and aggression to school engagement and academic expectations. The program and its evaluators then did reliability and validity testing to make sure that the tool accurately captured information about these risk factors and could measure reductions or increases in each.

Normally, the Research Board of the National Mentoring Resource Center cautions against developing evaluation instruments from scratch (this was one of the main motivations behind the development of our Measurement Guidance Toolkit). But this evaluation offers a nice example of when that might be necessary (trying to measure many factors in one survey), as well as a process to make sure that the instrument itself is accurate and valid.

But it is also worth noting, that while this tool did predict overall, composite dropout risk, the evaluation does not report which of those eight areas measured were driving the overall reductions. While the program certainly deserves credit for the positive outcomes reported, it is frustrating that the analysis seems to have avoided looking at which factors were most important and which levers of dropout prevention the program seems most adept at pulling. This makes it hard to improve the program over time by strengthening areas where little change was noted. It also makes it hard for other programs or researchers to build on this work and perhaps emphasize similar things in their own dropout prevention work. Combined with the issue noted above about not being sure if the program is addressing the right factors to turn around the dropout crisis nationally, we are left to wonder if the program is focusing on the right things or uncovering more nuanced information about the factors that this program can and can’t address. So while Peraj certainly seems like a step in the right direction in addressing school completion issues, and very much seems like a well-structured and managed program, it may require additional research to determine if it is having the ultimate impact it hopes for.

*Note: The National Mentoring Resource Center makes these “Insights for Mentoring Practitioners” available for each program or practice reviewed by the National Mentoring Resource Center Research Board. Their purpose is to give mentoring professionals additional information and understanding that can help them apply reviews to their own programs. You can read the full review on the National Mentoring Resource Center website.

ⁱ Secretarıa de Educacion Publica. (2012). Reporte de la Encuesta Nacional de Desercion en la Educacion Media Superior [Report of the National Survey of High School Dropout]. Retrieved from http://www.sems.gob.mx/work/models/sems/Resource/10787/1/images/Anexo_6Reporte_de_la_ENDEMS.pdf

ⁱⁱ Instituto Nacional para la Evaluacion de la Educacion. (2013). Panorama Educativo de Mexico 2012. Indicadores del Sistema Educativo Nacional. Educacion Basica y Media Superior [Educational Overview of Mexico 2012. Indicators of the National Educational System. Primary and Secondary Education]. Mexico: Author.

ⁱⁱⁱ Center for Promise (2015). Don’t quit on me: What young people who left school say about the power of relationships. Washington, DC: America’s Promise Alliance.