Pathways to Education

In considering the key takeaways from the research on this program that other mentoring programs can apply to their work, it’s useful to reflect on the features and practices that might have influenced its rating as “Promising” (that is, a program that shows some evidence that it achieves justice-related goals when implemented with fidelity).

1. When are the active ingredients of a “mentoring program” potentially not mentoring?

One of the first things one notices about the Pathways model is that there are four main pillars to the program, “counseling, academic, social, and financial”—with mentoring being part of the social category. But the mentoring that happens within that seems inadequately described in the study cited in the formal review. Mentoring is really only offered to 9^th and 10^th graders in the program and is described as being in the form of group activities, which includes “attending sporting events, theater, participating in creative arts, cooking, bowling, community recycling projects, and martial arts” as well as cognitive behavioral workshops. Mentees pick two of these types of activities to do a month. It is unclear from the description whether these groups are consistent over time or shuffle members continually, which may be an issue for a group mentoring approach since one of the hypothesized mechanisms of change in group mentoring programs is the bond and cohesive relationships formed by the group members. This would clearly be absent in this program if youth shift activities, even if they may get “mentoring” from the adults leading each one.

There are also other prominent “caring adult” roles in the program—most notably, there is the Student-Parent Support Worker (SPSW), who is tasked with forming a close relationship with the youth and his/her family and serving as a primary coordinator of all their services and supports. This person is clearly in a role where they could be forming mentoring-like relationships with youth in the program, but the study does not report much information on the level of closeness between these adults and youth, nor does it mention the amount or frequency of time they tend to spend with a typical youth (although with caseloads of up to 50 youth/families, it may not be all that much).

Youth are also assigned to tutors and other adults who work with them on a variety of academic subjects and other activities, including more focused college prep work later in their high school years.

So, that’s a lot of caring adults and an unclear description of the mentoring being provided by any of them. Perhaps most confusingly, “Big Brothers/Big Sisters” is listed in the evaluation report¹ as one of the group mentoring activities that youth can sign up for, but it is unclear if that means they are enrolled in a local Big Brothers Big Sisters agency or if they are doing that in addition to their group mentoring experiences.

None of this is to say that all these activities and this surplus of adults taking an active role is a bad thing or that they don’t constitute meaningful and impactful mentoring. It’s just that the evaluation doesn’t differentiate the contribution that each of these pillars makes in any testable way. We don’t know if that “social” pillar of the program is driving much of the change, none of it, or somewhere in between. We have noted before in other reviews of multi-component mentoring efforts that the role of a mentor can sometimes be ambiguous in these models with lots of moving parts and that evaluations can struggle to identify what contribution, if any, the mentoring being provided makes to the program’s overall impact. That seems to be the case here where clearly Pathways youth are receiving mentoring, perhaps lots of it, but the details in how that works in synergy with the other program components still remains a bit of a mystery.

2. The value of some financial incentives.

Another of those “pillars” that gets discussed in the evaluation report is the financial incentives of the program. This takes two forms, each of which may be rather critical in the outcomes assessed in the evaluation. First, the program incentivizes participation in the program by providing youth with free bus tickets/passes that allow them to not only come to the program, but also get themselves to school in an effort to combat chronic absenteeism. The provision of free transportation options has been suggested in other research to be a critical component of combating missed school days. The evaluation here suggests that these passes were a major driver of youth participation in the program, perhaps accentuated by the fact that many students did not live near where the services were offered. Programs that need to ensure that youth can overcome transportation challenges to both be in the program and be at school may want to consider providing this type of practical support. Yes, it costs money, but it’s hard to keep up in school or benefit from extra supports like the one offered in this program if one is not there.

The other financial incentive also speaks to both a reason to invest youth and family time in the program and one of the main outcomes: postsecondary enrollment. The program sets aside $1,000 (Canadian) for each year of participation that youth can use once they graduate toward postsecondary tuition costs. Although that’s not a huge sum of money by today’s standards of tuition, one can imagine that it makes the dream of college just a bit more realistic and affordable for low-income families. However, the role that this incentive played in the post-secondary enrollment findings is not explored in the evaluation. It is unclear whether this was the driving force for getting youth to apply to college or whether the improved grades and social skills developed by the other components were the agents of change or whether perhaps it was a combination of both.

It is interesting to consider, however, that when working with very low-income youth and families, sometimes simply purchasing practical items like a bus pass, or providing some much needed tuition assistance, may really spur both deeper program engagement and impact the eventual attainment of prized outcomes.

3. Honesty about subgroup findings is a good thing.

Just about any evaluation of a mentoring program is likely to find, if it looks, that it is working better for some youth than others. Here, the evaluation of the Pathways for Education program concludes that the program worked more effectively for youth with “with higher initial measured abilities,” as indexed by youth already doing fairly well in school and at home when entering the program. The evaluators freely admit that, based on their findings, the program “does not help all students equally” and note that it appears youth need to have some moderately successful academic foundation and social capital to build on. This is a somewhat refreshing admission in the world of program evaluations.

It does, however, beg questions about use of resources. Pathways is a program that prides itself on being available to every single eligible child living in a low-income area. It takes them all. Which, although admirable, means that resource are likely spread thin and that some youth are unable to get the perhaps more intensive support they need. The evaluators estimate program costs in the report at about $5,000 (Canadian) per student per year. That’s about $25,000 for a five year participant. That seems like a large investment when the results of the evaluation indicate that the program seemed to work best for youth who had some academic strengths to build on. There is a large philosophical debate to be had about whether a program should do the greatest good for the greatest number, knowing that some youth won’t benefit because they need more support than what can be offered, or whether it should focus scarce resources on those who need it most. We won’t attempt to answer the question of whether Pathways is distributing resources optimally here. Yet, it was interesting to see the evaluation talk openly about who the program appears to resonate with best while simultaneously touting an approach that tries to serve every child. The good news is that the program has enough evidence of effectiveness enough to get rated as “promising” in this rigorous review. One wonders if they could move one step up to “effective” in their evaluation evidence with just a bit more specified distribution of resources. This is a conversation every program should have when thinking about who is best served by the way things are and by the resources at hand.

*Note: The National Mentoring Resource Center makes these “Insights for Mentoring Practitioners” available for each program or practice reviewed by the National Mentoring Resource Center Research Board. Their purpose is to give mentoring professionals additional information and understanding that can help them apply reviews to their own programs. You can read the full review on the National Mentoring Resource Center website.

¹ Oreopoulus, Philip, Robert S. Brown, & Adam M. Lavecchia. 2017. “Pathways to Education: An Integrated Approach to Helping At-risk High School Students.” The Journal of Political Economy, 124(4): 947-984. https://doi.org/10.1086/692713