Promotor Pathways Program
*Note: The National Mentoring Resource Center makes these “Insights for Mentoring Practitioners” available for each program or practice reviewed by the National Mentoring Resource Center Research Board. Their purpose is to give mentoring professionals additional information and understanding that can help them apply reviews to their own programs. You can read the full review on the CrimeSolutions.gov website.
In considering the key takeaways from the research on this program that other mentoring programs can apply to their work, it’s useful to reflect on the features and practices that might have influenced its rating as “No effects” (that is, a program that has not shown definitive evidence of effectiveness)…
When looking at the Promotor Pathways program, we see an effort that, at least on paper, seems well-positioned for success. The Latin American Youth Center (LAYC) is a robust youth-serving organization serving immigrant and other high-needs communities in the greater Washington, DC area with a wide variety of services. This organization has attracted significant investment and community support over the years and is well-known as a leading service provider in the DC region. The Promotor program was started because youth themselves who had engaged in the many other services and supports offered by LAYC had told the program again and again that it was a relationship with a key member of the staff that had helped them stick with whatever services they were engaged in and ultimately find success. So rather than this program being driven from an adult perspective, the impetus to start it came directly from the young people being served.
The Promotor Pathways program also has the advantage of being well-funded with a lot of time and energy put into the design and implementation of the program by staff members and investors. The evaluation cited in the full Crime Solutions review was part of an almost $1,000,000 effort over several years to expand the size and scope of Promotor Pathways. This work, funded through the Social Innovation Fund, was built on evidence of promise for effectiveness of Promotor (a requirement for SIF funding) and was ambitiously intended to “lead to local government savings of $16 million over the next 10 years.”
So here we have a program that youth themselves said would be beneficial, housed in a thriving youth service setting, with substantial investment to do the work “right.” And yet the evaluation here indicates that local government probably didn’t see $16 million in savings as the program was found to not have much impact. So what went wrong?
1. The plural of anecdote is not data.
One thing to consider is the origin story of youth saying that a relationship with a staff member is what kept them going. No doubt the young people who said that to the program had that experience—they genuinely felt like that caring adult on the staff made them try harder, overcome the urge to give up, and keep coming back to LAYC as a source of multi-faceted support. But what happens when those are the only voices being heard on the issue? What percentage of youth served by LAYC felt that way or had that experience? Perhaps the “mentor-rich” experience of those youth was a rare circumstance, or only happened because of some other factor in those kids’ lives (including, simply, who they are as people). That “mentors kept me going” experience may have actually been quite rare or at least not fully understood.
For example, it’s also worth noting that the Promotores in the evaluated program were paid mentors who were unaffiliated with any of the other services at LAYC. It may be that those relationships really mattered when the relationship was with the staff who were already engaged in the direct services. Perhaps a “service neutral” mentor didn’t really connect with youth or seemed like more of an add-on that an integral part of the core LAYC services. It’s hard to say. But this serves as a bit of a caution to practitioners to take the preferences and ideas for service improvement of clients with a bit of a grain of salt. You may be getting input from only one vocal group while missing something important that might hinder your efforts if you build what is being asked for without deeper investigation. That may not have been the case here, but it’s worth noting that the genesis of this program was grounded in feedback that may have been biased in a particular direction.
2. That being said, the paid mentor aspect of Promotor Pathways seems very well thought out and executed.
There has been a lot of consternation about and interest in paid mentoring models over the last decade plus. Many have seen the struggles of volunteer-led efforts to connect mentors and youth for longer periods of time and with the intensity of trust and mutuality that we would hope for in most mentoring relationships and wondered if making the mentor a paid employee might not yield better results. Certainly our NMRC program reviews have found examples of paid mentoring being both successful and not (and meta-analyses by DuBois and colleagues have found no difference in effectiveness between paid and volunteer mentoring). What is striking about the Promotor program is that it seems to have many of the program elements in place that would lead to success. With this in mind, practitioners would be wise to note how the paid role is structured within LAYC, in spite of the lack of overall effectiveness in this one evaluation of the program to date. The details of this role seemed well-positioned for success
- 30 hours of training for the mentors on every conceivable topic for working with such a high-risk population of youth.
- 24/7 access to the Promotores, including via text, email, and phone. These paid mentors were tasked with literally always being there for the youth with whom they were matched.
- A decent mentor:youth ratio—The evaluation notes that there are about 15 Promotores currently in the program and this project served 165 mentees for the evaluation, which is a roughly 1:10 ratio. This seems reasonable given that these mentors had no other programmatic responsibilities.
- The mentor’s expected role included meeting with parents and other important adults in the youth’s life on a regular basis to coordinate care and make sure needs were being addressed.
- A meeting schedule that allowed for more frequent contact—the average pair in the program met 3.5 times a month, with nearly two-thirds of those meetings being in-person.
- Very low attrition of Promotores—only three of the mentors are reported to have ever left this role in the program, meaning that youth are not frequently being shuffled from one paid relationship to the next.
- As noted previously, the role is unaffiliated with other LAYC services and is intended to be focused solely on solving problems and helping youth get the most out of all the other things LAYC is offering.
- Long-lasting matches—All of this structure paid off with matches, at least in the evaluation study, that lasted an average of 15 months in the 18 month study window.
In addition to looking good “on paper” these relationships seemed to be successful in terms of relationship tone and closeness. Sixty percent of mentees felt “very close” to their Promotor, with another 30% feeling “somewhat close.” Nearly 90% felt like they could talk with their Promotor about a personal problem and 85% said they didn’t feel like their mentor was trying to control their lives but was, rather, simply a source of support. So there were many, many things about how this mentor role was conceived and implemented that this program got right. These relationships, and the role of the mentor itself, seemed well positioned for success.
3. The best laid plans don’t always work out.
Ultimately, though, the mentored youth in the evaluation did not really benefit from the experience in comparison to non-mentored LAYC participants as much as one would likely expect. In hindsight, the authors of the report note that 18 months may not be enough time to form a truly deep and lasting bond that can overcome what were some major risk factors for the youth being served. And that may be true in this case. But it’s also a bit of a bleak conclusion as most youth-serving organizations are unlikely to have youth, especially youth in the older age range of this program, engaged in their services for much longer than 18 months. And we have seen other examples of mentoring services reaching high-risk youth in some meaningful ways in similar timeframes.
There may be things LAYC can do to strengthen the program—connecting the work of the mentors to other staff to make sure that all staff providing direct services are able to coordinate information about a youth and make for a more fully “mentor-rich” environment; involving the family and other caring adults outside the program to an even greater degree; strengthening referrals to other service providers.
But there are two things that really stand out as potential next steps:
1. Do another evaluation, if possible.
This evaluation only compared Promotor participants to other LAYC youth who did not have a mentor. So all of the youth in this study were getting a very robust suite of services and supports. The mentoring was designed to mostly be a “boost” that would take the results from “good” to “even better”… That’s a tough window to hit and we have seen other reviewed programs struggle to show outcomes by not including a control group of youth who didn’t get any services at all. It might be that the Promotor participants would fare pretty well in comparison to similar youth receiving no LAYC services.
2. Perhaps target the program to a specific group of youth.
One of the interesting things about the evaluation is that youth who had a child of their own were much more likely to meet frequently with their mentors a high number of times (four times more likely to have more than 45 meetings). Higher levels of mentor-mentee interaction were associated with stronger outcomes. Based on these trends, one could argue that this service might be more impactful and efficient if directed at those most likely to meet with their mentors frequently and get a wider range of support. The study authors note that these youth likely had more needs because they were juggling parenthood in addition to engaging in the other LAYC services. But it might also be that being a parent spurred those youth to realize that they needed to take advantage of the supports and help being offered, in essence perhaps a bit of a “wake-up” call that made them want to engage in services even more than their non-parent peers. Regardless of the reason, one wonders what an evaluation might show if this was directed solely at LAYC youth who were parents, giving them extra support as they worked hard to care for both themselves and their child.
For more information on research-informed program practices and tools for implementation, be sure to consult the Elements of Effective Practice for Mentoring™ and the "Resources for Mentoring Programs" section of the National Mentoring Resource Center site.