Brief Instrumental School-Based Mentoring Program – Revised
*Note: The National Mentoring Resource Center makes these “Insights for Mentoring Practitioners” available for each program or practice reviewed by the National Mentoring Resource Center Research Board. Their purpose is to give mentoring professionals additional information and understanding that can help them apply reviews to their own programs. You can read the new full review on the Crime Solutions website. This review is an updated version of the program based on a fresh evaluation of the program. You can read the previous version’s insights here.
In considering the key takeaways from the research on this program that other mentoring programs can apply to their work, it’s useful to reflect on the features and practices that might have influenced its rating as “Promising” (that is, a program that shows some evidence that it achieves justice-related and other goals when implemented with fidelity).
1. The value of testing program improvements over time.
One of the best pieces of advice for youth mentoring programs is that they should always be tinkering with their approach and testing the results to see if aspects of the program can be improved. This can be as simple as adding fresh content to mentor training, beefing up match support check-ins, or directing mentor recruitment toward specific types of individuals because they might be a better fit for the role than previous mentors. Any good program will constantly be looking for subtle ways to make what they do better. And in the case of the Brief Instrumental School-Based Mentoring Program, we have a great example of the type of payoff that can happen with a program takes this kind of iterative approach to the development of the mentoring services over time.
An earlier version of this program was reviewed for Crime Solutions and found that have “No Effects,” meaning that it did not produce meaningful outcomes in many of the main categories in which it was trying to support youth. But as noted in the “Insights for Practitioners” for that iteration of the program, this is a program that has constantly evaluated and revised in a continuous improvement approach. In fact, the earliest version of the program showed evidence that it may have actually produced some harmful effects for youth. So the developers improved the training and content of the program and evaluated it again, finding that they had eliminated the harmful impacts but not quite reached their goal of meaningful positive outcomes.
So they revamped and tried again, once more emphasizing rigorous evaluation to see if their improvements worked. This version of the program added a host of improvements: the provision of a mentee manual, revised mentor training and supervision, more choice for mentors on match activities, and e-training and support. And these changes again paid off in a measurable way.
This figure, taken from the report of the evaluation conducted by McQuillin and Lyons, shows the notable improvements that are evident across the three versions of the program. In each of the outcome areas listed along the X axis, we can see that this latest version of the program outperformed the earlier versions, sometimes markedly, with several outcomes having effect size (level of impact) that are consistently well beyond the average of around .20 found in the last comprehensive meta-analysis of mentoring program effectiveness. This Figure is a powerful example of why we need to allow mentoring programs to have some missteps along the way so that they opportunity and time to try new ideas and improve weaker aspects of what they are trying to do. Unfortunately, many programs lose support and funding after an evaluation that shows poor or even less-than-desired results. Others may develop internal cultures that unfortunately do not support questioning current practices and thus miss out on opportunities to improve through innovation and evaluation. Policymakers often view programs as “working” or not, when the reality is that most programs need time to work out the kinks and a chance to find better ways of delivering their services. The story of this program really highlights why practitioners need to be constantly trying to improve what they do and why program stakeholders need to exhibit some patience as the program works toward its “best” design.
2. The more complicated the mentors’ task, the more supervision and support they need.
Among the many improvements that were made in this version of the Brief Instrumental School-Based Mentoring Program, perhaps none is more notable than the bolstering of the supervision and support offered to mentors. Although the evaluators of the program did not test to see if this enhanced supervision was directly responsible for the improvements in the program’s outcomes, it is quite plausible that this played a meaningful role.
For this version of the program, mentors were not only provided with considerable up-front training but also a wealth of support and guidance in delivering the curriculum of the program and integrating it into the relationship that each mentor had with his or her mentee. Keep in mind that this program asks mentors to engage in a number of highly particular conversations and activities with their mentees that build in a sequential fashion and provide youth with very specific skills and ways of thinking about themselves and their academics. Mentors are asked to engage in motivational interviewing, apply cognitive dissonance theory, and teach academic enabling skills. Doing these things well requires adhering closely to a set of phrases, behaviors, and talking points and the developers of this program didn’t want to simply do a training and leave that to chance (as they did in the past).
Before each session, mentors would meet with a site supervisor who would review the curriculum for that particular session, reinforce keys to delivering the content well, and answer mentor questions. After each session, mentors again met with site supervisors to go over their checklists of actions to see if they had covered all of the content they were supposed to during the session. Mentors were also encouraged to engage additional phone-based support with their supervisor as needed and were provided with a manual that they could refer to at any time to get more familiar with the content and practice the types of key messages they were supposed to be delivering to mentees.
The type of intensive “just-in-time” mentor training and supervision offered by this program is likely beyond what most school-based mentoring programs can offer their mentors with their current resources (it’s also worth noting that most programs are simply not asking mentors to engage in activities that are this tightly controlled and specific). Just about any program could, however, borrow the concept of pre- and post-match checkins with a site coordinator as a way of boosting program quality. These meetings could be brief but critically important to reviewing what the match is focusing on, sharing information about what the mentee has been experiencing recently (at school or at home), and reinforcing key messages or talking points that have the potential to either help make the relationship stronger or offer more targeted instrumental support. As the evaluators of this program note, “One persistent problem in SBM intervention research is the confusion surrounding what occurs in mentoring interactions and relationships,” which results in not only challenges in improving the program but also in helping others to replicate proven mentor strategies in other programs. Using the kind of rigorous pre-post mentor support used by this program might allow programs to understand what mentors and mentees are doing when they meet and make sure that important aspects of the program are delivered with fidelity, even if those important aspects are not as complicated as those in this particular program.
3. Remember to borrow evidence-supported practices and ideas from other youth development services.
We have frequently encouraged mentoring practitioners in these “Insights” pieces to borrow ideas and tools from other youth serving interventions, both mentoring and beyond. The Brief Instrumental School-Based Mentoring Program-Revised offers a great example of this in their sourcing of the “academic enablers” used by mentors. Rather than inventing a new set of tools and techniques, the program simply borrowed and adapted the materials from an intervention for children with ADHD that used coaches to help improve academic skills. The materials had already been used with good results in a short-term intervention, meaning they would fit into the timeline of the mentoring program. Their successful use with ADHD students previously also meant that they would be a likely good fit here with any mentees that had learning disabilities. So rather than reinventing a suite of activities to teach mentees about agenda keeping, planning, and organization skills, the program simply adapted something that already existed and fit their school-based model. This type of adaptation can save staff time and resources, while increasing the odds that the program is doing something that will be effective.
4. Don’t forget that mentoring programs are relationship programs.
The Insights about the earlier version of this program noted that there may have been challenges in developing relationship closeness given the program’s brief duration (8 sessions) and the highly scripted nature of the interactions (lots of curriculum delivery, not a lot of fun): “One wonders what the results might have been if this program had emphasized the ‘brief’ and/or the ‘instrumental’ just a bit less or otherwise figured out a way to give the participants more time to build a stronger relationship rather than an ‘alliance’.”
Well, this iteration of the program did exactly that by emphasizing more play, games, and freedom in session activities (provided that the core intervention was completed). As explained by the researchers: “the SBM program described in this study includes a variety of activities designed to promote a close relationship between the mentor and mentee because the quality of the mentoring relationship is theorized to be critical for helping mentees achieve their goals. Within an SBM context, brief effective instrumental models of mentoring that also include activities to develop close relationships may be an ideal ‘launch-pad’ for SBM programs with a developmental model to extend the period of the brief mentoring beyond the brief of mentoring.”
This serves as another example of this program learning from a prior attempt and doing something better (in this case strengthening the relationships themselves). It also provides some food for thought for any school-based mentoring program. Given that many school-based programs are brief and focused on fairly instrumental pursuits, how can these programs not only strengthen the relationships but also keep those relationships going once they get strong? It seems a shame to have a program forge new and meaningful relationships, only to use them in service of something that is, by design, short term. School-based programs should consider partnering with other programs or find other ways of using a “brief instrumental” program as a testing ground for relationships that can transfer to another program or setting if they “take root.” The developers of this program note that the ideal goal of school-based mentoring might be to “increase the immediate impact of mentoring on student outcomes and promote long-term mentoring relationships desired by developmental models of mentoring.”
It is unclear how many of this program’s brief matches (if any) went on to have longer-term engagement outside of the program. But all school-based programs should be asking “How can we keep these relationships going once they have tackled the targeting things we ask them to achieve?” Without exploring that, we may minimize the value of school-based mentoring and deny mentees and mentors the opportunity to grow something brief into something quite monumental.
For more information on research-informed program practices and tools for implementation, be sure to consult the Elements of Effective Practice for Mentoring™ and the "Resources" section of the National Mentoring Resource Center site.