*Note: The National Mentoring Resource Center makes these “Insights for Mentoring Practitioners” available for each program or practice reviewed by the National Mentoring Resource Center Research Board. Their purpose is to give mentoring professionals additional information and understanding that can help them apply reviews to their own programs. You can read the full review on the Crime Solutions website.
In considering the key takeaways from the research on this program that other mentoring programs can apply to their work, it’s useful to reflect on the features and practices that might have influenced its rating as “Promising” (that is, a program that has strong evidence that it achieved some of its justice-related goals).
1. It always helps to provide some follow up to ensure your initial success is maintained.
One of the interesting aspects of the Bottom Line model is that in addition to providing robust college planning and access services, they maintain a presence at several regional colleges and universities that tend to be the types of schools they often encourage youth to attend. These higher education institutions are well-aligned with the clients of Bottom Line as they offer a good combination of relative affordability, low dropout rates, and good academic reputations. And while some of Bottom Line’s outcomes on college acceptance and persistence beyond freshman year are presumably the result of helping students pick a good fit in the first place, the fact that they still have access to Bottom Line counsellors and supports even after they get on campus may well be a major reason why their college retention results look better than those of other services. Although the bulk of the work done in their model is done during the core application and acceptance process, the program seems to have added a valuable secondary component that helps ensure that their mentored youth get ongoing support for making progress toward the ultimate goal of college graduation.
We have seen other examples in our reviews over the years of programs using mentors or ongoing relationships to help maintain progress youth have made under a core set of services. One example can be found in the National Guard Youth ChalleNGe program, where youth are offered community mentors (that they help identify) as a way of maintaining the positive path they are on when they leave the residential portion of the program. Research on the program indicates that youth you stick with that mentor for several years after the initial residential experience tend to be maintaining their positive direction in a number of ways compared to youth who did not get a mentor or who stopped seeing them after some time. This is a good reminder to programs that, when possible, a little follow-up work can go a long ways after your initial services are complete. Other programs may be able to produce similar long-term trajectories through the use of alumni groups, occasional program “check-ins”, and ongoing training and learning opportunities once youth leave the core services.
2. Fidelity of implementation pays dividends.
One of the most helpful aspects of the evaluation work done by Carr and Castleman on behalf of Bottom Line is that they examined not only outcomes of the program, but also factors that may have led to those outcomes. This is particularly helpful when trying to understand why the program seems to have outperformed other college access programs and services in which youth in the control group may have participated. Among the factors that the evaluation looked at was the frequency and consistency of meetings with students. Most of the advisors in the program met with their students an average of almost once every month for 15 months, with a surprisingly high percentage of those being in-person office visits. We have reviewed other college access mentoring programs that offered a lighter touch with more emphasis on phone and text check-ins. Bottom Line seems to have emphasized the consistency and follow through of their meetings with students throughout the process and, not surprisingly given this, the students rated the influence of their advisors very highly. This might not have been the case if they hadn’t met with such regularity and at key points in the application process.
The consistency of service delivery was further examined in the evaluation with an analysis of the results of each advisor individually. This was designed to see if some advisors were more effective than others and if the overall good results were being driven largely by one or two “superstar” performers. Turns out that 19 of the 20 advisors had a net positive estimated outcome for the students they served compared to the control group students. (One has to wonder what the post-evaluation conversation was with the lone advisor whose students fared considerably worse.) Even more impressive is that in Bottom Line there is very little deliberate matching of advisors with youth who might be a “good fit” based on shared backgrounds, gender, or interests. The program essentially “randomly assigns” youth to any and all of the mentors, something they could only do if they had confidence that each advisor was able to faithfully walk each student through the activities and could form a strong working relationship with a wide variety of participants.
We put a lot of emphasis on matching mentors and youth based on surface-level similarities and interests in the mentoring field. But programs like Bottom Line that have a very clear set of goals and a structured approach to getting the young person from “point A” to “point B” should arguably really emphasize the consistency of the experience from mentor to mentor and work to ensure that regardless of who they are matched with that youth will get a positive relationship that hits all the critical tasks in their work together. Bottom Line likely achieved this fidelity of implementation through strong advisor training and by monitoring the progress of each match through the program. But now that they know their model can be delivered by many types of individuals, for a diverse array of students, with very little inconsistency, they will have an easier time replicating these results in other locations. They have learned that for their program the process is, in many ways, as important as the people.
3. Sometimes, all you need to do may be to change how young people are thinking about a situation.
In addition to finding that the program was producing positive results in most of the areas examined, the Bottom Line evaluation shed some light into how the program was influencing young people. Most notably, Bottom Line students were much more likely than control youth to indicate that affordability was a major factor in their application decisions. The Bottom Line model has several stages where cost-information is highly prioritized: when considering potential schools to apply to, when reviewing acceptance information and financial aid offerings, and in ultimately deciding where to attend. Other college access models also review these types of considerations, but the information provided by control youth indicated that this financial review was less emphasized than it was in Bottom Line and that they ultimately didn’t consider affordability as much when applying and enrolling. By getting these students to think a bit more deeply about the intersection of school quality and school cost, they helped them prioritize choosing a school that they could afford and attend for all four years of college, something that many students, especially those from lower-income backgrounds as in Bottom Line, tend to really struggle to make into a reality.
Other mentoring programs should think about what information might be critical to helping mentees reach their goals within the context of their programs. They may find that one of the best ways to achieve those goals is by encouraging mentors to focus on providing key information and helping youth enhance their ability to think about important factors in their decision-making. The outcome of a decision can only be as good as the thinking and information that informed it, and Bottom Line seems to have realized that their theory of change for students has some clear points where the influence of a mentor helps youth plan for their futures more effectively.
For more information on research-informed program practices and tools for implementation, be sure to consult the Elements of Effective Practice for Mentoring™ and the "Resources" section of the National Mentoring Resource Center site.