Early Start to Emancipation Preparation – Tutoring Program (ESTEP)

In considering the key takeaways from the research on this program (ESTEP) that other mentoring programs can apply to their work, it’s useful to reflect on the features and practices that might have influenced its rating, as well as some of the more interesting aspects of the program design.

When is mentoring not “mentoring?”

Most mentoring practitioners reading the Crime Solutions profile for this program will note that this program didn’t offer all that much mentoring compared to the other services offered. Although the tutors and youth certainly did spend considerable time together (40 hours of services over an average of 22 weeks), the average youth in the study got only five hours of formal “mentoring” time with his or her tutor/mentor. In the case of this particular program, that may have been a perfectly fine allotment of time⎯after all, the goals of this program were largely around improvements in academic skills and the program literally called itself a tutoring program. So it’s quite possible that the absence of positive outcomes for these particular students derived entirely from, say, a weakness of the tutoring curriculum and materials or some other aspect of the program that had nothing to do with the mentoring work that was happening in a largely ancillary and loosely defined manner.

We’ve seen other reviews of mentoring programs that have limited mentoring time (often amidst other program components and activities) determine that the program had less impact than one might expect, so perhaps this is unsurprising. Programs that are embedding mentoring within a suite of other supports should consider whether enough time and attention is being provided to the mentoring aspect. The fact that only 15% of the tutor-student relationships blossomed into the longer-term mentoring relationships the program was expecting perhaps serves as evidence that this aspect of the work was underemphasized. Mentoring treated as a limited (or even optional) add on seems unlikely to contribute much to program outcomes.

Setting and personnel matter

A few other factors about the program may have contributed to the lack of positive impacts for ESTEP:

1. The tutors tended to meet with their students in their foster homes or other system-provided residential settings. It’s possible that these students may have taken to the program more if it was designed to get them out of their foster homes and into the community. Foster homes, although in theory offering a safe environment to youth who need one, may not be a foster youth’s most favorite place to hang out. One wonders if simply changing the venue of the tutoring session to more of a “neutral” location may have put youth more at ease, allowed them to focus more fully on the work, and exposed them to new places and people (and perhaps a little more fun).
2. The program used community college students as the tutors. Although there is nothing inherently wrong with using these students in a tutoring/mentoring role, one wonders how well they were able to connect with the youth with whom they were matched. These youth were all older youth who had been exposed to some kind of trauma or neglect and, thus, may well have faced increased challenges in forming a strong working relationship with the program’s personnel. One wonders if these community college mentors had enough life experience, wisdom, and training to navigate the potential pitfalls of working with these youth. They may not have had a similar shared background and perhaps struggled to build that almost-instant rapport that individuals with a similar life history can develop. There are other programs serving foster youth that prioritize mentors who have experience in the foster care system themselves, and one wonders if that would have made a different here.

Implementing and evaluating programs for youth with so many needs is a challenge

ESTEP seemed to do a great job of targeting its services to the individuals they thought they could best reach and offering a well-defined set of services in a limited timeframe. The design of this program makes a lot of sense on paper. But as the evaluation of the program showed, it can be hard to keep everything so neat and tidy when a program meets the reality of the needs of higher risk youth and a diverse landscape of service providers. The researchers noted that 38% of the youth in the intervention group actually never did the tutoring for a variety of reasons. Additionally, 12% of the youth in the control group received some tutoring from other service providers, including a massive tutoring intervention led by the foster care system itself that coincided with this evaluation. Those two factors, along with an intent-to-treat evaluation design that tracks everyone, regardless of their participation, may have eliminated the detectability of any positive outcomes the program produced. The evaluators also noted that the age ranges and levels of reading comprehension of those served by the program fell outside of the anticipated sweet spot for which the program was designed. The evaluation also did not report much on whether the effects of the program varied as a function of different characteristics of the youth participating in the evaluation. As such, it’s hard to tell if certain subgroups of youth (e.g., those for whom the program was intended) benefitted from the program, while others did not. Just more evidence that doing this work in the real world is often messier than even the best program and evaluation designs can predict.

---

*Note: The National Mentoring Resource Center makes these “Insights for Mentoring Practitioners” available for each program or practice reviewed by the National Mentoring Resource Center Research Board. Their purpose is to give mentoring professionals additional information and understanding that can help them apply reviews to their own programs. You can read the full review on the CrimeSolutions.gov website.

For more information on research-informed program practices and tools for implementation, be sure to consult the Elements of Effective Practice for Mentoring™ and the “Resources for Mentoring Programs” section of the National Mentoring Resource Center site.