National Guard Youth ChalleNGe

In considering the key takeaways from the research on this program that other mentoring programs can apply to their work, there are several features of the program and its evaluation that may be useful to consider.

The promise and risks of truly transformative mentoring.

One of the most notable aspects of the National Guard Youth ChalleNGe (NGYC) program model is that it attempts to foster a deeply transformative change in the youth they serve: high school dropouts ages 16-18 who are unemployed, not heavily involved in the justice system, and who are disconnected from meaningful direction in life. The intervention starts with a two-week preparation phase (to determine youth suitability for the program) and an intensive six-month residential phase where youth live on a National Guard base and live a quasi-military life with an emphasis on discipline, hard work, and efficient time management. This is intended to create an environment where youth can begin working on the core aspects of the program: leadership development, academic excellence, healthy behaviors and fitness, responsible citizenship, life skills, and a host of other areas in which youth are expected to grow and develop. Toward the end of the Residential Phase, the program’s participants work with staff to arrange a postresidential “placement.” Acceptable placements include employment, education, and military service.

Community mentors are identified when youth enter the program and begin working with youth toward the end of the residential phase. Their intended role is to support the youth’s transition into an educational or career path and to help to reinforce any gains youth made during their time on the base. This mentoring phase is intended to last at least one year, but many matches last well beyond this formal period.

This transformational emphasis is inherent in the theory of change behind the program model. The notion that the residential phase will allow youth to shed their negative circumstances and unhealthy behaviors during this time and emerge with new skills, strengths, attitudes, and goals is a powerful one. It’s no surprise that the program tries to influence so many aspects of a youth’s life over a rather intense half year—that’s the point, really. It’s also no surprise that the evaluation conducted by MDRCⁱ looked at a wide variety of youth outcomes over a fairly long (36 month) time period. They were trying to determine if the program really did have an impact in all these different aspects of a youth’s life and whether the overall change was truly transformative for the long haul.

However, programs that have this ambitious scope also run the risk of finding that their model is better at producing some outcomes than others. A program model trying to produce one outcome for participants often has an easier task in front of it than one trying to achieve results on many outcomes. In the case of the ChalleNGe program, it did produce statistically significant positive outcomes in youths’ employment status and earning a high school diploma or GED (with more of that being on the GED side). These are meaningful achievements for participants and the program should be applauded for these accomplishments. But there were other important intended outcome areas where the program did not have an impact on participants compared to similar youth who didn’t get the intervention: arrests, delinquent behavior, marijuana and drug use, psychological distress, engagement in civic life and productive activities. There were even some areas where the control group youth fared better than the NGYC youth (being overweight and use of birth control).

The key takeaway from all this is that NGYC had mixed results, which is not surprising when one considers the many, many aspects of a youth’s life their model is intended to change. Simply put, the findings of the evaluation indicate that the program accomplished some of its goals, but not others. This is likely true of most mentoring programs and that has certainly been the case in other high-profile mentoring evaluations.

What results like this provide a program is the opportunity to strengthen their model. Mixed findings indicate that a program might be on the right path but could also take a fresh look at its theory of change and strengthen services in some key areas. Or, alternatively, perhaps, tighten the focus of the program on the outcomes that it has demonstrated that it can reliably deliver. Any evaluation of a program will highlight strengths and weaknesses. It’s the programs that seize that opportunity to continuously improve what they provide young people that wind up growing and producing better outcomes over time.

Mentoring programs like NGYC should be applauded for taking a bold, transformative approach to working with youth and seeking to help them set a new course for their lives. Over 110,000 youth have gone through the program over the last 20 years and no doubt many of them have benefitted greatly. The mentoring field arguably needs programs like NGYC with this kind of ambitious transformative approach. But we also need an environment where programs can evaluate their models, and share both good and bad results, without it being a punitive experience. So although a review of a program’s evidence of success might find mixed effects at one point in time, the most important thing is that those results lead to improved services and better outcomes for youth in the future. This is true of every program in the mentoring field, not just a program like NGYC.

Mentoring as a reinforcement of other services.

One of the most intriguing aspects of the NGYC model is its use of mentors as a transition support when youth leave the core residential phase of the program and reintegrate back into their communities. This idea of using mentors as almost a post-intervention support has become increasingly popular in recent years, with examples cropping up in programs that re-integrate juvenile and adult prisoners back into a community, programs that work with youth aging out of the foster care system, and workforce and education programs like YouthBuild in which mentors help youth transition from the program into their career or postsecondary education path.

In the case of the NGYC program, that promise seemed to be realized. In research conducted by Sarah Schwartz, Jean Rhodes, Renee Spencer, and Jean Grossman on the mentoring aspect of NGYC, it was found that at the 21 month follow-up youth who were meeting regularly with their mentor (74% of participants) showed benefits in terms of educational attainment and engagement in meaningful activity, whereas the remaining NGYC participants showed no significant differences from the control group. At a 38-month follow-up point, the NGYC participants who were still in contact with their mentors (56%) had significant benefits compared to the control group in a range of academic, vocational, and behavior outcomes.ⁱⁱ These preliminary findings suggest that ongoing contact with a mentor in the post-residential phase was instrumental in helping the NGYC outcomes really stick.

So thinking back to the discussion above about the overall impact of the NGYC program, it’s possible that stronger implementation of the mentoring and post-residential support might have improved some of the overall program outcomes at 36 months as seen in the evaluation conducted by MDRC. In fact, that evaluation unfortunately found that the “quality of the mentoring varied from site to site,” that mentor training was “executed inconsistently,” and that there were “varying levels of adherence to the mentoring model” (p. 13). This is a great example of how an evaluation can help programs understand the importance of fidelity to their models in making their theories of change work. The research on the mentoring component suggests that it was an impactful, but not consistently delivered aspect of the program. One has to wonder what putting more emphasis on this mentoring piece might have meant for some of the program’s overall effects. But clearly, a key takeaway for practitioners is that if mentors are to be used in supporting other intervention work, their role must be clearly defined and implemented with fidelity if it’s to be as effective as hoped.

Youth-initiated mentoring is a promising innovation in the field.

The other aspect of NGYC mentoring model that practitioners should note is that most of the matches made by the program were with adults whom the youth already knew in some capacity. These “youth-initiated” mentors were nominated by the youth (sometimes with the help of their parents), trained and supported by program staff, and tasked with providing a stable, familiar presence as these youth integrated back into their communities and pursued their goals.

Because these mentors were already familiar to the mentees, they could begin from a place of trust and understanding and avoid the awkward “getting to know each other” phase. These mentors were fairly likely to be of the same race and live in the same neighborhood as the mentees, removing some of the interpersonal and logistical barriers that can get in the way of a burgeoning mentoring relationship Starting from this familiar place seemed to help with match length (over half of the youth reported that they were still in contact with their mentors over three years later!) and perceived support (mentees reported receiving social-emotional support, guidance, and instrumental help from mentors in achieving their goals). There was also some feedback that these mentors, who came into the program a month or two before youth went home, were helpful in encouraging youth make it through the demanding residential portion of the program.

To further explore this innovative way of matching mentors and youth, our NMRC Research Board recently conducted a review of the practice of Youth-Initiated Mentoring, using information from the NGYC program and others. You can learn more about this new practice in their review, which found it to be a “promising” practice with notable potential for the mentoring field.

How should mentoring programs think about the relationship between costs and impacts?

One of the other notable aspects of the NGYC program is the relatively high cost per youth served. Estimates of the cost per youth of the ChalleNGe program run from $14,000 to $15,000, placing it well above the cost per youth estimates of most program models. The reasons behind this are obvious: it costs a lot of money to house and feed a young person for six months, in addition to all of the staff costs, costs of activities and outings, and other program infrastructure.

A 2012 cost-benefit analysisⁱⁱⁱ, however, which drew on the findings of the above evaluation, estimated that for every dollar spent on the program, society could receive as much as $2.66 of benefit, mostly in the form of higher lifetime earnings that would follow if the program was able to parlay the types of favorable educational and employment outcomes seen in the evaluation into ongoing education and good lifetime employment (while also having some minor outcomes related to justice system involvement).

So how should one think about these high costs and theoretical returns on investment in light of the program’s evaluation findings? There is no right or wrong answer to that question, but policymakers and funders must think about whether such intensive models really produce benefits that justify the considerable up-front expenditures. Would a lighter intervention for more youth actually produce more cumulative return on investment? Or should a program like this be reserved for youth with even higher levels of risk, such as those most likely to wind up in our wildly expensive prison system, rather than being directed at youth who have given up on high school for a variety of reasons but who may not be at notably elevated risk for involvement in crime? There are no clear answers to these questions, but high per youth costs will almost always invite scrutiny, especially when evaluation results show mixed impact.

Is a military-style intervention right for everyone?

One last aspect worth thinking about regarding the NGYC program model is the militaristic approach that it takes to transforming the lives of the youth it serves. It’s interesting that the program rests on the assumption that what these youth need is to be removed from their communities (voluntarily) and placed into a rigidly structured quasi-military environment in order to build needed discipline and skills. The removal from a harmful situation is the inherent idea behind any kind of residential program, such as a residential drug treatment or mental health intervention. It is interesting to see it applied here, however, to dropouts and a population of youth that, while struggling with some aspects of life, would not generally be regarded as being in eminent danger.

To the program’s credit, it does offer a two-week preparation phase in which youth are tested to see if they are up for the rigorous program and strict living situation that lies ahead. So the program does recognize that not all youth would be in a position to benefit from such an intensive intervention. The program is also voluntary—youth have the ability to leave the program if they choose.

But it’s hard not to compare the approach of this program with the notion of a military “boot camp” in general. The youth get their hair shaved off and live in barracks for six months, while receiving services provided by military personnel. There is an emphasis on discipline and self-control and a number of military-style physical activities like marching in formation and running obstacle courses as if training for military deployment. Although this approach has been proven to produce effective soldiers, it remains unclear if it is the right approach to producing effective young adults. Not many practitioners in the youth development space would argue against teaching youth to have more discipline, better coping skills, more respect for authority, and a stronger commitment to service and community. But military culture and civilian culture are distinct and individuals exposed to both can often struggle to move back and forth between those two worlds.

The results of the NGYC evaluations to date could be taken to suggest that its military-infused approach may not be the best fit for everyone. Some youth may need that level of intensity in an intervention; but for others, the removal from home and community and the deep emersion in a distinctly different (and deeply authoritarian) culture may prove too jarring or simply be too overwhelming. Hopefully future research can shed some light on which types of individuals thrive most in a military-style intervention such as this.

---

*Note: The National Mentoring Resource Center makes these “Insights for Mentoring Practitioners” available for each program or practice reviewed by the National Mentoring Resource Center Research Board. Their purpose is to give mentoring professionals additional information and understanding that can help them apply reviews to their own programs. You can read the full review on the Crime Solutions website.

For more information on research-informed program practices and tools for implementation, be sure to consult the Elements of Effective Practice for Mentoring™ and the “Resources” section of the National Mentoring Resource Center site.

---

ⁱMillenky, M, Bloom, D., Muller–Ravett, S., & Broadus, J. (2011). Staying on course: Three-year results of the National Guard Youth ChalleNGe evaluation. New York: MDRC. http://www.mdrc.org/publication/staying-course

ⁱⁱSchwartz, S., Rhodes, J., Spencer, R., & Grossman, J. (2013) Youth initiated mentoring: Investigating a new approach to working with vulnerable adolescents. American Journal of Community Psychology, 52(1-2), pp.155-69.

ⁱⁱⁱPerez–Arce, F., Louay C., Loughran D., and Karoly, L. (2012). A cost–benefit analysis of the National Guard Youth ChalleNGe Program. Santa Monica, CA: RAND Corporation. http://www.rand.org/pubs/technical_reports/TR1193.html