Patrick Boyle is the editor of Youth Today, and author of A Father's Place, a parenting column in Maryland's Gazette newspaper chain and on several Web sites. A 24-year newspaper and magazine veteran, Boyle has extensive experience covering youth issues as a reporter for The Watertown Daily Times and The Washington Times, and as a free lancer for the Washington Post, Baltimore Sun, Newsday, Child and Parenting magazines and ABC News, among others. His book, Scouts' Honor: Sexual Abuse in America's Most Trusted Institution, examined child molestation in the Boy Scouts of America. He has also served as senior editor of Car & Travel magazine and spokesman for AAA. He has a Master's degree in journalism from the University of Maryland.

Introduction: Activism in Action

Anyone who works with youth or advocates on their behalf knows all too well the hot words of the day: Evidence . . . Evaluations . . . Best Practices. Everyone wants to know what specific approaches to youth work make a measurable difference.

For more than a year, Youth Today has devoted extra time and space to evaluations of youth work practices. The objective has been to draw out the kinds of answers that people can apply to their work with youth. Those answers have been both encouraging and disappointing.

Every agency needs more mentors—so do mentors stay around longer and do a better job if they get paid? Even troubled kids love to play with and care for animals, but is there any evidence that working with dogs, horses and mules has a lasting impact? How about self-esteem classes? Has anyone found a way to consistently get older teenagers to come to and stick with after-school programs? And what would it take for your agency to carry out the Quantum Opportunities Program—one of the most successful youth interventions of recent times, and also one of the most expensive and difficult to operate?

Some of those answers are in the following stories, which look at how youth work practices are carried out, how they succeed and why they sometimes falter.

For site visitors who wish to click on a specific article, you can click on the article's title. If you prefer, you may scroll down to read all articles at once.

Mentoring Pays Off
Paid mentors do more than pal around—
but do evaluations show that they're worth the money?
By Patrick Boyle

An Evaluation of Volunteers Courts Controversy
Study of appointed advocates for children produces some surprising results
and a challenge for the group that asked for it.
By Barbara White Stack

Youth Work's Greatest Tails
Workers see big changes in troubled kids who work with animals,
but no one can dig up research on long-term impacts.
By Patrick Boyle

Getting Down on Self-Esteem
As research doubts value of "feel good" lessons,
programs shift to helping kids earn a sense of "self-efficacy."
By Madeleine Levin and Diana Zuckerman

Attracting Teens
They're a rare site at many after-school programs. A Boys & Girls Club experiment shows how to draw them in—then struggles to keep them interested.
By Dick Mendel

Federal Youth Programs And Policy Taken to Task
White House report calls for better evaluations to help decide
which programs to fund—and which to drop.
By Patrick Boyle

The Best Youth Program You Can't Afford
Struggles to copy QOP offer sobering lessons about best practice research
and 'this whole thing called reality.'
By Patrick Boyle

	If you prefer to read this article separately from the other articles listed in this section, please

Pay-As-You-Go Youth Work
Evaluations differ on the value of giving kids cash and other rewards to participate in youth development programs.
By Martha Shirk

On the last day of a month-long summer program in life skills, 14-year-old Shedrick Milton left the cozy office of the San Francisco Independent Living Skills Program with a couple of gift cards, some cookbooks and what amounts to money in the bank.
Shedrick, who is in foster care, can take more skill-building classes during his leisure hours throughout the school year, earning more gift cards and credits toward a stipend that could reach $2,500 by the time he graduates from the program in four years.

He is among thousands of youth around the country who receive financial or material rewards for taking part in youth development activities. Agencies use such incentives to motivate youth to avoid pregnancy, do their homework, stay in school, master life skills, train for jobs, save money, and serve on boards and advisory councils.
Until last year, a retired Nashville couple was paying $10 to every child who memorized the Ten Commandments. Twenty thousand children claimed the reward before the money ran out.

While the concept seems to be gaining popularity, only a handful of incentive-using youth programs have been evaluated. And because researchers have rarely separated the impact of the incentives from other program components, both proponents and opponents of the practice can cite findings to support their views.
In a paper that was widely distributed last year, respected researchers Robert J. Ivry and Fred Doolittle of MDRC, a nonprofit social research organization, argued that three decades of research prove that financial incentives solve the biggest problem most youth programs face - "namely, maintaining high levels of engagement, participation, and retention."

Ivry and Doolittle urged youth-serving agencies to make greater use of "participation bargains" that reward at-risk youth for acquiring specific competencies or achieving self-improvement benchmarks, such as abstaining from alcohol or drugs, earning academic credentials or completing training.

But some studies of specific programs suggest proceeding with caution. A study of a Canadian job training program for school dropouts found that incentives drew many youth who just wanted the money. A study of a short-lived pregnancy prevention program in Denver called Dollar-a-Day, based at a hospital, found that incentives got girls to show up, but did not reduce repeat pregnancies. And one evaluation of Quantum Opportunities Program (QOP) for youth at risk of dropping out of school found that incentives lost their effectiveness over the long term.

While cash incentives are commonplace at independent living programs for foster youth, some program administrators think the practice has gone too far and have switched to providing goods, such as movie passes or household items.

Researchers at the Harvard Family Research Project recently reviewed studies on incentives in youth programs and concluded that they "deserve greater attention as a strategy for improving youth's out-of-school-time program participation." The researchers stopped short of endorsing incentives.

"It does appear that it may make a difference in getting teens to participate in programs if you offer them incentives, but there is no research evidence to support this - only common sense," says Priscilla Little, the Harvard project's associate director.

Before integrating incentives into their programs, youth-serving agencies might want to consider the following questions:

Do incentives get youth to come? Arlene Hylton sees incentives working every day in the independent living program she runs for youth in San Francisco's foster care system. Incentives attract youth who would otherwise be reluctant to spend their leisure time in activities that adults think are good for them. "And once we get them here, they're hooked," she says.

The research validates Hylton's observations. The most recent evaluation of QOP - by Mathematica Policy Research - concluded that the hourly stipends "induced newly enrolled youth to attend program activities."

Incentives were also found to lure participants to Integrated Training Centres for Youth, a Canadian job-training program, as well as to the hospital-based pregnancy prevention program, Dollar-a-Day.

Do incentives keep youth coming?

Whether youth keep coming for the incentives or because they've gotten hooked is hard to determine.

For 10 years, Planned Parenthood of the Rocky Mountains has been attracting girls to its pregnancy prevention program in Denver with a $1-a-day payment for avoiding pregnancy. (This is different from the Dollar-a-Day effort that was run by a hospital.) After a dozen or so sessions, "The kids seem to forget about the money," observes Krista Anderson, the agency's vice president of education and training. "Sometimes they even leave without asking for it. What becomes the primary focus is really the knowledge and skills and the relationships they're getting from the program."

Two studies of QOP sites provide somewhat contradictory answers to the question of whether incentives keep youth coming. A study of the QOP pilot project, by researchers at Brandeis University, concluded that "financial incentives can be effective in maintaining student interest in and attendance at program events."
But a later study of the QOP demonstration project, by Mathematica, concluded that the incentives became less of a lure over time. Responding to that finding, plus observations from staff and participants, the Eisenhower Foundation last year increased the incentives to $1.25 or $1.40 an hour (from $1) at the six sites where it is replicating QOP. The foundation expects the higher payment to make the program more attractive to older youth, those whom the Mathematica researchers found most likely to stop participating.

Incentives were found by researchers to be important to maintaining participation in the Canadian job-training program, Integrated Training Centres. At one site, 37 percent of participants told evaluators they would have quit without the incentives.
Even with the incentives, however, the drop-out rate was high - between 22 percent and 37 percent, depending on the site. The evaluators attributed that primarily to an inadequate screening program. They concluded that attracting youth who are "there 'just for the money ...' is to some extent unavoidable, but it further demonstrates the difficulty the agencies face in having incentives work in the way they were intended."
At the San Francisco Independent Living Skills Program, Hylton changed her incentives several years ago, partly to keep youth coming back. The program used to pay youth $50 quarterly to attend life skills classes. Now, it combines immediate rewards, such as gift cards to stores like Old Navy, Target and T.J. Maxx, with the prospect of a longer-term payoff: the graduation stipend, which rises with the amount of effort a youth puts into preparing for independence from foster care.

"We keep telling them, 'the more you participate, the bigger your check will be,'" Hylton says.

One of Hylton's students, Cassandra Mitchell, 18, admits that it was the promise of a reward at the end that got her to keep coming to financial literacy workshops last summer, even though she didn't always find them interesting.

When she completed the six classes, she received a check for $100, plus an Individual Development Account that will match her savings for education or housing by a ratio of 3 to 1. Mitchell promptly deposited $500 from her summer earnings. When she saves $500 more, she'll be able to withdraw $3,000 to pay for her education or housing expenses at the University of the Pacific, where she is a freshman.

Do they help change youth behavior?

Sometimes yes, sometimes no. Usually, it's hard to tell. The key to behavior change seems to be the soundness of the program.

The Carrera Adolescent Pregnancy Prevention Program, operated by the Children's Aid Society of New York, reports a host of statistically significant positive outcomes among its participants, including lower pregnancy and birth rates. The program includes stipends for employment-preparation activities.

Dr. Michael A. Carrera, the program's director, believes that it's the overall quality of the program that's important, not the incentives. He prefers to think of the stipends as compensation for "substantive participation," rather than as incentives.

"The 'incentive' word is tied in my mind to someone saying, 'If you take the SAT tutoring, I'll buy you that pair of shoes you want,' " he says. "What I want to tie it to is something that is active and substantial and starts to teach them the equation that we've all learned, which is that when you get something done, you meet a deadline, you honor your obligations, you get your compensation."

Planned Parenthood claims positive behavior results from its Dollar-a-Day Program, which serves mostly Latina girls between 14 and 18. In each of the past 12 years, fewer than 10 percent of participants have gotten pregnant, Anderson says. (In comparison, the Alan Guttmacher Institute reports that about 15 percent of 15- to 19- year-old Hispanic girls in Colorado became pregnant in 2000.)

Finally, both QOP evaluations demonstrated that participation in the incentive-using program increased the likelihood of graduating from high school and enrolling in post-secondary school or training. Was it because of the incentives or the programming? The evaluators concluded that incentives "do not appear to operate effectively in the absence of a strong program featuring much personal contact with staff."

For whom do incentives work best?

Ivry and Doolittle of the MRDC suggest that incentives may be most effective with youth who do not care about conventional rewards. "Young people who are outside the mainstream may not be motivated solely by pragmatic goals like attaining an education credential, learning new skills related to employment, or even getting a job," their report notes.

Especially for older youth, they argue, offering financial incentives for participation in self-betterment activities is important because of competition from other pursuits. "Given the economic realities faced by at-risk youth, it is difficult for human capital development programs to compete with the secondary labor market and the underground economy," they say.

Larkin Street Youth Services works with the youth Ivry and Doolittle are talking about: homeless kids and runaways in San Francisco. "These are the hardest-to-serve young people, those who have been through multiple programs," says Sam Cobbs, director of program services. "They're sleeping on the street and wondering where their next meal is coming from."

Incentives are crucial for getting these youth "to do what they need to to get off the street," Cobbs says.

While younger youth respond well to incentives like blank CDs, phone cards or clothing vouchers, older youth and young adults prefer cash, Cobbs says. "Providing cash incentives is our attempt to set up our own economic system in-house, versus them going out on the streets and doing things to get money some other way," Cobbs says. "They're willing to sit in on a focus group or take a survey for $20 to put in their pocket."

The Eisenhower Foundation also regards incentives as an important part of the success of the QOP model, which targets highly disadvantaged youth. "It would surprise you to see how those kids treat those stipend checks," says Johnnie Gage, the foundation's chief operating officer. "It's huge to them. They know what they had to do to earn it, and the check demonstrates to them the connection between effort and reward. Most of the world is like that. And for the first time, they get it."
Incentives have also been shown to be effective with another historically hard-to-motivate group: the pregnant and parenting welfare-dependent teenagers targeted by Ohio's Learning, Earning and Parenting (LEAP) program. An evaluation found that participation in LEAP increased school completion by almost 20 percent and employment by 40 percent for teens who enrolled while still in school. At the suggestion of the MDRC evaluators, the state increased the incentives, adding a grade completion bonus of $62 and a graduation bonus of $200.

Where's the money?

Probably the greatest barrier to increasing the use of incentives is finding the money. Except for independent living programs, which are financed largely with highly flexible federal funds, many youth-serving programs are hampered by restrictions on spending.
"Some of our programs aren't in the position to give out as much," Gage says of the Eisenhower Foundation's QOP sites. "They've had to develop community resources to be able to give out bonuses. "

Ivry and Doolittle of MRDC argue that the public is likely to support incentives in youth programs as long as it believes it's getting something in return. As the nation's experience with welfare reform has shown, they argue, "The public generally approves of helping low-income people who are working hard to improve their lives. The public is therefore likely to respond favorably to policies that help young people who help themselves and who avoid risk-taking behaviors."

But the Harvard researchers caution that, given the cost, "it may be worthwhile to evaluate this strategy as an enhancement to an existing program prior to implementing it on a larger scale."

In addition to finding more money, providing such incentives also requires overcoming the skepticism of critics. "Incentives are likely to be not just ineffective but positively counterproductive in youth development programs, just as they've been shown to be in schools, families and work places," says motivational expert Alfie Kohn, author of

Punished by Rewards.

"The trouble is that there are actually two very different forms of motivation: intrinsic, which means doing something for its own sake, and extrinsic, which means doing something to get a reward," Kohn argues. "As extrinsic motivation goes up, intrinsic motivation - the kind we care about - tends to go down. This is one of the most frequently replicated findings in all of social psychology: The more you reward people for doing something, the more they tend to lose interest in whatever they had to do to get the reward."

Maybe so, maybe not. After all, many people are intrinsically and extrinsically motivated at the same time.

Besides, some program directors say they use incentives not just because they work, but because it's the right thing to do. Hylton views incentives primarily as a way of "honoring participation."

Cobbs, of Larkin Street, seconds that notion. "In our everyday lives, we have incentives," he notes. "Why shouldn't these young people, at least until they get to the point where there are natural incentives?"

Anderson, of Planned Parenthood, says the real value of paying girls a dollar a day not to get pregnant may be in its symbolism. "A lot of these kids are not your star athletes or your star students," she says. "They really don't get a lot of positive rewards for positive behavior.

"The weekly stipend says to them, 'Here's $7 because you're doing a really great job to improve yourself. Each day you're working really hard to do the right things [you] need to do to remain healthy.' "

Even if youth do participate in a program for extrinsic reasons, as Kohn suggests happens with incentives, they do get exposed to information that can aid their development.

Consider Cassandra Mitchell, the San Francisco girl who put her summer earnings into an Individual development Account. Mitchell says she probably would have saved her earnings even without the 3-to-1 match that she got through the IDA. "Since I grew up in a homeless family, it's important to me to have money," she says. "I'm always saving for a rainy day."

But to get the promised match, she had to sit through hours of financial literacy training. Ultimately, that's likely to help her manage her finances better.

Shedrick Milton is also both extrinsically and intrinsically motivated. "The gifts are good," Shedrick says of the rewards he got for taking life skills classes. "I'd be lying if I didn't say so. But they're not why I come. I come because I know I'm going to be on my own when I turn 18, and these are things I need to know.

	If you prefer to read this article separately from the other articles listed in this section, please

Mentoring Pays Off
Paid mentors do more than pal around—
but do evaluations show that they're worth the money?
By Patrick Boyle

Like many youth-serving agencies, Children's Village struggles to recruit and keep qualified mentors to guide its troubled kids. But mentor Carl Johnson has been showing up every day for four years, and it's no mystery why.
Mentoring is his job.

As the nation struggles to fulfill calls by its past several presidents to recruit a "million mentors," agencies such as Children's Village, Friends of the Children in Oregon and the Mentor Center in Nebraska are among the few that pay adults to fill that role. From such efforts, "encouraging track records are beginning to emerge," said a report issued in June by Public/Private Ventures (P/PV). Although the evidence is admittedly limited, P/PV believes there's enough to consider paid mentoring "a strategy with abundant potential."

The evidence also reveals difficulties in the approach, while the approach itself raises some fundamental questions for youth work - such as, "What's a mentor?"
"As soon as you start paying mentors, is it truly mentoring, or has it moved into some paraprofessional category?" asks Mark Fulop, director of the National Mentoring Center, based in Portland, Ore. The salaried mentor tends to resemble not so much a volunteer mentor as a combination case worker/street worker whose tasks include mentoring.

Paying mentors is "a very sticky issue," says Andrea Taylor, developer of Across Ages, a widely replicated drug prevention program that has experimented with mentor salaries and now pays only stipends to cover expenses. She notes that a recent grant announcement from the U.S. Department of Education for mentoring programs "stated unequivocally that you are not permitted to compensate the mentors in any way. Not even transportation costs. Nothing."

Some youth program directors think paying mentors is worth the added cost. One of them is David Allen, who directed a paid mentor experiment in Portland, Ore., called Project Youth Connect (PYC), which was funded by the U.S. Substance Abuse and Mental Health Services Administration. Allen echoes the experiences of many administrators who've tried to recruit and retain volunteer mentors:
"It has just been so difficult to keep anybody consistent, anybody showing up on a regular basis. They're gung-ho for the first week or two, and the relationship sort of fizzles out."

"Especially with at-risk kids," he adds, "once they open themselves up, [then] you don't come through, the kids are quick to say, 'It's just somebody else who came into my life and hopped right out.' We just went to the philosophy that we have to pay people." His agency, Self-Enhancement Inc., still does, even though funding for PYC has ended.

While many programs have succeeded in creating long-term relationships between volunteer mentors and youth - such as Big Brothers Big Sisters of America - in many communities the need exceeds the supply. At-risk youth, particularly those from the foster care or juvenile justice systems, typically require more time, expertise and emotional commitment than most volunteers can muster.

"The needs of the youth are too great and the demands on the counselor too many for anyone other than a paid professional to reasonably be expected to do this kind of work day in and day out, year in and year out," says a Children's Village evaluation of its Work Appreciation for Youth (WAY) program, which features paid mentors like Johnson.

The need is especially great for poor, minority youth, because many agencies cannot find enough adult mentors who match the cultural and social backgrounds of the kids. The Hmong American Partnership in St. Paul, Minn., wants stable, Hmong adults to mentor its Hmong youth - but "very few people [in the Hmong community] are stable," says Laura LaBlanc, director of Youth and Family Services. "They're working really hard just to get above poverty," and don't have time for intense volunteer commitments.
So the partnership hired a young Hmong adult as a mentor.

Such agencies typically pay mentors around $30,000 a year, going down to $25,000 at some PYC sites, and up to $50,000 at the Royal Project, a juvenile justice-focused program in Seattle.

Studies of these efforts have produced mixed results. (See story, page 21.) "'Effectiveness' evidence is thus far quite limited," the P/PV brief admits, making it "hard to make the persuasive case for paid mentor-counselors." But the report, "Guides for the Journey: Supporting High-Risk Youth with Paid Mentors and Counselors," says, "These programs have together produced an intriguing and positive track record."

Here is some of what that record shows:

Qualifications and Demands

Long Vang, the paid mentor at the Hmong partnership, echoes the views of many youth workers when he says that when you rely on volunteers, "you don't know who you're going to get."

When agencies pay, they can be more selective and demanding. They look for mentors with relevant education or job experience, particularly in recognizing and responding to such issues as substance abuse, mental illness and domestic violence.
At Children's Village, WAY Director Candace Rashada says she hires only mentors with bachelor's degrees in social sciences or human services, or with extensive experience in human services, "particularly with a disadvantaged population."
For example, Johnson has worked as a socio-therapist and in a detoxification program for youth and adults. Mentor/counselor Sarah Reid has a bachelor's degree in psychology and has worked at a vocational school for at-risk teens and at a shelter for mothers and children.

In St. Paul, it was only by creating a paid position that LaBlanc was able to hire the kind of mentor she wanted: a young Hmong from the local community. Vang, 29, moved to the United States with his family when he was 2 and had youth work experience at 4-H.

Once the mentors are on board, agencies put them through more training and reporting than they'd try with most volunteers. At PYC in Portland, the mentors had 40 hours of training at Self Enhancement, then shadowed a staff member for 16 hours before starting their work.

At the Mentor Center in Nebraska, which uses part-timers (at $7 to $9 an hour), the mentors attend monthly meetings with their supervisors and quarterly meetings with other mentors, and produce a report each time they meet with a youth.
"Being paid helps with the additional responsibilities," says Suzanne Vonderfecht, who coordinates paid and volunteer mentors at the center, which is part of the state's Behavioral Health Services system.

"You need a high level of accountability" to make sure the mentors are adequately serving troubled youth, Rashada says. "I don't know that you can do it with a volunteer."

The Mentors' Tasks

When the Royal Project set out to reduce disproportionate minority confinement and recidivism in King County, Washington, it hired adults to serve in the traditional role of mentors. But the youths "wanted more than just somebody who would hang out with them," says Debra Robinson, program manager for the Seattle-based project. "Youth today are a little more sophisticated than that, and their needs are more sophisticated than that. They need someone who can do more than just hang out."

So the project decided that the mentors would also be "life coaches," providing more extensive instruction and services, teaching the youth business principles and helping them set and work toward goals. It switched from several part-timers to one full-time mentor/coach with a caseload of 10 to 15, and it is looking for another, Robinson says. She says the mentor meets with the youth individually twice a week, as well as in groups.

Once they pay someone for a job with the "mentor" title, the agencies can expect a lot more than they would of a volunteer. The job "takes on a feel of case management," says Allen at Self-Enhancement in Portland.

Consider Johnson and Reid, full-time mentor/counselors at Children's Village, which bills itself as the nation's largest youth residential treatment facility. Some 300 boys reside on this 150-acre campus, tucked into a wealthy suburb a half-hour north of Manhattan. Most of the youth come by way of New York City's child welfare system and have significant mental and behavioral problems. Many stay for years, attending one of the three schools on campus.

Each year, 15 to 20 youth enter the WAY Scholarship program, which is delivered primarily by the mentor/counselors. The mentors begin working with the youth on campus and stick with them after they return to their families or age out. The Children's Village evaluation of WAY ("The WAY to Work," 2000) says these staffers are expected to make sure the youth receive "advocacy, information, encouragement, work ethics education, counseling and other services as needed to succeed in school and on the job … provide personal and intensive emotional support and practical guidance … be coaches, cheerleaders, surrogate parents, advocates, teachers and friends."

This might sound like souped-up after-care. The difference is the mentors' investment of time and emotional commitment. At Children's Village, the mentors are usually assigned 15 to 20 boys at a time, and they must:

• Follow the youths for five years, including after they leave campus. When a youth moves without notifying his mentor, the mentor must find him. Reid has tracked down families in shelters. "We do a lot of detective work," Johnson says.

• Meet face-to-face with each youth at least twice a month; keep in regular contact with important people in each youth's life (such as a grandmother), and file monthly reports about each youth and these contacts. For Johnson, these meetings often involve playing basketball, going to dinner or walking in and out of businesses in Manhattan to pick up job applications. "As we're walking and talking, things come out," he says.

• Show up to advocate for or help the youth with problems in such areas as school, the law or employment. The mentors talk with teachers and principals, help families find housing when they face eviction, appear in court for kids who've been arrested, and take the kids out for their birthdays when no one else plans to. When one youth got arrested and his mother wouldn't see him, Reid picked up clothes at his home and took them to him in detention.

• Be available around the clock. But unlike some volunteer mentors, the Children's Village staffers stress that they are not the youths' friends. "This is my job," Johnson says.

Time

Fulfilling these requirements takes a lot of time - a crucial element in helping troubled youth. "There are strong indications derived from the research on adult-youth relationships that some threshold level of 'intensity' (frequency of contact) and duration are important," the P/PV report notes, although it's not clear what the threshold is.

For the youth in these programs, the threshold is higher than most volunteers could manage. Children's Village has volunteer mentors for many youth on its campus, but notes in its evaluation that "for youth discharged from residential treatment, it [a volunteer] is probably not enough. Volunteers cannot be expected to go to the extreme lengths that many of the WAY Scholarship counselors had to go in order to develop and sustain relationships with the program participants."

At the Hmong-American Partnership, Vang initially meets with youth in the mentoring program once a week in groups and about twice a week one-on-one. As the youth move through the program's stages, they spend less time with Vang, because he needs to make time for newcomers.

That pattern of declining one-on-one time weakened the impact of the PYC programs. Each mentor typically took on new groups of youth in waves, and the only way to do that without perpetually hiring more staff is to give less individual attention to the kids already in the program, working with them more in groups.

At Self Enhancement's PYC in Portland, as a second group of youth came on board, the first group, "who had developed close bonds with their mentors, were upset about losing the one-on-one time," says an evaluation by RMC Research Corp.

When the Hmong group ran a PYC, LaBlanc saw that as the first group of youths got less time with their mentor, some of the measured gains reversed. "We suddenly got a real dip in the kids' academic performance," she recalls. "I thought, 'I've got 40 kids assigned to one mentor. … There are too many kids needing this guy's time."

(The Portland and St. Paul sites had versions of paid mentor programs before PYC and have continued them afterward, with caseloads reaching 50.)

Even with relatively moderate caseloads, evaluations of the PYC programs showed that paid mentors had difficulty meeting each youth individually for two hours a week, as required. "Most of the sites weren't able to do that," notes Kathy Zavela, who served as project director for a PYC at the University of Northern Colorado in Greeley.
Another problem, she says, was that the program provided intensive mentoring for only about half a year. "The length of mentoring may not have been sufficient enough to yield significant results," she says.

In contrast, Children's Village tries to follow the WAY Scholarship youth for up to five years. Friends of the Children, which operates in nine cities, tries to follow its youth for 12 years, through high school.

"It takes a good six months for a [mentoring] relationship to really solidify," says Taylor, the developer of Across Ages. Then progress can begin. She says Across Ages asks mentors to stay at least a year.

The Philadelphia-based program, which pairs youth with older adults, also tried salaried mentors under a PYC grant.

Sticking Around

Getting mentors to stay is one of the advantages that program administrators see in paying salaries. "I think it does add to their ability to make a longer commitment," says Vonderfecht at the Mentor Center in Nebraska.

For some, the strategy has worked. At the Hmong PYC, two of the three mentors stayed for the entire three-year grant, and the other stayed for two years, LaBlanc says. Vang, who was not part of PYC, has been at the agency for 10 years.
At Children's Village, the longitudinal study of WAY Scholarship youth showed that they averaged 2.5 changes in mentors during their time in the program.

Other programs have not fared so well. At Self-Enhancement in Portland, three of the four mentors hired for PYC left soon after the end of the first cycle (about seven months), while another left near the end of the three-year grant. They were replaced, but the RMC evaluation found that some youth consequently lost interest in the program. "Staff turnover almost certainly diminished some students' loyalty to the program and hindered the students' ability to bond with their mentor," the report said.
The turnover at Children's Village, albeit less frequent, has also isolated some youth. While the evaluation indicates that most of the youths felt good about their mentors even though they had several of them, it notes that "some youth became upset when the counselors left the program."

The evaluation quotes one youth as saying: "He informed me I was going to get another counselor. I didn't want that; it bothered me. For a while I was really mad. I just felt betrayed. I didn't want to get to know another counselor."

Coming in as a replacement mentor requires extra time and care, say mentors Johnson and Reid. But they say having one mentor for years doesn't appear to be as important to youths as having a person consistently dedicated to them.

"They want to know whether they can depend on you, whether you're going to do what you say you're going to do," Reid says.

Although paying people to be mentors might compel some to stay with a program for a while, the money can also buy an unstable staff. That's because those hired are professional youth workers; they want to eventually move up from these low- to moderate-paying positions.

At Self Enhancement, three of the four mentors who left PYC took other jobs within the agency, Allen says. "It kind of hurt the program," he says. "It was a fight" as the organization mulled over letting the mentors change jobs. But the agency's PYC grant was for just three years, he notes, and "people were looking at job security."

Turnover in paid mentoring programs is "inevitable," says Thomas Smith, author of the recent P/PV report. He calls the jobs largely "transitional positions" for people starting out in youth work or moving over from a related field, like teaching. "I don't think the normal case is going to be one mentor" for a youth's entire time in a program, he says.
Children's Village warns about this in materials for agencies looking to replicate WAY: "While the counselor relationship is paramount … job changes will occur, and participants need to feel connected to the program sufficiently to accept a new counselor."

Amy Baker, director of research at Children's Village, says the agency learned this from an effort in the early 1990s, funded by the U.S. Department of Labor, to replicate WAY at community-based organizations. When the sole mentor at a small program left, she says, the program virtually collapsed, because "kids had an attachment to the person, not the program."

The Consistency Key

That's why WAY strives to build a youth's attachment to the parent agency, Baker says. That can be tricky to carry out without playing down what has traditionally been the crucial element of mentoring: the intense relationship between the adult and the youth.

Paid mentoring tries to make that relationship more consistent. At issue is whether the money buys enough consistency - whether providing a youth with several paid mentors over five to 10 years is helpful enough to be worth the trouble. For most at-risk youth, the alternatives are an occasional volunteer mentor or none at all.
What's most important, says Zavela at the University of Northern Colorado, is "not so much whether a person's paid or not paid to be a mentor. It's whether there's a consistent adult presence in that [young] person's life."

	If you prefer to read this article separately from the other articles listed in this section, please

An Evaluation of Volunteers Courts Controversy
Study of appointed advocates for children produces some surprising results and a challenge for the group that asked for it.
By Barbara White Stack

The Court Appointed Special Advocate (CASA) program couldn't sound more apple pie, more thousand points of light.

CASAs are a cadre of 74,000 volunteers trained for dozens of hours, then dispatched to conduct independent investigations of child abuse and to represent the children's interests in courts around the nation. What could be wrong with that?

Virtually nothing, according to past evaluations. A qualitative consumer satisfaction survey of 23 CASA programs last year, commissioned by the National CASA Association, gave glowing reviews.

So, a second, more ambitious evaluation, a national one including a control group, seemed without risk.

That's not how it turned out.

While containing some information for national CASA to brag about - such as judges assigning CASAs to the most difficult cases, then frequently doing exactly what the advocates recommend - the report commissioned by the association delivers some surprisingly damning numbers.

It says CASAs, an overwhelmingly white and female group, spend little time on cases, and even less on those of black children. It says youngsters with CASAs are associated with more removal from parents, less kinship care and less reunification with parents.

CASA critics, including social workers who say problems in child welfare should be addressed by hiring more professionals rather than relying on volunteers, seized on those numbers, contending that they prove CASAs might actually harm children and families.

CASA officials focus on the positive findings and argue that the negative ones are questionable or need more study. Even the evaluation's author, Caliber Associates, has taken the unusual step of responding to CASA critics by stressing that some of the numbers may not prove anything because the controls in the study may have been faulty.

Whatever the truth about CASAs, the organization's experience with the study illustrates an increasingly important point for the youth field: the risks that groups take when complying with mounting demands from government and foundation funders to prove that what they do works.

Initial Findings

Michael Piraino, national CASA's chief executive officer for the past decade, says he sought funding for a thorough CASA evaluation long before accountability became the demand du jour. He recalls that in the mid-1990s, when he was searching for $1 million to pay for two evaluations and a data collection system to support them, funders wondered if they were worth doing. (See "Court Advocate Program Grows, But How Much Does It Help?" June 2000.)

Beginning in 1997 and ending in 2000, Piraino received three grants totaling $1 million from the David and Lucile Packard Foundation of Los Altos, Calif. The grants paid for software, called COMET, to help local CASA organizations track information, such as the hours that volunteers spent on cases and how many recommendations were accepted by judges. The grant also paid for two evaluations: the first, a qualitative study conducted by Pat Litzelfelner of the University of Kentucky College of Social Work, and the recently released quantitative study by Caliber.

In Litzelfelner's satisfaction survey, which began in 2001 and examined responses from 742 judges, lawyers, parents, foster parents and social workers, every group gave CASA a positive ranking on every question. The questions ranged from whether the respondent understood the role of a CASA (which got the highest scores) to whether the CASA visited the children regularly (which got among the lowest).