The banner of ECP was picked up by a number of subsequent social workers, and a
rather large (a nd not uncontroversial ) literalure has grown around Lhese nolions (e.g.,
Corcoran, 1985; Ivanoff, Blythe, & .8riar, 1987; Ivanoff, H.obinson, & Blythe, 1987;
G. MacDonald, 1994; Thyer, 1996). The influence of ECP has not been inconsiderable. For
example, in 1982, just 3 years fol lowin g the publicalion of F.mpirical Clinical Practice
(Jayaratne & Levy, 1979) , the curriculum policy statement of the Co un ci l on Social Work
Ed ucation (CSWE, 1982) included a new mandate Lh a l research co urses musl now Leach
“designs for the systematic evaluation of the student’s ow n practice . . . [and should] pre-
pare them systematically to evaluate their own practice and co ntr ibute to the generation
of knowledge for practice” (pp. 10- 11). Similar sta ndards still ca n be found in the curren t
CSWE guidel ines. Insistin g that individual practi tio ners co ndu cl systema tic outcome
evaluations of their own services was a remarkable professional standard, one that has not
ycl bee11 cm ul a lcd by educational and practice guidelines wiLhin clinical psychology o r
psychiatry in the present day. Reid ( 1994) provides a ni ce overview of the rise, influence,
an d dissemjnation of the ECP movement.
Empirically Supported Treatments
Subscquenl lo Lhc ECP movement within social work, a rclaled iniLiaLive developed
within clinical psychology called empirically validated treatments. During the mid – l 990s,
the president of Section lll (Society for a Science of Clinical Psychology) of Division 12
CHAPTCR 1 • IN l ROOUtTORY P RI NCIPLES OF S OCIAL W ORK R ESEARCH 5
(Clinical Psychology) of the American Psychological Association convened a Task Force
on Promotion and Dissemination of Psychological Procedures, a gro up charged with two
functions: (a) develop a scientifically defen sible set of criteria that can be used to deter-
mine whether a given psychological technique ca n be called empirically va lida ted and
( b) cond uct co mprehensive reviews of the research literature, apply these cr iter ia, a nd
come up with, in effec t, lists of psychological procedures that fulfill these criteria
and, therefore, can be co nsidered, in a scientific sense, empirically validated.
Th e evid enti ary sta ndards ultimately decided o n by the task force were actually rather
modest, consisting of the following criteria:
I. At least two good between-group design experiments demonstrating efficacy in one
or mo re of the following ways:
A. Superior to pill or psychological placebo or to another treatment
B. Equivalent LO an already established treatment in experiments with adequate
statistical power
II. A large ser ies of single-case design expe rim en ts ( N > 9) demonstratin g efficacy that
must have done the following:
A Used good experimental designs
B. Compared the intervention to another treatment (as in I.A. )
Among the further criteria are that the psychological techniques must be based on
well-proceduralized treatmenL manuals, that the characteristics of the client sam ples are
clearly defined, and th at the positive effects must have been demon strated by at least two
different inves tigators or investigatory teams. A psychological treatment m eeting the
preceding criteria co uld be said to be well established. A som ewhat less stringent set of cr i-
teria could be followed to potentially label a treat ment as probably efficacious (Chambless
et al. , 1996).
With the criteria in place, the task force busily got to work in seeing which psycholog-
ical treatments co uld be labeled empirically validated and probably efficacious, and reports
soon began appearing indicating empirically validated inLerventions for a wide array of
psychosocial disorders such as depression, panic disorder, pain, and schizophre nia. As
with the ECP movement within social work, the task force within psychology did not
escape controversy. For one thing, the task force recognized that labeling a treatm ent as
empirically validaled see med to close the discussio n off, implying perhaps a stronger level
of research evidence than was justified. Subseq uent reports of the task force used lhe
more t empered language of empirically supporled lreatments (ESTs) . Entire issues oflead-
ing professional journals (i.e., a 1996 issue of Clinical Psychology: Science and Practice, a
1998 issue of the Journal of Consulting and Clinical Psychology, a 1998 issue of
Psychotherapy Research) were devoted to the topic, as were considerable independent lit-
eratures (e.g., Sanderson & Woody, 1995). The influence of the EST movem ent also has
been strong, and the work of the Division 12 task fo rce was commented on extrem ely
favorably in Mental Health: A Report of the Surgeon General (Hatcher, 2000). The volume
titled A Guide lo Treatments That Work (Natha n & Go rman, 2007), now in its third edi-
tion, i.s an exemp lary resource for social workers seeking relatively current information
about empirically sup ported treatments for a wide va ri ety of m ental hea lth prob lems.
Division 12, Sec.:Lion HT (The Society for a Science of Clinical Psychology) co ntinues its
work in defining the cr iteria and language used to describe empirically suppo rted treat-
ments and maintains a Web site providing curre nt information on this influential initia-
tive (see http://www.psychology.sunysb.edu/eklonsky-/divisionl2/index.html).
6 Tll E HAIWBOOK OF S OCIAL W ORK R ES EARCH METHODS
Evidence-Based Practice
Coinc ident with the EST initiatives in clinical psychology have b een related activities in
med ic ine labeled evidence-based practice, defined as “the conscientious, explicit, and judi-
ciou s use of the current best evidence in making decisions about the care of individual
patients” (Sackett, Richardson, Rosenberg, & Haynes, 1997, p. 2). On its face, EBP would
not seem to be a radical notion, and indeed, most readers would ass ume that such a sta n-
dard already was in place in most of the hea Ith professions. St’td ly, to a great extent, this is
not the case, altho ugh a small but in fluen tit’tl grou p of health care providers is attempting
to make it so. EBP and EST actually are much more sophisticated var iants of th e earlier
ECP model of social work, but the spirit and intent of all three movements ECP (devel-
oped within social work), EST (developed within psychology), and EBP (developed
within medicine)-are the same. EBP is gradually supplanting the ECP and EST initia-
tives within social work and psychology. The current president of the Society for the
Science of Clinical Psychology (a section of Division 12 of the Amer ican Psychological
Association) pub lished an edito rial titled “Evidence-Based Psychoth erapy: A Graduate
Course Proposal” (Persons, 1999 ), and some social workers have begun using the EBP
language, most notably Gambrill (1999) w ith her thoughtful arLicle titled “Evidence-
Based Practice: An Alternative to Authority-Based Practice,” which introduced El3P to the
socia l work literature. The past decade has seen the publication of enough social work
books on the EBP topic to fill a bookshelf. The melding of these disciplinary perspectives
i11lo an interdisciplinary human services movemen t generically ca lled evidence-based prac-
tice seems likely. Consider Perso ns’s ( J 999) description of EBP:
T he evidence- based practitioner:
• Provides informed consent for treatment
• Relies on the efficacy data (especially from RCTs [randomized cl inical trials]) when
recomm ending and selecting and carrying out treatm ents
• Uses the empirical literature Lo gu ide decision-m aking
• Uses a systematic, hypothesis -testing approach to the treatment of each case:
o Begins with careful assessment
o Sets dear and measurable goals
o Develops and individualized fo rmulation and a treatment plan based on the
formulation
o Monitors progress toward the goals frequently and mod ifies or ends treatment
as needed (p. 2)
WeU, perhaps Jayaralne and Levy ( 1979) were simply two decades al1cad of their time. An
issue of the NASW News contained an article on the Surgeon General’s Report on Mental
Health and noted, “A challenge in the near term is to speed transfer of new evidence-based
treatments and prevenlion interventions into diverse service delivery settings and systems”
(O’ Neill, 2000, p. 6, italics added ). The Surgeon General’s repo rt itself states clearly,
Responding to th e calls of managed menta l health and behavioral heaJth ca re sys-
Lcms for evidence-based interventions will have a much needed and discernable
impact on practice …. It is essential to expand the supply of effective, evidence-
based services throughout the nation. (Hatcher, 2000, chap. 8, p. 453 )
EBP requires knowin g what helps socia l work clients and what does not help them.
It requires being able Lo distinguish b etween unverified opin-io ns a bout p sychosocial
CHAPTER l • I NTRODUCTORY P RINCIPLES OF SO CIAL WORK RESEARCH 7
interventions and facts about their effectiveness. And separa ting facts from fictions is
what science is prelly good at doing. Jot perfectly, and not without false starts, but the
publicly verifiable and potentially testable conclusion s of scientific research render this
form of knowledge buil ding an inherently self-correctin g one (in the long nm), a con-
siderable advan Lagc over o ther “ways of knowing.”
EBP differs from its precursor initiatives in that it does not tell socia l workers what
interven tions should be provided to clients. TL d oes not list so -called best practices, create
practice guidel ines, or develop lists of supposedly emp irically based treatments. Nor docs
it unduly privilege certain forms of evidence above all others. Each of the above three sen –
tences represents commo n misconceptions of EB P. EBP is actually a process of inquiry
offered to practitioners, described for physicians in Straus, Richardson, Galsziou, and
Haynes (2005), but readily adaptable to providers in all of the h uman service professions.
These steps are as follows (from Straus ct al., 2005, pp. 3-4) :
Step l: converting th e need for information (abou t prevention, diagnosis, prognosis,
therapy, causa tion, etc.) in to an answerable qucs Lion.
Step 2: tracking down Lhe besl evidence with which to answer that question.
Step 3: critically appraising that evidence for its validity (closeness to the truth),
impact (size of the effect), and applicability (usefulness in our clinical practice) .
Step 4: integrating the critical appraisal with our clinical expertise and wil11 our
patient’s unique biology, values, and circumstances.
Step 5: Evaluating our effectiveness and efficiency in executing steps 1-4 and seeking
ways to im prove them b oth for next time.
Each chapter in Straus et al. (2005) addresses on.e of these steps, and they have been
adapted for use by soc ial workers in an exce ll ent series of entries appearing in 171e Social
Worker’s Desk Reference (see Roberts, 2009, pp. 1115-1182). EBP states that social workers
need to be familiar with the best available evidence addressing the questions related to
client services and to their particular practice situation and to integrate their appraisal of
this information into an assessment of their own skills, the client’s preferences, relevant
professional and personal val ues and ethical standards, cost, feasibility, and resources . All
of these factors a re rcleva n L, no t just what the research evidence indicates. And by best
evidence, what is meant is not so-called gold-standard st udies such as randomi zed con-
trolled trials or meta -analyses (see later chapters on Lh esc LOpics in thi s book) but simply
t he best available relevan t ev idence. If there are no stud ies of superlative quality, then you
locate and assess those of lesser quality. Lots of evidence can go in to th e mix, including
quasi-experimental studies, single-subject studies, corrclational studies, descriptive work,
epidemiological evidence, qualitative investigations, case h istories, theory, and infom1ed
clinical opinion. There is always evidence for a social worker to consult, even if it is nol
evidence of the highest quality. As with ECP, EBP also encourages practitioners to evalu-
ate the outcomes of their work with individual cl ients using a research methodology
called si ngle-subject designs .
Another option is for social workers to co nsu lt systematic reviews (SRs) of the research
evidence related to various answerable que~tions invo lving assessment and interven tive
meth ods. The two gro ups most responsible for preparing high-quality and independent
SRs are called the Cochrane Collaboration (sec www.cochrane.o rg), focusing on issues
related to health care, and the Campbell Co ll aboration (see www.campbellcollaboration
.org), focusing on social welfare, education, and criminal justice. SRs are prepared by
8 THt HAN DBOOK OF SOC IAL W ORK RtSEARCH M El HODS
qualified research teams who obtain articles and reports from all over the world dealing
with a specific issue. These reports are minutely analyzed and critiqued and the collected
information surrunar ized in a readable format, with a take-away .message something like
Treatment Xis well-supported as an effective treatment for clients with Problem Y; The avail-
able evidence indicates that Treatment X is ineffective in helping clien ts with Problem Y;
Clients with Problem Y who receive Treatment X demonstrated impaired outcomes, com –
pared to clients who receive no treatment. You can see hmV” this information would be of
immense value to social workers. Here is a sampling of SRs currently available on the
Cochrane database that is of relevance to social workers:
• Behavioral and cognitive-behavioral therapy for obsessive-compulsive disorder in
children and adolescents
• Family intervention for bipolar disorder
• Family therapy for depression
• Psychological debriefing for preventing posttraurnatic stress disorder
• Psychotherapy for bulimia nervosa and binging
• Short-term psychodynamic psychotherapy for common mental disorders
And here are some fonnd on the Campbell Collaboration Web site:
• Cognitive-behavioral therapy for men who physically a buse their partner
• Cognitive-behavioral intervention for children who have been sexually abused
• Interventions intended to reduce pregnancy-related outcomes among adolescents
• School-based educational programs for the prevention of childhood sexual abuse
• Work programs for welfare recipients
These systematic reviews represent the highest quality and up -to-date critical appraisals
of the existing research literature addressing particular psychosocial and health problems
e:>..’})erienced by social work clients. They are a wonderful resource for practitioners seeking
such information and are integral to the conduct of evidence-based practice.
To sum marize, ECP suggested that social work treatment should be chosen based on
support via randomized controlled studies and that social workers need to evaluate the
outcomes of their practice with clients using single-system research designs. The EST ini-
tiative came up with a list of evidentiary criteria needed to label a given treatmen t as
“empirically supported.” Once these criteria were in hand, lists of psychosocial interven-
t ions meeting these standards were published. RBP provides more of a process to guide
clinical and practice decision making, which explicitly embraces evidence from many
sources (albeit urging one to pay particular attention to evidence of the highest quality)
and explicitly includes nonscientific considerations such as client preferences and values
into this decision -making process. In many ways, EBP is a more sophisticated and mature
conceptualization of the conduct of practice than ECP and EST, and these latter two in i-
tiatives largely have been subsurn.ed by EBP.
On Terms
The preceding brief overview helps to bring us to the present, wherein social work is
attempting to really implement our original aspirations pcrtainiillg to being based on a
foundation of scientific research. As in most intellectual undertakings, it always is helpful
In recent years, there has been increased pressure from funding agencies and
federal, state and local governments for greater effectiveness and accountability
of prevention and intervention programs. This rising demand for program
quality, and evidence of that quality, has fueled a growing interest in evidence‐
based programs (EBPs). However, there remains some confusion about what
constitutes an EBP, whether some EBPs are better than others, and the advantages
and disadvantages of implementing EBPs. In this Research to Practice brief, we
provide an overview of what it means for a program to be evidence‐based, discuss
the advantages and disadvantages of implementing EBPs, and point readers in the
direction of resources to help locate these programs and learn more about them.
What are evidence‐based programs?
A growing body of research in the social and behavioral sciences has demonstrated that
certain approaches and strategies for working with youth and their families can positively
impact important social problems such as delinquency, teen pregnancy, substance abuse and
family violence. Many of these effective approaches and strategies have been packaged into
programs targeting outcomes specific to individuals, schools, families, and communities.
Those programs that have been found to be effective based on the results of rigorous evaluations
are often called “evidence‐based.”
WHAT WORKS, WISCONSIN – RESEARCH TO PRACTICE SERIES
Evidence‐based programs:
An overview
ISSUE #6, OCTOBER 2007
BY SIOBHAN M. COONEY, MARY HUSER,
STEPHEN SMALL, AND CAILIN O’CONNOR
University of Wisconsin–Madison and University of Wisconsin–Extension
Evidence‐based programs: An overview 2
What Works, Wisconsin – Research to Practice Series, #6
The importance of rigorous evaluation
A rigorous evaluation typically involves either an
experimental design (like that used in randomized
controlled trials) or a quasi‐experimental design. In an
experimental design, people are randomly assigned to either
the treatment group, which participates in the program, or
the control group, which does not. After the program is
completed, the outcomes of these two groups are
compared. This type of research design helps ensure that
any observed differences in outcomes between the two
groups are the result of the program and not other factors.
Given that randomization is not always possible, a quasi‐
experimental design is sometimes used. In evaluations using
this design, the program participants are compared to a
group of people similar in many ways to the program
participants. However, because a quasi‐experimental
design does not randomly assign participants to program
and non‐program groups, it is not as strong a design as the
experimental approach. Because there may be unobserved
differences between the two groups of people who are
being compared, this design does not allow program
evaluators to conclude with the same certainty that the
program itself was responsible for the impacts observed.
Most programs have evaluation evidence from less
rigorous studies. Evaluations that do not include any type
of comparison group, for example, do not allow for any
conclusions to be made about whether the changes seen in
program participants are related to or caused by the
program. These studies sometimes show the promise of
positive results, but they do not allow the program to be
classified as evidence‐based. Programs with evidence from
less rigorous studies are often referred to as “promising”
programs.
An important element of EBPs is that they have
been evaluated rigorously in experimental or
quasi‐experimental studies (see box on this
page).
Not only are the results of these evaluations
important, but it is also essential that the
evaluations themselves have been subjected to
critical peer review. That is, experts in the field
– not just the people who developed and
evaluated the program – have examined the
evaluation’s methods and agreed with its
conclusions about the program’s effects. Thus,
EBPs often have evaluation findings published
in peer‐reviewed scientific journals.
When a program has sufficient peer‐
reviewed, empirical evidence for its
effectiveness, its developer will typi‐
cally submit it to certain federal
agencies and respected research
organizations for consideration. These
organizations “certify” or “endorse”
programs by including them in their
official lists of effective programs.
This lets others in the field know the
program meets certain standards of
effectiveness. (See Appendix A for
examples of these organizations.)
Simply put, a program is judged to be
evidence‐based if (a) evaluation re‐
search shows that the program pro‐
duces the expected positive results;
(b) the results can be attributed to the
program itself, rather than to other
extraneous factors or events; (c) the
evaluation is peer‐reviewed by
experts in the field; and (d) the
program is “endorsed” by a federal
agency or respected research
organization and included in their list
of effective programs.
Given this definition of an EBP, it is
important to distinguish the term
“evidence‐based” from “research‐
based.” Consider our earlier
description of how most, if not all,
EBPs were developed based on years
of scientific research on what program
components, such as content and
activities, are likely to work for youth
Evidence‐based programs: An overview 3
What Works, Wisconsin – Research to Practice Series, #6
and families. Because EBPs contain program
components with solid empirical bases, they can
safely be called “research‐based” programs.
However, the reverse is not true. Not all, or
even the majority, of research‐based programs
fit the definition of an EBP. Just because a
program contains research‐based content or was
guided by research‐based information, doesn’t
mean it has been proven effective. Unless it also
has scientific evidence that it works, it is
incorrect to call it “evidence‐based.”
Are some evidence‐based
programs better than others?
Programs that meet the definition of evidence‐
based are not all similarly effective or equally
likely to work in a given community.
For example, some EBPs have been evaluated
rigorously in several large‐scale evaluations that
follow participants for a long period of time.
Others have only undergone one or two less
rigorous evaluations (for example, those using
the quasi‐experimental design described on
page 2). Those programs that are shown to be
effective multiple times in experimental studies
are generally considered to be of a higher
standard.
Furthermore, many EBPs have been
successfully replicated and evaluated in a
variety of settings with a range of different
audiences. Others have only been evaluated
with a particular audience in a certain
geographical area, for example. When a
program has been shown to be effective in
different settings and with different audiences,
it is more likely that it will be effective when
implemented elsewhere.
Finally, EBPs can vary in the strength of their
effects. For example, one program may have
evidence that it reduces delinquent acts in its
participants by 10 percent over the subsequent
year, while another program has evidence of
reducing delinquency by 20 or 25 percent.
Generally, those programs that consistently pro‐
duce a greater effect than other programs are
thought to be better programs.
Thus, the level of evidence for effectiveness
varies across programs, and practitioners must
use a critical eye when judging where on the
continuum of effectiveness a program lies.
Advantages of evidence‐based
programs
There are numerous merits to adopting and
implementing EBPs. First, utilizing an EBP in‐
creases the odds that the program will work as
intended and that the public good will be
enhanced. There is also greater efficiency in
using limited resources on what has been proven
to work as compared to what people think will
work or what has traditionally been done.
Instead of putting resources toward program
development, organizations can select from the
growing number of EBPs, which are not only
known to be effective but also often offer well‐
packaged program materials, staff training, and
technical assistance. Using EBPs where
appropriate can thus be viewed as a responsible
and thoughtful use of limited resources.
The proven effectiveness that underlies EBPs
can help secure resources and support from
funding agencies and other stakeholders, such
as policy makers, community leaders, and
members of the targeted population.
Increasingly, funders and policy makers are
recommending, if not requiring, that EBPs be
used to qualify for their financial support.
Additionally, the demonstrated effectiveness of
these programs can facilitate community buy‐in
Evidence‐based programs: An overview 4
What Works, Wisconsin – Research to Practice Series, #6
and the recruitment and retention of program
participants.
A final benefit of EBPs is that they may have
cost‐benefit information available. This type of
information helps to convey the potential eco‐
nomic savings that can accrue when funds are
invested in a program. Cost‐benefit information
can be very influential in an era where
accountability and economic factors often drive
public policy and funding decisions.
Disadvantages of evidence‐based
programs
Despite the numerous advantages of EBPs,
there are some limitations that are important to
consider. A major constraint is the financial
resources needed to adopt and implement them.
Most EBPs are developed, copyrighted, and
sold at rather substantial costs. Program
designers often require that organizations
purchase curricula and other specially
developed program materials, that staff attend
specialized training, and that program
facilitators hold certain degrees or certifications.
Furthermore, EBPs are often intended to be im‐
plemented exactly as designed, allowing little
room for local adaptation.
Finally, organizations sometimes find that there
are few or no EBPs that are both well‐suited to
meet the needs of targeted audiences and
appropriate for their organization and local
community setting. This situation is especially
common when it comes to the promotion of
positive outcomes rather than the prevention of
negative ones. Because the development of
many EBPs was sponsored by federal agencies
concerned with addressing specific problems,
such as substance abuse, mental illness,
violence, or delinquency, there currently exist
many more problem‐focused EBPs than ones
designed specifically to promote positive
developmental outcomes like school success or
social responsibility.
Where to find evidence‐based
programs
Practitioners looking for an EBP to implement
in their community or learn more about these
programs will find the Internet to be their most
useful resource. As mentioned earlier, a number
of federal agencies and respected research
organizations “certify” or “endorse” programs
that meet the organizations’ specified standards
for effectiveness. Many of these agencies have
established on‐line registries, of lists of EBPs
that they have identified as effective. While
there are some differences in the standards used
by various organizations to assess whether a
program should be endorsed and thus included
on their registry, most share the primary criteria
regarding the need for strong empirical
evidence of program effectiveness.
Organizations that endorse EBPs typically limit
such endorsements, and thus their program
registry, to those programs that have shown an
impact on specific outcomes of interest to the
organization. For example, programs listed on
the Office of Juvenile Justice and Delinquency
Prevention’s Model Programs Guide have all
been shown to have an impact on juvenile
delinquency or well‐known precursors to
delinquency.
As previously mentioned, because the
development of many EBPs was funded by
federal agencies focused on specific problems,
most existing registries of EBPs are problem‐
oriented. Occasionally, EBPs are categorized
according to a strengths‐based orientation and
address outcomes related to positive youth
Evidence‐based programs: An overview 5
What Works, Wisconsin – Research to Practice Series, #6
WHAT WORKS, WISCONSIN: RESEARCH TO PRACTICE SERIES
This is one of a series of Research to Practice briefs prepared by the What Works, Wisconsin team at the
University of Wisconsin–Madison, School of Human Ecology, and Cooperative Extension, University of
Wisconsin–Extension. All of the briefs can be downloaded from http://whatworks.uwex.edu.
This series expands upon ideas that are discussed in What Works, Wisconsin: What Science Tells Us about
Cost‐Effective Programs for Juvenile Delinquency Prevention, which is also available for download at the
web address above.
This publication may be cited without permission provided the source is identified as: Cooney, S.M.,
Huser, M., Small, S., & O’Connor, C. (2007). Evidence‐based programs: An overview. What Works,
Wisconsin Research to Practice Series, 6. Madison, WI: University of Wisconsin–Madison/Extension.
This project was supported, in part, by Grant Award No. JF‐04‐PO‐0025 awarded by the Wisconsin
Office of Justice Assistance through the Wisconsin Governor’s Juvenile Justice Commission with funds
from the Office of Juvenile Justice and Delinquency Prevention.
development, academic achievement, school
readiness and family strengthening.
While registries of EBPs are usually organized
around the particular outcomes the programs
have been found to impact, many programs,
especially those focused on primary prevention,
often have broader effects than this pattern
would suggest. Many EBPs have been found to
be effective for reducing multiple problems and
promoting a number of positive outcomes. For
example, a parenting program that successfully
promotes effective parenting practices may not
only reduce the likelihood of particular
problems such as drug abuse or aggression, but
may also promote a variety of positive
outcomes like academic success or stronger
parent‐child relationships. For this reason, you
will often see the same program appear on
multiple registries that focus on different types
of outcomes.
Now, more than ever, practitioners have
available to them a wealth of EBPs that build on
the best available research on what works.
Unfortunately, they are currently underused
and often not well‐understood. Although EBPs
do have some limitations, they can contribute to
a comprehensive approach to preventing a
range of social and health‐related problems and
enhancing the well‐being of individuals,
families and communities.
Evidence‐based programs: An overview – Appendix A 6
What Works, Wisconsin – Research to Practice Series, #6
Appendix A
Evidence‐based program registries
The following websites contain registries, or lists of evidence‐based programs, that have met specific criteria
for effectiveness. Program registries are typically sponsored by federal agencies or other research organiza‐
tions that endorse programs at different rating levels based on evidence of effectiveness for certain participant
outcomes. The registries listed below cover a range of areas including substance abuse and violence preven‐
tion as well as the promotion of positive outcomes such as school success and emotional and social compe‐
tence. Generally, registries are designed to be used for finding programs for implementation. However,
registries can also be used to learn about evidence‐based programs that may serve as models as organizations
modify aspects of their own programs.
Best Practices Registry for Suicide Prevention
http://www.sprc.org/featured_resources/ebpp/index.asp
This registry, developed by the Suicide Prevention Resource Center (SPRC) and the American Foundation for
Suicide Prevention, includes two registries of evidence‐based programs. The first draws directly from a larger
registry‐ that of the Substance Abuse and Mental Health Administration’s (SAMHSA) National Registry of
Evidence‐Based Programs and Practices (NREPP). Users interested in finding out more about programs
drawn from this registry will be directed to the NREPP site. The second registry was developed by SPRC in
2005 and lists Effective and Promising evidence‐based programs for suicide prevention. This portion has fact
sheets in PDF format for users interested in learning more about the listed programs.
Center for the Study and Prevention of Violence, Blueprints for Violence Prevention
http://www.colorado.edu/cspv/blueprints/index.html
This research center site provides information on model programs in its “Blueprints” section. Programs that
meet a strict scientific standard of program effectiveness are listed. These model programs (Blueprints) have
demonstrated their effectiveness in reducing adolescent violent crime, aggression, delinquency, and sub‐
stance abuse. Other programs have been identified as promising programs. Endorsements are updated
regularly, with programs added to and excluded from the registry based on new evaluation findings.
The Collaborative for Academic, Social, and Emotional Learning (CASEL)
http://www.casel.org/programs/selecting.php
The Safe and Sound report developed at CASEL lists school‐based programs that research has indicated are
effective in promoting social and emotional learning in schools. This type of learning has been shown to con‐
tribute to positive youth development, academic achievement, healthy behaviors, and reductions in youth
problem behaviors. Ratings are given on specific criteria for all programs listed, with some designated
“Select” programs. This registry has not been updated since programs were reviewed in 2003.
Evidence‐based programs: An overview – Appendix A 7
What Works, Wisconsin – Research to Practice Series, #6
Exemplary and Promising Safe, Disciplined and Drug‐Free Schools Programs
http://www.ed.gov/admins/lead/safety/exemplary01/index.html
The Department of Education and the Expert Panel on Safe, Disciplined and Drug‐Free Schools identified
nine exemplary and 33 promising programs for this 2001 report. The report, which can be found at this site,
provides descriptions and contact information for each program. The focus is on programs that can be imp‐
lemented in a school setting whether in the classroom, in extra‐curricular activities, or as after‐school pro‐
gramming.
Helping America’s Youth
http://guide.helpingamericasyouth.gov/programtool.cfm
This registry is sponsored by the White House and was developed with the help of several federal agencies.
Programs focus on a range of youth outcomes such as academic achievement, substance use, and delin‐
quency, and are categorized as Level 1, Level 2, or Level 3 according to their demonstrated effectiveness. The
registry can be searched with keywords or by risk or protective factor, and is updated regularly to incorpo‐
rate new evidence‐based programs.
Northeast Center for the Application of Prevention Technology (CAPT) Database of Prevention Programs
http://www.hhd.org/capt/search.asp
This site features a simple or advanced search function to find substance abuse and other types of prevention
programs and determine their effectiveness according to a variety of criteria. Also included is information
about the sources those agencies used for their evaluations, contact information, websites, domains, relevant
references, and a brief description of each program.
Office of Juvenile Justice and Delinquency Prevention (OJJDP) Model Programs Guide
http://www.dsgonline.com/mpg2.5/mpg_index.htm
The OJJDP Model Programs Guide is a user‐friendly, online portal to prevention and intervention programs
that address a range of issues across the juvenile justice spectrum. The Guide now profiles more than 200
programs – rated Exemplary, Effective, or Promising – and helps communities identify those that best suit
their needs. Users can search the Guide’s database by program category, target population, risk and protec‐
tive factors, effectiveness rating, and other parameters. This registry is continuously updated and contains
more programs than other well‐known registries, although many of these are Promising rather than Exem‐
plary or Effective.
Promising Practices Network on Children, Families and Communities
http://www.promisingpractices.net/programs.asp
A project of the RAND Corporation, the Promising Practices Network website contains a registry of Proven
and Promising prevention programs that research has shown to be effective for a variety of outcomes. These
programs are generally focused on children, adolescents, and families. The website provides a thorough
summary of each program and is updated regularly.
Evidence‐based programs: An overview – Appendix A 8
What Works, Wisconsin – Research to Practice Series, #6
Social Programs that Work, Coalition for Evidenced‐Based Policy
http://www.evidencebasedprograms.org/
This site is not a registry in the conventional sense of the word in that it does not include and exclude pro‐
grams based on some criteria of effectiveness. Instead, it summarizes the findings from rigorous evaluations
of programs targeting issues such as employment, substance use, teen pregnancy, and education. Some of the
programs have substantial evidence of their effectiveness, while others have evaluation results suggesting
their ineffectiveness. Users are welcome to sign up for emails announcing when the site is updated.
Strengthening America’s Families: Effective Family Programs for Prevention of Delinquency
http://www.strengtheningfamilies.org/
This registry summarizes and rates family strengthening programs which have been proven to be effective.
Programs are designated as Exemplary I, Exemplary II, Model, or Promising based upon the degree, quality
and outcomes of research associated with them. A program matrix is also included, which can be helpful in
determining “at a glance” which programs may best meet community needs. This registry was last revised in
1999.
Substance Abuse and Mental Health Services Administration’s (SAMHSA’s) National Registry of
Evidence‐Based Programs and Practices
http://nrepp.samhsa.gov/
The National Registry of Evidence‐based Programs and Practices (NREPP) is a searchable database with up‐
to‐date, reliable information on the scientific basis and practicality of interventions. Rather than categorizing
programs as Model, Effective, or Promising, NREPP rates the quality of the research findings separately for
each outcome that has been evaluated, as well as readiness for dissemination. Users can perform customized
searches to identify specific interventions based upon desired outcomes, target populations and settings.
Youth Violence: A Report of the Surgeon General
http://www.surgeongeneral.gov/library/youthviolence/chapter5/sec3.html
This report designates programs as Model or Promising and goes further than many other registries to also
include a “Does Not Work” category. General approaches and specific programs for the prevention of youth
violence are described at three levels of intervention: primary, secondary and tertiary. This report has not
been updated since its publication in 2001, but it is rare in that it discusses the cost‐effectiveness of the
programs.
BY CAILIN O’CO
University of W
Meeting local n
WHAT WORKS,
A major appeal of evidence‐b
These programs have shown
significantly affect important o
demonstrated positive effects i
funders, and program practiti
evidence‐based program more a
only assume that a program w
according to the original progra
referred to as program fidelity. Un
practice. Practitioners often cha
them, whether intentionally or n
For many years, there was a d
evidence‐based programs was a
more to which approaches to p
undermine program effectivene
changes that are sometimes ma
then suggest strategies for main
based program implementation
Program fidelity and adaptation:
eeds without compromising program effectiveness
ISSUE #4, APRIL 2007
NNOR, STEPHEN A. SMALL, AND SIOBHAN M. COONEY
isconsin‐Madison and University of Wisconsin–Extension
WISCONSIN – RESEARCH TO PRACTICE SERIES
ased programs is their promise of effectiveness.
, through rigorous evaluations, that they can
utcomes for participants. The best of them have
n a number of different settings. For policymakers,
oners, that potential for effectiveness can make an
ttractive than an unproven program. However, we can
ill continue to have those effects if it is implemented
m design. Staying true to the original program design is
fortunately, true program fidelity is not easily achieved in
nge or adapt evidence‐based programs as they implement
ot.
ebate in the scientific literature over whether adaptation of
t all acceptable. In recent years, the conversation has turned
rogram adaptation are acceptable and which types are likely to
ss [1]. In this Research to Practice Brief, we explore the types of
de to programs and the effects those adaptations can have. We
taining program effectiveness, based on research about evidence‐
and adaptation.
Program adaptation and its effects Types of program adaptations
Acceptable adaptations:
Changing language – Translating and/or
modifying vocabulary
Replacing images to show youth and families
that look like the target audience
Replacing cultural references
Modifying some aspects of activities such as
physical contact
Adding relevant, evidence‐based content to
make the program more appealing to
participants
Risky or unacceptable adaptations
Reducing the number or length of sessions or
how long participants are involved
Lowering the level of participant engagement
Eliminating key messages or skills learned
Removing topics
Changing the theoretical approach
Using staff or volunteers who are not
adequately trained or qualified
Using fewer staff members than recommended
Programs are often adapted from their original
design when they are implemented by a new org‐
anization, in a new community, or by a new staff
member. Changes might be made to a program to
better meet the needs of the community where it is
being implemented, to fit the program within an
organization’s budget or calendar, or to accom‐
modate the preferences of the local staff members
facilitating it. While adaptations for some of these
reasons may be justified, changes to the content,
duration, or delivery style of the program can
diminish the program’s effects.
One very common reason for adapting a program
is a perceived cultural mismatch between a pro‐
gram and its targeted audience [2]. Although
research shows that, for example, juvenile delin‐
quency programs tend to be equally effective for
youth from many cultural backgrounds [3], cul‐
tural mismatch continues to be a concern in many
communities.1 A large study of the effectiveness of
the Substance Abuse and Mental Health Services
Administration (SAMHSA) Model Programs in
various settings offered some support for adap‐
tations designed to address cultural mismatch: the
researchers found that fidelity to the original pro‐
gram design was generally important to program
effectiveness, but less so when there was a cultural
mismatch [4]. In other words, in situations where
the “culture” of the program was different from
the culture of the target audience, adaptations
were less damaging to the program’s
effectiveness. However, another study found that
culturally adapted versions of a violence
prevention program had higher retention rates but
weaker outcomes when compared to the non‐
adapted program [5]. The authors suggest that,
although the adaptations made the program more
attractive to participants and improved retention
rates, the adaptations also may have eliminated
crucial elements of the original intervention,
making it less effective.
Other intentional changes to evidence‐based pro‐
grams may have similar effects, making a program
more attractive to potential participants or spon‐
soring agencies, but potentially reducing or elim‐
inating the positive effects of the program. Cutting
the number or length of program sessions is a
common example. Organizations often worry that
potential participants will be unable to commit to
a lengthy program, so they shorten it. Or, in times
of tight budgets, agencies may reduce the number
of staff involved in delivering a program or use
volunteers or paraprofessionals who do not have
adequate experience or training. Unfortunately,
reducing the “dosage” of a program, changing the
staff‐to‐participant ratio, or staffing the program
with less qualified personnel is likely to diminish
1 Issue #1 in this series addresses the issue of culture
and evidence‐based programs.
Program fidelity and adaptation 2
What Works, Wisconsin – Research to Practice Series, #4
the program’s effectiveness. Sufficient dosage and
the opportunity to form positive relationships
with well‐trained staff have been identified as
important principles of effective prevention pro‐
grams [6].
Eliminating parts of a program’s content and
shortening the duration or intensity of a program
are the riskiest forms of adaptation. A piece may
be removed that was critical to the program’s
effectiveness; there may not be enough time for
participants to develop a key skill, or to build the
relationships that are critical to the change pro‐
cess. On the other hand, adding material or
sessions to an existing program while otherwise
maintaining fidelity does not generally seem to
have a detrimental effect. One study even showed
that adaptations that added components to an
existing program tended to be more effective than
programs implemented without added compo‐
nents [7]. However, other researchers point out
that components added at the local level have the
potential to be counterproductive – for example,
adding lessons based on scare tactics, which
research has shown to be harmful – and can jeo‐
pardize the overall success of the program [8].
Another pervasive type of program adaptation
comes in the form of unintentional changes that
are made as the program is implemented over
time. This is sometimes referred to as “program
drift” [1]. These changes may happen when a fac‐
ilitator adjusts the program to fit his or her
facilitation style, eliminates content he or she
doesn’t like, or adds in pieces from other curricula
that may not support the goals of the program. As
the number of these changes grows, it becomes
less and less likely that the implemented program
will have the promised effects. For this reason, it is
not uncommon for evidence‐based programs to
require regular “re‐certification” of facilitators and
provide tools to measure program fidelity.
Strategies for maintaining
program effectiveness
If you have made the decision to implement a
proven or evidence‐based program, implementing
the program as it was designed should be your
goal. As you prepare for and implement an evi‐
dence‐based program, keep the following strat‐
egies in mind to minimize the risk of reducing
program effectiveness.
Select a program that meets your needs.
In general, the fewer changes that are made to
a program, the more likely it is to have the
desired effect on participants. It is almost
always preferable to choose a program that
won’t need to be adapted, rather than getting
into the risky business of program adaptation.
Before purchasing a program or deciding to
implement it, analyze the program, the cur‐
riculum, and any accompanying handouts or
recruitment materials to determine whether
they are appropriate for your target audience
and your organization. Look at the program
with a critical eye, considering whether it
would make sense and appeal to members of
your audience, and whether it seems like it
would have the desired effect on their know‐
ledge, skills, relationships, or behavior. To the
extent possible, find a program that will need
little to no adaptation for your circumstances.
If this is not possible, select a program that has
been adapted for other audiences in the past,
or one whose developer is willing to assist you
in the adaptation process.2
2 Issue #3 in this Research to Practice series addresses
guidelines for selecting an evidence‐based
program.
Program fidelity and adaptation 3
What Works, Wisconsin – Research to Practice Series, #4
Ensure that staff members are committed
to program fidelity. Decisions about pro‐
gram selection, adaptation, and implementa‐
tion should involve all relevant staff members,
particularly those who will be staffing or
facilitating the program. They need to be com‐
fortable with the material and the style of
interaction, and they also must commit to
delivering the program as agreed. Particularly
when a program is being implemented by
more than one staff member or in multiple
locations, commitment from all staff is critical
to ensuring program success.
Contact the program developer. Program
developers can tell you whether they or others
have already adapted the program for specific
audiences or circumstances. Program develop‐
ers can also provide you with key information
about the theory and assumptions that in‐
fluenced the program’s development. You can
typically find contact information for program
developers on registries of evidence‐based
programs.3 Getting input from the program
designer is one of the best ways to ensure that
any adaptations made to the program are
appropriate. Many program developers will
be happy to discuss your plans for implemen‐
ting or adapting the program.
Determine the key elements that make
the program effective. Sometimes called
core components analysis [1], this is the process
of identifying the “active ingredients” in the
program, those things that are responsible for
a program’s effectiveness. Ideally, you would
get this information from the program devel‐
oper based on his or her research and exper‐
ience. More and more evidence‐based pro‐
grams are identifying their core components,
often as they develop guidelines for imple‐
mentation and tools for measuring program
fidelity. For example, the Families and Schools
Together (FAST) program allows local imple‐
mentation planning teams to modify up to
60% of the program content as long as they
implement the core components identified by
the program developer [9].
Assess the need for cultural adaptation.
Keep in mind that culture is reflected both at
the surface level of the program, such as the
language and visual symbols used, and at a
deeper level, for example in the risk and pro‐
tective factors that are targeted [10].
• On the surface level, consider the lan‐
guage used in the program; the visuals,
examples and scenarios used; and the
activities that participants are asked to en‐
gage in. For example, when the Strength‐
ening Families Program for Parents and
Youth 10‐14 was adapted for use with
Hmong families in Wisconsin, changes
were made to de‐emphasize verbal and
physical expressions of love among family
members, and activities that required
physical contact were modified to make
the program culturally acceptable. These
types of changes, which tailor the existing
program to a particular group of par‐
ticipants, are unlikely to diminish the
program’s effectiveness.
• At the “deep” level of program struc‐
ture and goals, the cultural mismatch may
be more subtle and harder to pinpoint.
There is also a greater risk of reducing the
program’s impact when changes are made
at this level. Making adaptations to the
program’s targeted risk and protective
factors or other “deep” elements of the
program design should probably not be
attempted unless it’s done in collaboration
with the program’s developer. If you are
3 Program registries are on‐line guides to effective
programs created and maintained by federal
agencies or research organizations. Issue #3 in this
Research to Practice series includes information about
some of these registries and how to use them.
Program fidelity and adaptation 4
What Works, Wisconsin – Research to Practice Series, #4
considering these types of changes, look
first for a different program; one better
suited to meet your needs may be avail‐
able.
Stay true to the duration and intensity of
the original program. Several studies have
indicated that fidelity to program intensity, or
the amount of contact participants have with
the program and the length of time they are
involved in the program, is one of the keys to
replicating the effects of an evidence‐based
program [11]. It is also this aspect of a
program that is most likely to be changed,
whether because of budgetary concerns or in
an attempt to make the program more appeal‐
ing to participants. For example, a program
designed for seven weekly sessions may not
have the same effect if all the content is pre‐
sented in two full‐day sessions. Even though
all the content may be covered, participants
will not have time to practice the new skills
they are learning or to integrate these changes
into their everyday behavior. Similarly, con‐
densing the material to limit the program to
four weekly sessions will likely reduce the
program’s effectiveness: critical content may
be cut inadvertently, in addition to the risks
posed by reducing participants’ contact with
the facilitator and the program. It is important
to follow the program’s guidelines for how
often the program meets, the length of each
session, and how long participants stay
involved.
Take steps to avoid program drift.
Facilitators and managers should use tools
provided by the program developer to mon‐
itor program implementation. Be vigilant in
tracking how the program is implemented and
address any unintentional variation from the
original program design. This is particularly
important if a program is handed off from one
staff person to another, and when a program
is being implemented at multiple sites or by
multiple staff members.
Stay up‐to‐date with program revisions
and new materials. Evidence‐based pro‐
grams may be periodically revised and up‐
dated, or new materials may become available
as the program developers continue their
research on the program’s effectiveness. Check
the program’s website, or contact the program
developer and ask to be informed of any
updates to the program or materials.
In summary, program adaptation is not a task to
be taken lightly, if it occurs at all. Taking program
selection seriously, being thoughtful about what
(if anything) you change, and adding appropriate
components or topics as needed will help you to
avoid changing the core components of the
program and diluting its effectiveness.
Sources
1. U.S. Department of Health and Human Services.
(2002). Finding the balance: Program fidelity and
adaptation in substance abuse prevention: A state
of the art review. Retrieved March 1, 2006, from
http://modelprograms.samhsa.gov/pdfs/Finding‐
Balance1
2. Castro, F.G., Barrera, M., & Martinez, C.R. (2004).
The cultural adaptation of prevention
interventions: Resolving tensions between fidelity
and fit. Prevention Science, 5, 41‐45.
3. Wilson, S.J., Lipsey, M.W., & Soydan, H. (2003).
Are mainstream programs for juvenile
delinquency less effective with minority youth
than majority youth? A meta‐analysis of
Program fidelity and adaptation 5
What Works, Wisconsin – Research to Practice Series, #4
outcomes research. Research on Social Work
Practice, 13, 3‐26.
4. Emshoff, J., Blakely, C., Gray, D., Jakes, S.,
Brounstein, P., et al. (2003). An ESID Case Study
at the Federal Level. American Journal of
Community Psychology, 32, 345‐357.
5. Kumpfer, K.L., Alvarado, R., Smith, P., & Bellany,
N. (2002). Cultural sensitivity and adaptation in
family‐based prevention interventions. Prevention
Science, 3, 241‐246.
6. Nation, M., Crusto, C., Wandersman, A.,
Kumpfer, K., Morrisey‐Kaner, E., et al. (2003).
What works in prevention: Principles of effective
prevention programs. American Psychologist, 58,
449‐456.
7. Blakely, C.H., Mayer, J.P., Gottschalk, R.G.,
Schmitt, N., Davidson, W.S., et al. (1987). The
fidelity‐adaptation debate: Implications for the
implementation of public sector social programs.
American Journal of Community Psychology, 15, 253‐
268.
8. Elliott, D.S. & Mihalic, S. (2004). Issues in
disseminating and replicating effective prevention
programs. Prevention Science, 5, 47‐52.
9. Wisconsin Center for Education Research. (2006).
Families and Schools Together: Research and
program development: Implementation.
Retrieved December 19, 2006, from
http://www.wcer.wisc.edu/fast/imp/start.htm
10. Dumka, L.E., Lopez, V.A., & Carter, S.J. (2002).
Parenting interventions adapted for Latino
families: Progress and prospects. In J. Contreras,
K.A. Kerns, and A.M. Neal‐Barnett (Eds.), Latino
children and families in the United States: Current
research and future directions (pp. 203‐233).
Westport, Conn.: Praeger.
11. Allen, J.P., Philliber, S., & Hoggson, N. (1990).
School‐based prevention of teen‐age pregnancy
and school dropout: Process evaluation of the
national replication of the Teen Outreach
Program. American Journal of Community
Psychology, 18, 505‐524.
Program fidelity and adaptation 6
What Works, Wisconsin – Research to Practice Series, #4
WHAT WORKS, WISCONSIN: RESEARCH TO PRACTICE SERIES
This is one of a series of Research to Practice briefs prepared by the What Works, Wisconsin team at the University
of Wisconsin–Madison, School of Human Ecology, and Cooperative Extension, University of Wisconsin–
Extension. All of the briefs can be downloaded from www.uwex.edu/ces/flp/families/whatworks.cfm.
This series expands upon ideas that are discussed in What Works, Wisconsin: What Science Tells Us about Cost‐
Effective Programs for Juvenile Delinquency Prevention, which is also available for download at the web address
above.
This publication may be cited without permission provided the source is identified as: O’Connor, C., Small, S.A. &
Cooney, S.M. (2007). Program fidelity and adaptation: Meeting local needs without compromising program
effectiveness. What Works, Wisconsin Research to Practice Series, 4. Madison, WI: University of Wisconsin–
Madison/Extension.
This project was supported by Grant Award No. JF‐04‐PO‐0025 awarded by the Wisconsin Office of Justice
Assistance through the Wisconsin Governor’s Juvenile Justice Commission with funds from the Office of Juvenile
Justice and Delinquency Prevention.
The authors wish to thank Mary Huser of the University of Wisconsin–Extension and Gay Eastman of the
University of Wisconsin–Madison for their edits, comments, and suggestions in the development of this Research
to Practice brief.
http://www.uwex.edu/ces/flp/families/whatworks.cfm
- Program adaptation and its effects
- Strategies for maintaining program effectiveness
Sources
elines for Selecting an Evidence‐Based Program
What Works, Wisconsin – Research to Practice Series, #3
In recent years there has been a significant increase in the number of evidence‐
based programs designed to reduce individual and family problems and
promote healthy development. Because each program has undergone rigorous
testing and evaluation, program practitioners can reassure potential program
sponsors that the program is likely to be effective under the right conditions, with
the appropriate audience and with the proper implementation. However, knowing
which program is the “right” one for a particular setting and audience is not always
easy to determine. When selecting a program, it is important to move beyond current
fads or what the latest salesperson is selling and consider whether a program fits with
the local agency’s goals and values, the community setting and the needs of the
targeted audience. The long‐term success of a program depends on the program being
not only a good one, but also the right one.
Unfortunately, there is currently little research on how to best go about the process of
selecting an evidence‐based program. Consequently, the guidelines we present in this brief
are based primarily on our experiences working with community‐based organizations, the
experiences of practitioners, and common sense. We have identified a number of factors that
we believe should be considered when deciding which program is the most appropriate for a
particular audience and sponsoring organization. These factors can be grouped into three
general categories: program match, program quality and organizational resources. In order to
assist with the process of program selection, we have developed a set of questions to consider
when selecting an evidence‐based program for your particular agency and audience.
WHAT WORKS, WISCONSIN – RESEARCH TO PRACTICE SERIES
Guidelines for selecting an evidence‐based program:
Balancing community needs, program quality,
and organizational resources
ISSUE #3, MARCH 2007
BY STEPHEN A. SMALL, SIOBHAN M. COONEY,
GAY EASTMAN, AND CAILIN O’CONNOR
University of Wisconsin–Madison and University of Wisconsin–Extension
Guidelines for Selecting an Evidence‐Based Program 2
What Works, Wisconsin – Research to Practice Series, #3
Program match:
Questions to ask
How well do the program’s goals and objectives
reflect what your organization hopes to achieve?
How well do the program’s goals match those of
your intended participants?
Is the program of sufficient length and intensity (i.e.,
“strong enough”) to be effective with this particular
group of participants?
Are potential participants willing and able to make
the time commitment required by the
program?
Has the program demonstrated effectiveness with a
target population similar to yours?
To what extent might you need to adapt this
program to fit the needs of your community? How
might such adaptations affect the effectiveness of the
program?
Does the program allow for adaptation?
How well does the program complement current
programming both in your organization and in the
community?
The issues raised by program match, program
quality and organizational resources are overlap‐
ping. Selecting a program usually requires
balancing different priorities, so it’s important to
have a good understanding of all three of these
before determining the usefulness of a program
for a particular situation.
PROGRAM MATCH
A first set of factors to consider is related to how
well the program will fit with your purposes, your
organization, the target audience, and the com‐
munity where it will be implemented.
Perhaps the most obvious factor to consider is
whether the goals and objectives of a program are
consistent with the goals and objectives that the
sponsoring organization hopes to achieve. While
this may seem apparent, it is not uncommon for
sponsors to select a program because there is grant
money available to support it or everyone else is
doing it. Just because a program is the latest fad or
there’s funding to support it doesn’t necessarily
mean it is going to accomplish the goals of the
sponsoring organization or meet the needs of the
targeted audience.
A second aspect of program match involves
whether a program is strong enough to address
the level and complexity of risk factors or current
problems among participants. This refers to the
issue of adequate program duration and intensity.
Changing existing problem behaviors or counter‐
acting a large number of risk factors in partici‐
pants’ lives requires many hours of engaging
programming over a period of time. For example,
a short primary prevention program designed for
families facing few problems or risks may not be
effective for an audience already experiencing
more severe problems.
Another facet of program match concerns the
length of the program and whether your intended
audience will be willing and able to attend the
required number of sessions. Many evidence‐
based programs are of fairly long duration,
involving multiple sessions over weeks or
months. A common concern of program pro‐
viders is whether potential participants will
make such a long‐term commitment. Because
this is a realistic concern, program sponsors need
to assess the targeted audience’s availability for
and interest in a program of a particular length.1
The reality is, if people don’t attend, then they
can’t reap the program’s benefits. However, it is
also important to keep in mind that programs of
longer duration are more likely to produce
lasting behavior change in participants. Program
sponsors sometimes need to find a compromise
between the most effective program and one that
will be a realistic commitment for
participants.
Matching a program with the values and culture
of the intended audience is also critically import-
ant. Some programs are intentionally designed
for particular populations or cultural groups.
Most are more culturally generic and designed
1 Issue #2 in this series addresses strategies for
recruiting and retaining participants.
Guidelines for Selecting an Evidence‐Based Program 3
What Works, Wisconsin – Research to Practice Series, #3
Program quality: Questions to ask
Has this program been shown to be effective?
What is the quality of this evidence?
Is the level of evidence sufficient for your
organization?
Is the program listed on any respected evidence‐
based program registries? What rating has it
received on those registries?
For what audiences has the program been found
to work?
Is there information available about what
adaptations are acceptable if you do not
implement this program exactly as designed? Is
adaptation assistance available from the program
developer?
What is the extent and quality of training offered
by the program developers?
Do the program’s designers offer technical
assistance? Is there a charge for this assistance?
What is the opinion and experience of others who
have used the program?
for general audiences.2 It’s important to consider
whether the targeted audience will find the
program acceptable and will want to participate.
The ideal situation would be finding evidence that
a program is effective for the specific pop-
ulation(s) you intend to use it with. In that case,
you could reasonably expect the program to be
effective when it is implemented well.
Unfortunately, many evidence‐based programs
have only been evaluated with a limited number
of populations and under a relatively narrow
range of conditions. While many evidence‐based
programs are effective and appropriate for a range
of audiences and situations, it is rare to find a
program that is suitable or effective for every
audience or situation. In many cases, you will
need to carefully read program materials or talk to
the program’s designers to see whether adapting a
program or using it with an audience for which it
hasn’t been evaluated is reasonable.
Depending on the design, programs may or may
not be amenable to adaptation. If adapting a
program to a particular cultural group is
important, then program sponsors should serious-
ly consider whether such changes are possible.
Some program designers are willing to help you
with program adaptation so that the program’s
effectiveness will not be undermined by these
changes.3
Finally, when considering which program to
select, sponsors should consider whether the pro-
gram complements other programs being offered
by the sponsoring organization and by other
organizations in the community. The most
effective approaches to prevention and inter-
vention involve addressing multiple risk and
2 Issue #1 in this series addresses the issue of culture
and evidence‐based programs.
3 Issue #4 in this series will address issues of program
fidelity and adaptation.
protective factors, developmental processes and
settings. Any new program implemented in a
community should address needs that other
community programs fail to address, which will
help to create the kind of multi‐pronged approach
that leads to greater overall effectiveness.
PROGRAM QUALITY
A second set of factors to consider when selecting
a program are related to the quality of the pro‐
gram itself and the evidence for its effectiveness.
The program should have solid, research‐based
evidence showing that it is effective. For a pro‐
gram to be deemed evidence‐based, it must go
through a series of rigorous evaluations. Such
evaluations have experimental or quasi‐experi‐
mental designs – meaning they compare a group
of program participants to a similar group of
people who did not participate in the program to
determine whether program participation is assoc‐
iated with positive changes. These kinds of eval‐
Guidelines for Selecting an Evidence‐Based Program 4
What Works, Wisconsin – Research to Practice Series, #3
TABLE 1: Selected evidence‐based program registries
Blueprints for Violence Prevention
http://www.colorado.edu/cspv/blueprints/index.html
This registry is one of the most stringent in terms of endorsing programs as Model or Promising. Programs are
reviewed by an expert panel and staff at the University of Colorado, and endorsements are updated regularly.
Programs are added and excluded from the registry based on new evaluation findings.
Helping America’s Youth
http://guide.helpingamericasyouth.gov/programtool.cfm
This registry was developed with the help of several federal agencies. Programs focus on a range of youth
outcomes and are categorized as Level 1, Level 2, or Level 3 according to their demonstrated effectiveness. The
registry is updated regularly to incorporate new evidence‐based programs.
Office of Juvenile Justice and Delinquency Prevention Model Program Guide
http://www.dsgonline.com/mpg2.5/mpg_index.htm
This registry is one of the largest currently available and is continuously updated to include new programs.
Programs found on this registry are designated as Exemplary, Effective, or Promising.
Promising Practices Network
http://www.promisingpractices.net/
A project of the RAND Corporation, this registry regularly updates its listings of Effective and Promising
programs. Programs are reviewed and endorsed by project staff.
Strengthening Americaʹs Families
http://www.strengtheningfamilies.org/html/
Although this registry was last revised in 1999, it is the only registry with a focus specifically on family‐based
programs. Programs were reviewed by expert panels and staff at the University of Utah and the Center for
Substance Abuse Prevention. They were then designated as Exemplary I, Exemplary II, Model, or Promising.
Substance Abuse and Mental Health Services Administration (SAMHSA) National Registry of Evidence‐
Based Programs and Practices
http://www.nrepp.samhsa.gov
This recently re‐launched site no longer categorizes programs as Model, Effective, or Promising. Instead,
programs are summarized and the quality of the research findings is rated separately for each outcome that has
been evaluated. SAMHSA has also introduced a “Readiness for Dissemination” rating for each reviewed program.
Nominations are accepted each year for programs to be reviewed; SAMHSA funds independent consultants to
review nominated programs and update the registry.
uations allow for a reasonable assumption that it
was the program itself that changed people’s
knowledge, attitudes or behavior.
As funders and program sponsors become more
committed to implementing evidence‐based pro‐
grams, program developers are increasingly likely
to promote their programs as evidence‐based.
However, just because a program developer ad‐
vertises a program as evidence‐based doesn’t
mean that it meets the standards discussed above.
For example, a program might be “research‐
based,” but not “evidence‐based.” A research‐
based program has been developed based on
research about the outcomes or processes it add‐
resses. However, it has probably not been
subjected to the rigorous evaluations and real‐
world testing that are needed to designate a
program as evidence‐based. The simplest way to
determine evidence of a program’s effectiveness is
Guidelines for Selecting an Evidence‐Based Program 5
What Works, Wisconsin – Research to Practice Series, #3
Organizational resources:
Questions to ask
What are the training, curriculum, and
implementation costs of the program?
Can your organization afford to implement this
program now and in the long‐term?
Do you have staff capable of implementing this
program? Do they have the qualifications
recommended or required to facilitate the
program?
Would your staff be enthusiastic about a program
of this kind and are they willing to make the
necessary time commitment?
Can this program be implemented in the time
available?
What’s the likelihood that this program will be
sustained in the future?
Are your community partners supportive of your
implementation of this program?
to examine the designations given by well‐estab‐
lished and respected evidence‐based program
registries. Program registries classify programs at
different levels of endorsement based on evidence
of effectiveness for certain participant outcomes.
See Table 1 for an annotated listing of program
registries.
If a program is not listed on a respected registry,
then it is important to seek out scientific evidence
of the program’s effectiveness. At a minimum, you
should review any evaluation studies that have
been conducted by the program developer and
external evaluators. Ideally, these evaluations use
an experimental or quasi‐experimental research
design. Another sign of a high‐quality evaluation
is that its results have been published in a well‐
respected, peer‐reviewed, scientific journal.
An additional indicator of program quality to
consider is the level of training and follow‐up
support available from the program designers.
Some programs have a great deal of resources
available to help program implementers. These
resources can be especially important if you’re
working with a unique audience and need to
make adaptations or if program implementation is
particularly complex. As a general rule, more in‐
tensive training and more follow‐up support from
the program developer will increase the effective‐
ness and sustainability of a program over time.
Some programs provide excellent technical assis‐
tance; staff members are accessible and willing to
address questions that arise while the program is
being implemented. Often this technical assistance
is free, but sometimes program designers charge
an additional fee for it. Therefore, the benefits and
costs of technical assistance should be kept in
mind when selecting an evidence‐based program.
Finally, while the scientific literature and infor‐
mation from the program developer provide key
information about program quality, don’t over‐
look the experience of practitioners who have imp‐
lemented the program. Ask whether they
encountered any obstacles when implementing
the program, whether they believe the program
was effective, which audiences seemed to respond
most positively to the program, and whether they
would recommend the program for your sit‐
uation. This type of information is usually not
included in scientific program evaluations but is a
critically important consideration for most
practitioners.
ORGANIZATIONAL
RESOURCES
A final set of factors to consider when selecting a
program is related to the resources required for
carrying out the program. Consider whether your
organization has the expertise, staff, financial sup‐
port and time available to implement the pro‐
gram. Implementing evidence‐based programs is
usually fairly time‐ and resource‐intensive. For
example, evidence‐based programs often require
facilitators to attend multi‐day trainings or call for
facilitators with particular qualifications. Even if a
program is a good fit for your community, if your
organization doesn’t have the human or financial
resources to adequately implement the program,
its chances of success are limited.
Guidelines for Selecting an Evidence‐Based Program 6
What Works, Wisconsin – Research to Practice Series, #3
WHAT WORKS, WISCONSIN: RESEARCH TO PRACTICE SERIES
This is one of a series of Research to Practice briefs prepared by the What Works, Wisconsin team at the
University of Wisconsin–Madison, School of Human Ecology, and Cooperative Extension, University of
Wisconsin–Extension. All of the briefs can be downloaded from: http://whatworks.uwex.edu
This series expands upon ideas that are discussed in What Works, Wisconsin: What Science Tells Us about Cost‐
Effective Programs for Juvenile Delinquency Prevention, which is also available for download at the address above.
This publication may be cited without permission provided the source is identified as: Small, S.A., Cooney,
S.M., Eastman, G., & O’Connor, C. (2007). Guidelines for selecting an evidence‐based program: Balancing
community needs, program quality, and organizational resources. What Works, Wisconsin Research to Practice
Series, 3. Madison, WI: University of Wisconsin–Madison/Extension.
This project was supported by Grant Award No. JF‐04‐PO‐0025 awarded by the Wisconsin Office of Justice
Assistance through the Wisconsin Governor’s Juvenile Justice Commission with funds from the Office of
Juvenile Justice and Delinquency Prevention.
The authors wish to thank Mary Huser of the University of Wisconsin–Extension for her edits, comments, and
suggestions in the development of this Research to Practice brief.
In addition, when selecting a program it makes
sense to assess your organization’s long‐term
goals and consider which programs have the best
chance of being continued in the future. Programs
that require significant external funding are
especially prone to abandonment after the funding
runs out. Some programs are more readily adopt‐
ed by existing organizations and are easier to
support over the long run. Think about whether a
program has a good chance of being integrated
into the base programming of your organization.
Can the program be continued in the future with
existing staff and resources or will it always
require external support?
Lastly, because many evidence‐based programs
are resource intensive, think about collaborating
with other organizations in the community to
deliver a program. Selecting a program that meets
the needs of two or more agencies may allow for
the pooling of resources, thus enhancing the
likelihood that the program can be adequately
funded, implemented and sustained over time.
Additionally, such an arrangement can lead to
positive, long‐term partnerships with other com‐
munity agencies.
While all three of these factors are important,
some may be more crucial to your organization
than others. The key to selecting the best program
for your particular situation involves balancing
different priorities and trade‐offs and finding a
program that best meets these competing
demands. By selecting a high quality program that
matches the needs of your audience and com‐
munity and the resources of your organization,
you greatly enhance the likelihood that you will
have an effective program that will have a long‐
term impact and improve the lives of its
participants.
A Report to the Wisconsin Governor’s Juvenile Justice Commission
and the Wisconsin Office of Justice Assistance
Stephen A. Small, Arthur J. Reynolds,
Cailin O’Connor, and Siobhan M. Cooney
A joint initiative of the University of Wisconsin–Madison
Schools of Human Ecology and Social Work, and the
University of Wisconsin–Extension, Cooperative Extension
June 2005
This project was supported by funds from the
Office of Juvenile Justice and Delinquency Prevention, Office of Justice Programs.
What Works, Wisconsin
What Science Tells Us about Cost-Effective Programs
for Juvenile Delinquency Prevention
What Works, Wisconsin
What Science Tells Us about Cost-Effective Programs
for Juvenile Delinquency Prevention
Stephen Small, Arthur Reynolds,
Cailin O’Connor, and Siobhan Cooney
University of Wisconsin–Madison
School of Human Ecology
1300 Linden Drive
Madison, WI 53706
sasmall@wisc.edu or areynolds@waisman.wisc.edu
Acknowledgements
We are grateful to the Policy and Legislative Committee members of the Wisconsin
Governor’s Juvenile Justice Commission for their initiative in making this project
possible, and to the staff of the Wisconsin Office of Justice Assistance for their support
and collaboration throughout the project. We also appreciate the editorial assistance of
Mari Hansen of the University of Wisconsin–Madison/Extension.
The views expressed in this report are those of the authors and do not necessarily
represent the official positions or policies of the U.S. Department of Justice,
the State of Wisconsin, or other funders.
This publication may be cited without permission providing the source is identified as: Small, S.A.,
Reynolds, A.J., O’Connor, C., & Cooney, S.M. (2005). What Works, Wisconsin: What science tells us about
cost-effective programs for juvenile delinquency prevention. Madison, WI: University of Wisconsin–Madison.
Table of Contents
EXECUTIVE SUMMARY ……………………………………………………………………………………………………………..5
I. PROJECT OVERVIEW ………………………………………………………………………………………………………………….1
PROJECT HISTORY ………………………………………………………………………………………………………………….1
KEY CONCEPTS ………………………………………………………………………………………………………………………2
Risk-Protection Framework ………………………………………………………………………………………………..2
Positive Youth Development Approaches ……………………………………………………………………………..4
Human Capital Perspective ………………………………………………………………………………………………..4
Evidence-Based Programs and Practices ………………………………………………………………………………4
Registries of Evidence-Based Programs ………………………………………………………………………………..6
Classification of Prevention Programs………………………………………………………………………………….6
Cost-Benefit Analysis ………………………………………………………………………………………………………..7
INFLUENCE OF THE WASHINGTON STATE STUDY ON THE PRESENT REPORT ……………..8
II. COST-EFFECTIVE PROGRAMS TO PREVENT DELINQUENCY AND CRIME ……………………….9
PRINCIPLES OF EFFECTIVE PRIMARY AND SECONDARY PREVENTION PROGRAMS …….9
PRINCIPLES OF EFFECTIVE JUVENILE OFFENDER PROGRAMS ……………………………………….10
SUMMARY OF SELECTED COST-EFFECTIVE PROGRAMS…………………………………………………11
III. REVIEW OF PROGRAMS AND APPROACHES……………………………………………………………………….15
PRIMARY PREVENTION PROGRAMS ………………………………………………………………………………….15
Preschool Education…………………………………………………………………………………………………………15
Family Support Programs ………………………………………………………………………………………………..17
Social-Emotional Learning Programs…………………………………………………………………………………21
SECONDARY PREVENTION PROGRAMS…………………………………………………………………………….23
Family Training Programs ……………………………………………………………………………………………….23
Social Skills Training Programs ………………………………………………………………………………………..23
Mentoring Programs ……………………………………………………………………………………………………….24
Vocational/Job Training Programs …………………………………………………………………………………….25
Emerging and Unproven Programs……………………………………………………………………………………26
JUVENILE OFFENDER PROGRAMS ……………………………………………………………………………………..27
Juvenile Offender Program Categories and Considerations …………………………………………………..27
Diversion/Community Accountability Programs ……………………………………………………………….28
Therapeutic Interventions…………………………………………………………………………………………………30
Case Management/Multimodal Interventions……………………………………………………………………..31
Emerging and Unproven Programs and Approaches …………………………………………………………..32
IV. CONSIDERATIONS FOR IMPLEMENTING EVIDENCE-BASED PROGRAMS …………………….34
SELECTING AN APPROPRIATE EVIDENCE-BASED PREVENTION PROGRAM ………………..34
STRATEGIES FOR SUCCESSFULLY IMPLEMENTING PROGRAMS …………………………………….34
BARRIERS TO IMPLEMENTING EVIDENCE-BASED PROGRAMS ………………………………………36
CULTURAL/ETHNIC CONSIDERATIONS…………………………………………………………………………….37
V. RECOMMENDATIONS ……………………………………………………………………………………………………………..38
VI. CONCLUSIONS………………………………………………………………………………………………………………………….43
VII. REFERENCES ……………………………………………………………………………………………………………………………..44
APPENDIX A: EVIDENCE-BASED PROGRAM DETAILS
APPENDIX B: EVIDENCE-BASED PROGRAM DESCRIPTIONS
APPENDIX C: LIST OF PROGRAM REGISTRIES
APPENDIX D: GUIDELINES FOR SELECTING
EVIDENCE-BASED PROGRAMS
What Works, Wisconsin 5
EXECUTIVE SUMMARY
Significant advancements have been made
in the science of juvenile delinquency preven-
tion in recent years. There is considerable
evidence that some approaches are more effec-
tive than others in preventing crime and
reducing recidivism among youth. Research has
shown that implementing proven, scientifically
sound programs and interventions can have a
preventive effect, making it less likely that indi-
viduals will engage in crime and equipping
them to make positive contributions to society.
Many of these programs have been shown to
result in economic benefits to society far out-
weighing their costs. Through the use of
evidence-based programs,
practices and policies, the
state of Wisconsin can more
effectively address the
problem of juvenile delin-
quency while making the
best use of increasingly lim-
ited financial resources.
This report builds on
several recent efforts to analyze the growing
evidence in the field of delinquency prevention.
Most notably, the Washington State Institute for
Public Policy’s cost-benefit analysis of dozens of
prevention and intervention programs related to
juvenile delinquency provided the impetus for
this report. The Office of Juvenile Justice and
Delinquency Prevention’s Model Programs
Guide also informed our work. Drawing on
those resources, we highlight a number of
proven, effective programs and review the
strongest evidence available in several catego-
ries of interventions, from universal or primary
prevention with children and families through
community-based programs for juvenile offend-
ers. Special attention is also given to several
programs and approaches developed in Wiscon-
sin.
KEY CONCEPTS
A number of principles and frameworks
guide current thinking about the prevention and
treatment of youth problems in general and de-
linquency in particular. Most fundamental to
current approaches is the risk-protection
framework. Certain traits of individuals, fami-
lies, and communities have been identified as
risk factors for the development of anti-social
behavior and negative outcomes; criminologists
refer in particular to risk factors for delinquent
or criminal activity as “criminogenic factors.”
Other traits, called protective factors, make
negative outcomes less likely and positive out-
comes more likely. Identify-
ing risk and protective
factors makes it possible to
target relevant attitudes,
needs, or behaviors among
people at risk for negative
outcomes, and provide ser-
vices and resources that
promote positive outcomes.
A related perspective that has informed many
recent prevention and youth programming ini-
tiatives identifies “developmental assets,” the
building blocks necessary for healthy develop-
ment.
The human capital perspective is central to
understanding not only the reasoning behind
offering programs to at-risk individuals, but also
the concept of cost-benefit analysis. Human
capital refers to the investment of resources to
increase the social, emotional, and educational
skills of children, parents, and families. The hu-
man capital perspective emphasizes that
investments in young people’s education and
development can produce economic returns to
the general public and personal returns to indi-
viduals. All of these “returns” can be included
when estimating the effect of a program or in-
tervention.
Through the use of evidence-
based programs, Wisconsin can
more effectively address the
problem of juvenile delinquency
while making the best use of
limited financial resources.
What Works, Wisconsin Page ii
Putting the human capital perspective to
use, information on the effects of programs and
interventions can be converted into monetary
values and compared to the investment required
to achieve those effects. The economic benefits
of prevention programs cover a wide range, in-
cluding, but not limited to, reduced delinquency
and crime, increased economic well-being of
participants (and associated tax revenues), and
cost savings within major public service and
rehabilitative systems. Cost-benefit analysis
(CBA) offers a practical method for helping poli-
cymakers consider alternative programs when
faced with limited resources. CBA is a major
departure from traditional measures of program
effectiveness, which take into account only the
strength of a program’s impact, while ignoring
the costs. Using CBA, program options can be
ranked according to their effectiveness per dol-
lar of expenditure.
CBA necessarily builds on evidence gener-
ated from high-quality program evaluations.
The programs for which this type of information
is available are called evidence-based programs.
Evidence-based programs are those that have
been shown through scientific research and
evaluation to be effective and reliable. They
have been subjected to rigorous evaluation us-
ing comparison groups, often including long-
term follow-up to track various outcomes for
program participants and non-participants. A
number of federal agencies and research organi-
zations maintain registries of evidence-based
programs to guide the selection of programs for
prevention of problem behaviors. Evidence-
based programs are somewhat rare, but growing
in number.
REVIEW OF PROGRAMS
In this report, we review the available evi-
dence in ten categories of programs, grouped
into three broad areas:
PRIMARY PREVENTION
•
Preschool Education
•
Family Support Programs
•
Social-Emotional Learning Programs
SECONDARY PREVENTION
• Family Training Programs
• Social Skills Training Programs
•
Mentoring Programs
•
Vocational/Job Training Programs
JUVENILE OFFENDER PROGRAMS
• Diversion or Community
Accountability Programs
• Therapeutic Interventions
• Case Management/Multimodal
Interventions
Within each category, we highlight one or
two evidence-based programs, including cost-
benefit information whenever it is available. We
also discuss the practices and approaches that
appear to increase program effectiveness within
each category.
The most cost-effective prevention pro-
grams reviewed in this report include preschool
education, home visitation programs, and social
and emotional learning programs for elemen-
tary school children. In all of these programs,
the quality and intensity of services are high,
staff members are well trained, and the program
has a well-articulated vision with a strong con-
ceptual base. Although mentoring and job
training programs were also found to have good
evidence of effectiveness, their economic returns
are lower.
Among juvenile offender programs, the
strongest empirical evidence of cost-
effectiveness is for diversion programs and
therapeutic interventions that provide a range of
intensive services over relatively long periods of
time. Overall, there are fewer evidence-based
juvenile offender programs than prevention
programs. However, a number of principles of
effective intervention have been identified that
can increase the likelihood that a given program
or approach will be effective.
Throughout the review of programs, we also
identify emerging or unproven delinquency
prevention programs that appear to follow key
principles of effective programs but have not yet
What Works, Wisconsin Page iii
demonstrated reliable program impacts. These
include after-school programs and enhanced
probation and supervision programs. Addi-
tional research on these and other programs is
needed.
CONSIDERATIONS IN IMPLEMENTING
EVIDENCE-BASED PROGRAMS
Knowing that a program has undergone
rigorous evaluation and has strong evidence
that it works is an essential first step in moving
toward more effective, evidence-based practice.
However, implementing a program that will
have the desired effect involves a great deal
more than just using an evidence-based pro-
gram. Among other things, it requires that the
program selected be appropriate to the audi-
ence, that it is adequately funded and staffed,
and that the selected program is implemented
with fidelity. These less tangible matters are of-
ten overlooked by program sponsors, but are as
important as the program model itself if the
program is to have a positive impact. In addition
to these issues, there exist a number of practical
considerations related to the realities of program
administration, which are often barriers to the
use of evidence-based programs. These consid-
erations are articulated in the full report.
RECOMMENDATIONS
Based on our synthesis of the available evi-
dence on the effectiveness of juvenile crime
prevention programs, we make the following
recommendations for consideration by the Wis-
consin Governor’s Juvenile Justice Commission:
(1) Strongly support the use of evidence-
based prevention and intervention pro-
grams and practices.
(2) Educate policy-makers and practitioners
about evidence-based programs and prac-
tices and their practical and economic
benefits.
(3) Use results of cost-benefit analysis to bet-
ter prioritize funding of education and
social programs.
(4) Adopt an appropriate and validated as-
sessment tool in order to direct juvenile
offenders to the level of intervention and
supervision that is most likely to be effec-
tive for them.
(5) Develop mechanisms for disseminating
effective program models and good prac-
tice guidelines to practitioners and
decision-makers.
(6) Provide support for local-level delin-
quency prevention initiatives.
(7) Increase investments in research and de-
velopment (R & D) and in evaluation of
emerging, innovative, and promising pre-
vention programs.
(8) Provide a greater balance between preven-
tion and intervention programs and
strategies.
(9) Create new, state-level, operational poli-
cies that encourage cross-agency
collaboration and funding for prevention.
(10) Develop new state funding mechanisms
that are equitable and consistent with the
economic benefits of prevention pro-
grams.
These recommendations have the potential
to positively alter the future life chances of Wis-
consin youth, reduce crime, and contribute to
significant cost savings. However, putting into
action most of these recommendations will re-
quire both vision and courage – the vision to
look beyond short-term solutions and the cour-
age to challenge the status quo and adopt new
ways of operating. We hope that this report will
serve as an impetus for change and contribute to
the emergence of Wisconsin as a national leader
for innovative, scientific, and cost-effective poli-
cies and programs on behalf of its youth.
What Works, Wisconsin Page 1
I. PROJECT OVERVIEW
Significant advancements have been made
in the science of juvenile delinquency preven-
tion in recent years. There is considerable
evidence that some approaches are more effec-
tive than others in preventing crime and
reducing recidivism among youth. Research has
shown that high-quality implementation of evi-
dence-based programs and principles often
results in reduced delinquency and future re-
cidivism as well as economic benefits to society
that outweigh expenditures. Through the use of
evidence-based programs,
practices and policies, the state
of Wisconsin can more effec-
tively address the problem of
juvenile delinquency while
making the best use of increas-
ingly limited financial
resources.
In response to a request
from the Wisconsin Governor’s Juvenile Justice
Commission and the Wisconsin Office of Justice
Assistance, this report synthesizes the current
research on scientific approaches to delinquency
prevention and community-based intervention.
In addition, this report provides an overview of
the latest concepts, terms and models related to
juvenile delinquency prevention and interven-
tion. It concludes with recommendations
regarding future directions for the state of Wis-
consin that can lead to more effective and
economically viable approaches for reducing
juvenile crime, enhancing youth development
and contributing to responsible citizenship.
Three questions guide this report:
• What does science tell us about effective
approaches to preventing and treating
delinquency?
• What are the economic returns of the
most effective programs?
• What steps should the state take to de-
velop a more effective and cost-
beneficial strategy for preventing juve-
nile delinquency and future crime?
PROJECT HISTORY
Over the past five years, our knowledge
about cost-effective programs, practices and
principles in the field of juvenile justice and de-
linquency prevention has grown at a
phenomenal rate. The pace is so rapid that even
in the process of developing the direction and
scope of this report, new information became
available that led to modifica-
tions in what we and the
Wisconsin Governor’s Juvenile
Justice Commission intended
to accomplish through the
present report.
Initially, we had planned
to provide an update and ex-
tension of the Washington State Institute for
Public Policy’s 2001 report on The Comparative
Costs and Benefits of Programs to Reduce Crime [1].
However, within weeks of our beginning this
project, Washington State released a new ver-
sion of their report that essentially did just that
(see Benefits and Costs of Prevention and Early In-
tervention Programs for Youth [2]). The
availability of this new report meant that much
of the work on estimating costs and benefits for
delinquency and recidivism prevention pro-
grams had already been accomplished, freeing
us up to address other important issues.
The current report builds on existing re-
sources, including the Washington State report
as well as the Office of Juvenile Justice and De-
linquency Prevention’s (OJJDP’s) web-based
Model Programs Guide [3]. Consequently, the cur-
rent report includes a discussion of most of the
key findings from these previously released
documents but also goes well beyond them. We
provide here a fairly broad and inclusive sum-
mary of what is known about the prevention of
delinquency and recidivism in community-
Our knowledge about cost-
effective programs, practices
and principles in the field of
delinquency prevention has
grown at a phenomenal rate.
What Works, Wisconsin Page 2
based programs. Drawing on available cost-
benefit analyses, we also discuss what is known
about the potential cost savings of various ap-
proaches to prevention and community-based
juvenile offender programs. In addition, build-
ing on the OJJDP Model Programs Guide and
the numerous other guides and registries that
have been developed in recent years, we pro-
vide an overview of what is known about
proven, effective approaches to the reduction of
delinquency, including a synthesis of the princi-
ples that underlie successful programs. Finally,
we provide a review of some of the strategies
that have been found to be critical to the success-
ful selection and implementation of prevention
and early intervention programs.
KEY CONCEPTS
There are a number of principles and
frameworks that guide current thinking about
the prevention and treatment of youth problems
in general and delinquency in particular. In this
section, we provide an overview of some of the
most relevant frameworks, concepts and princi-
ples as a way to orient the reader to the current
scientific thinking in this area and provide a
foundation for our later discussion of what
works in delinquency prevention.
Risk-Protection Framework
Most scholars and professionals in the field
of prevention are guided by some variation of
the risk-protection framework. This approach
assumes that the best way to prevent delin-
quency or other problematic outcomes is to
reduce or eliminate risk factors and to increase
or enhance protective factors [4]. Risk factors
typically are defined as individual or environ-
mental markers that are related to an increased
likelihood that a negative outcome will occur [5,
6]. Conversely, protective factors usually are
defined as individual or environmental safe-
guards that enhance a person’s ability to
overcome stressful life events, risks or hazards
and promote adaptation and competence [7, 8].
Criminologists use the term “criminogenic fac-
tors,” which describes risk factors that (a) have
been empirically linked to delinquent or crimi-
nal behavior, and (b) are dynamic, or amenable
to change through intervention in an individ-
ual’s life [9].
Most prevention researchers and practitio-
ners view prevention within an ecological
framework [e.g., 5, 10, 11] which assumes that
risk and protective factors can exist both within
individuals and across the various settings in
which they live such as the family, peer group,
school, and community. Closely related is the
idea that most problems are multiply deter-
mined [11]. That is, there may be diverse paths
to the development of a particular problem like
delinquency, and efforts to address a single
cause are likely to fail, because most problems
have multiple causes. Similarly, the same risk
factor can be related to a variety of different out-
comes [8]. For example, many criminogenic
factors, which put youth at risk for delinquency,
also put them at risk for early parenthood or
school failure. Thus, efforts to prevent youth
problems must account for and target multiple
settings and risk factors [11, 12]. Figure 1 pre-
sents a list of common risk factors and protective
factors for juvenile delinquency organized by
ecological level.
What Works, Wisconsin Page 3
FIGURE 1.
Common Risk and Protective Factors for Juvenile Delinquency
RISK FACTORS PROTECTIVE FACTORS
INDIVIDUAL LEVEL
• Early initiation of problem be
havior
• Early and persistent antisocial behavior
• Low IQ
• Hyperactivity
• Rebelliousness
• Favorable attitudes toward deviant behavior
• Involvement in other problematic or dangerous be-
havior
• High IQ
• Intolerant attitudes toward deviant behavior
• Positive social orientation
• Ability to feel guilt
• Trustworthiness
FAMILY LEVEL
• Family history of criminal or delinquent behavior
• Family conflict or violence
• Favorable parental attitudes and involvement in
problem behavior
• History of maltreatment
• Parental psychopathology
• Teenage parenthood
• Poverty
• Good relationships with parents
• Good family communication
• Parents/caregivers who possess strong parent-
ing skills
PEER LEVEL
• Friends who engage in delinquent behavior
• Peer rejection
• Non-delinquent friends (or prosocially oriented
friends)
SCHOOL LEVEL
• Academic failure or poor performance beginning in
late elementary school
• Lack of commitment or bonding to school
• Low academic aspirations
• Positive commitment to school
• Academic achievement
• Strong school motivation
• Positive attitude toward school
COMMUNITY LEVEL
• Availability of drugs and weapons
• Low neighborhood attachment and community dis-
organization
• Media portrayals of violence
• Extreme economic deprivation
• Concentration of delinquent peer groups
• Non-disadvantaged neighborhood
• Low neighborhood crime
• Community norms and laws that condemn
drug use, crime and deviant behavior
• High neighborhood stability and cohesion
SOURCE: Adapted from Preventing & Reducing Juvenile Delinquency (2003), p. 105. By J.C. Howell. Thousand Oaks, CA: Sage Publi-
cations [13] and Child Delinquents: Development Intervention and Service Needs (2003), R. Loeber and D. Farrington, (Eds.). Thousand
Oaks, CA: Sage Publications [14].
What Works, Wisconsin Page 4
Positive Youth Development Approaches
An increasingly popular approach to prob-
lem prevention emphasizes the positive aspects
of youth development and well-being. This ap-
proach is especially common with traditional
youth serving agencies that provide after school
and non-formal educational programs, and with
initiatives concerned with facilitating youth in-
volvement in the community. In contrast to
traditional prevention programs, positive youth
development assumes that simply preventing
problems is not enough to prepare youth for
adulthood, and that the best way to prevent
problems from occurring throughout the life-
span is to promote the developmental potential
of young people [8, 15].
While there exist a number of positive youth
development models, in Wisconsin the most
widely used approach is the Search Institute’s
Developmental Assets framework [16]. This
model is built around 40 developmental assets,
defined as the building blocks that are crucial
for promoting healthy youth
development and well-being
[17]. According to the Search
Institute, assets center on the
relationships, social envi-
ronments, patterns of
interactions, and norms that
are central to promoting
youth development. Correla-
tional data from the Search Institute show that
the more developmental assets a young person
possesses, the fewer problems they exhibit (in-
cluding delinquency) and the greater the
likelihood that they will experience positive de-
velopmental outcomes like school success and
social responsibility [18].
Human Capital Perspective
Preventive interventions are increasingly
conceptualized from a human capital perspec-
tive, recognizing that social programs for
children and youth are “investments” that pro-
mote well-being among participants and for
society at large. Human capital is a general iden-
tifier for investments of human and financial
resources to increase the social, emotional, and
educational skills of children, parents, and fami-
lies. If they are substantial enough, these
resources can improve learning and behavior in
the short-term as well as economic and social
well-being in the long-term. Generally, the ear-
lier in the life course that an intervention occurs,
the greater its capacity for creating enduring
effects, with long-term consequences for indi-
viduals, their families, and the communities in
which they live [19, 20]. The human capital per-
spective emphasizes that investments in young
people’s education and development can have
economic returns to the general public as well as
personal returns to individuals, and that all of
these “returns” can be included when estimating
the effect of a program or intervention.
Evidence-Based Programs and Practices
The need for proven, effective, high quality
prevention and intervention programs remains a
high priority in Wisconsin and
across the nation.
Unfortunately, the effective-
ness of many current
programs and practices re-
mains unproven at best,
while some are known to be
ineffective or even harmful.
Increasingly, state, federal
and private funders are ask-
ing that the programs they
fund have solid research evidence that they
work. These programs are known as evidence-
based programs.
Evidence-based programs incorporate
strategies, activities and principles that have
been shown through scientific research and
evaluation to be effective and reliable. In order
for programs to be considered evidence-based,
they must go through a process that draws on
scientifically gathered information and uses a
commonly agreed upon criteria for rating re-
search interventions, principles and strategies.
Such programs:
The need for proven, effective,
high quality prevention and
intervention programs remains
a priority in Wisconsin and
across the nation.
What Works, Wisconsin Page 5
• Are based on a solid, scientific, theoreti-
cal foundation
• Have been carefully implemented and
evaluated using rigorous scientific
methods that usually include:
o A control or comparison group
o Well-established measures and
methods
• Have been evaluated in a variety of set-
tings with a range of audiences
• Have evaluation findings that have been
subjected to critical review by other re-
searchers and published in respected
scientific journals
• Have been certified as evidence-based
by a federal agency or well-respected
research organization based on the
above criteria
There has been some confusion regarding
the terms used to describe evidence-based pro-
grams. Some agencies and professionals refer to
them as research-based or science-based programs.
In addition, the term model programs is often
used interchangeably with exemplary programs.
In most cases, they are based on similar princi-
ples and criteria for scientific evidence and are
therefore comparable.
There is, however, a distinction between evi-
dence-based programs and evidence-based practices
or principles of effectiveness. Evidence-based prac-
tices and principles are elements that increase
effectiveness across a variety of programs, rather
than self-standing programs that can be ac-
quired and implemented. Researchers try to
identify these “active ingredients” to increase
the likelihood that programs – whether evi-
dence-based or not – will be effective.
It is important to keep in mind that the cur-
rent scientific evidence is at the level of the
program, not at the level of practices or princi-
ples. Because each program is an integration of
many elements, the contributions of individual
program components (staffing, learning activi-
ties, timing, or duration) to the total effect are
not estimated. Syntheses of the evidence across
different programs lead to identification of a
common set of principles, which are valuable,
but not “evidence-based” in the same way that
individual programs can be.
There are a number of reasons for the grow-
ing interest in the use of evidence-based
programs. The first is accountability. The public
wants to know that tax dollars are being spent
on programs and services that actually work.
Similarly, public agencies and their funders in-
creasingly want to invest their dollars in
programs that have scientifically demonstrated
their effectiveness. In fact, it is becoming more
common for funders to mandate that evidence-
based programs be used by the organizations
they fund. Another reason for the growth of
evidence-based programs is efficiency. Instead
of having to “reinvent the wheel,” agencies can
select from the growing number of programs
that are known to be well designed and have
undergone rigorous evaluation. Finally, and
perhaps most importantly, when these pro-
grams are implemented properly, there is good
evidence that they will have the positive impact
that they were designed to produce. In this way,
evidence-based programs enable limited public
resources to be used wisely.
The growth of evidence-based prevention
programs is a relatively recent phenomenon.
While there were few such programs prior to the
1990s, over the past decade the number of pro-
grams has grown quickly and continues to
increase at a rapid pace. Unfortunately, while
there has been a remarkable growth in the num-
ber of evidence-based prevention programs,
their adoption and use by practitioners lags far
behind. One recent study on the use of evidence-
based programs by schools found that less than
a third of the prevention programs used by pub-
lic schools were evidence-based [21]. In the field
of juvenile justice, the percentage of programs
that are evidence-based may be even lower.
What Works, Wisconsin Page 6
Registries of Evidence-Based Programs
As the number of evidence-based programs
has grown, various federal agencies and non-
profit research and education organizations
have developed web-based program registries
that certify and list programs that have met the
standards of being evidence-based. The most
rigorous of these are listed in Appendix C of this
report.
Many of these registries focus on a particu-
lar area of interest such as drug abuse, violence
or juvenile delinquency. Others are more gen-
eral in that they attempt to cover programs that
address a range of outcomes. In order for a pro-
gram to be included in a registry, it must have
met the sponsoring organization’s criteria of
being evidence-based. Some registries also in-
clude programs that are classified as promising.
These are programs that are based on good re-
search and theory and usually have some
preliminary evidence of being effective, but
have not been evaluated as rigorously, as often,
or with as much long-term follow-up as the
model or exemplary programs.
There is a great deal of overlap between
evidence-based program registries. This is true
even for registries that are fairly diverse in the
outcomes or problems that they are concerned
with. The reason for this is that many evidence-
based programs have been found to have a posi-
tive impact on addressing a range of
problematic and positive outcomes. For exam-
ple, some of the most effective family-based
programs have been found to not only reduce
drug abuse among youth, but also help prevent
aggressive behavior and promote positive de-
velopmental outcomes like school success.
Classification of Prevention Programs
Traditionally, three different types of ap-
proaches to prevention have been identified:
primary, secondary, and tertiary [22]. Primary
prevention is concerned with preventing the
initial occurrence of a problem within a normal
population. Secondary prevention involves in-
tervening with populations that show signs of
early problems, so that more serious problems
can be prevented. Tertiary prevention involves
the reduction of a problem among groups of
people already experiencing it [23].
A more recent development in the classifica-
tion of prevention programs comes from the
Institute of Medicine [24]. In this system, pro-
grams are classified based on the characteristics
of the audience they target. The three categories
of prevention programs are known as universal,
selective and indicated. Universal prevention in-
volves interventions directed at the general
public or an entire population (e.g., all middle
school students); selective prevention programs
are directed at a subgroup of a population that is
at risk of developing the problem but is not yet
exhibiting any difficulties; indicated prevention
involves interventions targeted at high risk in-
dividuals who show some signs or symptoms of
a problem. In terms of old and new terminology,
primary and universal prevention often are used
interchangeably, as are indicated and secondary
prevention [4].
COMMON LABELS
USED TO DESCRIBE
EVIDENCE-BASED PROGRAMS
Evidence-based programs and practices
are often referred to by different names by
various organizations. Though this can be
confusing, these labels are for the most
part, comparable. Some of the common
labels used to describe programs that meet
the criteria of evidence-based include:
• Exemplary Programs (U.S. De-
partment of Education)
• Effective Programs (CDC, NIDA,
DHHS)
• Model Programs (SAMHSA,
OJJDP, Blueprints, Surgeon Gen-
eral)
• Proven Programs (Promising Prac-
tices Network)
• What Works (Child Trends)
What Works, Wisconsin Page 7
Cost-Benefit Analysis
Cost-benefit analysis (CBA) offers a practical
method for helping policymakers consider al-
ternative programs while facing limited
resources. CBA is a major departure from tradi-
tional measures of program effectiveness, which
take into account only the strength of a pro-
gram’s impact, while ignoring the costs. Using
CBA, program options can be ranked according
to their effectiveness per dollar of expenditure.
In this report, we provide estimates of the eco-
nomic returns in 2004 dollars for a wide range of
crime prevention programs.
Levin and McEwan define CBA as the
“evaluation of alternatives according to their
costs and benefits when each is measured in
monetary terms” [25, p. 11]. The use of cost-
benefit analysis to document the payoffs of edu-
cation, prevention, and human service programs
has increased in recent years [2,
19]. When program outcomes
cannot be easily converted to
monetary terms, cost-
effectiveness analysis (CEA) is
recommended. In CEA, pro-
gram costs are in monetary
terms but benefits remain in the
metric of the outcome measure, such as the per-
cent decrease in a particular undesirable
behavior. The major advantage of CBA over
CEA, however, is that benefits for multiple out-
comes can be summarized in dollar terms,
expressed as either the net return (benefits mi-
nus costs) or return per dollar invested (benefits
divided by costs).
The ability to conduct a CBA depends on
whether or not it is possible to capture program
benefits and costs in dollar terms. Typically, it is
more difficult to calculate the dollar value of
benefits than costs. Calculating program costs is
relatively straightforward given that most costs
can be identified and monetized. However, be-
cause the outcomes of many prevention and
treatment programs are not limited to delin-
quency and crime, the estimated economic
returns must take into account all the effects of
program participation. Experiences that place
children and youth at risk of delinquency
should be taken into account (such as child mal-
treatment and school underachievement) as well
as later behaviors that are expected conse-
quences of delinquency (such as lower
educational attainment and economic hardship).
As with nearly any assessment of program im-
pact, another key requirement of CBA is that
equivalent outcomes are available for a control
group who received no program services (or
received “the usual services”) and for an ex-
perimental group whose members received the
modified or new program or services of interest.
Consequently, the economic benefits of a par-
ticular program are derived from the average
performance of program participants relative to
control group participants.
The economic benefits of prevention pro-
grams cover a wide range and
include youth delinquency,
adult criminal behavior and
related outcomes that can be
readily converted to economic
benefits. These outcomes in-
clude increased economic well-
being of participants (and asso-
ciated tax revenues) and cost
savings associated with the administration and
treatment of children and youth in major service
systems. The two categories that account for the
largest share of potential economic returns are
(a) cost savings to the criminal justice system
and crime victims and (b) increased earning ca-
pacity, sometimes measured indirectly by higher
educational attainment. The first category is a
benefit to the general public (i.e., taxpayer, gov-
ernments, and crime victims). The second is
primarily a benefit to program participants, but
also has direct benefits for society when in-
creased tax revenues are taken into
consideration.
CBA is generally used to report on the eco-
nomic benefits to society at large, which is the
sum of estimated benefits to the general public
and to program participants. The two major in-
dicators of these benefits are the net program
Cost-benefit analysis
(CBA) offers a practical
method for helping
policymakers consider
alternative programs.
What Works, Wisconsin Page 8
benefit and the benefit-cost ratio, which repre-
sents the economic benefit to society per dollar
invested in the program. Values above $1.00
indicate that economic returns exceed costs.
Note also that both actual and projected
benefits are generally included in estimated
benefits. Benefits can occur in the short- and
long-run. The extent to which evaluations ad-
dress long-term impacts depends on a range of
factors including the goals of the program,
whether sufficient time has elapsed to evaluate
target outcomes for program participants, and
whether investments have been made to evalu-
ate the effects of the program over the long run.
Primary prevention programs in the early child-
hood area, for example, are more likely to be
assessed for long-run effects a decade or more
post-program, whereas programs for juvenile
offenders usually have a much shorter time ho-
rizon and are unlikely to track participants past
the age of 18. It is worth noting that for many
programs, the research findings are still limited
to immediate program impacts or short-term
follow-up studies. However, as longer-term fol-
low-up becomes more widespread, evidence for
the impacts of these programs – and better esti-
mates of the long term cost savings – will
become available.
INFLUENCE OF THE WASHINGTON
STATE STUDY ON THE PRESENT
REPORT
A major impetus for this report was the re-
cent report commissioned and funded by the
Washington State legislature. Aos and col-
leagues at the Washington State Institute for
Public Policy estimated the economic benefits
and costs of a wide range of prevention and in-
tervention programs to enhance the well-being
of children and youth [2]. Unlike their earlier
report [1], the researchers measured economic
benefits associated with both crime prevention
and non-crime outcomes such as substance
abuse prevention and educational attainment.
The study’s investigation of the effects of differ-
ent programs was intended to advance public
policies on behalf of Washington’s families and,
by implication, those across the country.
The Washington State report is notable be-
cause it is the most complete account to date of
the benefits and costs of social programs for
children and youth. Spanning 1970 to 2003,
evaluations of seven types of programs were
synthesized including pre-kindergarten educa-
tion, child welfare/home visitation, youth
development, mentoring, teenage pregnancy
prevention, youth substance abuse prevention,
and juvenile offender programs. The assessed
outcomes for estimating economic benefits were
in seven areas: crime, substance abuse, educa-
tion outcomes (e.g., achievement, school
completion), teenage pregnancy and parent-
hood, suicide attempts, child abuse, and
domestic violence. Aos and colleagues found
that programs and intervention approaches in
four areas were particularly cost-effective. These
included youth development programs with the
goal of enhancing decision-making skills or par-
ent-child relations; juvenile offender programs
such as diversion interventions and family
therapies; home visitation programs that pro-
vide intensive services; and preschool education
for 3- and 4-year-olds.
As with the Washington State report, our
report includes information on cost-effectiveness
as a major indicator of program impact. We de-
scribe programs that were identified in their
report as well as in other publications. Never-
theless, our report differs in several important
ways. First, we emphasize programs that have
demonstrated relatively high levels of cost-
effectiveness in achieving their goals. Thus, the
programs we describe are a select list that we
believe provide the most payoff per dollar in-
vested. Second, we base our CBA information
on the original (or in some cases, most recent)
cost-benefit analyses conducted for each pro-
gram whenever possible. When no such original
analyses exist, we revert to the Washington State
study estimates, converted to 2004 dollar values.
Finally, the present report goes well beyond the
provision of cost-benefit information. The suc-
cess of any particular program involves many
What Works, Wisconsin Page 9
factors aside from cost-effectiveness, such as its
appropriateness for the target audience and how
well it is implemented. In this report we include
a thorough review of such factors so that deci-
sion makers and practitioners can have a more
complete picture of what it might take to select,
fund and implement a cost-effective, evidence-
based program directed at the prevention of ju-
venile delinquency.
II. COST-EFFECTIVE PROGRAMS TO
PREVENT DELINQUENCY AND CRIME
Based on a comprehensive review of the
available evidence on crime prevention pro-
grams, we describe effective programs for
children and youth in three major classes: pri-
mary prevention programs, secondary
prevention programs, and juvenile offender
programs. Primary prevention programs are
interventions such as preschool programs that
occur before children exhibit problems or anti-
social behaviors. Secondary prevention pro-
grams are implemented during the early cycle of
delinquency when misbehavior and conduct
problems are evident, but prior to full blown
delinquency. Juvenile offender programs, which
are often referred to as treatment programs or
tertiary prevention programs, are interventions
that occur after youth have been arrested. The
goal of juvenile offender programs is to prevent
recidivism. For the present report, we limit our
discussion of juvenile offender programs to non-
residential programs and approaches that are
delivered to juvenile offenders in the commu-
nity.
Before describing the most effective pro-
grams from our review, we summarize broad
principles that appear to make prevention and
juvenile offender programs effective and, for
many of them, exemplary.
PRINCIPLES OF EFFECTIVE PRIMARY
AND
SECONDARY PREVENTION
PROGRAMS
The most effective primary and secondary
prevention programs related to delinquency
share a number of key principles. Though there
is great value in using established, proven pro-
grams, that is not always possible. There will
always be new and emerging programs that do
not yet have the evaluation data to document
their impact. Drawing on a range of studies and
comprehensive reviews [26-28], these principles
can serve as guidelines to those who are devel-
oping new programs, wish to improve existing
programs, or have the responsibility for making
funding decisions about what is likely to work.
Primary and secondary prevention pro-
grams have similar principles of effectiveness.
Effective programs are:
• Delivered at a high dosage and inten-
sity – Effective programs tend to have
relatively greater amounts of contact
time with participants whether it be
number of sessions, hours, weeks, or
years.
• Comprehensive – Multi-component
programs that address a variety of risk
and protective factors are usually more
effective than single-component pro-
grams.
• Appropriately timed – The most effec-
tive programs address relevant factors
or processes at specific times of need,
and when participants are most recep-
tive to change.
• Developmentally appropriate – Pro-
grams should be age and develop-
mentally appropriate for the target au-
dience of children, youth, or adults.
What Works, Wisconsin Page 10
• Socio-culturally relevant – Tailoring the
program to the cultural traditions of
youth and their families enhances re-
cruitment, retention and sometimes
outcome attainment.
• Implemented by well-trained, effective
staff – The effectiveness of a program is
tied to the staff’s personal characteristics
such as efficacy and confidence, and
their level of training, either by educa-
tion or experience.
• Supported by strong organizations –
Effective programs receive administra-
tive support, have low employee
turnover rates, and have staff members
who share the same vision.
• Implemented using varied, active
methods – Interactive skills training
methods are much more likely than di-
dactic lecturing to increase program
effectiveness and client satisfaction.
• Based on strong theory – High-quality
programs have a strong theoretical justi-
fication, are based on accurate
information, and are supported by em-
pirical research.
• Evaluated regularly – Staff members are
able to make modifications and improve
program effectiveness when they sys-
tematically document and reflect on
implementation, processes, and results.
PRINCIPLES OF EFFECTIVE JUVENILE
OFFENDER PROGRAMS
Criminologists studying the effectiveness of
approaches and programs for juvenile offenders
[e.g., 9, 29, 30] have identified several principles
of effective intervention. These principles are
based on meta-analyses1 of hundreds of pro-
1 Meta-analysis is an increasingly common statistical
technique for analyzing and comparing the findings
of a body of previously conducted empirical studies.
A meta-analysis is a study of existing studies that
allows researchers to examine whether there are any
systematic patterns or findings that emerge across
studies.
gram evaluations and reviews of the records of
thousands of adult and juvenile offenders [31].
These principles include:
• The human service principle –
Punishment, control, and surveillance
will be ineffective at changing offenders’
behavior if human services are not also
provided.
• The risk principle – Offenders should
be assigned to interventions and ser-
vices based on level of risk of
recidivism, and that it is possible to clas-
sify offenders by risk of re-offending.
• The need principle – Interventions
should be selected based on the crimi-
nogenic needs (crime-related risk
factors) of the individual juvenile of-
fender, non-criminogenic needs are not
worth targeting.
• The responsivity principle – This prin-
ciple has two parts, based on
responding to offenders’ learning styles
and willingness to change:
o Interventions should take a be-
havioral approach, based on
cognitive and social learning theo-
ries, so that offenders can learn
new behaviors (“general respon-
sivity”)
o Interventions should be “fine-
tuned” to maximize response
from individual offenders, based
on their motivation, maturity,
learning style, anxiety level, etc.
(“specific responsivity”); and
• The fidelity principle – Programs
should be implemented with integrity
and fidelity to the original design, as re-
lates to staff training and retention,
duration and intensity of program con-
tact, and caseloads for staff.
Application of the principles of effective in-
tervention requires the use of an assessment tool
that provides the information needed to draw
conclusions about level of risk, criminogenic
needs, and the appropriateness of available dis-
What Works, Wisconsin Page 11
positions or diversion programs for an individ-
ual offender. Accurate assessment – above and
beyond the screening typically used to deter-
mine whether a juvenile should be detained or
referred for mental health services – is crucial to
effective intervention [32]. Several states have
developed standard assessments and require
their use at juvenile intake [33, 34]. Wisconsin’s
Juvenile Classification System [35] is currently
used to assess offenders placed in juvenile cor-
rectional facilities, but could be used more
widely with revision and re-validation.
SUMMARY OF SELECTED
COST-EFFECTIVE PROGRAMS
Table 1 describes prevention and juvenile
offender programs that have been demonstrated
to be among the most successful and cost-
effective programs in achieving their objectives,
either specifically in reducing delinquency and
criminal behavior, or for outcomes that place
children at significant risk of criminal behavior.
The programs featured are not the only ones
known to be effective, but were selected because
of their solid documentation of impact and, in
many cases, the availability of good cost-benefit
data. In addition, we felt that these programs
served as strong examples for the particular
category of program being highlighted.
Unlike the Washington State report, the em-
phasis in our report is on a select set of
programs we consider to be among the “best
bets” for delinquency prevention in Wisconsin
and in other states. Of course, new evidence on
emerging or established programs will further
inform policy decisions and complement the
evidence we present.
The selected programs and the evidence for
their effectiveness are described in the review of
programs and approaches that follows, in a
more detailed table in Appendix A, and in Ap-
pendix B, which also includes contact
information for individual programs. Other ef-
fective, evidence-based programs can be found
in the program registries that are listed in Ap-
pendix C.
An important emphasis of this report is the
economic benefits to society at large, as de-
scribed above. The main categories of benefits
for the programs we review are presented below
in order of their typical contribution to economic
benefits:
• Reduced costs for the administration
and treatment of individuals in the jus-
tice system as juveniles and adults. For
most studies, cost savings are projected
over adulthood based on the available
evidence on the program.
• Savings to crime victims including tan-
gible (e.g., hospitalization) and
intangible (e.g., pain and suffering) costs
associated with delinquency and crime.
• Increased earnings and compensation of
program participants in adulthood
based on actual earnings, employment,
or educational attainment at particular
ages. Estimates are for projected lifetime
earnings up to age 65.
• Increased tax revenues to state and fed-
eral governments based on estimated
earnings and compensation.
• Reduced costs for K-12 remedial ser-
vices including special education
placement and grade retention.
• Reduced costs for the administration
and treatment of children in the child
welfare system due to reported child
abuse and neglect. The tangible and in-
tangible savings to maltreatment victims
also are included.
• Reduced costs for the administration
and provision of social services includ-
ing welfare and food stamps.
• Reduced costs for substance abuse
treatment and related services.
We note that for some programs there are
unique benefit categories that are not listed
above. In center-based early childhood interven-
tions, for example, parents have increased time
available to devote to educational, economic, or
personal development. This is measured as a
What Works, Wisconsin Page 12
benefit to program participants. Savings associ-
ated with reduced rates of teenage parenthood
as well as health and mental health problems
also are applicable. Seemingly non-economic
outcomes such as achievement test scores and
conduct problem ratings can be converted to
economic benefits on the basis of their docu-
mented links to educational attainment and
juvenile crime [2].
TABLE 1.
Summary of Selected Effective Programs for Preventing Crime and Enhancing Well-Being
Per Participant Costs and Benefits in $2004
Program
Target group/
Focus
Intensity
and length
Major crime
prevention
impacts
Impacts linked to
crime prevention
Benefits
Costs
Net benefit
(Benefits –
Costs)
Return
per $1
invested
Primary Prevention
Preschool Education
Child-Parent
Centers
Child/
Enrichment,
Parent
involvement
Part day,
1-2 years
Arrests,
Incarceration
up to age 24
Ed attainment
Achievement
Abuse & neglect
78,732
7,755
70,977 10.15
High/Scope Perry
Preschool
Child/
Enrichment,
Home visits
Part day,
1-2 years
Arrests,
Incarceration
up to age 40
Achievement
Ed attainment
Income
145,414
16,648
283,995
16,648
128,766
267,347
8.74
(age 27)
17.07
(age 40)
Abecedarian
Project
Child/
Enrichment
Up to 8
years,
including
full day for
5 years
None reported Ed attainment
Achievement
Mother
employment
142,327
70,588
71,739 2.02
Family Support
Nurse-Family
Partnership –
High risk
Parent/
Home visits
2 hrs
biweekly,
2-2 ½ years
Arrests up to
age 15
Abuse & neglect
Substance use
Teen pregnancy
37,041
7,324
29,717 5.06
Strengthening
Families Program
10-14
Parent, child/
Groups
2 hrs
weekly,
7-14 weeks
Conduct
problems
Substance use
Peer pressure
6,833
874
5,959 7.82
What Works, Wisconsin Page 13
Per Participant Costs and Benefits in $2004
Program
Target group/
Focus
Intensity
and length
Major crime
prevention
impacts
Impacts linked to
crime prevention
Benefits
Costs
Net benefit
(Benefits –
Costs)
Return
per $1
invested
Social-Emotional Learning
Skills,
Opportunities,
and Recognition
School
population/
Parent and
teacher
training
Consistent
through
primary
grades
Violent
behavior
School behavior
Achievement
Alcohol misuse
14,810
4,712
10,100 3.14
Olweus Bullying
Prevention
Program
School
population/
Improving
school
environment
Consistent
through
school years
Vandalism,
fighting, theft,
and bullying
School behavior
— — —
Secondary Prevention
Family Training
Family Effective-
ness Training
Child,
Family/
Training and
therapy
1-1.5 hrs
weekly
13 weeks
Conduct
problems
Peer association
Family
functioning
— — —
Social Skills Training
Positive
Adolescent
Choices Training
Child/
Social skills
training
Unspecified Aggression
Violent
behavior
Arrests
None reported — — —
Mentoring
Big Brothers Big
Sisters
Youth/
Mentoring
2-4 times
monthly
12 months
Violent
behavior
School behavior
Achievement
4,166
4,117
49 1.01
Vocational/Job Training
Job Corps Youth/
Residential
training
Full day, 4-6
months
Arrests
Incarceration
Income
Ed attainment
Welfare use
22,883
15,804
7,079 1.45
Juvenile Offender Programs
Diversion
Adolescent Diver-
sion Project
Youth/
Case mgmt,
mentoring
6-8 hours
per week, 18
weeks
Reduced
recidivism
None reported 24,708
1,825
22,883 13.54
What Works, Wisconsin Page 14
Per Participant Costs and Benefits in $2004
Program
Target group/
Focus
Intensity
and length
Major crime
prevention
impacts
Impacts linked to
crime prevention
Benefits
Costs
Net benefit
(Benefits –
Costs)
Return
per $1
invested
Therapeutic Interventions
Multisystemic
Therapy
Youth/
Family
therapy
60 hours,
4 months
Re-arrests
Out-of-home
placement
Family function
Mental health
15,395
5,832
9,563 2.64
Functional Family
Therapy
Family/
Family
therapy
8-12 hours Re-offending
Sibling
delinquency
Family
communication
29,111
2,197
26,914 13.25
Multidimensional
Treatment Foster
Care*
Youth/ Foster
care with
therapeutic
treatment
Residential,
6-9 months
Re-arrests
Subsequent
days incarcer-
ated
Hard drug use 27,460*
2,524*
24,936* 10.88*
Case Management/Multimodal Interventions
Repeat Offender
Prevention
Youth/
Case mgmt;
center- and
home-based
services
Daily, 12-18
months
Sustained
petitions for
new offenses
Drug use
Completion of
court-ordered
obligations
— — —
Standard Services**
Juvenile Court Youth/
Formal
hearing,
adjudication,
& disposition
2-3 hearings;
length of
supervision
varies
N/A N/A Average
cost =
2,000
N/A N/A
Standard proba-
tion
Youth/
Monitoring &
supervision
Weekly
contact, min.
12 months
N/A N/A Annual
cost =
2,160
N/A N/A
Juvenile Correc-
tional Institution
Youth/
Confinement
& treatment
Residential;
length varies
N/A N/A Annual
cost =
68,255
N/A N/A
Note: The estimated economic benefits include actual and projected economic returns through adulthood. Estimates of both costs
and benefits across studies are based on different sets of assumptions and different lengths of follow up. Estimates from the
Washington State study are based on a different set of assumptions than those of the other reports.
* Costs and benefits for Multidimensional Treatment Foster Care (TFC) are given in comparison to regular group home treat-
ment. One year of TFC costs $2,524 more than group home care for the average participant, and yields $27,460 in benefits
per participant, as compared to group home residents. All other program costs and benefits are stated in comparison to “no
treatment.”
** Standard services are included in the table for the purpose of cost comparison.
What Works, Wisconsin Page 15
III. REVIEW OF PROGRAMS AND APPROACHES
We now summarize the evidence for pre-
vention and juvenile offender programs,
emphasizing findings from the programs de-
scribed in Table 1. We also summarize
principles that are consistent across the effective
programs in each category. As noted earlier,
these principles are derived from the available
evidence on particular programs, and have gen-
erally not been tested independently. However,
they are useful for the design and initial assess-
ment of new or untested programs and the
improvement of existing programs within these
categories.
In each category, we also briefly describe
emerging programs and programs currently in
wide use in Wisconsin. Emerging programs are
ones that appear promising but may not yet
have strong evidence of impact or cost-
effectiveness. Several of these programs seem
conceptually and operationally strong enough to
contribute to crime prevention in the future. We
are less certain of the effectiveness of others, but
include the available information in this review
because the approaches are
gaining in popularity in the
state and warrant more criti-
cal attention.
We address the pro-
grams and approaches
within ten categories in
three broad areas: Primary
Prevention, Secondary Prevention, and Juvenile
Offender Programs.
PRIMARY PREVENTION PROGRAMS
In the following section, we describe evi-
dence on programs for young people who are
relatively healthy and have not shown signs of
problem behavior leading to delinquency. How-
ever, they may be at risk of later delinquency
because they come from economically disadvan-
taged families or neighborhoods, have a parent
with a delinquency history, or possess other risk
factors. These programs generally begin early in
the life course and build general cognitive, aca-
demic, social, and emotional competencies.
Preschool Education
Preschool programs refer to education in the
early years of life before kindergarten. Although
they are implemented for children at all levels of
socio-economic status, the preschool programs
identified as most effective tend to serve eco-
nomically disadvantaged children. Center-based
preschool education programs are specifically
designed to promote children’s cognitive, psy-
chological, and social-emotional development.
Many preschool programs also provide family
services in a variety of forms ranging from home
visits and social service referrals to promoting
family-school partnerships. Family outcomes are
nevertheless conceptualized as secondary to
children’s. Because they follow the school calen-
dar, preschool programs have among the
highest dosages and greatest intensities of pre-
vention programs, ranging from 540 hours for a
part-day, one-year program to large multiples of
this amount for multi-year
and full-day programs that
provide comprehensive ser-
vices.
Scores of studies over
the years have found that
participation in preschool is
associated with positive
short- and longer-term school and social ad-
justment outcomes [36, 37]. Studies of three
programs reveal significant effects on many in-
dicators of well-being into adulthood, including
reduced crime [38-40]. These effects translate
into high economic returns. These three pre-
school programs are the High/Scope Perry
Preschool, the Chicago Child-Parent Centers,
and the Abecedarian Project. The Perry and
Child-Parent Center programs are part-day pro-
grams for 3- and 4-year-olds whereas
Abecedarian provides full-time educational day
care for five years. Evaluations of these pre-
Scores of studies over the years
have found that participation in
preschool is associated with
positive school and social
adjustment outcomes.
What Works, Wisconsin Page 16
school programs included well-matched com-
parison groups for assessing long-term
outcomes. As shown in Table 1, each program
has economic benefits that far exceed costs.
The Perry program has conducted follow-up
studies of its program participants when they
were 27 and 40 years old. The evaluations indi-
cate that the program group had fewer arrests
for violent, property, drug, and other crimes,
and spent fewer months incarcerated. They were
also less likely to repeat a grade, more likely to
graduate from high school, more likely to be
employed at ages 27 and 40, more likely to own
their homes, and less likely to use social ser-
vices. Based on the results up to age 40, we
estimate a return of $283,995 for the cost of
$16,648 per participant [38].
The Chicago Child-Parent Centers program
has followed a sample of 989 program partici-
pants and 550 comparison group members up to
age 24, and has found significantly lower rates
of juvenile arrest, adult convictions, and incar-
ceration among program participants. Program
participation also was associated with higher
educational attainment and school achievement
and with lower rates of child maltreatment and
remedial education [39].
The Abecedarian Project has shown a simi-
lar pattern of effects as the Perry and Chicago
programs, with the exception that no statistically
significant effects for crime prevention have
been detected, although the numerical differ-
ences favor the program group [40]. This finding
may be a function of the small sample size and
the low base rate of arrests in rural North Caro-
lina where the sample resides.
Although the costs of the programs are sig-
nificantly different from each other, the benefit-
cost ratios all exceed 2, ranging from $2.02 per
dollar invested for Abecedarian to $10.15 and
$17.07 per dollar invested for the Chicago and
Perry programs. The relatively low ratio for
Abecedarian is the result of the high cost per
participant (about $71,000 for five years). These
programs’ net economic benefits (benefits minus
costs) were the largest of all of the programs we
review in this report, ranging from about
$70,000 per participant for the Abecedarian pro-
ject and up to $267,000 per participant for the
Perry program. Note however, that the eco-
nomic returns for the Perry program may be a
function of the length of follow up, since the
other programs have just begun their long-term
adult follow-ups, and may prove to have similar
long-term benefits.
The primary sources of the economic bene-
fits of the programs were the increased earnings
of the participants (and the resulting tax reve-
nues) and public savings due to reduced crime,
averted crime victim costs, and reduced need for
rehabilitation and treatment. The crime-related
savings were the largest economic benefits by
far for the Perry Preschool and the Chicago
Child-Parent Centers program.
There appear to be five common elements of
effectiveness across these programs and in the
field at large. First, the preschool staff members
were well trained and earned competitive sala-
ries. Teachers for the Perry and Chicago
programs had at least bachelor’s degrees with
certification in early childhood, while the Abe-
cedarian teachers earned salaries that were
competitive with those of public school teachers,
which is unusual for such programs. Second,
comprehensive services were provided in the
form of center-based education for children and
support services for families either through
home visits, intensive school involvement, or
health and nutrition services. For example, the
Chicago program emphasized family-school
partnerships and resource mobilization. The
third common feature of the programs was the
relatively high dosage and intensity of services
as compared to other prevention programs.
Children in the Perry and Chicago programs
had close to 1,000 hours of participation, while
participation in the Abecedarian project ap-
proached 5,000 hours. Child to staff ratios were
small and ranged from about four to eight chil-
dren per staff member. In addition, each had a
curriculum philosophy that was well imple-
mented. Finally, each of the programs provided
What Works, Wisconsin Page 17
a literacy-rich learning environment with a wide
range of age-appropriate educational activities.
Although not subjected to cost-benefit
analysis, many other preschool programs have
demonstrated positive effects on early and later
school performance that may lead to significant
levels of cost-effectiveness. These include among
others, Head Start [41], the Syracuse Family
Development Research Program [42], projects
in the Cornell Consortium for Longitudinal
Studies on Child Abuse and Neglect [36], and
state-funded programs in Georgia, Illinois, and
Oklahoma [43].
Family Support Programs
One of the most important influences on the
lives of children and youth is the family setting
in which
they grow up.
A plethora of research
has documented the importance of the family in
the prevention of delinquency and other prob-
lems and the promotion of healthy, positive
development. Poor parenting skills and parent-
child relations often lead to a range of negative
outcomes for youth, including criminal and
other risk-taking behavior. Most of the evidence-
based primary prevention fam-
ily programs that have been
found to reduce delinquency
fall into one of two types: home
visitation and parent educa-
tion/training programs.
Home visitation programs
Home visitation programs are an increas-
ingly popular approach to family-focused
prevention. It is estimated that over half of a
million children in the United States are cur-
rently enrolled in some type of home visitation
program [44]. Most home visitation programs
are directed at first-time mothers when the
women are pregnant or when the child is very
young. The major goals are to improve preg-
nancy outcomes, promote child development,
reduce parental stress, prevent child maltreat-
ment, and increase effective parenting. These
programs are rarely focused directly on delin-
quency prevention and crime reduction per se,
but several long-term studies have found that
some home visitation programs have the poten-
tial to reduce such problematic outcomes while
more generally improving the lives of mothers
and their children [45, 46]. Home visitor pro-
grams vary along several dimensions, including
the target population(s), intended outcomes,
type of service provider, duration, intensity, and
the types of services available. However, they all
share three assumptions: 1) that parents play a
critical role in well-being and development of
their children; 2) that intervening and support-
ing parents as early as possible is the best
approach; and 3) that resources and education
should be brought directly to families rather
than expecting families to seek them out on their
own.
The diversity of home visiting programs has
made the synthesis of their effects rather diffi-
cult. Overall, the literature on home visiting
indicates that in some programs, at-risk families
with young children who receive home visita-
tion services fare better than control group
families on multiple outcomes. A recent meta-
analysis of home-visiting pro-
grams found that families in
home-visiting programs did not
have significantly less parental
stress or reports of child abuse
than other families [47]. In addi-
tion, mothers in these programs
did not differ from the control
group mothers in terms of self-
sufficiency, employment, or dependence on so-
cial services. However, positive effects did
emerge on some measures, such as parenting
attitudes and behavior and children’s cognitive
and social-emotional development.
Although the evidence for the effectiveness
of some home visiting programs has been
mixed, one program has emerged as the exem-
plary program of its kind by consistently
demonstrating an array of positive results in
rigorous evaluations. Evaluations of the Nurse-
Family Partnership Program (NFP) [45, 46] have
found that compared to controls, individuals
One of the most important
influences on the lives of
children and youth is the
family setting in which
they grow up.
What Works, Wisconsin Page 18
born to women in the program experienced
fewer incidences of running away from home in
adolescence. In addition, these adolescents had
fewer arrests, convictions, probation violations,
and sexual partners, and used less alcohol and
tobacco. Furthermore, this home visiting pro-
gram resulted in numerous positive outcomes
affecting the participating mothers and other
family members. These outcomes included sig-
nificant reductions in maternal substance abuse
during pregnancy, fewer incidences of child
maltreatment, smaller family size, fewer closely
spaced births, and less reliance on welfare [45,
46].
The costs of home visit-
ing programs can be fairly
high, especially if profes-
sional home visitors are
used. For example, the esti-
mated cost for the NFP
program is $7,324 per family;
at the lower end, the cost per
family for implementing
other home visitor programs
like Healthy Families America is nearly two-
thirds less. However, the impact and economic
benefits of the more expensive NFP program are
significantly greater and make the former pro-
gram a much more worthy investment. The net
benefit of the NFP program is $29,717, which
translates into a benefit of $5.06 for every dollar
invested. In contrast, a less effective program
like Healthy Families America yields a return
significantly less than one dollar for every dollar
invested [2].
Research has identified a number of charac-
teristics of the NFP program that have
contributed to its success and cost-effectiveness
[48]. First, home visits begin early, well before
the child is born, and last for about two years
after the child’s birth. In other words, support-
ing a mother before her child is born helps
ensure that she gets off to a good start during
the critical first few months of parenthood.
Equally important is that the program is of suffi-
cient duration and intensity, involving more
than two full years of regular home visits. Each
home visitor carries a caseload of no more than
25 families, allowing for appropriate intensity of
services. Another reason for NFP’s effectiveness
is that it is comprehensive, targeting parental
and environmental health, care-giving skills,
positive maternal development, and social sup-
port from family and friends. Home visitors also
assess the needs of individual families and pro-
vide referrals to appropriate services in the
community. One critical aspect of the program,
often overlooked by other home visiting ap-
proaches, involves the degree of supervision
provided to staff. Each home visitor is part of a
supervised team of 8 to 10 home visitors who
meet regularly to discuss
clients and receive ongoing
guidance from a supervisor.
A final critical aspect of NFP
is the level of professional
qualifications required of the
home visitors. This program
uses only trained nurses as
home visitors. Most other
home visiting programs
have used other types of professionals, parapro-
fessionals and volunteers as home visitors with
much less success.
These features of the NFP program help to
explain why it has been so successful in produc-
ing positive outcomes for mothers and their
children. Unfortunately, the remarkable success
of a few home visitor programs like NFP has led
to the rapid growth of other home visiting pro-
grams, many of which fail to take into account
the key ingredients that make such programs
effective. There has been a tendency to dilute
such programs by using less qualified home
visitors, not providing staff with regular training
and guidance, and significantly reducing the
number of home visits families receive. While
these tactics will notably reduce the costs of the
program, they do so at the expense of impact
and cost-effectiveness.
Parent education and training programs
Research clearly demonstrates that parents
and other adult caregivers are among the most
The remarkable success of some
home visitor programs has led
to the rapid growth of other
home visiting programs, many
of which fail to take into
account the key ingredients that
make such programs effective.
What Works, Wisconsin Page 19
important influences for promoting positive
child and adolescent development and reducing
problematic outcomes like drug abuse, violence
and delinquency. In fact, most scholars agree
that a positive family environment which in-
cludes parental supervision; positive, supportive
communication; clear rules and guidance; and
consistent discipline are “the major reasons
youth do not engage in delinquent or unhealthy
behaviors” [27, p. 457]. In response to the impor-
tant role that parents play in guiding and
protecting their children, numerous parent edu-
cation and training programs have emerged
over the past several decades. Unfortunately,
while there are thousands of parent educa-
tion/training programs being offered throughout
the United States and Wisconsin, very few of
them are proven, evidence-based programs. Na-
tional estimates indicate that only 10% of family
support practitioners are implementing evi-
dence-based programs [27].
Parent training programs are generally be-
havioral and skill-focused, and fall into two
formats: 1) training for parents alone and
2) training for both parents and children.
Programs that only include parents usually
involve skills training, directed at bringing
about cognitive, affective and behavior changes
in parents. Parents are also taught how to in-
crease their positive interactions with their
children, reward positive behavior and ignore
undesirable behavior, and improve communica-
tion and child compliance. A review of parent-
only programs found that this approach tends to
be most effective for families with younger chil-
dren (ages 3 to 10) and when the program also
includes additional sessions where parents are
given time to address their own issues [27].
Although meta-analyses of parent-only pro-
grams have indicated that some are quite
effective at preventing youth problem behav-
iors, programs that incorporate training for both
parents and children have been found to be
more effective in producing positive outcomes
for children [49]. Programs that also incorporate
a child-focused component tend to be most ef-
fective when they include opportunities for
children to learn how to manage anger and emo-
tions; accept and give constructive criticism or
praise; develop problem solving, decision-
making, peer resistance, and communication
skills; and develop friendships with prosocial
peers. In addition, some programs give parents
and children the opportunity to practice their
skills together, leading to the promotion of more
positive parent-child interaction and communi-
cation. Perhaps the most important reason why
parent-child family programs are more effective
than parent-only programs is that they address
more risk and protective factors. In addition to
enhancing parenting skills, they also promote
the child’s social competence and the facilitation
of positive family interactions and communica-
tion. Such programs have been found to work
best for elementary and middle school children.
Research on both parent-only and family-
focused interventions indicates that the most
effective programs share a number of important
features [50]. Such programs are comprehensive,
addressing a broad range of risk and protective
factors in the family environment. Programs are
also interactive, requiring family members to
interact with each other (and other group mem-
bers, when appropriate). Programs that use
didactic instruction are less effective in produc-
ing change in individuals. Research has
indicated the timing of programs to be critical;
first, the program implementation must occur
when family members are amenable to change.
In addition, programs are generally more effec-
tive when they intervene earlier in the child’s
life, especially before problems have occurred or
become severe. Programs beginning later carry
the burden of correcting years of dysfunctional
family interactions. Finally, the program’s dura-
tion should be sufficient for the types of
relational and behavioral changes that are
sought. For example, highly dysfunctional fami-
lies require a higher-intensity intervention for a
longer period of time. It should be noted that
most current evidence-based parenting pro-
grams involve a minimum of 5 sessions, with
most requiring 10 or more sessions. In contrast,
What Works, Wisconsin Page 20
many of the unproven parenting programs cur-
rently in use consist of fewer sessions, which
may in part explain their ineffectiveness.
One of the reasons that parenting programs
can be so cost-effective is that by helping parents
acquire new childrearing and relational skills, a
Wisconsin Ideas:
FAMILIES AND SCHOOLS TOGETHER (FAST)
One widely implemented prevention program, Families and Schools Together (FAST), was devel-
oped in Wisconsin and continues to attract the attention of prevention-oriented researchers and
practitioners. The FAST program, developed by Dr. Lynn McDonald of the Wisconsin Center for Edu-
cation Research, University of Wisconsin–Madison, is currently implemented in 45 states in the U.S.
and five countries internationally.1 It has been recognized as an evidence-based program by the Sub-
stance Abuse and Mental Health Services Administration (SAMHSA), the Strengthening America’s
Families project, and the Office of Juvenile Justice and Delinquency Prevention (OJJDP).
This program is built on the belief that parents are necessary prevention agents in their children’s
lives. The program targets children 4 to 12 years old and their families. Initially the program was de-
veloped for high-risk children and their families; it is currently also being implemented as a universal
program in schools in high-risk communities. Each program site serves 5 to 25 families at a time for 8 to
10 weeks of intensive group-based intervention. Families come together, often at their children’s school,
for family meals, parent-child play time, and parent support groups. Following graduation from the
intensive 8-week intervention, parents participate in FASTWORKS, comprised of two years of monthly
community planning sessions.2
The FAST program has shown positive results in four experimental evaluations. To date, no long-
term follow up studies have been conducted to determine FAST’s effects on juvenile delinquency, al-
though current data indicate that FAST can alter several risk and protective factors for delinquency.
Compared to control group children, FAST children at post-test showed lower levels of aggression and
anxiety, and higher levels of social skills and academic performance. In addition, parents in the FAST
program increased their school and community involvement.2, 3, 4
Costs for FAST implementation are estimated at $1,200 per family.4 Currently no cost-benefit analy-
sis of the FAST program is available, although a preliminary analysis is planned for Fall 2005.5
Recently an adaptation of FAST, called Baby FAST, has begun implementation at 10 sites in Wis-
consin. Baby FAST serves new parents, particularly teenage parents, with children 0 to 3 years old in 8
weekly group meetings. Program developers are currently evaluating aspects of the Baby FAST pro-
gram for effectiveness.6
1 FAST Program Description. Retrieved May 20, 2005, from http://www.wcer.wisc.edu/fast/
2 McDonald, L. FAST Fact Sheet. Retrieved May 9, 2005, from
http://www.wcer.wisc.edu/fast/how/FACTsheets/FACTSheet_18
3 Development Services Group for the Office of Juvenile Justice and Delinquency Prevention. (2004). Model programs guide
(Version 3.2). Retrieved March 18, 2005, from
http://www.dsgonline.com/Model_Programs_Guide/Web/mpg_index_flash.htm
4 Substance Abuse and Mental Health Services Administration (SAMHSA). SAMHSA Model Programs. Retrieved May 18,
2005, from http://modelprograms.samhsa.gov/
5 Trahan, M., FAST Program Manager, Wisconsin Center for Education Research, UW–Madison. Personal communication
with S. Cooney, May 19, 2005.
6 Baby FAST. Retrieved 2005, May 24, from http://www.wcer.wisc.edu/fast/how/Baby%5FFAST/
What Works, Wisconsin Page 21
new family environment is created that remains
in place long after the program is over. As a re-
sult, the intervention can have long-lasting
effects not only for a target child, but also for his
or her siblings. Such sibling spillover effects are
not usually taken into account in cost-benefit
estimates of such programs, which likely results
in conservative estimates of the programs’ true
benefits.
One of the most well-studied and effective
family-focused prevention programs is the
Strengthening Families Program for Parents
and Youth 10 to 14 (SFP 10-14). This universal,
primary prevention program is implemented
with families who have children between the
ages of 10 and 14. It provides opportunities for
parents and children to work separately on skill
building and then participate in activities to-
gether to practice their skills and address
common issues. The SFP 10-14 program runs for
seven sessions with the option of four additional
booster sessions. Evaluations of the program
have found that children who participated were
less likely to use alcohol and other drugs, were
less aggressive, had fewer conduct problems,
and resisted peer pressure better than youth in
the control group at a four-year follow-up [3,
51]. Program parents were found to show more
affection and set more appropriate limits for
their children. The relatively brief nature of the
program (7 sessions) along with its well-
documented and impressive impacts has made
this an increasingly popular program. In Wis-
consin, the University of Wisconsin-Extension,
Cooperative Extension, has made this a priority
program and has begun to implement it around
the state, often in cooperation with other local
agencies. The program is relatively inexpensive
to implement and has been found to have an
impressive benefit-cost ratio. The program has
been found to save $7.82 in avoided future costs
for every dollar invested.
Social-Emotional Learning Programs
The last century has seen increased pressure
on schools to prevent unhealthy and delinquent
behavior in children. Schools are now expected
to provide an education that prepares children
not just for future work, but also for fully re-
sponsible citizenship. Unfortunately, the
individual-level factors found in children with
behavior problems can influence others in the
school setting. Moreover, when a large number
of children with these characteristics attend the
same school, the result can be a school environ-
ment replete with aggression, bullying, gangs,
and disrespect for people and personal property.
Some researchers in the area of school-based
prevention have suggested Social-Emotional
Learning (SEL) programs as an effective ap-
proach for schools that want to prevent student
problem behaviors. SEL is defined as “the proc-
ess of developing the ability to recognize and
manage emotions, develop caring and concern
for others, make responsible decisions, establish
positive relationships, and handle challenging
situations effectively” [52]. Programs of this type
provide frameworks for the whole school sys-
tem, including parents and community
members, to fully promote the social and emo-
tional competencies of students. Proponents of
this category of programs contend there are five
skills that can be taught in schools for the maxi-
mum benefit of student development [52]:
(1) Self-awareness, defined as knowing and
understanding one’s own feelings,
thoughts, and abilities;
(2) Social awareness, defined as knowing
and understanding others’ thoughts and
feelings, and having the ability to inter-
act with diverse individuals;
(3) Self-management, defined as setting
goals and overcoming obstacles to
achievement, and handling personal
emotions in a positive, constructive
manner;
(4) Relationship skills, defined as beginning
and maintaining positive relationships,
resisting negative pressure from peers,
resolving conflict peacefully, and coop-
erating with others; and
What Works, Wisconsin Page 22
(5) Responsible decision-making, defined
as considering the advantages and dis-
advantages of actions and taking
responsibility for one’s own behavior.
Recent meta-analyses indicate that imple-
menting prevention programs in schools can
decrease school dropout and truancy, substance
abuse, conduct problems, delinquent behavior,
and drug use [53]. The Skills, Opportunities,
and Recognition (SOAR) program, formerly
known as the Seattle Social Development Pro-
ject, has been found to be an especially cost-
effective program for the prevention of youth
delinquency. This program, which targets chil-
dren in grades 1 through 6 in urban areas,
consists of two components: cooperative, devel-
opmentally-appropriate teaching practices in the
classroom and optional parental education
classes. The program is somewhat unique in that
it targets both family and school risk factors.
An evaluation of this program in Seattle
elementary schools found that at the beginning
of the 5th grade, program students, compared to
control group students, reported higher scores
on measures of family bonding, communication,
involvement, and proactive management. Stu-
dents participating in the program perceived
school to be more rewarding, and they were
more attached and more committed to school.
Program students also reported lower levels of
alcohol use and delinquency [54]. Another
evaluation found that six years after completing
the program, the program participants showed
positive outcomes on multiple measures com-
pared to control group individuals. Program
students reported better behavior in school,
greater attachment to school, and greater im-
provement in academic achievement. Program
participants were also less likely than controls to
report committing violent acts, drinking alcohol
heavily, engaging in sexual intercourse, and
having more than one sexual partner [55].
Cost-benefit analyses reveal that the benefits
of the SOAR program significantly outweigh the
costs. The program costs approximately $4,712
per participant, but the gains to society total
$14,810, a $3.14 return for every dollar invested.
Research has highlighted several character-
istics of successful SEL programs. First, no one
practice or program is likely to have a large, sus-
tainable effect on behavior [53]. Instead,
providing continuous, multi-faceted instruction
will produce the most beneficial effects. For ex-
ample, the success of the SOAR program can be
explained in part by the persistence of its im-
plementation in each school. When schools
implement the SOAR program, they do so con-
tinuously from first through sixth grade. In
addition, effective school-based programming is
developmentally appropriate. The implementa-
tion of SOAR varies at each grade level to target
skills most appropriate for the program stu-
dents. The optional parenting education
programs are also adjusted to the ages of the
program students. Because of the differing de-
velopmental needs and strengths of youth in
elementary, middle, and high schools, effective
SEL programs necessarily take different forms
for different ages [56]:
Elementary school. This time in a child’s life is
filled with changes in the cognitive and social
realms. Effective programs implemented during
this period help children develop their personal
and social competencies to achieve pro-social
goals. Developing the ability to accurately proc-
ess and positively respond to social information
is important at this stage. Children in elemen-
tary schools are ideal for programs that promote
non-violent values and develop conflict resolu-
tion skills. Because violent behavior is learned,
programs that start early can prevent violence
throughout the life span.
Middle school. Young adolescents face pre-
dictable changes in cognitive abilities, physical
maturation, and increased influence from their
social environment. Exploratory and non-
conforming behavior is normal at this stage.
However, the adoption of a healthy lifestyle and
the ability to make responsible decisions affects
present and future quality of life. Effective pro-
grams at this stage engage families, provide life
What Works, Wisconsin Page 23
skills training for the adoption of healthy behav-
iors, and provide productive extra-curricular
activities for students.
High school. Although programs target com-
petencies that continue from the middle school
years, here the focus is on students becoming
mature, responsible, productive citizens. Effec-
tive high school programs aim to develop the
skills needed for success in adult life including
those related to health and physical activity,
personal relationships, decision-making, creativ-
ity, employment, citizenship, and moral/ethical
values.
School-based primary prevention programs
other than SOAR have shown positive out-
comes. One such program is the Olweus
Bullying Prevention program, a universal pre-
vention program designed for school-wide
implementation. The program aims to reduce
and prevent bullying and victimization in
schools. An evaluation using a pre-test and post-
test design in 42 Norwegian schools found re-
ductions in bullying, victimization, vandalism,
fighting, theft, and truancy. Students also re-
ported more positive attitudes toward their
school after participating in the program [57].
Unfortunately, no cost-benefit analyses or long-
term follow-up studies have been conducted on
the Bullying Prevention program to date.
SECONDARY PREVENTION
PROGRAMS
This section describes programs for children
and youth who already exhibit some problem
behaviors within family or school settings but
have not been arrested and processed as juvenile
offenders. Secondary prevention programs aim
to remediate these problem behaviors before
they become more serious.
Family Training Programs
As noted above, a plethora of research has
documented the importance of the family in the
prevention of youth delinquency and the pro-
motion of healthy, positive development.
Family Effectiveness Training (FET) has
emerged as an effective, family-focused secon-
dary prevention program. FET targets Hispanic
families with children 6 to 12 years old in which
family conflict, often the result of both family
development and acculturation processes, is
occurring. The program posits that behavioral
disorders of the child are the result of maladap-
tive family processes. A therapist or facilitator
works with the family members in 13 weekly
sessions to strengthen current positive family
interactions and to provide treatment for nega-
tive patterns of interaction. An evaluation of this
program found that compared to a control
group, youth in the training had fewer conduct
and personality problems, demonstrated greater
maturity, and held more positive self-concepts.
Families improved their functioning relative to
controls. At a six-month follow-up assessment,
intervention families continued to fare better
than controls on most measures [58]. Unfortu-
nately, cost-benefit analyses have not been
conducted on this family-focused secondary
prevention program.
The Strengthening Families program de-
veloped at the University of Utah has been
recognized by several agencies as an evidence-
based family training program [59]. This secon-
dary prevention program is similar to the
SFP 10-14 primary prevention program, but it
requires a greater number of sessions. Evalua-
tions of this program have found significant
reductions in youth conduct disorders and posi-
tive changes in several aspects of family
functioning [60]. To date, this program has not
been subjected to a cost-benefit analysis.
Social Skills Training Programs
As noted above, schools are now a primary
delivery site for interventions related to youth
delinquency. The Positive Adolescent Choices
Training (PACT) program, a secondary preven-
tion program targeting African-American
students with histories of aggressive behavior
and/or victimization, has demonstrated positive
effects in evaluation research. The program
trains students on social skills, anger manage-
ment, and violence education through culturally
What Works, Wisconsin Page 24
appropriate videotaped vignettes and opportu-
nities for discussion and reinforcement. An
evaluation of this program indicates that in rela-
tion to comparison youth, program youth
showed less physical aggression in school and
had better social skills, fewer contacts with the
juvenile court system, fewer violence-related
charges, and fewer criminal offenses in general.
At a 2-year follow-up assessment, program
youth and comparison youth had both increased
their rates of criminal offending. However,
comparison youth had rates twice that of pro-
gram participants [61]. Unfortunately, cost-
benefit analyses have not been conducted on
this program.
Mentoring Programs
In recent years, mentoring programs have
received increased interest from policy-makers
and practitioners concerned with the decreased
availability of parental support and supervision
in many young people’s lives. Mentoring is usu-
ally defined as a relationship between an adult
and a young person in which the adult provides
guidance and support. Mentoring programs are
based on the assumption that children and ado-
lescents can benefit from a close relationship
with an adult other than a parent. Proponents of
these programs believe that mentoring can in-
fluence youth through several pathways [62].
First, social and emotional development is fos-
tered through close relationships characterized
by communication and trust. Such positive rela-
tionships can increase a young person’s self-
worth and hopes for the future. Second, cogni-
tive development is aided by interactions with
more knowledgeable, capable adults. These in-
teractions give youth the opportunity to think
and communicate critically about issues impor-
tant to them. Finally, the role modeling
provided by successful adults can decrease
youth’s beliefs that their opportunities in life are
very limited.
Mentoring programs vary along several di-
mensions. The goals of the programs vary such
that some are broad enough to cover all positive
youth developmental outcomes, while others
may focus on one specific area, such as improv-
ing academic achievement or preventing
substance abuse. Some programs consist primar-
ily of one-on-one interactions between mentors
and matched youth. Others are more compre-
hensive, offering tutoring, life-skills training, or
workshops for parents. Despite these variations,
mentoring programs share some commonalities.
Matches are often made by the gender, race,
geographic location, and shared interests of the
mentor and youth.
Evaluation data reveal that this approach,
when well implemented, can positively influ-
ence both criminal and non-criminal behavioral
outcomes in youth. For example, a recent sum-
mary evaluation of ten youth mentoring
programs found that overall, mentored youth do
better than other youth on measures of academic
performance, educational attainment, and pro-
social attitudes and
relationships.
In addition,
mentored youth sometimes do better than oth-
ers on measures of substance abuse and
delinquent behavior. According to this report, it
appears that at-risk youth in economically dis-
advantaged neighborhoods and/ or single
parent homes are the youth most likely to bene-
fit from positive mentoring relationships [63].
The Big Brothers Big Sisters of America
(BBBSA) program is the most frequently im-
plemented mentoring program in the United
States. Currently, BBBSA serves youth through
more than 500 organizations across the United
States. The program matches children and ado-
lescents from single-parent families with a
volunteer adult mentor to provide support in
their lives. Mentors and matched youth meet
two to four times a month for at least a year. An
extensive evaluation of the program yielded
several notable findings. Program youth, com-
pared to those in the control group, were 46
percent less likely to begin using drugs, 27 per-
cent less likely to begin using alcohol and one-
third less likely to hit someone. In addition,
youth who were mentored in the program
missed half the number of school days, showed
greater academic improvement, and had better
relationships with their family and their friends
What Works, Wisconsin Page 25
when compared to youth in the control group
[64].
Current cost-benefit analyses indicate that
BBBSA is not substantially cost-effective accord-
ing to the outcomes measured. At a cost of
$4,117 per participant, the bene-
fits only amount to $4,166, a net
gain of $49 per youth. This trans-
lates into $1.01 gained for every
dollar invested in the program.
Evaluation research has in-
dicated several characteristics of
programs that foster successful mentoring rela-
tionships. These principles can help guide
decision-making for both practitioners and pol-
icy-makers. Most importantly, effective
mentoring programs take steps to encourage
long-lasting relationships. For example, BBBSA
requires participants to agree to maintain their
mentoring relationship for at least one year. Re-
search has indicated that longer mentoring
relationships produce better outcomes for youth
[65]. Mentoring matches lasting longer than a
year produce the most positive benefits for
youth, including improvements in academic,
behavioral, and psychological domains. Matches
that last at least six months but less than one
year produce some positive effects, but not to
the extent that longer matches do. In addition,
matches lasting less than three months produce
negative effects, including a decline in the self-
worth and scholastic competence of youth par-
ticipants. Research has also indicated that
matches lasting less than six months can lead to
increases in alcohol use of youth participants.
For these reasons, program implementers must
be especially careful to make good matches and
to encourage the continuation of the relation-
ship.
Effective programs provide ample training
and support to mentors before and during the
mentoring relationship. In several studies, those
mentors who received the most training had the
longest lasting relationship with their youth par-
ticipants [63]. In addition, effective programs
take steps to facilitate interaction between youth
and their adult mentors. Programs may organize
activities for matches and thus decrease the
chance that the relationship will fail out of lack
of interest. Research has also shown that men-
toring relationships are less likely to fail if the
pair engages in activities that reflect the interests
of the youth rather than those of
their mentors [63]. Finally, the
most effective programs have a
history of internally monitoring
their program and improving
their practices based on evalua-
tion data. BBBSA is the most
successful program model of its kind; its success
can at least partially be attributed to its history
of evaluating program implementation and out-
comes.
Vocational/Job Training Programs
Vocational and job training programs are
built on the premise that increasing education
and employment opportunities will combat ju-
venile delinquency and adult criminal behavior.
This strategy is implemented in an attempt both
to prevent youth from first committing delin-
quent acts and to discourage future criminal
behavior in youth already in contact with the
system. Evaluations of vocational training pro-
grams have produced mixed results. Some
programs have produced negligible or negative
impacts on employment, delinquency, and other
outcomes, essentially producing costs greater
than benefits [20, 66]. Others, such as the Job
Corps, have demonstrated more positive results.
According to the U.S. Department of Labor,
Education and Training, over 60,000 youth ages
16 to 24 participate in a Job Corps program
every year. Only one long-term evaluation of
participant outcomes, the National Job Corps
Study, has been conducted for the Job Corps
program [67]. Evaluation results indicated that
those participating in Job Corps had signifi-
cantly fewer arrests and convictions, and less
time incarcerated, than those in the control
group. Job Corps participants spent more time
in academic classes than did the control group,
and overall had larger increases in their educa-
Effective mentoring
programs take steps to
encourage long-lasting
relationships.
What Works, Wisconsin Page 26
tion and skill levels. The evaluation also re-
vealed that the Job Corps participants earned
more money than the control group individuals
after completing the program. Cost-benefit
analyses indicate that the benefits of the Job
Corps program outweigh its costs. At a cost of
approximately $15,804 per participant, program
participation yields about $22,883 in benefits to
society, a net gain of $7,079 per youth. This
translates to a benefit of $1.45 for each dollar
invested in the program.
Emerging and Unproven Programs
Two increasingly widespread approaches
that many practitioners and policymakers be-
lieve have promise for preventing delinquency
are after-school programs and alternative
schools. Below we examine existing evidence for
the effectiveness of each of these strategies.
After-school programs
After-school programs have been widely
heralded for the prevention of juvenile delin-
quency. Recent research has repeatedly found
evidence that juvenile crime
peaks during the after-school
hours, from approximately
2 pm to 6 pm [68]. Most hy-
pothesize that this peak
occurs due to a lack of adult
supervision. Thus, many
communities have consid-
ered implementing after-
school programs as a strategy to reduce delin-
quent behavior. Such programs are offered prior
to and following regular school sessions and
typically include services such as tutoring, vol-
unteer opportunities, second language
instruction, computer instruction, athletic
events, and recreational activities.
A recent meta-analysis of after-school pro-
grams indicates that these programs can reduce
delinquency for middle-school but not elemen-
tary-school students [68]. In addition, after-
school programs were found to increase youths’
intentions not to use drugs. Participation in af-
ter-school programs was somewhat related to
increases in social skills and associations with
positive peers, all of which were related to re-
ductions in delinquency. Those programs
emphasizing social skills had the largest effects
on reducing problem behavior. The elementary
school programs in this meta-analysis generally
did not focus on social skills, one possible ex-
planation for their lack of effectiveness.
Researchers have pointed out that one of the
problems with after-school programs as a pre-
vention strategy is that the most at-risk youth
are unlikely to join and regularly attend after-
school programs [i.e., 68]. Thus, one cannot as-
sume that after-school programs will reduce
delinquency in those youth most at-risk. At this
point in time, no cost-benefit analyses have been
conducted on after-school programs.
Alternative schools
Of the 127 charter schools that the Wiscon-
sin Department of Public Instruction lists on its
website, almost half are described as serving
students who are truant, aggressive, disruptive,
at-risk for school failure, or who have been ex-
pelled from regular public
schools [69]. Alternative
schools are often viewed as a
viable solution for problem
or delinquent youth who do
not succeed in the typical
school setting. Unfortu-
nately, the empirical
research documenting the effectiveness of alter-
native schools is seriously deficient. Several
evaluations have found alternative schools to be
ineffective, while other evaluations have found a
few, small, positive effects. For example, one
evaluation of three alternative schools found
that in-school problem behaviors decreased for
alternative school students relative to peer coun-
terparts, but this effect did not hold for
delinquent behavior in general [70]. The most
substantial evidence to date comes from a meta-
analysis of 57 alternative school evaluations. It
found that alternative schools had a small, posi-
tive effect on school performance, attitudes
One of the problems with after-
school programs as a prevention
strategy is that at-risk youth are
unlikely to join and regularly
attend after-school programs.
What Works, Wisconsin Page 27
toward school, and self-esteem but had no sig-
nificant effect on delinquency [71].
JUVENILE OFFENDER PROGRAMS
In this section, we consider non-residential
programs and approaches that are delivered to
juvenile offenders in the community, whether
they are formal dispositions given by a juvenile
court, or diversion programs that keep first-time
offenders out of the courts. Residential pro-
grams are not considered here except to the
extent that they often serve as the “control
group” for comparing rates of recidivism and
cost-effectiveness of community-based pro-
grams.
Two frameworks are rele-
vant for understanding current
thinking about intervention
with juvenile offenders. First,
the principles of effective inter-
vention help us to understand
on a broad level what kinds of
approaches will work and what
will not. These principles were described in
more detail in Section II. The principles include:
• the human service principle;
• the risk principle;
• the need principle;
• the responsivity principle, which has
two parts: “general responsivity” and
“specific responsivity”; and
• the fidelity principle.
Adherence to the principles of effective in-
tervention is best achieved with the use of a risk
and need assessment tool administered to juve-
nile offenders when they come into contact with
law enforcement or the juvenile court. Sound
assessment should inform all decisions about
court processing and dispositions so that each
offender receives the level of supervision and
services that are most likely to be effective.
Second, the balanced and restorative justice
(BARJ) model, endorsed by Wisconsin’s juvenile
justice legislation, provides the philosophical
grounding for many current approaches to ju-
venile offender programming. BARJ emphasizes
that responses to delinquency must ensure pub-
lic safety and offender accountability, while at
the same time supporting offenders to develop
the competencies that will help them become
productive, law-abiding adults. All of the pro-
grams and approaches described in the review
below can be employed in a BARJ approach.
Juvenile Offender Program Categories and
Considerations
Through meta-analyses and comparisons of
different program types, researchers have found
that programs implemented in the community
are
significantly more likely to
reduce recidivism than are
programs implemented in resi-
dential settings [72, 73]. Within
the realm of community-based
interventions, though, there is a
wide range of effectiveness
among particular programs
and approaches. Some interventions are more
likely than others to have the desired effect on
juvenile offenders, while others actually appear
to encourage recidivism.
Evidence for the effectiveness of various re-
sponses to juvenile delinquency is far weaker
than evidence for the effectiveness of prevention
programs. The fields of primary and secondary
prevention are increasingly characterized by
evidence-based programs that are available for
purchase and use. In the area of juvenile of-
fender programs, on the other hand, there are
very few of these “packaged” programs that are
disseminated for wider use. Individual states or
jurisdictions are more likely to develop their
own programs based on approaches that appear
promising. This results in programs that may go
by the same name and incorporate some com-
mon elements, but which cannot be assumed to
have the same effect on juvenile offenders.
Moreover, very few of these programs have
been rigorously evaluated, making it difficult to
know if they are truly effective.
Programs implemented in
the community are
significantly more likely to
reduce recidivism than are
residential programs.
What Works, Wisconsin Page 28
In recent years, there has been movement
within the juvenile justice field toward evidence-
based programs. However, currently available
evidence speaks more to general approaches
than to specific programs. The quality of that
evidence leads us to conclude that some of these
approaches hold promise for reducing recidi-
vism among juvenile offenders, but limits our
ability to say with confidence that certain ap-
proaches “work.” In the following review, we
present general approaches as well as specific
programs within several categories. We also dis-
cuss several approaches that hold promise, or
are emerging as popular, community-based re-
sponses to delinquency but have not been
empirically proven to be effective.
Note that the categories of programs and
approaches used in this report are not mutually
exclusive. In fact, there is significant overlap
between the categories. Many of the programs
and approaches described are sometimes used
as diversion from the juvenile court; all of them
can be used as elements in a balanced and re-
storative justice model; and case management
and multimodal programs employ many types
of interventions which may be included in other
categories here.
Diversion/Community Accountability
Programs
Diversion describes any response to juvenile
delinquency in which adjudication is postponed
while the offender completes a rehabilitative
program.2 Diversion programs are also referred
to as community accountability programs. Due
to the costs of juvenile court processing (esti-
mated at about $2,000 per offender [37]) and
other advantages of keeping low-risk offenders
2 Community-based treatments are often referred to
as “placement diversion” programs, because the
young offender is diverted from residential place-
ment. Diversion is also sometimes used to refer to
release to parents with a warning but no services or
restrictions. In this report, “diversion” refers to diver-
sion from formal adjudication into a structured
program that provides services and/or makes de-
mands of offenders.
out of the juvenile court system, diversion pro-
grams hold promise as a cost-effective response
to delinquency. One such program, the Adoles-
cent Diversion Project, has been shown to be
highly effective. Diversion programs vary
widely, and not all are as effective as ADP, but
this is an area that shows promise for cost sav-
ings, reduced caseloads in the juvenile justice
system, and reduced recidivism among offend-
ers.
Diversion or community accountability pro-
grams are designed to ensure accountability
among low-risk offenders while reducing
caseloads and expenses for the juvenile justice
system. This allows resources to be used on of-
fenders who pose more of a risk and require
more supervision and services. However, crimi-
nologists and juvenile justice professionals
express several legitimate concerns about diver-
sion programs [13, 74]. First, these programs can
result in “net widening,” bringing youth into
unwarranted contact with the justice system,
and resulting in undue punishment as well as
inefficient use of resources. This is related to a
concern about the constitutionality of sanctions
being determined outside the confines of the
juvenile court and without due process. Another
concern is differential treatment; statistics show
that white youth are more likely to be diverted
from adjudication than are minority youth who
are arrested for similar offenses [75]. Finally, the
goals of offender accountability and public
safety may not be accomplished if diversion
programs are weak or ineffective, and offenders
do not benefit from the experiences.
To address these concerns and ensure that
diversion programs are used fairly and effi-
ciently,
• clear guidelines should be in place for
determining whether offenders warrant
inclusion in a diversion program, re-
lease without diversion, or formal
processing through the juvenile court
[e.g., 76], ideally attached to the use of a
standard, validated, risk and needs as-
sessment tool [e.g., 33, 34];
What Works, Wisconsin Page 29
• offenders should be informed of their
right to refuse participation in a diver-
sion program if they contest the charges
against them; and
• consequences for lack of compliance
with the demands of a diversion pro-
gram should be serious, and should be
enforced quickly.
When compared to regular juvenile court
processing, diversion programs cost signifi-
cantly less, and many have achieved outcomes
better than or similar to those of juvenile court.
The Washington State study concluded that di-
version programs in general, and ADP in
particular, were cost-beneficial alternatives to
regular juvenile court processing.
The Adolescent Diversion Project (ADP),
also known as the Michigan State Diversion Pro-
ject, is one of the few evidence-based programs
in the diversion category. ADP pairs diverted
juvenile offenders with college student men-
tors/caseworkers. The students, mainly juniors
and seniors, receive training in behavioral inter-
vention and advocacy through a semester-long
course. Students and offenders then work one-
on-one, for 6–8 hours per week, for 18 weeks.
Several experimental evaluations of this model
revealed that the young offenders responded
well to one-on-one work with college students,
whether they used behavioral contracting, youth
advocacy/case management, or therapeutic ap-
proaches. Recidivism was lower among ADP
participants than among both offenders who
were formally processed through the juvenile
court and offenders who were released to their
parents [74]. Due to its reliance on college stu-
dents, ADP is a relatively low-cost program to
implement, at $1,825 per participant. With bene-
fits of $24,708 per participant based on reduced
recidivism, ADP is a very cost-effective pro-
gram. We estimate a return of $13.54 per dollar
invested in ADP.
Emerging and Unproven Diversion Programs
Specialized courts separate from the juvenile
court are another form of diversion programs
that have grown rapidly in recent years. Efforts
are underway to systematically evaluate the ef-
fectiveness of both Teen Courts and Juvenile
Drug Courts, but at this point in time, the evi-
dence is still inconclusive as to whether these
approaches are effective.
Teen Courts are an increasingly popular
approach to dealing with status offenses and
non-violent juvenile crime. The National Youth
Court Center lists 37 Teen Courts in Wisconsin
on its website [77], associated with counties,
municipalities, and schools around the state.
One explanation for the rapid rise of Teen
Courts is that they are relatively inexpensive to
operate, because they rely on youth and adult
volunteers (including offenders sentenced to
community service) to fulfill the roles of attor-
neys, judges, and jury members. Teen Courts
also present an opportunity for interested non-
delinquent youth to learn about the justice sys-
tem and make a meaningful contribution to their
community [78]. However, many argue that
Teen Courts are essentially net-widening pro-
grams. Initial evaluations of Teen Courts have
found that they are most effective with low-risk
offenders, while higher-risk offenders are less
likely to comply with their sentences, and show
no reduction in recidivism compared to high-
risk offenders participating in other justice pro-
grams [79-81].
Juvenile Drug Courts provide increased
supervision and accountability for substance-
using offenders, who report back to the Drug
Court at regular intervals throughout their re-
covery. Drug Courts also link offenders and
their families to treatment services, and sanction
lapses in offenders’ recovery. Proponents claim
that Drug Courts help to reduce the caseload of
the regular courts and reduce subsequent sub-
stance abuse and re-arrest. However, because
Drug Courts are a relatively new phenomenon,
it is not yet possible to draw conclusions about
their effectiveness for reducing recidivism and
substance abuse among juvenile drug offenders
[82].
What Works, Wisconsin Page 30
Therapeutic Interventions
There is significant evidence that providing
juvenile offenders with proven, well-
implemented therapeutic treatment reduces the
likelihood that they will re-offend. In addition to
several comprehensive, evidence-based treat-
ment programs, there is evidence from multiple
sources that many types of interventions are
more effective when they incorporate a thera-
peutic component, particularly cognitive-
behavioral therapy.
The evidence-based programs in this cate-
gory involve therapeutic treatment for offenders
and their families, based on the assumption that
the roots of delinquent behavior are found not
only within young offenders
themselves, but also in their
families and in how they relate
to their peer groups, schools,
and communities. These pro-
grams are characterized by
regular meetings of family
members with a therapist.
Multisystemic Therapy
(MST) has shown success in reducing recidi-
vism among participants, as compared to
recipients of usual community-based mental
health services [83] and compared to offenders
who received individual therapy [84]. MST
therapists provide in-home treatment to families
of juvenile delinquents, with a focus on improv-
ing parenting effectiveness by empowering
parents and teaching them new parenting skills.
Average treatment includes about 60 hours of
contact over a 4-month period, with therapists
available 24 hours a day, seven days a week.
Caseloads range from four to six families at a
time. MST costs per family are estimated at
$5,832. The Washington State report highlighted
MST as a consistently cost-effective intervention
for juvenile delinquency, with an estimated av-
erage return of $15,395 per juvenile offender, or
$2.64 for every dollar spent on the program.
Functional Family Therapy is a flexible ap-
proach, grounded in clinical theory, which is
used to treat a variety of complex problems in
the families of high-risk and delinquent youth.
Families receive an average of 8–12 hours of
therapy. Several rigorous evaluations indicate
that FFT can reduce recidivism among juvenile
offenders by 20 to 60 percent, and is highly ef-
fective compared to other treatments. The
program also reduces future offending by sib-
lings of the target youth [85]. Per-family costs
are estimated at $2,197. The Washington State
Institute for Social Policy reports a return of
over $13 per dollar invested in FFT programs,
representing $29,111 in benefits per delinquent
youth.
Multidimensional Treatment Foster Care
(TFC) is an alternative to group home place-
ment, in which a troubled or
delinquent youth receives
treatment through trained fos-
ter parents and continues to
attend school in his or her home
community. Case managers
supervise the foster parents,
coordinate needed interven-
tions at school, and develop an
individualized behavior man-
agement program to be implemented by the fos-
ter parents. The youth’s family of origin is also
trained in behavior management techniques to
promote better supervision and continuing in-
tervention when the juvenile returns home.
Evaluations of TFC have found it to be more
effective than group home placement for reduc-
ing subsequent offending and substance abuse,
with only slightly higher costs [86]. The added
cost of roughly $2,500 per youth results in bene-
fits of over $27,000, or $10.88 for every dollar
invested, making TFC a cost-beneficial ap-
proach.
Finally, although it is not embodied in one
particular program, cognitive-behavioral (CB)
therapy has emerged as an evidence-based prac-
tice for responding to juvenile delinquency. CB
theory is grounded in the principle that deviant,
impulsive behavior stems from deviant, impul-
sive thinking. CB therapies target these
unproductive thought processes and teach of-
fenders new skills and behavioral strategies.
Providing juvenile
offenders with proven,
well-implemented
therapeutic treatment
reduces the likelihood that
they will re-offend.
What Works, Wisconsin Page 31
Andrews and colleagues found that cognitive-
behavioral approaches were among the most
effective of all treatment types, especially when
the intervention was appropriate for the of-
fender (as defined by the principles of effective
intervention) [29]. Lipsey and colleagues also
found CB therapy to be effective in another
meta-analysis [87]. CB therapy is often delivered
to offenders in small groups, which allows for
role-playing, such as practicing skills for resist-
ing negative peer pressure. However, as with
other types of programs, bringing together
groups of delinquent youth can also have nega-
tive consequences. Based on his review of the
research, LaTessa recommends treatment
groups of no more than eight offenders and fre-
quent meetings, at least twice a week through
the treatment period [88].
A number of studies support the use of CB
therapeutic treatment in the context of other in-
terventions. Lipsey conducted a meta-analysis of
hundreds of delinquency interventions, and
concluded that interventions incorporating prac-
tical, skills-based, behavioral or cognitive-
behavioral treatment components were more
effective at reducing recidivism than others, re-
gardless of the type of intervention overall [72].
Other evaluations have found that program ef-
fectiveness did not improve when insight-
oriented counseling, psycho-dynamic counsel-
ing, or other therapeutic approaches not
grounded in cognitive and behavioral theories
were offered to participants [89].
Case Management/Multimodal Interventions
The last decade has seen a movement to-
wards integrating or coordinating services for
children and youth, particularly those with seri-
ous emotional disturbances, who are involved in
the juvenile justice and child welfare systems.
This movement has generated several ap-
proaches for connecting children and families to
existing resources and coordinating the services
they receive from various programs and agen-
cies. As these approaches are used in the field of
juvenile justice, they incorporate many of the
program types already discussed in this report.
Mounting evidence suggests that these case
management or multimodal interventions can
be effective in managing offenders in the com-
munity and reducing re-offense.
Interventions in this area are built around
strong assessments of juveniles at intake and/or
at disposition, to determine their needs and the
needs of their families, and to ensure that they
receive the appropriate services. These pro-
grams are most effective when funding streams
are blended to allow care coordinators freedom
in connecting families to the services they need.
When services are coordinated and even co-
located, youth and families can access the re-
sources they need to resolve or manage issues in
their lives and get young offenders back on
track toward successful adulthood [13].
California’s Repeat Offender Prevention
Program (ROPP) is the only evidence-based
program in this category. ROPP is a case as-
sessment and management program for juvenile
offenders who are assessed to be at high risk of
recidivism. It is one component of “The 8% Solu-
tion,” a graduated sanctions model that was
developed after research in Orange County, CA,
revealed that about 8% of offenders were re-
sponsible for over half of all referrals to the
juvenile courts. High-risk offenders report to a
Youth and Family Resource Center in their
community, where they receive intensive case
management, sometimes referred to as “an indi-
vidualized intensive supervision program” [3]
that integrates services from a variety of youth-
and family-serving agencies. This includes co-
ordination of educational and rehabilitative
services, facilitation of community service and
restitution completion, drug testing, in-home
family support, and supervision of the offender.
A 2002 report to the state legislature concluded
that ROPP participants showed significant im-
provements in school attendance and academic
achievement, were more likely to complete resti-
tution and community service obligations,
tested positive for drugs less often, and were
less likely to recidivate for new offenses than
were control group members who received stan-
dard probation services [90]. No cost-benefit
What Works, Wisconsin Page 32
analysis has been conducted for ROPP to deter-
mine whether its benefits justify the additional
cost of the program when compared to tradi-
tional probation.
Emerging and Unproven Programs and
Approaches
Enhanced Probation and Supervision Programs
Probation has often been called the corner-
stone of the juvenile justice system. According to
the Office of Juvenile Justice and Delinquency
Prevention, roughly 60% of adjudicated juve-
niles are put on probation [91].3 Concerns about
the effectiveness of probation for reducing de-
linquent activity both during and after the
probation period led to the development of en-
hanced probation services, either delivered
through the juvenile justice system or through
social service agencies. The defining characteris-
tic of these programs is that they increase the
amount of contact between offenders and proba-
tion officers or caseworkers. Many enhanced
probation programs also intensify the surveil-
lance of young offenders, for example with the
use of electronic monitoring, and provide ser-
vices such as therapeutic intervention or
3 These statistics do not distinguish between proba-
tion alone and enhanced probation and supervision
programs.
Wisconsin Ideas:
THE WRAPAROUND APPROACH
Wisconsin is a leader in developing coordinated service teams, integrated services projects, and
wraparound programs for families with involvement in multiple systems, such as child welfare, juve-
nile justice, and mental health services.1 Wraparound Milwaukee has attracted national attention for
the favorable results it demonstrated in a non-experimental evaluation. Milwaukee has also developed
the FOCUS Program, which combines short stays in a child caring institution (which provides out-of-
home care and supervision) with longer-term wraparound services for delinquent boys.
Wraparound programs provide treatment and service coordination for delinquent and non-
delinquent youth with mental health disorders, with the goal of keeping youth in the community and
with their families when possible. Using blended funding from the juvenile justice and child welfare
systems, Wraparound Milwaukee allows families to select from among an array of services and provid-
ers, and provides “care coordination” to ensure the best use of resources.2 Wraparound programs,
coordinated service teams, and integrated services projects accomplish similar goals in other communi-
ties around the state.
The FOCUS Program, a Milwaukee county pilot project that began in 2003, aims to reduce dispro-
portionate minority confinement by diverting delinquent boys with mental health diagnoses from
correctional placements, instead placing them in a child caring institution for 4–5 months and provid-
ing Wraparound Milwaukee services for them and their families afterwards.3
Neither Wraparound Milwaukee nor the FOCUS program has been subjected to evaluations rigor-
ous enough for a cost-benefit analysis to be completed. In 2000, Wraparound Milwaukee reported that
650 youth were served in the community (at a monthly cost of about $3,300 per participant) with the
same funds that otherwise would have served only 350 institutionalized youth (at a monthly cost of
about $5,000 per bed).2
1. Wisconsinʹs Collaborative Systems of Care (WCSOC). WCSOCʹs Resource Website. Retrieved May 12, 2005, from
http://www.wicollaborative.org/
2. Kamradt, B. (2000). Wraparound Milwaukee: Aiding youth with mental health needs. Juvenile Justice, 7(1), 14-23.
3. Milwaukee Countyʹs DMC Project. (2005). Goals and Activities: FOCUS Program – An alternative to correctional place-
ment. Waukesha, WI: Presented at the Juvenile Law Seminar, April 14, 2005.
What Works, Wisconsin Page 33
treatment for alcohol and other drug abuse is-
sues.
Intensive supervision is widely used in Wis-
consin. The Division of Juvenile Corrections
estimates that roughly one-half of Wisconsinʹs
counties are using intensive supervision [92].
According to state statutes, caseworkers in
intensive supervision and probation (ISP) pro-
grams must have at least one face-to-face contact
each day with each juvenile offender in their
caseload, not to exceed 10 juveniles at any given
time [93]. Programs may use electronic monitor-
ing, and juveniles may be taken into custody in
response to a violation of probation, at the
caseworker’s discretion.
Evaluations of ISP programs have shown
them to be roughly as effective as other standard
approaches. An evaluation of an ISP program in
Mississippi showed that participants were
slightly less likely than standard probationers to
recidivate in the year following program com-
pletion [94]. Another study in Michigan found
that ISP probationers were no more likely to re-
cidivate than their counterparts who were
placed in state residential facilities [95]. These
two studies drew different conclusions about
the cost-effectiveness of ISP because of their dif-
ferent control groups. ISP was deemed to be a
cost-effective alternative to commitment to the
state in Michigan, because it achieved the same
results at less than one-third of the cost. How-
ever, ISP was not found to be
cost-effective in the Mississippi
study because its small effect
did not justify the increased
costs when compared to stan-
dard probation. One meta-
analytic review found no effect
of ISP on recidivism, as com-
pared to regular probation [96].
Giving greater weight to studies with larger
sample sizes, the researchers actually concluded
that ISP increased the likelihood of recidivism.
However, the authors did conclude that inten-
sive supervision programs that also offer
offenders therapeutic treatment are more likely
to be effective than those that simply increase
supervision [96].
Because there have not been any rigorous
evaluations of Wisconsin’s ISP model, we cannot
be certain whether these programs are likely to
be effective at reducing recidivism or achieving
other outcomes. However, based on evaluations
of ISP programs in other locations, Wisconsin’s
guidelines for caseload size and frequency of
contact appear to describe an effective approach
to reducing recidivism, particularly if treatment
and services are made available to offenders
beyond what is required by the statutes.
Day treatment centers are another emerging
approach that increase supervision and provide
treatment options for offenders with substance
abuse issues. Day treatment appears to be a rela-
tively popular intervention in Wisconsin,
particularly for young offenders. No control-
group studies have been completed, but non-
experimental studies of day treatment centers
have shown reduced recidivism among offend-
ers who reported to day treatment [3, 97].
Restorative Justice Programs
While a wide variety of programs and ap-
proaches can be incorporated into the balanced
and restorative justice (BARJ) model, “restora-
tive justice” also describes specific approaches
that focus on repairing the harm done by juve-
nile offenders. Restorative justice involves
offenders in restoring damaged
property to its original condi-
tion, repaying victims for their
financial losses, providing ser-
vices to victims or to the
community at large to make up
for damage done, and apolo-
gizing to victims.
Very few experimental evaluations have
been carried out on restorative justice programs
in this country. Most of the existing evidence
relates to victim satisfaction with the process,
rates of reaching an agreement, and rates of sen-
tence completion and agreement fulfillment.
Victim-offender mediation is the only approach
Community service and
restitution may contribute
to reductions in recidivism
when used within a
restorative framework.
What Works, Wisconsin Page 34
that has been extensively evaluated, and even
then, the evidence is encouraging but not con-
clusive [98, 99]. We cannot say for sure whether
such approaches are effective at reducing recidi-
vism, nor can we come to any conclusions about
the relative costs and benefits of these programs
with the available information.
It appears that community service and res-
titution may contribute to reductions in
recidivism when used within a restorative
framework. Several studies found that when
these approaches were used just as “punish-
ment,” they were not effective at reducing
recidivism. However, when they were used to
repair the harm of the crime and develop job-
related skills or lasting social connections, they
were more effective [13, 100]. In a meta-analysis,
Lipsey found that programs that coupled resti-
tution with competency development were
effective in reducing recidivism [101]. This in-
cluded assistance from probation officers or
caseworkers in finding appropriate job place-
ments to earn money to pay damages, or
meaningful volunteer placements where offend-
ers learned marketable skills.
IV. CONSIDERATIONS FOR IMPLEMENTING
EVIDENCE-BASED PROGRAMS
Knowing that a program has undergone
rigorous evaluation and has strong evidence
that it works is an essential first step in moving
toward more effective, evidence-based practice.
However, whether a program is effective in-
volves a great deal more than just using an
evidence-based program. Among other things, it
requires that the program selected be appropri-
ate to the audience, that it is adequately funded
and staffed, and that the selected program is
implemented with fidelity. These less tangible
matters are often overlooked by program spon-
sors, but are as important as the program model
itself if a program is to have an impact. In addi-
tion to these issues, there exist a number of
practical considerations related to the realities of
program administration, which are often barri-
ers to the use of evidence-based programs. In
the following section, we briefly review some of
these considerations.
SELECTING AN APPROPRIATE
EVIDENCE-BASED PREVENTION
PROGRAM
Knowing that a program has undergone
rigorous testing and evaluation can reassure
potential program sponsors that the program is
likely to be effective under the right conditions
and with the appropriate audience. However,
knowing which program is the “right” one for a
particular setting and audience is not always
easy to determine. There are a number of critical
factors that planners need to consider when se-
lecting a program for their organization or
community. In Appendix D, we provide a de-
tailed set of guidelines to assist program
planners in the task of selecting an appropriate,
evidence-based program for their particular
agency and audience.
STRATEGIES FOR SUCCESSFULLY
IMPLEMENTING PROGRAMS
Selecting an appropriate, proven, evidence-
based program is just the first step in having a
successful program and achieving desired out-
comes. It is becoming increasingly clear that a
program’s success is also highly dependent on
how well it is implemented. Well-designed, evi-
dence-based programs will not produce desired
results if they are implemented poorly. For a
program to be successful, it needs people who
care about and “champion” the program, quali-
fied staff members who have access to training
and technical assistance, and an organization
that is ready and willing to implement the pro-
gram as designed.
What Works, Wisconsin Page 35
A major goal of the OJJDP-supported Blue-
prints project at the University of Colorado has
been to identify those factors that are critical for
effective program implementation [57]. Several
key elements need to be in place if a program is
to succeed. Based on their evaluation of success-
fully implemented evidence-based programs,
the Blueprints team identified factors that are
important for successful program implementa-
tion: site assessment, effective organization,
qualified staff, program champions, program
integrity, training and technical assistance, and
implementation fidelity.
Site assessment. Members of the sponsoring
organization or program site should come to-
gether to examine and assess their needs,
commitment and resources before they actually
implement the program. This might include dis-
cussion of the program’s feasibility, including
potential problems and obstacles that might
arise and ways to overcome them. This step can
provide a forum for program staff and sponsors
to gain a better understanding of the program’s
operation and requirements, which in turn can
lead to increased enthusiasm for the program by
the staff, as well as less fear and resistance.
Effective organization. It is important for
groups planning to implement a program to
have a strong organization in place that includes
solid administrative support, agency stability, a
shared vision and interagency links. Adminis-
trative support is critical because it is
administrators who usually hold the power to
allocate resources and make program decisions,
but it is usually program staff who are responsi-
ble for implementing the program. If they are
not in synch with one another, or if staff do not
feel supported from above, the chances of a pro-
gram failing increases significantly. Effective
organizations also have a high degree of staff
stability. When there is high staff turnover, there
may be less continuity in how the program is
delivered. Staff may be less prepared to deliver
the program, and delays in implementation may
occur. Effective organizations also have a shared
vision of the program’s goals and objectives and
how they fit with the organization’s philosophy
and goals. Without a common vision, programs
may be implemented differently by staff mem-
bers, leading to inconsistency, confusion and
even friction between staff and administrators.
Finally, programs are more likely to succeed
when other organizations and programs are
supportive of them and where there are estab-
lished interagency links. This is especially
important for programs where a participant’s
intervention plan requires coordination with
other agencies or programs.
Qualified staff. An often overlooked aspect
of effective programs is the quality and com-
mitment of the staff who will be implementing
it. Many assume that the effectiveness of a pro-
gram resides primarily in the program activities,
but it is becoming increasingly clear that the
quality, training and commitment of the staff are
also essential. The Blueprints study found that
well implemented, successful programs had
staff members who were motivated to do the
program, felt a sense of ownership for it, and
had the credentials, skills and experience
needed to do the required tasks. While the im-
portance of these staff characteristics may sound
obvious, it is not uncommon to have staff mem-
bers implementing a program who are
unprepared or unqualified. Moreover, when
program decisions are made by administrators
without the input of staff, the staff implement-
ing the program may not be committed to it and
may resent having to spend their time on it.
A final staff issue that appears to be very
commonplace is the lack of adequate time to
implement the program. In many cases, when a
site decides to implement a new prevention or
intervention program, it is added on top of exist-
ing programs or work with clients. In other
words, additional responsibilities are added to
existing ones without an adjustment in staff
workloads. When this occurs, staff members are
more apt to become overworked, frustrated and
dissatisfied with the program, leading to reluc-
tance to devote the time, energy and
commitment necessary to implement the pro-
gram well or even at all.
What Works, Wisconsin Page 36
Program champions. Most successfully im-
plemented programs have what is typically
referred to as a program champion. This is
someone in the sponsoring organization who is
enthusiastic about the program and possesses
enough organizational power and staff rapport
to influence decisions and implement plans.
They usually assume the role of the program’s
director or coordinator. For this reason, champi-
ons are often from the administrative level, but
they can come from any level of the sponsoring
organization and need not be limited to a single
person. If there is no one in the organization to
champion a program, its chances of success are
significantly diminished.
Training and technical assistance. Central to
a program’s success is the provision of strong
training and ongoing technical assistance to
support the program. Without adequate train-
ing, staff members may not be prepared to
implement the program or implement it well.
The Blueprints study found that staff who were
trained were more likely to implement their
programs and were more likely to do so with
greater fidelity to the original model. They were
also more confident and better prepared to
overcome problems that arose. Other research
has found that programs with well-trained staff
are more likely to have favorable participant
outcomes [102]. In addition, the availability of
ongoing technical assistance can help program
staff address unforeseen problems and obstacles
that often arise when implementing a new pro-
gram.
Implementation fidelity. A common finding
of successful evidence-based programs is that
they are implemented with fidelity. Fidelity re-
fers to how well a program is implemented
according to the original program design. When
programs are disseminated to new settings it is
not uncommon for the new site to make modifi-
cations in the program’s design or delivery.
Often these changes make the program less ef-
fective, leading to poorer outcomes [103, 104].
There are several aspects of fidelity that pro-
gram staff should consider when implementing
a new program:
• Adherence: Does the program include the
components and materials as it was
originally designed or have core com-
ponents of the program been modified
or dropped? Is the program being used
with the appropriate audience?
• Exposure or Dosage: Are the number,
length and frequency of sessions the
same as the program’s original design,
or have they been significantly modified
or shortened?
• Quality of program delivery: Are the indi-
viduals delivering the program well
trained and prepared to deliver the pro-
gram in the manner prescribed?
• Participant responsiveness: Are the par-
ticipants engaged and involved in the
program as originally intended?
BARRIERS TO IMPLEMENTING
EVIDENCE-BASED PROGRAMS
As this report has demonstrated, there are a
growing number of programs that have been
found to be effective in reducing delinquency
and related problems. However, a number of
significant barriers are likely to keep practitio-
ners from using such programs. Understanding
these barriers can be a first step in developing
strategies to overcome them.
Perhaps the greatest barrier to the use of
evidence-based programs is that many practi-
tioners and policy makers have little or no
understanding of what they are and why they
can be beneficial. Moreover, they do not know
where to find them. Consequently, a first step in
the wider dissemination of evidence-based pro-
grams is to educate practitioners and policy
makers about what they are, their practical and
economic benefits, and where they can be found.
An equally important obstacle to the adop-
tion of evidence-based programs is the
significant financial and human resources that
many of these programs require. One of the rea-
sons that evidence-based programs work is that
they are intensive and long term. However,
What Works, Wisconsin Page 37
many practitioners may be reluctant to make
such a long-term investment or commit the re-
quired number of staff because it draws time
and resources from other initiatives. This is one
of the most common reasons given by program
administrators and staff for using less intensive,
unproven programs.
Because evidence-based programs often
need to be purchased from the program design-
ers, their initial costs can be fairly significant,
especially when compared to untested programs
or programs devised by agency staff. In addi-
tion, many evidence-based programs can only
be implemented by individuals who have un-
dergone formal training by the original program
designers or their representatives. This is usu-
ally an additional expense and can be a financial
burden to agencies that are already strapped for
funding. Moreover, to ensure that programs are
implemented well and that staff learn from the
process and make ongoing improvements, pro-
grams must be monitored and evaluated. To do
this right, program staff may need ongoing ac-
cess to technical assistance on program design,
implementation and evaluation. Without such
access, implementation problems may occur that
can undermine a program’s effectiveness.
CULTURAL/ETHNIC
CONSIDERATIONS
As Wisconsin and the nation become more
ethnically and culturally diverse, it is increas-
ingly important to take into account how both
policies and programs might affect minority
populations. This is especially crucial in the ju-
venile justice system where some racial/ethnic
groups are disproportionately represented. An
increasingly common question raised by policy
makers and practitioners is whether commonly
implemented programs that are not specifically
tailored to minority audiences are equally effec-
tive for minority and majority youth.
A recent meta-analytic study by Wilson,
Lipsey and Soydan examined this question and
provides some initial insight [105]. They found
that delinquency prevention programs, as a
group, were equally effective for minority and
white youth. However, they did find some
trends suggesting that the effectiveness of some
particular programs was slightly less for minor-
ity youth than white youth.
These findings indicate that programs can
be equally effective for multiple groups without
being tailored to any particular group. It may
well be that programs specifically tailored to
particular minority populations are more effec-
tive than culturally blind, mainstream programs.
However, because there are so few culturally
tailored programs – and even fewer studies that
examine their effectiveness – this question can-
not yet be answered empirically.
While considering this question, it is impor-
tant to note that disproportionate minority
contact (DMC) with the juvenile justice system is
not a problem that can be solved solely by tailor-
ing prevention and intervention programs for
particular cultural and ethnic groups. Unfortu-
nately, current practices in the arrest, detention,
disposition, and release of juveniles tend to fa-
vor white youth over minority youth at every
stage in the process [75, 106]. While making pre-
vention and intervention programs more
effective is a desirable goal and a step in the
right direction, concrete changes in the practices
that bring youth into the juvenile justice system
are more likely to make an impact on DMC in
the short-run.
What Works, Wisconsin Page 38
V. RECOMMENDATIONS
Based on our synthesis of the available evi-
dence on the effectiveness of juvenile crime
prevention programs, we present some recom-
mendations for enhancing the quality and
impact of juvenile crime prevention programs in
Wisconsin.
(1) Strongly support the use of evidence-
based prevention and intervention pro-
grams and practices. The scientific
knowledge base regarding the prevention
and treatment of juvenile delinquency has
reached a critical mass. Although a great
deal is known about the types of ap-
proaches that are likely to be effective, the
use of these evidence-based programs and
practices lags far behind. Much current
practice is based on past history and good
intentions. The effectiveness of current
practice could be significantly improved if
practitioners were to adopt evidence-
based practices and funders were to direct
funding toward programs and practices
supported by current scientific evidence.
We have identified a number of proven
and cost-effective programs in several
categories of prevention that provide a
good foundation for future crime preven-
tion.
(2) Educate policy-makers and practitioners
about evidence-based programs and
practices and their practical and eco-
nomic benefits. A major barrier to the
adoption of evidence-based programs and
practices is the common lack of under-
standing of what these programs and
practices are and why they are more
credible than less scientific approaches.
The benefits of evidence-based programs
are not limited to their ability to bring
about effective, verifiable impacts. They
also encourage efficiency in that local con-
stituencies/agencies can draw on existing
programs without needing to spend time
creating their own. This allows limited
human and financial resources to be used
wisely. The use of evidence-based pro-
grams also has important political
benefits, especially in a period of limited
funding. Particularly when cost-benefit
analysis can be calculated, evidence of
program effectiveness can be used to as-
sure policy-makers, funders and
practitioners that such programs are a
wise and worthy investment. Finally, edu-
cation about evidence-based programs
must emphasize the importance of im-
plementing programs with fidelity to the
original program design. Diluted pro-
grams are likely to violate the principles of
effectiveness and unlikely to produce the
same effects as the originals.
(3) Use results of cost-benefit analysis to
better prioritize funding of education
and social programs. In a time of increas-
ingly limited fiscal resources, greater
scrutiny of existing programs and services
becomes essential. Cost-benefit analysis
and other impact evaluations are espe-
cially important because they can identify
the most efficient use of taxpayer dollars
for crime prevention and other outcomes.
Although there are many criteria to be
used in funding decisions and not all ef-
fective programs are analyzed for returns,
increased funding for and use of economic
analyses of social programs are some of
the best ways to determine the most effi-
cient use of public investments in young
people.
(4) Adopt an appropriate and validated as-
sessment tool in order to direct juvenile
offenders to the level of intervention and
supervision that is most likely to be ef-
fective for them. A high-quality
assessment tool helps to ensure that juve-
nile justice funds are being used
What Works, Wisconsin Page 39
effectively. To the extent that assessment
tools are attached to standards for diver-
sion, detention, and disposition, they also
ensure fairness and reduce concerns about
differential treatment from one offender to
another based on race, class, or other per-
sonal characteristics unrelated to
likelihood of re-offending. Several states
and jurisdictions have developed high-
quality assessment tools. Multnomah
County, Oregon, for example, has devel-
oped the “Risk Assessment Instrument,”
used both at intake and at preliminary
hearings, which assigns point values to
various criminogenic factors that influence
an offender’s risk of recidivism [107]. Of-
fenders with few risk factors are then
diverted away from court processing and
into appropriate diversion programs.
Washington State has also developed a
“Juvenile Court Pre-Screen Assessment”
[34]. Wisconsin’s Juvenile Classification
System [35] has been validated for use
with offenders placed in juvenile correc-
tions but could be revised and re-
validated for use with all juvenile offend-
ers earlier in their contact with law
enforcement and juvenile justice.
(5) Develop mechanisms for disseminating
effective program models and good prac-
tice guidelines to practitioners and
decision-makers. In order to facilitate the
adoption of evidence-based programs and
practices, decision-makers and practitio-
ners need to have access to the latest
knowledge base on what works, as well as
technical assistance for selecting and im-
plementing high-quality programs.
Getting this information out to those who
need it will require some investment and
creativity. Several strategies that we be-
lieve hold promise for facilitating the
wider dissemination and use of evidence-
based programs and practices include:
• “What Works” practitioner guides for
various categories of programs and
approaches
• Dissemination of this report in print
and online, with links to other re-
sources like the OJJDP Model
Programs Guide and the Juvenile Jus-
tice Evaluation Center
• Developing a system to keep practi-
tioners updated on the latest
programs and practices, such as the
planned training series to be offered
in partnership with the Department of
Corrections, and the development of a
“What Works, Wisconsin” web site
that is regularly updated
• Developing a statewide infrastructure
to provide regular training on evi-
dence-based programs and practices,
assistance on how communities can
select, implement, fund and sustain
such programs (including how to
work collaboratively with other agen-
cies), and a system for ongoing
technical assistance for all aspects of
program implementation and evalua-
tion.
(6) Provide support for local-level delin-
quency prevention initiatives.
Ultimately, most decisions about delin-
quency prevention are made and
implemented locally, even though they
are often funded by state and federal
sources. Without local support and action,
most prevention initiatives will have little
chance of success. Because effective pre-
vention approaches need to be
comprehensive, targeting multiple risk
and protective factors, their success de-
pends on the cooperation of multiple
private and public agencies. Such collabo-
ration becomes even more important if
initiatives involve the use of time- and re-
source-intensive evidence-based programs
and practices. In addition, the success of
such community initiatives depends on
developing a local infrastructure to help
plan and implement them. Sharing such
models and providing local support and
What Works, Wisconsin Page 40
guidance for their development around
the state is critical. Fortunately, there are a
number of potential partners (like UW
Cooperative Extension) who may be able
to serve as key partners in the dissemina-
tion of such knowledge and as catalysts
for local organizing and action.
(7) Increase investments in research and de-
velopment (R & D) and in evaluation of
emerging, innovative, and promising
prevention programs. The National Sci-
ence and Technology Council estimates
that of the total annual expenditures of so-
cial programs for young people, only 1/3
of 1 percent goes to R & D along the lines
of the evaluation research documented in
this report [108]. In comparison, national
R & D investments in energy, biomedical
sciences, and transportation average 2 to 3
percent of total expenditures (estimated
by GDP). R & D investments are needed
both to assess the effectiveness of existing
programs and to support innovation to
address emerging issues and needs at the
local, state, and national levels. Resources
should be set aside for developing new
programs and for rigorously evaluating
interventions and policies that have been
developed within Wisconsin or have been
implemented widely in the state. For ex-
ample, there is a pressing need for
development and evaluation of preven-
tion programs tailored to specific ethnic or
cultural groups to ensure that all Wiscon-
sin residents have access to programs and
services that effectively meet their needs.
Through set-aside funds or matching
grants with state and local governments,
we recommend that R & D investment in
crime prevention approach the 2 to 3 per-
cent commonly invested in other areas.
(8) Provide a greater balance between pre-
vention and intervention programs and
strategies. Current funding practices at all
levels emphasize treatment over preven-
tion, despite the fact that the evidence for
the effectiveness of prevention approaches
to delinquency far surpasses the evidence
about the effectiveness of treatment. The
National Science and Technology Council
estimates that of the $555 billion devoted
annually to children and youth up to age
21 through established sources (e.g., K-12
education, juvenile justice, health), only
$15 billion or 2.7% of total expenditures
went to investments that could be classi-
fied as prevention [108]. Thus, only about
1 dollar of every 40 public dollars goes to
prevention programs like the ones re-
viewed in this report. Public expenditures
in Wisconsin do not appear to be different
than in other states. By increasing the ratio
of public investments for prevention rela-
tive to treatment, public policies for
children and youth would be better
aligned with the current evidence base.
(9) Create new, state-level, operational poli-
cies that encourage cross-agency
collaboration and funding for preven-
tion. Traditional funding practices are
outdated and do not support new under-
standings of what makes for effective
action. Categorical funding streams that
provide support only in response to nar-
rowly defined problems and are available
only when problems become chronic or
severe dominate the terrain, making it al-
most impossible to create multi-pronged,
responsive community-based support sys-
tems that would support effective
interventions. Wraparound programs, for
example, may be most effective when
funding streams are blended to allow care
coordinators freedom in connecting fami-
lies to the services they need. To
strengthen effectiveness of crime preven-
tion efforts, adequate and stable funding
for prevention services is essential.
(10) Develop new state funding mechanisms
that are equitable and consistent with the
economic benefits of prevention pro-
grams. Much of the funding and almost
all of the human resources for the pro-
grams we examined are provided locally,
What Works, Wisconsin Page 41
often by community groups or agencies.
However, as the Washington State report
points out, the long-term economic sav-
ings do not primarily accrue to the local
jurisdiction but to the state, which benefits
by reducing its need to invest in future re-
habilitation, incarceration and other
related programs [2]. Because of this eco-
nomic imbalance in the benefits of
prevention, local jurisdictions may lack
the incentives and resources necessary to
invest in the types of evidence-based pro-
grams that are going to have long-term
impacts and lead to future state savings. It
is important for state policy-makers to
understand how long-term savings are
likely to accrue from initial local invest-
ments in community-based prevention
and consider funding mechanisms that are
appropriate.
A similar issue arises at the level of state
agencies. Most prevention programs are
broad in their impact, leading to reduc-
tions in a range of problematic outcomes
and the promotion of a variety of positive
developmental consequences. For exam-
ple, investments in quality, evidence-
based early childhood programs not only
benefit the educational system but also the
welfare, juvenile justice and corrections
systems. State funding, however, contin-
ues to follow very narrow streams,
focusing on specific outcomes such as the
prevention of child abuse, delinquency or
drug abuse. If prevention is going to work
within the state, significant reorganization
is needed on how state agencies fund pre-
vention.
We recommend that the following mecha-
nisms for prevention funding be
considered, several of which have been es-
tablished in other states:
• Form a state-level prevention com-
mission comprised of representatives
of state agencies that invest in preven-
tion, similar to California’s Children’s
Commission. Each agency would an-
nually contribute funds to be invested
in evidence-based programs overseen
by the commission. Two to three per-
cent of the total amount of funding
would be reserved for research and
evaluation.
• Create a public/private endowment
for funding evidence-based preven-
tion programs. Similar to investment
strategies in biotechnology, the state
would provide base levels of funding
for cost-effective prevention pro-
grams, which could then be matched
by local communities and the private
sector. Minnesota uses such an en-
dowment to fund preschool
programs.
• Issue state bonds to finance preven-
tion initiatives that are likely to
provide high economic returns. Given
the availability of evidence on the
cost-effectiveness of many prevention
investments, the state of Wisconsin
could finance targeted investments by
issuing bonds. The programs funded
by the bonds would have a strong re-
search base from which to define the
length of time needed for the program
to recover the interest on the bonds
and provide positive economic re-
turns. While issuing state bonds for
specific prevention programs is un-
precedented, Wisconsin and
California currently issue bonds for
general revenue outlays.
• Develop a check-off box on the state
income tax form for voluntary contri-
butions to prevention program
funding. As implemented in Illinois
and other states, taxpayers could con-
tribute any dollar amount to
prevention programs overseen by the
state prevention commission or a
specified government agency. Among
the options for contributions could be
What Works, Wisconsin Page 42
preschool education, child abuse pre-
vention, birth to three services, and
domestic violence.
• Redirect a portion of funds from exist-
ing expenditures to prevention.
Current categorical funding for many
education and justice system pro-
grams is heavily weighted toward
remediation or treatment. Rebalancing
the allocations even by a small per-
centage would provide needed funds
to implement cost-effective prevention
programs. For example, in K-12 edu-
cation, the state Department of Public
Instruction and local school districts
receive millions of dollars per year in
Title I block grants to schools serving
low-income students. Nearly 95% of
these funds are directed toward re-
medial education, while less than 5%
goes to prevention programs such as
preschool education.
What Works, Wisconsin Page 43
VI. CONCLUSIONS
The costs of juvenile crime and delinquency
deplete both our human and economic re-
sources. Delinquency and crime negatively
affect our families, schools and community life
while increasing the economic burden on fed-
eral, state and local government. For example, in
2004, the cost of housing a juvenile offender in
corrections in Wisconsin for 12 months was
$68,255 – and is projected to increase nearly 20%
by 2007. When these expenditures are consid-
ered along with financial and emotional costs to
crime victims, as well as costs to society for fu-
ture adult crime and incarceration, it becomes
clear that the prevention of delinquency is both
a social and economic priority.
Over the past decade, our knowledge of cost
effective programs and practices related to de-
linquency prevention and treatment has grown
significantly. Unfortunately, a great deal of cur-
rent practice is based on past customs and good
intentions and does not draw on this rich, scien-
tific knowledge base. Our review found that
there is a great deal of valuable and valid scien-
tific knowledge that could be used to inform
current practice. The utilization of such knowl-
edge by decision makers and juvenile justice
professionals has the potential to increase the
effectiveness of current programs and practices
and lead to long-term economic benefits.
This report identified programs and ap-
proaches that can significantly reduce the odds
that children and youth will engage in delin-
quent behavior, thus increasing the likelihood
that they will be contributing members of soci-
ety and helping taxpayers avoid the high costs
of juvenile treatment and adult incarceration.
The cost of almost all of these programs is a
small fraction of the costs of treatment and fu-
ture crime.
The most cost effective prevention programs
reviewed include preschool education, home
visitation programs, parent education, and so-
cial and emotional learning programs for school-
aged children. In all of these programs, the qual-
ity and intensity of services are high, staff
members are well trained, and the program has
a well-articulated vision with a strong concep-
tual base. Although mentoring and job training
programs were also found to have good evi-
dence of effectiveness, their economic returns
are lower. Among juvenile offender programs,
the strongest empirical evidence of cost-
effectiveness is for diversion or community ac-
countability programs and therapeutic
interventions that provide a range of intensive
services over relatively long periods of time.
We also examined several promising crime
prevention programs that appear to follow key
principles of effective programs but have not yet
demonstrated reliable program impacts. These
include after-school programs and enhanced
probation and supervision programs. Addi-
tional research on these and other programs is
needed.
This report concludes with a series of rec-
ommendations that have the potential to
positively alter the future life chances of Wis-
consin youth, reduce crime, and contribute to
significant cost savings. These recommendations
include changes in how funding decisions are
made; how practitioners are trained and sup-
ported; how programs are selected,
implemented and evaluated; and how state
agencies coordinate and fund prevention initia-
tives. However, putting into action most of these
recommendations will require both vision and
courage – the vision to look beyond short-term
solutions and the courage to challenge the status
quo and adopt new ways of operating. We hope
that this report will serve as an impetus for
change and contribute to the emergence of Wis-
consin as a national leader for innovative,
scientific, and cost-effective policies and pro-
grams on behalf of its youth.
What Works, Wisconsin References
VII. REFERENCES
1. Aos, S., Phipps, P., Barnoski, R., & Lieb, R. (2001). The comparative costs and benefits of programs to
reduce crime (Version 4.0). Olympia: Washington State Institute for Public Policy.
2. Aos, S., Lieb, R., Mayfield, J., Miller, M., & Pennucci, A. (2004). Benefits and costs of prevention and
early intervention programs for youth. Olympia: Washington State Institute for Public Policy.
3. Development Services Group for the Office of Juvenile Justice and Delinquency Prevention.
(2004). Model programs guide (Version 3.2). Retrieved March 18, 2005, from
http://www.dsgonline.com/Model_Programs_Guide/Web/mpg_index_flash.htm
4. Durlak, J. (1997). Successful prevention programs for children and adolescents. New York: Plenum
Press.
5. Coie, J., Watt, N., West, S., Hawkins, J., Asarnow, J., Markman, H., Ramey, S., Shure, M., & Long,
B. (1993). The science of prevention: A conceptual framework and some directions for a national
research program. American Psychologist, 48, 1013-1022.
6. Kirby, L. & Fraser, M. (1997). Risk and resilience in childhood. In M. Fraser (Ed.), Risk and
resilience in childhood (pp. 10-33). Washington, D.C.: NASW Press.
7. Rutter, M. (1987). Psychosocial resilience and protective mechanisms. American Journal of
Orthopsychiatry, 57, 316-331.
8. Small, S.A. & Memmo, M. (2004). Contemporary models of youth development and problem
prevention: Toward an integration of terms, concepts, and models. Family Relations:
Interdisciplinary Journal of Applied Family Studies, 53(1), 3-11.
9. LaTessa, E.J. (1999). What works in correctional intervention. Southern Illinois University Law
Journal, 23, 415-426.
10. Kelly, J.G. (1986). An ecological paradigm: Defining mental health consultation as a preventative
service. In J.G. Kelly and R. Hess (Eds.), The ecology of prevention: Illustrating mental health
consultation (pp. 1-36). New York: Haworth Press.
11. Small, S.A. & Luster, T. (1994). An ecological, risk-factor approach to adolescent sexual activity.
Journal of Marriage and the Family, 56, 181-192.
12. Bogenschneider, K.P., Small, S.A., & Riley, D.A. (1990). Risk and protective factors in adolescent
development. Paper presented at the Cooperative Extension National Youth-At-Risk Summit,
Washington, D.C.
13. Howell, J.C. (2003). Preventing and reducing juvenile delinquency: A comprehensive framework.
Thousand Oaks, CA: Sage Publications.
14. Loeber, R. & Farrington, D. (Eds.). (2003). Child delinquents: Development, intervention, and service
needs. Thousand Oaks, CA: Sage Publications.
15. Roth, J., Brooks-Gunn, J., Murray, L., & Foster, W. (1998). Promoting healthy adolescents:
Synthesis of youth development program evaluations. Journal of Research on Adolescence, 8,
423-459.
16. Benson, P.L. (1997). All kids are our kids: What communities must do to raise caring and responsible
children and adolescents. San Francisco, CA: Jossey-Bass.
What Works, Wisconsin References
17. Benson, P.L., Leffert, N., Scales, P.C., & Blyth, D.A. (1998). Beyond the ʺvillageʺ rhetoric: Creating
healthy communities for children and adolescents. Applied Developmental Science, 2, 138-159.
18. Scales, P.C. & Leffert, N. (1999). Developmental assets: A synthesis of the scientific research on
adolescent development. Minneapolis: Search Institute.
19. Heckman, J. (2000). Policies to foster human capital. Research in Economics, 54, 3-56.
20. Karoly, L.A. (2001). Investing in the future: Reducing poverty through human capital
investments. In S.H. Danziger and R.H. Haveman (Eds.), Understanding poverty (pp. 314-356).
New York: Russell Sage Foundation.
21. Ringwalt, C., Ennett, S., Vincus, A., Thorne, J., Rohrbach, L., & Simons-Rudolph, A. (2002). The
prevalence of effective substance use prevention curricula in U.S. middle schools. Prevention
Science, 3(4), 257-265.
22. Caplan, G. (1964). The principles of preventive psychiatry. New York: Basic Books.
23. Bloom, M. (1996). Primary prevention practices. Thousand Oaks, CA: Sage Publications.
24. Institute of Medicine. (1994). Reducing risks for mental health disorders: Frontier for preventive
intervention research. Washington, D.C.: National Academy Press.
25. Levin, H.M. & McEwan, P.J. (2001). Cost-effectiveness analysis: Methods and applications (2nd ed.).
Thousand Oaks, CA: Sage Publications.
26. Bond, L.A. & Carmola Hauf, A.M. (2004). Taking stock and putting stock in primary prevention:
Characteristics of effective programs. The Journal of Primary Prevention, 24(3), 199-221.
27. Kumpfer, K.L. & Alvarado, R. (2003). Family-strengthening approaches for the prevention of
youth problem behaviors. American Psychologist, 58(6), 457-465.
28. Nation, M., Crusto, C., Wandersman, A., Kumpfer, K., Morrisey-Kaner, E., Seybolt, D., & Davino,
K. (2003). What works in prevention: Principles of effective prevention programs. American
Psychologist, 58(6/7), 449-456.
29. Andrews, D.A., Zinger, I., Hoge, R.D., Bonta, J., Gendreau, P., & Cullen, F.T. (1990). Does
correctional treatment work? A clinically relevant and psychologically informed meta-analysis.
Criminology, 28(3), 369-404.
30. Gendreau, P. & Andrews, D.A. (1990). Tertiary prevention: What the meta-analyses of the
offender treatment literature tell us about ʺwhat works.ʺ Canadian Journal of Criminology, 32(1),
173-184.
31. Lowenkamp, C.T. (2004). Results and lessons learned from Ohio: The principles of effective intervention.
Retrieved February 20, 2005, from
http://www.uc.edu/criminaljustice/Articles/Lessons_Learned.ppt
32. Borum, R. (2003). Managing at-risk juvenile offenders in the community: Putting evidence-based
principles into practice. Journal of Contemporary Criminal Justice, 19(1), 114-137.
33. Risler, E.A., Sutphen, R., & Shields, J. (2000). Preliminary validation of the juvenile first offender
risk assessment index. Research on Social Work Practice, 10(1), 111-126.
34. Barnoski, R. (2004). Assessing risk for re-offense: Validating the Washington State Juvenile Court
Assessment. Olympia: Washington State Institute for Public Policy.
What Works, Wisconsin References
35. Wisconsin Division of Juvenile Corrections. (no date). Wisconsin Juvenile Classification System.
Madison, WI: Department of Corrections, Division of Juvenile Corrections.
36. Consortium for Longitudinal Studies. (1983). As the twig is bent… Lasting effects of preschool
programs. Hillsdale, NJ: Erlbaum.
37. Karoly, L.A., Greenwood, P.W., Everingham, S.S., Hoube, J., Kilburn, M.R., Rydell, C.P., Sanders,
M., & Chiesa, J. (1998). Investing in our children: What we know and donʹt know about the costs and
benefits of early childhood interventions. Santa Monica, CA: RAND.
38. Schweinhart, L.J., Montie, J., Xiang, Z., Barnett, W.S., Belfield, C.R., & Nores, M. (in press).
Lifetime effects: The High/Scope Perry Preschool study through age 40. (Monographs of the High/Scope
Educational Research Foundation, 14). Ypsilanti, Michigan: High/Scope Educational Research
Foundation.
39. Reynolds, A.J., Temple, J.A., Robertson, D.L., & Mann, E.A. (2002). Age 21 cost-benefit analysis of
the Title I Chicago Child-Parent Centers. Educational Evaluation and Policy Analysis, 24(4), 267-303.
40. Masse, L.N. & Barnett, W.S. (2002). A benefit-cost analysis of the Abecedarian early childhood
intervention. New Brunswick, NJ: National Institute of Early Education Research.
41. Garces, E., Thomas, D., & Currie, J. (2002). Longer term effects of Head Start. American Economic
Review, 84, 900-915.
42. Lally, J.R., Mangione, P.L., Honig, A.S., & Wittner, D.S. (1988). More pride, less delinquency:
Findings from the ten-year follow-up study of the Syracuse University Family Development
Research Program. Zero to Three, 8(4), 13-18.
43. Gilliam, W.S. & Zigler, E.F. (2004). State efforts to evaluate the effects of prekindergarten: 1977-2003.
New Haven, CT: Yale University Child Study Center. Available at
http://nieer.org/resources/research/StateEfforts .
44. Rapoport, D. & OʹBrien-Strain, M. (2001). In-home visitation programs: A review of the literature.
Retrieved May 10, 2005, from http://www.sphereinstitute.org/publications/OCProp10LitRev
45. Olds, D., Henderson, C.R., Jr., Cole, R., Eckenrode, J., Kitzman, H., Luckey, D., Pettitt, L., Sidora,
K., Morris, P., & Powers, J. (1998). Long-term effects of nurse home visitation on childrenʹs
criminal and antisocial behavior. Journal of the American Medical Association, 280(14), 1238-1244.
46. Olds, D., Pettitt, L.M., Robinson, J., Henderson, C., Jr., Eckenrode, J., Kitzman, H., Cole, B., &
Powers, J. (1998). Reducing risks for antisocial behavior with a program of prenatal and early
childhood home visitation. Journal of Community Psychology, 26(1), 65-83.
47. Sweet, M.A. & Appelbaum, M.I. (2004). Is home visiting an effective strategy? A meta-analytic
review of home visiting programs with young children. Child Development, 75(5), 1435-1456.
48. National Center for Children, Families, and Communities. Nurse-Family Partnership. Retrieved
April 5, 2005, from http://www.nccfc.org/pdfs/descriptionNFP
49. Webster-Stratton, C. & Hammond, M. (1997). Treating children with early-onset conduct
problems: A comparison of child and parent training interventions. Journal of Consulting and
Clinical Psychology, 65(1), 93-109.
50. Eckenrode, J., Izzo, C., & Campa-Muller, M. (2002). Early intervention and family support
programs. In R.M. Lerner, F. Jacobs, and D. Wertlieb (Eds.), Handbook of Applied Developmental Sci-
ence (Vol. 2, pp. 161-195). Thousand Oaks, CA: Sage.
What Works, Wisconsin References
51. Strengthening Americaʹs Families. Retrieved April 1, 2005, from http://strengtheningfamilies.org/
52. Collaborative for Academic, Social, and Emotional Learning. (2003). Safe and Sound: An Educa-
tional Leaderʹs Guide to Evidence-based Social and Emotional (SEL) Programs. Retrieved April 5, 2005,
from http://www.casel.org/downloads/Safe%20and%20Sound/1A_Safe_&_Sound
53. Wilson, D.B., Gottfredson, D.C., & Najaka, S.S. (2001). School-based prevention of problem
behaviors: A meta-analysis. Journal of Quantitative Criminology, 17(3), 247-272.
54. Hawkins, J., Catalano, R.F., Morrison, D.M., OʹDonnell, J., Abbott, R.D., & Day, L.E. (1992). The
Seattle Social Development Project: Effects of the first four years on protective factors and
problem behaviors. In J. McCord and R.E. Tremblay (Eds.), Preventing antisocial behavior:
Interventions from birth through adolescence. New York: Guilford Press.
55. Hawkins, J.D., Catalano, R.F., Kosterman, R., Abbott, R., & Hill, K.G. (1999). Preventing
adolescent health-risk behaviors by strengthening protection during childhood. Archives of
Pediatric and Adolescent Medicine, 153, 226-234.
56. Weissberg, R.P. & Greenberg, M.T. (1997). School and community competence-enhancement and
prevention programs. In I.E. Sigel and K.A. Renninger (Eds.), Child psychology in practice (5th ed.,
Vol. 5, pp. 139-161). New York: John Wiley & Sons.
57. Mihalic, S., Fagan, A., Irwin, K., Ballard, D., & Elliot, D. (2004). Blueprints for Violence Prevention.
Washington, D.C.: Office of Juvenile Justice and Delinquency Prevention.
58. Szapocznik, J., Santisteban, D., Rio, A., Perez-Vidal, A., Santisteban, D., & Kurtines, W.M. (1989).
Family Effectiveness Training: An intervention to prevent drug abuse and problem behaviors in
Hispanic adolescents. Hispanic Journal of Behavioral Sciences, 11, 4-27.
59. University of Utah – Department of Health Promotion and Educaton.Strengthening Families
Program. Retrieved May 25, 2005, from www.strengtheningfamiliesprogram.org
60. Substance Abuse and Mental Health Services Administration (SAMHSA). SAMHSA Model
Programs. Retrieved May 18, 2005, from http://modelprograms.samhsa.gov/
61. Yung, B.R. & Hammond, W.R. (1998). Breaking the cycle: A culturally sensitive violence
prevention program for African-American children and adolescents. In J.R. Lutzker (Ed.),
Handbook of child abuse research and treatment. New York: Plenum.
62. Rhodes, J.E. & Roffman, J.G. (2002). Relationship-based interventions: The impact of mentoring
and apprenticeship on youth development. In R.M. Lerner, F. Jacobs, and D. Wertlieb (Eds.),
Handbook of applied developmental science (Vol. 2, pp. 225-236). Thousand Oaks, CA: Sage.
63. Jekielek, S.M., Moore, K.A., Hair, E.C., & Scarupa, M.S. (2002). Mentoring: A promising strategy for
youth development. Washington, D.C.
64. Tierney, J.P., Grossman, J.B., & Resch, N.L. (1995). Making a Difference: An Impact Study of Big
Brothers Big Sisters. Philadelphia: Public/Private Ventures.
65. Grossman, J.B. & Rhodes, J.E. (2002). The test of time: Predictors and effects of duration in youth
mentoring relationships. American Journal of Community Psychology, 30(2), 199-219.
66. Greenberg, D.H., Michalopoulos, C., & Robins, P.K. (2003). A meta-analysis of government-
sponsored training programs. Industrial and Labor Relations Review, 57(1), 31-53.
What Works, Wisconsin References
67. Burghardt, J., Schochet, P.Z., McConnell, S., Johnson, T., Gritz, R.M., Glazerman, S.,
Homrighausen, J., & Jackson, R. (2001). Does Job Corps Work? Summary of the National Job Corps
Study. Princeton, NJ: Mathematica Policy Research, Inc.
68. Gottfredson, D.C., Gerstenblith, S.A., Soule, D.A., Womer, S.C., & Lu, S. (2004). Do after school
programs reduce delinquency? Prevention Science, 5(4), 253-266.
69. Wisconsin Department of Public Instruction. Charter Schools in Wisconsin. Retrieved April 5, 2005,
from http://www.dpi.state.wi.us/dpi/dfm/sms/csindex.html
70. Gottfredson, D. (2001). Schools and delinquency. New York: Cambridge University Press.
71. Cox, S.M., Davidson, W.S., & Bynum, T.S. (1995). A meta-analytic assessment of delinquency-
related outcomes of alternative education programs. Crime and Delinquency, 41(2), 219-234.
72. Lipsey, M.W. (1992). Juvenile delinquency treatment: A meta-analytic inquiry into the variability
of effects. In T. Cook, et al. (Eds.), Meta-analysis for explanation: A casebook (pp. 83-127). New York:
Russell Sage Foundation.
73. Greenwood, P.W. (1996). Responding to juvenile crime: Lessons learned. The Future of Children,
6(3), 75-85.
74. Davidson, W.S., Redner, R., Amdur, R.L., & Mitchell, C.M. (1990). Alternative treatments for
troubled youth: The case of diversion from the juvenile justice system. New York: Plenum Press.
75. Snyder, H.N. & Sickmund, M. (1999). Minorities in the juvenile justice system. Retrieved April 6,
2005, from http://www.ncjrs.org/pdffiles1/ojjdp/179007
76. Wood-Westland, S. (2002). Nebraska Pretrial Diversion Guidelines and Resources. Retrieved March
28, 2005, from
http://www.ncc.state.ne.us/pdf/juvenile_justice_materials/03_DiversionManual
77. National Youth Court Center. (2005). National Youth Court Center – Listing by state. Retrieved
February 16, 2005, from http://www.youthcourt.net/national_listing/United_states/queryWI.asp
78. Pearson, S.S. & Jurich, S. (2005). A national update: Communities embracing youth courts for at-risk
youth. Washington, D.C.: American Youth Policy Forum.
79. Butts, J.A., Buck, J., & Coggeshall, M.B. (2002). The impact of Teen Court on young offenders.
Philadelphia: Urban Institute, Justice Policy Center.
80. Harrison, P., Maupin, J.R., & Mays, F.L. (2001). Teen court: An examination of processes and
outcomes. Crime and Delinquency, 47(2), 243-264.
81. Minor, K.I., Wells, J.B., Soderstrom, I.R., Bingham, R., & Williamson, D. (1999). Sentence
completion and recidivism among juveniles referred to teen courts. Crime & Delinquency, 45(4),
467-480.
82. Roman, J., Butts, J.A., & Rebeck, A.S. (2004). American drug policy and the evolution of drug
treatment courts. In J.A. Butts and J. Roman (Eds.), Juvenile drug courts and teen substance abuse
(pp. 27-54). Washington, D.C.: The Urban Institute Press.
83. Henggeler, S.W., Clingempeel, W.G., Brondino, M.J., & Pickrel, S.G. (2002). Four-year follow-up
of multisystemic therapy with substance-abusing and substance-dependent juvenile offenders.
Journal of the American Academy of Child & Adolescent Psychiatry, 41(7), 868-874.
What Works, Wisconsin References
84. Borduin, C.M., Mann, B.J., Cone, L.T., & Henggeler, S.W. (1995). Multisystemic treatment of
serious juvenile offenders: Long-term prevention of criminality and violence. Journal of Consulting
& Clinical Psychology, 63(4), 569-578.
85. Alexander, J.F., Pugh, C., Parsons, B.V., Sexton, T., Barton, C., Bonomo, J., Gordon, D., Grotpeter,
J.K., Hansson, K., Harrison, R., Mears, S., Mihalic, S., Ostrum, N., Schulman, S., & Waldron, H.
(2000). Functional Family Therapy. In D.S. Elliot (Ed.), Blueprints for Violence Prevention: Book 3.
Boulder, CO: Center for the Study and Prevention of Violence, Institute of Behavioral Science,
University of Colorado.
86. Chamberlain, P. (2003). Treating chronic juvenile offenders: Advances made through the Oregon multi-
dimensional treatment foster care model. Washington, D.C.: American Psychological Association.
87. Lipsey, M.W., Chapman, G.L., & Landenberger, N.A. (2001). Cognitive-behavioral programs for
offenders. Annals of the American Academy of Political and Social Science, 578, 144.
88. Latessa, E.J. (2005). What works and what doesn’t in reducing recidivism: The principles of effective
intervention. Milwaukee, Wisconsin: Presented to the Wisconsin Governorʹs Juvenile Justice
Commission, March 14, 2005.
89. Maloney, D. (1994). Clearer by the day. In Wisconsin Family Impact Seminars Briefing Report:
Promising approaches for addressing juvenile crime. Madison, WI: Center for Excellence in Family
Studies, University of Wisconsin–Madison.
90. Board of Corrections. (2002). Repeat Offender Prevention Program: Final report to the Legislature.
Sacramento, CA: Author.
91. Puzzanchera, C., Stahl, A.L., Finnegan, T.A., Tierney, N., & Snyder, H.N. (in press). Juvenile Court
Statistics 2000. Pittsburgh, PA: National Center for Juvenile Justice.
92. Wisconsin Division of Juvenile Corrections.DJC technical assistance to counties. Retrieved October
28, 2004, from http://www.wi-doc.com/index_juvenile.htm
93. Wisconsin Statutes, Juvenile Justice Code, 938.534 – Intensive Supervision Program (2003)
94. Robertson, A.A., Grimes, P.W., & Rogers, K.E. (2001). A short-run cost-benefit analysis of
community-based interventions for juvenile offenders. Crime and Delinquency, 47(2), 265-284.
95. Barton, W.H. & Butts, J.A. (1990). Viable options: Intensive supervision programs for juvenile
delinquents. Crime and Delinquency, 36(2), 238-256.
96. Gendreau, P., Goggin, C., Cullen, F.T., & Andrews, D.A. (2000). The effects of community
sanctions and incarceration on recidivism. Forum on Corrections Research, 12(2), 10-13.
97. Howell, J.C. (1998). Guide for implementing the comprehensive strategies for serious, violent, and chronic
juvenile offenders. Washington, D.C.: U.S. Department of Justice, Office of Justice Programs, Office
of Juvenile Justice and Delinquency Prevention.
98. Nugent, W.R., Williams, M., & Umbreit, M.S. (2004). Participation in victim-offender mediation
and the prevalence of subsequent delinquent behavior: A meta-analysis. Research on Social Work
Practice, 14(6), 408-416.
99. Umbreit, M.S., Coates, R.B., & Vos, B. (2004). Victim-offender mediation: Three decades of
practice and research. Conflict Resolution Quarterly, 22(1), 279-303.
What Works, Wisconsin References
100. Schneider, A.L. (1990). Deterrence and juvenile crime: Results from a national policy experiment. New
York: Springer-Verlag.
101. Lipsey, M.W. (1999). Can rehabilitative programs reduce the recidivism of juvenile offenders? An
inquiry into the effectiveness of practical programs. Virginia Journal of Social Policy and the Law, 6,
611-641.
102. Taggart, V.S., Bush, P.J., Zuckerman, A.E., & Theiss, P.K. (1990). A process evaluation of the
District of Columbia ʺKnow Your Bodyʺ project. Journal of School Health, 60, 60-66.
103. LaTessa, E.J. (2004). The challenge of change: Correction programs and evidence-based practices.
Criminology & Public Policy, 3(4), 547-560.
104. Gendreau, P., Goggin, C., & Smith, P. (1999). The forgotten issue in effective correctional
treatment: Program implementation. International Journal of Offender Therapy and Comparative
Criminology, 43(2), 180-187.
105. Wilson, S.J., Lipsey, M.W., & Soydan, H. (2003). Are mainstream programs for juvenile
delinquency less effective with minority youth than majority youth? A meta-analysis of outcomes
research. Research on Social Work Practice, 13(1), 3-26.
106. Pope, C.E., Lovell, R., & Hsia, H.M. (2002). Disproportionate minority confinement: A review of the
research literature from 1989 through 2001. Retrieved April 6, 2005, from
http://ojjdp.ncjrs.org/dmc/pdf/dmc89_01
107. National Center for Juvenile Justice. (March 4, 2005). State Juvenile Justice Profiles: Oregon.
Retrieved April 20, 2005, from http://www.ncjj.org/stateprofiles/
108. National Science and Technology Council. (1997). Investing in our future: A national research
initiative for Americaʹs children for the 21st century. Washington, D.C.: Executive Office of the
President, Office of Science and Technology Policy.
APPENDIX A:
EVIDENCE-BASED PROGRAM DETAILS
TYPE PROGRAM NAME
USED IN
WI?
REGISTRIES
(See App. C)
DELIVERY
SITE/
CONTEXT
AGE
RANGE OF
TARGET
AUDIENCE
TARGETED
POPULATIONS
DOSAGE/
DURATION
PRIMARY PREVENTION
PRESCHOOL
CHICAGO CHILD-PARENT
CENTERS
Yes 6,8,7,10,11 School 3-9 years Economically
disadvantaged children
in urban area
Up to 6 years
HIGH SCOPE PERRY
PRESCHOOL
No 1,2,5,6,7,8,9, 10,
11
School 3-4 years Low SES children 1 to 2 years
ABECEDARIAN PROJECT No 9,10,11 School 6-12 weeks
old through
kindergarten
Low SES children Up to 8 years, with
5 days per week
for 5 years
FAMILY SUPPORT
NURSE-FAMILY
PARTNERSHIP
No 1,2,4,6,8,9,11 Family 0-2 years Children of low-income,
first-time mothers
Weekly-monthy
visits, beginning in
pregnancy and
ending when child
is age 2.
STRENGTHENING FAMILIES –
10-14
Yes 2,3,4,8 Family 10-12 years General population 7 2-hour sessions +
4 boosters
SOCIAL-EMOTIONAL LEARNING
SKILLS, OPPORTUNITIES,
AND RECOGNITION (SOAR),
formerly known as SEATTLE
SOCIAL DEVELOPMENT
PROJECT
Unknown 1,2,3,5,6,7,8,11 School Grades 1-6 Elementary school
children in urban, high
crime areas
Multi-year;
consistent during
school year
OLWEUS BULLYING
PREVENTION
Yes 1,2,7,8 School 6-15 years Elementary and junior
high school students
Multi-year;
consistent during
school year
Page 1 of 6
APPENDIX A:
EVIDENCE-BASED PROGRAM DETAILS
EVALUATED OUTCOMES-
DELINQUENCY
RELATED
EVALUATED OUTCOMES-
NON-DELINQUENCY
RELATED
AGE/TIME AT
LAST FOLLOW-
UP
COSTS PER
YOUTH
BENEFITS PER
YOUTH
NET
BENEFIT
RETURN
PER $1
INVESTED
PRIMARY PREVENTION
PRESCHOOL
Compared to control group: less
delinquency up to age 18, fewer
multiple arrests, fewer arrests
for violent offenses
Compared to control group:
greater school readiness at
kindergarten, higher scores in
reading and math in third grade,
less likely to repeat a grade, less
likely to receive special
education, higher scores on life
skills competency exams, higher
ratings of parent involvement,
fewer reports of child
maltreatment, greater
educational attainment
Age 21 years $7,755 $78,732 $70,977 $10.15
Compared to control group:
reductions in anti-social
behavior and misconduct, fewer
fights, fewer criminal justice
contacts, fewer arrests, less time
on probation
Compared to control group:
greater educational attainment
for the females but not males,
stronger commitment to school,
less likely to have mental
impairment, higher job earnings,
more likely to own their own
homes, less welfare assistance
Age 40 years $16,648 $283,995 $267,347 $17.07
None Compared to control group:
higher math and reading scores,
less likely to repeat a grade, less
likely to receive special
education, higher IQ and school
achievement
Age 21 years $70,588 $142,327 $71,739 $2.02
FAMILY SUPPORT
As Adolescents: less likely to
run away, less likely to be
arrested, less likely to be
convicted of a crime
As Adolescents: fewer sexual
partners, less cigarette smoking.
For mothers: less substance use
during pregnancy, smaller
family size, fewer closely spaced
births, less welfare assistance,
fewer cases of child abuse and
Age 15 years $7,324 $37,041 $29,717 $5.06
Fewer conduct problems Less likely to use substances,
less likely to affiliate with a
delinquent peer group. Parents
more affectionate and set better
limits for children.
4 years $874 $6,833 $5,959 $7.82
SOCIAL-EMOTIONAL LEARNING
Fewer violent acts Better behavior in school, more
attachment to school, greater self-
reported achievement, less likely
to be sexually active, less likely
to have multiple sex partners,
less likely to drink alcohol
heavily
6 years $4,712 $14,810 $10,100 $3.14
Decrease in vandalism, fighting,
theft, bullying
Improved classroom order and
discipline; more positive
attitudes toward school
2.5 years
Page 2 of 6
APPENDIX A:
EVIDENCE-BASED PROGRAM DETAILS
TYPE PROGRAM NAME
USED IN
WI?
REGISTRIES
(See App. C)
DELIVERY
SITE/
CONTEXT
AGE
RANGE OF
TARGET
AUDIENCE
TARGETED
POPULATIONS
DOSAGE/
DURATION
SECONDARY PREVENTION
FAMILY TRAINING
FAMILY EFFECTIVENESS
TRAINING (FET)
Unknown 1,8 Family 6-12 years At-risk Hispanic/Latino
youth and their families
13 1-1.5 hour
sessions
SOCIAL SKILLS TRAINING
POSITIVE ADOLESCENT
CHOICES TRAINING (PACT)
Unknown 8 School 10-18 years African American
student with conduct
problems and/or
histories of victimization
MENTORING
BIG BROTHERS BIG SISTERS
OF AMERICA
Yes 1,2,8 Community 6-18 years Disadvantaged/at-risk
youth; youth from single-
parent families
2-4 times per
month for at least
one year
VOCATIONAL/JOB TRAINING
NATIONAL JOB CORPS Yes 8 School and
Community
16-24 years Economically
disadvantaged youth
Average 8 months
JUVENILE OFFENDER PROGRAMS
DIVERSION
ADOLESCENT DIVERSION
PROJECT, also known as
MICHIGAN STATE
No 8,9 Community 10 to 18 Youth entering justice
system
6-8 hours a week
for 18 weeks
THERAPEUTIC INTERVENTIONS
MULTISYSTEMIC THERAPY No 1,2,4,7,8 Home,
school, and
community
12-17 years Juvenile offenders at
high risk of out-of-home
placement
Average 60 hours
of therapist contact
in a 4-month
period
FUNCTIONAL FAMILY
THERAPY
No 2,4,7,8 Family 11-18 years High-risk/acting out
youth (delinquency,
violence and substance
use) and their families
Average 8 to 12
hours, no more
than 26 hours
(MULTIDIMENSIONAL)
TREATMENT FOSTER CARE
Yes 1,2,3,4,7,8 Foster home 11-18 years Juvnile offenders;
adolescents with chronic
antisocial behavior or
emotional disturbances
6-9 months
CASE MANAGEMENT/MULTIMODAL
REPEAT OFFENDER
PREVENTION PROGRAM
No 8 Community 8 to 15 First-time offenders
under 15.5 years old and
exhibiting at least three
risk factors
Varies, average 12-
18 months
Page 3 of 6
APPENDIX A:
EVIDENCE-BASED PROGRAM DETAILS
EVALUATED OUTCOMES-
DELINQUENCY RELATED
EVALUATED OUTCOMES-
NON-DELINQUENCY
RELATED
AGE/TIME AT
LAST FOLLOW-
UP
COSTS PER
YOUTH
BENEFITS PER
YOUTH
NET
BENEFIT
RETURN
PER $1
INVESTED
SECONDARY PREVENTION
FAMILY TRAINING
Decrease in youth conduct
problems
Decrease in associations with
delinquent peers, improved self-
concept, improved family
functioning
6 months
SOCIAL SKILLS TRAINING
Less physical aggression, less
involvement with the juvenile
court system, less violence-
related charges, fewer offenses
Immediate post-
test
MENTORING
Compared to control group: less
likely to hit someone in the past
year
Compared to control group: less
initiation of alcohol and drug
use, fewer school absenses,
higher grades, higher scholastic
competence scores, better
relationships with family and
friends
18 months after
found eligible to
participate
$4,117 $4,166 $49 $1.01
VOCATIONAL/JOB TRAINING
Reduction in recidivism;
compared to control group:
fewer arrests, fewer convictions,
less time incarcerated
Compared to control group:
more time in academic classes,
larger increases in education and
skill levels, more money earned
after completing the program
48 months after
entering program
$15,804 $22,883 $7,079 $1.45
JUVENILE OFFENDER PROGRAMS
DIVERSION
All but one of the treatment
models outperformed the
control group for reduction in
2 years $1,825 $24,708 $22,883 $13.54
THERAPEUTIC INTERVENTIONS
Reduction in long-term re-arrest
rates
Improved family functioning,
decreases in youth mental health
problems, reduction in
subsequent out-of-home
placements
4 years $5,832 $15,395 $9,563 $2.64
Reductions in recidivism,
reductions in sibling entry into
high-risk behaviors
Improved family positive
communication, reduced
negative/blaming
communication styles
5 years $2,197 $29,111 $26,914 $13.25
Compared to group home
placements: fewer subsequent
arrests, fewer subsequent days
incarcerated
Less hard drug use 12 months $2,524 $27,460 $24,936 $10.88
CASE MANAGEMENT/MULTIMODAL
More likely to complete court-
ordered obligations for
restitution, work, and
community service; Fewer
sustained petitions for new
offenses; Less likely to abscond
Reduced percentage of positive
drug tests; Improved school
attendance; Improved grade
point average; Less likely to fall
below grade level
2 years after
original offense
N/A
N/A N/A N/A
Page 4 of 6
APPENDIX A:
EVIDENCE-BASED PROGRAM DETAILS
TYPE PROGRAM NAME
USED IN
WI?
REGISTRIES
(See App. C)
DELIVERY
SITE/
CONTEXT
AGE
RANGE OF
TARGET
AUDIENCE
TARGETED
POPULATIONS
DOSAGE/
DURATION
STANDARD SERVICE MODEL
JUVENILE COURT Yes N/A Justice
system;
Community
N/A Juvenile offenders 2-3 hearings
STANDARD PROBATION Yes N/A Justice
system;
Community
N/A Juvenile offenders Varies; avg. 12-15
months
JUVENILE CORRECTIONAL
INSTITUTION
Yes N/A Justice
system;
Institution
N/A Juvenile offenders Varies
Page 5 of 6
APPENDIX A:
EVIDENCE-BASED PROGRAM DETAILS
EVALUATED OUTCOMES-
DELINQUENCY RELATED
EVALUATED OUTCOMES-
NON-DELINQUENCY
RELATED
AGE/TIME AT
LAST FOLLOW-
UP
COSTS PER
YOUTH
BENEFITS PER
YOUTH
NET
BENEFIT
RETURN
PER $1
INVESTED
STANDARD SERVICE MODEL
N/A N/A N/A $2,000
average cost
N/A N/A N/A
N/A N/A N/A $2,160
annual cost
N/A N/A N/A
N/A N/A N/A $68,255
annual cost
N/A N/A N/A
Page 6 of 6
What Works, Wisconsin Appendix B
APPENDIX B: EVIDENCE-BASED PROGRAM DESCRIPTIONS
CHICAGO CHILD-PARENT CENTERS
The Child-Parent Centers aim to promote academic achievement by providing service to preschoolers,
kindergarteners, and first- through third-grade children and their families. The centers provide a stable
learning environment with small class sizes to educationally and economically disadvantaged children.
This community-based intervention provides both educational and family-support services. The Centers
encourage parental involvement in the school and in the children’s learning.
For more information:
Sonja Griffin
sogriffin@csc.cps.k12.il.us
(773) 553-1958
HIGH/SCOPE PERRY PRESCHOOL
The High/Scope Perry Preschool targets disadvantaged children 3 to 4 years old. The preschool program
seeks to foster positive intellectual, social, and physical development in participating children. Ulti-
mately, the program aims to decrease crime, teenage pregnancy, and use of welfare by improving the
employment prospects for its participants.
For more information:
High/Scope Educational Research Foundation
http://www.highscope.org/
ABECEDARIAN PROJECT
The Abecedarian program begins at 6 to 12 weeks of age and ends when the child enters kindergarten.
Primarily it is a preschool program in which children receive developmentally appropriate education and
health care. Parents attend group meetings and often receive home visits.
For more information:
FPG Child Development Institute
University of North Carolina
http://www.fpg.unc.edu/
NURSE-FAMILY PARTNERSHIP
The Nurse-Family Partnership is a home-visiting program targeting low-income, first-time mothers and
their babies. Home visits begin during pregnancy and continue until the child is 2 years old.
For more information:
National Center for Children, Families, and Communities
University of Colorado
http://www.nccfc.org/
What Works, Wisconsin Appendix B
STRENGTHENING FAMILIES FOR PARENTS AND YOUTH 10 TO 14
The Strengthening Families for Parents and Youth 10-14 program initially separates parents and children
to work on behavioral and cognitive skills, and then brings them together to practice their skills in family
activities. The program runs for seven sessions with four booster sessions.
For more information:
Iowa State University Extension
http://www.extension.iastate.edu/SFP
SKILLS, OPPORTUNITIES, AND RECOGNITION (SOAR), formerly known as
SEATTLE SOCIAL DEVELOPMENT PROJECT
The SOAR program aims to promote strong bonds between families and schools. The program consists of
two components: cooperative, developmentally-appropriate teaching practices in the classroom and op-
tional parental education classes. The program is designed to prevent or reduce conduct problems, peer
rejection, and academic failure in elementary school students in urban areas.
For more information:
Social Development Research Group
University of Washington
http://depts.washington.edu/ssdp/
OLWEUS BULLYING PREVENTION
The Bullying Prevention program is a universal prevention program designed for school-wide interven-
tions. The program aims to reduce and prevent bullying and victimization in schools.
For more information:
National Center of Rural Justice and Crime Prevention
Clemson University
http://virtual.clemson.edu/groups/ncrj/
FAMILY EFFECTIVENESS TRAINING
Family Effectiveness Training targets Hispanic and Latino families with children 6 to 12 years old. The
training focuses on families in which problems, such as negative peer influence on the child, poor behav-
ior/conduct of the child, or poor parent-child communication, occur. The therapist or facilitator works
with the family members to establish good communication and order within the family.
For more information:
Center for Family Studies
University of Miami School of Medicine
http://www.cfs.med.miami.edu/
What Works, Wisconsin Appendix B
POSITIVE ADOLESCENT CHOICES TRAINING (PACT)
The PACT program typically targets at-risk African-American youths aged 10 to 18, but can be adapted to
other populations, such as youths with conduct disorder or a history of victimizations. This program
teaches social skills and problem-solving techniques through videotaped scenarios involving African-
American youths. The training focuses on issues related to interpersonal violence.
For more information:
The Center for Child and Adolescent Violence Prevention
Wright State University
http://www.wright.edu/sopp/ccavp/
BIG BROTHERS BIG SISTERS OF AMERICA
Currently, Big Brothers/Big Sisters serves youth through more than 500 organizations across the United
States. The program matches children and adolescents from single-parent families with a volunteer men-
tor adult to provide support in their lives. Mentors and matched youth meet two to four times a month
for at least a year.
For more information:
Big Brothers/Big Sisters of America
www.BBBSA.org
JOB CORPS
Job Corps provides economically disadvantaged youths 16 to 24 years of age education and vocational
training in a residential living setting. The Job Corps environment aims to provide the skills necessary for
job attainment.
For more information:
U.S. Department of Labor
http://jobcorps.doleta.gov/
What Works, Wisconsin Appendix B
ADOLESCENT DIVERSION PROJECT/MICHIGAN STATE DIVERSION PROJECT
Juvenile offenders receive behavioral therapy and/or other services from college students, mainly juniors
and seniors, who receive training in behavioral intervention and advocacy. Students and offenders work
one-on-one, for 6-8 hours per week, for 18 weeks. Variations on the program place emphasis on family
involvement in the intervention, relationship building between client and caseworker, or on different be-
havioral therapy approaches.
For more information:
Department of Psychology
Michigan State University
william.davidson@ssc.msu.edu
MULTISYSTEMIC THERAPY
The family of a juvenile offender receives in-home treatment from therapists who are available 24 hours a
day, seven days a week. Therapists have caseloads of 4-6 families at a time, and they focus on improving
parenting effectiveness by empowering parents and teaching them new parenting skills. Average treat-
ment includes about 60 hours of contact over a 4-month period.
For more information:
Multisystemic Therapy Services
http://www.mstservices.com/
FUNCTIONAL FAMILY THERAPY
Functional Family Therapy is designed to treat high-risk youth and their families. The therapy practice is
grounded in clinical theory but is flexible enough to be adapted to treat a variety of complex family prob-
lems.
For more information:
Functional Family Therapy
http://www.fftinc.com/
(MULTI-DIMENSIONAL) TREATMENT FOSTER CARE
In this alternative to group home placement, a troubled or delinquent youth receives treatment through
trained foster parents and continues to attend school in his or her home community. Case managers su-
pervise the foster parents, coordinate needed interventions at school, and develop an individualized
behavior management program to be implemented by the foster parents. The youth’s family of origin is
also trained in behavior management techniques to promote better supervision and continuing interven-
tion when the juvenile returns home.
For more information:
TFC Consultants, Inc.
http://www.mtfc.com/
What Works, Wisconsin Appendix B
REPEAT OFFENDER PREVENTION PROGRAM / THE 8% SOLUTION
All juvenile offenders in California are assessed for recidivism risk at intake. Lower risk offenders are
assigned to the Intensive Intervention Program (for medium-risk) or the Immediate Accountability Pro-
gram (for low-risk). High-risk offenders under age 15½ are referred to the Youth and Family Resource
Center in their community, where several youth-serving agencies work together to devise a case plan.
(Other programs are available for high-risk, older adolescents.) Participants attend school at the center,
are provided transportation to and from the center, receive intensive in-home services for their families,
and may also participate in other center programs, including:
• Parent education and teen parenting classes
• Health screening and health education services
• Drug and alcohol abuse services
• Mental health screening and services
• Afternoon recreation and specialized programming
• Saturday field trips and community service projects
For more information:
Orange County Probation Department – The 8% Solution
http://www.oc.ca.gov/probation/solution/index.asp
California Board of Corrections, Repeat Offender Prevention Program
http://www.bdcorr.ca.gov/cppd/ropp/ropp.htm
What Works, Wisconsin Appendix C
APPENDIX C: LIST OF PROGRAM REGISTRIES
1. Substance Abuse and Mental Health Services Administration (SAMHSA) Model Programs
U.S. Department of Health and Human Services
http://modelprograms.samhsa.gov/
2. Blueprints for Violence Prevention
Center for the Study and Prevention of Violence
University of Colorado
http://www.colorado.edu/cspv/blueprints/index.html
3. Exemplary and Promising Safe, Disciplined and Drug-Free Schools Programs
U.S. Department of Education
http://www.ed.gov/admins/lead/safety/exemplary01/index.html
4. Strengthening Americaʹs Families
University of Utah
http://www.strengtheningfamilies.org/html/
5. The Collaborative for Academic, Social, and Emotional Learning (CASEL)
University of Illinois – Chicago
http://www.casel.org/about_sel/SELprograms.php
6. Promising Practices Network
RAND Corporation
http://www.promisingpractices.net/benchmark.asp?benchmarkid=52
7. Youth Violence: A Report of the Surgeon General
http://www.surgeongeneral.gov/library/youthviolence/chapter5/sec3.html
8. Office of Juvenile Justice and Delinquency Prevention (OJJDP) Model Program Guide
http://www.dsgonline.com/mpg_non_flash/mpg_index2.htm
9. Price, R. H., Cowen, E.L., Lorion, R.P., and Ramos-McKay, J. (Eds.). (1988). 14 Ounces of Prevention: A
Casebook for Practitioners. Washington, D.C.: American Psychological Association.
10. Crane, J. (Ed.). (1998). Social programs that work. New York: Russell Sage Foundation.
11. Weissberg, R. P., and Greenberg, M.T. (1997). School and community competence-enhancement and
prevention programs. In I. E. Sigel, and Renninger, K.A. (Eds.), Child psychology in practice (5 ed., Vol.
5). New York: John Wiley & Sons.
What Works, Wisconsin Appendix D
APPENDIX D: GUIDELINES FOR SELECTING
EVIDENCE-BASED PROGRAMS
Knowing that a program has undergone rigorous testing and evaluation can reassure potential pro-
gram sponsors that the program is likely to be effective under the right conditions and with the
appropriate audience. However, knowing which program is the “right one” for a particular setting and
audience is not always easy to determine. There are a number of critical factors that planners need to con-
sider when selecting a program for their organization or community. Here we provide a detailed set of
guidelines to assist program planners in the task of selecting an appropriate, evidence-based program for
their particular agency and audience. Below we provide a summary of some of these principles.
The goals and objectives of the program should be consistent with the goals and objectives of the sponsoring or-
ganization and the targeted program participants.
Program sponsors should have a good handle on the outcomes that they would like to
achieve. When determining outcomes it is important to consider not only the goals and interests
of the program’s sponsors, but also the desires and needs as seen from the perspective of poten-
tial participants. Appropriate programs should be keyed to the assets and risk and protective
factors that are relevant to the target audience.
Questions to consider:
• What are your goals and objectives for implementing this program? How well does the
program address them?
• What are the goals and objectives of the participants? How well does the program ad-
dress them?
• Does the program address the risk and protective factors most relevant to the sponsor’s
and participants’ goals? Does it build assets and resources in those areas that are most
important?
• Is the program of sufficient length and intensity to meet the goals of the facilitator
and/or participants?
There should be sufficient resources and time available to implement the program.
Some evidence-based programs are fairly expensive to implement, requiring many resources
and a significant time commitment. For example, they may require that facilitators attend multi-
day trainings offered by the program’s developers or that the program be facilitated by profes-
sionals with particular qualifications. Program sponsors need to assess whether they have the
human and financial resources that a program demands.
Questions to consider:
• Are there sufficient financial resources to implement the program?
• Is special training required of program facilitators? How accessible and affordable are
materials and training?
• Does the sponsoring organization(s) have staff members who are willing to make the
time commitment? Do the staff members have the skills and experience needed to im-
plement the program?
• Can the program be implemented within the time available?
What Works, Wisconsin Appendix D
The program’s assumptions and activities should be consistent with the values and cultural practices of the tar-
get audience.
Taking into account the values and culture of the intended audience is critically important.
Some programs are intentionally designed for certain populations or cultural groups. Others are
more generic and designed for universal audiences. It’s important to consider whether particular
programs are compatible with particular groups.
Questions to consider:
• Does the program take into account the class, cultural and historical backgrounds of the
participants?
• Are the outcomes and practices consistent with the values and norms of the target audi-
ence?
• Does the program take into account developmental, gender and individual differences of
both adults and children?
• Are the assumptions made about human nature, development and how people learn and
change consistent with the cultural beliefs of the target audience?
The program should be flexible enough to be adapted to the local setting and situation.
In order for an evidence-based program to be effective, it needs to be implemented in a way
that is similar to its original, documented design. Most programs have well-specified program
components that should be implemented with close correspondence to the original model. This is
known as program fidelity. On the other hand, in order to meet local needs and promote pro-
gram ownership, it may be necessary to adapt a program to the local conditions where it will be
delivered. This is known as program adaptation. Depending on the design, some programs are
more easily adapted.
Questions to consider:
• How rigid or flexible is the program?
• Can the program be adapted to the needs or culture of your audience?
• Is the program’s designer available to assist with local adaptation?
The program should be found effective for the specific population(s) with which you are working.
In order for a program to be deemed evidence-based, it must go through a series of rigorous
evaluations and meet certain standards. Even so, most evidence-based programs have only been
evaluated with a limited number of populations and under a relatively narrow range of condi-
tions. While it is likely that most well-established evidence-based programs will be effective and
appropriate for a range of audiences and situations, they will not be suitable or effective for every
audience or situation. When considering the merits of a program for a particular setting, it is im-
portant to examine whether there is evidence that the program will be appropriate for the target
audience.
What Works, Wisconsin Appendix D
Questions to consider:
• Under what conditions has the program been found to be effective? Are these conditions
similar to yours?
• Has the program been evaluated with audiences similar to your target population, and
do the results indicate that the program is effective?
• Are participants similar to your target population satisfied with the program?
The program should have a reasonably high probability of being sustained in the future.
When planning a program it makes sense to think long-term and consider which programs
are most likely to be continued in the future. Programs that require significant external funding
are especially prone to abandonment after the funding runs out. Some programs are more readily
adopted by existing organizations and are easier to support over the long run.
Questions to consider:
• Will there be sufficient financial and human resources to continue the program in the fu-
ture?
• Is there enough local commitment to the program so that it will be continued in the fu-
ture, especially when external funding is no longer available?
Evidence-based intervention and services for high-risk
youth: a North American perspective on the challenges of
integration for policy, practice and research
James K. Whittaker
Charles O. Cressey Endowed Professor Emeritus, School of Social Work, University of Washington, Seattle,
Washington, USA
A B S T R AC T
This paper explores the cross-national challenges of integrating
evidence-based interventions into existing services for high-resource-
using children and youth. Using several North American model
programme exemplars that have demonstrated efficacy, the paper
explores multiple challenges confronting policy-makers, evaluation
researchers and practitioners who seek to enhance outcomes for
troubled children and youth and improve overall service effective-
ness. The paper concludes with practical implications for youth and
family professionals, researchers, service agencies and policy–makers,
with particular emphasis on possibilities for cross-national
collaboration.
Correspondence:
James K. Whittaker,
School of Social Work,
University of Washington,
4101 Fifteenth Avenue NE,
Seattle, WA 98105-6299,
USA
E-mail: jimw@u.washington.edu
Keywords: children in need (services
for), evidence-based practice,
research in practice, therapeutic
social work
Accepted for publication: January
2009
I N T R O D U C T I O N
Across many national boundaries and within multiple
service contexts – juvenile justice, child mental and
child welfare – there is a growing concern about a
proportionately small number of multiply challenged
children and youth who consume a disproportionate
share of service resources, professional time and public
attention. While accurate, empirically validated popu-
lation estimates and descriptions remain elusive. The
consensus of many international youth and family
researchers, including those reported by McAuley
and Davis (2009) (UK), Pecora et al. (2009a) (US)
and Egelund and Lausten (2009) (Denmark) in this
present volume seems to be that some combination of
externalizing, ‘acting-out’ behaviour, problems with
substance abuse, identified and often untreated mental
health problems, experience with trauma and challeng-
ing familial and neighbourhood factors are often, and
in various combinations, manifest in the population of
children and youth most challenging to serve. Many of
these find their way into intensive out-of-home care
services, and Thoburn (2007) provides a useful
window into the out-of-home care status of children in
14 countries and offers useful observations on
improvements in collecting administrative data for
child and family services to inform both policy and
practice. Others call for a critical re-examination of the
present status of ‘placement’ as a central fulcrum
in child and family services policy and practice
(Whittaker & Maluccio 2002).
A sense of urgency is conveyed by the fact that
many child and youth clients of ‘deep-end’, restrictive
(out-of-home) services disproportionately represent
underserved and often socially excluded families and
communities of colour, and pose additional challenges
in service planning around the cultural compatibility
of proffered interventions (Blasé & Fixsen 2003;
Barbarin et al. 2004; Miranda et al. 2005). Important
work in this area includes ethnic and cultural
Author note: Portions of this paper in earlier form were
presented by the author at the 8th and 10th annual EUSARF
International Conferences at the University of Leuven,
Belgium, 9–11 April 2003 and the University of Padova,
Italy, 26–29 March 2008.
doi:10.1111/j.1365-2206.2009.00621.x
166 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd
mailto:jimw@u.washington.edu
variations on known effective practices. Lau (2006),
for example, offers a nuanced and sensitive treatment
of actual and potential adaptations in existing parent
training models. A basic concern with questions of
equity and social justice, coupled with a growing scep-
ticism about the efficacy of traditional residential,
‘place-based’ services, has heightened the search for
more preventive, family- and community-based, cul-
turally congruent service alternatives. All of this is set
against a backdrop of concern about the state’s ability
to provide effective parenting oversight and support
for children in care, as well as those who remain with
their families (Bullock et al. 2006). Fortunately, this
search is occurring at a time when researchers in many
countries are shedding light on mechanisms of risk
and resilience (Sameroff & Gutman 2004), change
processes involved in effective interventions (Biehal
2008) and the challenges faced by parents in multiply
stressed environments (Ghate & Hazel 2002; Ghate
et al. 2008) that are rich in their potential for contri-
butions to intervention design and evidence-informed
practice.
The primary purpose of this paper is to examine
some of the challenges and opportunities in incorpo-
rating evidence-based strategies and interventions
into existing service systems to better meet the needs
of high-resource-using children and youth. The
growing corpus of empirical research on promising
treatment strategies offers, if not clear-cut prescrip-
tions, then rich implications for future policy initia-
tives and service experiments.
Indeed, the pursuit of evidence-based practice, in
its many forms, increasingly attracts the attention of
those who plan, deliver and evaluate critical treatment
and rehabilitative services for vulnerable children and
their families across national boundaries and regions.
While definitions of ‘evidence-based practice’ empha-
size different dimensions of that construct, the
common themes of bringing ‘science-to-service’, and
its reciprocal ‘service-to-science’, are increasingly
evident in the child, youth and family services systems
in many European countries and North America, as
well as elsewhere. Simultaneously, reform efforts in
the USA and many European countries press for
community-based, family-oriented, non-residential
alternatives to traditional residential care and treat-
ment programmes for acting-out children and youth
with identified mental health problems (Chamberlain
2003; Weisz & Gray 2008). However, the impulse for
service reform and the availability of at least some
empirically validated model interventions do not of
themselves constitute a sufficient basis for system
reform, but instead serve to illuminate some of the
many fault lines that exist in the child and family
services field:
• The continuing tensions between ‘front-end’, pre-
ventive services and ‘deep-end’ highly intensive
treatment services and the unhelpful dichotomies
these tend to create and perpetuate
• The tensions between a widely shared desire to
adopt more evidence-based practices and the genu-
inely felt resistances to these, particularly when they
are used in a rigid fashion that requires strict adher-
ence to established protocols with little opportunity
for experimentation, customization or practitioner
discretion. For example, as one family support
researcher recently observed, we need much more
fine-grained analyses of the actual lived experience
of client families with the services offered to them
(S. P. Kemp 2008, personal communication). Such
analyses will almost certainly involved a ‘mixed-
methods’ approach using qualitative measures and
methods to augment quantitative studies
• The tension, as manifested in North America and
elsewhere between evidence-based and culturally
competent practices, reflects, among other things,
antagonism towards certain practice strategies
based on perceptions of the under-representation of
ethnic minorities in the study samples on which
certain models have been validated
As model programmes proliferate and are increas-
ingly removed from the particular political and cul-
tural niches within which they were developed, we
would do well to heed the cautions offered by Munro
et al. (2005) that researchers, planners and youth and
family practitioners are at a moment in time when
cross-national perspectives are critical in helping iden-
tify new ways of both framing problems and shaping
service solutions. Cross-national dialogue can help in
identifying different formats for collecting, analysing
and utilizing routinely gathered client information,
analysing subtle local adaptations of internationally
recognized evidence-based services and examining
the effects of differing policy contexts on service
outcomes.
T H E Q U E S T F O R M O R E E F F E C T I V E
I N T E R V E N T I O N S
For the remainder of this paper, I wish to do three
things: (1) briefly identify where we are in our search
for effective (evidence-based) interventions; (2) assess
how we are doing in increasing their availability to
high-resource-using troubled youth and their families;
Evidence-based intervention for high-risk youth J K Whittaker
167 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd
and (3) identify some particular challenges faced by
the individual practitioner, the social agency and the
public policy context in furthering the shared goal of
improving outcomes, and thus life prospects for
troubled children. The author’s bias will soon be
readily apparent. First, as one who has spent a lifetime
trying to bring both the precision of research methods
and the richness of research findings to the ‘shop
floor’ of children’s agency practice, I am convinced
that the evidence-based practice movement will not
succeed until it is embraced by those closest to the
children: the child and youth care workers, the social
workers, teachers, family support workers and others
who, with parents, toil on the front lines of helping.
This is not in my view a one-way street – Science-to-
Service – but presumes a vital feedback loop from
Service-to-Science where the insights and hypotheses
of those most directly involved in interventions
(including parent and child consumers) inform and
improve successive generations of applied research
studies. Second, I readily acknowledge the North
American bias apparent in many of my examples – I
write of what I know best – while recognizing a deeply
felt need in my country for European and other cross-
national perspectives if we are ever to achieve success
with our internal efforts at improving outcomes.
The search for evidence-based practices with chil-
dren and families is now well underway on both sides
of the Atlantic. Kazdin and Weisz (2003), Weisz
(2004), Burns and Hoagwood (2002), Macdonald
(2001), Pecora et al. (2009b) and McAuley et al.
(2006) survey effective interventions in child welfare
and child mental health services, as well as review
current research on service populations that will
inform the creation of novel interventions.
The simple, nominal definition of evidence-based
practice offered by Professor Geraldine MacDonald of
Queen’s University in Belfast provides a useful start-
ing point:
Evidence-based practice indicates an approach to decision-
making which is transparent, accountable and based on careful
consideration of the most compelling evidence we have about
the effects of particular interventions on the welfare of indi-
viduals, groups and communities. (MacDonald 2001, p. xviii)
It is clear that debates about what constitutes the
sufficiency and quality of evidence – where to set the
bar for rigour, how to distinguish evidence-based vs.
evidence-informed practice – continue apace both in
academic and practitioner discourse even as the
evidence-based practice movement as a whole contin-
ues to raise its profile in policy and services. These
competing definitions and nuances are, in toto, a sign
of health as they simply serve to underscore one or
another aspect of what is emerging as a more fulsome
understanding of what evidence-based practice con-
sists of. These aspects include, but are not limited to:
• a dual focus on aetiology and outcomes
• the incorporation of ethics and values as key com-
ponents
• the development of a collaborative process with
affected client groups
• a commitment to transparency in processes and
accountability
Many practitioners and practice researchers have
participated in the work of international groups such as
the Campbell and Cochrane Collaborations (Littell
2008) – originating in the health field – that attempt to
sift, sort and categorize the state of the evidence around
particular illnesses, socio-behavioural problems or
social welfare concerns. Many have also experienced –
closer to home – the increasing impact of national, state
and regional initiatives designed to increase the content
of proven, efficacious practices into child, youth and
family service systems. Such initiatives typically use
two strategies, often in combination:
Positive Reinforcement: e.g. ‘Laying Flowers Along Certain
Pathways’ by encouraging adoption of selected efficacious
model interventions. (One notes in passing that ‘efficacy’ of a
given intervention often increases in proportion to the dis-
tance from its country of origin!)
Coercion: e.g. Penalizing a programme, agency or practitioner
whose interventions do not reflect a sufficient quantity of
evidence-based practice according to an agreed-upon time
schedule. In the USA, this typically means that a practitioner
or service agency follows a prescribed protocol for interven-
tion or risks losing reimbursement for services rendered.
M OV I N G F R O M
‘ E F F I C AC Y- T O – E F F E C T I V E N E S S ’
In the USA at the moment, there is growing respect
for the complexities involved in moving from pilot
demonstrations of effective child, youth and family
interventions to broad-scale application: i.e. moving
from ‘efficacy’ to ‘effectiveness’ (Jensen et al. 2005;
Weisz & Gray 2008). What these terms signify are:
1. That individual investigators can demonstrate sig-
nificant results for novel treatments over standard (or
traditional) services through carefully controlled, rig-
orously conducted studies often including random-
ized controlled trials: the ‘gold standard’ of clinical
research. That is, they can demonstrate efficacy.
Evidence-based intervention for high-risk youth J K Whittaker
168 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd
2. Yet, these impressive results do not, on close
examination, appear to influence what might be
thought of as routine, day-to-day practice as con-
ducted in more familiar agency settings. Thus, the
evidence-based practice movement, while demon-
strating efficacy, cannot as yet demonstrate overall
effectiveness.
What explains this disconnect? Lisbeth Schorr, an
astute analyst of child and family services innovation,
sums it up succinctly: ‘Successful programs’, she says,
‘do not contain the seeds of their own replication’
(Schorr 1993, quoted in Fixsen et al. 2005).
Thus, if we are truly interested in effectiveness – i.e.
achieving wide-scale adoption of proven efficacious
interventions, we need to look beyond efficacy studies:
(1) to those contextual elements that influence prac-
tice decisions and client outcomes (Kemp et al. 1997);
and (2) to a different kind of research undertaking
that focuses directly on the processes involved in suc-
cessful adoption of proven efficacious interventions
(Weisz & Gray 2008).
John Weisz, one of the nation’s leading research
analysts in child mental health and a professor of
psychology at Harvard University as well as President
of the Judge Baker Children’s Center in Boston,
points the way forward on what is needed to ultimately
resolve the efficacy/effectiveness challenge:
A very important focus for the next stage of research on
interventions for children will be the effective implementation
of evidence-based practices by practitioners in service settings.
This will require an active collaboration between the research-
ers who develop and test interventions and the clinical, child
welfare, and education professionals who serve children and
families. (J.R. Weisz 2008, personal communication)
E X P L O R I N G T H E L A N D S C A P E O F
E V I D E N C E – B A S E D S E R V I C E S F O R
H I G H – R I S K YO U T H
Let us proceed, then, by exploring the context within
which evidence-based services are nested. Here, we
find some common and proximate elements familiar
to all who labour in the child and family services field,
as well as a few more distal forces that, nonetheless,
have a potential for considerable impact on the
identification, validation and eventual integration of
evidence-based practices. I will refer, briefly, to more
or less typical examples from within the US context.
Model intervention programmes
For purposes of illustration, I offer three interventions
that have received considerable attention in children’s
mental health services in the USA, and which have
been the objects of numerous community replications
and research study both in North America and else-
where (Whittaker 2005). These include:
• Multisystemic Therapy (MST), developed principally
by Dr Scott Henggeler, a psychologist now at the
Department of Psychiatry and Behavioral Sciences,
Medical University of South Carolina (Henggeler
et al. 1998; Schoenwald & Rowland 2002;
Henggeler & Lee 2003). http://www.mstservices.com
• Treatment Foster Care (MTFC), developed in several
clinical/research teams in the USA and represented
here by the model (Multi-dimensional Treatment
Foster Care) principally developed by Dr Patricia
Chamberlain and colleagues at the Oregon Social
Learning Center – a highly influential applied
behaviour analysis developmental research centre –
one of whose founding members is Dr Gerald
Patterson (Chamberlain & Reid 1998; Chamberlain
2002, 2003). http://www.MTFC.com
• Wraparound Treatment, a novel, team-oriented,
community-centred intervention developed by a
variety of individuals including the late Dr
John Burchard, formerly Professor of Clinical
Psychology at the University of Vermont, John Van
Den Berg, Carl Dennis and others beginning
in the early 1980s (Burns & Goldman 1999;
Burchard et al. 2002). http://www.rtc.pdx.edu/
PDF/PhaseActivWAProcess
[While space does not permit in depth analysis here,
the interested reader is directed to the previously cited
references, as well as to the web sites for each of these
three models that include multiple references to com-
pleted and in-progress research and demonstration
efforts, as well as specifics on programme principles
and components. A variation of the of the MTFC
model designed for younger children in regular foster
care is described in this present volume by Price et al.
(2009)].
These three interventions are specifically designed
to provide alternative pathways for children who
otherwise would be headed into more costly and
restrictive residential provision. Dr Barbara Burns,
Professor of Psychology at Duke University in North
Carolina and a principal author of the children’s
mental health section of our latest Surgeon General’s
Report on Mental Health (US Department of Health
and Human Services 1999) provides a succinct ratio-
nale for why this is warranted:
The most critical question for the future is, what will it take
to convince payers, public and private, to support the
Evidence-based intervention for high-risk youth J K Whittaker
169 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd
http://www.mstservices.com
http://www.MTFC.com
http://www.rtc.pdx.edu
interventions that are backed up by evidence about improved
outcomes? Assuming that the pool of dollars available for
mental health treatment will not increase, it will be necessary
to shift resources away from institutional care (which lacks
evidence of effectiveness) toward community alternatives.
This will require a reduction in funds allocated to institu-
tional care, where a significant portion of the child mental
health money is still being spent. (Burns & Hoagwood 2002,
p. 13)
While reviews of residential care in both the UK
(Sinclair 2006) and the USA (Whittaker 2006)
confirm a move away from residential services,
recent comparative international contributions have
urged critical re-examination of the multiple varieties
of residential service (Courtney & Ivaniec 2009) to
meet the needs of at least some high-resource-
using youth. In part, this sentiment reflects the fact
that theory and model development, particularly in
the arena of intensive residential services has lan-
guished as development of comparable family-
centred services has flourished. Some have urged the
development of a conceptual schema for intensive
services – e.g. the ‘prosthetic environment’ – which
transects more traditional residential, family and
community boundaries is strengths-oriented and
incorporates educational, socialization and family
support services along with intensive treatment
(Whittaker 2005).
In focusing here on a few programme models spe-
cifically designed to serve as alternatives to residen-
tial care and treatment, and other forms of intensive
out-of-home service, one must acknowledge omis-
sion of a great deal of promising, empirically based
work that is presently being done with a wide range
of family-, school- and community-centred interven-
tions that is both more preventive in its focus and
appropriate for a much wider population of children
and families than space allows us to examine here.
See, for example, Carolyn Webster Stratton’s Incred-
ible Years Program (Beauchaine et al. 2005) and the
work of many others whose contributions in such
areas as family support illuminates a segment of ser-
vices more preventive in focus (Kemp et al. 2005;
Lightburn & Sessions 2006) and the contribution of
Jackson et al. (2009).
What, then, are the similarities and differences
of these three promising interventions? A recent
review (Burns & Hoagwood 2002) yields the
following:
1. All three interventions adhere to ‘systems of care’
values: The ‘systems of care’ framework derives from
both our National Institute of Mental Health and
private foundation initiatives in the 1980s, and is
defined as:
A comprehensive spectrum of mental health and other neces-
sary services which are organized into a coordinated network
to meet the multiple and changing needs of children and
adolescents with severe emotional disturbances and their
families. (Stroul & Friedman 1986, p. xx)
The system of care thus defined is based on three main ele-
ments. First, the mental health service system efforts are
driven by the needs and preferences of the child & family and
are addressed by a strengths-based approach. Second, the
locus and management of services occur within a multi-
agency collaborative environment grounded in a strong com-
munity base. Third, the services offered, the agencies
participating and programs generated are responsive to cul-
tural context and characteristics. [Though, as noted, this
remains a contested area with respect to some communities of
color.] (Burns & Hoagwood 2002, p. 19)
2. All three interventions are delivered in a commu-
nity – home, school, neighbourhood – context as
opposed to an office
3. All have operated in multiple service sectors:
mental health, juvenile justice, child welfare
4. All were developed and evaluated in ‘real world’
community settings, thus enhancing external validity
5. All show preference for the model treatment con-
dition in multiple randomized controlled trials
6. All lay claim to being less expensive to provide than
institutional care (Burns & Hoagwood 2002, p. 7).
Differences of course exist. For example, both MST
and MTFC possess a higher degree of specificity with
respect to intervention components than does wrap-
around. As of this writing, MST has perhaps the
strongest evidentiary base, particularly in clinical trials
showing positive effects, though some recent reviews,
including one by Prof. Julia Littell of Bryn Mawr
University in Pennsylvania conducted for the Camp-
bell Collaboration, have raised critical questions about
the evidence base offered in support of MST (Littell
2005, 2008). Finally, from a staffing perspective, MST
appears to make higher use of master’s-level-trained
professionals in service delivery than either MTFC or
wraparound.
To these three model programmes, we must of
course add numerous other evidence-based treatment
techniques targeted to specific conditions and prob-
lems, as reflected in recent reviews by Kazdin and
Weisz (2003), Weisz (2004) and Chorpita et al.
(2007). These model intervention programmes do not
of course exist in a vacuum, but both influence and are
influenced by a host of other elements in a typical state
or regional context in the USA.
Evidence-based intervention for high-risk youth J K Whittaker
170 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd
P U B L I C , V O L U N TA R Y A N D P R O P R I E TA R Y
S E R V I C E P R OV I D E R S
Model programmes such as MST, MTFC and wrap-
around are typically adopted by some segment of the
mixed system of service agencies (Public/Voluntary/
Proprietary) that make up the delivery system in a
given state, county or municipality. Public service pro-
viders are typically service funders as well, creating in
the view of some voluntary agencies an unequal influ-
ence in terms of what particular models are selected
for adoption, as well as on the masking of true admin-
istrative costs of programme implementation, given
the public sector’s economies of scale and presumed
ability to mask start-up costs. Given the wide varia-
tions in state and county service systems within the
USA, there are some anecdotal reports of the ten-
dency of certain model programmes to bend and
shape themselves into a widely varying array of
funding arrangements (referred to as ‘pretzelling’) in
order to gain a foothold and a leverage in a given
public system (K. Blasé 2007, personal communica-
tion) with the result that local service providers may
be held to similar outcome and process standards
while enjoying widely varying reimbursements to
support their efforts.
N AT I O N A L , R E G I O N A L A N D L O C A L
R E S E A R C H C E N T E R S A N D R E S O U R C E
N E T W O R K S
In addition to evidence-based programme models
that typically have their own internal capacity for pro-
gramme development, marketing, training, evaluation
and dissemination, a wide variety of university and
institute-based resource networks and research centres
play an increasingly important role in the promotion
of evidence-based programmes and practices. For
example, the National Implementation Research
Network (NIRN) was begun at the University of South
Florida as part of a larger effort to bring science-based
information to the forefront of child mental health
practice. Recently relocated to the University of
North Carolina, NIRN has done significant work in
documenting national, state and regional capacity to
support model programme development, and has
provided consultation to individual states and organi-
zations on effective strategies for integrating evidence-
based practices into the fabric of existing services
(Fixsen et al. 2005). For more information, see: http://
www.fpg.unc.edu/~NIRN/. The California Evidence-
Based Clearinghouse for Child Welfare Practice is
funded by the California Department of Social Ser-
vices, Office of Child Abuse Prevention and guided by
a state advisory committee and a National Scientific
Panel. The Clearinghouse provides guidance on
selected evidence-based practices in simple straightfor-
ward formats, reducing the consumer’s need to
conduct literature searches, review extensive literature
or understand and critique research methodology
(http://www.cachildwelfareclearinghouse.org/). The
Clearinghouse has developed a six-tiered schema for
sorting out promising programmes ranging from
‘Well-Supported – Effective Practice’ to ‘Concerning
Practice’ (e.g. shows negative effects on clients and/or
potential for harm).
A legislatively generated state institute, the
Washington State Institute on Public Policy (WSIP)
was created by the Washington state legislature to
conduct cost/benefit and a range of other studies on a
variety of classes of intervention, including child
welfare and early intervention (http://www.wsipp.
wa.gov/board.asp). Its generally thorough and well-
executed analyses have achieved wide dissemination
beyond the region and are frequently cited by model
programme developers as confirmation of their effec-
tiveness. Methodological concerns have recently been
raised about the general quality of intervention
research reviews (Littell 2005, 2008), including those
generated by WSIP, and within local practice commu-
nities, one hears anecdotally some concerns about the
potential for overly concrete inferences by legislative
bodies and funding sources whose attention may
extend only to the executive summary section of
detailed reviews of model programmes and not to the
caveats and nuances contained in their appendices
and footnotes.
Beyond these particular exemplars, there are a wide
variety of government-, university- and institute-
based research centres and clearinghouses devoted to
the identification, review, evaluation and promotion
of evidence-based practices. Such centres are not
typically coordinated, resulting oftentimes in an over-
load of information for busy practitioners desirous of
identifying the most appropriate interventions for
troubled youth and their families. The problem is
intensified as estimates place the number of docu-
mented treatments for children and adolescents in
excess of 500 (Kazdin 2000). Here, the work of Dr
Bruce Chorpita at the University of Hawaii offers at
least a partial solution. For a number of years, Chor-
pita’s research team has been refining a ‘common
elements’ approach to identified evidence-based
treatments and then matching these with identified
Evidence-based intervention for high-risk youth J K Whittaker
171 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd
http://www.fpg.unc.edu/~NIRN
http://www.fpg.unc.edu/~NIRN
http://www.cachildwelfareclearinghouse.org
http://www.wsipp.wa.gov/board.asp
http://www.wsipp.wa.gov/board.asp
problem clusters and characteristics of youth and
families in service systems. The model’s particular
focus on the practitioner’s adoption of discrete strat-
egies, as opposed to whole-cloth approaches, is
directly addressed to one of the major identified bar-
riers to the implementation of evidence-based prac-
tice: the resistance to treatment manuals (Chorpita
et al. 2007). In a related area, the empirical research
of Professor Charles Glisson of the University of Ten-
nessee and colleagues sheds important light on the
organizational factors that may impede or enhance
the uptake of evidence-based practices in service set-
tings: e.g. organizational structure, organizational
culture and organizational climate (Glisson et al.
2008).
J U D I C I A L A N D L E G I S L AT I V E I N I T I AT I V E S
Vocal community advocacy calling attention to service
inadequacies and lacunae – for example, failure to
meet the mental health needs of children in the state
foster care system or excessive numbers of placement
changes – frequently end up in the court system. The
resultant settlements, or ‘consent decrees’, can exert
considerable direct and indirect pressure on the
service system to adopt particular models of evidence-
based practice as a remedy to the perceived problem.
In addition, within an individual state or jurisdiction,
there are not infrequently legislative initiatives
designed to promote certain evidence-based practices,
as well as initiatives generated from within the public
service agency itself. Taken together with the already
identified promotional efforts of model programme
developers, sometimes augmented by the largesse of
voluntary foundations that seek to promote particular
strategies for service improvement, the resulting pres-
sure for individual practitioners and voluntary service
agencies to follow certain prescribed pathways to
practice can be intense.
E V I D E N C E – B A S E D P R AC T I C E : M E E T I N G
T H E C H A L L E N G E O F I M P L E M E N TAT I O N
For each of the features of the evidence-based practice
landscape – model programmes, public and voluntary
service providers, individual youth and family practi-
tioners, research centres and clearinghouses, legisla-
tive and judicial bodies and client communities – there
are challenges to achieving the generally agreed-upon
goal of improving outcomes for high-resource-using
youth and their families through the adoption of
proven, efficacious practices. While these challenges
vary depending on the point one occupies in the
overall landscape of evidence-based practice, there
appears to be a growing consensus in the USA for a
far more intensive focus on what some have termed
‘implementation science’:
. . . the scientific study of methods to promote the systematic
uptake of clinical research findings and other evidence-based
practices into routine practice. (Implementation Science: UK:
on-line journal)
Thus, while different in their focus: (1) the previ-
ously cited efforts of NIRN to identify effective path-
ways for the integration of evidence-based practices
into existing service systems (Fixsen et al. 2005); (2)
the plea from research scholars like Julia Littell (2008)
and others to bring more rigour, precision and sys-
tematization to the scientific review processes for
evidence-based approaches; and (3) the numerous
contributions of senior research analysts like John
Weisz and others (Weisz & Gray 2008), directed
towards identifying pathways for bringing practitio-
ners and researchers into a closer working relation-
ship, are best viewed as part of a unified effort. There
is, I believe, a growing awareness that integration of
proven efficacious practices in youth and family work
will happen only when there is a fully functioning
infrastructure to support desired changes and various
individual actors see their ‘part’ in relational to the
‘whole’.
Thus, for model programme developers, there is the
critical task of identifying what are the active ingredi-
ents in their interventions. What are the necessary and
what are the sufficient components in a service unit of
MST, MTFC or wraparound? Despite the fact that
raising the question of ‘active ingredients’ leads one,
ineluctably, to what noted child psychiatry researcher
Peter Jensen calls ‘the soft underbelly’ of evidence-
based treatment, it is an area of critical importance
for future research (Jensen et al. 2005). The costing
and ‘scaling-up’ implications alone of adding even a
modest increment of evidence-based practice to exist-
ing services warrants seeking answers to the question:
‘How much of what is enough’? There has been an
understandable resistance on the part of many model
programme developers to disaggregate their interven-
tions for fear of compromising treatment integrity,
and thus weakening outcomes. That said, it is heart-
ening to note the flexibility of some models to cus-
tomize their interventions to fit the needs of particular
service populations and environmental niches. The
previously cited modification of the MTFC pro-
gramme reported elsewhere in this volume offers one
Evidence-based intervention for high-risk youth J K Whittaker
172 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd
such excellent example (Price et al. 2009). We must
find ways to hold model programme developers harm-
less for their results if they are willing to experiment
with modifications of their ‘packages of service’ to
address particular needs.
From the perspective of the individual service
agency – whether it is a voluntary body, local authority
or a large, public bureaucracy – a key question vis-à-
vis the adoption of exemplary evidence-based pro-
gramme models concerns the basic strategy for
implementation: is it to be additive or integrative?
The additive approach that appears to prevail in
many sectors of service in the present US context
means that service agencies adopt one model pro-
gramme at a time, adhering strictly to the interven-
tion, assessment, training and evaluation protocols of
the developer. This is meant to insure model fidelity
and programme (treatment) integrity and to prevent
what has been called ‘program contamination’. The
result, in the author’s view, can lead to an encapsula-
tion of discrete programmes – each with its identified
staff and protocols for assessment and intervention –
within a single agency structure. This results in fewer
opportunities for cross-fertilization (e.g. common, or
cross-training) and is silent on the preferred order of
implementation: e.g. does it make a difference which
model is adopted first? Moreover, the administrative
complexities involved in managing multiple discrete
programmes in a single agency can be considerable,
particularly in smaller units with limited supervisory
resources.
An alternate, or what might be termed an integra-
tive approach, would seek to identify common ele-
ments across successful model programmes and train
towards those. The previously cited work of Chorpita
et al. (2007) provides a potentially valuable founda-
tion to such an approach. A slight variation on the
integrative approach would be to identify a common
platform of foundational knowledge and skill – e.g.
around client engagement – or ‘therapeutic alliance’,
the present legatee of the old concept of ‘relation-
ship’ (Rauktis et al. 2005), and first establish that
core competency with all staff before moving to
incorporate the specific strategies and techniques
contained within successful model programmes. At
present, the enthusiasm for what might be called the
‘intervention-du-jour’ seems to suggest a continua-
tion of the additive, seriatim, approach at the
expense of the integrative.
Behind the specific issue of the preferred method
for adopting efficacious model interventions lies the
broader question of the service agency’s capacity to
integrate, utilize and generate practice-focused
research. Weisz and others have proposed models for
closer integration of research and practice (Hoagwood
et al. 2002; Weisz & Gray 2008) within the service
agency, but at the moment these are not widely in
evidence. Whittaker et al. (2006) offer a five-stage
model template for integrating evidence-based prac-
tice in a child mental health agency, including logic
modelling of existing programmes as a means of
developing a common language of service, including
implicit theories of change, selected evaluation activi-
ties, strategic researcher–agency staff partnerships and
benchmarking against practice models of national sig-
nificance. A barrier to building research capacity in
existing agencies is that present contracts are typically
tied to designated services, not to building an infra-
structure supportive of research.
Challenges for the service agency include data
management where the adoption of electronic records
lags in certain sectors and where many agencies lack
the capacity to systematically analyse routinely gath-
ered data at either the case or the aggregate level. As
noted, the proliferation of assessment and evaluation
measures – often tied to specific programmes – adds
complexity to the data management needs of the
service agency. In the critical arena of supervision, the
question arises of the adequacy of a single supervisor
to provide oversight and support to a staff operating
in widely disparate intervention models with their
differing change theories, assessment protocols,
outcome measures and time frames. In the related
arena of training, similar problems can be found,
including almost exclusive reliance on the use of
external (and often expensive) consultants during
the start-up phase of a model intervention with
unclear plans for transition of oversight to internal
agency staff. Moreover, the determinants for training
foci in some agencies remain strongly with worker
interests and are not necessarily related to client
characteristics. Of particular concern among many
smaller, voluntary service settings in the USA is the
factor of agency history. Many such agencies were
residential in their origins, typically following a
pathway from orphanage to treatment setting. Thus,
boards of governors and major donors may be more
oriented to place-based services and ‘bricks &
mortar’ than to community-based programme alter-
natives. A quote from a senior head in one such
agency captures the tension for those in leadership:
‘How do I insure that my program plan for the agency is
in synchrony with my business plan?’ (K. Scott 2002,
personal communication).
Evidence-based intervention for high-risk youth J K Whittaker
173 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd
S U M M A R Y A N D TA K E – AWAY M E S S AG E S
The growing corpus of research on evidence-based
approaches to work with troubled youth and their
families offers both hope and challenge. In the area of
alternatives to residential services for high-resource-
using youth, model programmes like MST, MTFC
and wraparound appear to hold much promise, but
their full-scale adoption into existing service systems
will require addressing a series of complex implemen-
tation challenges. Public sector children’s services
often work with families who are challenging to
engage and for whom permanency and continued
child safety remain as core service objectives. The
sensitive application of evidence-based programmes
and practices into the real world of contemporary
child and family practice must necessarily involve
parents, social workers, model developers and
researchers in bidirectional communication. All of
these efforts will benefit greatly from sustained and
multilevel cross-national collaboration. Of the several
US originated programme models identified in this
brief review, virtually all are intruding to some degree
on the work-plans of service planners, evaluation
researchers, supervisors and practitioners in Europe,
Australia and elsewhere. Similarly, interventions or
intervention components as varied as ‘Patch’ (geo-
graphically centred, generalist services) (Adams &
Krauth 1995) and ‘Family Group Conferencing’
(Pennell & Anderson 2005) have come to the USA
from the UK, New Zealand and elsewhere in recent
years. Since all of these ‘imports’ will likely undergo
modification and appear again as ‘exports’, it behoves
staff at all of the above levels to carefully track how
these novel interventions are being incorporated into
widely differing political, geographic, cultural and
organizational contexts. Fortunately, the wide avail-
ability of instantaneous, direct, point-to-point elec-
tronic communication and the increasing prominence
of cross-national journals, networks and conferences
make such communication more possible than ever.
From a research perspective, the widely varying envi-
ronments into which model programmes are being
introduced hold the distinct possibility for compara-
tive research, including natural experiments.
For youth and family practitioners, service agencies
and researchers and policy-makers, some concrete
take-aways include:
1. For practitioners
• Challenge the ‘conventional wisdom’ of practice
wherever it resides – including in your own per-
sonal theories of change: for example, ‘insight is a
requisite for behaviour change’; ‘longer service
produces better outcomes’.
• Seek out and read one up-to-date review of inter-
ventions most relevant to the children and youth
you presently work with. Discuss what you have
garnered with peers.
2. For researchers
• Focus on application to real-world practice in
your dissemination efforts: for example, ‘What are
the top five practice implications of your latest
study and where might these most usefully be dis-
seminated?’
• Seek practice-partners for agency-centred
research projects specifically focused on issues of
implementation of evidence-based practices.
3. For child and family service agencies
• Discover first what is working within the agency
and build on that as a foundation before purchas-
ing ‘off-the-shelf ’ models.
• Develop an internal capacity to systematically
analyse routinely gathered data at the case level
and aggregate level and ‘mine’ this information to
inform practice.
4. For the service system/policy level
• Here, and speaking from a parochial perspective,
with all of our resources in the USA, we are sorely
in need of a new structure or body within a state or
authority that ‘connects-the-dots’ between relevant
service policy, research and practice in support of
enhancing the implementation of evidence-based
practices to improve outcomes. While the title and
organizational form for such a body proves elusive
– clearinghouse? executive steering committee for
evidence-based programme improvement? – its key
function should be to focus laser-like attention on
the question of what is most important in
evidence-based practice implementation in a state,
local authority or region: What do we need to learn
over the next 12 months? How will we learn it?
How will we decide ‘what-trumps-what’: Cost?
Urgent service need? Level of evidence? Cultural
relevancy? Organizational compatibility? Special
opportunity to experiment with proven efficacious
model programmes? While the fruits of such a new
body would be experienced locally, one hopes that
its field of vision and, eventually, its impact would
extend cross-nationally.
In sum, evidence-based practice has added greatly to
the ‘tool kit’ of social services in the identification of
proven efficacious models of intensive intervention
such as those referenced earlier in this paper.That said,
the task of scaling-up these exemplars is proving to be
Evidence-based intervention for high-risk youth J K Whittaker
174 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd
complex and challenging, and will require both
focused attention on the multiple contextual elements
that impede and enhance the adoption of evidence-
based alternatives, as well as a critical re-examination
of existing biases – for example, the current and often
reflexive negative attitudes towards residential provi-
sion in any form – that underlay current services plan-
ning. Both of these conversations will be greatly
enhanced by multilevel, sustained and data-oriented
cross-national collaboration among practitioners,
service planners and researchers. Fortunately, through
technological innovations such as electronic commu-
nication, the means for such collaboration are close at
hand. High-resource-using youth and their families
presently in, or at risk of entering the intensive services
system, will be the ultimate beneficiaries of our efforts.
R E F E R E N C E S
Adams, P. & Krauth, K. (1995) Working with families and com-
munities: the patch approach. In: Reinventing Human Services:
Community and Family Centered Practice (eds P. Adams & K.
Nelson), pp. 87–109. Transaction-Aldine Publishers, Piscat-
away, NJ.
Barbarin, O., McCandies, T., Coleman, C. & Atkinson, T. (2004)
Ethnicity and culture. In: Intervention with Children and Fami-
lies: An Interdisciplinary Perspective (eds P. Allen-Meares &
M.W. Fraser), pp. 27–54. Pearson,
Boston, MA.
Beauchaine, T.P., Webster-Stratton, C. & Reid, M.J. (2005)
Mediators, moderators and predictors of one-year outcomes
among children treated for early onset conduct problems: a
latent growth curve analysis. Journal of Consulting and Clinical
Psychology, 73, 371–388.
Biehal, N. (2008) Preventive services for adolescents: exploring
the process of change. British Journal of Social Work, 38, 444–
461.
Blasé, K.A. & Fixsen, D.L. (2003) Evidence-based programs and
cultural competence (20pp). Available from Tampa, FL: Univer-
sity of South Florida, Louis de la Parte Florida Mental Health
Institute, The National Implementation Research Network.
Bullock, R., Courtney, M., Parker, R., Sinclair, I. & Thoburn, J.
(2006) Can the Corporate State Parent. Children and Youth
Services Review, 28, 1344–1358.
Burchard, J.D., Bruns, E.J. & Burchard, M.I. (2002) The wrap-
around approach. In: Community Treatment for Youth: Evidence-
based Interventions for Severe Emotional and Behavioral Disorders
(eds B.J. Burns & K. Hoagwood), pp. 69–91. Oxford Univer-
sity Press,
New York.
Burns, B.J. & Goldman, S.K. (1999) Promising practices in wrap-
around for children with serious emotional disturbance and their
families. Systems of Care: Promising Practices in Children’s
Mental Health, 1998 series, Volume IV. Washington, DC:
Center for Effective Collaboration and Practice, American
Institutes for Research.
Burns, B.J. & Hoagwood, K. (eds) (2002) Community Treatment
for Youth: Evidence-based Interventions for Severe Emotional and
Behavioral Disorders. Oxford University
Press, New York.
Chamberlain, P. (2002) Treatment foster care. In: Community
Treatment forYouth: Evidence-based Interventions for Severe Emo-
tional and Behavioral Disorders (eds K. Hoagwood & B.J.
Burns), pp. 117–139. Oxford University Press, New York.
Chamberlain, P. (2003) Treating Juvenile Offenders: Advances
Made through the Oregon Multidimensional Treatment Foster Care
model. American Psychological Association, Washington, DC.
Chamberlain, P. & Reid, J.B. (1998) Comparison of two com-
munity alternatives to incarceration for chronic juvenile
offenders. Journal of Consulting and Clinical Psychology, 66,
624–633.
Chorpita, B.F., Becker, K. & Daleiden, E.L. (2007) Understand-
ing the common elements of evidence-based practice: miscon-
ceptions and clinical examples. Journal of the American
Academy of Child and Adolescent Psychiatry, 46, 647–652.
Courtney, M.E. & Ivaniec, D. (eds) (2009) Residential Care of
Children: Comparative Perspectives. Oxford University Press,
New York and Oxford.
Egelund, T. & Lausten, M. (2009) Prevalence of mental health
problems among children placed in out-of-home care in
Denmark. Child & Family Social Work, 14, 156–165.
Fixsen, D.L., Naoom, S.F., Blasé, K.A., Friedman, R.M. &
Wallace, F. (2005) Implementation Research: A Synthesis of the
Literature. University of South Florida, Louis de la Parte
Florida Mental Health Institute, Tampa, FL. The National
Implementation Research Network (FMHI Publication
#231). For more information, Available at: http://www.
fpg.unc.edu/~NIRN/ (accessed 18 February 2009).
Ghate, D. & Hazel, N. (2002) Parenting in Poor Environments:
Stress, Support & Coping. Jessica Kingsley Pub
lishers,
London.
Ghate, D., Hauari, H., Hollingworth, K. & Lindfield, S. (2008)
Key Elements of Effective Practice in Parenting Support in theYouth
Justice Context: Source Document. Youth Justice Board, London.
Glisson, C., Landsverk, J., Schoenwald, S.K., Kelleher, K.,
Hoagwood, K.E., Mayberg, S. et al. (2008) Assessing the
organizational social context (OSC) of mental health services
for implementation research and practice. Administration and
Policy in Mental Health and Mental Health Services Research,
35, 98–113.
Henggeler, S.W. & Lee, T. (2003) Multisystemic treatment of
serious clinical problems. In: Evidence-based Psychotherapies for
Children and Adolescents (eds A.E. Kazdin & J.R. Weisz), pp.
301–322. The Guilford Press, New York.
Henggeler, S.W., Schoenwald, S.K., Borduin, C.M., Rowland,
M.D. & Cunningham, P.B. (1998) Multisystemic Treatment of
Antisocial Behavior in Children and Adolescents. Guilford Press,
New York.
Hoagwood, K., Burns, B.J. & Weisz, J.R. (2002) A profitable
conjunction: from science to service in children’s mental
health. In: Community Treatment for Youth: Evidence-based
Interventions for Severe Emotional and Behavioral Disorders (eds
B.J. Burns & K. Hoagwood), pp. 327–339. Oxford University
Press, New York.
Evidence-based intervention for high-risk youth J K Whittaker
175 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd
http://www
Jackson, A., Frederico, M., Tanti, C. & Black, C. (2009) Explor-
ing outcomes in a therapeutic service response to the emo-
tional and mental health needs of children who have
experienced abuse and neglect in Victoria, Australia. Child &
Family Social Work, 14, 198–212.
Jensen, P.J., Weersing, R., Hoagwood, K.E. & Goldman, E.
(2005) What is the Evidence for Evidence-Based Treatments?
A hard look at our soft underbelly. Mental Health Services
Research, 7, 53–74.
Kazdin, A.E. (2000) Psychotherapy for Children and Adolescents:
Directions for Research and Practice. Oxford University Press,
Oxford and New York.
Kazdin, A.E. & Weisz, J.R. (2003) Evidence-based psychotherapies
for children and adolescents. The Guilford Press, New York.
Kemp, S.P., Whittaker, J.K. & Tracy, E.M. (1997) Person-
environment Practice:The Social Ecology of Interpersonal Practice.
Transaction-Aldine Publishers,
Piscataway, NJ.
Kemp, S.P., Allan-Eckard, K., Ackroyd, A., Becker, M.F. &
Burke, T.K. (2005) Community family support meetings: con-
necting families, public child welfare and community
resources. In: Child Welfare for the 21’st Century: A Handbook of
Practice (eds G.P. Mallon & P.M. Hess), pp. 102–117. Colum-
bia University Press, New York.
Lau, A.S. (2006) Making the case for selected and directed
cultural adaptations of evidence-based treatments: examples
from parent training. Clinical Psychology: Science and Practice,
13, 295–310.
Lightburn, A. & Sessions, P. (eds) (2006) Handbook of
Community-Based Clinical Practice. Oxford University Press,
Oxford and New York.
Littell, J.H. (2005) Lessons from a systematic review of effects of
multisystemic therapy. Children and Youth Services Review, 27,
445–463.
Littell, J.H. (2008) Evidence-based or biased? The quality of
published reviews of evidence-based practices. Children and
Youth Services Review, 30, 1299–1317.
McAuley, C. & Davis, T. (2009) Emotional well-being and
mental health of looked after children in the UK. Child &
Family Social Work, 14, 147–155.
McAuley, C., Pecora, P.J. & Rose, W. (eds) (2006) Enhancing the
Well-being of Children and Families through Effective Interven-
tions: International Evidence for Practice. Jessica Kingsley Pub-
lishers, London.
Macdonald, G. (2001) Effective Interventions for Child Abuse and
Neglect. John Wiley & Sons Ltd., New York.
Miranda, J., Guillermo, B., Lau, A., Kohn, L., Wei-Chin, H. &
LaFromboise, T. (2005) State of the science on psychosocial
interventions for ethnic minorities. Annual Review of Clinical
Psychology, 1, 113–142.
Munro, E., Stein, M. & Ward, H. (2005) Comparing how dif-
ferent social, political and legal frameworks support or inhibit
transitions from public care to independence in Europe, Israel,
Canada and the United States. International Journal of Child
and Family Welfare, 8, 197–198.
Pecora, P.J., White, C.R., Jackson, L.J. & Wiggins, T. (2009a)
Mental health of current and former recipients of foster care:
a review of recent studies in the USA. Child & Family Social
Work, 14, 132–146.
Pecora, P.J., Whittaker, J.K., Maluccio, A.N., Barth, R.P. &
DePanfilis, D. (2009b) The ChildWelfare Challenge: Policy, Prac-
tice and Research, 3rd edn. Transaction-Aldine Publishers,
Piscataway, NJ.
Pennell, J. & Anderson, G.R. (2005) Widening the Circle: The
Practice and Evaluation of Family Group Conferencing with Chil-
dren,Youths and Their Families. NASW Press, Washington, DC.
Price, J.M., Chamberlain, P., Landsverk, J. & Reid, J. (2009)
KEEP foster parent training intervention: model description
and effectiveness. Child & Family Social Work, 14, 233–242.
Rauktis, M.E., De Andrade, V., Doucette, A.R., McDonough,
A. L. & Reinhart, S. (2005) Treatment foster care and rela-
tionships: understanding the role of therapeutic alliance
between youth and treatment parent. International Journal of
Child & Family Welfare, 8, 146–163.
Sameroff, A.J. & Gutman, L.M. (2004) Contributions of risk
research to the design of successful interventions. In: Interven-
tion with Children and Families: An Interdisciplinary Perspective
(eds P. Allen-Meares & M.W. Fraser), pp. 9–27. Pearson,
Boston, MA.
Schoenwald, S.K. & Rowland, M.D.v2002) Multisystemic
therapy. In: Community Treatment forYouth: Evidence-based Inter-
ventions for Severe Emotional and Behavioral Disorders (eds B.J.
Burns & K. Hoagwood), pp. 91–116. Oxford University Press,
New York.
Schorr, L.B. (1993) Effective Strategies for Increasing Social
Program Replication/Adaptation. Seminar on Effective Strategies
For Increasing Social Program Replication/Adaptation. National
Association of Social Workers, Washington, DC.
Sinclair, I. (2006) Residential care in the UK. In: Enhancing the
Well-being of Children and Families through Effective Interven-
tions: International Evidence for Practice (eds C. McAuley, P.J.
Pecora & W. Rose), pp. 203–216. Jessica Kingsley Publishers,
London.
Stroul, B.A. & Friedman, R.M. (1986) A System of Care for
Severely Emotionally Disturbed Children & Youth. CASSP Tech-
nical Assistance Center, Georgetown University, Child Devel-
opment Center, Washington, DC.
Thoburn, J. (2007) Globalization and Child Welfare: Some Lessons
from Cross-national Study of Children in Out-of-Home Care.
Social Work Monographs, University of East Anglia, Norwich,
England.
US Department of Health and Human Services (1999) Mental
Health: A Report of the Surgeon GENERAL. US Department of
Health and Human Services, Substance Abuse and Mental
Health Services Administration, National Institutes of Health,
National Institute of Mental Health, Rockville, MD.
Weisz, J.R. (2004) Psychotherapy for Children and Adolescents:
Evidence-based Treatments and Case Examples. Cambridge
University Press, Cambridge.
Weisz, J.R. & Gray, S. (2008) Evidence-based psychotherapy
for children and adolescents: data from the present and a
model for the future. Child and Adolescent Mental Health, 13,
54–65.
Evidence-based intervention for high-risk youth J K Whittaker
176 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd
Whittaker, J.K. (2005) Creating ‘prosthetic environments’ for
vulnerable children: emergent cross-national challenges for
traditional child & family services practice. In: In the Best
Interests of Children and Youth: International Perspectives (eds H.
Grietens, W. Hellinckx & L. Vandemeulebroecke), pp. 99–119.
Leuven University Press, Leuven.
Whittaker, J.K. (2006) Residential care in the US. In: Enhancing
the Well-being of Children and Families through Effective Interven-
tions: International Evidence for Practice (eds C. McAuley, P.J.
Pecora & W. Rose), pp. 217–227. Jessica Kingsley Publishers,
London.
Whittaker, J.K. & Maluccio, A.N. (2002) Re-thinking ‘child
placement’: a reflective essay. Social Service Review, 76, 108–
134.
Whittaker, J.K., Greene, K., Schubert, D., Blum, R., Cheng, K.,
Blum, K. et al. (2006) Integrating evidence-based practice in
the voluntary child mental health agency: a template for clini-
cal and organizational change. American Journal of Orthopsy-
chiatry, 76, 194–201.
Evidence-based intervention for high-risk youth J K Whittaker
177 Child and Family Social Work 2009, 14, pp 166–177 © 2009 Blackwell Publishing Ltd
