Methods for Successfully Adopting Evidence-Based Programs
Implementing Evidence-Based Practices and Measuring Results:
- Getting to Outcomes 2004: Promoting Accountability through Methods and Tools for Planning, Implementation, and Evaluation. Retrieved on March 14, 2007 from the Rand Corporation
- Basic Guide to Outcomes-Based Evaluation for Non-Profit Organizations with Very Limited Resources. Retrieved on March 15, 2007 from the Free Management Library
Logic Models: The Logic Model provides an effective method for charting progress from initial and short-term outcomes toward intermediate and long-term outcomes. A Logic Model focuses on identifying the logical links between the outcomes you desire, your program assumptions or theories, and your program strategies or services.
- Logic Model Builder for Child Abuse and Neglect Prevention/Family Support Programs. Retrieved on March 15, 2007 from the Child Welfare Information Gateway
- Program Planning and Development. Retrieved on March 15, 2007 from the University of Missouri Extension
- Program Development and Evaluation. Retrieved on March 15, 2007 from the University of Wisconsin
- Guidelines and Framework for Designing Basic Logic Model. Retrieved from the Free Management Library
How to Keep to the Fidelity of Evidence-Based Programs
What is Fidelity? - Implementing a model program as designed with the recommended techniques, in the recommended setting, to the recommended population, for the recommended time and duration.
Why is Fidelity important?
- Anticipated research-based outcomes are based on adherence to the program model and design.
- Participation can be impacted.
- Anticipated outcomes may be negatively impacted, or unanticipated outcomes may occur.
What are Fidelity criteria components?
- Written Curriculum
- Key Replication Components
- Standardized Training
- Defined Target Population
- Defined Setting
- Defined Number and Duration of Sessions.
Who Develops the Criteria?
- The developer of the evidence-based program usually provides the fidelity criteria. This information is often provided on the developer’s web site.
- Federal agencies that identify or endorse evidence-based programs may also have this information.
- What training requirements does my staff need?
- Is it one-time training or on-going training?
- Are there frequency/intensity/dosage requirements for implementation?
- Are there caseload limits for staff?
- Is there a curriculum manual that is used in association with this program?
- Is there technical assistance available to us while we implement the program?
- Are there fees associated with my agency using this program?
- Are there specified evaluation tools that we need to use?
Adaptation and Fidelity
Adaptation can be defined as changing an aspect of the evidence-based program:
- To be culturally relevant
- To meet community norms or standards
- To meet partner or key stakeholder expectations
- To better meet participant needs
- To adapt to facilitator/trainer knowledge and skills.
Research shows that adaptations are more effective when:
- Underlying program theory is understood;
- Core program components have been identified;
- Both the community and needs of the population have been carefully defined.
(Source: Achieving Outcomes: A Practitioner's Guide to Effective Prevention SAMHSA, 2002.
The ultimate goal is to maintain the basic integrity of a program while matching the innovation to the unique features of the setting and the preferences/reactions of the relevant population. (Jason, Durkal & Holton-Walker, 1984)
To Adapt or Not to Adapt?
The decision to adapt a program is difficult and certain steps are recommended:
- Familiarize yourself with the program
- Contact the program developer
- Check needs assessment data
- Identify adaptation needs
- Consult with key stakeholders
- Develop a plan.
Source: Achieving Outcomes: A Practitioner's Guide to Effective Prevention SAMHSA, 2002.
Why Self-Assess for Program Fidelity?
- In order to ensure the best program outcomes, it is important to assess that the program is being implemented as designed.
- To help determine whether the program you are implementing fits your program goals and target population.
- To ensure the program is being implemented as designed.
- Conduct random visits of sessions
- Attend facilitator training to verify training is consistent with the program design
- Document the length and duration of sessions
- Interview the participants or facilitators
- Monitor adherence to intervention as originally developed and evaluated
- Identify significant deviations from key components as they arise
- Correct deviations by providing:
- “Coaching”
- Helpful resources
- Booster training
Fidelity Resources: