When thinking about program process evaluation, most people focus on assessing the effectiveness and outcomes of their program. However, they may overlook evaluating how the program was delivered or implemented, which can significantly impact those outcomes. Not all evaluations center on outcomes; some, like formative or implementation evaluations, focus on understanding how a program is being carried out.
This two-part blog series will explore the 5 W’s of evaluating program implementation:
- Why it’s important to evaluate program processes
- Who should lead the evaluation
- What methods are best for assessing implementation
- Where the evaluation should take place
- When is the right time to conduct this type of assessment
In this first post, we’ll address the first two W’s, while the next blog will cover the remaining three.
WHY CONDUCT A PROCESS EVALUATION
Let’s start with the “why.” Evaluating program implementation helps organizations gain insights into how their programs are functioning. It also acts as an accountability tool, addressing important questions like:
- Is the program being executed as designed?
- Does the current approach maintain fidelity to the program model?
- Is the program implemented consistently across different sites and staff?
- What types of services are being delivered, and how often?
- Are the correct procedures being followed?
- Is the program effectively reaching its intended audience?
It’s crucial to define your goals for evaluating program implementation. For example, you should determine whether the program is being carried out as planned or confirm adherence to its model. Whatever your objectives, having a clear purpose and understanding of what you hope to learn will guide the process.
Currently, we’re partnering with the Wabash YMCA’s 21st Century Community Learning Center to assess their program delivery. Each center must work with an external evaluator to conduct an implementation assessment as part of state requirements. Here’s what we aim to uncover and why:
- Evaluate whether the program aligns with its intended design and meets state standards.
- Analyze attendance trends to understand the population being served.
- Use findings to inform and improve program strategies in future years.
WHO SHOULD CONDUCT YOUR PROCESS EVALUATION
When determining who will conduct your process evaluation, you have the option of either identifying an internal staff member (e.g., program manager or quality assurance) from your organization or hiring an external evaluator. Many organizations find that there are challenges with an internal team member: they may not be objective, they don’t have a fresh perspective, and they often have other job responsibilities beyond the evaluation.
For the reasons mentioned above, it is beneficial to have an external evaluator (like TCG!). An external evaluator will be able to assess the operations of your program from an unbiased lens. This is especially helpful if a program has multiple sites. An external evaluator can assess all sites/facilitators for consistency more objectively than a program staff member. (If you’re interested in learning more about how to evaluate multi-site programs, view our blog post here!).
In our evaluation project with the Wabash YMCA, the decision to conduct an evaluation with an external group was made by their funders. This decision ensures that the evaluation is high quality and objective.
The other three W’s will be discussed in a later blog post, so stay tuned! In the meantime, contact us today to learn more about our evaluation services!
