When it comes to program evaluation, people often think of evaluating the effectiveness and outcomes of their program. They may not think about evaluating how the program was administered or delivered, which may affect the program outcomes. There are several types of valuable evaluations that do not focus on outcomes. One type of evaluation, called “process or formative evaluation”, assesses how a program is being implemented.
In this two part blog series, we are going to cover the 5 W’s of a Process Evaluation:
- Why conduct a process evaluation
- Who should conduct a process evaluation
- What methods to use to conduct a process evaluation
- Where to conduct a process evaluation
- When to conduct a process evaluation
In this first blog in the series we will cover the first two W’s. The next blog will discuss the other three.
WHY CONDUCT A PROCESS EVALUATION
Let’s start with the “why”. A process evaluation helps an organization better understand how their program is functioning and operating. Process evaluations also serve as an accountability measure and can answer key questions, such as:
- Is the program operating as it was designed and intended?
- Is the current implementation adhering to program fidelity?
- Is the program being implemented consistently across multiple sites and staff, if applicable?
- What type and frequency of services are provided?
- What program procedures are followed?
- Is the program serving its targeted population?
It is important to determine what you want to learn from your process evaluation. Maybe you want to assess if the program is being implemented as it was intended or you want to know if the program model is being followed. Whatever the reason, you want to be clear about why you are completing the process evaluation and what you hope to learn.
We are currently working with the Wabash YMCA’s 21st Century Community Learning Center to evaluate their program implementation. Each center is required to work with an external evaluator to conduct a process evaluation. Here is what we hope to learn and the why of this evaluation:
- The evaluation will assess if the program has been implemented as it was intended and if it is adhering to state standards;
- This evaluation will capture the population served through the assessment of attendance trends;
- The findings from the process evaluation will be used for program improvement in subsequent years.
WHO SHOULD CONDUCT YOUR PROCESS EVALUATION
When determining who will conduct your process evaluation, you have the option of either identifying an internal staff member (e.g., program manager or quality assurance) from your organization or hiring an external evaluator. Many organizations find that there are challenges with an internal team member: they may not be objective, they don’t have a fresh perspective, and they often have other job responsibilities beyond the evaluation.
For the reasons mentioned above, it is beneficial to have an external evaluator (like TCG!). An external evaluator will be able to assess the operations of your program from an unbiased lens. This is especially helpful if a program has multiple sites. An external evaluator can assess all sites/facilitators for consistency more objectively than a program staff member. (If you’re interested in learning more about how to evaluate multi-site programs, view our blog post here!).
In our evaluation project with the Wabash YMCA, the decision to conduct an evaluation with an external group was made by their funders. This decision ensures that the evaluation is high quality and objective.