Evaluation is one of many services we provide for clients at Transform Consulting Group. Evaluation is important for any program because it helps organizations determine if a program is effective or if the change they set to accomplish actually occurred. In a previous blog we shared the 4 key steps to evaluation that we follow. These steps are (1) establish clear outcomes, (2) create or modify data tools and systems, (3) analyze the data, and (4) use data to make informed decisions.
Some of our clients offer their program 
services across multiple sites. This program model can be useful because it can save time, money and resources (i.e. hiring one director to manage multiple sites). As evaluators, we appreciate multi-site programs as well. Here are some reasons why multi-site evaluations are important and useful for evaluators:
- Increases knowledge about what works in different settings
- Allows for larger sample size when gathering data
- Creates opportunities for sharing experience and knowledge about what works
- Helps evaluators see sustainability on a wider scale
At TCG, we are currently evaluating a couple
21st Century Community Learning Centers (CCLC) Program grantees. The clients implement the program across multiple sites, usually different schools within a school district. We use the following tips to help us evaluate multiple sites for one program.
3 Tips to Evaluate Multi-Site Programs
- Observe each program site: Site observations help evaluators see how a program is implemented across each site. Is the approach and curriculum consistently followed? What is each site doing well compared to other sites? Where do improvements need to be made? Observations give evaluators important context and understanding when it is time to analyze the data. For the 21st CCLC program, the clients operate both summer, before and after school programs. We have built into our evaluation plan time to observe each site during the summer, at the beginning of the school year and the end of the school year.
- Collect data by site: Each site will have unique factors that may influence the program and overall data. When we are able to look at data by individual site, it will help inform recommendations for improvement and possible best practices. For example, we recently worked with a multi-site program to assess their workforce retention data. They identified several locations that had really high turnover and a few sites with lower turnover rates. By disaggregating the data by each site, we were equipped to inform some strategic decision-making.
- Solicit stakeholder feedback: Gathering input from key stakeholders is a critical component of our evaluation process (see these past blog posts: How to Engage Diverse Stakeholders in your Planning and 4 Tools to Collaborate with Key Stakeholders). For the 21st CCLC program, we sought feedback from program staff. We wanted to know if all staff understand the goals and outcomes of the program since it’s a new program for the organization and they have hired several new staff. If the responses differed, we can report back to the administrators that better messaging may be needed to ensure the program is being implemented effectively and meeting the state requirements.
Once data and information has been collected by site, the evaluator determines the best way to report the information. Depending on the audience for the evaluation report, the data may be presented differently. For an external audience (public, the funder, etc.), program summary results may be reported. For internal audiences (staff, board of directors, volunteers), it may be helpful to present the data at a disaggregated level by each site or location. There may be steps to take to better align sites to meet program goals and outcomes.
While there may be some additional steps needed to evaluate multi-site programs, our overall four-step approach to evaluation stays the same. Good evaluations often lead to recommendations for improvement. This is an opportunity to discuss data collected and implications for future programming, including ongoing program evaluation practices within the organization. If you like our approach, contact us today to learn about how we can help you with your next evaluation.
