Category Archives: Evaluation

Putting Equitable Evaluation into Practice

Evaluation

Evaluation ProcessDoes your evaluation process accurately represent the population you serve? The marginalized and less privileged? We shared in this blog that evaluation is a process that helps organizations determine if the change that they set to accomplish actually occurred. We follow a 4-step evaluation process: (1) establish clear outcomes, (2) create or modify data tools and systems, (3) analyze the data, and (4) use data to make informed decisions. 

In all our work with clients, we want to ensure we’re best representing all populations when doing evaluation. That’s why we’re learning more about an emerging initiative that is transforming the way evaluators think about their work – equitable evaluation. 

At TCG, we’re always learning! We recently attended a lunch and learn titled “Equitable Evaluation: Whose Voice Counts?!” hosted by the Dorothy A. Johnson Center for Philanthropy at Grand Valley State University. The presenter Equitable evaluation presentationshared examples about how equity is undermined based on traditional evaluation practices. She then shared an overview of the core equitable evaluation principles. The work of evaluation is for and contributes to equity. Evaluations should answer critical questions about the effect of a program on populations, underlying drivers of inequity, and why history and culture might be affecting the work and potential changes. In addition, evaluation design and follow-through should be based around the values of equity, meaning multi-culturally valid and set toward participant ownership.

We [the audience] were challenged to explore the opportunities to implement equitable practices into our own evaluation work, so that’s what we [TCG] did!

This presentation got us intentionally thinking about our four-step process, how equity is included and opportunities for improvements.  

Putting Equitable Evaluation into Practice

Equity can be applied throughout our  four steps, creating a fair and impartial process from beginning to end. Some examples for before, during, and after an evaluation that help lead to an equitable evaluation include: equitable evaluation handout

  • Before the evaluation, ensure the program evaluation questions and design are equitable. This means that the evaluation is multi-culturally valid and geared towards participant ownership. 
  • This thought process can be carried on throughout your evaluation. During the evaluation process, consider equity when determining appropriate stakeholders and the method of collecting feedback from those stakeholders. What factors should be considered for different stakeholders (environment, demographics, politics, influences, etc.)? What methods will be most effective to gather feedback? What influencers may be impacting stakeholder responses, or lack thereof? 
  • After, review the feedback from the evaluation through a lens of equity. What and/ or who was missing from the process? Determine the next steps to follow through based on the results of the evaluation. 

How are you ensuring equitable evaluation in your practice? You can learn more about the Equitable Evaluation Initiative on the EEI website. If you’re stuck in your evaluation process or don’t feel like you have a system in place to effectively collect equitable data – let’s chat!

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

Learn About Indiana’s Youngest Children with the 2019 ELAC Annual Report!

Communities Education Evaluation Government Research & Analysis

2019-elac-annual-reportIndiana’s Early Learning Advisory Committee (ELAC) released its 2019 Annual Report. Each year, ELAC completes a needs assessment on the state’s early childhood education system and then recommends solutions.

We want to share some quick highlights and key takeaways from this year’s needs assessment.  ELAC focuses on ensuring early childhood education is accessible, high-quality, and affordable to all families. 

Are Children Ages 0-5 Receiving High-Quality Care?

  • Of the 506,257 children in Indiana ages 0-5, 64% need care because all parents are working. This includes both working parents who are single and households where both parents work outside the home. Figure 9
  • Of those children who need care, only 40% are enrolled in known programs. The other three fifths of children receive informal care—from a relative, friend, or neighbor.
  • Of the young children who need care, only 16% are enrolled in high-quality programs. A high-quality early childhood education program not only ensures that children are safe, but also supports their cognitive, physical, and social-emotional development. 

Are Children in Vulnerable Populations Receiving High-Quality Care?

  • Indiana makes funding assistance available for early childhood education for children from low-income families.
  • Indiana does not collect data on children in other vulnerable populations, such as children in foster care and children affected by the opioid epidemic.
  • Overall, due to lack of data, Indiana does not know the kind of care received by children in vulnerable populations.

What Trends Are There in Early Childhood Education?

  • Since 2014, Indiana has made progress by enrolling more of the children who need care in known early childhood education programs. 
  • Over the past 5 years, Indiana has consistently enrolled fewer infants and toddlers than preschoolers in known and high-quality programs. Figure 31
  • Compared to 2012, more early childhood education programs are participating in Paths to QUALITYTM, Indiana’s quality rating and improvement system.
  • In addition, significantly more programs have earned high-quality designations of either Level 3 or Level 4 since 2012.

What Trends Are There in the Early Childhood Education Workforce?

  • Indiana’s early childhood education workforce is more diverse than the K-12 workforce but not as experienced.
  • Nationally, the early childhood education workforce earns $4-$7 less per hour than the average hourly wage of all occupations.

What is the Unmet Need in the Early Childhood Education System?

  • There has been a persistent need in early childhood education programs for more available spots for infants and toddlers.
  • Despite overall improvements, there are still some communities in Indiana with no high-quality early childhood education programs.
  • The tuition cost of high-quality early childhood education programs remains unaffordable, and the available financial assistance for low-income families is insufficient.

How Can I Find Out More?

  • Read the 2019 ELAC Annual Report, which includes statewide data on Indiana.
  • ELAC also publishes an interactive dashboard that allows you to learn more about specific data points. You can also easily present data to stakeholders.
  • The interactive dashboard contains both state- and county-level data. Use the map to select your county, and hover over the data to learn more!

2019-elac-interactive-dashboard

Transform Consulting Group is proud to support ELAC’s work by pulling this needs assessment and interactive report together!

Does your organization, agency, or coalition need to better understand your community or a key issue, but you don’t know how to get started? We are skilled in collecting quantitative data from multiple data sources and pulling it together in a visually-appealing, user-friendly report. Contact us to learn how we can help you complete your next needs assessment!

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

5 W’s of a Process Evaluation: Part 2

Education Evaluation Government Nonprofit

In a recent blog post, we introduced the first two W’s of a process evaluation:

  1. Why conduct a process evaluation
  2. Who should conduct a process evaluation

This blog post will cover the remaining three W’s:

  1. What methods to use to conduct a process evaluation
  2. Where to conduct a process evaluation
  3. When to conduct a process evaluation
WHAT METHODS TO USE WHEN CONDUCTING A PROCESS EVALUATION

There are several different data tools and methods you can use during a process evaluation. It may be helpful to use a combination of these methods!

  • Review documentation: It can be helpful to review staff logs, notes, attendance data and other program documents during a process evaluation. This method will help you to assess if all staff are following program procedures and documentation requirements.
  • Complete fidelity checks: Many programs/curriculums come with fidelity checklists for assessing program implementation. This is especially important if you are implementing an evidence-based program or model. Programs may have a set number of required sessions and guidelines for how frequently they should occur. You can use fidelity checklists to assess if the program’s implementation is consistent with the original program model.
  • Observe: Observations can be especially helpful when you Y Observationshave multiple sites and/or facilitators. During observations, it’s crucial to have a specific rating sheet or checklist of what you should expect to see. If a program has a fidelity checklist, you can use it during observations! If not, you should create your own rubric.
  • Collect stakeholder feedback: Stakeholder feedback gives you an idea of how each stakeholder group is experiencing your program. Groups to engage include program staff, clients, families of clients and staff from partner programs/organizations. You can use interviews, surveys, and focus groups to collect their feedback. These methods should not focus on your clients’ outcomes, but on their experience in the program. This will include their understanding of the program goals, structure, implementation, operating procedures and other program implementation components.

In our evaluation project with the Wabash YMCA’s 21 Century Community Learning Center, we used a combination of the methods described above. Our staff observed each program site using a guiding rubric. Our team collaborated beforehand to make sure they had a consistent understanding of what components to look for during observations. We also collected stakeholder feedback by conducting surveys with students, parents and teachers. The content of these surveys focused on their experiences and knowledge of the program. After the program was complete, we reviewed documentation, including attendance records and program demographic information.

WHERE TO CONDUCT A PROCESS EVALUATION

You should conduct a process evaluation wherever the program takes place. To capture an accurate picture of implementation, an evaluator needs to see how the program operates in the usual program environment. It is important to assess the implementation in all program environments. For example, if a program is being implemented at four different sites, you should assess the implementation at each site.

In our evaluation project with the Wabash YMCA, we assessed the program implementation at three different school sites. This involved physically observing the program at each site as well as reviewing records and documentation from each site. Being in the physical environment allowed us to assess which procedures were used consistently among sites. It also helped us identify program components that needed improvement.

WHEN TO CONDUCT A PROCESS EVALUATION

An organization can conduct a process evaluation at any time, but here are a few examples of times when its use would be most beneficial:

  • A few months to a year after starting a new program, you can conduct a process evaluation to assess how well your staff followed the implementation plan.
  • When you’re thinking about making a change to a program, a process evaluation will help you determine in what program areas you need to make changes.
  • If your program is not doing well, conduct a process evaluation to see if something in your process is interfering with program success.
  • When your program is doing well, conduct a process evaluation to see what in your process is making it successful.
  • If you’ve had issues with staff turnover, conducting a process evaluation can help identify gaps in staff training, professional development and ongoing support that may be contributing to the turnover rate.

To determine when to conduct a process evaluation, it is also important to consider the capacity of your organization. Make sure that your staff will have enough time to devote to the evaluation. Even when using an external evaluator, staff may need to spend extra time meeting with evaluators or participating in focus groups/interviews.

We conducted our evaluation with the Wabash YMCA at the end of their first year of program implementation. Evaluating their first year of implementation allows us to provide them with recommendations on how to improve the program’s implementation in future years. We will conduct a similar evaluation during the next three subsequent years to track their operations and processes over time.

If your organization needs support in conducting a process evaluation, contact us today to learn more about our evaluation services!

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

5 W’s of a Process Evaluation: Part 1

Evaluation Government Nonprofit

When it comes to program evaluation, people often think of evaluating the effectiveness and outcomes of their program. They may not think about evaluating how the program was administered or delivered, which may affect the program outcomes. There are several types of valuable evaluations that do not focus on outcomes. One type of evaluation, called “process or formative evaluation”, assesses how a program is being implemented.

In this two part blog series, we are going to cover the 5 W’s of a Process Evaluation:

  1. Why conduct a process evaluation
  2. Who should conduct a process evaluation
  3. What methods to use to conduct a process evaluation
  4. Where to conduct a process evaluation
  5. When to conduct a process evaluation

In this first blog in the series we will cover the first two W’s. The next blog will discuss the other three.

WHY CONDUCT A PROCESS EVALUATION

Let’s start with the “why”. A process evaluation helps an organization better understand how their program is functioning and operating. Process evaluations also serve as an accountability measure and can answer key questions, such as:Screen Shot 2018-07-27 at 4.38.23 PM

  • Is the program operating as it was designed and intended?
  • Is the current implementation adhering to program fidelity?
  • Is the program being implemented consistently across multiple sites and staff, if applicable?
  • What type and frequency of services are provided?
  • What program procedures are followed?
  • Is the program serving its targeted population?

It is important to determine what you want to learn from your process evaluation. Maybe you want to assess if the program is being implemented as it was intended or you want to know if the program model is being followed. Whatever the reason, you want to be clear about why you are completing the process evaluation and what you hope to learn.

We are currently working with the Wabash YMCA’s 21st Century Community Learning Center to evaluate their program implementation. Each center is required to work with an external evaluator to conduct a process evaluation. Here is what we hope to learn and the why of this evaluation:

  1. The evaluation will assess if the program has been implemented as it was intended and if it is adhering to state standards;
  2. This evaluation will capture the population served through the assessment of attendance trends;
  3. The findings from the process evaluation will be used for program improvement in subsequent years.

WHO SHOULD CONDUCT YOUR PROCESS EVALUATION

When determining who will conduct your process evaluation, you have the option of either identifying an internal staff member (e.g., program manager or quality assurance) from your organization or hiring an external evaluator. Many organizations find that there are challenges with an internal team member: they may not be objective, they don’t have a fresh perspective, and they often have other job responsibilities beyond the evaluation.

For the reasons mentioned above, it is beneficial to have an external evaluator (like TCG!). An external evaluator will be able to assess the operations of your program from an unbiased lens. This is especially helpful if a program has multiple sites. An external evaluator can assess all sites/facilitators for consistency more objectively than a program staff member. (If you’re interested in learning more about how to evaluate multi-site programs, view our blog post here!).

In our evaluation project with the Wabash YMCA, the decision to conduct an evaluation with an external group was made by their funders. This decision ensures that the evaluation is high quality and objective.

The other three W’s will be discussed in a later blog post, so stay tuned! In the meantime, contact us today to learn more about our evaluation services!

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

How A Needs Assessment Can Support Your Head Start Program

Education Evaluation Nonprofit Research & Analysis

Most organizations that receive federal or state funding and even private funding are required to complete some type of needs assessment. This might be part of their grant application or annual program update. The purpose of the needs assessment (which we talk more about this here and here) is to help organizations align their services to meet the needs of their targeted population and geographic service area.

Head Start and Early Head Start grantees are one type of grant program that must complete a comprehensive needs assessment every five years as part of their grant application. They are also required to complete an annual needs assessment update. In addition to the local grantees, each state has a Head Start Collaboration Office. They too are required to complete an annual needs assessment based on federal priorities to inform their annual plan and funding priorities.Blog Image

Transform Consulting Group has worked with Head Start and Early Head Start programs at every level from the local grantee level to the state collaboration office and even the federal Office of Head Start. With these partners, we have helped with writing grant applications, managing data systems, completing strategic plans, supporting implementation of new grants, and of course completing needs assessments. Based on our breadth of experience with Head Start, we have some tips to share in how to best complete and leverage your Needs Assessment:

  1. Gather Quantitative Data

The 5-Year Community Assessment must include a variety of data points such as community demographics, data about Head Start eligible children and families, education, health, social services, nutrition, housing, child care, transportation, community resources, and the list goes on. During the other 4 years of the grant period, local grantees must do a Community Assessment Annual Update. This update includes any significant changes in data around key areas such as the availability of pre-kindergarten, child and family homelessness, and other shifts in demographics and resources.

  1. Gather Stakeholder Feedback

We’ve talked a lot about stakeholder engagement in past blogs (here and here). The 5-Year Community Assessment includes gathering input from community partners, parents, and staff. We do this through the use of surveys (electronic or paper), focus groups, and interviews. This is a great opportunity to hear from your key stakeholders, build buy-in and engagement, and strengthen existing relationships.

  1. Create Visually Appealing Needs Assessment Reports

We pride ourselves on creating visually appealing reports that are user-friendly for all audiences and talk about it in this blog.  You can see examples of our Head Start needs assessment reports here and here. We have also taken these reports to create fact sheets about the need for services across different service areas or to summarize the impact / footprint of the Head Start and Early Head Start program.

In more recent years, we have started developing data dashboards that summarize the community needs assessment. Organizations are putting these dashboards on their website like this example here.  By doing this, Head Start programs can be a great resource in the community of comprehensive data about young children and families that other partners can use for planning purposes.Screen Shot 2019-05-22 at 12.04.29 PM

  1. Share and Use Your Data

After your organization has invested all of this time and effort in completing your needs assessment you want to make sure you use it to drive programming and services. This is where having a visually appealing report, some infographic facts and / or a data dashboard are so important. It makes sharing them internally with your staff and parents as well as externally with partners that much easier! We love to share this information at policy council meetings, family events, and community partner meetings.  

Does this process sound overwhelming to you? Do you feel like you are in data overload? We can help! You don’t have to do this alone.

Head Start programs, like many federally funded programs, are tasked to track and monitor a lot of data and information especially for compliance purposes. Evidence can be seen of that in the reporting requirements of the needs assessments, along with other state and federal regulations. Most Head Start programs do not have one primary database, so data is often stored in many ways across several systems and staff members. TCG can help review these systems, provide recommendations, assist in analyzing data, and offer training to staff about data systems and best practices around data collection and analysis.

We have the Head Start knowledge and the data expertise to support your needs assessment  and data management needs. Consider how TCG can help your Head Start program today. Contact us to learn more!

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

4 Steps to Create a Dashboard

Evaluation General Information

In today’s information age, organizations are overwhelmed with the amount of information that they collect, track and monitor. Non-profit leaders must decipher all the data to determine what is meaningful and relevant to share with staff, funders, Boards of Directors and other community partners. A dashboard is a great tool to bring all the critical elements together in a user-friendly report.

Through our program evaluation and research and analysis services, we help organizations create dashboards. Here are four steps to create an effective dashboard:

  1. Determine the audience for the dashboard. A dashboard is customized for the audience meant to view and use the dashboard, so first an organization needs to determine the intended audience. Then an organization needs to determine the key takeaways that you want the targeted audience to get from this dashboard. 0Lastly, the organization should really focus on the information that is most important and relevant for this audience.
  1. Decide on what the dashboard is tracking now that the audience is determined. A dashboard is meant to communicate progress over time, such as monthly, quarterly or annually. In addition, data in the dashboard can be used to compare different data sets, such as geographic locations, sites or populations. These factors need to be determined to provide the appropriate context for decision makers.
  1. Determine the visuals that will be most effective in communicating the message. In most cases, we work to fit a dashboard on one page. This does not provide much “real estate”, so you must be intentional about the visuals used to grab the audience’s attention and display the key messages [Sidebar: this is why we use Tableau!]. A dashboard does not have much room for wording and explanation.
  1. Determine the delivery of the dashboard. In most cases, dashboards are “static” or print reports that are shared via handouts or electronically. However, with the growing development of software programs, more interactive dashboards are being created. In some cases, there may be value in creating both a static dashboard that is completed annually and an interactive dashboard that is updated real-time.

Dashboard Blog imageAs we shared in this blog post about creating a needs assessment and annual report, we mentioned the state dashboard and county profile that we created for ELAC. After conducting the first state needs assessment on young children, ELAC realized that the amount of data and information was overwhelming. ELAC was inspired by the Annie E. Casey Foundation’s 16 indicator dashboard in their annual data book and used this to create ELAC’s dashboard.

Following the four steps above and inspired by the Annie E. Casey Foundation’s dashboard, we worked with ELAC to create a state dashboard that:

  • Identifies four focus areas related to children and families, high-quality care programs, education workforce, and kindergarten readiness with 16 key indicators.
  • Compares progress over time (dependent upon the release of new/ old data)
  • Uses arrows (a visual tool) to depict if the numbers represent improvement (arrow goes up) or worsening (arrow goes down).

After a few years of creating the ELAC state dashboard, communities across the state of Indiana were asking for this same data at the local level. We worked with ELAC and our state data partners to gather the county-level information to create a two-page county dashboard and profile. The first page of the county profile mirrors the state dashboard with a few exceptions.

Instead of comparing progress over time, the county dashboards compare the county data to the state data.  Following steps #1 and #2 above, we focused on the audience for the county dashboards who said that having their data in the context of the state data would be helpful to know if they are doing better or worse.  Therefore, that ranked as a higher priority than comparing their data over time.

The second page of the ELAC county dashboard was new and provided the opportunity to add visuals (charts and graphs) to depPicture1ict the key findings in the full narrative report. The visuals help to communicate complex information in simple charts.

Using a data visualization software program like Tableau is critical to not only help make the dashboards visually appealing but also to automate the process. In this case, we created 93 unique dashboards for the state and all 92 counties. While the ELAC dashboards are currently only static reports, there is the option and feature (with Tableau) of making them interactive like the Indiana Commission of Higher Education’s College Readiness Dashboard. One of our good friends, the M.A. Rooney Foundation, has also been working to transpose K-12 data for schools and community partners into meaningful dashboards.

Are you ready to get started in creating a dashboard for your organization? We would love to work with you to help you focus in on the key indicators important for your organization and create a dashboard that informs decision-making! Contact us today for help.

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

4 Tips for Getting Started with Tableau

Evaluation Research & Analysis Technology

Have you ever seen beautiful charts or dashboards that make the data “pop” in the report or presentation and wondered how could you do that?  At Transform Consulting Group, we have made a lot of charts and graphs to help our clients evaluate their programs and understand important information in a way that is easy to digest. We work to find the most efficient ways to assist our clients with the data that they need to make informed, timely decisions. 2016 Percent of Annual Income a family pays for high-quality careOne way to do this is staying current with data analysis and visualization software.

The data visualization software that we are crushing on these days is Tableau. It is essentially an accelerated version of “pivot tables”.  If any of you are familiar with Excel, then you know pivot tables.  A pivot table is a tool that we use to determine the relationship between two or more data points. For example, when we were working with TeenWorks, a college and career readiness program, to see if their students are enrolling and persisting in college, then we might want to dig deeper in understanding who the students are that are not persisting, what schools are they enrolled in, what type of school is it (public or private, 2-year or 4-year), what is their major, and what is their gender and other socioeconomic statistics.

These additional data points help tell the story of what change is occurring and how that could impact the program model, partnership development, target clients, professional development and so many other factors.  Tableau helped our team answer these questions and more to better understand the relationship of our client’s program to its intended outcomes.pg 19


Recently, Transform Consulting Group used Tableau to complete a statewide needs assessment on Indiana’s youngest children ages 0-5 by pulling together data from multiple agencies and partners. The analysis resulted in the Indiana’s Early Learning Advisory Committee’s (ELAC) help 2017 Annual Report. The intended audience for the report are policy makers who do not have a lot of time to read technical reports, but Tableau equipped out team with creating a visually-appealing report that draws attention to the key findings.

These are our top four tips of getting started with Tableau:

  1. Use Tableau support. There are many support options through Tableau. One option is the Tableau Community, which allows users to connect and ask or answer questions for each other. This can be a quick way to find answers to a common problem or question that users have.  For example, we were havProjected Employment Needsing difficulty with one of our state maps and Tableau Community had a solution that we were able to implement.
  2. Another option is to contact a Tableau consultant through Tableau. A consultant can provide customized personal training and guidance, which might be especially helpful for a new staff person using Tableau and/or a special project (like a dashboard). The consultant won’t do the work for you but is available along the way for further questions and guidance as you complete your project.  
  3. Organize your data. Tableau can be picky about how the original data is organized and certain charts require different data formatting. Before getting started, it is helpful to organize your data into one spreadsheet. Transform Consulting Group prefers to use Google Sheets because it allows multiple people to work in a document and view changes real time, but Excel or Numbers could also work.
  4. Work with a Tableau expert.  Your project might be beyond the capacity (time and knowledge) of your current team, so partnering with a group or individual who has used Tableau might be a more efficient and effective solution.

If your organization needs help with analyzing and visualizing your data, contact Transform Consulting Group for a free consultation!

Disclaimer: This is not a sponsored post.  We were not asked by Tableau to write this post.  This is our own opinion.

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

When do you need a Feasibility Study?

Evaluation Fundraising Strategies Program Development Research & Analysis

Whether you call it a feasibility study, a needs assessment, or a readiness assessment, you typically need one when your organization is looking to take on a large initiative such as a capital fundraising campaign, adding a new program or service, or expanding into a new market.  When do you need a feasibility study?

At Transform Consulting Group (TCG), we perform feasibility studies that often include the following steps:  gathering stakeholder feedback, surveying the board of directors, scanning the environment, completing a financial analysis, and conducting a community assessment.  There is a lot to consider when taking on a large initiative, and each feasibility study is going to look slightly different depending on what is being considered and how your organization operates.

Our team has served numerous clients during their feasibility study process. Here’s a look at those projects and how the client decided it was time for a feasibility study:

  • Meeting Community Need

Community leaders in Jay County wanted to investigate the feasibility of converting an old elementary school building into an early childhood center. The Portland Foundation hired our team to facilitate a site analysis of existing school buildings, assess the existing early childhood education landscape, and create a marketing and business plan for implementation.Portland Foundation Feasibility Study Cover

  • Launching a New Program/Service

Shepherd Community Center wanted to see if their organization and service area were a good fit before adopting the Center for Working Families program model. For this engagement, we held focus groups, facilitated internal and external assessments, and completed a logic model to identify the resources, inputs, outputs and outcomes aligned to support the implementation of the Center for Working Families program.

  • Assessing Annual Performance

All Head Start organizations are required to submit annual needs assessments to inform their strategic plan goals and objectives.  The Indiana Head Start State Collaboration Office hired our team to perform their report that shows how Indiana Head Start grantees compare locally and nationally and how well the state is responding to federal priority areas.  

  • Relocating or Opening a New Location

Before you consider relocating or opening an additional location in a new community, we recommend 3 steps to determine feasibility in this blog.

  • Fundraising

Our funding analysis and fund development plan are two fundraising strategies that might be part of your feasibility study.  If you’re looking to launch a capital campaign, they are two strategies that should definitely be incorporated as well as interviews with major donors and staff and surveying the board of directors.

Completing a feasibility study isn’t a quick task, nor should it be.  Make sure your organization is ready to go before committing valuable resources to a project or campaign.  Need assistance with one or more elements of your feasibility study? Not sure where to start? Contact us today to see how we can help!

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

5 Tips to Implement an Evidence-based Program

Education Evaluation Nonprofit Program Development

When awarding funding, philanthropic funders want to invest in “what works” and is proven effective. Many funders show preference for programs and practices that are evidence-based. Implementing an evidence-based program is a great way for grant seekers to demonstrate that they are also committed to “what works”.

For example, the Richard M. Fairbanks Foundation recently awarded funding to over 20 schools and school districts as part of their Prevention Matters initiative.  Prevention Matters is a three-year grant initiative aiming to help Marion County schools identify, implement and sustain proven substance use prevention programs.

To apply for this funding, schools selected an evidence-based substance use prevention program that aligned with their needs. In their proposal, schools had to demonstrate that they had a strong plan for implementation and sustainability. Developing such a plan can be a daunting task, but is crucial for successful implementation. We worked with Bishop Chatard and the North Deanery Schools of the Archdiocese of Indianapolis to help them develop their implementation plan and proposal (Which was fully funded by the Fairbanks Foundation! Learn more about our fundraising services here.). Here are 5 tips we used to help them prepare to successfully implement their evidence-based program!

1. Select an Evidence-based ProgramWhat Works Image (1)

First, you need to find a program that aligns with the needs you are trying to address. For example, if you are a school looking to prevent substance use and violence, while also promoting positive youth development, you may choose to implement the Botvin LifeSkills Training curriculum.

Taking the time to research available programs is crucial to ensuring successful implementation and maximum impact. To learn more about how to find an evidence based program, check out this blog!

2. Assess your Organization’s Capacity

Once you have selected an appropriate evidence-based program, it is important to assess your current funding and staffing capacity. You want to assess if your current organizational capacity will allow you to implement the program with fidelity. Fidelity refers to the extent to which you deliver your program as the original program model intended. Evidence-based programs are  proven effective and that effectiveness relates to how the program is implemented. Therefore, fidelity to the model is crucial to successful implementation.

Completing a feasibility study is a great way to assess your capacity and readiness. A well designed feasibility study will help an organization assess 1) if what they are thinking of implementing is possible and 2) how to consider implementing it. Check out this blog to learn more about completing a feasibility study.

The assessment of your capacity may indicate that you need to make some organizational changes. For example, you might need to tweak your program budget to purchase necessary materials and/or hire additional staff. Making these operational and workforce investments will lead to more successful implementation and program outcomes.

3. Create an Implementation Plan

Next, it’s time to flesh out your implementation plan. This plan should include a timeline and should specify staff members’ responsibilities for program related tasks. Many evidence-based programs have a set number of required sessions and guidelines for how frequently they should occur. Make sure that your implementation plan aligns with program requirements.

4. Train and Prepare Staff

Once you create your implementation plan, provide training for staff involved in the implementation. Involved staff should have a clear understanding of the program goals, activities, and their responsibilities throughout implementation. Your implementation plan should also include continued professional development opportunities and training for staff, to ensure continued high quality implementation.

5. Establish Continuous Monitoring Procedures

Once you begin implementing the program, you want to continuously monitor your fidelity to the program model. Many evidence-based programs come with accompanying fidelity checklists. It is important to identify a staff member, or an outside evaluator, who will conduct observations of the program to evaluate the implementation. You can use observations and fidelity checklists to assess if the program’s implementation is consistent with the original program model.

If your organization is looking for support in choosing, implementing or evaluating an evidence-based program, contact us today to learn more about our program development and evaluation services!

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

Does Your Organization Have a Continuous Quality Improvement Process?

Evaluation Nonprofit Program Development Strategic Planning

During a time where communities and policies are changing, it is important to ensure the programs and services within those communities are constantly evolving to meet the needs of families. The Continuous Quality Improvement (CQI) model is an ongoing process for organizations to be able to determine whether or not a change made led to an improvement in quality. In order to move toward making the necessary improvements, a review of what occurred is conducted through a CQI process like the Plan-Do-Study-Act (PDSA) cycle.

Steps to Complete a PDSA Cycleplan-act-do-study-cycle4

At Transform Consulting Group, we utilize this consistent approach when working with organizations to help them find solutions.

Plan:

Before beginning your PDSA cycle, identify the problem or issue you would like to address for quality improvement. The problem identified will guide your purpose for the review. Once you have decided what to focus on, consider the the following steps to plan to test your idea for change:

  • Define the goals
  • Define your research question(s)
  • Make predictions
  • Determine details for implementation of change or intervention
  • Plan of action for data collection

During this phase, we work with the client to thoughtfully plan to implement their new or current program/ service. So often, organizations jump immediately to step 2 – “Do” – without completing this critical first step. During the planning phase, we define what we hope to accomplish especially if we are proposing a change. Then we determine how the proposed change/ intervention will be implemented and work through all of the details. Lastly, we finalize how data will be collected.

Do:

This phase of the PDSA cycle requires you to conduct the test for the change or intervention. It is during this phase that you will complete the following tasks:

  • Carry out the intervention
  • Collect data
  • Begin data analysis

This step in the process is what most organizations know and are doing. Organizations are delivering interventions every day with their services. They might be intentionally or unintentionally modifying their intervention. The “Do” step in this process is not new to organizations. It is wrapping it around the other three steps that makes this work transformational!

Study:

The study phase of the cycle occurs after you have completed your intervention. You then analyze the data to study what did or did not occur. Organizations will want to review their predictions and assumptions before conducting the test. You will want to take the following steps during this phase of the cycle:

  • Complete data analysis
  • Compare data to predictions
  • Summarize the information

Organizations often skip over this step in the process or do not spend enough time thoughtfully reviewing the data. For some organizations, their data can be considered “high stakes” and there is a tendency to want to focus on the positive changes/ results that occurred and glance over the changes that did not occur or the benchmarks that were not met. During this phase, it is so important for an organization to be transparent and honest with themselves when reviewing the data.

Act:

Based on the summarized information, this last phase of the cycle allows you to determine what modifications may be needed to ensure that the goals you set will be met. Your organization may decide to modify a program element or change how a service is delivered; you may decide to target a different population or use a new curriculum. Once you have determined whether or not to adapt, adopt, or abandon your intervention, you will be prepared to do the following:

  • Plan next cycle
  • Decide whether the change can be implemented

During this last step, your organization takes all of the information gathered to make data-informed decisions that will ultimately improve your results. This is the exciting part of the process and one that you don’t want to skip. This step and the overall PDSA process will help your organization continue to improve the quality of services provided and impact in the community.

In this blog, “Is it time to redesign your program?”, we shared several examples of clients we helped use the PDSA process to test and implement new interventions/ modifications to improve their outcomes. The CQI process allows organizations to have a plan of action once a problem or service gap has been identified.

At Transform Consulting Group, we follow this consistent approach when helping you find solutions to accelerate your impact. If you are looking to improve the quality of a service or program to facilitate positive change, contact us today!

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin