Category Archives: Evaluation

4 Steps to Establish Evaluation SOPs

4 eval stepsAn evaluation plan is a roadmap to help organizations validate the impact of their work. In this past blog post, we describe our four-step evaluation process. Step 2 of this process involves creating or modifying data tools and systems. When establishing evaluation systems, it is important to create standard operating procedures, or SOPs.

An SOP is a step by step procedure that outlines necessary processes within an organization. SOPs actualize your organization’s processes into actionable tasks in a structured and uniform format. They are used to make sure staff are following the same process when completing an organizational activity. SOPs establish consistency and accountability among staff and improve efficiency and quality.

When it comes to evaluation, SOPs help to ensure data collection is valid and reliable and also help you avoid data errors.They provide structure and guidance to staff who may not focus on evaluation and data on a daily basis. SOPs equip them to support your organization’s evaluation work and integrate it into their day-to-day responsibilities. SOPs also help to mitigate the effect that staff changes and transitions may have on your evaluation activities. By having SOPs in place, you can introduce new staff to your evaluation procedures early on in their onboarding and training and improve the continuity and sustainability of your evaluation work.

Last spring we worked with the Center for Leadership Development (CLD) on an evaluation project and created SOPs for all of their evaluation tools and procedures. We worked with them to align the SOPs to the structure of other organizational procedures which helped them to integrate the SOPs successfully and smoothly across all programs. 

Here are four steps we completed with CLD that you can also use to get started with creating evaluation SOPs for your organization.

1. Create a list of the different procedures you want to document and systemize.

We recommend creating an SOP for all evaluation-related tools you utilize. These may include:

    • Data collection tools (surveys, assessments, academic data/data share agreements, registration/enrollment forms, etc.)
    • Databases and programs you use to store your data (Oracle, Apricot, etc.)
    • Data analysis and visualization programs (Excel, Tableau, SPSS, etc.)

2. Create a template or format that makes sense for your organization. 

We recommend that all of your SOPs include the following elements:

    • Purpose: Overview of why to follow the procedure
    • Responsibility: Indicate who is responsible for completing specific tasks outlined in the SOP; Make sure to indicate the JOB TITLE of the person responsible, rather than their name so there is no confusion in cases of staff turnover
    • Instructions: List all steps and tasks to complete when following the procedure
    • Timeline for completion: Indicate when to complete each task
    • Administration/Materials: Include a list of any materials, attachments, and references that staff should use when completing the procedure

Here is an example of a general SOP template:

Screen Shot 2020-01-15 at 9.25.13 PM

The SOPs for your data collection tools should include two distinct sections detailing the procedures for collecting and analyzing data. Both of those sections should include the elements listed above. 

3. Draft SOPs or delegate their creation to staff

assess-01You do not need to create all of your evaluation SOPs at one time. Create a prioritized list/schedule of which SOPs are most important to create first. To make the drafting process more manageable, consider training multiple staff members to create SOPs using the template. Make sure these staff members are familiar with the evaluation tools and systems. One staff member should be responsible for reviewing and approving all the evaluation procedures to check for consistency in formatting and structure. 

4. Train staff on following new SOPs 

Once you create your evaluation SOPs, make sure they don’t just sit on a shelf! The SOPs need to be used and followed consistently by all relevant staff. Provide training and guidance for your staff on the new SOPs. Point out how the activities and tasks fit within their day-to-day responsibilities.

At Transform Consulting Group, we know the importance of demonstrating the impact of your work. Whether you are just now beginning to evaluate your program or you already have an established evaluation plan in place, we would be happy to help you create SOPs to strengthen the sustainability and effectiveness of your evaluation activities. Contact us today for more information!

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

When and How to Create a Strategic Plan

We often talk with clients who identify barriers to their success, but they don’t realize that they may need to create a strategic plan in order to accomplish their goals. In other cases, our clients do recognize their need for a new strategic plan, but they lack the expertise and capacity to develop it on their own. We help organizations navigate this process!

Should My Organization Create a Strategic Plan Now?

We wrote a blog to answer the question When Is It Time to Change Your Program? The factors in that blog may also indicate that your organization needs a new strategic plan: 

  • Are your programs not having the desired impact?
  • Do you lack sufficient funding?
  • Has new industry research emerged?
  • Are your community’s demographics changing?

In addition, there are other factors that signal a need for strategic planning:

  • Was your current strategic plan completed 5+ years ago?
  • Is your organization newly created?
  • Is large-scale systems change underway?

Newly Created Organizations

We recently worked with clients impacted by two of these factors, and one was a newly created organization. Kosciusko County’s Child Care and Early Learning Coalition, LaunchPad, was established in October 2018. In their first year, the coalition hired a director and gathered stakeholder feedback. However, they struggled to gain momentum without a strategic plan. Within their coalition, they didn’t have the strategic planning expertise they needed. They reached out to us for assistance creating a strategic plan. 

Launchpad LogoOur team came alongside LaunchPad, meeting them where they were in their work. Instead of restarting the process of gathering stakeholder feedback, we used the information already available. Due to that, we could shorten the first two steps of our strategic planning process. Then, we quickly moved on to facilitating consensus and developing a strategic plan. This enabled us to meet LaunchPad’s time frame and set them up for their next steps to accomplish their goals! 

Large-Scale Systems Change

The other client we recently partnered with is 4C of Southern Indiana. One of 4C’s primary functions is their role as a Child Care Resource and Referral (CCR&R) agency. The state of Indiana recently enacted large-4C-logo-color-black-fontscale systems change that impacted CCR&Rs and other agencies.

In order to adjust to these changes and plan for the future, 4C asked us to facilitate a strategic planning process for them. In our work with 4C, we connected with diverse stakeholders for feedback. We reached consensus on a new plan with the board, staff, and key partners. We created an implementation plan for them, and now they’re putting it into action!
4C Strategic Planning Retreat

Have you determined that you need help with your strategic planning to build capacity? Contact us today to chat about your goals and how we can help!

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

2019 Year in Review

This year was another big year for Transform Consulting Group. In 2019, we were thrilled to once again work with a variety of clients and important causes across Indiana and Michigan. 

We are thankful for all that we accomplished as a team to help accelerate the impact of our clients in communities across the two states. We summarized a snapshot of the highlights from 2019 below. 

Building Capacity:

  1. We secured over $1.6 million in grants for clients and created development plans to increase and diversify funding
  2. We facilitated 8 strategic plans which included internal and external organization assessments, research, building consensus with stakeholders, and mapping out specific steps to achieve the goals. Many of these projects were community wide initiatives that involved a diverse group of partners utilizing a collective impact framework. 
  3. We shared our knowledge and expertise at over 15 trainings at conferences and more than 20 trainings with clients. 

Facilitating Evaluation, Research, & Analysis:

  1. We continue expanding our data visualization skills and created 31 data dashboards to make data user-friendly, accessible, and manageable for clients. View a few examples here.
  2. We completed 9 program evaluations that included setting clear goals, developing tools and systems, data analysis, and an evaluation plan.
  3. We developed 9 community needs assessment data reports to help organizations understand their community and target population to appropriately plan for and align programming and services.

Mobilizing Communities, Partners, & Systems:

  1. We created and disseminated 60 surveys for stakeholder feedback to help direct our clients’ next steps. 
  2. We’re passionate about Collective Impact and worked with Community Foundations and other partners across the state to identify and address big issues in their communities. 
  3. We helped launch a new state contract that connects a number of partners at the state and local level. 

Company Highlights: 

  • As a woman-owned business, it was a huge accomplishment to officially become a certified Women’s Business Enterprise
  • We increased our number of clients by over half in 2019. Of those clients, nearly half were brand new!
  • We were 2019 Big Data Visualization Challenge finalists for the 2nd year. The Challenge used data to highlight recommendations for improving health outcomes for Indiana mothers and infants. We were provided datasets on maternal health and infant mortality, and then tasked with creating a visualization solution. You can view the dashboard we created and our recommendations here
  • We expanded our reach with clients across Indiana and Michigan who represented so many different sectors in Education and Community Development including: 
    • Adult Education
    • Alternative Education
    • Child Welfare
    • College & Career Readiness
    • Domestic Violence
    • Early Childhood Education
    • Economic Development
    • Homelessness
    • Home Visiting
    • K-12 Education
    • School Counseling
    • Substance Abuse Prevention
    • Veteran Services
    • Youth Development

As we head into the New Year, we are grateful for the organizations who trust our team and partnered with us in 2019. We have exciting things ahead in 2020, and we would love to work with you! Do you have a clear plan for your upcoming year? Whether you’re looking to strengthen  your organization’s capacity, use your data to drive impact, or mobilize your community and partners to tackle big issues, we would love to help! Contact us to set up a call to talk further in the New Year.

Website Image(Click here to enlarge image)

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

3 Tips For Managing Program Data

 Do you have a system for effectively and efficiently storing and monitoring program data? At Transform Consulting Group, we work with many clients who are required to report on program outcomes and impact, but they often lack systematic approaches to managing their data. We know data can drive program impact (check out this blog), but in order to use the data, you first have to collect and accurately report information. Managing Multiple Data Systems

One type of organization required to report on program outcomes and impact is Head Start. Head Start programs, at all levels (federal, state, local, and internal), are required to collect program data and report on it regularly. This data includes information about staff, students, families, facilities, maintenance, etc. The list goes on! Not only is data required for reporting, but ideally organizations are tracking data to evaluate their impact and inform program decisions. Currently, there is not an all inclusive system for Head Start programs to store and monitor this data, so programs are left to find and use multiple systems.  

Geminus Head Start in northwest Indiana sought our help to create a customized monthly report and interactive dashboard (see our dashboard examples here) that connects their data systems and equips their team to use the data to improve student and family outcomes. Along the course of our partnership, we came up with several tips to manage data that are helpful whether you’re a Head Start organization or other similar program wanting to effectively and efficiently manage data. 

1. Develop Procedures and Data Management Plans for Data Systems 

If you have a long list of data, several tools and systems are needed to track the information. To keep everything organized, we found it necessary to develop procedures around tools and systems. This creates consistent and efficient data entry.

Having procedures in place is especially helpful when hiring  new staff members or shifting responsibilities between teams. One example of a tool we use to manage several data systems is a Standard Operating Procedure (SOP). A SOP is applied to a specific tool, like a child assessment. The SOP outlines its own purpose and when to use it. It also explains who is responsible, related instructions about data entry or export, and timelines. 

2. Assign and Use Unique Identifiers

Similar data elements are often tracked across several systems. For example: a student’s name being used in attendance, assessment collection, and family engagement tracking. If organizations try to link data based on a name, there could be problems caused by inconsistent spelling or duplicate names (two people with the same name). A unique identifier (series of numbers and/or letters) matches program data with ease across many systems. This will help ensure efficient and accurate data reporting. 

Geminus Head Start uses PROMIS as a main data entry and management system. This system automatically assigns unique identifiers to students, families, programs, etc. These identifiers are then entered into other tracking systems during data entry. Look for a similar identification feature in your organization’s main data tracking system. 

3. Minimize Manual Spreadsheet Tracking

Staff often feel more comfortable developing their own spreadsheet to track organizational information related to their role. However, that’s one more source your team will need to keep track of when searching for and entering data. Manual entry also leads to inconsistencies in spelling and entry depending on the individual and time of entry. There are databases available that allow custom report building. PROMIS, the software Geminus Head Start uses, allows for custom report building. SurveyMonkey or Google Forms can also be used to build custom surveys or fillable forms. Custom reports have efficient features, such as drop down menus for consistent entry options. These reports can be built for several topic areas, but linked within one system creating a connection between common data elements (students, families, sites, etc.). 
Geminus meeting 3

We know these three tips are not limited, but they set your organization on a path for effective and efficient data collection. Do you need help gathering and connecting multiple data sets? Do you want to create customized dashboards to visualize benchmarks and filter data? Does your staff need trained on using data to make informed decisions? If you answer yes to any of these questions, let’s chat! We’d love to provide solutions to accelerate your impact through data.

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

Using Data to Tell Your Story

We say we’re #datanerds at Transform Consulting Group. However, for a communication and marketing person like myself, I will admit that data intimidates me. I prefer using words and emotions to convey ideas rather than numbers and excel sheets. So, how does that translate to our data-driven work at TCG? 

In Nancy Duarte’s book “Data Story” she says, “Facts aren’t as memorable as stories.” She highlighted an experiment that revealed 5% of people remember individual statistics while 63% remember the stories. 

In our work – as an organization and for clients – we have to have both data and storytelling if we want to make an impact. If you search through our blogs, we have several data related posts (here, here, here, and here). We also have a whole marketing series (here, here, here, and here). This blog is bringing together those 2 worlds – with 5 tips for using data in your storytelling.  

5 Tips for Using Data to Tell Your Story

 

  1. Keep it simple. Numbers and data may seem far from simple at times, especially when you’re dealing with really complicated issues. The reality is though, most of your stakeholders are not technical experts. They don’t know the lingo. They don’t know your measurables. You don’t want to bog down your audience with so much “meat” that they can’t absorb anything. Simple is better. Be clear, straight to the point, and look at your data through the lens of an “everyday” person.
  2. Make it relatable. Why is the data important? Who does it impact? It’s hard for even the biggest #datanerd to get excited about an excel sheet full of numbers or a report with a bunch of charts. However, if you explain the implications of the data, people will connect. We like to call this the “so what?”

    What does it mean for your community when student graduation rates decline? How does it impact employers when there aren’t enough early childhood education programs available for working families? What does it mean for your community to have a high rate of child maltreatment? Don’t just spout off facts and figures, explain the why and significance of your data.

    Extra Tip: Know your audience. Know who you are talking to so you can shape your message in a way that relates to the person in front of you. Your data story should be very different for your potential client like a parent versus a potential funder or partner.
  3. Utilize clear graphs and slides. When surveying top executives from large organizations across the county, Duarte found the majority preferred simple visuals to get to the point. They requested a bar graph, pie chart or line graph. We create some pretty fancy data dashboards here at TCG, but we know that the most important data point needs to be the first thing you see. Don’t bury it with too many special features, graphics, or animations.  Also, be mindful of creating clear titles to describe your graphs and charts. Be intentional with the type of chart you have selected, including the colors.
  4. Structure your story. We learn from a young age that a story needs to have 3 components: a beginning, middle, and end. How does that translate when using data? The beginning of your story should highlight the pain point. Set the stage for what is the current reality and need. The “messy middle” as Duarte says, is where you highlight the obstacles or hurdles getting in the way of progress or impact. The end is where you present the solution.
  5. Make the data stick. How do you talk about the magnitude of the data? The data should connect to something familiar if you want your message to stick. If the numbers are great, express that clearly in your message and tone. If the numbers need improvement, then be direct and express disappointment. It is possible to generate emotions from numbers in your delivery. 

At Transform Consulting Group, we are passionate about the many causes our clients represent. We know your work is important. So, what’s next for you? Whether you’re struggling with gathering and analyzing data to inform decision making or struggling to use that data to craft your story of impact – we want to help! Let’s work together to turn your data into action. 

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

3 Tips to Use Your Data to Drive Program Improvement

Nationally, only about half of students who start college actually complete and earn college degrees, according to the National Student Clearinghouse. It is much worse for students who start at a community college or 2-year degree program. This means many students are increasing their debt (from student loans) and not reaping the financial benefits of a college degree and higher earnings. In addition, colleges are losing money when students dropout. It’s a lose-lose game.3 Tips to Use Your Data to Drive Program Improvement

Colleges have been getting increasing pressure from the federal government and others to improve their college completion rates. They have turned to data analytics to better understand how they could intervene earlier with students who might be at risk of not completing their degree and dropping out.

When I first heard this podcast from The Hechinger Report on how “Colleges are using big data to track students in an effort to boost graduation rates, but it comes at a cost” it made me want to listen closer.  As a #datalover and #datanerd, this topic certainly peaked my interest. I am also a first generation college graduate and can personally relate to this topic of college dropouts since many (over half) of my high school peers didn’t finish their college degree.

This is not a new strategy (“predictive analytics”) as companies like Amazon and Google are doing this all of the time with our digital footprint. However, it’s new for colleges and non-profit organizations.  

Georgia State is a case study example of how their university has embraced data analytics to improve college completion rates. Their university now has one of the highest rates of college graduation for public universities in the country, and they have closed the racial equity gap. Students of color are graduating at the same rate as white students at Georgia State. 

So how did Georgia State get there? They used their data to drive and inform program improvement. We’ve talked about this here and here.

The podcast also shares the struggles that other universities face in implementing these changes. It’s not enough to purchase the data analysis software, but you also need trained staff who are able to analyze and interpret the data to take action on it. 

We’ve put together 3 tips to get started with using your data to drive program improvement that’s not only based off of the success of Georgia State but our work with other clients who want to improve their impact.

  1. Have a system in place to collect and track meaningful data. Georgia purchased a data system to help them bring all of their data together and identify patterns. At Transform Consulting Group (TCG) we are big fans of using Tableau Software (see more here and here). We love Tableau, because it can make your data easier to review and understand. We are also adept at using whatever data systems our client has to pull out the information that we need to inform decision-making.
  2. Have trained staff who can analyze and interpret the data. Getting the data from your system is the first step. Then you need to have individuals who know how to identify patterns, ask inquisitive questions and develop recommendations. We are big fans of forming an “Impact Team” at an organization who is trained on analyzing and reviewing your data and can help drive action based on the results. Learn more here.
  3. Have a process to determine program improvement changes. Once you have your data and results, you are now at the fun part – you get to take action on changes to make to improve your results! We find that sometimes organizations don’t have a clear process in place to determine what changes – such as a new curriculum, staff training, client outreach – they will make based on the results. We follow a Continuous Quality Improvement Process using the “Plan – Do – Study -Act or (PDSA)” framework to determine what action steps we will take. Other similar processes might be “Lean/Six Sigma”. At Georgia State they decided to hire additional advisors to use the data to reach out to students at risk. This is one of the many changes they have made, based on their data results, to help students stay on track with college graduation. The goal here is to be intentional on how you will use your data to make programmatic changes and then study the result of those changes to know if it’s working or not.

What are some goals that you are not currently satisfied within your organization? Have you considered how data analytics could help you focus in on improving your results?  Bill Gates said, “Without measurement there is no shared accountability.“ We would love to help you improve your impact through data analytics. Contact us to learn more.

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

Why Break Down Data?

When you’re using data to make decisions, are you also taking time to break down data to learn more? Perhaps you are struggling to understand the needs of different parts of the population you serve. Maybe you’re noticing different outcomes in different groups but don’t know why. When you break down data, you can see what’s hidden within your overall results. 

One important reason to break down data is to help your clients who are experiencing multiple adverse events. Our team at Transform Consulting Group worked with Coalition for Homelessness Intervention and Prevention (CHIP) to understand the challenges of people experiencing both homelessness and domestic violence. We took a close look at their data. Then we came up with recommendations on how to best meet the needs of this population.

A significant portion of individuals who are homeless have also experienced domestic violence. In Marion County, 21% of individuals in the Homelessness Management Information System (HMIS) had lived in households with reported domestic violence (DV). DV-homelessness-data

Based on national best practices and local data, CHIP and its partners identified potential system and policy improvements. They gathered feedback from domestic violence survivors experiencing homelessness via surveys and focus groups. 

The data and research revealed the need for targeted public policy and legislative protections for this population. When domestic violence survivors leave their relationships, they face economic hardships that put them at risk for homelessness. One policy solution allows survivors to remain in their rental home after the perpetrator is removed from the lease.

Breaking down the data led to this policy recommendation that is specific to domestic violence survivors. This policy change goes beyond what’s relevant for all individuals experiencing homelessness.

In other instances, breaking down data can be particularly helpful to ensure you meet the needs of the most marginalized people. Are children from high-income families able to access your programming more easily than other children? Are your participants of color seeing the same gains as your white participants? Do those in rural areas achieve the same positive results as urban communities? Looking at data in this way can help you focus on equity for your vulnerable populations. 

Researching national best practices revealed that domestic violence survivors, in particular, benefit from meeting with advocates in locations other than their office. Survivors face transportation and other logistical barriers. This can mean it’s much easier if an advocate comes to their home or neighborhood.

There are more details on all the data and findings in the Report on Domestic Violence Survivors Experiencing Homelessness in Marion County that Transform Consulting Group prepared for CHIP. CHIP-DV-report-cover

No matter what your organization’s mission is, breaking down data can help you learn more about different segments of the population you’re serving. Do you see better outcomes when participants have been in your program for more than six months? Is your curriculum more effective for younger children?

In addition to breaking down your data, check out our other blogs on making sure you’re data literate and putting data into context

If you aren’t using data to look at segments of the population you serve, then you might be missing what is (or isn’t) working well in your program. Let us know if you need help with data analysis or program evaluation!

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

How A Needs Assessment Can Support Your Head Start Program

Most organizations that receive federal or state funding and even private funding are required to complete some type of needs assessment. This might be part of their grant application or annual program update. The purpose of the needs assessment (which we talk more about here and here) is to help organizations align their services to meet the needs of their targeted population and geographic service area.

Head Start and Early Head Start grantees are one type of grant program that must complete a comprehensive needs assessment every five years as part of their grant application. They are also required to complete an annual needs assessment update. In addition to the local grantees, each state has a Head Start Collaboration Office. They too are required to complete an annual needs assessment based on federal priorities to inform their annual plan and funding priorities.Blog Image

Transform Consulting Group has worked with Head Start and Early Head Start programs at every level from the local grantee level to the state collaboration office and even the federal Office of Head Start. With these partners, we have helped with writing grant applications, managing data systems, completing strategic plans, supporting implementation of new grants, and of course completing needs assessments. Based on our breadth of experience with Head Start, we have some tips to share in how to best complete and leverage your Needs Assessment:

  1. Gather Quantitative Data

The 5-Year Community Assessment must include a variety of data points such as community demographics, data about Head Start eligible children and families, education, health, social services, nutrition, housing, child care, transportation, community resources, and the list goes on. During the other 4 years of the grant period, local grantees must do a Community Assessment Annual Update. This update includes any significant changes in data around key areas such as the availability of prekindergarten, child and family homelessness, and other shifts in demographics and resources.

  1. Gather Stakeholder Feedback

We’ve talked a lot about stakeholder engagement in past blogs (here and here). The 5-Year Community Assessment includes gathering input from community partners, parents, and staff. We do this through the use of surveys (electronic or paper), focus groups, and interviews. This is a great opportunity to hear from your key stakeholders, build buy-in and engagement, and strengthen existing relationships.

  1. Create Visually Appealing Needs Assessment Reports

We pride ourselves on creating visually appealing reports that are user-friendly for all audiences and talk about it in this blog. You can see examples of our Head Start needs assessment reports here and here. We have also taken these reports to create fact sheets about the need for services across different service areas or to summarize the impact / footprint of the Head Start and Early Head Start program.

In more recent years, we have started developing data dashboards that summarize the community needs assessment. Organizations are putting these dashboards on their website like this example here.  By doing this, Head Start programs can be a great resource in the community of comprehensive data about young children and families that other partners can use for planning purposes.Screen Shot 2019-05-22 at 12.04.29 PM

  1. Share and Use Your Data

After your organization has invested all of this time and effort in completing your needs assessment you want to make sure you use it to drive programming and services. This is where having a visually appealing report, some infographic facts and / or a data dashboard are so important. It makes sharing them internally with your staff and parents, as well as externally with partners, that much easier! We love to share this information at policy council meetings, family events, and community partner meetings.  

Does this process sound overwhelming to you? Do you feel like you are in data overload? We can help! You don’t have to do this alone.

Head Start programs, like many federally funded programs, are tasked to track and monitor a lot of data and information especially for compliance purposes. Evidence can be seen of that in the reporting requirements of the needs assessments, along with other state and federal regulations. Most Head Start programs do not have one primary database, so data is often stored in many ways across several systems and staff members. TCG can help review these systems, provide recommendations, assist in analyzing data, and offer training to staff about data systems and best practices around data collection and analysis.

We have the Head Start knowledge and the data expertise to support your needs assessment and data management needs. Consider how TCG can help your Head Start program today. Contact us to learn more!

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

4 Steps to Complete a Program Evaluation

Program evaluation helps organizations determine if the change that they set to accomplish actually occurred. Change can be knowledge gained, attitude change, or behavior change. For example, did a literacy tutoring program help the students who were not reading on grade level actually catch up to reading on grade level by the end of the program period? Or, are the low-income children who participated in a high-quality pre-K program ready for kindergarten?  

At Transform Consulting Group, we are data nerds and love helping organizations develop and implement evaluation plans to assess their change by using a four-step process.
4 eval steps

Program Evaluation in Four Steps:

  1. Establish clear outcomes
  2. Create or modify data tools and systems
  3. Analyze the data
  4. Use data to make informed decisions

The first step is to ensure there are clear outcomes in place that support an organization or program’s goals. We work to develop SMART outcomes: specific, measurable, achievable, relevant, and timely. In this step, we typically review or develop a logic model, which is just a technical description of aligning your programs to the change (or outcomes) that you want to accomplish. For example, a college-readiness program may want to increase the number of students who can (1) identify a major or career they are interested in pursuing after high school; or (2) understand how to apply for college financial aid.  

The second step focuses on having the right data tools and systems in place to measure and report back on the designated outcomes. Transform Consulting Group helps organizations determine the most appropriate tool(s) to collect, track, and monitor accomplishment of the identified outcomes. We consider organizational capacity (staff time, knowledge, and budget) when suggesting data tools. Some examples could include participant surveys, assessments, and student academic records. We work to have data tools that are valid and reliable and will provide the data necessary to monitor progress.

The third step is to analyze the data once it has been collected, and present the results in an easily-understood format. Data is measured to determine whether program outcomes were met and the change, if any, that occurred. This is often the step where organizations get stuck, because they don’t have the staff time or knowledge to complete the analysis. [Stay tuned for a separate post on simple ways to analyze your data!] We tell our clients that this is the fun part because we can see if what they set out to accomplish actually occurred!  

Some common research questions that drive many organizations to conduct program evaluations to get answers include:

  • Are program participants being reached as intended? If yes, why? If not, we also want to know why.
  • To what extent are desired program changes occurring? Was there a significant difference or just a small difference? Is there a specific group that is not being impacted?
  • Is the program worth the resources it costs? What is the “return on investment” for this program or service?

The fourth step is to discuss the results of the program evaluation and make informed decisions based on what the data tells us. We will compile a summary report and/or slide deck presentation of the evaluation data for stakeholders internally and externally to review the results and discuss the implications. Good evaluations often lead to recommendations for improvement such as enhanced professional development, diversified participant recruitment strategies, and/or program model changes. This is an opportunity to discuss data collected and implications for future programming, including ongoing program evaluation practices within the organization.

In today’s era of accountability, what gets measured gets done. If you don’t measure results, you can’t tell success from failure[1]. Transform Consulting Group equips organizations to celebrate their successes and inform opportunities for growth. Contact us today for more information on how Transform Consulting Group can help assess the impact of your organization.  
[1] Reinventing Government, Osborne and Gaebler, 1992.

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin

4 Steps to Complete a Feasibility Study

Too often non-profits and government agencies immediately begin implementing a new program or service area. They see a need with their clients or a gap in the existing services, so they elect to help meet that need. This all sounds good, right? The challenge is that there has not been enough time to complete a comprehensive planning and assessment process to develop the program or service. One service we offer our clients to meet this need is completing a feasibility study.plan-act-do-study-cycle4

We follow the Plan-Do-Study-Act or “PDSA” continuous quality improvement cycle (learn more in this blog).  We help clients assess, design, launch and evaluate programs and services in order to meet community needs and apply the latest research. When following this approach, we most often find that clients tend to skip the first step “Plan” and jump straight to “Do” as mentioned above. We work to help our clients thoughtfully plan out their services, programs, and interventions before they implement them to get the impact and desired change they are working towards.

Implementing a feasibility study is a great tool to complete a thoughtful planning process. A well designed feasibility study will help an organization assess 1) if what they are thinking of implementing is possible and 2) how to consider implementing it.

Shoes at ArrowsWe worked with a group of community leaders in Jay County to complete the feasibility of converting an old elementary school building into an early childhood center. Like many rural communities, Jay County has a declining population that has impacted their local schools in continuing to operate multiple school buildings, which has resulted in school consolidations and closures. At the same time, their rural community also struggles with attracting new employers due to a lack of child care for a growing workforce. Their community leaders had the idea of converting a closed elementary school into an early childhood center but wanted assistance in completing a feasibility study first.

4 Steps to Complete a Feasibility Study

 

1. Market Analysis

During this step you want to gather key information about your targeted population. This includes collecting demographic information from online public sources. This helps create a composite of your targeted community and population. We also suggest completing a landscape assessment to identify any other organization providing similar services or working with the target population. Lastly, it’s important to gather some qualitative feedback from various key stakeholders in the community to determine what they think the needs and gaps are as well as build community will for possibly launching a new service. This can be done through focus groups, surveys, and key informant interviews.

The purpose of this step is to ensure that there is in fact a need for your proposed program/ service. Check out this blog for more insight on completing a community needs assessment!

2. Program Design

During this step you will want to complete some research on your targeted service area. For Jay County, we are gathering the latest research on early childhood program models and services that lead to the desired outcomes they are seeking. Our landscape scan is also looking at existing program models in the community so as to not duplicate existing options but to consider complementary program models that will meet the needs of communities. If you are seeking external funding, you may want to adopt or align your program around research-based models that have demonstrated outcomes. This will provide confidence to potential funders in implementing a new program.

The purpose of this step is to determine the best model and design for implementing your program. Check out this blog for more tips on finding evidence-based programs.

3. Business Model

The next step is to develop the business model for operating the program. During this phase of the feasibility study you will gather important financial information that will help you understand what it will cost to implement the program and potential sources of funding. You should create a budget and possibly complete some financial forecasting to show start-up costs and when the program would “break even” or be self-sustaining. This step should also assess the operations behind implementing the program, which includes the staffing model, materials and services, training, facility, technology, equipment and other program needs.

With Jay County, we completed walk-throughs of three possible locations with an architect and construction group to inform the best location to operate an early childhood center. This informed the potential capacity to serve children, the staffing needs and ultimately budget the break down for start-up costs versus ongoing maintenance costs. The purpose of this step is to think through all of the components needed to successfully implement the program.

Check out this blog for some tips to establish financial goals.

4. Communications Plan

The last (and sometimes forgotten) step is to develop a communications strategy if you decide to launch the new program. After spending all of this time assessing and planning the design of the program, you want to ensure that the targeted audience knows about the program and enrolls/ participates. The communications plan would include determining the current knowledge base in the community, so there might need to be some education and awareness about why you are providing this service especially if it is new and different.

In Jay County, we are created a PR Campaign through a series of op-eds penned by different key stakeholders (employers, teachers, judge, doctor, etc.) in the community all talking about why expanding early childhood is critical to meet the community’s needs. Your communications plan should include the different channels (social media, newspaper, radio, text, mailings, etc.) that residents use to gather information. In a parent survey (our potential client for early childhood services), we asked them where they get their information and their preferred method of communication. Based on this assessment, develop a start-up marketing plan and community education plan for the proposed new program that will meet participation goals and engage the key stakeholders and partners in the community.

Check out this blog for tips on creating an op-ed campaign and this blog for getting media attention.

Completing a feasibility study may seem unnecessary or slow down your timeline, but the time you invest up front will see a return in a well thought out model that will be set up for success and to accomplish your goals. Completing intentional design through the PDSA model is a critical differentiator for Transform Consulting Group and many clients point specifically to this process improving their own internal operations which accelerates impact. Contact us if we can help you complete a feasibility study!

Share this article:Share on Facebook
Facebook
Tweet about this on Twitter
Twitter
Email this to someone
email
Share on LinkedIn
Linkedin