Category Archives: Evaluation

Does Your Organization Have a Continuous Quality Improvement Process?

During a time where communities and policies are changing, it is important to ensure the programs and services within those communities are constantly evolving to meet the needs of families. The Continuous Quality Improvement (CQI) model is an on-going process for organizations to be able to determine whether or not a change made led to an improvement in quality. In order to move towards making the necessary improvements, a review of what occurred is conducted through a CQI process like the Plan-Do-Study-Act (PDSA) cycle.

Steps to Complete a PDSA Cycleplan-act-do-study-cycle4

At Transform Consulting Group, we utilize this consistent approach when working with organizations to help them find solutions.

Plan:

Before beginning your PDSA cycle, you will need to have identified a problem you would like to address for quality improvement. The problem identified will guide your purpose for the review. Once you have chosen a change idea to focus on you will need to go through the following steps to plan to test your idea for change:

  • Define the goals
  • Define your research question(s)
  • Make predictions
  • Determine details for implementation of change or intervention
  • Plan of action for data collection

During this phase, we work with the client to thoughtfully plan to implement their new or current program/ service. So often, organizations jump immediately to step 2 – “Do” without completing this critical first step.  During the Planning, we define what we hope to accomplish especially if we are proposing a change.  Then we determine how the proposed change/ intervention will be implemented and work through all of the details.  Lastly, we finalize how data will be collected.

Do:

This phase of the PDSA cycle requires you to conduct the test for the change or intervention. It is during this phase that you will complete the following tasks:

  • Carry out the intervention
  • Collect data
  • Begin data analysis

This step in the process is what most organizations know and are doing. Organizations are delivering interventions everyday with their services. They might be intentionally or unintentionally modifying their intervention.  The “Do” step in this process is not new to organizations.  It is wrapping it around the other three steps that make this work transformational!

Study:

The study phase of the cycle is the time where once you have completed your intervention, you analyze the data and study what did or did not occur.   Organizations will want to review their predictions and assumptions made before conducting the test. You will want to take the following steps during this phase of the cycle:

  • Complete data analysis
  • Compare data to predictions
  • Summarize the information

Organizations often skip over this step in the process or do not spend enough time thoughtfully reviewing the data. For some organizations, their data can be considered “high stakes” and there is a tendency to want to focus on the positive changes/ results that occurred and glance over the changes that did not occur or the benchmarks that were not met.  During this phase, it is so important for an organization to be transparent and honest with themselves when reviewing the data.

Act:

Based on the summarized information, this last phase of the cycle allows you to determine what modifications may need to be made to ensure that the goals you set will be met. Your organization may decide to modify a program element or change how a service is delivered; you may decide to target a different population or use a new curriculum.  Once you have determined whether or not to adapt, adopt, or abandon your intervention, you will be prepared to do the following:

  • Plan next cycle
  • Decide whether the change can be implemented

During this last step, your organization takes all of the information gathered to make data-informed decisions that will ultimately improve your results.  This is the exciting part of the process, and one that you don’t want to skip. This step and the overall PDSA process will help your organization continue to improve the quality of services provided and impact in the community.

In this blog, “Is it time to redesign your program?”, we shared several examples of clients who we used the PDSA process to help them test and implement new interventions/ modifications to make to their programs to improve their outcomes. The CQI process allows organizations to have a plan of action once a problem or service gap has been identified.

At Transform Consulting Group, we follow this consistent approach when helping you find solutions to accelerate your impact. If you are looking to improve the quality of a service or program to facilitate positive change, contact us today!

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

How to Find an Evidence-Based Program

What Works Image (1)Often in writing grants and implementing programs, funders want to invest in “what works” and has been proven effective. Implementing an evidence-based program is a way to demonstrate proven results. If your organization wants to prevent child abuse, you might consider implementing an evidence-based home visiting program like Nurse Family Partnership or Healthy Families of America that has shown to prevent child abuse and promote parent-child engagement.

Across the country more funders, including the federal government have been focused on investing in evidence-based programs as well as expanding the number of evidence-based programs across different sectors. The reasons might vary, but funders want are committed to improving the different social issues affecting individuals and communities across the country.

How do you know what programs are evidence-based or are available to replicate in your community? There are some national clearinghouses that have been created to identify different evidence-based programs models depending on your industry. Below is a list of sources to find evidence-based practices for several focus areas.

List of Evidence-Based Program Registries:

Perhaps you have a program that you think could be evidence-based and replicated in other communities. Most of the resources listed above are seeking applicants from promising practices to submit their program for inclusion in the respective registry. They outline specific criteria to meet their standards for inclusion.

Are you interested in getting help evaluating your current program to demonstrate that it is evidence-based? We can help you with our evaluation services.

Maybe you are interested in implementing an evidence-based program or seeing if one of them is right for you. There is no need to reinvent the wheel when there is an abundance of evidence-based programs to support focus areas like education, youth development and overall health and well-being. It’s all about finding a program that aligns with your organization’s mission and expertise. At Transform Consulting Group, we can assist organizations to find evidence-based practices that align with their goals. Contact us today for a consultation!

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

Four Tips to Use Your National Student Clearinghouse Data

NSC Data BlogIs your organization working to get more students to and through college? How are you tracking your results and their progress? For many schools and organizations, this can be difficult because you are not in direct communication with the students anymore.

National Student Clearinghouse

One solution to track post-secondary attainment rates is the National Student Clearinghouse (NSC). NSC has information covering about 96% of all U.S. enrollment in public and private collegiate institutions. The NSC data includes information on college enrollment, degree types, institution type and dropout and graduation rates.

NSC data will answer several of your key questions, including:

  • How many students are enrolling in post-secondary education and when (1st year after high school or 3 years later)?
  • What colleges or institutions are students selecting?
  • How long are students enrolling (it tracks by semester)?
  • What majors and minors are students selecting?
  • How many are withdrawing or dropping out?
  • How many are graduating?

Here at Transform Consulting Group, we have analyzed NSC for several organizations working to get more students to and through college, such as TeenWorks and the Center for Leadership and Development. We have identified 4 tips to help your school or organization collect, understand, and analyze NSC data.

1. PREPARE YOUR DATA

Before you can get an update on the status of your students/ alumni’s post-secondary information, you must first submit a data request to NSC. NSC is very specific about how they want to receive your student file. Review NSC’s specific format for submitting your data request.

2. REVIEW AND CLEAN DATA
Now that you have correctly submitted your data request to NSC, they will return an aggregate data sheet and a student data sheet.

The aggregate data sheet gives a summary of the entire data set, including the top institutions attended, 4-year versus 2-year enrollment, number and percent awarded degrees, public versus private attendance, and in-state versus out-of-state attendance.

NSC Aggregate Headings

When you see the returned student data sheet, it may seem overwhelming! The sheet breaks down 33 data points organized by column for each student found in the database. This data sheet will help you answer the above key questions.

To help avoid that overwhelming feeling, NSC created a “StudentTracker Detail Report Excel Guide for Colleges and Universities”, breaking down the detailed steps to help understand and use the information. Essentially, you will spend a significant amount of time cleaning and formatting the data. Then you can analyze the data in Excel (using pivot tables) or a data analysis software. We like to use Tableau to analyze our clients data, which you can read about here.

3. ALIGN DATA
Hopefully, now that overwhelmed feeling is gone! In order to take your NSC data to the next level and really understand what is working and not working at your school or organization related to postsecondary attainment, you need to combine your NSC data with some of your internal data. Take your returned student sheet data file and match it with some of your internal data. Then you could answer some additional key questions:

  • What impact did ______ program have on our students’ postsecondary rate?
  • Did students who were involved in more programs or multiple years at our organization have higher postsecondary rates?
  • Did different groups of students (low-income, first generation college grads, gender, race, geographic neighborhoods, etc.) have higher or lower postsecondary rates?

This further alignment and assessment of comprehensive data helps you identify which programs and services lead to the best or worst outcomes for students.

4. SHARE THE INFORMATION top majors CLD
Now what? You have your postsecondary outcomes with the context of additional program information, but how are you interpreting the data and sharing it internally and externally with key stakeholders? We recommend creating visuals to highlight the key data points. Studies show that a majority of people are visual learners. We suggest checking out a series of blogs Transform Consulting Group has written about creating infographics and other visuals to help tell the data story. Lastly, and most importantly, you can now make data informed decisions to improve your postsecondary outcomes!

Conclusion

Data not your thing? Still feeling overwhelmed by the NSC process and information? Transform Consulting Group is here to help. We have worked with several schools and organizations to help make sense of their postsecondary outcomes and National Student Clearinghouse data. Contact us today to learn more about how we can help!

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

Why is a Program Evaluation Impact Team so Important?

Impact Image- blogHow does an organization know it’s meeting its goals and objectives?  An outside team can be hired to put tools and systems in place, which is a good start, but investing money in outside consulting only can leave organizations floundering in a constantly evolving reality.  What if instead of investing solely in an outside group, an organization invests simultaneously in its own people?  It is exceedingly important to invest in the right team of people to help an organization to push forward, to evaluate and sustain systems, while keeping a “pulse” on the organization.  That’s what an impact team does.     

In a past blog, we talk about the 4 steps of a program evaluation. The impact team would work closely with the outside evaluator to complete the four steps and keep the evaluation cycle going beyond the consultation engagement! They are essentially your internal “CQI” or continuous quality improvement team.

An impact team is a cross-cutting team of staff that come together on a regular basis (e.g., quarterly) to ask critical questions, review data, and make meaning of the information; basically, to integrate program evaluation into the organization.  They would discuss how the data is being collected to ensure the processes and systems in place are being followed. They would also review the big goals identified in the logic model and discuss if the targets have been met and why/ why not.  Lastly, and most importantly, they would identify data-informed recommendations to improve the outcomes.

Transform Consulting Group worked closely with Center for Leadership and Development to implement a system for in-house evaluation and train their impact team to ensure integration of evaluation within the organization and cross-department alignment. Policies and procedures were put into place to guide system processes and outline impact team member roles and responsibilities. The team was trained on how to collect data using the agreed-upon tools.  A data management plan and schedule were created to align with the organization’s programming schedule and keep the impact team on track with data collection and analysis throughput the year.  This ensured new evaluation results would always be ready in accordance with important fundraising events and annual strategic planning sessions.

Who should be on an impact team?  The executive decision-makers?  All management?  Maybe just the staff who worked on the programs being evaluated?  Only if you want a myopic view of your organization’s reach.  The impact team should consist of staff at all levels to have different perspectives.  Information will be interpreted differently by each person on the team based on their unique capabilities, experiences and strengths.

What is interesting?  What stands out?  Some results may be more obvious to different members of the team.  This is what makes a great impact team.  Take time to think about the best people at each level of the organization and don’t be afraid to adjust those involved as things change.  An impact team will be engaged in the most crucial elements of the organization.  Make sure to include members capable of critical thinking and connecting the dots—systems level thinkers, not just task-masters.  Those who can perform high-level analysis, problem solving, and decision making are essential, but don’t exclude those doing work on the ground floor.  Their understanding of what it takes to execute the organization’s vision on a daily basis is invaluable.  Build a team from all levels that is excited, engaged, and willing to be honest about what works and what doesn’t.

This is the start of where the big decisions are made.  The informational “tools” to make big decisions start with the evaluation data (the hard evidence) and the folks who can interpret what the data is saying.  An organization’s impact team would present evaluation results and subsequent recommendations to the board and leadership team regularly.  Organizations are equipped with make data-informed recommendations for decision making, such as modifying programming elements, letting go of a program, refining the target population, requiring more professional development for program staff, adjusting program dosage for participants, and other organizational or programmatic changes.

Transform Consulting Group can help you create and sustain a winning program evaluation impact team for your organization.  Please contact us today to learn more!

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

4 Ways to Share Data Publicly

Congratulations! You successfully gathered data to look deeper into the effectiveness of your programs, decided WHO you’re sharing the information with and now it’s time to figure out HOW to share the data.

We talked about the different audiences you should share your data with in this blog, such as internally with staff and volunteers and externally with funders and partners. Once you determine your audience, you may decide on a variety of ways to communicate your latest information.

First, it’s important to know your audience and be willing to shape your message in a way that is easy to understand and compelling. Some things to consider about your audience may include their age, educational background, industry or non-industry, and economic levels. For example, the way you communicate to a wealthy, influential donor should look different than the way you communicate to your staff or clients.

Here are some examples of how you can share data with a variety of audiences:

  1. Screen Shot 2017-06-12 at 11.27.32 AMAnnual Report

An annual report is a great way to package your year’s efforts and data. You can use several visual elements to make important data stand out, include photos of the people you serve, while also including descriptive text to provide additional information. For tips on how to spice up your annual report, check out this blog

  1. Email BlastScreen Shot 2017-06-12 at 11.00.23 AM

Many of you are already regularly emailing your target audience, so consider using those email blasts to spotlight recent data. If your data shows positive results of a programs, then share it! Include photos or success stories from clients to highlight and make sure you’re presenting the facts in an “easy-to-read”, visual way.

We did this at TCG and sent out a “Year in Review” email blast to current and potential clients highlighting our efforts and successes throughout the calendar year.

  1. Marketing Materials

If your data is showing huge successes, then you want to make sure EVERYONE hears about it!

  • Brochures: Consider re-creating marketing materials to highlight specific results and outcomes beyond just the standard details about programs provided and “numbers served”.
  • Letterhead: Add a simple line at the bottom of your organization’s letterhead such as “95% of students enrolled in our summer programs saw significant improvements in test scores.”
  • Email Signatures: Have all staff members update their email signatures to include a link to the Annual Report or another statement on a specific data point.

The great thing about using marketing materials is that you are able to communicate data to people outside your circle. EVERYONE who receives an email or letter from you can see your data and it doesn’t have to be a person who already has a stake in your organization.

  1. Social Media Campaign

Your data tells a story about your organization and social media platforms are great for sharing!

  • Upload your Annual Report online and share the link often on your channels.
  • Share pieces of data from the annual report on a weekly basis through a specific data point or graphic.
  • Swap out your Facebook or Twitter banner to highlight a specific success.
  • Make your social media posts visual!  One tip to keep in mind: before you post long winded paragraphsScreen Shot 2017-06-12 at 10.50.34 AM packed full of numbers, try creating infographics to communicate the data in a visual way. You can then upload your infographics as photos on your social media channels and make it easy for your followers to share.

We did this recently when sharing 2016 stats for the Indiana Heart Gallery to our social media followers. While the infographic to the right doesn’t tell the whole story, it does gives a quick look at the numbers. For more tips on creating infographics, check out our past blogs here and here.

It’s important to note that the examples listed above are just a starting point. The great thing about sharing data and telling your story is that the possibilities for HOW you do it are endless. Get creative and don’t be afraid to try out new things!

Whether you’re at step one and need help gathering data or you already have great data compiled and need help sharing it, contact Transform Consulting Group today and we’d love to chat!

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

Our Go To Sources for Data

Whether you are working on a needs assessment, evaluating a program or starting a new project, reliable data is important to help make smart choices. Our Go to Sources for Data- Blog image2

Due to the internet, data is available at our fingertips, but the endless options can become overwhelming.  Have you ever spent hours sorting through links and data sources to find one specific indicator such as the number of individuals in your community with post-secondary education or the poverty rate for a certain age group?

At Transform Consulting Group, we are data nerds and can help point you in the right direction to find the data you need to move your organization forward. We have a cheat sheet of the key organizations and resources that we utilize to help us find the right data for our clients.

There are a few resources that provide comprehensive data and the majority of information that you need.  However, there may be times where you want to go deeper with the data in your understanding.  Therefore, you may need to use some additional resources to find the information you are seeking.

Comprehensive Child Data
  • Child Trends — Child Trends is a national nonprofit research organization focused on improving the lives of children, youth, and their families. Their databank features data by topic, including child maltreatment/welfare, early childhood, families, health, and social and emotional learning.
  • Screen Shot 2017-06-13 at 11.35.27 AMKids Count Data Center — Kids Count is a project of the Annie E. Casey Foundation and has comprehensive data on child and family well-being.  Each state has a designated entity to manage and compile the state Kids Count data, which is the Indiana Youth Institute for Indiana. Individual county profiles are available and updated annually. 
Education Data
  • State Departments of Social Services — Many state departments of social services or human services oversee child care for young children and school-age children.  In Indiana, the Family and Social Services Administration (FSSA) Office of Early Childhood and Out-of-School Learning (OECOSL) collect data on early childhood education and out-of-school time programs and would be the resource to identify the number of programs serving young children and/or provide before and after school care for school-age children as well as the cost of care and other important factors.
  • State Departments of Education — Each state department of education has information about individual schools, corporations and their students.  In Indiana, the  Indiana Department of Education (IDOE) shared this data online via “Compass”, which is a searchable database to access educational attainment, testing scores, and free/reduced lunch rates.  Additional information can also be requested.
  • Lumina Foundation — Lumina is committed to increasing the post-secondary attainment of Americans to 60%.  They have national, state and metro level data available.  Check out the recently released 2017 Stronger Nation Report! We use the Lumina data when working with our college and career readiness and Center for Working Families clients.
Population Data
  • U.S. Census Bureau — The Census Bureau provides quality data about the people and the economy. It features a few data resource tools, including Quick Facts, the American FactFinder, and the American Community Survey. Information is available at a variety of geographic levels, including national, state, county, city and town, township, region, census tract and more.
  • State Population Resources — Some states have created online resources that summarize the Census data for their state.  In Indiana, STATS Indiana is a great resource to find quick profiles and maps on population, housing, education, income and poverty, health and employment.  In Central Indiana, SAVI is another great resource with detailed information and profiles. SAVI features tools to analyze and visualize Central Indiana Community Data. These websites tend to be more user-friendly than the Census.
Health
  • State Departments of Health — Each state department of health has access to health specific information.  In Indiana, this is the Indiana State Department of Health (ISDH).  ISDH recently launched a new website, Stats Explorer, to make it easier to access health data at the state and county level.  Here you can find information about the prevalence of drug overdoses, sexually transmitted diseases, births and deaths, cancer, infectious diseases and many other health issues, all in one convenient location. Previously, the data was available through multiple pages on the ISDH website.
Economic
  • State Departments of Workforce Development — Each state has an agency or office focused on workforce development and will have a variety of information available for the public.  In Indiana, the Department of Workforce Development (DWD) features workforce and economic data on a separate website, Hoosiers by the Numbers. Here people can find data by state or county on economics, education, employment/ unemployment, wages, and labor force.

There are many more data sources available online as well as via special request. Once you collect the data, don’t forget to make it visually appealing for your audience! Check out our blogs to find some additional tips on using your data.  If it is still too overwhelming, give us a call to see how we can help gather the data you need to inform your work. 

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

You Finally Have Data! Who Should You Share It With and Why?

Data-BlogImagine your organization recently conducted a program evaluation.  Data was gathered from one of the programs offered and an analysis of the data revealed your program outcomes were met, some even exceeded!  This is exciting news and you want to shout it out from the nearest rooftop… We understand this urge, but there are better (and safer) ways to share your evaluation data.  In this blog, we discuss different audiences with whom an organization might share their evaluation results and the benefits of sharing them. Stay tuned for a future post discussing how to share your evaluation data.

At Transform Consulting Group, we love seeing organizations use data to measure and achieve success!  We also love helping organizations “visualize” success in easy-to-understand ways using simple charts and graphics.  Once these data visuals are audience-ready, what are the next steps?  Does an organization need to be intentional about sharing program evaluation data?  Why?

Data is only as good as how it is used.  If an organization collects data, analyzes it and makes beautiful visuals with it, but few people actually see the results, what is gained from the experience?  Probably not much!  This connects back to a blog we wrote called, 4 Steps to Complete a Program Evaluation.  Step number four in the process is to discuss the results of an evaluation and make data-informed decisions.  In order to do this, evaluation data must be shared with the right people.  

Our team typically compile a summary report and/or slide presentation of an organization’s evaluation data for stakeholders to review the results and discuss their implications.  A stakeholder is anyone who has a “stake”—an interest, concern, or investment—in an organization and/or program achieving (or not achieving) its goals.   We categorize data sharing into two different “buckets” -internal and external.  As it sounds, sharing data internally is with folks on the inside of an organization, and external data sharing is with folks on the outside looking in. Within each bucket, we’ve identified three important stakeholder groups and how sharing evaluation results can be beneficial, no matter what the data “says”.

Internal Stakeholders

1. Employees:  It would be challenging to find a dedicated employee who would not care to see evidence of their daily efforts actually paying off and leading to positive change.  The achievement of program outcomes reflected in data can help validate the combined efforts of all staff involved in a program’s design (if applicable) and delivery.  It is important to share evaluation results with staff at all levels and not just employees at the top.  Too often, we see that the information does not trickle down.  Sharing, showcasing and celebrating success builds morale and encourages staff to continue doing great work.  In the business sector, this would be akin to celebrating a top-sales month! 

On the other hand, results of an evaluation may reveal that a program is not achieving its intended outcomes and uncover potential reasons why not.  Evaluation data can “shed light” on issues that staff and/or leadership were not even aware of that could hinder the ability to make an impact.  Based on sharing these results, employees will know to expect workflow or programmatic changes in the near future without confusion or surprise.

At TCG, we recently worked with an organization that provides college and career readiness counseling for high-school students to evaluate their 8-week summer program.  During orientation, students were given a pre-survey to assess their knowledge of college and career readiness subject matter before participating in the program.  On the last day, students were given a post-survey to measure knowledge gained as a result of completing the program.  Evaluation data was presented to program staff and leadership in easy to understand charts and graphs.  Staff became excited when they saw how much college and career readiness knowledge students gained as a result of participating in the program!  They were also glad to find out which areas students reported knowing the least about.  As a result, staff could strengthen those areas of the program prior to the next round of incoming students.  

2. Volunteers:  Organizations that depend on volunteer work are always on the lookout for more help.  Unfortunately, unpaid labor can be hard to come by, especially if volunteers don’t feel like their time and effort is making a difference.  If an organization can show community impact as a result of volunteerism, their volunteers are more likely to feel validated and remain committed to the organization.  Then they may even recruit more volunteers!  In this case, evaluation data helps to promote satisfaction and the feeling of reward.  Volunteers are motivated to stay put and not seek out other volunteer opportunities when they know that their time and talent is making a difference.

3. Board of Directors:  The Board can use evaluation data to begin planning and discussing the future of an organization.  If the data supports outcomes consistently being met, a Board may decide to expand the program’s service delivery to a larger area or broader audience.  If the expansion is successful, the organization sustains a larger community impact and the Board (hopefully) feels a sense of accomplishment and pride in the organization.

For evaluations resulting in unmet program outcomes, the Board may recommend program model changes, or commit to focus on a problem area (i.e. staff professional development) during the upcoming year.  Either way, sharing program evaluation results with Board members equips them to make informed decisions about what is best for the organization going forward.

External Stakeholders

1. Funders:  Funders want to see a return on their investment into any organization.  Many funders require organizations to conduct program evaluations to remain accountable for the results that their funds are directly supporting.  Bottom line, funders want to know if the program is worth the resources that it costs.  Evaluation results help “quantify” worth for funders by showcasing to what extent the desired program changes are occurring.  If a program performs as expected, it is likely that funding dollars will keep flowing.  

Alternatively, if evaluation data shows program outcomes not being met and insignificant or no change is occurring, a funder may decide not to invest or discontinue investing in an organization.  Funders are often supporting multiple organizations at once and want to feel confident their money is well-spent and producing the best results!

2. Partners:  Evaluation data may help make a case for two or more organizations to join forces and provide combined programs or services within a community to maximize impact.  Sharing data can be a positive step in the direction of collaborating and working towards common goals.  Local programs may unknowingly be competing for clients, resources or limited funding.  Some are likely struggling to meet goals.  Awareness of such issues, made apparent through evaluation data, could spark ideas to collaborate and leverage partnerships to provide joint programming.  The result?  Sharing data may lead organizations to do what is best for the community, while also doing what is best for their budget!  

Ideally, routine program evaluation and data sharing will keep organizations accountable to each other in the future.  On the other hand, evaluation data can show when partnerships are no longer working and should be realigned or dissolved.

3. The Public:  Sharing data with communities helps to legitimize an organization’s purpose in the public eye.  Program evaluations are one way to demonstrate community impact.  Evaluation data goes a step farther to show how much impact an organization or program is making.  Nonprofit organizations are public agencies with a responsibility to communicate back to the public about the goodwill that they are making from the public’s investment in them!

Foundations, grant makers and other funders research shared public data about an organization or program to determine whether or not to invest in it.  Evidence of successful programming can suggest an organization itself is well-managed.  This inspires confidence that funding dollars will also be used wisely and generate the greatest return on investment.

Internally, sharing evaluation results can galvanize an organization–arming employees (program staff and management) and the Board of Directors with data they need to ensure program outcomes are being met and take appropriate actions when issues arise.  External stakeholders use data to verify an organization’s credibility and hold it accountable to the outcomes it seeks to achieve.  Ultimately, stakeholders use data to keep an organization on track to accomplish its overall goals.

It’s hard to win a game if your team has no idea of the score—or even worse, what game they are playing.  Sharing evaluation data helps to keep an organization’s impact transparent and everyone involved on the same page.  Stay tuned for our next blog in this series about how to share data and use it to tell your story within different communication channels!  Talk with one of our team members today and learn how you can get the word out about your evaluation results!

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

4 Steps to Create a Dashboard

In today’s information age, organizations are overwhelmed with the amount of information that they collect, track and monitor.  Non-profit leaders must decipher through all the data to determine what is meaningful and relevant to share with staff, funders, Board of Directors and other community partners.  A dashboard is a great tool to bring all the critical elements together in a user-friendly report.

Through our program evaluation and research and analysis services, we help organizations create dashboards.  Here are a four steps to create an effective dashboard:

  1. Determine the audience for the dashboard. A dashboard is customized for the audience meant to view and use the dashboard, so first an organization needs to determine the intended audience. Then an organization needs to determine the key takeaways that you want the targeted audience to get from this dashboard.  Lastly, the organization should really focus on the information that is most important and relevant for this audience.
  1. Decide on what the dashboard is tracking now that the audience is determined. A dashboard is meant to communicate progress over time, such as monthly, quarterly or annually. In addition, data in the dashboard can be used to compare different data sets, such as geographic locations, sites or populations. These factors need to be determined to provide the appropriate context for decision makers.
  1. Determine the visuals that will be most effective in communicating the message. In most cases, we work to fit a dashboard on one-page. This does not provide much “real estate”, so you must be intentional about the visuals used to grab the audience’s attention and display the key messages [Sidebar: this is why we use Tableau!]. A dashboard does not have much room for wording and explanation.
  1. Determine the delivery of the dashboard. In most cases, dashboards are “static” or print reports that are shared via handouts or electronically. However, with the growing development of software programs more interactive dashboards are being created.  In some cases, there may be value in creating both a static dashboard that is completed annually and an interactive dashboard that is updated real-time.

Dashboard Blog imageAs we shared in this blog post about creating a needs assessment and annual report, we mentioned the state dashboard and county profile that we created for ELAC.  After conducting the first state needs assessment on young
children, ELAC realized that the amount of data and information was overwhelming. ELAC was inspired by the Annie E. Casey Foundation’s 16 indicator dashboard in their annual data book and used this to create ELAC’s dashboard.

Following the four steps above and inspired by the Annie E. Casey Foundation’s dashboard, we worked with ELAC to create a state dashboard that:

  • Identifies four focus areas related to children and families, high-quality care programs, education workforce, and kindergarten readiness with 16 key indicators.
  • Compares progress over time (dependent upon the release of new/ old data)
  • Uses arrows (a visual tool) to depict if the numbers represent improvement (arrow goes up) or worsening (arrow goes down).

After a few years of creating the ELAC state dashboard, communities across the state of Indiana were asking for this same data at the local level.  We worked with ELAC and our state data partners to gather the county level information to create a two-page county dashboard and profile.  The first page of the county profile mirrors the state dashboard with a few exceptions.

Instead of comparing progress over time, the county dashboards compare the county data to the state data.  Following steps #1 and #2 above, we focused on the audience for the county dashboards who said that having their data in the context of the state data would be helpful to know if they are doing better or worse.  Therefore, that ranked as a higher priority than comparing their data over time.

The second page of the ELAC county dashboard was new and provided the opportunity to add visuals (charts and graphs) to depPicture1ict the key findings in the full narrative report.  The visuals help to communicate complex information in simple charts.

Using a data visualization software program like Tableau is critical to not only help make the dashboards visually appealing but also to automate the process.  In this case, we created 93 unique dashboards for the state and all 92 counties.  While the ELAC dashboards are currently only static reports, there is the option and feature (with Tableau) of making them interactive like the Indiana Commission of Higher Education’s College Readiness Dashboard. One of our good friends, the M.A. Rooney Foundation, has also been working to transpose K-12 data for schools and community partners into meaningful dashboards.
Are you ready to get started in creating a dashboard for your organization? We would love to work with you to help you focus in on the key indicators important for your organization and create a dashboard that informs decision-making! Contact us today for help.

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

4 Steps to Complete a Program Evaluation

Program evaluation helps organizations determine if the change that they set to accomplish actually occurred. Change can be knowledge gained, attitude change, or behavior change.  For example, did a literacy tutoring program help the students who were not reading on grade level actually catch up to reading on grade level by the end of the program period?  Or, are the low-income children who participated in a high-quality pre-k program ready for kindergarten?  

At Transform Consulting Group, we are data nerds and love helping organizations develop and implement evaluation plans to assess their change by using a four-step process.
4 eval steps

Program Evaluation in Four Steps:

  1. Establish clear outcomes
  2. Create or modify data tools and systems
  3. Analyze the data
  4. Use data to make informed decisions

The first step is to ensure there are clear outcomes in place that support an organization or program’s goals.  We work to develop SMART outcomes: specific, measurable, achievable, relevant, and timely.  In this step, we typically review or develop a logic model, which is just a technical description of aligning your programs to the change (or outcomes) that you want to accomplish.  For example, a college-readiness program may want to increase the number of students who can (1) identify a major or career they are interested in pursuing after high school; or (2) understand how to apply for college financial aid.  

The second step focuses on having the right data tools and systems in place to measure and report back on the designated outcomes.  Transform Consulting Group helps organizations determine the most appropriate tool(s) to collect, track and monitor accomplishment of the identified outcomes.  We consider organizational capacity (staff time, knowledge and budget) when suggesting data tools. Some examples could include participant surveys, assessments, and student academic records.  We work to have data tools that are valid and reliable and will provide the data necessary to monitor progress.

The third step is to analyze the data once it has been collected, and present the results in an easily-understood format.  Data is measured to determine whether program outcomes were met and the change, if any, that occurred.  This is often the step where organizations get stuck, because they don’t have the staff time or knowledge to complete the analysis. [Stay tuned for a separate post on simple ways to analyze your data!]  We tell our clients that this is the fun part, because we can see if what they set out to accomplish actually occurred!  

Some common research questions that drive many organizations to conduct program evaluations to get answers include:

  • Are program participants being reached as intended?  If yes, why?  If not, we also want to know why.
  • To what extent are desired program changes occurring?  Was there a significant difference or just a small difference?  Is there a specific group that is not being impacted?
  • Is the program worth the resources it costs?  What is the “return on investment” for this program or service?

The fourth step is to discuss the results of the program evaluation and make informed decisions based on what the data tells us.  We will compile a summary report and/or slide deck presentation of the evaluation data for stakeholders internally and externally to review the results and discuss the implications.  Good evaluations often lead to recommendations for improvement such as enhanced professional development, diversified participant recruitment strategies, and/or program model changes.  This is an opportunity to discuss data collected and implications for future programming, including ongoing program evaluation practices within the organization.

In today’s era of accountability, what gets measured gets done. If you don’t measure results, you can’t tell success from failure[1].  Transform Consulting Group equips organizations to celebrate their successes and inform opportunities for growth.  Contact us today for more information on how Transform Consulting Group can help assess the impact of your organization.  
[1] Reinventing Government, Osborne and Gaebler, 1992.

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

Using A Needs Assessment Process to Plan a New Program

When a business plans to open a new store or restaurant, the business first conducts market research to ensure a good fit and a successful venture. In the public sector, we call this a “needs assessment”.

Whether your organization is a non-profit, hospital, school or government agency, more than likely you have been asked or initiated a needs assessment.  Why?  The needs assessment is a great opportunity to step back and build understanding about the target population (aka, intended client), their strengths, needs, concerns and goals, as well as taking a look internally at your own organization to ensure a good fit.

In Indianapolis, we have a Center for Working Families (CWF) program model that was developed by the Annie E. Casey Foundation to help low-income families achieve long-term financial stability. A group of local funders and partners help support and coordinate the CWF model to ensure a good fit with communities and strong fidelity at implementation.

Transform Consulting Group has helped a few organizations complete a needs assessment and project implementation plan to implement CWF in Indianapolis.  Most recently, we worked with Shepherd Community Center, a multi-service center on the Near Eastside of Indianapolis. Achieving CWF status would designate Shepherd Community Center as a location where low-income families could access a coordinated or bundled set of three key services to help lift them out of poverty and achieve long-term financial stability. In order to demonstrate the need for CWF in the neighborhood that Shepherd Community Center supports, Transform Consulting Group conducted a community needs assessment within the Near Eastside neighborhood of Indianapolis and completed an internal assessment of the organization.

Completing the needs assessment allowed Transform Consulting Group staff an opportunity to connect with the target audience through in-person focus groups and surveys at key community gatherings as well as meetings with key community partners.  We also gathered key indicators from the Census and other external data sources. The information collected included demographics, income levels, education levels and goals, goals, and perceived gaps in community services. Combined with the stakeholder feedback, we developed a comprehensive profile of the targeted community.   Data was analyzed and key findings shared with Shepherd Community Center leadership.  Results from the needs assessment not only demonstrated a strong need in the community, but will also serve as a baseline of information for future CWF planning and programming within this community.

Understanding the needs of your community or “market” is the foundation for successful organizational programming that will have the greatest, long-lasting impact. Transform Consulting Group looks forward to new opportunities helping organizations like Shepherd Community Center get a pulse on their community needs and effectively provide support. Contact Transform Consulting Group today to learn more about how we can assist your organization.

 

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn