Category Archives: Evaluation

How to Measure Character and Leadership Development

In many youth development and education programs, improving character and leadership development are common goals. Measuring them can become challenging due to their close relationship and broad understanding. This is why it is important to establish clear definitions of character development and leadership development before completing an evaluation.

We are currently working with a college and career readiness program,  the Center for Leadership Development (CLD), on their internal data collection and evaluation process. Two key principles of CLD are character development and leadership effectiveness. In order to determine the best way to measure these two focus areas, we walked CLD Leadership through a process where the team defined the two key focus areas, assessed their alignment across the 13 different programs and services, developed short-term IMG_0398outcomes that align with the definitions, and mapped their alignment to long-term aspirational outcomes.

1. Establish Clear Definitions

The Center for Leadership Development has a model that utilizes principles like character development and leadership effectiveness to measure the college and/or career readiness of its participants. In order to effectively measure the two principles separately, the team needed to establish clear definitions of character development and leadership effectiveness for the organization. For example, one-character trait of college and career readiness is discipline. One must be disciplined in order to complete assignments on time whether it be in a school or professional setting.

2. Ensure Alignment with Programming

To ensure programs had the proper alignment with the core principles, the CLD leadership team identified programs with strong focuses on character development and leadership effectiveness. We walked the leadership team through each program to ensure the programs focuses aligned with the two principles and their definitions.  This process creates an understanding with the team and makes sure the measures are connected to the curriculum or services.

3. Develop Short-term Outcomes

Once character development and leadership effectiveness were defined, CLD’s leadership team began to brainstorm short-term outcomes for these focus areas. These short-term outcomes were developed based on the established definitions. For example, the CLD leadership team identified self-discipline as part of what defines character development for the organization. If the program has a focus on character development, there should be a short-term outcome that will provide a measure in order to show an increase, decline or no change in the self-discipline of a participant. This will help the staff understand if the curriculum and program is addressing these specific areas and moving closer to accomplishing its goals.

4. Align to Long-term Outcomes

The CLD leadership team also needed to ensure that the principles aligned with the long-term goals of the organization. The team reviewed research and discussed the rationale behind why character and leadership development are critical components to propel students to a college and career path. The goal of ensuring alignment between the principles and long-term goals is to avoid losing sight of the long-term vision and impact, and to make sure everything remains connected.

Many programs that focus on developing strong character and leadership abilities in youth focus on these efforts simultaneously. It can be challenging to measure character traits and leadership traits separately when character development sets the foundation for building strong leadership skills. Although this challenge exists, organizations can overcome this hurdle by first establishing clear, organizational definitions in order to move forward with creating short-term outcomes to measure the impact of a program and the organization as a whole. We at Transform Consulting Group are here to support the data and evaluation needs for your organization. Contact us today!  

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

Summarizing the 2018 ELAC Interactive Annual Report

Indiana’s Early Learning Advisory Committee (ELAC) recently released its new 2018 Annual Report that was discussed in this blog post.

interactive-reportThis year, a new feature was developed along with the Annual Report – an Interactive Annual Report dashboard using Tableau. This dashboard allows the audience to take a deeper dive into the early childhood education data included in the Annual Report.

While the visualization of a data dashboard can be exciting, it can also be overwhelming. The interactive dashboards were inspired by the Indiana Commission of Higher Education’s use of dashboards. ELAC saw the opportunity to share the data that has been collected in a user-friendly format for community stakeholders. When you are browsing the new interactive dashboard, make sure to check out these eight key features!

8 Key Features of the ELAC Interactive Dashboard:

  1. There are five main sections of the dashboard: (1) Young Children and Families (2) Accessibility (3) High-Quality (4) Affordability and (5) Kindergarten Readiness. Simply, select the rectangle tab for the section you want to see.
  2. Data is compiled from multiple sources: The data that ELAC reports comes from multiple sources. A dashboard is a good format to pull together multiple data points and present it in a user-friendly format. ar-sources
  3. Each chart is included to answer a key research question: Check out the gray boxes to identify the questions that the data is answering. This can guide the information that you are seeking to find.
  4. Different charts are utilized to visualize the data: Each tab includes a variety of charts to answer the key research questions. For example, maps are included in each section to display how the data varies across the state.
  5. Data can be filtered by different categories: Charts have the option of being filtered by location, age or program type. There are filtering options throughout the dashboard, at the top of pages or along individual charts. filters
  6. Definitions and data sources are included: The Accessibility, High-Quality and Affordability tabs feature a “Hover for Help” option at the top of the page that features definitions related to content on the page. Throughout any page of the dashboard, hover over charts for more data and definitions. hover-example
  7. Data includes a ranking of counties: Each tab features a county ranking chart to help counties easily identify how they compare to the rest of Indiana’s 92 counties.
  8. Data can be shared or downloaded: The Tableau Toolbar is located on the bottom right of the dashboard. You can click on it Undo/ Redo/ Reset filters applied. You can share the dashboard with the url link and also via social channels, and you can also download it as a PDF.

If you have questions or comments about the ELAC Interactive Annual Report dashboard, email elacindiana@gmail.com or contact Transform Consulting Group.

Like what you see? Transform Consulting Group can help your organization develop a data dashboard customized to your needs. Contact us today for a consultation!

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

How Are Indiana’s Youngest Children Doing? The 2018 ELAC Annual Report Gives Insight.

Indiana’s Early Learning Advisory Committee (ELAC) just released its 2018 Annual Report—the fifth since ELAC’s inception in 2013. Annually, ELAC completes a needs assessment for the state’s early learning system and recommends solutions. The goal is to baseline where Indiana is using key indicators and to make best practice recommendations to address the gaps. The result of this year’s annual needs assessment is three key reports and tools: 

ELAC’s seven appointed members work alongside 150 workgroup volunteers who focus on different aspects of the state’s early learning system. All this energy centers on providing early childhood care and education that is accessible, high-quality, and affordable to all families.

Screen Shot 2018-01-05 at 11.40.14 AM

How Are Children Ages 0-5 Doing Today?

  • Of the 506,761 children in Indiana ages 0-5, 65% need care because all parents are working. This includes working parents who are single as well as households where both parents work outside the home.Figure 3
  • Of those children who need care, only 41% are enrolled in known programs. The other three-fifths of children are in informal care settings—with a relative, friend, or neighbor—where the quality of care is unknown.
  • Of the young children who need care, only 15% are enrolled in high-quality programs. A high-quality program not only ensures that children are safe, but also supports their cognitive, physical, and social-emotional development for kindergarten readiness and beyond.

What Are Some Of Indiana’s Accomplishments On Behalf Of Young Children?Figure 15

  • There are more high-quality early childhood care and education programs available. In 2012, Indiana had just over 700 high-quality programs. There are now almost 1,200.
  • Today there are 4.5 times more children enrolled in high-quality programs than there were five years ago.
  • Over half of the counties increased their number of high-quality programs.

What Is The Unmet Need Identified In The 2018 ELAC Annual Report?

  • There are communities in Indiana with no high-quality programs.
  • The tuition cost of high-quality early childhood care and education programs is unaffordable, and the available financial assistance for low-income families is  insufficient.
  • There is a lack of high-quality seats for infants. Only 7% of children ages 0-5 in high-quality programs are infants. Tuition Comparison

How Can I Find Out More?

  • As in past years, ELAC has published a full annual report, which includes statewide data on Indiana.
  • ELAC has also compiled updated 2018 county-level data for all 92 Indiana counties to aid local stakeholders and coalitions in their work. Use the map to select your county. You can review your county’s profile in an interactive dashboard or a PDF report!
  • There is a newly created feature this year! ELAC published an interactive dashboard with all of the data in the annual report—allowing you to learn more about specific data points and easily present data to stakeholders. There are also comparisons between counties to see how well your community is doing compared to others.

Transform Consulting Group is proud to support ELAC’s work to help each of our youngest learners reach their full potential!

Transform Consulting Group can also help your organization or coalition with data analysis, creating dashboards to visualize your data, and meaningful reporting. Contact us to multiply your impact!

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

Does Your Organization Have a Continuous Quality Improvement Process?

During a time where communities and policies are changing, it is important to ensure the programs and services within those communities are constantly evolving to meet the needs of families. The Continuous Quality Improvement (CQI) model is an on-going process for organizations to be able to determine whether or not a change made led to an improvement in quality. In order to move towards making the necessary improvements, a review of what occurred is conducted through a CQI process like the Plan-Do-Study-Act (PDSA) cycle.

Steps to Complete a PDSA Cycleplan-act-do-study-cycle4

At Transform Consulting Group, we utilize this consistent approach when working with organizations to help them find solutions.

Plan:

Before beginning your PDSA cycle, you will need to have identified a problem you would like to address for quality improvement. The problem identified will guide your purpose for the review. Once you have chosen a change idea to focus on you will need to go through the following steps to plan to test your idea for change:

  • Define the goals
  • Define your research question(s)
  • Make predictions
  • Determine details for implementation of change or intervention
  • Plan of action for data collection

During this phase, we work with the client to thoughtfully plan to implement their new or current program/ service. So often, organizations jump immediately to step 2 – “Do” without completing this critical first step.  During the Planning, we define what we hope to accomplish especially if we are proposing a change.  Then we determine how the proposed change/ intervention will be implemented and work through all of the details.  Lastly, we finalize how data will be collected.

Do:

This phase of the PDSA cycle requires you to conduct the test for the change or intervention. It is during this phase that you will complete the following tasks:

  • Carry out the intervention
  • Collect data
  • Begin data analysis

This step in the process is what most organizations know and are doing. Organizations are delivering interventions everyday with their services. They might be intentionally or unintentionally modifying their intervention.  The “Do” step in this process is not new to organizations.  It is wrapping it around the other three steps that make this work transformational!

Study:

The study phase of the cycle is the time where once you have completed your intervention, you analyze the data and study what did or did not occur.   Organizations will want to review their predictions and assumptions made before conducting the test. You will want to take the following steps during this phase of the cycle:

  • Complete data analysis
  • Compare data to predictions
  • Summarize the information

Organizations often skip over this step in the process or do not spend enough time thoughtfully reviewing the data. For some organizations, their data can be considered “high stakes” and there is a tendency to want to focus on the positive changes/ results that occurred and glance over the changes that did not occur or the benchmarks that were not met.  During this phase, it is so important for an organization to be transparent and honest with themselves when reviewing the data.

Act:

Based on the summarized information, this last phase of the cycle allows you to determine what modifications may need to be made to ensure that the goals you set will be met. Your organization may decide to modify a program element or change how a service is delivered; you may decide to target a different population or use a new curriculum.  Once you have determined whether or not to adapt, adopt, or abandon your intervention, you will be prepared to do the following:

  • Plan next cycle
  • Decide whether the change can be implemented

During this last step, your organization takes all of the information gathered to make data-informed decisions that will ultimately improve your results.  This is the exciting part of the process, and one that you don’t want to skip. This step and the overall PDSA process will help your organization continue to improve the quality of services provided and impact in the community.

In this blog, “Is it time to redesign your program?”, we shared several examples of clients who we used the PDSA process to help them test and implement new interventions/ modifications to make to their programs to improve their outcomes. The CQI process allows organizations to have a plan of action once a problem or service gap has been identified.

At Transform Consulting Group, we follow this consistent approach when helping you find solutions to accelerate your impact. If you are looking to improve the quality of a service or program to facilitate positive change, contact us today!

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

How to Find an Evidence-Based Program

What Works Image (1)Often in writing grants and implementing programs, funders want to invest in “what works” and has been proven effective. Implementing an evidence-based program is a way to demonstrate proven results. If your organization wants to prevent child abuse, you might consider implementing an evidence-based home visiting program like Nurse Family Partnership or Healthy Families of America that has shown to prevent child abuse and promote parent-child engagement.

Across the country more funders, including the federal government have been focused on investing in evidence-based programs as well as expanding the number of evidence-based programs across different sectors. The reasons might vary, but funders want are committed to improving the different social issues affecting individuals and communities across the country.

How do you know what programs are evidence-based or are available to replicate in your community? There are some national clearinghouses that have been created to identify different evidence-based programs models depending on your industry. Below is a list of sources to find evidence-based practices for several focus areas.

List of Evidence-Based Program Registries:

Perhaps you have a program that you think could be evidence-based and replicated in other communities. Most of the resources listed above are seeking applicants from promising practices to submit their program for inclusion in the respective registry. They outline specific criteria to meet their standards for inclusion.

Are you interested in getting help evaluating your current program to demonstrate that it is evidence-based? We can help you with our evaluation services.

Maybe you are interested in implementing an evidence-based program or seeing if one of them is right for you. There is no need to reinvent the wheel when there is an abundance of evidence-based programs to support focus areas like education, youth development and overall health and well-being. It’s all about finding a program that aligns with your organization’s mission and expertise. At Transform Consulting Group, we can assist organizations to find evidence-based practices that align with their goals. Contact us today for a consultation!

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

Four Tips to Use Your National Student Clearinghouse Data

NSC Data BlogIs your organization working to get more students to and through college? How are you tracking your results and their progress? For many schools and organizations, this can be difficult because you are not in direct communication with the students anymore.

National Student Clearinghouse

One solution to track post-secondary attainment rates is the National Student Clearinghouse (NSC). NSC has information covering about 96% of all U.S. enrollment in public and private collegiate institutions. The NSC data includes information on college enrollment, degree types, institution type and dropout and graduation rates.

NSC data will answer several of your key questions, including:

  • How many students are enrolling in post-secondary education and when (1st year after high school or 3 years later)?
  • What colleges or institutions are students selecting?
  • How long are students enrolling (it tracks by semester)?
  • What majors and minors are students selecting?
  • How many are withdrawing or dropping out?
  • How many are graduating?

Here at Transform Consulting Group, we have analyzed NSC for several organizations working to get more students to and through college, such as TeenWorks and the Center for Leadership and Development. We have identified 4 tips to help your school or organization collect, understand, and analyze NSC data.

1. PREPARE YOUR DATA

Before you can get an update on the status of your students/ alumni’s post-secondary information, you must first submit a data request to NSC. NSC is very specific about how they want to receive your student file. Review NSC’s specific format for submitting your data request.

2. REVIEW AND CLEAN DATA
Now that you have correctly submitted your data request to NSC, they will return an aggregate data sheet and a student data sheet.

The aggregate data sheet gives a summary of the entire data set, including the top institutions attended, 4-year versus 2-year enrollment, number and percent awarded degrees, public versus private attendance, and in-state versus out-of-state attendance.

NSC Aggregate Headings

When you see the returned student data sheet, it may seem overwhelming! The sheet breaks down 33 data points organized by column for each student found in the database. This data sheet will help you answer the above key questions.

To help avoid that overwhelming feeling, NSC created a “StudentTracker Detail Report Excel Guide for Colleges and Universities”, breaking down the detailed steps to help understand and use the information. Essentially, you will spend a significant amount of time cleaning and formatting the data. Then you can analyze the data in Excel (using pivot tables) or a data analysis software. We like to use Tableau to analyze our clients data, which you can read about here.

3. ALIGN DATA
Hopefully, now that overwhelmed feeling is gone! In order to take your NSC data to the next level and really understand what is working and not working at your school or organization related to postsecondary attainment, you need to combine your NSC data with some of your internal data. Take your returned student sheet data file and match it with some of your internal data. Then you could answer some additional key questions:

  • What impact did ______ program have on our students’ postsecondary rate?
  • Did students who were involved in more programs or multiple years at our organization have higher postsecondary rates?
  • Did different groups of students (low-income, first generation college grads, gender, race, geographic neighborhoods, etc.) have higher or lower postsecondary rates?

This further alignment and assessment of comprehensive data helps you identify which programs and services lead to the best or worst outcomes for students.

4. SHARE THE INFORMATION top majors CLD
Now what? You have your postsecondary outcomes with the context of additional program information, but how are you interpreting the data and sharing it internally and externally with key stakeholders? We recommend creating visuals to highlight the key data points. Studies show that a majority of people are visual learners. We suggest checking out a series of blogs Transform Consulting Group has written about creating infographics and other visuals to help tell the data story. Lastly, and most importantly, you can now make data informed decisions to improve your postsecondary outcomes!

Conclusion

Data not your thing? Still feeling overwhelmed by the NSC process and information? Transform Consulting Group is here to help. We have worked with several schools and organizations to help make sense of their postsecondary outcomes and National Student Clearinghouse data. Contact us today to learn more about how we can help!

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

Why is a Program Evaluation Impact Team so Important?

Impact Image- blogHow does an organization know it’s meeting its goals and objectives?  An outside team can be hired to put tools and systems in place, which is a good start, but investing money in outside consulting only can leave organizations floundering in a constantly evolving reality.  What if instead of investing solely in an outside group, an organization invests simultaneously in its own people?  It is exceedingly important to invest in the right team of people to help an organization to push forward, to evaluate and sustain systems, while keeping a “pulse” on the organization.  That’s what an impact team does.     

In a past blog, we talk about the 4 steps of a program evaluation. The impact team would work closely with the outside evaluator to complete the four steps and keep the evaluation cycle going beyond the consultation engagement! They are essentially your internal “CQI” or continuous quality improvement team.

An impact team is a cross-cutting team of staff that come together on a regular basis (e.g., quarterly) to ask critical questions, review data, and make meaning of the information; basically, to integrate program evaluation into the organization.  They would discuss how the data is being collected to ensure the processes and systems in place are being followed. They would also review the big goals identified in the logic model and discuss if the targets have been met and why/ why not.  Lastly, and most importantly, they would identify data-informed recommendations to improve the outcomes.

Transform Consulting Group worked closely with Center for Leadership and Development to implement a system for in-house evaluation and train their impact team to ensure integration of evaluation within the organization and cross-department alignment. Policies and procedures were put into place to guide system processes and outline impact team member roles and responsibilities. The team was trained on how to collect data using the agreed-upon tools.  A data management plan and schedule were created to align with the organization’s programming schedule and keep the impact team on track with data collection and analysis throughput the year.  This ensured new evaluation results would always be ready in accordance with important fundraising events and annual strategic planning sessions.

Who should be on an impact team?  The executive decision-makers?  All management?  Maybe just the staff who worked on the programs being evaluated?  Only if you want a myopic view of your organization’s reach.  The impact team should consist of staff at all levels to have different perspectives.  Information will be interpreted differently by each person on the team based on their unique capabilities, experiences and strengths.

What is interesting?  What stands out?  Some results may be more obvious to different members of the team.  This is what makes a great impact team.  Take time to think about the best people at each level of the organization and don’t be afraid to adjust those involved as things change.  An impact team will be engaged in the most crucial elements of the organization.  Make sure to include members capable of critical thinking and connecting the dots—systems level thinkers, not just task-masters.  Those who can perform high-level analysis, problem solving, and decision making are essential, but don’t exclude those doing work on the ground floor.  Their understanding of what it takes to execute the organization’s vision on a daily basis is invaluable.  Build a team from all levels that is excited, engaged, and willing to be honest about what works and what doesn’t.

This is the start of where the big decisions are made.  The informational “tools” to make big decisions start with the evaluation data (the hard evidence) and the folks who can interpret what the data is saying.  An organization’s impact team would present evaluation results and subsequent recommendations to the board and leadership team regularly.  Organizations are equipped with make data-informed recommendations for decision making, such as modifying programming elements, letting go of a program, refining the target population, requiring more professional development for program staff, adjusting program dosage for participants, and other organizational or programmatic changes.

Transform Consulting Group can help you create and sustain a winning program evaluation impact team for your organization.  Please contact us today to learn more!

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

4 Ways to Share Data Publicly

Congratulations! You successfully gathered data to look deeper into the effectiveness of your programs, decided WHO you’re sharing the information with and now it’s time to figure out HOW to share the data.

We talked about the different audiences you should share your data with in this blog, such as internally with staff and volunteers and externally with funders and partners. Once you determine your audience, you may decide on a variety of ways to communicate your latest information.

First, it’s important to know your audience and be willing to shape your message in a way that is easy to understand and compelling. Some things to consider about your audience may include their age, educational background, industry or non-industry, and economic levels. For example, the way you communicate to a wealthy, influential donor should look different than the way you communicate to your staff or clients.

Here are some examples of how you can share data with a variety of audiences:

  1. Screen Shot 2017-06-12 at 11.27.32 AMAnnual Report

An annual report is a great way to package your year’s efforts and data. You can use several visual elements to make important data stand out, include photos of the people you serve, while also including descriptive text to provide additional information. For tips on how to spice up your annual report, check out this blog

  1. Email BlastScreen Shot 2017-06-12 at 11.00.23 AM

Many of you are already regularly emailing your target audience, so consider using those email blasts to spotlight recent data. If your data shows positive results of a programs, then share it! Include photos or success stories from clients to highlight and make sure you’re presenting the facts in an “easy-to-read”, visual way.

We did this at TCG and sent out a “Year in Review” email blast to current and potential clients highlighting our efforts and successes throughout the calendar year.

  1. Marketing Materials

If your data is showing huge successes, then you want to make sure EVERYONE hears about it!

  • Brochures: Consider re-creating marketing materials to highlight specific results and outcomes beyond just the standard details about programs provided and “numbers served”.
  • Letterhead: Add a simple line at the bottom of your organization’s letterhead such as “95% of students enrolled in our summer programs saw significant improvements in test scores.”
  • Email Signatures: Have all staff members update their email signatures to include a link to the Annual Report or another statement on a specific data point.

The great thing about using marketing materials is that you are able to communicate data to people outside your circle. EVERYONE who receives an email or letter from you can see your data and it doesn’t have to be a person who already has a stake in your organization.

  1. Social Media Campaign

Your data tells a story about your organization and social media platforms are great for sharing!

  • Upload your Annual Report online and share the link often on your channels.
  • Share pieces of data from the annual report on a weekly basis through a specific data point or graphic.
  • Swap out your Facebook or Twitter banner to highlight a specific success.
  • Make your social media posts visual!  One tip to keep in mind: before you post long winded paragraphsScreen Shot 2017-06-12 at 10.50.34 AM packed full of numbers, try creating infographics to communicate the data in a visual way. You can then upload your infographics as photos on your social media channels and make it easy for your followers to share.

We did this recently when sharing 2016 stats for the Indiana Heart Gallery to our social media followers. While the infographic to the right doesn’t tell the whole story, it does gives a quick look at the numbers. For more tips on creating infographics, check out our past blogs here and here.

It’s important to note that the examples listed above are just a starting point. The great thing about sharing data and telling your story is that the possibilities for HOW you do it are endless. Get creative and don’t be afraid to try out new things!

Whether you’re at step one and need help gathering data or you already have great data compiled and need help sharing it, contact Transform Consulting Group today and we’d love to chat!

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

Our Go To Sources for Data

Whether you are working on a needs assessment, evaluating a program or starting a new project, reliable data is important to help make smart choices. Our Go to Sources for Data- Blog image2

Due to the internet, data is available at our fingertips, but the endless options can become overwhelming.  Have you ever spent hours sorting through links and data sources to find one specific indicator such as the number of individuals in your community with post-secondary education or the poverty rate for a certain age group?

At Transform Consulting Group, we are data nerds and can help point you in the right direction to find the data you need to move your organization forward. We have a cheat sheet of the key organizations and resources that we utilize to help us find the right data for our clients.

There are a few resources that provide comprehensive data and the majority of information that you need.  However, there may be times where you want to go deeper with the data in your understanding.  Therefore, you may need to use some additional resources to find the information you are seeking.

Comprehensive Child Data
  • Child Trends — Child Trends is a national nonprofit research organization focused on improving the lives of children, youth, and their families. Their databank features data by topic, including child maltreatment/welfare, early childhood, families, health, and social and emotional learning.
  • Screen Shot 2017-06-13 at 11.35.27 AMKids Count Data Center — Kids Count is a project of the Annie E. Casey Foundation and has comprehensive data on child and family well-being.  Each state has a designated entity to manage and compile the state Kids Count data, which is the Indiana Youth Institute for Indiana. Individual county profiles are available and updated annually. 
Education Data
  • State Departments of Social Services — Many state departments of social services or human services oversee child care for young children and school-age children.  In Indiana, the Family and Social Services Administration (FSSA) Office of Early Childhood and Out-of-School Learning (OECOSL) collect data on early childhood education and out-of-school time programs and would be the resource to identify the number of programs serving young children and/or provide before and after school care for school-age children as well as the cost of care and other important factors.
  • State Departments of Education — Each state department of education has information about individual schools, corporations and their students.  In Indiana, the  Indiana Department of Education (IDOE) shared this data online via “Compass”, which is a searchable database to access educational attainment, testing scores, and free/reduced lunch rates.  Additional information can also be requested.
  • Lumina Foundation — Lumina is committed to increasing the post-secondary attainment of Americans to 60%.  They have national, state and metro level data available.  Check out the recently released 2017 Stronger Nation Report! We use the Lumina data when working with our college and career readiness and Center for Working Families clients.
Population Data
  • U.S. Census Bureau — The Census Bureau provides quality data about the people and the economy. It features a few data resource tools, including Quick Facts, the American FactFinder, and the American Community Survey. Information is available at a variety of geographic levels, including national, state, county, city and town, township, region, census tract and more.
  • State Population Resources — Some states have created online resources that summarize the Census data for their state.  In Indiana, STATS Indiana is a great resource to find quick profiles and maps on population, housing, education, income and poverty, health and employment.  In Central Indiana, SAVI is another great resource with detailed information and profiles. SAVI features tools to analyze and visualize Central Indiana Community Data. These websites tend to be more user-friendly than the Census.
Health
  • State Departments of Health — Each state department of health has access to health specific information.  In Indiana, this is the Indiana State Department of Health (ISDH).  ISDH recently launched a new website, Stats Explorer, to make it easier to access health data at the state and county level.  Here you can find information about the prevalence of drug overdoses, sexually transmitted diseases, births and deaths, cancer, infectious diseases and many other health issues, all in one convenient location. Previously, the data was available through multiple pages on the ISDH website.
Economic
  • State Departments of Workforce Development — Each state has an agency or office focused on workforce development and will have a variety of information available for the public.  In Indiana, the Department of Workforce Development (DWD) features workforce and economic data on a separate website, Hoosiers by the Numbers. Here people can find data by state or county on economics, education, employment/ unemployment, wages, and labor force.

There are many more data sources available online as well as via special request. Once you collect the data, don’t forget to make it visually appealing for your audience! Check out our blogs to find some additional tips on using your data.  If it is still too overwhelming, give us a call to see how we can help gather the data you need to inform your work. 

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn

You Finally Have Data! Who Should You Share It With and Why?

Data-BlogImagine your organization recently conducted a program evaluation.  Data was gathered from one of the programs offered and an analysis of the data revealed your program outcomes were met, some even exceeded!  This is exciting news and you want to shout it out from the nearest rooftop… We understand this urge, but there are better (and safer) ways to share your evaluation data.  In this blog, we discuss different audiences with whom an organization might share their evaluation results and the benefits of sharing them. Stay tuned for a future post discussing how to share your evaluation data.

At Transform Consulting Group, we love seeing organizations use data to measure and achieve success!  We also love helping organizations “visualize” success in easy-to-understand ways using simple charts and graphics.  Once these data visuals are audience-ready, what are the next steps?  Does an organization need to be intentional about sharing program evaluation data?  Why?

Data is only as good as how it is used.  If an organization collects data, analyzes it and makes beautiful visuals with it, but few people actually see the results, what is gained from the experience?  Probably not much!  This connects back to a blog we wrote called, 4 Steps to Complete a Program Evaluation.  Step number four in the process is to discuss the results of an evaluation and make data-informed decisions.  In order to do this, evaluation data must be shared with the right people.  

Our team typically compile a summary report and/or slide presentation of an organization’s evaluation data for stakeholders to review the results and discuss their implications.  A stakeholder is anyone who has a “stake”—an interest, concern, or investment—in an organization and/or program achieving (or not achieving) its goals.   We categorize data sharing into two different “buckets” -internal and external.  As it sounds, sharing data internally is with folks on the inside of an organization, and external data sharing is with folks on the outside looking in. Within each bucket, we’ve identified three important stakeholder groups and how sharing evaluation results can be beneficial, no matter what the data “says”.

Internal Stakeholders

1. Employees:  It would be challenging to find a dedicated employee who would not care to see evidence of their daily efforts actually paying off and leading to positive change.  The achievement of program outcomes reflected in data can help validate the combined efforts of all staff involved in a program’s design (if applicable) and delivery.  It is important to share evaluation results with staff at all levels and not just employees at the top.  Too often, we see that the information does not trickle down.  Sharing, showcasing and celebrating success builds morale and encourages staff to continue doing great work.  In the business sector, this would be akin to celebrating a top-sales month! 

On the other hand, results of an evaluation may reveal that a program is not achieving its intended outcomes and uncover potential reasons why not.  Evaluation data can “shed light” on issues that staff and/or leadership were not even aware of that could hinder the ability to make an impact.  Based on sharing these results, employees will know to expect workflow or programmatic changes in the near future without confusion or surprise.

At TCG, we recently worked with an organization that provides college and career readiness counseling for high-school students to evaluate their 8-week summer program.  During orientation, students were given a pre-survey to assess their knowledge of college and career readiness subject matter before participating in the program.  On the last day, students were given a post-survey to measure knowledge gained as a result of completing the program.  Evaluation data was presented to program staff and leadership in easy to understand charts and graphs.  Staff became excited when they saw how much college and career readiness knowledge students gained as a result of participating in the program!  They were also glad to find out which areas students reported knowing the least about.  As a result, staff could strengthen those areas of the program prior to the next round of incoming students.  

2. Volunteers:  Organizations that depend on volunteer work are always on the lookout for more help.  Unfortunately, unpaid labor can be hard to come by, especially if volunteers don’t feel like their time and effort is making a difference.  If an organization can show community impact as a result of volunteerism, their volunteers are more likely to feel validated and remain committed to the organization.  Then they may even recruit more volunteers!  In this case, evaluation data helps to promote satisfaction and the feeling of reward.  Volunteers are motivated to stay put and not seek out other volunteer opportunities when they know that their time and talent is making a difference.

3. Board of Directors:  The Board can use evaluation data to begin planning and discussing the future of an organization.  If the data supports outcomes consistently being met, a Board may decide to expand the program’s service delivery to a larger area or broader audience.  If the expansion is successful, the organization sustains a larger community impact and the Board (hopefully) feels a sense of accomplishment and pride in the organization.

For evaluations resulting in unmet program outcomes, the Board may recommend program model changes, or commit to focus on a problem area (i.e. staff professional development) during the upcoming year.  Either way, sharing program evaluation results with Board members equips them to make informed decisions about what is best for the organization going forward.

External Stakeholders

1. Funders:  Funders want to see a return on their investment into any organization.  Many funders require organizations to conduct program evaluations to remain accountable for the results that their funds are directly supporting.  Bottom line, funders want to know if the program is worth the resources that it costs.  Evaluation results help “quantify” worth for funders by showcasing to what extent the desired program changes are occurring.  If a program performs as expected, it is likely that funding dollars will keep flowing.  

Alternatively, if evaluation data shows program outcomes not being met and insignificant or no change is occurring, a funder may decide not to invest or discontinue investing in an organization.  Funders are often supporting multiple organizations at once and want to feel confident their money is well-spent and producing the best results!

2. Partners:  Evaluation data may help make a case for two or more organizations to join forces and provide combined programs or services within a community to maximize impact.  Sharing data can be a positive step in the direction of collaborating and working towards common goals.  Local programs may unknowingly be competing for clients, resources or limited funding.  Some are likely struggling to meet goals.  Awareness of such issues, made apparent through evaluation data, could spark ideas to collaborate and leverage partnerships to provide joint programming.  The result?  Sharing data may lead organizations to do what is best for the community, while also doing what is best for their budget!  

Ideally, routine program evaluation and data sharing will keep organizations accountable to each other in the future.  On the other hand, evaluation data can show when partnerships are no longer working and should be realigned or dissolved.

3. The Public:  Sharing data with communities helps to legitimize an organization’s purpose in the public eye.  Program evaluations are one way to demonstrate community impact.  Evaluation data goes a step farther to show how much impact an organization or program is making.  Nonprofit organizations are public agencies with a responsibility to communicate back to the public about the goodwill that they are making from the public’s investment in them!

Foundations, grant makers and other funders research shared public data about an organization or program to determine whether or not to invest in it.  Evidence of successful programming can suggest an organization itself is well-managed.  This inspires confidence that funding dollars will also be used wisely and generate the greatest return on investment.

Internally, sharing evaluation results can galvanize an organization–arming employees (program staff and management) and the Board of Directors with data they need to ensure program outcomes are being met and take appropriate actions when issues arise.  External stakeholders use data to verify an organization’s credibility and hold it accountable to the outcomes it seeks to achieve.  Ultimately, stakeholders use data to keep an organization on track to accomplish its overall goals.

It’s hard to win a game if your team has no idea of the score—or even worse, what game they are playing.  Sharing evaluation data helps to keep an organization’s impact transparent and everyone involved on the same page.  Stay tuned for our next blog in this series about how to share data and use it to tell your story within different communication channels!  Talk with one of our team members today and learn how you can get the word out about your evaluation results!

Share this article:Share on FacebookShare on Google+Tweet about this on TwitterEmail this to someoneShare on LinkedIn