Program Dev & Impact Evaluation

By Melinda McAliney, MSW

‍

Congratulations on making it to this point in establishing a strong and sustainable ministry! For most new organizations, the temptation is to now jump in and start delivering services. The need is great, and you want to help, right? Having worked with hundreds of nonprofits and ministries over my career, I would like to humbly suggest caution. Take the time now to do some research, develop your logic model, and identify your evaluation methods. It will ensure that you’re delivering services that make an impact and will help prevent your team from conflating how busy you are/how many people you are serving with effectiveness. Being thoughtful and intentional now will maximize your kingdom impact.

Research

There are two types of research you should do right away: research similar services already available in your community, and research what has been proven to work (and, conversely, not work).

‍

What Exists

Reach out into your community and learn what other churches/faith communities and nonprofits are doing. Talk with elected officials and review any master plans or vision documents for your community. This important step not only helps you learn about what’s going on in your community, but also helps you establish your collaborative network. As amazing as your services will be, you won’t be able to do everything for those that come to your doors (nor should you). Thus, create a robust network of other programs and services you can refer to and collaborate with. This step also helps prevent you from delivering duplicative services.

‍

What Works

Not every idea is a good or effective one. You’ve felt God calling you to address a need in your community. That’s wonderful! Now, honor that calling by spending some time learning about what works, and what’s most effective in moving the needle and affecting real, measurable change. Over the past 30+ years, nonprofits have been challenged to demonstrate their effectiveness. You can benefit from that research and build your services around those that have been shown to work, as well as avoid those services that have been shown to make little to no difference.

‍

As you do this research, be sure to consider context. Just because a workforce development program worked in the urban core of Chicago, doesn’t mean that it will have the same impact in rural Mississippi. Try to find programs that serve a similar population and context to yours.

‍

Logic Models

Now it’s time to start laying out your ministry on paper. A good way to do this is by creating a Logic Model. Logic models are a simple, visual way to show your program and all of its components at a very high level. Here is what a typical logic model looks like:

‍

Inputs

(Resources)

What do you need in order to do what you do?

Activities

(Services)

What will you do?

Outputs

How much will you do? How many will you serve?

Outcomes

(Results)

What difference will it make?

Materials, Staff, Equipment, Space, Equipment, Training

Align with Budget and Narrative

Activities, Services,

Efforts

Activities

Numbers Served

Quantification of your activities

Knowledge, Beliefs, Attitudes, Skills

Behavior

Status/Condition

‍

The temptation is to start with the Activities column, but doing that is like the tail wagging the dog. Start with your outcomes and get really, really clear on the change you seek in those you serve. Humans typically go through three steps in achieving change – knowledge/skill/attitude/belief, behavior, and finally condition/status. Let’s say you want to create a ministry to help prevent child abuse/neglect. With that in mind, here are outcome definitions and an example of an outcome statement:

‍

• Condition/Status (longer-term change)

• What will be different about your clients’ lives as a result of your program? 

• Ex.  68 of 80 (85%) families who participate in at least 10 parenting skills sessions will stay intact for at least 6 months after program completion. 

• Behavior (intermediate change)

• What will your clients do differently as a result of your program?

• Ex. 72 of 80 (90%) parents who participate in at least 10 parenting skills sessions will demonstrate more positive interaction with their children. 

• Knowledge, Skill, Attitude, Belief (short-term change)

• What will your clients know or believe as a result of your program?

• Ex. 75 of 80 (94%) parents who participate in at least 10 parenting skills sessions will gain knowledge of positive parenting strategies. 

‍

It seems easy, but once you dive in, this may end up being the most challenging part of your ministry development journey. Spend time creating outcomes that are ultimately measurable and are right-sized for your ministry. For example, if you’re developing a ministry that reads to first graders, it’s impractical to say that one of your outcomes will prevent high school dropout. A more appropriate condition/status outcome may be that at the end of the school year, first grade students are reading at grade level. See the difference?

‍

Once you’re clear on your outcomes, work on developing your activities – what is all the “stuff” you are going to do? Keep your outcomes in front of you so that you make sure that your activities directly point to and support your outcomes. This can help prevent mission creep or doing too many activities that, while people on your ministry team may like them or have the skills to provide them, they don’t actually lead to your outcomes. 

‍

Next, create your outputs. This is basically a quantification of your activities. For example, if you provide a 10-week parenting class (activity), your output would quantify how many parenting classes you will provide during the year.

‍

The final step is creating your inputs/resources. This is when you get to see if you’ve built an Audi program with Chevy resources. Rather than setting your ministry up for failure or burnout, go back and rethink the program. You may need to scale back on your activities in the short-term as you grow your resources.

‍

There! You’ve done it! You’ve created your Logic Model. Congratulations! You now have a valuable tool that you can use in sharing your ministry with key stakeholders, including donors.

‍

You’re not done yet, though. The next step is evaluation.

‍

Evaluation

In a nutshell, evaluation is all about being able to determine if you did the things you said you were going to do with the people you said you were going to do it with AND did you achieve the outcomes you said you were going to achieve. I have encountered programs throughout my career that, after conducting a simple evaluation, realized their services were not as effective as they thought they were (they were equating how busy they were – the “stuff” – with effectiveness), and some found out that they were having no impact at all. Don’t be like them.

‍

People tend to get nervous whenever the topic of evaluation comes up.  They often start envisioning double-blind research studies, databases, consultants, and the like. Evaluation can be all of this. It can also be a simple pre/post survey. It’s okay to start small and simple. The point is to do something to make sure your ministry is having the intended impact.

‍

Using our example in the previous section of a child abuse/neglect prevention program, following are some simple ways you could measure your outcomes.

• Outcome:  68 of 80 (85%) families who participate in at least 10 parenting skills sessions will stay intact for at least 6 months after program completion. 

• Send a survey or call at 6 months asking if the family is still intact, if they have had any contact with the Department of Family Services, etc.

• Outcome:  Ex. 72 of 80 (90%) parents who participate in at least 10 parenting skills sessions will demonstrate more positive interaction with their children. 

• Observe parents interacting with their children.
• Distribute a survey asking parents about their interaction with their children.

• Outcome:  Ex. 75 of 80 (94%) parents who participate in at least 10 parenting skills sessions will gain knowledge of positive parenting strategies. 

• Distribute a pre/post survey at the end of 10 sessions.
• Conduct a focus group at the end of 10 sessions to ask what parents learned.

‍

Once you have gathered the data, look at it! Schedule regular sessions with key staff/volunteers throughout the year, but no less than annually, to dig into the data. Are you achieving the outcomes you wanted? If not, why not? Be slow to point the finger at your participant(s). Instead, look at your program – is a key element missing? Are you offering services when your population isn’t available? Are you asking for too big of a time commitment? To get these answers, you may want to conduct a focus group or distribute a survey to participants. 

The important thing is to understand your data and respond to it. Don’t keep doing the same thing and expecting different results. Consider how you can use the data to get better and drive even better outcomes for those you serve. 

‍

Closing

Walking through these steps will help you build a strong foundation for your ministry. The actual process is much more messy than it may first appear, and that’s okay. Change is messy. Balance your plans with the inherent messiness, and something beautiful will emerge.

God’s blessings as you embark on your incredible ministry journey. May He bless your efforts richly!

‍