- Impact Austin
Discovery Days 2023: Session One Recap - How Do Nonprofits Measure Impact?
Our 2023 learning series takes a different approach from past Discovery Days. Instead of exploring community needs and issues, our three sessions dive deep into one aspect of grantmaking: impact. Impact Austin's Grant Review Committees (GRCs) regularly ponder how to assess and compare intended outcomes and potential impacts among the dozens of grant applications received in each focus area. With that in mind, our 2023 theme developed.
The January 27 session of Discovery Days 2023 delivered a beautifully-organized presentation by Brenna Cummings and Jerrica Witte, with a welcome and introduction by Executive Director Demetria Caston. With so much content to share, we've added several of their slides into this recap. See the recorded video below, and check out speakers' bios here.
Jerrica notes that, while this webinar's information is especially helpful to Impact Austin's GRC volunteers, it's also intended for anyone wanting to engage with nonprofits and understand how they might evaluate their work.
Brenna emphasized that impact is all about scale; there are different categories of impact depending on the scale of change. So, for social impact to occur, there is a culmination and combination of actors and actions creating such a large change that it can be measure at a social level. But nonprofits typically focus on participant outcomes and program outcomes, and this session primarily focused in that space.
Outcomes are direct and measurable changes that result from an action or service. Additional comments regarding nonprofit outcomes include these:
Programmatic outcomes allow nonprofits to evaluate how well they deliver their programs and services, helping them determine any changes to be made for greater effectiveness. These outcomes might also be shared with a funder.
Participant outcomes can be affected by external real-life forces on individuals outside of the programs serving them.
Long-term outcomes are typically applicable to more established nonprofits, advanced in the services they deliver and how they track and measure participant progress, with a lot of internal tools for monitoring.
Among nonprofits, near-term outcomes may be most broadly measured.
Brenna then led the discussion to the elements of a nonprofit programs, the logic model upon which a program would be constructed. She noted that, when we read grant proposals, much of what we're reading is one of these elements below. The logic model shows how a nonprofit intends to get to desired participant or programmatic outcomes.
To illustrate the graphic above, Brenna applied the example of a workforce development program that could theoretically be proposed for Impact Austin funding.
Inputs could be new staff, salaries for staff, case management software, supplies and training materials, a facility. Funders fund the inputs! As funders, do we see a compelling and understandable set of inputs to bring the desired outputs?
Activities are the actions to be completed to facilitate desired outcomes.
Outputs show completed activities.
Outcomes show measurable change. These changes can be binary (getting a job/not getting a job) or a range that measures progress. We want to make sure we go beyond "reach" and get to the effects.
We next explored the reasons that nonprofits collect data. Certainly donors and funders want to understand what a gift or grant helped to deliver.
As a funder, Impact Austin pays attention to the data and outcomes that our grantees report and shares this information with our members. This content is available to the public in our Annual Reports, in Grant Outcomes videos on our YouTube channel, and on the See Our Impact page of our website. In earlier years Impact Austin grant outcomes were printed and shared with members at each Annual Meeting
Brenna gave illustrations for the three measures above, again using the example of a workforce training program.
A baseline assessment might measure a participant's current income level and skill level in Excel and Word.
A survey done every year might measure a participant's reported satisfaction with job and career.
Indicators could assess job readiness by measuring program attendance, access to transportation, typing skill, Excel skill.
Sometimes a funder may want to dig deeper with a nonprofit to understand more about their outcomes. Sensitivity to a nonprofit's capacity to plan for evaluation and to collect data should be a component of a funder's partnership with the nonprofit. Consider a data collection plan that is respectful.
The main part of the presentation closed with discussion of how funders can be better participants in the nonprofit ecosystem.
As a funder, act as a partner and a resource.
Really learn about the nonprofit, its mission, its history.
Understand where the nonprofit is in its organizational lifecycle.
Learn about the participants! Do we understand the "voice" of those populations? What are their challenges? What do we NOT already know? Don't lose sight of the individuals and families being served.
Don't add burden with funding requirements. Might some applicants be turned off or choose to turn away from partnership with us because our requirements are too strenuous?
Remember that evaluation requires resources: technology, software, staffing. How can we partner with a nonprofit's ability to provide evaluation? Some funders provide grants specifically for this purpose.
Consider equity in the data collection. Are assessments biased?
Trust Based Philanthropy is a space that Impact Austin is moving toward. How can we best deploy a trust-based ethos while also wanting to know the results of our nonprofit investments?
Trust the nonprofit to be the expert.
Partner with the nonprofit to convey impact. What resources does the nonprofit have?
What is reasonable to expect from the nonprofit in terms of reporting impact? What type of data recording "fits?"
Consider the funder-nonprofit relationship to be a collaborative effort. What does the nonprofit need to do that is important to them? What outcomes are important to them?
Final thoughts from our presenters:
Brenna - If nonprofit impact is a book, we have provided the introduction.
Jerrica - Data is powerful.
Not yet signed up for Discovery Days 2023? Please register here
and join us for two more sessions in February 2023.