Jean A. King - How do you measure the impact of the humanities?

Jean A. King is Distinguished Teaching Professor in the Department of Organizational Leadership, Policy and Development at the University of Minnesota where she serves as Director of the Minnesota Evaluation Studies Institute. A long-time writer on program evaluation, she joined the Minnesota Humanities Center Board in 2012 and currently serves on the Program and Development Committees and the advisory group for the Education Initiative, as well as supporting staff in developing evaluation capacity.

There are two reasons why I feel comfortable discussing this topic. First, I am a program evaluator with over 35 years’ experience. Second, I was an English major in college and remain a proud member of Garrison Keillor’s Professional Organization of English Majors (POEM). Plus I have one really good example that demonstrates such impact. Twenty years ago when I lived in New Orleans, I successfully answered the quiz-of-the-day question at an ice cream parlor: “What is the correct version of the following line from Hamlet: ‘Also, poor Yorick, I knew him well.’” Without hesitation I recited the correct line (“Alas, poor Yorick! I knew him, Horatio…”) and earned a free ice cream cone. When I told my mother, she noted joyfully that I had demonstrated the tangible value of an English major and four years of liberal arts college tuition. Talk about impact!
But systematically measuring the impact of the humanities moves well beyond a free ice cream cone. Multiple challenges confound the task, three of which emerge by examining three key words:

• Humanities- An initial challenge stems from the need to be explicit about what, exactly, constitute “the humanities.”  People don’t necessarily agree on what to include under the umbrella nor what should count as a humanities “program” or intervention. The first challenge, then, is being clear about exactly what we are trying to measure the impact of.

• Measuring- A second challenge arises because, even if we can agree on that, from a psychometric perspective, measuring the effects is a daunting task. Quantitative methods generate numbers, and qualitative methods produce thick descriptions and stories, but how can any method document meaningful impact? What instruments and data collection processes will produce credible evidence for those who control humanities budgets--legislators, policy makers, and funders?

• Impact- I routinely encounter well-intentioned people who want to prove causal assertions--to find irrefutable evidence that a certain set of activities leads directly and uniquely to specific measurable outcomes. Herein lies the third and most daunting challenge. While it may seem like a straightforward task, it is simply not possible to measure such impact without making a sizeable number of assumptions about how the multiple systems in which programs operate work. In my experience there is virtually never a straight line connecting program activities, outputs, and short- and long-term outcomes, all of which lead to ultimate impact.

Thankfully, there are two reasons why these challenges honestly don’t bother me. First, we evaluators can humbly design studies that help intended users practice what my colleague Michael Quinn Patton calls “reality testing.” We can provide sound evidence about what is occurring before, during, and after programs. Second, and more importantly, I would argue that some things hold intrinsic value, even if we can’t demonstrate it with 21st century rigor. To my mind, the humanities fall into that category. They define us as human beings and bring us together in community, and that is a good in and of itself, whether or not we can prove it.

---