There are two reasons why I feel comfortable discussing this topic. First, I am a program evaluator with over 35 years’ experience. Second, I was an English major in college and remain a proud member of Garrison Keillor’s Professional Organization of English Majors (POEM). Plus I have one really good example that demonstrates such impact. Twenty years ago when I lived in New Orleans, I successfully answered the quiz-of-the-day question at an ice cream parlor: “What is the correct version of the following line from Hamlet: ‘Also, poor Yorick, I knew him well.’” Without hesitation I recited the correct line (“Alas, poor Yorick! I knew him, Horatio…”) and earned a free ice cream cone. When I told my mother, she noted joyfully that I had demonstrated the tangible value of an English major and four years of liberal arts college tuition. Talk about impact!
But systematically measuring the impact of the humanities moves well beyond a free ice cream cone. Multiple challenges confound the task, three of which emerge by examining three key words:
- Humanities- An initial challenge stems from the need to be explicit about what, exactly, constitute “the humanities.” People don’t necessarily agree on what to include under the umbrella nor what should count as a humanities “program” or intervention. The first challenge, then, is being clear about exactly what we are trying to measure the impact of.
- Measuring- A second challenge arises because, even if we can agree on that, from a psychometric perspective, measuring the effects is a daunting task. Quantitative methods generate numbers, and qualitative methods produce thick descriptions and stories, but how can any method document meaningful impact? What instruments and data collection processes will produce credible evidence for those who control humanities budgets--legislators, policy makers, and funders?
- Impact- I routinely encounter well-intentioned people who want to prove causal assertions--to find irrefutable evidence that a certain set of activities leads directly and uniquely to specific measurable outcomes. Herein lies the third and most daunting challenge. While it may seem like a straightforward task, it is simply not possible to measure such impact without making a sizeable number of assumptions about how the multiple systems in which programs operate work. In my experience there is virtually never a straight line connecting program activities, outputs, and short- and long-term outcomes, all of which lead to ultimate impact.