What can program evaluation learn from deliberative assessment? (a thinking ‘out loud’ post)

Wednesday night the Oregon Program Evaluators Network (OPEN) held a great “networking & community learning” event. We did a little beer-drinking, a lot of networking, and heard from a number of members and friends who shared briefly about a project, or a concept or tool they’ve learned about recently.

I shared about deliberative assessment. Deliberative assessment is an approach to higher ed assessment that I learned about during a visit on our campus with electronic portfolio and assessment guru Darren Cambridge (of American Institutes for Research).

Darren says that “Assessment means making student learning visible so that it can inform programmatic and curricular innovation and demonstrate effect on learning and identity development.” (see link to slideshare at bottom of post for source of this and below quotes)

With regard to deliberative assessment, he adds that:

• “Students are privileged informants about their own learning.
• Evidence of learning needs to come from multiple contexts, and the relationships between them need to be articulated.
• Assessment should be a system of deliberative processes inclusive of all stakeholders that makes programs more responsive to them.”

So where traditional assessment approaches might seek to standardize and compare student performance to outcomes that are a set of standards established by higher powers (perhaps even via standardized tests – yuck – not that I have an opinion about that…). In contrast, a deliberative approach would “capture standards all stakeholders value as enacted in practice and examining alignment of both student and programmatic performance.”

A benefit of deliberative assessment, then, is that it takes a collaborative approach. It recognizes and makes use of the fact that universities and students need each other to learn. Programs then are to think of themselves as accountable to students. So instead of telling students they all must meet X and Y standards, we articulate what we ‘promise’ students will know, based on work done to determine what we (including students) value and need.

Can you see connections here to program evaluation approaches? To formative and impact evaluation? To developmental evaluation? To collaborative and empowerment and participatory evaluation? To supporting action based on data? I do!

Ultimately, this illustrates why I think that the two fields (higher ed assessment and program evaluation) can learn from one another. We use different terminology, but we often are coping with the same challenges, and having similar successes. AND, it’s these types of approaches, in assessment and program evaluation, that resonate with me as authentic, meaningful, and lending themselves to action based on the results of their work.

And, to take a bit of a tangent here, but to underscore my feelings about what the assessment field might teach program evaluation… I’ll also point out that a central function of assessment is measuring and understanding learning (duh, right?). Program assessment findings are or can easily be framed by the question “what are we (said program) learning?” and “what might we do or change as a result from learning whatever it is that we are learning?” And I’ll ask this: Aren’t we ultimately trying to learn about something when we do program evaluation as well? Couldn’t a program evaluation project benefit from intentionally framing our work this way? Calling out learning as a central goal for the project? And if we were to explicitly frame evaluation findings around learning, might we (both evaluators and program administrators) be more likely to apply that learning — to use the evaluation findings to make improvements, etc.?

Anywhoo, here’s a link to a presentation from Darren Cambridge strikingly similar to the one we saw during his visit and containing the quotes used in this post, and another to his book E-Portfolios for Lifelong Learning and Assessment which covers deliberative assessment, among other things, in much greater detail.