Michael Quinn Patton, “Measuring Learning: Developmental Evaluation”, Tamarack Institute, 2007/05/10

Measurements appropriate for a dynamic state may not be appropriate for a static state.

Michael Quinn Patton is an independent organizational development consultant and has written five major books on the art and science of program evaluation, including the influential “Utilization-Focused Evaluation,” first published in 1978, in which he emphasized the importance of designing evaluations to insure their usefulness, rather than simply creating long reports that may never get read or never result in any practical changes.

Michael’s most recent book, Getting to Maybe: How the World is Changed, a collaboration with Frances Westley and Brenda Zimmerman, was published in 2006.

Traditional evaluation is often about finding a model that can work across time and space, that can be replicated. The most wide-spread models of traditional evaluation include summative and formative evaluations.

Summative evaluations are usually done at the end of a program to assess whether the model that was proposed for the program worked. Formative evaluations are done at the beginning of a program while you’re working out the bugs and improving the model. You do a formative evaluation in to order get ready for summative evaluation — both assume that you can work out the best model, then proceed to implement it.

In developmental evaluation, you expect the world to be dynamic, so that you will never arrive at a summative state. In innovative processes, participants may find they never get to a model for very long, or in a meaningful way. Using a summative approach during a developmental stage of an innovative process may do harm by imposing a static data collection model on a very dynamic adaptive process. Looking for a recipe may force your community and the people in it to fit your model and your prescription – contrast this to the metaphor of raising a child – where there is no recipe, but you try different things until you find what works.

Developmental evaluation fits when a group needs ways to get periodic feedback, reflect on it, then act on it. The Stacey Matrix can be helpful to help a group determine if you need a developmental evaluation approach. The matrix measures the degree of certainty about the efficacy of an intervention and the amount of agreement there is within a group.

Learning Centre – Communities Collaborating for Impact

Interview MP3 audio

Q&A MP3 audio

Advertisements
About

David Ing blogs at coevolving.com , photoblogs at daviding.com , and microblogs at http://ingbrief.wordpress.com . See .

Tagged with: ,
Posted in Talk Audio Download
One comment on “Michael Quinn Patton, “Measuring Learning: Developmental Evaluation”, Tamarack Institute, 2007/05/10
  1. The evaluation is for improvement not for policing. However, people from developing countries need to understand. This approach and philosophy will work when people implement projects sincerely and honestly.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s

Translate
Beyond this media queue
This content is syndicated to Twitter. For professional perspectives, look to Coevolving Innovations; for a photoblog, look to Reflections, Distractions.
  • Open Innovation Learning, Book Launch
    Video and audio recordings of the launch of my book, Open Innovation Learning, may be a more accessible preliminary way into the content, with the open access publication better as a reference.
  • Eight infographics on Systems Methods (UToronto iSchool 2018)
    The UToronto iSchool graduate student groups created 8 infographics reflecting impressions on the systems methods most relevant to their research in winter 2018.
  • Negotiating Order with Generative Pattern Language
    A workshop at PLoP 2017 framed dialogue as "Creating Order of" and "Negotiating Order with" frames of reference, to encourage collective sensemaking
  • Exploring the Context of Pattern Languages
    Pattern language is not for wicked problems, said Max Jacobson, coauthor with Christopher Alexander of the 1977 A Pattern Language: Towns, Building, Construction.  In addition, the conventional definition of an Alexandrian pattern as “a solution to a problem in context” when applied to social change might better use the term “intervention”, rather than “solu […]
  • Multiparadigm Inquiry Generating Service Systems Thinking
    Expanding pattern language to service systems through multiparadigm inquiry sweeps in research by scholars contemporaneous to Christopher Alexander.
  • Christopher Alexander, Horst Rittel, C. West Churchman
    Christopher Alexander (pattern languages), Horst Rittel (wicked problems) and C. West Churchman (the systems approach) were neighbours on campus at U.C. Berkeley in the 1960s and 1970s. What might we synthesize from their joint wisdom?
  • 2018/05 Moments May 2018
    Barely recovered from jet lag from China, bounced back to west coast for some continuing research. Returned home as spring turns into summer.
  • 2018/03 Moments March 2018
    A month in Toronto, as I came out of the Air Cast for my Achilles Tendon injury from December, and started physiotherapy.
  • 2018/04 Moments April 2018
    Relearning to walk, after ankle in cast, across Shanghai, Wuhan and Vancouver, with DY as my sherpa.
  • 2018/02 Moments February 2018
    A second month when the only occasions to leave the house required my spouse to accompany me.
  • 2018/01 Moments January 2018
    Ankle in a cast, a limited mobility month, maximum 10km from home
  • 2017/12 Moments December 2017
    Completed round-the-world trip Helsinki-Hameenlinna-London, then busy holiday season including an Achilles tendon injury from parkour, and our da shou Double 60 celebration.
  • The Systems Approach and its Enemies Helps Us Find the Morality of a Revised Democracy | van Gigch | 2006
    In a book series celebrating C. West Churchman, John P. van Gigch digests (and portends to extend) The Systems Approach and its Enemies. On enemies … 4.1 A MATTER OF DEFINITIONS: ADVERSARIES VERSUS ENEMIES I note the similarity/difference between the words ‘enemy’ and ‘adversary.’ Other authors use the word adversary (ies) to denote all the […]
  • Restoring Legitimacy to the Systems Approach | Clinton J. Andrews | 2000
    A public policy professor, Clinton J. Andrews, looks at how The Systems Approach may encounter problems in skepticism from engineering practice. The systems approach is one general way of going about tackling a problem; some others include the experimental, political, moral, religious, and aesthetic approaches [1,p. 5], [2]. The systems approach to a problem […]
  • The Systems Approach: Its Variety of Aspects | Richard Mattessich | 1982
    An informed view of the Systems Approach from 1982.  (Richard Mattessich was a well-respected professor at UBC when I started in the doctoral program in 1982, but I wouldn’t get to appreciate the Systems Approach as described by C. West Churchman until the ISSS 1998 meeting). In his latest work [The Systems Approach and its […]
  • A logic model for philanthropic effectiveness | Peter Frumkin | 2006
    Program evaluation can be approached from the philanthropic perspective. In searching for ways to give money effectively, donors have many options and confront a wide range of theories about how to achieve impact. It is possible to think about these theories as falling into three main categories: theories of change, theories of leverage, and theories […]
  • Program Logic Models and Theory of Change | Kellogg Foundation | 2004
    From the program evaluation community, with the W.K. Kellogg Foundation … The program logic model is defined as a picture of how your organization does its work – the theory and assumptions underlying the program. A program logic model links outcomes (both short- and long-term) with program activities/processes and the theoretical assumptions/principles of t […]
  • Restoring Manjaro Grub after Ubuntu upgrade
    On a multi-boot Linux computer where Ubuntu has already been installed, adding on Manjaro Linux installs its own version of Grub (that I’ll call Arch-Grub) that is different but compatible with that previously installed (that I’ll call Debian-Grub). Updating Ubuntu to a newer version (or installing an older version) restores Debian-Grub, replacing the workin […]
Contact
I welcome your e-mail. If you don't have my address, here's a contact page.
%d bloggers like this: