Measurements appropriate for a dynamic state may not be appropriate for a static state.
Michael Quinn Patton is an independent organizational development consultant and has written five major books on the art and science of program evaluation, including the influential “Utilization-Focused Evaluation,” first published in 1978, in which he emphasized the importance of designing evaluations to insure their usefulness, rather than simply creating long reports that may never get read or never result in any practical changes.
Michael’s most recent book, Getting to Maybe: How the World is Changed, a collaboration with Frances Westley and Brenda Zimmerman, was published in 2006.
Traditional evaluation is often about finding a model that can work across time and space, that can be replicated. The most wide-spread models of traditional evaluation include summative and formative evaluations.
Summative evaluations are usually done at the end of a program to assess whether the model that was proposed for the program worked. Formative evaluations are done at the beginning of a program while you’re working out the bugs and improving the model. You do a formative evaluation in to order get ready for summative evaluation — both assume that you can work out the best model, then proceed to implement it.
In developmental evaluation, you expect the world to be dynamic, so that you will never arrive at a summative state. In innovative processes, participants may find they never get to a model for very long, or in a meaningful way. Using a summative approach during a developmental stage of an innovative process may do harm by imposing a static data collection model on a very dynamic adaptive process. Looking for a recipe may force your community and the people in it to fit your model and your prescription – contrast this to the metaphor of raising a child – where there is no recipe, but you try different things until you find what works.
Developmental evaluation fits when a group needs ways to get periodic feedback, reflect on it, then act on it. The Stacey Matrix can be helpful to help a group determine if you need a developmental evaluation approach. The matrix measures the degree of certainty about the efficacy of an intervention and the amount of agreement there is within a group.