Date post: | 16-Mar-2016 |
Category: |
Documents |
Upload: | martin-lotfabadi |
View: | 219 times |
Download: | 0 times |
DevelopmentalEvaluation:
Applying Complexity Conceptsto Enhance Innovation and Use
Michael Quinn Patton SIG October 4,2010
Michael Quinn Patton
1
In the beginning…
Michael Quinn Patton SIG October 4,2010 2
Michael Quinn Patton SIG October 4,2010
Innovation and Evaluation
Systems thinking & complexity science
as frameworks for conceptualizing interventions:
Developmental Evaluation
3
5 types of DE
1. Ongoing Development
Developmentvs.
Improvement
Michael Quinn Patton SIG October 4,2010 4
Blandin Community Leadership Program
Michael Quinn Patton SIG October 4,2010 5
Beyond Formative and Summative to
Developmental Evaluation
as an option in the repertoire of approaches
Michael Quinn Patton SIG October 4,2010 6
Mrs. McCave and her 23 Daves
7Michael Quinn Patton SIG October 4,2010
Michael Quinn Patton SIG October 4,2010
Challenge:Matching the evaluation process and design to the nature of the situation:
Contingency-basedEvaluation
8
Michael Quinn Patton SIG October 4,2010
Conditions that challenge traditional model-testing evaluation
• High innovation• Development• High uncertainty• Dynamic• Emergent• Systems Change
AdaptiveManagement
andDevelopmental
Evaluation
9
Michael Quinn Patton SIG October 4,2010
First DE Type
Ongoing development in adapting a project, program, strategy, policy, or other innovative initiative to new conditions in complex dynamic systems.
10
Michael Quinn Patton SIG October 4,2010
Mintzberg on Strategy
Unrealized StrategyIntendedStrategy
Deliberate Strategy
Realized Strategy
Emergent Strategy
11
Michael Quinn Patton SIG October 4,2010
12
Uncertainty and Emergence“No battle plan ever survives contact with the
enemy.” Field Marshall Helmuth Carl Bernard von Moltke
“Everyone has a plan…until he gets hit.”Former World Heavyweight boxing champion, Mike Tyson
5 Types of DE2. Pre-formative development of a potentially
scalable innovation:• Developing an innovation to the point
where it is ready for traditional formative and summative evaluation
• Pre-formative developmental evaluation works with emerging ideas and visionary hopes in a period of exploration to shape them into a potential model that is a more fully conceptualized, potentially scalable intervention.
Michael Quinn Patton SIG October 4,2010 13
Michael Quinn Patton SIG October 4,2010
PANARCHY MODEL
14
Michael Quinn Patton SIG October 4,2010
Stored
HARVESTING LESSONS
DEVEOPMENTAL EVALUATION
FORMATIVE
SUMMATIVE
Phases of Technological &
Social Innovation 15
Polio Example Bill & Melinda Gates Foundation: $1 billion on polio
eradication: Vertical, focused strategy "There's no way to sugarcoat the last 12 months,"
Bruce Aylward, a WHO official, told Mr. Gates in June -- the virus was rippling through countries believed to have stopped the disease.Mr. Gates asked: "So, what do we do next?“
New strategy: Disease-specific wars can succeed only if they also strengthen the overall health system in poor countries.
http://online.wsj.com/article/SB10001424052702303348504575184093239615022.html?mod=WSJ_hps_InDepthCarousel
Michael Quinn Patton SIG October 4,2010 16
5 Types of DE
3. Adapting effective general principles to a new context as ideas and innovations are taken from elsewhere and developed within a new setting, the work of developmental evaluation in the dynamic middle between top-down and bottom-up forces of change.
Michael Quinn Patton SIG October 4,2010 17
Michael Quinn Patton SIG October 4,2010
Fundamental Issue:How the World Is Changed
Top-down dissemination of “proven models”
versusBottoms-up adaptive management
18
Michael Quinn Patton SIG October 4,2010
Prescriptive Models vs. Adaptive Principles
Identifying effective principles for adaptive management
(bottoms-up approach)versus
Identifying and disseminating “proven” models
(top down approach)19
Michael Quinn Patton SIG October 4,2010 20
Top-down dissemination of “proven models”
Developmental Evaluation: Navigating the murky,
complex, dynamic MIDDLE
Bottom-up adaptive management
Michael Quinn Patton SIG October 4,2010 21
Personal Factor
Damiano example, Jean Gornick
21
5 Types of DE4. Major systems change and cross-scale
developmental evaluation• Providing feedback about how major systems
change is unfolding• Evidence of emergent tipping points• System inter-relationships as the “unit of
analysis”• Evaluating strategy
Michael Quinn Patton SIG October 4,2010 22
Michael Quinn Patton SIG October 4,2010
Systems
• Parts are interdependent such a change in one part changes all parts
• The whole is greater than the sum of the parts
• Focus on interconnected relationships• Systems are made up of sub-systems and
function within larger systems
23
Michael Quinn Patton SIG October 4,2010 24
Michael Quinn Patton SIG October 4,2010
Understanding the Elephant
from a Systems Perspective
25
Michael Quinn Patton SIG October 4,2010 26
Michael Quinn Patton SIG October 4,2010 27
Michael Quinn Patton SIG October 4,2010 28
Michael Quinn Patton SIG October 4,2010 29
Michael Quinn Patton SIG October 4,2010 30
Michael Quinn Patton SIG October 4,2010
The relationship between what goes in and what comes
out What conceptual framework informs evaluation?
31
Michael Quinn Patton SIG October 4,2010
Teen Pregnancy Program Example
32
Michael Quinn Patton SIG October 4,2010
Logic Model for Pregnant Teens Program1. Program reaches out to pregnant teens
2. Pregnant teens enter and attend the program (participation)
3. Teens learn prenatal nutrition and self-care (increased knowledge)
4. Teens develop commitment to take care of themselvesand their babies (attitude change)
5. Teens adopt healthy behaviors: no smoking, no drinking,attend prenatal clinic, eat properly (behavior change)
6. Teens have healthy babies (desired outcome)
33
Michael Quinn Patton SIG October 4,2010
Systems web showing possible influence linkages to a pregnant teenager
Teachers/ other adults
Youngpregnantwoman's
attitudes &behaviors
Herparents &
other familymembers
Child'sfather &
peers
Prenatal program
staff
Her peer group
34
Michael Quinn Patton SIG October 4,2010
Program systems web showing possible institutional influences affecting pregnant teenagers:
SCHOOL SYSTEM
Youngpregnantwomen's
attitudes &behaviors
PrenatalClinic andHospitalOutreach
Church
Prenatal program
Other community-based youth programs
Other Systems-- welfare-- legal -- nutrition programs-- transportation-- child protection-- media messagesContext factors-- politics-- economic incentives-- social norms-- culture-- music
YouthCulture
35
5 Types of DE
5. Developing a rapid response in the face of a sudden major change or a crisis, like a natural disaster or financial melt-down:
• Exploring real time solutions and generating innovative and helpful interventions for those in need.
• Dealing with high uncertainty, turbulence, turmoil, high stakes, and often conflict.
Michael Quinn Patton SIG October 4,2010 36
Refugee Camps
37Michael Quinn Patton SIG October 4,2010
38Michael Quinn Patton SIG October 4,2010
About this book
Preview this book
Shake Hands with the Devil By Roméo Dallaire
This is a preview. The total pages displayed will
be limited. L
Loading...
Loading...
39Michael Quinn Patton SIG October 4,2010
resultSearch Booksresultoe9S6SgfGeneral RPP1PP1ACfU3U1E
Michael Quinn Patton SIG October 4,2010 40
Photo by Lynsey Gornick
Michael Quinn Patton SIG October 4,2010
Seeing Through A Complexity Lens
“You don't see something until you have the right metaphor to let you perceive it”. Thomas Kuhn
41
Getting to Maybe: How the World Is
Changed? Frances Westley,Brenda Zimmerman, Michael Q. PattonRandom House
Canada,2006
Michael Quinn Patton SIG October 4,2010 42
Michael Quinn Patton SIG October 4,2010
Complex Nonlinear Dynamics• Nonlinear: Small actions can have large
reactions. “The Butterfly Wings Metaphor”• Emergent: Self-organizing, Attractors• Dynamical: Interactions within, between, and
among subsystems and parts within systems can volatile, changing
• Getting to Maybe: Uncertain, unpredictable, uncontrollable
• Co-evolution: Integrating innovation and evaluation
43
Three Cups of TeaBaltistani proverb:
First cup you share, you are a stranger.
Second cup, you are an honored guest.
Third cup, you are in relationship.
44Michael Quinn Patton SIG October 4,2010
Michael Quinn Patton SIG October 4,2010
“A Leader's Framework for Decision Making” by David J. Snowden and Mary E. Boone, Harvard Business Review,November, 2007:
Wise executives tailor their approach to fit the complexity of the circumstances they face.
45
Wise evaluators tailor their approach to fit the complexity of the circumstances they face.
Michael Quinn Patton SIG October 4,2010 46
Michael Quinn Patton SIG October 4,2010
5 Types of DE
1. Ongoing development and adaptation2. Preformative evaluation to support
exploration and innovation3. Supporting local adaptation of general
principles to navigate top-down and bottom-up forces for change
4. Evaluating major systems change5. Evaluating in turbulent, disaster situations
47
Michael Quinn Patton SIG October 4,2010
Challenge:Matching the evaluation process and design to the nature of the situation:
Contingency-basedEvaluation
48
Michael Quinn Patton SIG October 4,2010 49
Michael Quinn Patton SIG October 4,2010
ReferencesDevelopmental Evaluation: Applying Complexity
Concepts to Enhance Innovation and Use. Guilford Press, June 2010.
Getting to Maybe: How the World Is Changed? Frances Westley, Brenda Zimmerman, Michael Q. Patton, Random House Canada, 2006
Utilization-Focused Evaluation, 4th ed., Michael Quinn Patton, Sage, 2008.
50