Download as PDF, PPTX
Uploaded byPattie Pattie
Patton 30aug
The document discusses the necessity of evolving evaluation practices to address complex global challenges such as climate change, inequality, and systemic transformation. It highlights the principles of blue marble evaluation, emphasizing global thinking, interconnections, and sustainability as key criteria for assessing initiatives and interventions. The text also critiques traditional evaluation methodologies and advocates for inclusive and adaptive approaches that recognize the importance of long-term, transformative impacts on diverse systems.
More Related Content
Patton 30aug
- 2.
- 4.
- 6. Think globally, Act locally& globally, Evaluate the Interactions
- 7. EvalPartners The international movement to strengthennational evaluation capacities
- 8.
- 10.
- 12. Global problems transcend nationaland agency boundaries
- 13. • Climate change •Economic turbulence • Refugees • Virulent infectious diseases • Dying oceans • Global cyber-terrorism • International drug cartels • Human trafficking • Weapons trafficking • Poverty and inequality • Multi-national corporate collusion
- 14. Global problems…. • Thedefinitions of the problems are disputed • The “facts” are a matter of intense debate • Politics and special interests dominate: -- national interests -- multi-national corporate interests -- agency agendas -- competition for resources • The stakes are huge
- 15. Evaluation grew up inthe “projects” Evaluation grew up in the projects
- 16. Evaluation Traditional Evaluating… • Grants • Projects& Programs • Clusters of grants • Goal attainment • Outcomes • Implementation Generating... • Findings & Lessons • Recommendations Nontraditional & New Directions: Evaluating… • Mission fulfillment • Strategy • Innovation • Advocacy campaigns • Policy change • Systems Change • Complex dynamic interventions 16
- 17. Innovations & Challenges: Evaluating… •Community impacts • Regional initiatives • Networks and collaborations • Leadership • Inclusiveness and diversity • Innovation • Collective impact • Scaling • Environmental ecosystem sustainability • Global systems change & TRANSFORMATION 17
- 20.
- 21.
- 22.
- 25. From evaluating fishing to evaluating the sustainabilityof the fishing cosystem
- 26.
- 28.
- 29. Are we readyfor…. Evaluators without borders
- 31. Blue Marble Evaluation CorePrinciples
- 33.
- 34. 34 RIGOROUS SCIENTIFIC THINKING: ❑ Criticalthinking ❑ Inferential thinking ❑ Skill in interpreting evidence ❑ SYSTEMS THINKING
- 35. Carl Sagan, cosmologist “Scienceis a way of thinking much more than it is a body of knowledge.” 1934 - 1996
- 38. “We can notsolve our problems with the same level of thinking that created them.”
- 40. Blue Marble Global ScalePrinciple Design Guidance Evaluation Criterion and Question Engage using a global perspective. Look beyond nation- state borders and boundaries to affect transnational, regional, and global patterns, interactions, and dynamics. Global scale fidelity: To what extent and in what ways is an initiative or intervention truly global in both processes and results? Blue Marble Global Scale Principle
- 48. Geologic Time • Earth’shistory has been broken up into time units based on the fossils contained within the rocks. These units are part of the Geologic Time Scale. • The time scale is broken up into units called eons, eras, periods, and epochs
- 52.
- 54. Peter Drucker, managementscholar “The greatest danger in times of turbulence is not the turbulence— It is to act with yesterday’s logic.” 1909-2005
- 55.
- 63.
- 64.
- 65.
- 66. Grouping SDGs credit: AzoteImages for Stockholm Resilience Centre
- 67.
- 71.
- 73. Getting to Maybe: Howthe World Is Changed? Frances Westley, Brenda Zimmerman, Michael Q. Patton 73
- 74. “We are thefirst generation to know that we are destroying the planet, and the last generation that can do anything about it.” WWF (2018)
- 77. “In Scotland, weare living as if we have the resources of three planets when we only have one. Australia is consuming five planets of resources. For the world as a whole, we are consuming the Earth’s resources as if we have 1.7 planets. We only have one.” Terry A’Hearn, CEO, Scottish Environmental Protection Agency
- 78. “Climate change really doeschange everything. If we are going to meet global targets or stabilize global temperatures, then we are going to have significant change whether we like it or not.” Professor Ioan Fazey, director of the Centre for Environmental Change and Human Resilience, University of Dundee.
- 79. Suzanne Moser, Stanford Woods Institutefor the Environment Transformations, If it is life we want
- 80. Kumi Naidoo, Launch Executive Directorof the African Civil Society Initiative “The most important challenge for leadership of our time, whether the leaders are from the private sector, from government, or from civil society, is how can we tell it like it is, speak truth to power, say that we are five minutes till midnight on climate change.”
- 81. “I globe trotfrom one meeting to another. Some of us are caught up in this cycle and there seems to be no way out of it. Anyway, this year (2017), I participated in nine international meetings. I was active in all of them either as part of the organizing group or as a presenter. There is one thread connecting all of them: ‘transformation.’” Million Belay is Coordinator of the Alliance for Food Sovereignty in Africa (AFSA), a broad alliance of different civil society actors that are advocating for and working towards food sovereignty and agroecology in Africa.
- 82.
- 83. “We believe thattransformational change requires that we craft new and better solutions at all scales through a systems-level approach and deep collaboration between philanthropy, researchers, grassroots movements, the private sector, farmers and food systems workers, Indigenous Peoples, government, and policymakers. Transformation means realizing healthy, equitable, renewable, resilient, and culturally diverse food systems shared by people, communities, and their institutions.” (https://futureoffood.org/about-us/ Ruth Richardson, based in Toronto, is Executive Director of the Global Alliance for the Future of Food
- 85. Transformation involves redefinitionof goals (e.g.: from producing energy to producing sustainable energy) which arise from a new understanding about the way things work (eg.: carbon emissions result in climate change) and produce fundamental change in operating logics (e.g.: from “mining” of nature, to harmony with nature). This usually involves deep shifts in power structures (e.g.: away from carbon extractors).
- 86. GEORGE FLOYD: “ICAN’T BREATHE.”
- 88. SYSTEMS CHANGE …beyond projects •Deep •Broad •Long-term
- 89. Understanding the Elephant from a SystemsPerspective 89
- 90.
- 91.
- 92.
- 93.
- 94.
- 95.
- 96.
- 97.
- 98. The relationship betweenwhat goes in and what comes out What conceptual framework informs situation recognition and sense-making? 98
- 99. Principles of SystemsThinking •Interrelationships •Perspectives •Boundaries •Dynamics
- 100.
- 101. Complexity concepts informingtransformation • Emergence: Self-organizing, Attractors • Nonlinear: Small actions generate large effects (Butterfly effect) • Dynamical: Interactions within, between, and among subsystems and parts within systems can volatile, changing • Getting to Maybe: Uncertainty; unpredictable; uncontrollable; unanticipated consequences • Coevolution: Interdependence • Adaptation: Response to turbulence, changing conditions • Tipping points: Not knowable in advance, clear afterwards • Critical mass: Multiple pathways converging 101
- 103. The Water ofSystems Transformation
- 105.
- 106.
- 107. Microfinance: Grameen Bank •“solidarity group lending” • dividing a loan among several recipients • limited to women, a stipulation created by Yunus • He felt that women were more reliable, typically, at fulfilling their payment obligations and cooperating with their peers • women would overwhelmingly invest in their families’ wellbeing • envisioned microfinance as a vehicle for female empowerment + material gains. 107
- 108.
- 109.
- 110.
- 111.
- 112. Dimension Definition Relevance Strategicadvances that accelerate or shift the trajectory of progress toward low carbon and/or climate resilient development in targeted countries and sectors. Scale Catalytic processes that significantly expand and diffuse the development and deployment of low carbon and climate resilient technologies, infrastructure, and other innovations, increasing their supply and access. Systemic change Deep, fundamental shifts in patterns of individual, institutional, community, and/or private sector decision making, actions, and behaviors in targeted markets or other systems that advance low carbon and climate resilient development. Sustain- ability Transformational changes that are designed to be financially, economically, environmentally, socio-politically, or physically robust and resilient. These changes are durable and lasting in ways that lessen the likelihood of reverting back to past practices and persist over time. Dimensions of Transformational Change for the Climate Investment Fund
- 113.
- 114. Carol H. Weiss Theoryof Change: “Nothing as Practical as a Good Theory” 1995 Aspen Institute Nov. 7, 1926 - Jan. 8, 2013 114
- 116.
- 119.
- 121. Michael Scriven. Thelogic of evaluation. 121
- 122. The logic ofevaluation is a four-step process that includes: 1) defining the criteria that will be used to evaluate something; 2) setting standards of performance on those criteria; 3) measuring the actual performance; and 4) synthesizing the results to reach an evaluative judgment
- 123. Michael Scriven. Thesynthesis problem 123 How to combine the factual premises with the value premises.
- 124. Criteria Matter 1. Criteriaconstitute the nuclear core of evaluation’s energy function: rendering judgment. Without criteria there can be no judgment. Without judgment there can be no evaluation. 2. Criteria express, manifest, encompass, make explicit, and operationalize what is valued. Criteria mediate the conversion of values into judgments. 3. Criteria prioritize what is important. 4. Criteria direct what questions to ask, data to collect, and results to highlight. 5. Criteria focus evaluation reporting and conclusions: To what extent and in what ways have criteria been met?
- 125. OECD/DAC criteria, 1991 •Relevance (Project addresses priorities) • Effectiveness (Project outcomes) • Efficiency (Project cost-benefit) • Impact (Results) • Sustainability (continuity)
- 128.
- 131.
- 132. The traditional, normalpractice of evaluation Footings: Language of problem and solution (versus systems and adaptations) Project-oriented and solution-focused (versus transformation-driven and adaptation oriented) Assumption of a ‘right’ answer (versus accepting that inquiry and learning are ongoing) Oriented to goal achievement (vs systems trransformation) Assumes solutions that can be engineered (versus embracing complex systems thinking) Evaluation Roundtable April 2019 132
- 133.
- 134.
- 135.
- 137. Peter Drucker “The greatest dangerin times of turbulence is not the turbulence – it is to act with yesterday’s logic.”
- 138.
- 142. Greenwashing Disinformation disseminated byan organization so as to present an environmentally responsible public image. Greenwashing is the process of conveying a false impression or providing misleading information about how a company’s products are more environmentally sound. Greenwashing is considered an unsubstantiated claim to deceive consumers into believing that a company's products are environmentally friendly. Greenwashing is a play on the term "whitewashing," which means using misleading information to gloss over bad behavior.
- 143.
- 144. SUSTAINABLE SYSTEMS TRANSFORMATION EVALUATIONCRITERIA 2. Complex Dynamic Systems Focus -- Systems are clearly the evaluand -- Natural and human system(s) are targeted as interrelated and interdependent -- Complexity theory and understandings
- 146. Seeing Through A ComplexityLens “You don't see something until you have the right metaphor to let you perceive it”. Thomas Kuhn 146
- 147. Inspired by JeffConklin, cognexus.org Time Complex development situations are ones in which this… 147
- 148.
- 149. Turns out tobe this… Time 149
- 150.
- 152. SYSYTEMS COMPLEXITY • Interconnections •Perspectives • Boundaries • Dynamics • Emergence • Nonlinearities • Uncertainty • Uncontrollable • Adaptation Beyond Projects and Programs
- 155. SUSTAINABLE SYSTEMS TRANSFORMATION CRITERIA 3. ResilientEcological Sustainability. Evaluate transformational sustainability as manifesting ecosystem resilience and adaptability at the nexus between humans and the environment. • Employ a dynamic view of sustainability. • Make the ecosystem viability the focus of sustainability not a program, project, or intervention. • Not sustainability as continuity
- 156.
- 159. The International DevelopmentEvaluation Association held its 2019 conference in Prague. The theme was Evaluation for Transformative Change. The conference concluded with a “Declaration” of 10 recommendations, one of which addressed sustainability as a universal evaluation criterion: In all our evaluations, we commit to evaluating for social, environmental and economic sustainability and transformation…. (IDEAS, 2019) SUSTAINABILITY as a universal criterion in evaluations
- 161.
- 162. • The CanadianEvaluation Society (CES) is revising its strategic plan and making environmental sustainability a guiding principle. • The 2020 conference organizing committee has also selected environmental sustainability as one of its foundational values.
- 163. 4. Eco-efficient FullCost Accounting • Document and assess the full costs and benefits of systems transformations, including economic, social, and environmental dimensions. • Compare the full costs and benefits of baseline versus transformed systems. Evaluate whether, how, and to what extent transformational engagement generates net eco-efficient benefits.
- 164.
- 165. Business Roundtable Redefines thePurpose of a Corporation to Promote “An Economy That Serves All.”
- 166. 25 Years AgoI Coined the Phrase “Triple Bottom Line.” Here’s Why It’s Time to Rethink It. HARVARD BUSINESS REVIEW John Elkington
- 170. Ecoefficiency Creating and deliveringgoods and services while using renewable resources and creating less waste (zero) and pollution (zero). Full cost accounting TEEB: The economics of ecosystems and biodiversity
- 173.
- 174. GLOBAL ALLIANCE FORTHE FUTURE OF FOOD
- 175. 5. Diversity/Equity/Inclusion (DEI). •Evaluate how transformational engagement manifests the values of diversity, equity, and inclusion. • Evaluate whether, how, and to what extent transformational engagement enhances systems level diversity, equity, and inclusion.
- 178. Oxfam: World's 22richest men have more than all women in Africa
- 179. Oxfam, January 2020,Davos • The number of billionaires has doubled in the past decade • At least 119 billionaires worth about $500 billion attended Davos this year, Bloomberg News reported. • 2,153 billionaires now have more wealth than the 4.6 billion poorest people on the planet.
- 180. VALUES: DEI •Diversity • Equity • Inclusion
- 181. All evaluations should addressequity Jara Dean-Coffey
- 182.
- 184. • The Diversity,Equity & Inclusion Working Group (DEIWG)) is twofold: • to advise the Board on issues relating to diversity, equity and inclusion within CES, and • to promote inclusive evaluation practice within the evaluation community, in reflection of the CES’s revised competencies and ethical principles.
- 185. 6. Interconnectedness momentum Identify,understand, and evaluate the interconnections that are essential and integral to transformation. • Evaluate whether, how, and to what extent interconnections among people, networks, institutions, ideas, and movements are deepened and enhanced to support, nurture, catalyze, and accelerate transformational trajectories. • Evaluate whether, how, and to what extent dysfunctional and constraining interconnections are disrupted and broken to liberate positive transformational energy and momentum.
- 187. “When everything isconnected to everything else, for better or worse, everything matters.” — Bruce Mau Canadian designer and educator Evaluation matters
- 188.
- 189. TRANSFORMATION INTERCONNECTIONS GLOCAL IMPACTS: Micro-meso-macrointerconnections LOCAL GLOBAL
- 190. Think globally Act locally& globally Evaluate interactions TRANSFORMATION
- 191.
- 192.
- 193.
- 194. Posterity – notHistory, nor the Future • This is part of a new ethics for this era. It requires imagining and ‘designing’ futures fully cognisant of the people who will inhabit them. Evaluation professionals use the past to help inform the future. But do our evaluation questions and criteria reflect a clear notion of the responsibility of this generation to future generations, the need to break from the past – and hence the stewardship that is needed to ensure sustainable development ‘for posterity’?
- 195.
- 196. Criteria for EvaluatingTransformation 1. Transformation Fidelity 2. Complex adaptive systems focus 3. Resilient adaptive capacity 4. Ecoefficiency (full cost accounting) 5. Diversity, EQUITY, Inclusiveness 6. Interconnectedness momentum
- 197. DAC Technical Criteria • Relevance(project addresses priorities) • Effectiveness (project outcomes) • Efficiency (project cost- benefit) • Impact (Project results) • Sustainability (continuity) • Coherence Transformation Values- Based Criteria • Transformation Fidelity • Complex adaptive systems focus • Resilient adaptive capacity • Ecoefficiency (full cost accounting) • Diversity, EQUITY, Inclusiveness • Interconnectedness momentum
- 200. Your questions &comments