Asian Cities Climate Change Resilience Network Initiative Final Evaluation
1 like2,919 views
The document presents a summative evaluation of the Rockefeller Foundation's Asian Cities Climate Change Resilience Network (ACCCRN) initiative, which aims to enhance climate resilience among vulnerable urban populations in Asia. Launched in 2008, the initiative has allocated approximately $59 million to help cities build capacity for climate resilience, share practical knowledge, expand networks, and ensure effective management and accountability. The evaluation highlights both successes and challenges faced during the initiative, providing lessons and recommendations for future resilience efforts.
![SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 55
surprisingly absent from ACCCRN, and urged ACCCRN
to ensure that in its second half, poor and vulnerable
people would be fully included in implementation
projects, and would have their priorities heard, their
adaptive capacity increased, and their lives improved.
There is little evidence that the lives of the poor and
vulnerable have improved. The main reason for this is
simply the paucity of measurement of the quality of
citizens’ lives [although this was a recommendation of
the MTE].
In addition to the little measurement that has been
undertaken in ACCCRN to demonstrate changes in
lives, measurement of the quality of a life is a field of
study in its own right, from the basic discussion about
how to measure poverty (The Economist, 2011), to the
more wide-ranging debates about well-being (Stiglitz
et al., 2009). The ACCCRN Results Framework provides
two measures against which it planned to assess “lives.”
• Measurable improvements in the well-being of
citizens within ACCCRN cities (disaggregated by
those defined as poor, vulnerable to climate change
effects, and by gender) – to be measured through
city-level interviews, reflective group analysis and
national-level perceptions. This is a qualitative
measure, which can be appropriate to obtain
information on the multi-dimensional aspects of
lives and to obtain people’s own views, using their
own self-anchored metrics.
• Decrease in proportion of poor (men and women)
within total population in ACCCRN and new cities
– to be measured through measurable impact
on poverty as a result of UCCR activity. This is
a quantitative measure, which should draw on
available national and sub-national survey data.
Properly determining the impact of ACCCRN on lives
would require a pre-ACCCRN baseline. This can be
achieved through recall techniques for qualitative data,
and quantative data should be available from archives.
Ideally, this would also include comparator cities where
ACCCRN has not worked, to take account of any
underlying trends affecting lives.
In the absence of any of these data, the evaluation
cannot make an assessment of whether the lives of
poor and vulnerable people have improved. Even
when the data are available, the indicators would still
need quantification with target levels. In other words, a
judgementwouldstillhavehadtobemadeonhowmuch
improvement in well-being and how much decrease
in the proportion of poor people would reasonably
constitute impact. Given that ACCCRN is running
for a further two years, there is still an opportunity
to collect the data and undertake this high-level
impact assessment. This is not a recommendation,
as the judgement as to whether this would be useful
information for the Foundation is for the Foundation to
take.
The second reservation about this impact objective
is the extent to which the lives of the poor and
vulnerable were expected to be affected by ACCCRN
interventions and, thus, how much impact might be
expected. The classical approach to poverty reduction
is to design poverty-targeting, pro-poor interventions.
However, as noted in assessing the TOC, ACCCRN is
predicated on the assumption that: “By working at the
city level you can have more impact on the poor and
vulnerable communities than by working exclusively
with poor and vulnerable.” Given the above concern
about measurement, it is clear there is not enough
empirical evidence from ACCCRN to either prove or
disprove this assumption.
An analysis of the targeting, in general and of targeting
the poor and vulnerable in particular, (which is
analyzed in Annex 8), shows a variable approach to
the consideration of beneficiaries, and poverty and
vulnerability targeting. Some grants target whole
systems or whole-of-city (e.g. UCCR urban plans
for Chiang Rai, hydrological modeling in Quy Nhon),
others aim to partially or wholly benefit the poor and
vulnerable through area-based or problem-based
targeting (e.g. flood early warning system in Semarang,
waterborne disease surveillance system for Indore),
and some do not appear to have considered the issue.
Overall, it is not possible to assess the impact on urban
dwellers, other than by anecdote and through inference](https://image.slidesharecdn.com/acccrn-finalevaluation-reissueddec2016-170107001156/85/Asian-Cities-Climate-Change-Resilience-Network-Initiative-Final-Evaluation-77-320.jpg)
![SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 63
Lastly, on knowledge, the successes of ACCCRN will be
undermined if there are attempts to reify the approach for
the sake of legacy. The knowledge and the approach will
continue to evolve and be taken up by others. The legacy
is also contingent on keeping the ACCCRN knowledge
repository available in on-line space. This is not mutually
dependent on the networking function. The collection
needs to be kept in as many libraries as possible and the
un-gating fee should be paid for any gated materials, in
order to widen the audience.
8.2 Stakeholders
A key aspect of sustainability is whether the cadre of
champions continues to promote UCCR and UCCR
approaches. A number of activities are underway to
do that, including linking to national organizations and
training bodies. Sustainability should not be heavily
predicated on the continued existence of an ACCCRN
network. Where members of the cadre are in touch with
each other and sharing ideas, it will assist sustaining
UCCR, but it is not essential. The prospects for clusters
of ACCCRN’s network of relationships being sustained
upon completion of grant spending are most positive
via membership of existing, mostly national, networks
in countries such as Vietnam and Indonesia. These
granteesandtheirnetworksareprogressivelyintegrating
a climate change mitigation and adaptation lens into a
broader urban development framework. The extent to
which the concept of climate change resilience persists
as an organizing principle of their work depends to some
extent on the priorities of their next donor relationship.
Does the ACCCRN network need to be resilient and
sustain indefinitely into the future? Research into
networks established by IDRC suggest that donor-sup-
ported networks can be useful and valid time-bound
channels for dissemination of knowledge, but only for
as long as:
• the knowledge remains current and useful
• network members continue to see value in relating
with each other and being part of the network
• required resources remain available.
The success and sustainability of the ACCCRN
network depends on the willingness of its members
to participate. Neither the MTE nor this evaluation
have found evidence that ACCCRN partners are
seeking to become more regionally networked. In
Vietnam and Indonesia, there are emerging examples
of interdependent and cooperative styles of working
among a few ACCCRN-supported organizations,
independentofACCCRNsupport.Buttheheterogeneity
of country actors coupled with the external contexts
in India and Thailand means these countries are less
well placed for nurturing collaborative organizational
relationships for knowledge sharing, advocacy, and
building communities of practitioners.
8.3 Policies and plans
The development process needs to be the main vehicle
for addressing climate change. Hence building UCCR
needs to be a mainstream activity, and mainstreaming
needs to be centered around development planning
(Roman et al., 2012). The extent to which this is the
case appears variable.
In some cities, such as Surat, there is little connection.
In others, such as Bandar Lampung, the CRS informed
the approach to adaptation in the Mid-term Regional
Development Plan 2010-2015 of Bandar Lampung.
Field research suggests that this was partly because:
“all the key selected consultants [for the Mid-Term
Plan] have been part of the City Team” and, where
the consultants were different (such as for the Bandar
Lampung’s Spatial Planning (RTRW) Document for
2010-2030), “the VA and the City Resilience Strategy
are not” (Lassa and Nugraha, 2014).
Uptake of the CRS into formal planning in Bandar
Lampung is also attributed to defining UCCR as a
planning and financial development issue, not only
an environmental one, as well as having the CRS
translated into “development language” through its
incorporation into the local development plans by the
Local Planning Development Board, BAPPEDA.](https://image.slidesharecdn.com/acccrn-finalevaluation-reissueddec2016-170107001156/85/Asian-Cities-Climate-Change-Resilience-Network-Initiative-Final-Evaluation-85-320.jpg)
Asian Cities Climate Change Resilience Network Initiative Final Evaluation
- 1. Summative Evaluation The Rockefeller Foundation Asian Cities Climate Change Resilience Network Initiative September 2014 VERULAM ASSOCIATES LTD THE RockefellerFoundation MONITORING&EVALUATIONOFFICE Supported by
- 2. About Verulam Associates Verulam Associates is a UK-based international development consultancy company with an associate company in Bangladesh. Verulam’s focus is on organizational learning, delivered through evaluation, review, policy advice and support for organizational change. With a long-standing commitment to work in South and Southeast Asia, Verulam’s clients include the World Bank, Asian Development Bank, the European Commission, leading bilateral aid agencies including the UK Department for International Development, Swedish International Development Cooperation Agency, Norwegian Agency for Development Cooperation, Swiss Agency for Development and Cooperation, and the Government of the Netherlands, as well as major international and South Asian NGOs. Verulam’s approach is based on process consultancy and informed by clinical organizational understandings and insights. Verulam has undertaken regular monitoring of the Rockefeller Foundation’s ACCCRN Initiative since 2009 and conducted the Initiative’s Mid-Term Evaluation in 2011 and this Summative Evaluation in 2014. About The Rockefeller Foundation For more than 100 years, The Rockefeller Foundation’s mission has been to promote the well-being of humanity throughout the world. Today, The Rockefeller Foundation pursues this mission through dual goals: advancing inclusive economies that expand opportunities for more broadly shared prosperity, and building resilience by helping people, communities, and institutions prepare for, withstand, and emerge stronger from acute shocks and chronic stresses. Monitoring and Evaluation at the Foundation Committed to supporting learning, accountability, and performance improvements, the Foundation’s Monitoring and Evaluation team works with staff, grantees, and partners to monitor and evaluate the Foundation’s pathways to impact in the short- and long-term, and to capture lessons about what works and what doesn’t across the Foundation’s diverse portfolio. The contents of this report are the views of the authors and do not necessarily reflect the views or policies of the Rockefeller Foundation. © 2014, The Rockefeller Foundation Cover photo: ©Gitika Saksena for Robin Wyatt Vision
- 3. Summative Evaluation The Rockefeller Foundation Asian Cities Climate Change Resilience Network Initiative September 2014 THE RockefellerFoundation MONITORING&EVALUATIONOFFICE
- 4. ©NicDunlop
- 5. Table of contents Acronyms iv Acknowledgements vii Preface ix Summary overview xi 1. Introduction 1 1.1 Climate change and urbanization 1 1.2 The Rockefeller Foundation and ACCCRN objectives 2 1.3 ACCCRN mid-term evaluation 3 1.4 ACCCRN summative evaluation 4 1.5 Methodology 4 1.6 Report 5 2. ACCCRN implementation 7 2.1 Growth of ACCCRN 7 2.2 Grantees and grants 8 3. Relevance and design 15 3.1 The relevance of ACCCRN 15 3.2 ACCCRN’s theory of change and design assumptions 16 3.3 Consideration of governance factors 19 3.4 Consideration of socio-economic factors 23 4. Efficiency 25 4.1 Use of Foundation resources 25 4.2 Grants and grantees 26 4.3 Program outcomes vs. city projects 29 4.4 Internal communications 30 5. Effectiveness 33 5.1 Articulating the UCCR concept 33 5.2 Putting the ACCCRN design into practice 37 5.3 ACCCRN as a network 40 5.4 Communications, publications and M&E grants 43 5.5 Engagement and leverage 45 6. Influence 47 6.1 Actors and audiences 47 6.2 Influence at the international level 48 6.3 Influencing at national and city levels 48 6.4 ACCCRN’s wider influence 49 6.5 ACCCRN’S influence in the Foundation 50
- 6. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVEii 7. Impact 53 7.1 Demonstrating a range of approaches, processes and practices 53 7.2 Impact on urban dwellers 54 7.3 Capacities of a cadre of champions 56 7.4 City impact 57 8. Sustainability 61 8.1 Knowledge and learning 61 8.2 Stakeholders 63 8.3 Policies and plans 63 8.4 Finance 64 9. Conclusions 65 9.1 Outcomes 65 9.2 Impacts 67 9.3 Successes 68 9.4 Challenges 68 10. Lessons and recommendations 71 10.1 Lessons for ACCCRN 71 10.2 Lessons for the Rockefeller Foundation 72 10.3 Recommendations 73 References 81 LIST OF TABLES Table 1: Breakdown of grants by outcome 10 Table 2: Breakdown of grants by country and city 10 Table 3: Breakdown of all ACCCRN grants, by grantee and level 11 Table 4: Theory of change assumptions 17 Table 5: Comparison of resilience indicator frameworks 36 Table 6: Six different ACCCRN models 38 Table 7: Summary of current and proposed UCCR sustaining actions 62 LIST OF FIGURES Figure 1: Map of cities in asia working with ACCCRN and related initiatives 7 Figure 2: Phasing in ACCCRN 8 Figure 3: Process steps in ACCCRN phases 2 and 3 9 Figure 4: Process steps in ACCCRN phases 2 to 4 9 Figure 5: Breakdown of grants by country. 10 Figure 6: The ISET urban climate resilience planning framework 34 Figure 7: Idealized ACCCRN approach 53
- 7. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE iii Figure 8: Diverse 1st and 2nd generation ACCCRN models 54 Figure 9: Closing the CRS loop with a resilience review 75 Figure 10: Linking CRSs with overall planning and finance 76 ANNEXES Annex 1: Terms of reference Annex 2: Evaluation matrix Annex 3: Grant data Annex 4: Climate change as a wicked problem Annex 5: The UCCR concept Annex 6: The ACCCRN models Annex 7: Evolution of the ACCCRN Network Annex 8: List of city projects targeting the poor and vulnerable Annex 9: ACCCRN leverage Annex 10: People met All annexes listed can be found in the separate annex document for this report.
- 8. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVEiv Acronyms 100RC 100 Resilient Cities ACCCRN Asian Cities Climate Change Resilience Network ADB Asian Development Bank APCO APCO Worldwide Inc. APEKSI Association of Indonesian Municipalities AR4/5 Fourth/Fifth Assessment Report BBC British Broadcasting Corporation CC Climate change CCA Climate change adaptation CCCO Climate Change Coordination Office CCCI Cities and Climate Change Initiative CCR Climate change response CDIA City Development Initiative for Asia COHED Centre for Community Health and Development COP21 Conference of the Parties 21 CRS City resilience strategy CtC Challenge to Change DFID Department for International Development DNPI Dewan Nasional Perubahan Iklim – National Council on Climate Change (Indonesia) DONRE Department of Natural Resources and Environment DRR Disaster risk reduction GEAG Gorakhpur Environmental Action Group GFDRR Global Facility for Disaster Reduction and Recovery GIZ German International Cooperation GoI Government of Indonesia Green ID Green Innovation and Development Centre HABITAT III Third UN Conference on Housing and Sustainable Urban Development HFA Hyogo Framework for Action IAP ICLEI-ACCCRN Process IATI International Aid Transparency Initiative ICA Indonesia Climate Alliance ICCCAD International Centre for Climate Change and Development ICLEI International Council for Local Environmental Initiatives ICRIER Indian Council for Research on International Economic Relations IDRC International Development Research Centre IDS Institute for Development Studies
- 9. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE v IFI International Finance Institutions IFRC International Federation of Red Cross and Red Crescent Societies IIED International Institute for Environment and Development IPCC International Panel on Climate Change IRAD Integrated Research and Action for Development ISET Institute of Social and Environmental Transition JNNURM Jawarlal Nehru National Urban Renewable Mission KfW KfW Bankengruppe – German development bank M&E Monitoring and evaluation M-BRACE Mekong Building Resilience in ACCCRN Cities to Climate Change MCI Millennium Cities Initiative MoE Ministry of Environment MONRE Ministry of Natural Resources and Environment MoPW Ministry of Public Works MOU Memorandum of understanding MTE Mid-Term Evaluation NCCC National Climate Change Committee NGO Non-governmental organization NIUA National Institute of Urban Affairs NISTPASS National Institute for Science and Technology Policy and Strategy Studies OECD Organisation for Economic Cooperation and Development PDIA Program-driven interative adaptation PEARL Peer Exchange and Reflective Learning RAN-API Climate Change Adaptation National Action Plan (Indonesia) RAN-PI Climate Change National Action Plan (Indonesia) RF The Rockefeller Foundation SAG Strategy and Alignment Group SC Steering committee SDC Swiss Agency for Development Cooperation SDG Sustainable Development Goal SLD Shared learning dialogue SRC Stockholm Resilience Centre TARU TARU Leading Edge Pvt. Ltd. TEI Thailand Environmental Institute TERI The Energy Research Institute TOC Theory of change
- 10. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVEvi TORs Terms of reference UCCR Urban Climate Change Resilience UCCRP Urban Climate Change Resilience Partnership UCCRTF Urban Climate Change Resilience Trust Fund UCR-CoP Urban Climate Resilience Community of Practice UNFCCC United Nations Framework Convention on Climate Change UN-HABITAT United Nations Human Settlements Program UNISDR UN Office for Disaster Risk Reduction USAID United States Agency for International Development VA Vulnerability assessment VFM Value for money VIAP Vietnam Institute for Architecture, Urban and Rural Planning WG Working group WUF World Urban Forum
- 11. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE vii Acknowledgements This evaluation was undertaken by Verulam Associates Ltd., the ACCCRN Monitoring and Evaluation Grantee, under the technical direction of Paul Thornton. The evaluation team, led by Julian Barr (Co-Chair of Itad), included Chris Albertyn, Dr. Aditya Bahadur, Moho Chaturvedi, Dr. Wijitbusuba Ann Marome, Sarah Standley and Vu Xuan Nguyet Hong. Itad’s Sarah Standley led on compiling data and other research evidence and Susie Bartlett did a sterling job of anchoring all our logistics. At Verulam Associates, management support was provided by Masum Khan, peer review by Paul Thornton and quality assurance and grant management by Dr. Hilary Thornton. The Rockefeller Foundation Evaluation Office in New York and the ACCCRN team members in Bangkok and New York were very accommodating of the evaluation team and responsive to all our requests. In particular, Laura Fishler and Fern Uennatornwaranggoon gave excellent support and advice, not least on the ACCCRN documents. We would like to thank all those who agreed to meet with the evaluation team and responded willingly with time and information including ACCCRN grantees at country and regional level, ACCCRN partners at city and national level, and the wide range of other key informants. We would also like to offer particular thanks to those who made invaluable contributions in arranging a complicated agenda with many meetings and intricate itineraries – specifically: Piva Bell at Mercy Corps and Irvan Pulungan at the International Council for Local Environmental Initiatives (ICLEI) in Indonesia; Pakamas Thinphanga at the Thailand Environmental Institute (TEI), Bangkok; Dr. Umamaheshwaran Rajasekar at TARU Leading Edge Pvt. Ltd. (TARU), Bijay Kumar Singh at the Gorakhpur Environmental Action Group (GEAG), and Sunandan Tiwari at ICLEI in India; and Ngo Le Mai at ISET-Vietnam.
- 13. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE ix Preface Launched in 2008, the Asian Cities Climate Change Resilience Network (ACCCRN) Initiative aimed to catalyze attention, funding, and action for building the climate change resilience of vulnerable cities and people in Asia. Given that current estimates forecast that about 55 percent of Asia’s population will be living in urban centers by 2030, the ACCCRN Initiative is built on the premise that cities can take actions to build climate resilience – including drainage and flood management, ecosystem strengthening, increasing awareness, and disease control – which can greatly improve the lives of poor and vulnerable people, not just in times of shock or stress, but every day. At the time the initiative was launched, the concept of urban resilience and models for implementing it were nascent and emergent. ACCCRN proved to be an important experiment and “learning lab” for the Foundation and its grantees and partners to build capacity in cities to better understand and implement resilience solutions to the often devastating shocks and stresses of climate change. The initiative was effective in the initial 10 ACCCRN cities and, later, in an additional 40 cities. As part of our Foundation-wide commitment to learning and accountability to our grantees, partners and stakeholders, we undertook an independent evaluation of the work of the initiative in 2014 to assess what worked well and not so well in ACCCRN. Conducted by Verulam Associates and ITAD, who also conducted a mid-term evaluation of the ACCCRN Initiative in 2011, this summative evaluation highlights successes, but also provides an important moment to reflect on the challenges we faced and on what we can do better or differently going forward. We are pleased to share the results of this evaluation and to contribute to the broader learning process in the field of urban resilience. The evaluation has provided a valuable opportunity for the Foundation to reflect on the impact and promise of this body of work. We remain committed to the continued leverage of our experience and our extensive network to enable broader interest in and commitment to urban climate change resilience. The success of ACCCRN has already directly informed other resilience efforts at the Foundation, most significantly 100 Resilient Cities, a $150 million effort to help build urban resilience in 100 cities around the world. Organizations, systems, and societies, like humans, are not born with resilience – we learn it, we adapt it, and we improve upon it. Nancy MacPherson Ashvin Dayal Managing Director, Evaluation Associate Vice President, Managing Director, Asia The Rockefeller Foundation The Rockefeller Foundation
- 14. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVEx ©GitikaSaksenaforRobinWyattVision
- 15. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE xi Summary overview Our planet’s climate is unequivocally warming. At the same time, the global population continues to urbanize, with many of the risks of climate change concentrated in urban areas. In Asia, the fastest urbanizing region of the world, 64 percent of the population will live in cities by 2050. The 500 million Asians who currently live in slums are among those most vulnerable to climate change. In 2007, in order to address this nexus of urbanization, climate change, poverty, and vulnerability in Asia, the Board of Trustees of The Rockefeller Foundation invested $70 million in support of its Climate Change Initiative. The initiative has three distinct and separate components: i) the Asian Cities Climate Change Resilience Network (ACCCRN) component, which is the focus of this evaluation, ii) the African Agriculture Climate Change Resilience component, and iii) the US Climate Change Policy component. The objectives of the overall Climate Change Initiative are to: 1. build climate change resilience of poor and vulnerable urban populations in the developing world through developing, promoting, and disseminating models for community resilience 2. build climate change resilience of poor and vulnerable smallholder farmers in Africa through climate change-sensitive agricultural development practices 3. increase funding and support for climate change resilience of poor and vulnerable people in the United States and potentially in the developing world, by influencing US mitigation policy and practice. The Asian Cities Climate Change Resilience Network (ACCCRN) Of the total funds approved for the overall Climate Change Initiative, approximately $42 million were set aside to implement the Asia Cities Climate Change Resilience Network (ACCCRN) component over a six-year period: 2007–2012. Of the $20 million approved for the two-year initiative extension, $17 million was allocated for ACCCRN, bringing the total ACCCRN budget to $59 million. Granting was extended for a further two years to 2014, with all activities concluding in 2016. ACCCRN had four intended outcomes. • Outcome 1 - Capacity. Improve the capacity of ACCCRN cities to plan, finance, coordinate, and implement climate change resilience. • Outcome 2 – Knowledge, learning, and deepening of experience. Share practical knowledge on urban climate change resilience (UCCR) in order to deepen the quality of awareness, engagement, demand, and application by ACCCRN cities and other stakeholders.
- 16. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVExii • Outcome 3 – Expansion, networking, scaling-up. Expand UCCR, with ACCCRN and new cities taking action through existing and additional financial, policy, and technical support generated by a range of actors, particularly new donors. • Outcome 4 – Organizational excellence, management, accountability, and learning. Ensure that The Rockefeller Foundation’s ACCCRN team operates effectively and efficiently, and is relevant and accountable to stakeholders and the context in which it operates, providing leadership and contributing to the Foundation’s strategy and mission. The 2011 mid-term evaluation of ACCCRN found four areas of need. 1. Understanding. ACCCRN was a pioneering and highly relevant initiative, enabling, supporting, and exploring approaches and methodologies to vulnerability assessment and the design of city-level resilience plans, but the understanding of resilience was weak in some cities and needed to be strengthened in order for the initiative to succeed. 2. Documentation. The UCCR approach needed continued reflection and documentation. 3. Sharing and institutionalization. Sharing of UCCR experiences needed to be stepped up, as did the adoption of UCCR ideas and the institutionalization of the city-level advisory committees and working groups. 4. Networking. There was no compelling value proposition for another network as proposed initially by ACCCRN but there was definitely a demand for networking and sharing of knowledge and experience among individuals and groups. In 2014, a summative evaluation was conducted to assess ACCCRN and make recommendations regarding the following. 1. Influence and impact. Assess to what extent ACCCRN’s influence and impact can be attributed to The Rockefeller Foundation’s support. 2. Outcomes and achievements. Assess both intended and unintended outcomes and achievements of the initiative, and the extent to which they appear to be sustainable beyond the Foundation’s involvement. 3. Relevance and added value. Assess ACCCRN’s relevance and value added to the evolution of UCCR in Asia and the new work of the Foundation, including the validity and evolution of the theory of change and hypothesis1 of the initiative in relation to challenges of UCCR in Asia. 4. Leadership and management. Assess the efficiency and effectiveness of the Foundation’s leadership and management of the Initiative. 5. Knowledge contributions. Assess the quantity and quality of the knowledge contributions of ACCCRN to the fields of urban climate change resilience (UCCR) and resilience in general, as well as to network development efforts. The evaluation also sought to make recommendations to the Foundation on further actions needed to nurture and sustain the achievements of the initiative including resource mobilization and stakeholder 1 “Demonstrating contextually appropriate models of urban climate resilience, combined with cross-learning and support for replication and scaling up, can contribute to improved and more rapid development of urban climate resilience models throughout the developing world.”
- 17. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE xiii engagement, optimal timing and considerations for the Foundation’s consolidation strategy for ACCCRN, and implications of the work of ACCCRN for the new resilience work, particularly 100RC, the Global Resilience Partnership (GRP), and the overall resilience goal of the Foundation. Overall conclusions The summative evaluation concurs with the mid-term findings that ACCCRN was a well-conceived, timely and needed idea, and has been highly relevant to Asian cities where the twin-pressures of urbanization and climate change are keenly felt. It concludes overall that ACCCRN is a successful initiative which has developed a relevant approach to UCCR, tested and adapted this approach in a range of cities, built capacities to strengthen UCCR, produced a good body of published documentation on ACCCRN, and leveraged new funding and actors for building UCCR. As might be expected in a large and complex enterprise such as this, not everything has gone as planned. The sustainability at city level has yet to be tested, and it has proven difficult to establish a network of UCCR practitioners. Leveraging a fully collegiate way of working and realizing the potential complemen- tarity of grants has been suboptimal, and it has taken longer than anticipated to mesh The Rockefeller Foundation, DFID, and ADB ways of working to operationalize the Urban Climate Change Resilience Trust Fund (UCCRTF). Specific findings by Outcome OUTCOME 1 – CAPACITY 1. Capacity of city partners to plan, finance, coordinate, and implement climate change resilience strategies has improved in all cities. ACCCRN’s multi-stakeholder approach, and its iterative, learn- ing-by-doing modality has created new (more resilient) relationships among city actors, and in most cities, has improved the ownership of and commitment to building UCCR. The model of entering cities through a city climate change working group (of various modalities) was a pragmatic starting point. 2. City working groups have acquired new skills. Acquired skills, such as for undertaking vulnerability analyses and developing resilience strategies, mean increased ability to cope with the risks implied by climate change. However, it is less evident that capacities to deal with the uncertainty aspects of climate change have been emphasized. OUTCOME 2 – KNOWLEDGE, LEARNING, COLLABORATION 3. Individual and shared learning, and practical knowledge have contributed to building UCCR. Although this has been more individual than organizational, there have also been broad increases in appreciation of UCCR when, for example, systems-wide UCCR projects, such as early warning and surveillance systems were implemented, and when UCCR ideas were taken up into planning and policy, as seen in Vietnam and Indonesia.
- 18. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVExiv 4. ACCCRN and its partners have invested strongly in documenting the experiences and lessons. The International Institute for Environment and Development (IIED) research and publication grant has been effective in helping ACCCRN achieve a presence in the academic space around urban climate change, particularly around the operationalization of supporting cities to prepare for climate change. There are some obstacles to knowledge sharing regionally, due to language barriers, and nationally, due to the technical level of the writing. OUTCOME 3 – EXPANSION, NETWORKING, SCALING UP 5. Achievement of the network/networking objectives in ACCCRN has been challenging. Factors include the multifaceted aims of a network, and the diversity and geographic spread of potential members. It is not certain that the new attempt at networking has sufficiently addressed these. However, there is greater prospect for success in country-level networks that link to pre-existing groups and structures. 6. Achievements on scaling, replication, and leveraging have been much better. ACCCRN granted to ICLEI - Local Governments for Sustainability (ICLEI) to expand ACCCRN to 40 more cities (including in new countries), using a lighter-touch approach. Similarly, Mercy Corp in Indonesia has extended the approach to six more cities, and the Institute for Social and Environmental Transition (ISET) has levered new funds from USAID to implement an ACCCRN-based approach in four new cities in Vietnam and Thailand. A major result on leverage has been the establishment of the UCCRP with DFID, ADB and USAID. 7. Increased commitment to UCCR can be seen at country and city levels. At country level, there are examples of national commitment to UCCR-related issues, and at city level, governments have expressed interest in investing in UCCR. However, overall, the influence ACCCRN has achieved at the national level has been variable, related to the extent to which climate change has been given priority in policy-making circles and the strategy ACCCRN has taken in engaging in this space. OUTCOME 4 – ORGANIZATIONAL EXCELLENCE, MANAGEMENT, ACCOUNTABILITY, AND LEARNING 8. The Rockefeller Foundation ACCCRN team structure has allowed the Foundation to gain an in-depth knowledge of UCCR and how it gets put into practice. This placed the team in a strong position to lever other actors. The Strategic Advisory Group (SAG) was established to address concerns about coordination of grantees, though its focus has mostly been tactical. Overall, the evaluation considers that the potential complementarity and synergies of the mix of grantees that were brought together have not been realized. IMMEDIATE IMPACT – A DIVERSE RANGE OF UCCR APPROACHES, PROCESSES, AND PRACTICES 9. ACCCRN has generated a variety of models around a core ACCCRN approach. These processes, approaches, and practices have been tailored by grantees and host cities according to the expertise and orientation of the facilitating grantee, and in response to the national and city context. With at least seven emerging “models,” ACCCRN’s objective to develop and test a “diverse range of effective approaches, processes, and practices” has been fully met and a number have been scaled out.
- 19. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE xv ULTIMATE IMPACT – RESILIENCE AND CAPACITY OF A GROWING NUMBER OF ASIAN CITIES IS ENHANCED AND, THROUGH THIS WORK, THE LIVES OF POOR AND VUL- NERABLE MEN AND WOMEN ARE IMPROVED 10. A growing number of Asian cities have improved their UCCR through ACCCRN. These cities have contributed a number of smaller pieces in the larger UCCR jigsaw. Quantifying the actual improvement in UCCR is difficult, not least because ACCCRN was experimental in nature when the concept of resilience was just starting to take hold, and its cities have not adopted a measurement framework to assess impact. Successes Four particular ACCCRN successes highlight the importance of the pioneering approach initiated by the initiative. 1. UCCR as a relevant, operational approach. The Rockefeller Foundation was a “first mover” in setting up ACCCRN as a pioneering initiative to understand and build UCCR. The development of a multistakeholder-based process of studies (including vulnerability analysis) and collective reflection on these led to the production of a City Resilience Strategy, and is now recognized as an appropriate mechanism to help cities appreciate and build UCCR. In this novel field, working with an emergent strategy and following a learning-by-doing approach was entirely appropriate. The resultant learning on how to operationalize city-level intervention on UCCR is valued by a range of stakeholders. 2. UCCR literature. ACCCRN has created a large body of published research and documentation of the ACCCRN process and experience. Knowledge has been one of the initiative’s major outcomes. 3. UCCR partnership. A major success for ACCCRN has been its formalization of the Urban Climate Change Resilience Partnership (UCCRP) and its attendant Trust Fund. UCCRP is managed by the Asian Development Bank (ADB) but implemented jointly with ACCCRN. It is funded jointly by DFID and USAID. 4. Reputational legacy. ACCCRN’s grantees and participating cities have emerged with a strong, collective reputation for innovation in the field of UCCR. Furthermore, The Rockefeller Foundation has emerged with good reputational capital for having been an early, committed, and professionally engaged funder in this important and emerging field. Challenges Some areas have proven to be more challenging for ACCCRN. 1. Networking. As ACCCRN has progressed, its conception of “network” has become increasingly complex and “catch-all” in its proposed functions. While the interdependent links in proposed network objectives are conceptually coherent, the heterogeneous target membership (including individuals and organizations), geographic scope, language and cultural diversities, and expectations of multiple functions all mitigate against attracting active membership engagement. The emergence of more active national networks, linked to existing groupings and with a local identity and purpose, appears to be more realistic than a multi-faceted ACCCRN-oriented network. 2. Working collaboratively. While the spread of grants in ACCCRN was generally appropriate, major granting has been exclusively to single grantees, with little incentive to collaborate. The situation may
- 20. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVExvi be characterized as collegial rather than collaborative, and thus there have been some inefficiencies in achieving collective results 3. Closing the experimental loop. Experimentation in resilience building in a young field was appropriate. With seven practical ACCCRN models emerging from the natural experiment, analysis has yet to be done across all these models. Given the diversity of contexts (mainly of governance) and of partner approaches, it remains for the analysis to address the questions “what works, where, and why?” 4. Closing the city resilience strategy cycle. City resilience strategies (CRSs) identify vulnerabilities to climate change and prioritize a set of actions to address them. The process followed in creating the strategies was as important as the finalized documents. However, they have not been well linked into formal city development plans or finance mechanisms. If CRSs are not mainstreamed, they risk becoming peripheral. 5. Determining a city’s resilience. UCCR is a difficult concept to articulate and, with such a multi- dimensional concept, it is hard for city stakeholders to know whether their city is indeed becoming more resilient. The extent to which resilience is being built in cities, against any sort of calibration, is very subjective. To have a better feedback loop on whether resilience is improving, cities need a more routine approach to city-led monitoring and evaluation of the outcomes of UCCR actions, and some sort of assessment framework against which they can judge whether they are becoming more resilient. 6. Ensuring policy traction. The theory of change considered that knowledge, empirical evidence, and hands-on capacity building would achieve commitment to UCCR. This oversimplified the political and economic realities of cities and took for granted the importance of the national policy context in the motivating cities to engage. Lessons Lessons for ACCCRN. Having developed and tested an approach to UCCR, the key lessons for ACCCRN are about how to make UCCR practice stick for a whole range of stakeholders. 1. Messaging. Resilience is a difficult concept, and UCCR more so. ACCCRN messaging on UCCR is undoubtedly credible, but much of it has prioritized city projects. Much has also been in lengthy documents, targeted at more advanced audiences which practitioners have not found accessible. The lesson calls for considering the importance of messaging to non-technical practitioners and policymaker audiences, in appropriate languages and cultural styles. 2. Networking. Networking and building coalitions of connectors and champions is part of getting the message across. ACCCRN has given much attention to networking, though not always successfully. The value proposition of the to-be-refreshed ACCCRN network is not yet clear. Country networks are emerging as the most functional model for ACCCRN. The lesson calls for giving consideration to the people who will sell and scale the UCCR message. 3. Strategic opportunism. ACCCRN has been strategic at creating its own opportunities, convening donors at Bellagio, forming a partnership with ADB and DFID, and hosting and presenting at various conferences. Overall, however, it has not been close enough to national and global policy processes to identify strategic opportunities to promote the UCCR agenda. The lesson is to use political economic
- 21. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE xvii analysis during the early stages of engagement (at every level), and work with grantees who have the capability to use these analyses to inform strategic opportunism. General lessons from ACCCRN. The evaluation provides reflections and lessons from ACCCRN that are relevant and generalizable to The Rockefeller Foundation. 1. Discovery. ACCCRN helped shape and simultaneously achieve results in a new and complex area. The use of emergent strategy and experimentation were appropriate management approaches in this environment. This was time well invested. The lesson for the Foundation is to have some flexibility in the time it allocates to new initiatives in their discovery phase, with the available time related to the complexity and novelty of the field. 2. Tailoringtocontext.Theinitiativehasatleastsevenparallelmodelsconcurrentlyinexecutionacross two generations of models. This diversity was envisaged in the ACCCRN design, and contextually adapted models are more likely to be accepted and endure. The lesson reminds that one-size-fits-all approaches historically fail. A key aspect of tailoring to local context is understanding and relating to the local and national political economy. 3. Granting. ACCCRN gave grants to individual organizations, and much time and energy was then invested in trying to get the grantees to work together. This is paradoxical. It resulted in collegiality, but not collaboration. The lesson indicates that initiatives should consider collaborative grants and improved levels of transparency about grants within an initiative. 4. Working with other funders. The UCCR Partnership is a demonstration of the credibility of the UCCR concept, and of the Foundation’s convening and influencing power. However, operational- izing the idea has been difficult. The lesson has been that developing joint guidance on operating the trust fund, and achieving agreements from four funders with quite different funding models and operational requirements has been time-consuming – first in understanding the differences, and then in reaching consensus. 5. Paralleling100ResilientCities(100RC).ACCCRNcitiesreportedbeingconfusedbyTheRockefeller Foundation having two different streams of urban resilience funding. Some city stakeholders had interpreted 100RC as “ACCCRN 2.0” - an opportunity to access a further stream of funding to continue city projects. The lesson is that there is a need for a clear narrative, shared by ACCCRN and 100RC of the differences and complementarities between the two initiatives. It is also important to develop a narrative for the ACCCRN cities that do not qualify for 100RC funds, so that this is not seen as “failure.” 6. Adding a gender lens. ACCCRN is gender neutral. This means it has not been possible to see the different effects of its actions on men and on women. The lesson for the Foundation is that, due to the way the initiative has been designed and implemented, it does not appear to have lived up to the Foundation’s vision of, and support for, gender equality as a key element of inclusive economies.
- 23. 1 SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 1 Introduction The Asian Cities Climate Change Resilience Network (ACCCRN) was launched in 2007 to test and demonstrate a range of actions to build climate change resilience in urban areas. ACCCRN is one of three components of the Foundation’s Developing Climate Change Resilience Initiative, the only one with a focus on urban resilience in Asia. In its formative phase, ACCCRN worked with 10 Asian cities across India, Indonesia, Thailand and Vietnam. Since the 2011 mid-term evaluation, activity has expanded to include Bangladesh and the Philippines with about 50 cities engaged in some level of ACCCRN-related work. By the end of implementation in 2016, this is likely to exceed 100 cities.2 1.1 Climate change and urbanization ACCCRN was designed to address the twin pressures of climate change and urbanization. The Fifth Assessment Report (AR5) of the Intergovernmental Panel on Climate Change (IPCC, 2014) stated 2 The climate change and urbanization context of ACCCRN and the ACCCRN design concept and implementation details are described more fully in the Mid-Term Evaluation (Barr et al., 2012) and ACCCRN’s own documentation, including its monitoring reports, available at: http:// www.acccrn.org/ Warming of the climate system is unequivocal, and since the 1950s, many of the observed changes are unprecedented over decades to millennia. The atmosphere and ocean have warmed, the amounts of snow and ice have diminished, sea level has risen, and the concentrations of greenhouse gases have increased. In considering the impacts of this, the report went on to say: Many global risks of climate change are con- centrated in urban areas (medium confidence). Steps that build resilience and enable sustainable development can accelerate successful climate- change adaptation globally. As the planet continues to warm, so the world will continue to urbanize. Sustainable development challenges will be increasingly concentrated in cities, particularly in the lower-middle-income countries where the pace of urbanization is fastest (UNDESA, 2014). The Rio +20 Conference on Sustainable Development (UN, 2014) recognized that cities can lead the way towards economically, socially, and environmentally sustainable societies, but a holistic approach to urban planning and management is needed to improve living standards of urban and rural dwellers alike.
- 24. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE2 Whileglobally,2014figuresfind54 percentoftheworld’s population residing in urban areas, in Asia the figure is 48 percent. However, projections that the global proportion will reach 66 percent by 2050 also project thattheAsianurbanpopulationwillreachapproximately 64 percent (UNDESA, 2014). This indicates that Asia is the fastest urbanizing area of the world, with the urban population increasing by 1.5 percent annually. The region is making substantive investments to respond to this growth (e.g. infrastructure investment), which creates “a window of opportunity to make cities more – not less – resilient to a range of climate impacts, by influencing the growth trajectory of urban areas, given the long lifecycle of such investments” (The Rockefeller Foundation, n.d.). The urban population is not uniform. World Bank figures show that about 25 percent of the global population living below the poverty line is in cities. Within cities in Asia, between 24 percent and 35 percent of people live in slums – almost 500 million across Asia (UN-HABITAT, 2012). These urban poor are also those most vulnerable to the negative effects of climate change (UN-HABITAT, 2013). ACCCRN, therefore, aims to address the nexus of urbanization, climate change, and poverty and vulnerability. 1.2 The Rockefeller Foundation and ACCCRN objectives The Rockefeller Foundation’s mission is to promote the well-being of humanity, globally. It does this by advancing inclusive economies for shared prosperity, and through helping people, communities, and institutions to build resilience. The Foundation works at the intersection of four focus areas – advance health, revalue ecosystems, secure livelihoods and transform cities. Cities have always faced stresses, but the bases for this focal area include the rate at which change is occurring in cities, the immense scale of growing urbanization around the world, and climate change as the “great threat multiplier” (Rodin, 2014a). The Rockefeller Foundation now has a growing portfolio of resilience initiatives, including the recently announced Global Resilience Partnership (Rodin, 2014b), jointly funded with the United States Agency for International Development (USAID), and the centennial urban-specific resilience initiative 100 Resilient Cities (see 100 Resilient Cities website). Both build on and extend the experience of ACCCRN, which retains its particular focus on climate change as the stressor on city systems. Thus through ACCCRN, the Foundation aims to contribute to achieving the ultimate impact: Enhanced resilience and capacity of a growing number of developing country/Asian cities in relation to current and future climate risks, and improved lives of poor and vulnerable men and women.3 With the more immediate impact being: A diverse range of effective approaches, processes, and practices demonstrated in ACCCRN cities that build urban climate change resilience that incorporates the priorities of poor and vulnerable communities.4 ACCCRN seeks to contribute to impact by achieving three external outcomes and one internally focused outcome.5 • Outcome 1 – Capacity There is improved capacity to plan, finance, coordinate, and implement climate change resilience strategies within ACCCRN cities. • Outcome 2 – Knowledge, learning and deepening of experience (knowledge, learning and collaboration) Individual and shared learning and practical knowledge to build urban climate change resilience 3 For the original full ACCCRN Results Framework see Verulam/ACCCRN, 2010, and for the revised version, see Verulam/ACCCRN, 2013. 4 ibid 5 These outcomes were revised in 2013, to adjust Outcome 2 from net- worked knowledge and learning to individual and shared learning, and Outcome 3 to incorporate networking as a component of expansion and scaling-up.
- 25. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 3 deepens the quality of awareness, engagement, and application by ACCCRN cities and other stakeholders. • Outcome 3 – Expansion, networking, scaling-up (money, leverage, networks) UCCR is expanded with ACCCRN and new cities sharing experience through existing and new networks, and taking action through existing and additional support (finance, policy, technical) generated by a range of actors. • Outcome 4 – Organizational excellence, management, accountability and learning (management) The Rockefeller Foundation ACCCRN Team operates effectively efficiently and is relevant and accountable to stakeholders and the context in which it operates. It also provides leadership in The Rockefeller Foundation and contributes to its strategy and mission. 1.3 ACCCRN mid-term evaluation The aims of the formative mid-term evaluation (MTE) conducted in 2011 (Barr et al., 2012), included: • to assess the ongoing relevance and rationale of the Initiative to the field of urban climate change resilience (UCCR) in developing countries, and to the needs of key stakeholders • to assess the underlying hypothesis that demonstrating contextually appropriate models of UCCR, which, combined with cross-learning and support for replication and scaling-up, can contribute to improved and more rapid development of urban climate resilience models throughout the developing world • to assess the effectiveness of the Initiative in delivering its outputs and in making progress towards achieving its outcomes in the first phase of execution (2008- 2010) • to make recommendations for mid-course corrections to the Foundation on the Initiative. Key findings from the MTE included: • ACCCRN was “in the right place at the right time,” enabling, supporting and exploring approaches and methodologies to vulnerability assessment and the design of city-level resilience plans • the sustainability of a UCCR approach would depend on the success of continued reflection on the process, documentation and sharing of UCCR experiences, adoption of UCCR ideas, and the institutionalization of the city-level advisory committees and working groups • high likelihood that ACCCRN’s high-level objective of “a diverse range of effective approaches, processes, and practices to build UCCR” would be demonstrated in the 10 cities, and that this would scale-out to other cities. The MTE recommended a one-year extension of ACCCRN to ensure that the lessons from Phase 3 (i.e. working with city stakeholders on resilience projects) would be fully reflected on, documented and shared. The Board of Trustees of The Rockefeller Foundation adopted this recommendation with a time extension and an additional $17 million. 1.4 ACCCRN summative evaluation This summative evaluation had three primary purposes:6 1) Learning to contribute to the Foundation’s knowledge about the outcomes and impact of ACCCRN in order to inform future work in the area of UCCR and resilience, as well as the Foundation’s work more widely. 2) Accountability to The Rockefeller Foundation President and Board of Trustees and other key stakeholders for the expenditure of funds (approx. $59 million), staff, and other resources in relation to the achievements of the Initiative. 6 The Terms of Reference (TORs) of the current Summative Evaluation, carried out in March 2014, are presented in Annex 1.
- 26. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE4 3) Asapublicgood,contributingknowledgeonurban climate change resilience and on approaches to evaluating climate change and resilience. With these objectives of broadly assessing the successes (or otherwise) of the ACCCRN program, this evaluation has considered the achievements, challenges, and success factors in the different contexts in which the Initiative has been implemented. Thus, the evaluation has: • assessed the relevance of ACCCRN to the evolution of UCCR in Asia and the new work of the Rockefeller Foundation • assessed the efficiency and effectiveness of the Foundation’s leadership and management of the Initiative, including the allocation of Foundation resources, outputs delivered, thought leadership, and building effective partnerships and alliances to implement and sustain the Initiative • assessed the influence and impact achieved, in particular the extent to which critical stakeholders have been motivated and stimulated to change attitudes, behavior, practices, and systems in support of UCCR in ACCCRN cities, and to what extent this can be attributed to the Foundation’s support • considered outcomes and achievements of the Initiative as well as impacts, and assessed the extent to which the achieved outcomes – both intended and unintended – are sustainable beyond the Foundation’s involvement • highlighted the knowledge contributions of ACCCRN to the fields of UCCR and resilience in general, as well as to network development efforts. 1.5 Methodology Evaluation matrix The evaluation objectives and evaluation questions have been combined to create an evaluation matrix as the structural framework for the evaluation (Annex 2). In developing the matrix, some of the questions related to influence and effectiveness have been moved within the matrix to give better coherence around the objectives. Data collection Data were collected through different approaches. A major tool was semi-structured interviews against a checklist of questions, with: • ACCCRN grantees and partners in the capital cities of all four original ACCCRN countries and the initial 10 ACCCRN cities (the evaluation did not sample the cities – all 10 were visited) • partners and stakeholders in: International Council for Local Environmental Initiatives (ICLEI) replication cities, Shimla in India and Sukabumi in Indonesia; Mercy Corps/Association of Indonesian Municipalities (APEKSI) replication and best practicecityofBlitarinIndonesia;andnon-ACCCRN Mekong Building Resilience in ACCCRN Cities to Climate Change (M-BRACE) replication cities of Udon Thani in Thailand and Hue in Vietnam • climate change leaders, policy makers and practitioners in Asia and globally • international and regional ACCCRN grantees, partner organizations and other climate change funders in Asia and globally • staff of The Rockefeller Foundation in Bangkok and New York, including the members of the Executive Team, ACCCRN management and those involved with other relevant initiatives, particularly resilience initiatives. The quantitative and financial data from the ACCCRN grant portfolio were analyzed to examine the granting patterns. A plan was also developed to undertake a separate assessment of all ACCCRN city projects to provide an evidence base for the evaluation, but this did not occur. ACCCRN is somewhat unusual in the development sector for several reasons. Being an initiative rather than a program, it is at a larger scale and a longer duration than most programs, so has generated a very large volume of documentary material. Within this, it has generated a large body of public domain documents
- 27. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 5 about itself, many of them in peer-reviewed journals. Thus, there is already a considerable body of published material on ACCCRN in existence. The Rockefeller Foundation has created a repository of some 2,000 ACCCRN documents, including grant documentation, regional trip reports, work plans, conference reports, financial reporting, budgets, and monitoring reports. These and other peer-reviewed and grey literature documents in the public domain were reviewed. Particular attention was paid to the Verulam monitoring reports, including the annual country-level monitoring reports and the annual monitoring synthesis, as well as the Monitoring The Field reports, as these are ACCCRN’s primary source of objective progress information. 1.6 Report Following this Chapter 1 review of ACCCRN implementation, particularly since the Mid-Term Evaluation, Chapters 2 through 7 present the evaluation findings according to the main objective areas of the TORs: relevance and design, efficiency, effectiveness, influence, impact, and sustainability. Subsequently, the major findings and lessons and recommenda- tions of the evaluation are given in Chapters 8 and 9, respectively.
- 29. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 7 general approach has also spread horizontally to other initiatives involving ACCCRN grantees, but not funded by the Rockefeller Foundation7 (Figure 1). 7 For example, the USAID-funded M-BRACE program that ISET is leading in Vietnam and Thailand. 2ACCCRN Implementation 2.1 Growth of ACCCRN ACCCRN activity commenced in 2008, working in 10 cities across four countries. It has since expanded into additional “replication cities,” with various grantees receiving ACCCRN funding. The 7 For example, the USAID-funded M-BRACE program that ISET is leading in Vietnam and Thailand. INDIA Leh Shimla Singla Barisal Mongla Jorhat Guwahati Makati City Core ACCCRN Cities ACCCRN Cities Mekong-Building Climate Resilant Asian Cites (A program supported by USAID with additional support from the Rockefeller Foundation) Core ACCCRN CitiesShillong Da Nang Lao Cai Hue Chang Rai Bashirhat Gorakhpur Saharsa INDONESIA VIETNAM PHILIPPINESTHAILAND BANGLADESH Hat Yai Bandar Lampung Semarang Udon Thani Phuket Quy Nhon Can Tho Surat Bhubaneswar Panaji Mysore Quezon City Palembang Cirebon Blitar Probolinggo Dumai Indore Chubaneswar FIGURE 1: Map of cities in Asia working with ACCCRN and related initiatives Source: www.acccrn.org.
- 30. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE8 ACCCRN has operated through four distinct phases with some temporal overlap. The basic aspects of these phases are the following. • Phase 1: City scoping. Identification of suitable cities with which to work. Completed in late 2009. • Phase 2: City engagement. Engaging city stakeholders, forming UCCR working groups, undertaking iterative Shared Learning Dialogues (SLDs) with stakeholders, undertaking Vulnerability Assessments (VAs), producing City Resilience Strategies (CRSs).8 Completed in 2011. • Phase 3: Project implementation. Selection and implementation of resilience projects in cities. First funding round announced in March 2011. • Phase 4: Scaling-up, replication and networking. Concurrent with Phase 3, included scaling ACCCRN’s approach and concepts up and out to other cities and national levels, bringing in knowledge and learning partners, leveraging new funds for UCCR. Figure 3 depicts the main steps in Phases 2 and 3. Figure 4 shows a progressive schematic from Phase 2 to Phase 4. This illustrates the emphasis ACCCRN has placed on documenting and promoting the Phase 2 and 8 For further explanation of Phase 2 see the MTE, Rockefeller Foundation 2010, and Moench et al. 2011. 3 approaches, getting from initial engagement with a purposively formed UCCR working group, to production of a CRS, to implementation of resilience-building city projects. It shows the mainly case-study-based research on city projects and the communication of these as evidence of ACCCRN’s successes. 2.2 Grantees and grants ACCCRN has evolved a structured approach to its grantee architecture. Some grantees, notably Arup and ISET, have been associated with ACCCRN through all four phases with different grants delivering a range of different outputs. However, the grant landscape in ACCCRN has three main components. • Country coordinators. ISET in Vietnam, Mercy Corps in Indonesia, Thailand Environment Institute (TEI) in Thailand, with India being split between TARU Leading Edge Pvt. Ltd. which also coordinates the two cities Indore and Surat, and the Gorakhpur Environmental Action Group (GEAG) which coordinates Gorakhpur. Additionally, TARU and GEAG in India, and NISTPASS in Vietnam were early subgrantees. • International/regional partners with specific roles. APCO Worldwide Inc. (APCO) (communications); Arup (various roles, including FIGURE 2: Phasing in ACCCRN Phase 1 201020092008 2011 2012 2013 2014 Phase 2 Phase 3 Phase 4 Source: Verulam Associates, 2014.
- 31. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 9 management support to ACCCRN, technical support to some country coordinators, and networking and knowledge management); Verulam Associates (M&E); and ISET (technical support and documentation). • Phase 4 grantees. grantees focused on replication, networking and scaling-up, including: International Council for Local Environmental Initiatives (ICLEI) – replication cities; International Institute for Environment and Development (IIED) – research and publication; International Centre for Climate Change and Development (ICCCAD) – training; and Intellicap – private sector. To date, ACCCRN has commissioned $59.3 million in 107 separate grants to 29 grantees.9 Table 3 below lists the 29 grantees, the number and value of their grants, and whether they were for regional or country 9 A full list of grants is given in Annex 3. purposes, or re-granting for city projects – of which there were 38 grants, valued at $15.99 million. An analysis of spending by outcome10 reveals that the greatest proportion of granting and grant value was allocated to Outcome 3 – which received 69 percent of grants related at least in part to Outcome 3, and 41.6 percent of grant value (Table 1). Granting has also been analyzed by city (Table 2). It is evident that cities in Vietnam and Indonesia have received more grants numerically, and a higher value of grants, than the cities in India and Thailand. The same applies to the countries as a whole (Figure 5). 10 All grants with a start year of or after 2010 are included in the analysis. Prior to 2010, grants were not parsed by outcome. Of the 89 post-2010 grants, 15 were excluded from the analysis due to of lack of information on outcome specificity. Thus, the analysis included 74 post-2010 grants, with a total value of $42.1 million. FIGURE 3: Process steps in ACCCRN Phases 2 and 3 FIGURE 4: Process steps in ACCCRN Phases 2 to 4 Form UCCR working group Form UCCR working group Vulnerability assessment Vulnerability assessment Shared learning dialogue Shared learning dialogue City resilience strategy City resilience strategy ACCCRN city projects ACCCRN city projects Process reflection: • Documentation • Communication • Networking Research: • Documentation • Communication Adaptation Replication Evidence for leverage Source: Verulam Associates, 2014. Source: Verulam Associates, 2014.
- 32. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE10 TABLE 1: Breakdown of grants by outcome Outcome Number of grants by Outcome % of grants by Outcome* Total grant amount ($) allocated per Outcome % of the total value ($) by Outcome Outcome 1 44 59 $12,999,024 $30.88 Outcome 2 44 59 $7,447,212 $17.69 Outcome 3 51 69 $17,494,325 $41.56 Outcome 4 1 1 $329,992** $0.78 Source: Verulam Associates, 2014. * The sum is greater than 100% as most grants contributed to more than one outcome ** Only one post-2010 grant is allocated to Outcome 4: Arup grant 2010 CLI 310, for which 20% ($329,992) of the total grant amount ($1,649,960) was earmarked for Outcome 4. TABLE 2: Breakdown of grants by country and city Country Indonesia India Thailand Vietnam City Bandar Lampung Semarang Gorakhpur Indore Surat Chiang Rai Hat Yai Can Tho Da Nang Quy Nhon Total grant value ($) $1,252,369 $1,904,340 $782,016 $1,302,833 $813,792 $650,000 $597,950 $1,649,934 $1,849,236 $1,661,232 Total number of grants* 4 6 2 3.5 1.5 2 2 3.66 4.66 3.66 Source: Verulam Associates, 2014. * Some grants were split between two or more cities FIGURE 5: Breakdown of grants by country Indonesia India Thailand Vietnam Vietnam Thailand india Indonesia $3,156,709 $2,898,641$1,247,950 $5,160,402 Source: Verulam Associates, 2014.
- 33. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 11 TABLE3:BreakdownofallACCCRNgrants,bygranteeandlevel ORGANIZATIONALLGRANTSREGIONALANDCOUNTRYGRANTS(R&C)CITYGRANTS No.Amount($) %ofNo.%of$value No.Amount($) %ofNo.%of$value No.Amount($) %ofNo. %of$ value ofallgrantsofgrantee’sgrantsofgrantee’sgrants APCOWorldwide Limited 2$1,450,0001.92.42$1,450,000100.0100.00-0.00.0 AshokaThailand1$200,0000.90.31$200,000100.0100.00-0.00.0 AsianDevelopment Bank(ADB) 1$5,000,0000.98.41$5,000,000100.0100.00-0.00.0 AsianDisaster PreparednessCentre (ADPC) 3$174,6552.80.33$174,655100.0100.00-0.00.0 Centrefor CommunityHealth andDevelopment 1$300,0000.90.51$300,000100.0100.00-0.00.0 ChallengetoChange2$401,9871.90.70-0.00.02$401,987100.0100.0 CitiesDevelopment InstituteofAsia 1$74,2330.90.11$74,233100.0100.00-0.00.0 Foundation- administered 1$54,0000.90.11$54,000100.0100.00-0.00.0 Gorakhpur Environmental ActionGroup (GEAG) 4$1,877,5863.73.24$616,42025.032.83$1,261,16675.067.2 IndianCouncil forResearchon International EconomicRelations (ICRIER) 2$550,0001.90.92$550,000100.0100.00-0.00.0 InstituteforSocial andEnvironmental Transition(ISET) 20$17,284,13118.729.17$10,464,29435.060.513$6,819,83765.039.5 Institutefor DevelopmentStudies (IDS) 1$134,8420.90.21$134,842100.0100.00-0.00.0
- 34. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE12 ORGANIZATIONALLGRANTSREGIONALANDCOUNTRYGRANTS(R&C)CITYGRANTS IntegratedResearch andActionfor Development(IRADe) 2$220,3001.90.42$220,300100.0100.00-0.00.0 Intellicap3$862,0752.81.53$862,075100.0100.00-0.00.0 IntellectualCapital AdvisoryServices PrivateLimited 1$29,5000.90.01$29,500100.0100.00-0.00.0 International CouncilforLocal Environmental Initiatives(ICLEI) 6$2,936,0505.64.96$2,936,050100.0100.00-0.00.0 International FederationofRed CrossandRed CrescentSocieties (IFRC) 1$1,000,0000.91.71$1,000,000100.0100.00-0.00.0 InternationalInstitute forEnvironmentand Development(IIED)* 6$4,251,1605.67.26$4,251,160100.0100.00-0.00.0 MercyCorps16$7,866,82115.013.37$4,460,11243.856.79$3,406,70956.343.3 NationalInstituteof UrbanAffairs(NIUA) 1$320,7000.90.51$320,700100.0100.00-0.00.0 OveArup&Partners InternationalLimited 6$4,586,2605.67.76$4,586,260100.0100.00-0.00.0 RosieSjögren2$176,0161.90.32$176,016100.0100.00-0.00.0 Societyfor ParticipatoryResearch inAsia(PRIA) 1$300,0000.90.51$300,000100.0100.00-0.00.0 TARULeadingEdge PrivateLimited 8$4,078,4067.56.92$1,300,42033.331.96$2,777,98675.068.1 ThailandEnvironment Institute(TEI) 9$3,074,9108.45.24$1,756,96044.457.15$1,317,95055.642.9
- 35. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 13 ORGANIZATIONALLGRANTSREGIONALANDCOUNTRYGRANTS(R&C)CITYGRANTS TheEnergyand ResourcesInstitute (TERI) 2$335,4001.90.62$335,400100.0100.00-0.00.0 UnitedNations EconomicandSocial Commissionfor AsiaandthePacific (ESCAP) 1$75,0000.90.11$75,000100.0100.00-0.00.0 VerulamAssociates2$1,550,0001.92.62$1,550,000100.0100.00-0.00.0 VietnamInstituteof Architecture,Urban andRuralPlanning (VIAP) 1$158,9460.90.31$158,946100.0100.00-0.00.0 Totals:107$59,322,978100.00100.0069$43,337,34364.573.138$15,985,63535.5N/A Averages:3.69$2,045,619 Source:VerulamAssociates,2014. * Forcontractingreasons,thegranttoICCCADinBangladeshhasbeenchannelledthroughIIED.Approximately$2millionoftheamountindicatedherehasbeenforIIED’spublicationsgranteerole,andtheremainderforICCCAD.
- 37. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 15 3 Relevance and design In reviewing ACCCRN’s relevance, the mid-term evaluation concluded that ACCCRN was a relevant response to urban climate change, i.e. it focuses on rapidly urbanizing second-tier cities, it works through processes that involve subnational government and local stakeholders,11 it links to planning, and it has helped shape and promote the idea of UCCR. The summative evaluation has reached a similar assessment. In general, ACCCRN continues to be relevant. It is a well-conceived idea, which was timely and needed and, in particular, is highly relevant to Asian cities where the twin pressures of urbanization and climate are keenly felt. However, a more nuanced view emerges when unpacking the relevance question at different scales – internationally and in the countries and cities where it has worked – and to various stakeholders. TheTORsalsocalledforanassessmentoftherelevance of ACCCRN in the evolution of UCCR in Asia and the new work of the Foundation, including an assessment of the Foundation’s comparative advantage and value added in the UCCR field. Here, the evaluation addresses those questions and assesses the relevance of the design to the need and current thinking on the challenges of climate change and how ACCCRN relates to them. 11 Most international climate change frameworks mainly address national governments and “do not indicate a clear process by which local governments, stakeholders and actors may participate” (UN-HABITAT, 2011). 3.1 The relevance of ACCCRN Although, in an environmental context, the concept of resilience has been in use since the 1970s, the Foundation was an early entrant into the area of urban climate change and the development of the UCCR field. This raises the question of whether it has used this positioning effectively to shape the field and debates about UCCR. And if so, has it shaped them appropriately? This evaluation considers that the design of the ACCCRN approach to improving cities’ UCCR, which has evolved over the life of the Initiative, has been relevant. The basis for this finding comes from an examination of general approaches to this type of problem. Climate change is widely considered a “wicked” problem,12 meaning a problem that is difficult or impossible to fix, for reasons such as: incomplete or contradictory knowledge, the number of people and opinions involved, the large economic burden, and the interconnected nature of these problems with other problems (Kolko, 2012). Adding the urban dimension to climate change serves to increase the problem’s “wickedness.” Climate change has now been termed a “super-wicked problem,” since it is one of a new class of global environmental problems also characterized by: 12 See Annex 4 and APSC, 2007; Brown et al., 2010, and Davoudi et al., 2009, for a fuller explanation of the concept of a wicked problem.
- 38. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE16 time running out for solutions, a weak or absent central authority, those who cause the problem also seeking to create a solution, and irrational discounting that pushes responses into the future (Levin et al., 2012). The emerging schools of thought about dealing with wicked problems emphasize that social learning offers particular strengths, especially in areas such as climate change, where a plurality of knowledge improves understanding of the issue and ways of addressing it (Collins and Ison, 2009; Hackman et al., 2014). In social learning, it is the process of “co-creation of knowledge, which provides insight into the causes of, and the means required to transform, a situation. Social learning is thus an integral part of the make-up of concerted action” (Ison and Collins, 2008). Climate change is also seen as a complex problem13 (Ramalingam, 2013). Understanding complexity requires experimentation. Experimentation is considered more successful than analysis in identifying the risks or accurately predicting solutions. Thus, solutions tend to be emergent, meaning they require developing approaches, experimenting, evaluating and repeating as necessary, and then amplifying the experiments that are shown to work. Experimentation is also now being seen more widely as an approach and a framing lens for urban climate change.14 Both social learning and complexity thinking apply to the ACCCRN/UCCR context. The ACCCRN design appropriately adopted a social learning approach through its Shared Learning Dialogue (SLD) process. 13 Complexity is increasingly being used as a lens to understand the real world, with a complex situation being one in which the relationship between cause and effect can only be perceived in retrospect, but not in advance (also see Annex 4, and Snowden and Boone, 2007). 14 Anguelovski, et al. 2014; Bulkeley and Castán Broto, 2013 and Castán Broto and Bulkeley, 2013. Bulkeley and Castán Broto state: “We suggest that such interventions might fruitfully be considered in terms of exper- iments, partly in order to signify their potential but more significantly to recognise their often tentative nature, the sense of testing or establish- ing (best) practice that frequently accompanies their development, and the ways in which they are used as a means of supporting or contesting knowledge claims and discursive positions. …Here, we do not use ex- periment in the formal scientific sense of the term but rather to signify purposive interventions in which there is a more or less explicit attempt to innovate, learn or gain experience.” However, while ACCCRN has actively taken different approaches in different contexts, it is less clear that these were specifically designed to be experimental approaches to a complex problem. Scope now exists to frame the various approaches taken by different grantee-city combinations as an experiment, and review the learning that has emerged. An experimentation approach is consistent with thinking about how innovation can be encouraged. Compared with most other funders, foundations have “extraordinary discretion to experiment and try new things” (Kasper and Marcoux, 2014). ACCCRN has been experimental, as it has moved ahead in fits and starts, with a strategy that was emergent. Grantees have been charged with finding new solutions. Emergent processes are more complicated to manage and take more time than linear processes that deliver blueprint strategies. The approach has been the right one, but it has taken time, more time than the Foundation is comfortable with in relation to its current generation of initiatives. 3.2 ACCCRN’s theory of change and design assumptions Theory of Change (TOC). ACCCRN was one of the first initiatives of The Rockefeller Foundation to take a results-basedapproachwithaninitialresultsframework developed early in process. This was reviewed and revised by the ACCCRN team with a related theory of change (TOC) facilitated by Verulam as the M&E granteein2010,andsubsequentlysharedwithgrantees and used as the basis for the regular monitoring (Verulam/Rockefeller ACCCRN Team, 2010). After the 2012 round of monitoring, the results framework was revised, and this revised version has been used subsequently (Verulam/ACCCRN, 2013). Following the MTE, outlines of both the results framework and TOC were included in the Initiative summary given to grantees along with their grant letters (The Rockefeller Foundation, n.d.).
- 39. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 17 The TORs asked how, and in what ways, the TOC and assumptions of ACCCRN have informed the Initiative and reflected its learning. ACCCRN’s TOC for achieving impact is centered on three outcomes: • improved capacity within cities as the basis for implementation of plans and projects and for climate resilience • improved information flows among ACCCRN partners and between ACCCRN partners and external stakeholders (other cities, climate experts, donors, national governments, etc.) • the combination of robust plans/projects within cities and good communication to outside stakeholders to leverage additional funding and technical support for climate resilience work within existing cities, as well as for new cities in Asia. As a normative model, the theory of building capacity through learning-by-doing, sharing these experiences among the parties involved and others, and using these to leverage expansion makes sense. These have also been the three main outcome areas of the programme, so the TOC has strongly informed the practice and implementation. The evaluation found weaknesses in the TOC with respect to the formation of a strong “community” among ACCCRN members and the level of effort required to leverage uptake at national level. The translation of any TOC into practice is contingent on the extent to which the assumptions underlying the theory hold. The ACCCRN assumptions have held to a varied extent (Table 4). TABLE 4: Theory of change assumptions Choices of strategy and approaches Achievement of Outcomes Achievement of Impact • Working with city governments is a necessary but not sufficient condition to address UCCR needs and is necessary to ensure long- term financial and institutional action to build UCCR. • Examples and knowledge of what cities are doing are sufficient to increase the prioritization of UCCR issues within their planning/budget frameworks. • An iterative learning process improves the set of interventions to build resilience and achieve local ownership. • A multi-stakeholder process leads to co-generation of local knowledge. • We will need to work in a multiplicity of environments to generate models and learning. • Lessons learned are transferable across cities. • Pools of money to support replication from donor agencies will be available within the next 1–2 years, and we will be able to tap into this. • Resilience improvements are measurable and credible to other city governments and subject matter experts. • City governments will continue to push this agenda after direct Rockefeller Foundation support is withdrawn. • Those networking functions that prove of value will be financially supported beyond the current 3-year funding window, either from The Rockefeller Foundation, governments, or donors. • The Rockefeller Foundation has adequate staff to take on what may be an increasing workload in the coming 2–3 years. • Supporting development of practical models will do more to enhance resilience than allocating a similar sum of funds for research and analytics. • This is a young field, and the most effective strategy to achieve impact is through direct experimentation in resilience building. • Multi-stakeholder processes to develop local resilience plans and interventions will lead to greater local ownership, and thus more successful and sustainable resilience interventions • By working at the city level, you have more impact on the poor and vulnerable communities than by working exclusively with poor and vulnerable • Models of UCCR can be created, implemented and documented within the program’s time frame and within the budget. Source: Verulam Associates/ACCCRN, 2013.
- 40. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE18 Strategy and approach assumptions. The evaluation concurswithallthestrategyandapproachassumptions, except: “Examples and knowledge of what cities are doing are sufficient to increase the prioritization of UCCR issues within their planning/budget frameworks.” We find that examples and knowledge are insufficient to drive planning and budgeting prioritization. In this area, ACCCRN has demonstrated a weak orientation towards the realities of national, subnational, and city governance. The political realities of decision-making and prioritization at these levels need more than pilots and evidence. While examples have been influential, they are not sufficient. Outcome assumptions. In relation to outcome assumptions, it has proven harder than assumed to leverage other donor funds for replicating the ACCCRN approach to UCCR and the assumption about “tapping into” has in part proven unfounded. The UCCR Partnership and Trust Fund with the Asian Development Bank (ADB) and Department for International Development (DFID) has proved slow and difficult to operationalize. While initiatives by other donors have drawn on ACCCRN, there has been little complementary direct financing at city level. The assumption that “Resilience improvements are measurable and credible to other city governments and subject matter experts” has not fully held. This is principally because ACCCRN has not developed an agreed approach to measuring resilience. Thus evidence for increasing resilience in cities is mainly anecdotal or at best partial. These stories of resilience (with some exceptions) have mainly been at the project level, but have nonetheless still been credible to other cities, as evidenced from the investment in expansion cites, which have drawn on ACCCRN examples. The assumption that: “Those networking functions that prove of value will be financially supported beyond the current three year funding window, either from RF, governments, or donors” also has not held. The networking functions have yet to demonstrate sufficient value that others wish to fund them. A final attempt at networking is underway, but with a four-year ACCCRN grant and a view from the Foundation that it is more important that this stimulates networking than focuses on creating a sustainable network. Impactassumptions.Inrelationtoimpactassumptions, it was assumed that: “Supporting development of practical models will do more to enhance resilience than allocating a similar sum of funds for research and analytic.” This has partially held. ACCCRN is widely credited with having developed ways to operationalize an approach to improve UCCR. However, there are some areas which could have been usefully developed alongside the practical models including UCCR indicatorsandmeasurementframeworks.Asisreflected in ACCCRN itself: “Climate change adaptation in cities requires strengthening urban governance, addressing the underlying drivers of vulnerability, and building on past approaches to develop resilience. Research helps with all of these” (IIED, 2014). Furthermore, since ACCCRN has not been fully run as an experiment, there is no “control city” to inform the argument that impact could have been increased by investing in research and analytics as opposed to projects on the ground. The evaluation concurs with the assumption that: “This is a young field, and the most effective strategy to achieve impact is through direct experimenta- tion in resilience building.” However, one of the conclusions of the evaluation is that ACCCRN has not been managed as an experiment. Grants and city projects have been commissioned in such a way as to promote experimentation, but this is not the same as an experiment, since the whole-of-program analysis, which would close the experimental loop and draw cross-sectional conclusions has not been conducted. This also relates to the final impact assumption: “Models of UCCR can be created and implemented ACCCRN has demonstrated a weak orientation towards the realities of national, subnational, and city governance.
- 41. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 19 and documented within the program’s time frame and with the budget.” Models can be, and have been, implemented and documented, but individually. A comparative analysis is largely missing. However for the avoidance of doubt, it needs to be stated that while models have been implemented, this is not the same as cities having become resilient within the projects’ timeframe and budget. The assumption that “by working at the city level you can have more impact on the poor and vulnerable communities than by working exclusively with poor and vulnerable” is neither proven nor disproven. The literature contends that resilience does not automatically equate with poverty reduction (Béné et al., 2014). Indeed, if the distributional and social justice aspects of resilience are not considered, the reverse may hold, but there is not enough empirical evidence from ACCCRN either way in regard to this assumption. Some targeted city projects such as typhoon-resistant housing in Da Nang and ward-level interventions in Gorakhpur have benefitted the poor, but our assessment is that these are projects targeted at the poor and vulnerable, not “working at the city level,” though their selection did consider scalability as a criterion. In their selection and identification, most city-level projects did consider benefits to the poor and vulnerable as a “gateway criterion” (i.e. must-have), although the mechanism was not always a direct one. Finally, the core of this TOC is that impact (building UCCR) will be achieved through building capacity in cities and sharing knowledge on UCCR. The linear model fundamentally overlooks the governance and politics dimensions, both upwards towards the national tableau and downwards towards engagement with citizens. It is discussed further below. 3.3 Consideration of governance factors In examining the relevance of ACCCRN and considering governance factors, the various national contexts are worth touching. • India. In India, the 74th Constitutional Amendment aims at decentralized governance, and devolves urban governance to the state level, providing states with authority to selectively devolve powers to urban local bodies. While an increasing number of states have devolved powers to the city level, states such as Uttar Pradesh, in which Gorakhpur is located, have not. Most Indian cities are highly dependent upon states for financing, with very limited direct taxation and revenue sources available to cities. Requisite development plans made by cities are consolidated into district plans at state level to take into account issues of adjoining rural areas. Unless national and state policies make provision for specific local services and processes, it is challenging for those cities to allocate any significant funding to, for example, UCCR. • Indonesia. After years of highly centralized government, Indonesia launched a major initiative for local self-government in 2001, with substantial devolution of policy and budgets. City government has gradually developed in the intervening years, and further powers have been devolved. Planning, development and the related issue of climate mitigation and adaptation are among those policy areas where guidance from the center and implementation at city level are closely linked. This high level autonomy for local government focuses attention on cities themselves and less directly on local government associations which are organizationally weak, e.g. the Association of Indonesian Municipalities (APEKSI). However, such associations are key to city-level engagement. The role of the Government of Indonesia (GoI) with respect to climate change, as with other policies, is largely to provide guidance and advice, though there continues to be significant project investment from the center. The three relevant GoI bodies in this respect are the Ministry of Environment (MoE) which is responsible
- 42. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE20 for formulating policies and coordination in the field of environment and control of environmental impacts, the Ministry of Public Works (MoPW) which is in charge of all GoI public works, and the National Development Planning Agency which is responsible for national planning and for advising on planning at subnational level. Climate change became a major feature of public policy in Indonesia when it hosted the 13th session of the Conference of the Parties (CoP 13) to the United Nations Framework Convention on Climate Change (UNFCCC) in 2007. The focus of the CoP on deforestation was particularly pertinent for Indonesia and raised the profile of climate change moregenerally.TheNationalActionPlanonClimate Change (RAN-PI) was launched that year and the following year the President established the Dewan Nasional Perubahan Iklim (DNPI) – National Council on Climate Change – within the President’s secretariat as a high-level focal point to coordinate the implementation of the climate change action plan and to strengthen Indonesia’s position in international forums on climate change control. • Thailand. Thailand’s institutional authority has a historical legacy of fragmentation through multiple agencies with autonomous, overlapping, or conflictual mandates. Tensions and contradictions, often highly politicized, also exist between central and local governing bodies. The structuring and financing of local government provides on-going challenges for mainstreaming and integration of UCCR concepts and practices. • Vietnam has strong top-down/centrally-led policy decision-making and regulation. All laws and almost all policies related to urban development or climate change come from central government level. Other than its five largest cities, which have a relatively high level of management autonomy, cities have limited power or autonomy in decision-making, particularly those related to city development strategy and policies. • The government of Vietnam views climate change as a very high priority in its policy agenda and has put a comprehensive policy response in place. The current five-year Social-Economic Development Plan (2011–2015) recognizes climate change as a significant threat to development and reaffirms commitments on climate change response (CCR) policies. The key strategies/action plans include: the 2007 National Strategy for Natural Disaster Prevention, Response and Mitigation; the 2011 NationalClimateChangeStrategy;the2012National ActionPlantoRespondtoClimateChange;the2013 Vietnam Green Growth Strategy; the 2013 Party’s Resolution No.24-NQ/TW on “active in response to climate change, improvement of natural resource management and environmental protection;” and the 2014 National Action Plan on Green Growth. Action plans on CCR are formulated at all levels (government, line ministries and provinces/cities). The National Climate Change Committee (NCCC) chaired by the Prime Minister was established in 2012 and a Standing Office for NCCC was set in the Ministry of Natural Resources and Environment (MONRE) to support the NCCC works relating to climate change policy response. This sets out a set of very different contexts with varied policy and fiscal restraints on cities in respect of climate change. Now, some six years into ACCCRN, it is evident that the type of UCCR approach has differed in the countries in relation to the context, including some aspects of governance. For example, in Thailand, flexibility of the ACCCRN process has enabled the Thailand Environmental Institute (TEI) to try a number of approaches in working with the cities and to engage broader provincial and national decision-makers. The challenging political context in Thailand limited results from the initial approach that TEI took. Therefore, TEI adopted a strategy of longer-term awareness and knowledge building to increase appreciation of the relevance of UCCR perspectives within the politically strained context. This has resulted in a learning approach that
- 43. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 21 has featured a range of workshops and exchanges between and within cities and with mixtures of stakeholders, including municipality officials (including mayors), government agencies and departments, NGOs, academics and researchers, businesses and the private sector. Governance is directly relevant to UCCR. A study conducted for ACCCRN identifies five ways in which good governance can affect cities’ ability to build resilience (Tanner et al., 2009). These are: • decentralization and autonomy – cities with some degree of local-level decision-making authority combined with good working relationships with national and state or provincial governments are able to implement policies and programmes more effectively and efficiently than cities where deci- sion-making is centralized with higher levels of government • accountability and transparency – a transparent, open planning system not only engages important stakeholders, it also educates them about the trade-offs that will be part of any climate planning process • responsiveness and flexibility – cities require flexible agencies and management systems suited to responding to and anticipating what may result from climate change • participation and inclusion – as climate change tends to disproportionately affect the poorest and most vulnerable groups first and most severely, engaging these groups in planning and decision- making is critical for building climate resilience • experience and support – cities experienced in developing integrated, people-centered early warning systems for extreme events are well placed to make progress toward climate change resilience, and also benefit from the experience of local, national and international civil society organizations and research organizations. As identified in the MTE, some aspects of these were better addressed in ACCCRN’s conceptualization than others. In general, governance factors were not well considered in the selection of the original 10 cities. Furthermore, there was not sufficient attention paid to the national-level governance and policy context at that time. National policy was not prioritized. The ACCCRN theory of change emphasized building a body of credible practice from the demonstration of projects in cities as a driver for UCCR. It was assumed that this evidence would be sufficient for decision makers to take notice and then act. This is a linear, techno-rational model that does not take account of the wider governance context and the complexity of planning processes. There is a contrast between the “swamp of real life issues,” in which social learning approaches can help, and a “high ground of techno- rational approaches” (Schön, 1983). This contrast is still relevant to addressing urban climate change. Although not stated in the TOC, it would appear that an implicit and, ultimately, unreliable assumption at the heart of ACCCRN was that national policy and urban governance are rational processes informed by and responsive to emerging scientific and other academic evidence. As evident from the previous bullet points, cities vary widely in the way they are run. Governance differs according to the extent of political power decentralized to the city, balance of political power within the city (strong mayors, executive teams and large councils making all decisions), role and level of independence of the senior administrative team, levels of fiscal decen- tralization and financial autonomy, and the models and approaches to planning and service delivery. South Asian countries have centrally accountable senior bureaucrats who wield more control over planning and finance than the elected city politicians. Southeast Asian countries, generally, have strong mayor models with various degrees of decentralization … political economy analysis was not routinely applied in developing the initiative, and this was a deficit.
- 44. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE22 from national government in practice. Vietnamese cities often have little decentralized power, which is mediated by strong community-level political party structures. However larger, economically strong cities do have more de facto room for maneuver. Therefore, to be effective, a sound appreciation of context and drivers is needed. Key among these is an understanding of the particular political economy15 of any given city – the formal and informal powers and institutions that define how cities really tick. Overall, political economy analysis was lacking in ACCCRN. Over time, the approaches, such as the Climate Change Coordination Offices (CCCOs) in Vietnam, have adapted to the political economy context, but political economy analysis was not routinely applied in developing the Initiative, and this was a deficit. The ACCCRN process has been very important in allowing stakeholders to feel their way through uncharted territories – and a good amount of time has been provided for this to unfold and enable wider ownership and adoption. However, it is now evident that the limited or lack of early engagement by key ACCCRN actors with strategically important senior people within city government to get their buy-in to the process of developing city resilience strategies was an error. As a consequence, the initial process of developing CRSs has concluded without cities officially adopting or incorporating the strategies into their operations and budget processes. However, many cities have now dedicated some budget to climate change areas. ACCCRN has been variable in understanding and responding to the important political drivers of city decision-making. Some of this variability related to the orientation and abilities of country partner staff working in cities, but ACCCRN has not explicitly 15 Political economy analysis aims to situate development interventions within an understanding of the prevailing political and economic pro- cesses in society – specifically, the incentives, relationships, distribution, and contestation of power between different groups and individuals, all of which greatly impact on development outcomes. Such an analysis can support more effective and politically feasible donor strategies, as well as more realistic expectations of what can be achieved, over what timescales, and the risks involved. See McLoughlin, 2009. “done governance.” There has been an underlying expectation that city governments would change how they did things, based upon the emergence of rational, technical information. While ways of working with cities, and increasingly on national stages in Indonesia and Vietnam, have been developed, there remain sustainability questions relating to the levels of embeddedness in formal governance processes. The Kennedy School of Government has developed a critique of why government reform fails (Andrews et al., 2012), and UCCR is, in part, reform of the way city government works to address climate change (the assessment above partly hinges on ACCCRN using a reform lens). The key is that organizations pretend to reform by changing what policies and organizational structures look like, as a camouflage in the absence of real change in the way they operate. The danger is that “capabilitytraps”persistwheremimicry(ofstructureand form) is rewarded over function. To overcome this, they have developed an approach called problem-driven iterative adaptation (PDIA). This is based on principles including “allowing the local selection and articulation of concrete problems to be solved” – not being driven by external experts, encouraging experimentation and positive deviance, “promoting active experiential (and experimental) learning with evidence-driven feedback built into regular management and project decision making,” and encouraging scaling by diffusion rather than top-down. Interestingly, it would appear that the ACCCRN design is largely in line with this thinking, and thus seen as a relevant design: • the UCCR working groups are not uniform; they have been created to function in the local context • the SLDs helped articulate local problems • experimentation has been encouraged.16 The key, therefore, is the extent to which the SLD-CRS processes have been mainstreamed into government. 16 Whether experimentation is a feature of ACCCRN implementation is discussed in Chapter 4: Effectiveness. Where there could be increased attention is the use of more evidence-based feedback loops to drive decision-making and further cycles of experimentation.
- 45. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 23 This varies across cities and countries. The SLD-CRS processcanbeseenintwoparts:SLDsrelatetoawareness, coalitions, and the “softer” conditions to pursue UCCR, while CRSs relate to the formal codification of these efforts. Evidence from across the ten cities indicates that some have become aware of, but not adopted, the formal documents, others the reverse, and some – both. Vietnam’s socio-economic, institutional and governance factors have been better taken into account, as seen in contextually relevant institutional architecture and alignment with (and shaping of) national policy. The models of CCCOs in the three Vietnamese ACCCRN cities have been established differently, depending on their institutional and political context, e.g. the selection of ACCCRN local partners: City People’s Committee in Can Tho, the City Office in Da Nang, and the provincial Department of Natural Resources and Environment (DONRE) in Binh Dinh/Quy Nonh. Of these, DONRE is considered the most sustainable one, as it sits directly under the People’s Committee authority. In Indonesia, Mercy Corps’ understanding of the institutional context helped gain traction, which has beenstrengthenedthroughitsrelationshipwithAPEKSI. There is also evidence that wider issues of urban policy and planning – including relevant stakeholders – were used as an entry point for Semarang’s and Bandar Lampung’s CRSs. In India, the desire for policy change was there, but there was no strategy or activity towards this end. The evaluation heard that ACCCRN could have done more if it had undertaken robust political economy analyses in the cities. InThailand,currentgovernancesystemsandstructures do not lend themselves to incorporating the complex multi-dimensional conceptions of UCCR. However, TEI has evolved with its city partners’ strategic approaches that engage with socio-economic and governance matters associated with promoting climate change resilient cities. TEI has adopted a more overt resilient city engagement approach (as opposed to a primary climate change entry point) with the M-BRACE cities of Udon Thani and Phuket. 3.4 Consideration of socio-economic factors As noted, resilience does not automatically equate with poverty reduction (Béné et al., 2014) and, indeed, if the distributional and social justice aspects of resilience are not considered, the reverse may hold (Slater, 2014). Care needs to be taken that the notion of resilience as an immutable natural characteristic does not subsume other critical social objectives in addressing climate change (Friend and Moench, 2013). Thus, “resilience isn’t just about hard infrastructure and building codes— it has a strong social and community component as well” (Rodin, 2014a). Discussions with NGOs in India highlighted a growing trend in which city authorities are using regulations for slum clearance and shifting poor people outside of city boundaries, relinquishing responsibility for upgrading. Research by TERI on ACCCRN processes (Sharma et al., 2013; 2014) noted that the relevant authorities in Indore, Guwahati, Bhubaneswar, and Shimla considered theinvolvementofurbanpoorandcommunitygroupsin the ACCCRN processes to be impractical, as the issues under consideration were too technical. Engagement with community representatives was limited “to the initial stages when seeking inputs on identifying the primary risks and to an extent during implementation of the pilot projects in the core cities” (Sharma, 2014). Within socio-economics, the specific area of gender equality should be addressed. The evidence is that women are particularly vulnerable to the risks associated with urban poverty (ADB, 2013), and it is commonly these more vulnerable sectors of society that are most adversely affected by climate change. This is an important area for the Foundation, which states on its website: “We believe gender equality and women’s leadership are key to realizing our dual visions Nonetheless, ACCCRN appears largely “gender-blind.”
- 46. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE24 of achieving more equitable growth and strengthening resilience against the shocks and disruptions of our world.” Nonetheless, ACCCRN appears largely “gender- blind”. With few exceptions (e.g. women’s housing, Vietnam), citizens have not been viewed through a gender lens and, thus, little has been done to address women’s and girls’ particular vulnerabilities, to mobilize them for the UCCR response, or to use building UCCR as a socially transformative response. This situation is disappointing, but not unusual. Women are not normally perceived as part of the solution. Climate policies frequently treat women only as “vulnerable beneficiaries rather than as rights-holding citizens who need to be recognised for the agency, skills and experience they can contribute” (Skinner, 2011). They misstheopportunityofusingclimatechangeresponses to improve social justice, and playing transformative roles by challenging existing gender disparities.
- 47. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 25 4 Efficiency This section considers how well the various resources available to ACCCRN were utilized in pursuit of its objectives and the related TOR questions. 4.1 Use of Foundation resources The MTE found that The Rockefeller Foundation was fairly hands-on in the way it managed ACCCRN, that the valueofgrantsmanagedperstaffmemberwasinlinewith other initiatives, and that climate change grantees were positive about the Foundation as a grantor, although they felt the burden of administration could be lighter. The Rockefeller Foundation ACCCRN team has continued to be fairly hands-on into Phases 3 and 4, and indeed staff numbers have been maintained as the Initiative has entered consolidation. An involved staff is considered appropriate from several perspectives. Kasper and Marcoux (2014) cite an example of a Foundation that takes an especially hands-on approach in helping shape and guide early-stage ideas as they move from concept to implementation, with much interaction and feedback from stakeholders. They note, “this type of assistance often requires extra staff time and specialized expertise on the part of the funder, but the foundation recognizes that early ideas are often malleable, and that targeted interventions that better connect services with potential users can dramatically increase the chances of success.” In a field-forming/field-shaping initiative such as ACCCRN, in which granting is more like experi- mentation, then it is right that the grantors have close sight of, and good interaction, with the evolving grants. The other aspect of resource use in experimental initiatives is the amount of time and money that is appropriate for the search and development phases. To some extent, being Bangkok-based has allowed ACCCRN some leeway, enabling it to learn how to build the UCCR plane while also flying it slowly. More compressed search and development phases, moving rapidly into may have led to some less tailored and, therefore less accepted, approaches. In deciding timeframes for its initiatives, the Foundation may wish to consider using some type of scale of complexity and novelty. Phase 3 saw The Rockefeller Foundation ACCCRN team’s close involvement with selection of city projects. … being Bangkok-based has allowed ACCCRN some leeway, enabling it to learn how to build the UCCR plane while also flying it slowly.
- 48. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE26 There is some question as to whether the grant application and feedback process could have involved less Foundation staff time. However, as ACCCRN moves into Phase 4, which is concerned with scaling-up and scaling-out, it can capitalize on the Foundation’s convening power – and can do so more effectively if the Foundation is credible and convincing on UCCR. The proximity that the team has had with UCCR in practice means this is the case. If the Foundation wishes to have this level of agency in up-scaling its initiatives, then a hands-on approach is no bad thing. Value for money The pressure for efficient use of funds is less for ACCCRN grantees than it is for those in bilateral, and increasingly multilateral, aid programmes. The Value for Money (VFM) agenda, with its focus on “the 3Es” – economy, efficiency, and effectiveness – and an underlying desire to put costs against monetized benefits, has driven sharp attention to costs in many part of the development world (World Bank Institute, 2013; DFID, 2011; Australian Government, n.d.). The Rockefeller Foundation is in general less concerned about such metrics, such as economy – daily cost of grantee staff time, or efficiency – cost per CRS produced. The Foundation is correctly concerned about effectiveness and achievement of grant objectives and ACCCRN outcomes, though not in VFM terms as cost-effectiveness – the cost per outcome achieved. Nonetheless, there would be merits of some comparative analysis of good use of funds and economies of scale. 4.2 Grants and grantees Grant phasing Roughly three phases of granting are apparent in ACCCRN. • Initial granting. This was mainly to international organizations that injected ideas and got the Initiative moving. Some of the earliest granting was task oriented and thus characterized by short, specific grants. • The main Phase 2/3 granting. A at this stage, the structuralarrangementofcountrycoordinatorswas established, supported by the main international grantees who provided management and technical assistance inputs. Specialist grantees such APCO (communications) and Verulam Associates (M&E) also began their work. • Gap-filling and scaling and replication granting. Some of these grants filled gaps, particularly at national policy levels, such as TERI in India, and IIED as an international publications grantee. Others were designated to support specific Outcome 3 objectives, such as ICLEI for replication cities and International Centre for Climate Change and Development (ICCCAD) for training. This phasing, roughly follows a continuum of: country coordinators ➝ city grants ➝ policy sphere ➝ documentation ➝ training for scaling. The switch early on from task-oriented granting for specific results to longer-term grants with broader objectives, including institution building in the grantee organizations and partners, was a distinctive change of strategy. Longer-term grants allow building of relationships both between the Foundation and grantee, and between the grantee and partners in the city or country. They also enable grantees to build their own expertise in a developing field. If these longer-term relationships can be managed to ensure that the Initiative operates as a collective learning laboratory, this can be very productive. However, being locked into a few long-term relationships means that ACCCRN has risked creating monopoly suppliers of UCCR soft services in some markets. We did hear views that ACCCRN has built technical capabilities in international organizations at the expense of local ones. During the initial phase of granting, the Foundation faced the … being locked into a few long-term relationships means that ACCCRN has risked creating monopoly suppliers of UCCR soft services …
- 49. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 27 problem of finding grantees to work on UCCR while the field was largely unformed. It found, among others, ISET, with expertise in socio-ecological and natural resources systems, and Arup, with city expertise. TEI, TARI, Arup, GEAG and Mercy Corps were then brought on board. Criticisms that the evaluation came across in relation to early grantee selection were twofold: • there was some shortage of grantee staff with core urban expertise • the Foundation had sought the wrong skill sets. Initially, coordination, convening power, communications and an understanding of the local political economy proved more important than technical capacity. Having a techno-rational bias led to an emphasis on resilience and city expertise, whereas there may have been local agencies in some countries that could have led strong foundation building. Some grantee organizations were also found to have weaknesses in generic project management skills and utility with project management tools. This was particularly the case where prime grantees were re-granting, but not considered to be adding enough value in this process, for example by building project management capacity. These skills were assumed, and technical assistance to support their development would have been useful. For some grantees, it was not until Verulam started its monitoring based on indicators within the results framework that these deficiencies became evident. Some second-tier grantees also said that they felt that the prime grantees acted to some extent as gatekeepers to ACCCRN. However, Arup did recognize and include the issue of project management capacity in its scope of work for the Phase 3 grant. It worked with all country partners through a train-the- trainer approach to build skills throughout the project cycle. Grantee management The MTE characterized the management model as “hub-and-spoke.” The Strategy and Alignment Group (SAG) was formed to improve interactions among grantees. Overall, ACCCRN has used the language of partnership and tried to manage its grantee relationships in partnership mode. In some cases, grantee relationships have been more contractual – service provision – but in most cases, there has been a productive tension between the freedom a grant provides and the necessity to deliver some immutable aspects of the funder’s vision. As discussed in the section on relevance, an approach characterized by experimentation is very appropriate to the wicked problem for urban climate change, and it is becoming used as an approach to appreciating responses to urban climate change. This is akin to seeking innovative solutions, which organizations pursue when facing complex problems – a process the Foundation’s senior management has written about in terms of identifying common challenges (Khan and Joseph, 2013). One of the top three challenges is collaboration, because the innovation process usually involves multiple players whose “different experiences and approaches, multiple commitments to different groups affected by the problem … and … diverse resources, disparate views, and separate goals” can be integrated to produce an innovative solution greater than the sum of its parts. The Foundation recognizes that collaboration can be derailed by various individual, disciplinary, and organizational concerns. Khan and Joseph (2013) also discuss “innovation dissonance,” or the productive tensions of people working through their differences and finding common ground in the innovation process. Both working together and some skirmishing over conceptual territory and intellectual property have been evident between grantees in ACCCRN. ACCCRN has awarded grants to diverse organizations to develop and test innovative UCCR approaches and solutions. However, the extent to which collaboration has All ACCCRN grants were to single grantees, with little incentive to work together.
- 50. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE28 been leveraged is questionable. All ACCCRN grants were to single grantees, with little incentive to work together. This might be characterized as multiple parallel tracks, linked by information sharing – collegiality rather than collaboration. The SAG has revealed that “learning loops” are restricted to the country level, i.e. little ACCCRN-wide or cross learning is occurring. For example, in Indonesia, the ICA forums, convened by Mercy Corps, provide an opportunity for ICLEI and Mercy Corps to meet and discuss priorities. However, the absence of a strategy for joint working limits regular meetings or for follow-up of actions agreed at the ICA. In India, at the national level, there is no evidence that many partners have a shared understanding or even knowledge of the TOC, or that their individual project-specific efforts are part of a bigger design. Efforts at convening ACCCRN partners in India to reflect upon UCCR-related developments have met with very limited success. Grantees operate in a highly competitive funding context. As the grants do not have contractually binding obligation for engaging in higher order reflection, learning and cooperation with other partners, there has been no motivation to do so. The separation of TARU and GEAG as implementing partners created an unhelpful tension in efforts to establish the needed collaboration and exchange. This led to disruption for the India program, which would have benefitted from more critical mass across the cities. With no clear design at the beginning of the program, organic outcomes became difficult to undo later on when TARU became the designated national partner. Coordination effort at national level in India was limited since many of the national partners were brought into the program late. These partners have had very little linkage to city projects (except as occasional subjects of research) and have not effectively engaged at the city-level. There were times when energy was absorbed by some tense relationships between grantees, for example around the UCCR conceptual territory. Partners reported an ACCCRN milieu which tended towards being competitive, and sometimes confrontational. This could have been reduced by either exploring other grant types (such as collaborative or consortium grants, or grants with some form of mutual interdependency) or a higher degree of transparency about what each grantee was working on. The detail of each grantee’s grant was only apparent bilaterally to the grantee and the Foundation. This created multiple opportunities for confusion, inefficiency, overlap, and resentment. A higher standard of transparency in initiatives would improve their smooth running. This may just be within the confines of an initiative, but many development organizations are now signed up to the International Aid Transparency Initiative (IATI), including US-based foundations, such as the Bill and Melinda Gates Foundation and the William and Flora Hewlett Foundation. The Rockefeller Foundation also may wish to consider going in this direction. One area of grant management, or even grant architecture, that might have been given more emphasis in the absence of collaborative grants is the fulcrum (or mutually reinforcing) relation between the different kinds of grants. For example, there is no evidence in India of any substantial linking between the National Institute of Urban Affairs (NIUA) grant at national level and the city grants. Similarly, national organizations conducting research were not providing any service or significant feedback to cities, except perhaps the emerging feedback provided to Surat by the Indian Council for Research on International Economic Relations (ICRIER) on temperature and worker productivity. Grantees clearly have interacted and, in some places such as Vietnam, interaction (networking, even) is well developed. But across the Initiative, it was not built in by design. Partners reported an ACCCRN milieu which tended towards being competitive, and sometimes confrontational.
- 51. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 29 Grantee performance If an initiative is structured around single organization grants with a tendency to see grantees as service providers, then the result is likely to be less holistic and experimental than it might have been. The system drives grantees towards performing within the confines of their grant. Thus, for example, IIED was contracted to support other grantees in documentation and ensure 20 papers on UCCR were published over two years. This is not the same as having a remit to undertake crosscutting research on ACCCRN. Grantee performance is not easy to assess, since i) the structure of the Foundation’s grants ties grantees to deliverables – which are tangible products, mostly reports – rather than development results, and ii) the unit of analysis for the results framework monitoring was the city and the country, rather than the grantee. Grantee performance could be inferred from this, but it was not the focus of the assessment. Thus, while the Foundation staffinBangkokhasbeencloseenoughtogranteestobe generally aware of performance issues, these have been managed in real time, and grantees been given time to improve in areas of weakness, rather than resorting to use of an explicit performance framework. The evaluation found that country coordinator grants have generally worked well in Vietnam and Indonesia. They have played a major role in implementing ACCCRN, collaborating well with other grantees and building relationships with national agencies, NGOs, and development donors on UCCR. They have helped promote UCCR successfully, through conferences, national forums, and publications. Support to cities has been valued, although a theoretical bias was mentioned in Vietnam. In India, which has two coordinators (TARU and GEAG) and operates on a much greater scale, it has been difficult for either grantee to play a national role or convene ACCCRN India grantees to discuss, reflect and strategize. Recipients perceive TARU as a very effective and expert service provider while GEAG is seen as more focused on Gorakhpur. In Thailand, TEI has had to find indirect ways to work at a national scale, e.g. through universities. The only grant that has been closed was Arup’s grant to “support the creation of a learning and policy network that includes current ACCCRN partners and other key institutions relevant to urban climate change resilience.” The bases for the closure were that i) by the second half of 2012, country partners were much more engaged in network development within their countries than had been foreseen, and it became clear that Arup was no longer ideally placed to address country-based networking challenges and opportunities in line with the needs, and that ii) Arup had experienced staffing changes during the life of this grant, making it somewhat more challenging to deliver within this changed context. 4.3 Program outcomes vs. city projects One critique heard about ACCCRN is that it has functioned as a program more than an initiative, i.e. that it has been quite task-oriented and focused its attention on the city projects. However, as discussed, the city projects are a major part of the design of ACCCRN – they provide key capacity-building opportunities through learning-by-doing, and are intended to genuinely contribute to improved city resilience. They have undoubtedly been an important and integral part of ACCCRN’s success – showing how UCCR can work on the ground. We also recognize that ACCCRN has had an eye on a bigger picture, particularly the ADB-DFID-RF Urban Climate Change Resilience Partnership (UCCRP), donor convenings at Bellagio, and engaging laterally in trying to lever private sector funding. Thus the “program” critique does not hold. As noted in Chapter 1, 107 ACCCRN grants worth $59.3 million were committed, and of these, 38 grants worth $15.9 million were for re-granting to cities. This is … city projects are a major part of the design of ACCCRN – they provide key capacity- building opportunities …
- 52. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE30 27 percent of the whole. Given that one of ACCCRN’s major successes is its having shown how UCCR can be operationalized (through SLD ➝ CRS ➝ city resilience projects), this appears to be a proportionate investment. There is, however, some concern over what the projects themselves have really achieved. There are indications that not all cities have seen ACCCRN holistically, and that the CRS has been seen as an exercise to qualify for project funding. For example, in Surat, the CRS is a shelved document. Thus, where there is not broader institutionalization of resilience, the projects (and the project investment) risk being a means to an end – not part of a bigger resilience picture. There is a suggestion that, despite not being institutionalized, the process of producing the CRSs served as a catalyst for stakeholders to develop new institutions (such as the Climate Trust in Surat) and strategies. This argument has some merit, although the question remains as to whether stakeholders could have been catalyzed in other ways, without focusing on a document that then had low utility. Some respondents are concerned that the projects are not delivering the promised contribution to UCCR and without a clear measurement framework for UCCR, it is uncertain what the city projects have brought to building resilience. There is also concern that projects are not being properly monitored and that ACCCRN has significantly shifted its attention to replication (Outcome 3). There is a need to ensure that the city projects are properly consolidated. There is now a considerable risk that, in consolidation, ACCCRN will experience fragmentation – not only of city projects, but also across the broad range of fronts it is pursing. In the field visits, we found that country grantees and city sub-grantees had capacity gaps in their ability to execute the city projects effectively, for example on implementation planning, risk assessment and having an exit strategy. Some of this was addressed through technical assistance, such as that provided by Arup (2014). What can now only be a counterfactual is to question whether there would have been less need to invest in projects if the CRSs had been better embedded in formal city government planning systems. An alternate approach that might have been worth testing, would have been to develop the CRS with the government to get full institutional buy-in (as seems to be the approach for the UN Human Settlements Program (UN-HABITAT) City Climate Change Initiative (CCCI), and then to look to projects and project finance. Countries have had a significant focus on city-level work and,lateranincreaseinprogram-levelattention.However, the program-level attention to non-city grantees does not appear well coordinated or mutually reinforcing. This issue of coordination, rather than balance of effort, between cities and program levels was found in Thailand and India, with very little linkage between national level and city projects. In both Indonesia and Vietnam, urban development programming has been informed by the ACCCRN projects. At the national level in Indonesia, the National Development Planning Agency is supporting and encouraging cities to mainstream climate change adaptation into city development plans. In order to do so, cities need to submit resilience planning and vulnerability assessments, informed by a multi-stake- holder consultation process. ACCCRN activities are being used as pilots for RAN-API. Currently, there are 15–20 pilot sites (cities and provinces) for Indonesia’s National Action Plan, and ACCCRN cities are among these. 4.4 Internal communications The TORs asked us to consider communications and networking. This section focuses on the Strategy and Alignment Group (SAG) as the main vehicle for …we found that country grantees and city sub-grantees had capacity gaps in their ability to execute the city projects effectively…
- 53. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 31 internal communications. Findings on networking and knowledge management are covered in Chapter 4: Effectiveness. The MTE, critical of the structure of grantor-grantee relationships in ACCCRN, recommended a more web-like structure with more and deeper relationships directly among grantees. The SAG, comprising key rep- resentatives from major grantees, was formed in 2011. Arup designed and managed the SAG process under its grant. The original (2011) objectives of the SAG were to: • strengthen the level of alignment among key ACCCRN partners in relation to overall program goals, progress towards achievement of results, key emerging lessons, priorities, opportunities, and challenges • provide collective strategic guidance to ACCCRN, informing, advising, and guiding decisions on program priorities. SAG meetings occur every six months. More recent meetings have adjusted the objectives to: • maximize the long-term impact of ACCCRN by focusing on the organizational learning and change (institutionalizing) and the strategies that are needed at an organizational level, as well as external incentives and support • provide an opportunity for reflection, increasing alignment, and peer-to-peer learning. Evaluation respondents have found the SAG useful as a forum for sharing information and learning about what is going on in the Initiative and, to some extent, as a coordination body. SAG meetings have tended to be strong on sharing information on progress (including reviewing the reports of the monitoring grantee), considering ACCCRN strategy and tactics, facilitating practical coordination, and reflecting on organizational and individual behaviours and how the Initiative is working. In this, the SAG has largely met its objectives. However, it has stopped short of really examining the ACCCRN approach and making cross-sectional analysis across countries and grantees. This is a nuanced point, since the SAG has been reflective, but it has been reflective on “howarewedoingonrunningthisinitiative?”–essentially firstorderreflections.Secondorderreflectionson“howto best build UCCR in different socio-political and physico- climatic contexts?” have not been a joint enterprise in the SAG. However, it is worth noting that this type of cross-sectional analysis has been undertaken and published by several grantees individually (Sharma et al., 2013; 2014; Kernaghan and da Silva, 2014; Reed et al., 2013) and by The Rockefeller Foundation (Brown etal.,2012).Furtherpublicationsareinprogress,including a NISTPASS-led paper that builds on a previous paper “Climate Adaptation Planning in Vietnam – A review of local government experience,” and a paper on Indonesia titled “Learning from climate change vulnerability assessments in Indonesia.” ACCCRN did make strategic adjustments to its initial approaches, and let the experimental process approach have time to differentiate across different countries. However, if there had been more systematized engagement with, and reflection on, the relevance of the theory of change, there could have been more significant second-order learning on the nature of the intervention and testing of the assumptions made. The ACCCRN approach has facilitated grantees in developing their own “ACCCRN flavors” and, accordingly, there have been diverse models. With the exception of the 2013 Monitoring Synthesis Report (Verulam, 2014), these have not been compared. It is not clear whether this is because of a tacit concern about how grantees would react to their approaches being compared. Nonetheless, discussions during the evaluation with major grantees revealed an appetite for The ACCCRN approach has facilitated grantees in developing their own “ACCCRN flavors” and, accordingly, there have been diverse models.
- 54. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE32 such a cross-sectional analysis to close the loop on the “ACCCRN experiment.” Finally, the series of SAG meetings has been seen to have variable quality and utility. Some, such as in early 2014, were concrete and focused. Others have been criticized for being very granular, with long debates and unresolved tensions among partners. Any future SAG meetings should include the completion of a proper cross-sectional analysis of the UCCR interventions.
- 55. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 33 Effectiveness This chapter considers how well ACCCRN has delivered on key outcome areas. It centers on the evaluation questions that cover: articulation of the UCCR concept, resilience of the ACCCRN network, effectiveness of external communications and the effectiveness of particular service grants. The chapter first considers how well the UCCR concept has been understood and communicated, and then how well the ACCCRN design has been translated into practice. ACCCRN’s communications and M&E grants are then reviewed. The chapter concludes with an assessment of networking in ACCCRN and the success of its engagement and leverage objectives. 5.1 Articulating the UCCR concept The complexity of resilience is an obstacle to comprehension (see Annex 5). Few stakeholders in ACCCRN really appreciate the multi-dimensional, complex nature of resilience in the round. Country coordinators do, but have struggled to explain it to others nationally and in cities. UCCR contains three elements: urban, climate change, and resilience. Most stakeholders have focused on the climate change first, and urban second (though urban as place, rather than urbanization as a pressure), and struggled with resilience. Thus, in Vietnam, ACCCRN’s promotion of UCCR has been taken up through the national climate response. In Indonesia, it has had traction in national climate adaptation work, although buy-inhasbeenbroaderbecauseMercyCorpshasbeen able to define UCCR as a planning and development financing issue, not only an environmental one. This has led to incorporation in local development plans. In cities, UCCR is interpreted variously as disaster risk reduction (DRR), city greening, environmental- ism, climate adaptation, and climate mitigation. Part of the issue relates to language and translation problems (Thai does not even really have a word for “city”), and part to the concept itself. Therefore, the matter of articulating UCCR clearly is a fundamental one. There are at least four distinct definitional and articulation challenges to putting UCCR into practice: • adding the linguistic translation of “resilience” into the various regional languages and lingua franca • explaining the academically complicated concepts of resilience and UCCR in simpler language • translating the academic concept of resilience into a set of actions that can be operationalized • defining resilience and UCCR with a degree of precision that allows measurement. In addition to the direct translation problem, the way in which climate change resilience has been absorbed into the national climate change context has not been 5
- 56. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE34 consistent. Thus, in Vietnam, climate change resilience has been taken as climate change response, and in Indonesia, climate change adaptation (CCA). ACCCRN has been permissive, possibly pragmatic, in accepting these alternative translations or working definitions. As described by Friend and Moench (2013), and evident from the evaluators’ city visits, direct translation of “resilience” is difficult, and the meaning of the words used for resilience carry different connotations to those underlying “resilience.” The difference is not merely semantic. At best, some of the substance of resilience is lost, and at worst, it is severely watered down (e.g. to city greening). There is a well-rehearsed debate on whether resilience is an end-point or outcome, or whether, as is now more widely accepted, it is an ability or capacity. This leads to a variety of definitions of resilience and developments of the concept (these are detailed in Annex 5). ACCCRN itself defines resilience as: “the capacity of an individual, community, or institution to dynamically and effectively respond to shifting climate impact circumstances while continuing to function and prosper. Simply, it is the ability to survive, recover from, and even thrive in changing climatic conditions” (ACCCRN.org website). Arup’s working definition of city resilience for its Arup/Rockefeller City Resilience Framework is: “the capacity of cities to function, so that the people living and working in cities – particularly the poor and vulnerable – survive and thrive no matter what stresses or shocks they encounter” (Arup, 2014). MONITOR IDEN TIFY ACTIONS PRIORITIZ E DESIGN IMPLEMENT AGENTSRESPONSIVENESS•LEARNRE SOURCEFULNESS SYSTEMS SAFE FAILURE•FLEXIBILITY&DIVERSITY MODUL ARITY&REDUNDANCY INSTITUTIONS DECISION-MAKING INFORMATION • INSTITUTIONS CLIMATE EXPOSURE AGENTS SYSTEMS Infrastructure Ecosystems LOCAL KNOWLEDGE CATALYTIC AGENT SCIENTIFIC KNOWLEDGE BUILDING RESILIENCE UNDERSTANDING VULNERABILITY SHARED LEARNING A B C FIGURE 6: The ISET Urban climate resilience planning framework Source: Friend and MacClune, 2013
- 57. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 35 These short definitions, while clear in their intent, do not fully articulate what resilience is made up of. Simple on the surface, they do not convey the complexity of resilience and certainly not UCCR. There is no shortage of attempts to do this (as discussed in Annex 5) and ACCCRN has contributed to this development. ISET’s framework has the benefit of being dynamic and providing guidance for a process of trying to build UCCR. This works conceptually, as a high-level model, to articulate the interplay between exploring the problem and testing solutions that is required in addressing wicked problems. It is a dynamic model, which suggests the processes required to build UCCR. However, it can be argued that UCCR needs to be defined in terms other than as process components. Itshouldalsobenotedthat,inpractice,ACCCRNhasnot completely adhered to the model. Areas of divergence areshowninthethree(yellow)ovals,labelledA,B,andC. A relates to findings that some of the city engagement (Phase 2) and SLD processes were felt to be dominated by a climate science perspective. B relates to the level of monitoring by cities and stakeholders on the extent to which resilience has been built, and C relates to the extent to which reflection and learning continue to be a key feature of the overall UCCR enterprise. The ACCCRN approach (ISET framework) is an important contribution to learning how to operationalize UCCR, and through communications and replication, the approach is spreading. However, it should be recognized that it is not unique. For example, the UN-HABITAT Cities and Climate Change Initiative (CCCI) has published a guide and accompanying Toolkit (UN-HABITAT, 2014) for urban planners planning for climate change. The CCCI approach is similar to ACCCRN’s and references ACCCRN work. However it follows a normative planning cycle (plan- do-review) whereas ACCCRN’s stakeholder-based review steps (monitor-evaluate-adjust-modify) appear to be weak. ACCCRN has shied away from a measureable definition of resilience, producing instead definitions as process components, principles, and simple text. This presents problems of both management and measurement as cities and other stakeholders lack a clear means to assess whether their efforts are leading to greater resilience. Therefore, in articulating UCCR, there is a clear need for indicators to put flesh on the thin bones of explanation. ISET has produced a set of UCCR indicators, which it hastestedonandwithACCCRNcities(Tyleretal.,2014). Arup (2014) has developed a City Resilience Index under a grant from the Foundation’s resilience team, not from ACCCRN. This index is designed to cover multiple types of shock and stress to cities, not only climate change. It is already being rolled out by agencies, including 100RC and the World Bank (in Vietnam), although it is still at pilot stage. Both these frameworks have been developed from a systems perspective and aim to capture the status of governance subsystems, human sub-systems, and infrastructure and services systems. Both frameworks have benefitted greatly from their creator’s involvement in ACCCRN. There is a move, led by the Stockholm Resilience Center, to consider sustainability as living within a “safe operating space for humanity” (Rockström et al., 2009). This means living within a hard outer boundary set by the limits of the physical environment, and a soft inner boundary set by the limits of socio-economics and social justice. The question is how humans navigate the “safe and just space” – the resilient pathways – between the two boundaries. As has been shown in ACCCRN CRSs, these pathways combine hard and soft disciplines and views, and help people navigate UCCR is an attractive concept, and the Foundation was an early entrant in this field, though there are others running in parallel tracks, which has implications for legacy.
- 58. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE36 turbulences and uncertainties, particularly those driven by climate change. Both the ISET and Arup frameworks contain elements of these hard and soft boundaries, and are compatible with the “safe operating space” concept. The Arup index will break down further into about 50 subindicators and up to 150 variables. Once tested, it has the potential to be a well-articulated urban resilience measurement framework, although it may also have heavy data demands in an environment where there is a paucity of comparable data. The ISET framework, tested in eight ACCCRN cities, does not contain indicators per se, but instead provides a structure for identifying site-specific (and project- specific) indicators. Thus, it has the advantage of being relevant to the individual cities, but the disadvantage –comparedtotheArupindex–ofnotbeingcomparable across cities. The potential advantage of the index’s cross-city calibration needs to be assessed, and a “resilience review” may provide a good opportunity to stress-test the framework and obtain feedback from potential users – including an assessment of their capacity to use this framework for planning, monitoring, and decision-making. UCCR is an attractive concept, and the Foundation was an early entrant in this field, though there are others running in parallel tracks, which has implications for legacy. As the concept is evolving, a techno-rational, climate science-dominated definition is broadening to be more integrated, including governance dimensions. The particular strengths of the ACCCRN definitions and thence implementation are: ISET urban resilience indicators ARUP city resilience indicators Resilience elements Characteristics Indicators Resilience categories “What” • Physical infrastructure • Ecosystems Flexibility and diversity Reliable mobility & communications Urban services Redundancy and modularity Continuity of critical services Safe failure Reduced physical exposure “Who” Agents – individuals, households, and organizations Responsiveness Finance including contingency funds Economy and society Resourcefulness Social stability and security Capacity to learn Collective identity & mutual support “How” Institutions Rights and entitlements Minimal human vulnerability Health and wellbeing Decision-making Livelihoods & employment Information Safeguards to human life & health Application of new knowledge Integrated development planning Leadership and strategy Empowered stakeholders Effective leadership & management Source: Tyler et al., 2014. Source: Arup, 2014. TABLE 5: Comparison of resilience indicator frameworks
- 59. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 37 • iterative – the design includes learning loops • integrated – the design aims to bring together components from across sectors and services • inclusive – the design aims to involve a wide range of stakeholders. The “three circles” concept behind UCCR – climate, urban, people – is widely understood, though different countries and grantees have emphasized different aspects of the circles. UCCR is nonetheless a difficult concept; resilience itself is difficult to communicate. There is currently a measurement gap. Having a comprehensive set of indicators would help put flesh on the thin definitional bones that prevail and thus narrow the measurement gap. Several indicator frameworks exist, and these need more piloting and review. 5.2 Putting the ACCCRN design into practice In this section, we consider two aspects of the design in practice: first, how the core approach has been adapted under different country-city-grantee combinations, and second, how the social learning principles of ACCCRN have translated into practice. Adaptations of the ACCCRN approach The idea of developing models and disseminating them for replication, central to the conception of ACCCRN, was articulated in its Immediate Impact Statement which proposed: “a diverse range of effective approaches, processes, and practices to build urban climate change resilience … ” This aim was subsequently articulated by Moench et al. (2011) as: “generating replicable models and interventions for climate adaptation in medium-sized Asian cities. The Initiative takes an action research approach that has catalyzed city level actors to assess key climate stresses and potential vulnerabilities and to propose measures to respond to them.” During the 2013 monitoring, six distinct cases of approach, process, and practice were identified (Verulam, 2014) and endorsed and developed by the Foundation ACCCRN team (Rockefeller, 2014). The core ACCCRN elements of initial awareness, knowledge sharing and learning, vulnerability assessment and the development of a city resilience strategy as the precursor to developing and promoting project- and program-level city activity were present in all cases. However, the monitoring analysis found that the adaptations differ with respect to: • the skill base and orientation of the facilitating agency • the entry point within the city • the nature of the city team membership and functioning • the relationship with city government • the extent of internal/external facilitation once initiated • the reach and depth of activities • the emerging institutional framework of UCCR in the city. The assessment is that of these factors, the most important are i) the context of the socio-political national and city levels, and ii) the pre-existing competence and orientation of the grantee. Of these models, four are first generation, as they emerged from the initial 10 cities. ISET’s USAID-funded Mekong-Building Climate Resilience in Asian Cities (M-BRACE) Program applied the experience from four of the 10 cities to four new cities. However, it has taken the understanding of the dynamics of urbanization as its starting point, rather than climate change, and seen urbanization as a transformative process driven by regional economic integration. It has also seen urbanization as a governance challenge, working through SLD processes to focus on what stakeholders identify as critical gaps in land use planning and enforcement. The ICLEI model is also considered a second-gener- ation or replication model. Having received a grant in
- 60. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE38 2010 to develop a streamlined toolkit and test it in three Indian cities (Shimla, Bhubaneswar and Mysore), ICLEI then received a major upscaling grant (2012 CAC 311; $1.75 million) in 2012 to expand the ACCCRN footprint by up to 40 more cities. The streamlined approach, now known as the ICLEI-ACCCRN Process (IAP), is a distilled version of the typological ACCCRN approach, but with distinctive ICLEI elements (such as the entry point being city government) – making it essentially an “ACCCRN-lite” approach. ICLEI has since received a further “engagement building” grant of $600,000 (2013 CAC 308) to provide small grants to stakeholder groups, not cities, to reinforce the broader engagement, which is a critical part of the UCCR process. In addition to the IAP, the evaluation reviewed two further second-generation approaches: Mercy Corps’ work with the APEKSI Best Practice Transfer Program 1. Community empowerment neighborhood/ward focused approach – Gorakhpur/GEAG GEAG is unique in being a city-based NGO that was already active in the environmental field. Its community-based model is not present elsewhere. By focusing on a neighborhood, it has embedded elements of UCCR at a depth not achieved elsewhere. 2. Technocratic project approach – e.g. Surat/TARU TARU has strength in its technical expertise. Its approach has centered on the development of a quality portfolio of practical citywide and focused UCCR projects in collaboration with city government and other partners. The design and implementation of projects in Surat and Indore has demonstrated UCCR in specific practical contexts. 3. Multi-stakeholder engagement approach – e.g. Semarang/Mercy Corps In Indonesia, the original notion of multiple stakeholder engagement has sustained from the initial awareness raising through implementation. Broad-based city teams continue at the center of the process, liaising with other involved stakeholders from city government, academia, civil society, and private sector. 4. City climate cell – CCCO/ISET Vietnam The particular nature of Vietnam’s political context presented both challenges and opportunities. The response was to establish climate-related expertise within city structures. The CCCOs are now permanent parts of the city administration, providing a UCCR resource not present in other cities. 5. Choice of entry point – climate specific or problem identification/governance – ISET/TEI (ACCCRN vs. M-BRACE) The M-BRACE program (being implemented by ISET and TEI in Vietnam and Thailand) adapts the original ACCCRN model. Instead of taking a climate-specific entry point to UCCR, the discussion with stakeholders focuses on their own identification of problems facing the city. In practice, this has centered more on governance issues than climate-specific problems. 6. Light touch city facilitation – ICLEI ICLEI has more recently designed a thinner and shorter version of the ACCCRN engagement process. Even in its early stages, the leadership and ownership of the city government, continued support from ICLEI through its membership network, and the higher political profile are yielding results not seen in other approaches (this is expanded upon in Annex 6 as one of ACCCRN’s second-generation models). TABLE 6: Six different ACCCRN models Source: Verulam, 2014, and The Rockefeller Foundation, 2014.
- 61. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 39 (BPTP) in Indonesia, and ISET’s M-BRACE Program (see above) in Vietnam and Thailand.17 There are thus at least seven emerging models (four first generation and IAP, M-BRACE and BPTP), which well fulfills ACCCRN’s aim for a “diverse range of effective approaches, processes, and practices.” It also exemplifies the use of experimentation suggested as the preferred approach to dealing with complex and wicked problems and to stimulating innovation (as discussed in Chapter 2). In addition to ACCCRN’s initial analysis, Arup and TERI have conducted some multi-city reviews, but they have not used this type of typology as the unit of analysis to examine what works where, and why. The evaluation recommends development of this cross-sectional analysis to reveal the time- and cost-effective means of enabling a city-owned and managed process, which can result in a credible CRS. This could form a basis from which to motivate access to climate-resilience and related funding streams. This analysis should thus consider the factors identified in the 2013 monitoring: • the national and local governance and socio-economic context • the nature of the grantee/facilitator (including its institutional purpose, positional power, and primary relationships) • the entry point for engaging in UCCR (ICLEI uses its network to engage first with the city government while Mercy Corps began by identifying individuals who represented the widest range of stakeholders) • the nature of the city working group/team (e.g. its membership including government, CSOs, and academia as well as its structural form, officers, mandate, location, frequency of meetings, and funding) • the reach and depth of activities (e.g. scope and scale of projects, city-wide awareness activities, monitoring activities). 17 See Annex 6 for a more detailed description of these three replication models. It also should consider a further four clusters of factors derived from Kernaghan and da Silva (2014), namely: • the levels, mechanisms, use, and exchange of local and formal knowledge • the spread, role, and influence of diverse stakeholders • the extent of linkage with local and national plans and policies • the linkage to municipal and national public finance, donor, and private sector funding. Social learning in practice The evaluation has judged a social learning approach to UCCR-building to be a relevant design concept (as introduced in Chapter 2). The following discusses the extent to which this successfully guided the ACCCRN’s implementation. In applying social learning, tension can emerge in two areas: • from espousing a social learning approach and (as in many cities) simultaneously taking a techno-ra- tional, climate science-led approach • from a phase-shift between an espoused social learning approach (which was very evident in Phase 2 of ACCCRN, although running alongside an expert-led approach), and a more linear, city project-driven approach in Phase 3). ACCCRN and in particular ISET documentation (e.g. Moench et al., 2011), supports and promotes the appro- priateness of a social learning approach to building UCCR (ACCCRN’s “espoused theory”). However, a number of evaluation respondents held the view that Phase 2 was less socialized than documents suggest, being dominated by a more didactic approach that privileged climate science and climate scientists The idealized version of the ACCCRN approach undoubtedly features an iterative multi-stakeholder learning and dialogue- based diagnostic and planning process that results in a CRS.
- 62. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE40 (the “theory in practice”). This is supported by TERI’s analysis of ACCCRN interventions in India (Sharma et al., 2013; 2014), although India is not the only country where we heard this critique. It is also the impression that is conveyed by the CRSs. The idealized version of the ACCCRN approach undoubtedly features an iterative multi-stakeholder learning/dialogue-based diagnostic and planning process that results in a CRS. The purpose of a CRS is to: “produce a set of strategies to cope with climate change impacts that will be integrated into urban development policy” and to “translate strategies into several adaption18 actions to enhance resilience” (ACCCRN, 2010) and then to prioritize adaptation activities and prepare draft proposals for the prioritized actions. The publication of the CRSs marked ACCCRN’s formal shift from Phase 2 to Phase 3, but it also marked something of a phase shift in approach. It is at this point that stakeholders start to behave in a more normative, linear, project-delivery manner. The elements of iterative learning at the city level appear to feature much less going into Phase 3. That is not to say that ACCCRN has abandoned learning. Indeed the Arup- facilitated knowledge forums, ISET-led write-shops, and IIED-led research and associated meetings have stimulated reflection and learning, and there are some useful examples of city learning, such as City-to-City Workshops and City Learning Exchanges in Thailand. Also, the working groups and city teams are valued in all locations. The multi-stakeholder, multi-disci- plinary format is novel and many new connections and exchanges of ideas have happened. However, the finding is that the function of the groups/teams changed once city projects commenced. They shifted to become much more operationally focused, concerned with managing the delivery of the city projects. They deal more with the first-order problems of managing projects than with second-order reflection on becoming more resilient. While the groups or teams 18 In Indonesia, adaptation has been the term commonly used, rather than resilience. The two are essentially used synonymously. continue to meet, they are less clear about their future roles, purpose and function. They recognize that they are useful in helping address climate change-related problems, but in India, the Surat Climate Change Trust (SCCT) does not fit into the everyday budgeting and functioning of local government. In Bandar Lampung, thecityteamisconcernedaboutwhatwillhappenwhen ACCCRN funding expires and is shaping itself towards more sectorally oriented subgroups that will be tasked with raising funds. These two examples illustrate that social learning is not high on their agendas, yet it would be a basis for keeping them together. 5.3 ACCCRN as a network The TORs ask: “How resilient is the ACCCRN cities network? Is it sustainable, useful, and distinctive? What lessons emerge for Rockefeller on building networks?” This raises a number of definitional questions about the networking objectives in ACCCRN. In 2013, the outcomes were revised from networking being part of Outcome 2: “Network for knowledge, learning and engagement:Sharedpracticalknowledgetobuildurban climate change resilience …” to Outcome 3: “Expansion, networking, scaling up: UCCR is expanded with ACCCRN and new cities sharing experience through existing and new networks … ”. The shift from network to networking and the inclusion of non-ACCCRN networks is a noteworthy change of direction for this aspect of ACCCRN. The MTE noted the absence of a compelling value proposition for a network of ACCCRN cities, and that a country-by-country type of network was emerging. It recommended a focus on linking UCCR champions and city organizations into existing networks. In institution building, there is a choice between building new or buying into and strengthening existing institutions. The ACCCRN instinct has been to build an ACCCRN network, rather than to buy into existing networks, though this is changing.
- 63. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 41 The ACCCRN instinct has been to build an ACCCRN network, rather than to buy into existing networks, though this is changing in the more recent phase of network grant-making. The build option is the more difficult, requiring a compelling value proposition which can then build a critical membership mass. Achieving critical mass of membership and spontaneous network activity also has been a challenge. ACCCRN’s original networking conception was of a “cities network for sharing practical knowledge on urban climate change resilience.” However, since then, the core purpose of the network in ACCCRN has been somewhat mutable, with the approach and structure altering in phases alongside changes in the intended function. • Phase 1: “sharing practical knowledge on urban climate change resilience” – facilitated by The Rockefeller Foundation. • Phase 2: “knowledge management, organizational learning and reflection, identity formation” – to enable internal reflection on successes and failures of Phase 2, and in turn, to inform the network transition to Phase 3. • Phase 3: “a learning and policy network” – a mul- tifunctional network that has: “a unique identity distinct from that of the ACCCRN projects at the city level – but making sure the network has The Rockefeller Foundation brand clearly and consistently” and aims to: “Manage multiple and diverse membership, facilitate provision of services and support to members – promote shared ownership to avoid hub and spoke.” • Phase 4: “Expanding country and regional networks” – institutions embrace and disseminate ACCCRN-generated and other urban climate change resilience (UCCR) resources, tools, skills, and knowledge to promote UCCR – a multifunc- tional network with a membership of urban climate change resilience champions, including both individual practitioners and institutions. This evolution of the ACCCRN network, described in more detail and analyzed in Annex 7, draws in part on the framework developed by Mendazabel (2006 a, b). Knowledge hub, knowledge forums and other learning and networking Oncetheinitialintensiveworkwithcitieswascompleted in Phase 1, there was less convening of the wider network. The last event involving all city partners and other grantees was in Bali in 2011. Beginning with Phase 2, network activity has centered on the knowledge hub as an electronic medium and on the knowledge forums and other learning events that have brought together smaller groups. The sharing and exchange of knowledge is generally regarded as a highly social process, often predicated on the establishment of relationships and a level of trust. However the ability to accomplish this in ACCCRN has been tempered by the practical constraints of geography and resources. To enable knowledge sharing across this wide geography, an information technology solution has been implemented in the form of the ACCCRN Knowledge Hub. The evaluation found that aspects of the Knowledge Hub were supply-driven, but without the demand to justify them. A similar initiative by the World Bank had also failed in trying to build a community of practice around “understanding risk,” and lessons had not been learned from this. Such initiatives aimed at building a community of practice need to have clear incentives, commitment, and shared investments in achieving mutually beneficial outcomes that serve the primary work purposes of the participants. The Knowledge Hub was too ambitious in scope with too few people to create a critical mass. The success of online forums depends on the value users place in the content as well as on user community size, the trust implicit among members, and the so-called “power law of participation” which defines a rule of thumb that 90 percent of members lurk, 9 percent respond to content, and 1 percent create original content. In addition, participation was limited by language for many who made use of the Hub at city level, especially in Vietnam.
- 64. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE42 Motivation is best stimulated through face-to-face meetings of the member community. This occurred across the whole ACCCRN community through the KnowledgeForumsandwithmoreselect groupsthrough other learning events. However participation was limited, the focus was specific to learning, and there was no continuity or wider network benefit. These events were valued but respondents indicated that this was largely due to the personal knowledge gained, rather than any bonding across the community or development of network products other than publications, which inevitably only involved small numbers of actors. National networks From a number of dimensions – language, context, building on existing networks, physical proximity, stronger binding forces, and the greater ability to have a blended network (online and face-to-face components) – efforts to build networks on UCCR are more likely to succeed if they focus on the national level, bringing together individuals (political leaders and professional practitioners), community groups, civil society organizations, private companies, research institutions, international organizations, and national and subnational government authorities to exchange knowledge, share good practices, and learn together about urban climate change resilience. This is already proving to be the case with specific climate change networks and through established urban networks.19 ACCCRN network facilitation The first grant dedicated to developing the ACCCRN network was given to Arup during Phase 3. The Arup 19 Established urban networks include the Vietnam Urban Forum that predates ACCCRN (http://www.citiesalliance.org/node/3341; http://uda. com.vn/news/category/45/vi-vn/english-corner.aspx) and the related Vietnamese Urban Climate Resilience Community of Practice (http:// urbanclimatevn.com/); the Indonesia Climate Change Alliance; ICLEI national networks; and national networks of local government in India and Indonesia. analysisofnetworking(Arup,2014)wasverythoroughly researched and scoped. However, this analysis took time to arrive, risked being over-engineered, and by the time Arup was ready to commission it, it no longer had the right staff expertise in place. Conflicts with national initiatives seem to have arisen, and by the time this grant was terminated, more momentum had been lost. The role of Mercy Corps, as the new network grantee, includes: • sustaining the engagement of current ACCCRN partners and actively engaging new UCCR practitioners and institutions to become part of the network and to expand membership beyond the region • managing the acccrn.org website and knowledge portal • continuing to foster the sharing of lessons learned from ACCCRN through programmed activities, including small meetings, convenings, and webinars • linking to and translating knowledge generated by individual champions and country partners to the regional level for dissemination and wider sharing • facilitating an ACCCRN Network governance structure and process • developingmechanismsforlong-termsustainability, in terms of both funding and participation. This is somewhat at odds with views heard from ACCCRN during the evaluation – that a sustaining network should not necessarily be the aim, and that even a well-facilitated forum that delivers benefits over its four-year life and meets its objectives may fade when ACCCRN funding completes. Investing time and effort in building a network that may sustain once direct funding runs out, versus running a network that maximizes its networking and knowledge-shar- ing functions within a finite lifespan, are two different strategies which may be sequential, as short-term benefits to members may increase the likelihood of longer term support. However, The Rockefeller Foundation, Mercy Corps, and ICCCAD need a clearer shared view on this in order to determine whether members do value the networking, and thus, whether a longer term strategy is justified. … efforts to build networks on UCCR are more likely to succeed if they focus on the national level …
- 65. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 43 The shape of the Phase 4 networking grant was under development during and immediately after the evaluation. At the time of the evaluation, the evaluators were concerned that the new grant should clearly build on the analysis and learning of the Phase 3 grant, and give sufficient weight to supporting national learning initiatives/communities of practice on urban climate change, which are broader than ACCCRN. The Rockefeller Foundation has asked Mercy Corps, as networking grantee, to build a network that can provide services to 100RC, thus adding another function and a further delivery pressure to an already ambitious remit. This brings into sharp relief the question of whether the ACCCRN network should aim to have a high-level international identity (as is suggested by this 100RC-related objectives), or center on fostering a series of national-level networks, as seems to be the emerging pattern. The glue for the ACCCRN network has been ACCCRN funding. The knowledge and networking incentives have not yet sufficiently convinced members and potential members of the network to keep it self- sustaining (if indeed networks ever are). The national networks tend to have stronger binding forces and appear more likely to succeed. It is not yet evident that the Mercy Corps- (and eventually ICCCAD-) led network will overcome the obstacles. The evaluation found that all actors do not have the same understanding of the incentives, demands for, and functions of an ACCCRN network. As ACCCRN has moved into its consolidation phase, the need for an established “ACCCRN” network has appeared to arise more from within The Rockefeller Foundation than from those who have received support via ACCCRN. There is, at least within three of the countries, a set of country actors motivated to interact with a sense of purpose. These actors have ideas and experiences to exchange, and feel comfortable in doing so. The direction is thus much more towards national networks. Where stakeholders have a remaining interest in an ACCCRN network is at least in part related to the prospect of resources and funding opportunities to continue with implementing programs and projects of interest. ACCCRN’s efforts would be better directed towards supporting national initiatives, with potentially some meta-linkage among national networks and linkage into existing international networks, such as the Knowledge Centre on Cities and Climate Change (K4C) and UNISDR Resilient Cities. ACCCRN has a wealth of documented knowledge on UCCR and aims to maintain the acccrn.org website. However, while hosting a single repository of ACCCRN publications is good for hit rates and reputation, it places an indefinite hosting burden on ACCCRN. Therefore, the networking grant should also aim to ensure that the ACCCRN electronic library is mirrored and hosted with a number of other major urban and climate change portals. 5.4 Communications, publications and M&E grants The TORs ask, “How effective was it to have communications and M&E functions operating as grants? And what lessons emerge for Rockefeller more widely from these types of grants?” External communications grant. The external communications grant has not always progressed smoothly, although the initial problems of coming to grips with the task and the nature of a grant-based relationship were overcome, and performance is now good. The grant was complicated because there was no dedicated “client” across the Foundation’s ACCCRN team and partners. This made consensus building difficult and time consuming. The knowledge and networking incentives have not yet sufficiently convinced members and potential members of the network to keep it self-sustaining.
- 66. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE44 The grantee had to tread a difficult middle ground – it was neither close to the action (geographically or in relation to the subject matter) nor to some of the client drivers (more promotional aspects of Foundation communication from New York). Entirely in-house communications may indeed risk erring towards Foundation PR rather than UCCR messaging. But when using an external agency, a contrast may be made betweenanagencythatworksacrossmanyclient-types, including corporate social responsibility, and a mission- driven, communications-for-development organization such as British Broadcasting Corporation (BBC) Media Action. For other initiatives, the Foundation would still benefit from using a dedicated grantee, but could use it more effectively by commencing the grant early in the Initiative, working with a communications-for- development specialist, ensuring there is a dedicated communications focal point within the Foundation, and taking a more coordinated approach using social media in the Initiative. Publications grantee. Documentation was a shared activity of a number of grantees but the MTE found it was slow to take off and somewhat fragmented. More recently, ACCCRN commissioned a dedicated publications grantee to capture lessons on the ground and create communications material for national and international policy audiences and academics. This has worked well and contributed strongly to filling the document gap evident at the MTE, and to filling the gap in documentation in UCCR. The risk for assessing the success of this type of communications grantee relates to the type of results expected. Funders often expect to see instrumental results (change in policy or practice) closely attributed to pieces of research, whereas conceptual results are far more common – meaning more indirect and accumulative changes in knowledge, understanding, and attitudes (Nutley et al., 2007). The main lesson is that the grant commenced too late in the Initiative. Too early and there would have been little to publish on, but as it was, time was consumed in learning the intertwined roles and responsibilities of ACCCRN partners. Hence, an opportunity to be closer to the M&E grantee, and publish M&E deliverables, was missed. This points to a larger issue of grants architecture, which was emergent. As a take-away from ACCCRN, the Foundation should reflect on the lessons from the type and sequence of grants that typically provide a skeleton to an initiative. Communications, documentation, and M&E all have a beneficial role at early stages. The grantee commissioned a mid-term reflection on its role in ACCCRN (Allen and Yap, 2014). The review found anappropriatecombinationofapproaches,well-tailored to the research and researcher environment in the four ACCCRN countries. For example, in India, this included working closely with a number of city-level partners to produce peer-reviewed journal outputs, and channelled funds for further research through existing country partners. In Thailand, it meant collaborating with ISET and TEI to convene a series of Roundtable Expert Meetings held around key themes of urbanization, vulnerability, and climate change impact. Overall, this grant has: • improved the documentation, publication, and dissemination of research already underway, with the IIED-ACCCRN Working Paper series being an important vehicle • addedcredibilityandbrandrecognitiontoACCCRN outputs • enhanced the knowledge production capability of ACCCRN partners. M&E grantee. The M&E grantee has served as both a critical friend to The Rockefeller Foundation and ACCCRN partners, as well as a strategic advisor. This has been a useful and valued function. The regular progress reporting against the Results Framework has been the main utility from the grant. This is a unique role in the Initiative, providing the “helicopter view” of progress. The Foundation maintains good contact with its grantees, and thus awareness of what they are doing …
- 67. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 45 The Foundation maintains good contact with its grantees, and thus awareness of what they are doing, but the Monitoring Reports are the only place in which this is brought together in an aggregate manner, rated (using a red-amber-green traffic light system), and assessed against the Results Framework. Other progress reporting is either subjective or against activities in the grant letters. For these reasons – especially the presentation of an aggregate, objective, ratings-based progress assessment against the Results Framework – an M&E grantee is a role that the Foundation should routinely build into its initiatives. A number of grantees found the advice from the M&E grantee to be useful for other reasons, such as advice on using standard project management tools and techniques. However, grantees did not make the most of the assessments in the Monitoring Reports; some were self-reflective, but most saw them as a tool for upward reporting to the Foundation. 5.5 Engagement and leverage Outcome 3 states: “UCCR is expanded with ACCCRN and new cities taking action through existing and additional support (finance, policy, technical) generated by a range of actors.” A key focus for ACCCRN has therefore been on mobilizing these additional funds for UCCR. Defining this “leverage” and the level of attribution implied with it is not always precise. Nonetheless, it is evident that ACCCRN and ACCCRN partners have played an important role in mobilizing funds for UCCR above and beyond those in the Initiative grants.20 Internationally ACCCRN’s major leverage result is the Urban Climate Change Resilience Partnership (UCCRP). This partnership, initially a partnership with DFID21 20 In addition to the discussion here, see Annex 9 for examples of policy and financial leverage within ACCCRN. 21 DFID’s contribution is through its programme Managing Climate Risks for Urban Poor: http://devtracker.dfid.gov.uk/projects/GB-1-203842/ andtheADB,22 iswheretheUCCRTrustFund(UCCRTF) sits. The Trust Fund commenced in December 2013, but negotiations about its operating parameters were drawn out. However, it is an excellent demonstration of The Rockefeller Foundation’s convening power – with the Foundation23 seen as quite influential in bringing resilience thinking into the ADB’s urban operational planning. Catalyzing the UCCRTF funds around UCCR has brought much visibility to the issue within the ADB. DFID, a major investor of about $140 million, is keen to ensure an urban poverty focus is achieved in the Trust Fund. Given ACCCRN’s patchy achievement in this area, such attention is needed. Part of the delay in getting the UCCRTF operational has been the donors’ contrasting policy emphases and approaches. The opportunity to bring influence to bear at this level is an added advantage requiring sensitivity and nuance. As well as agreeing to procedures, time has been invested through a contract to Arup on translating ACCCRN lessons and approaches for the ADB, and aligning expectations and mutual learning about grant mechanisms versus loan vehicles. The transaction costs have been very high to date, but the potential exists for a step up in the scale and scope of UCCR traction. ADB’s stock-in-trade is investment in hard infrastructure. Investment in, and appreciation of, the softer aspects of UCCR are a potential value add from the partnership. The only regional example of leverage is ISET’s successful application to USAID for funding for the M-BRACE program. This funding enabled ISET to complement its ACCCRN work with two additional cities in Thailand (working with TEI) and Vietnam using a similar approach but developing a new model (as discussed in Chapter 3). Nationally Leverage and engagement by ACCCRN at the national level is less substantial and varies across the four 22 USAID has also committed to joining UCCRP with a $5 million com- mitment. Though modest, it is politically powerful to have another donor join, especially one that traditionally tends to engage in bilateral relations. 23 The ADB sees UCCRP as relating to The Rockefeller Foundation, rather than specifically ACCCRN.
- 68. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE46 countries. While engagement is evident in Indonesia and Vietnam, progress has been much harder to achieve in India and Thailand. However, there has been some subnational (state-level) leverage in India, and Vietnam has had shifts in national budget allocations that can be attributed to ACCCRN influence. Country coordinators needed the skills and access to engage nationally. In all cases, this needed to be developed and so has taken time – something grantee selection did not seem to take into account, although it is recognized that few organizations are equally as adept at national/policy-level engagement and city level engagement. For example, a relatively small consultancy company in India had to rely on individual points of contact, which, with staff changes, was not sustained. While ACCCRN has found access difficult and has not been able to leverage any substantial additional funds at national level, the recent establishment of the Urban Institute of India (Jadhav, 2014) suggests that with sufficient political access, focus, and intent, it is possible to leverage significant additional funding for urban and environmental issues in India. In Vietnam and Indonesia, international grantees needed to rely on national staff developing the knowledge and gaining access, confidence, and reputation before engagement was realized. In Thailand, constraints in the broader enabling environment (planning, governance, and finance) are a limiting factor in realizing national uptake and scaling. As a direct result of TEI’s ACCCRN experience, a five-year funding agreement is close to being concluded with the International Development Research Centre (IDRC). Through this agreement, TEI will partner with the University of Toronto to promote a research and capacity development partnership with Thai Universities that specifically addresses UCCR. In all cases, it appears the complexity of the process was underestimated and the linkage from city experience to national change was not as linear as assumed. There has been more success in influencing the donor community at national level. Positive examples of learning from ACCCRN, appreciation of city and project impact, and in some cases, co-financing, have been seen in all four countries. However major shifts in policy and financing by donors have been limited. City level This is where there is much more evidence of success. In all ten cities, city governments have provided at least some financial support, and in some cases, national (and in India, state) government support has been forthcoming. Summary In summary, ACCCRN leverage is most evident at the international (UCCRP) and city levels with more mixed experience nationally. Projects have attracted the most interest which has resulted in financing – including fiscal space being created in city and national budgets in some cases. Policy change, even at city level, has been less well evidenced. CRSs have been acknowledged but not incorporated holistically into city planning and budgeting. In addition, while there are examples of national policy discourse beginning to include resilience alongside climate adaptation, UCCR as a concept has yet to gain traction. Overall fiscal leverage has exceeded expectations while policy engagement is only just beginning to become apparent in a sustainable way. The convening power of the Foundation has been a major factor not only with UCCRP – the impact of the early donor convening hosted at Bellagio is still quoted by many informants. ACCCRN grantees have become successful in leveraging funds as experience has developed and in some cases are beginning to gain ground in policy environments where the timeframe is much longer. ACCCRN leverage is most evident at the international (UCCRP) and city levels with more mixed experience nationally.
- 69. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 47 6 Influence Influence is not normally considered a standard evaluation criterion (OECD DAC, 2010). However, in the TORs, it appeared across a number of objectives for the evaluation. Therefore for coherence, when creating the evaluation matrix (Annex 2), we grouped all the evaluation questions relating to influence, recognizing that a number of these relate to Outcome 3 and have been covered in part in the leverage section of Chapter 4. ACCCRN has created a strong footprint of published material, and its ideas and approaches are having ripples in Asia and beyond. Influence is spreading through grantees utilizing the approaches in work outside ACCCRN, and through communications activities – such as the website and presentations at the ICLEI Resilient Cities Conferences in Bonn. There is also horizontal spread, such as to the Best Practice Transfer Programme in Indonesia and to new cities through ICLEI’s network, and vertical spread, such as influencing a Prime Ministerial Decree on climate change in urban master planning in Vietnam. The wider evidence of influence leading to financial leverage and engagement has been described in Chapter 4. 6.1 Actors and audiences It is important to be clear who within ACCCRN can influence which external actors, and by which means. This applies at all levels – international, national, and city. Essentially it is asking “who is the salesforce,”24 meaning who is currently best placed to promote UCCR, and who will continue to promote UCCR when ACCCRN concludes? Without such an analysis, it is not possible to balance the opportunism and more strategic aspects of influencing. While there have been successes, the evaluation found little coherence or coordination of ACCCRNs influencing activities overall. There have been major achievements but much more might have been achieved. Those actors ACCCRN may seek to influence can be clustered into four main interrelated audiences: i) international actors involved in the global policy and financing of climate change and UCCR, including leading politicians, policy influencers, donors, and funders, ii) those responsible for the related 24 Arup Phase 4 grant proposal: Engaging professional institutions. ACCCRN has created a strong footprint of published material, and its ideas and approaches are having ripples in Asia and beyond.
- 70. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE48 implementation and resourcing frameworks, including policy actors at regional, national, and city levels, iii) practitioners engaged in implementation, including city managers, and professionals such as engineers, planners, and architects, and iv) wider audiences that could be interested or affected including academics, NGOs, civil society, the private sector, and ultimately citizens. There are different avenues to influence this spectrum of audiences, with some applying selectively and others that have wider appeal. 6.2 Influence at the international level The Rockefeller Foundation and high-profile individuals and institutions within the ACCCRN circle have had the greatest influence, often through set piece events such as the Bellagio convening and IPCC meetings. Here face-to-face interactions have worked well with both donors and funders. The Bellagio convenings utilized the Foundation’s convening power to maximum effect to gain attention and profile for ACCCRN even before the practice had been established. To a large extent, the Bellagio event can be credited with creating sufficient momentum on its own to build the UCCRP. ACCCRN’s thinking and knowledge contributions are clearly evident in the Fifth Assessment Review (AR5) of the IPCC. Five ACCCRN associates were among the authors of Chapter 8 – Urban Areas: Aromar Revi, David Satterthwaite, Debra Roberts, Jo da Silva and David Dodman. The influence of ACCCRN (alongside others) seems evident in the AR Review’s section on Understanding Resilience for Urban Centres in Relation to Climate Change. AR5 notes that: “Since AR4, a much larger and more diverse literature has accrued on current and potential climate change risks for urban populations and centers. The literature on urban ‘adaptation’ and on building resilience at city and regional scales has also expanded… .” In addition, several ACCCRN-related publications are cited, namely: Brown et al., 2012; de Silva et al., 2012; Moench et al., 2011; Tanner et al., 2009; and Tyler et al., 2010. 6.3 Influencing at national and city levels Proof of concept achieved through city practice was assumedtobethemaintoolforinfluencebutexperience hasshownthatotherentrypointstopolicydialoguecan also be effective. ACCCRN’s focus has been on UCCR. This is an elegant concept, joining Urban (U),25 climate change (CC) and resilience (R). However, the level of influence that UCCR has achieved in the four countries is closely related to the extent to which climate change has been given priority in policy-making circles. This is important, because it is the CC dimension of UCCR that has primarily received attention. Thus in Vietnam, traction is around CC response, and in Indonesia, around CC adaptation. In India, where CC has received less policy attention, ACCCRN has had less influence, particularly in terms of national policy. The proof-of-concept evident in ACCCRN cities has helped ACCCRN have an influence on shaping national policy in Vietnam and Indonesia. Influence has been less in Thailand where the political situation is less tractable. TEI does not have the right leverage in the politicized space and has consequently adjusted its influencing strategy towards a longer-term approach of working with universities – encouraging UCCR research and related teaching of a new cadre of professionals who will take up government positions in the future. In India, engagement with national players has been limited and should arguably have come earlier 25 Urban as place and as a system, more than urbanization as a pressure. ACCCRN needs to make the approach and process more efficient, by distilling it to its minimum extent while maintaining its functionality…
- 71. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 49 in the program. Greater policy influence may have been achieved through targeting state governments – possibly through focusing on cities in one state only. Influencing also needs to consider practical uptake of the lessons ACCCRN has produced in operation- alizing the process of producing a city resilience strategy. The ACCCRN approach is not seen as rapid or efficient. This is because its grantees have been developing, using, and reflecting on their approaches – essentially action research. To be more influential, ACCCRN needs to make the approach and process more efficient, by distilling it to its minimum extent while maintaining its functionality, and then making practical and simply written guidance available. The forthcoming ICLEI-ACCCRN Process (IAP) toolkit may do this. To be influential, it will need to be well promoted and disseminated. The Foundation’s ACCCRN staff has also drafted three UCCR points-of-view documents, expected to be published shortly, which distill lessons on the case for UCCR, city projects and the key learning from the wider ACCCRN experience. These would have been helpful sooner, for example to influence the design of 100RC. 6.4 ACCCRN’s wider influence With wider audiences, the influence of those “institutions less subject to political cycles – research institutions, think tanks, civil society organizations” (Brown et al., 2012) is largely through documentary material, and frequently through making this available on the Internet. The MTE found a paucity of documentary material from ACCCRN in the public domain, and recommended that ACCCRN capitalize on the learning from Phase 3. Since then, ACCCRN and its partners have invested strongly in documenting experiences and lessons. The IIED research and publication grant has been effective in helping ACCCRN achieve a presence in the academic space around urban climate change, particularly around the operationalization of supporting cities in preparing for climate change. This body of published work is important for supporting the work of the influencers in ACCCRN, and influencing some of the target audiences. However, it is not ideal for all audiences. On many occasions, the evaluation heard that ACCCRN material was too technical or academically written for practitioner audiences, some of the material was located behind publishers’ pay walls, and to be part of a legacy of influence, it needed to have a maximum availability. According to the publisher Elsevier, an ISET article on UCCR (Friend and Moench, 2013) is one of the most cited (i.e. influential) articles in the journal Urban Climate. Google Scholar currently yields 326 hits for the search term “ACCCRN” and 21 for “UCCR.” While the Scopus academic citation database shows a reasonable citation rate for the top peer-reviewed publications on ACCCRN, the influence of these citations in the urban climate change space needs to be viewed in terms of the spread of citations – i.e. the number of citations that are cross-citations from within the ACCCRN family. Within the ACCCRN family there is both fragmentation – partners mainly but not exclusively write up “their part” – and an indication of a pattern of self-citation and cross-citation. Recent reviews done by those outside ACCCRN (Bene et al., 2013) do not cite ACCCRN papers to the same extent. In this regard, ACCCRN partners need to challenge themselves to “look over the parapet” and ensure that cross- and self-citation do not create an ACCCRN bubble. There is a need to maintain a sound grasp of the emerging resilience field through, for example, engaging with wider thinking on resilience at events like the Resilience 2014 Conference (www.resilience2014.org), which at least one partner did attend. Social media reach is accepted as a measure of influence and impact. The field of altmetrics relates to assessing the impact of research and tools, used for organizations and individuals, such as the online tool “Klout” which ranks users according to social influence. At a very simple level, ACCCRN makes use of social
- 72. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE50 media, but has a relatively low reach: 306 Facebook and 347 Twitter followers (compared with C40 Cities’ 3,878 and 16,600 followers respectively). Conferences are a part of the influencing portfolio, and ACCCRN has targeted and been supportive of the ICLEI series of Resilient Cities conferences in Bonn. Such global meetings, which bring together a critical mass of major actors, provide key opportunities for influencing. This year’s World Urban Forum (WUF) offers a very good example of the influence that can be achieved (Brodzinsky, 2014). For example, the 2014 WUF formed a new global collaboration for helping cities improve resilience to disaster and climate risk, as well as to economic and other systemic shocks. This collaboration – which includes as signatories the UN Human Settlements Programme (UN-Habitat), UN Office for Disaster Risk Reduction (UNISDR), Inter-American Development Bank, The Rockefeller Foundation and the 100 Resilient Cities Centennial Challenge, the C40 Cities Climate Leadership Group, ICLEI Local Governments for Sustainability, the World Bank, and the Global Facility for Disaster Reduction and Recovery (GFDRR) – aims to improve the flow of knowledge and financial resources necessary to help cities become more resilient (World Bank, 2014). The partnership will also mobilize support for the post-2015 urban resilience agenda, including the post-2015 Sustainable Development Goals, the climate change framework, the Hyogo Framework for Action, and the Habitat III agenda – all key opportunities in the near future.26 Finally, in regard to work with professionals to extend ACCCRN’s influence and legacy, the picture is variable, and shows little engagement with professional institutions responsible for urban planning, engineering, and design of cities. In Vietnam, ACCCRN demonstrated this type of influence with a grant to the Vietnam Urban Development Planning Association to develop curricula on UCCR and train urban planning professionals. 26 While this cannot be attributed directly to ACCCRN, and it is urban resilience rather than UCCR, it is part of the wider ACCCRN-Rockefeller Foundation, UCCR- urban resilience continuum of influence. 6.5 ACCCRN’s influence in the Foundation Resilience has long been a focal area in the Foundation. It was a pillar when ACCCRN was first conceived in 2007, and it has become more central to its work. The impact of Superstorm Sandy battering New York in October 2012 added a particularly real emphasis to the Foundation’s work on urban resilience and resilience to climate-related events. ACCCRN is only a part, albeit an important part, of the Foundation’s work on resilience. Although its work on rural climate change resilience in Africa ended in 2011, the overall resilience portfolio has grown. New initiatives in this space include: • 100 Resilient Cities (100RC) Initiative – closest in intent to ACCCRN, though it considers stresses more broadly than climate change, and will select 100 cities globally to receive support and funds to become more resilient • Global Resilience Partnership (GRP) – co-funded by USAID and aims to help millions of people in the Sahel, Horn of Africa, and South and Southeast Asia build stronger and more resilient futures • Resilience by Design – a portfolio of investigations that increases understanding of resilience and a set of processes that increases the resilience of communities in the US through leveraging significant federal funding • InnovativeFinance–anefforttodevelopinnovative finance products in partnership with International Finance Institutions (IFIs), which can create new incentives for investing in urban resilience globally. ACCCRN preceded all of these initiatives by several years. So, to what extent has it influenced them? ACCCRN is indeed seen as influential within the Foundation. Its influence has two main dimensions: • the existence of ACCCRN, its city projects and its examples of success have given the Foundation the confidence to drive forward its larger resilience agenda, particularly urban resilience, knowing that it has a good empirical foundation on which to base it
- 73. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 51 • the higher level conceptual basis of ACCCRN, in terms of working at the nexus of climate change, urbanization, and poverty, and the ideas about the nature of resilient systems, have been influential. The Foundation’s meta-view of urban resilience has been captured in a draft Urban Point of View, which draws heavily on ACCCRN. However, this evaluation has found that the practical lessons on “doing resilience” – such as the need for “soft” aspects such as adapting to local socio-political system and social learning recognizing plural views, alongside “hard” project investments – have transmitted less well. Some respondents felt this was a consequence of distance, e.g. ACCCRN is Bangkok-based, and initiatives not based in New York are less accessible. Countervailing views were also heard, as some ACCCRN staff are based in New York, and the Managing Director for Asia is regularly in, or in contact with, New York, and chairs the resilience group. Likewise divergent views were heard about the accessibility of ACCCRN materials. Some had to search the Internet to find them, while others had very productive interactions directly with ACCCRN staff members, who shared materials. The point is that the Foundation does not appear to have clear ways for new initiatives to learn from old ones, despite it being in the interest of both to do so.
- 74. CEWO
- 75. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 53 7 Impact The TORs ask that the evaluation consider the extent to which ACCCRN has had an impact according to the aims in its ultimate and intermediate impact statements. In relation to intermediate impact, this chapter considers the extent to which ACCCRN has demonstrated a diverse range of effective approaches, processes and practices. In relation to the ultimate impact – the impact on the lives of urban dwellers – it considers the extent to which the capacities of a cadre of UCCR champions have been built, and whether cities are more resilient. 7.1 Demonstrating a range of approaches, processes and practices The ACCCRN approach is a step-wise model that progresses from engaging with the city through the formation of a multi-stakeholder UCCR working group, to conducting a vulnerability assessment with them, reflecting on this in an iterative learning process (the SLD), and then developing a city resilience strategy (CRS) (Figure 7). The CRS provides the basis for prioritizing resilience-building actions, usually through city projects. Adaptations of the ACCCRN Approach, presented in Section 4.2, identify four distinct first-generation ACCCRN models and three second-generation models. These are shown in Figure 8. As seen from the schematic, ACCCRN has generated a rich variety of models (i.e. processes, approaches, and practices) around the core set of steps in the idealized approach. These have been tailored by grantees and host cities to respond to the national and city context and to the expertise and orientation of the facilitating grantee. Therefore, the evaluation finds that ACCCRN has completely met its intermediate impact objective of FIGURE 7: Idealized ACCCRN approach Vulnerability assessment Shared Learning Dialogue City resilience strategy City projects Form UCCR working group Source: Verulam Associates, 2014.
- 76. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE54 demonstratinga“diverserangeofeffectiveapproaches, processes, and practices.” These practices have built urban climate change resilience and incorporated the priorities of poor and vulnerable communities to varying extents. The impact objective is to demonstrate the diverse range. ACCCRN has gone beyond that to analyse them too. All the initiatives, with the exception of a couple of the most recent second generation models, have been written up and reflected on individually, or collectively (Sharma et al., 2013; 2014; Kernaghan and da Silva, 2014; Reed et al., 2013). However, if this diversity is viewed as an experiment, the whole experiment has yet to be analysed to fully appreciate the pros and cons of these diverse approaches across contexts. This is the subject of one the evaluation’s recommendations. 7.2 Impact on urban dwellers By design, ACCCRN was intended to contribute to “Enhanced resilience … of a growing number of … cities, and improved lives of poor and vulnerable men and women.”27 The MTE concluded that citizens are 27 ACCCRN ultimate impact statement. FIGURE 8: Diverse 1st and 2nd generation ACCCRN models * Two models (shown in grey) are less well documented, with GEAG and NIUA in India cascading an approach through training, but not hands-on involvement. 1st Generation 2nd Generation 1. Community- empowerment + NGO: GEAG, Gorakhpur CEAG Cascaded NGOs* 5. Best practice transfer cities: Mercy Corps, Indonesia 6. Problem-centered entry: ISET/TEI; MBRACE 7. Govt-entry/ACCCRN-lite: ICLEI (IAP) NIUA India Cascaded training* 2. Technocratic project approach: Taur, Indore, Surat 3. Multi-stakeholder engagement approach: Semarang/Mercy Corps 4. City climate cell: CCCO/ISET Vietnam Climate specific entry point: TEI, Thailand ACCCRN Source: Verulam Associates, 2014.
- 77. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 55 surprisingly absent from ACCCRN, and urged ACCCRN to ensure that in its second half, poor and vulnerable people would be fully included in implementation projects, and would have their priorities heard, their adaptive capacity increased, and their lives improved. There is little evidence that the lives of the poor and vulnerable have improved. The main reason for this is simply the paucity of measurement of the quality of citizens’ lives [although this was a recommendation of the MTE]. In addition to the little measurement that has been undertaken in ACCCRN to demonstrate changes in lives, measurement of the quality of a life is a field of study in its own right, from the basic discussion about how to measure poverty (The Economist, 2011), to the more wide-ranging debates about well-being (Stiglitz et al., 2009). The ACCCRN Results Framework provides two measures against which it planned to assess “lives.” • Measurable improvements in the well-being of citizens within ACCCRN cities (disaggregated by those defined as poor, vulnerable to climate change effects, and by gender) – to be measured through city-level interviews, reflective group analysis and national-level perceptions. This is a qualitative measure, which can be appropriate to obtain information on the multi-dimensional aspects of lives and to obtain people’s own views, using their own self-anchored metrics. • Decrease in proportion of poor (men and women) within total population in ACCCRN and new cities – to be measured through measurable impact on poverty as a result of UCCR activity. This is a quantitative measure, which should draw on available national and sub-national survey data. Properly determining the impact of ACCCRN on lives would require a pre-ACCCRN baseline. This can be achieved through recall techniques for qualitative data, and quantative data should be available from archives. Ideally, this would also include comparator cities where ACCCRN has not worked, to take account of any underlying trends affecting lives. In the absence of any of these data, the evaluation cannot make an assessment of whether the lives of poor and vulnerable people have improved. Even when the data are available, the indicators would still need quantification with target levels. In other words, a judgementwouldstillhavehadtobemadeonhowmuch improvement in well-being and how much decrease in the proportion of poor people would reasonably constitute impact. Given that ACCCRN is running for a further two years, there is still an opportunity to collect the data and undertake this high-level impact assessment. This is not a recommendation, as the judgement as to whether this would be useful information for the Foundation is for the Foundation to take. The second reservation about this impact objective is the extent to which the lives of the poor and vulnerable were expected to be affected by ACCCRN interventions and, thus, how much impact might be expected. The classical approach to poverty reduction is to design poverty-targeting, pro-poor interventions. However, as noted in assessing the TOC, ACCCRN is predicated on the assumption that: “By working at the city level you can have more impact on the poor and vulnerable communities than by working exclusively with poor and vulnerable.” Given the above concern about measurement, it is clear there is not enough empirical evidence from ACCCRN to either prove or disprove this assumption. An analysis of the targeting, in general and of targeting the poor and vulnerable in particular, (which is analyzed in Annex 8), shows a variable approach to the consideration of beneficiaries, and poverty and vulnerability targeting. Some grants target whole systems or whole-of-city (e.g. UCCR urban plans for Chiang Rai, hydrological modeling in Quy Nhon), others aim to partially or wholly benefit the poor and vulnerable through area-based or problem-based targeting (e.g. flood early warning system in Semarang, waterborne disease surveillance system for Indore), and some do not appear to have considered the issue. Overall, it is not possible to assess the impact on urban dwellers, other than by anecdote and through inference
- 78. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE56 from the projects. It is not yet possible to say whether a whole-of-city or a targeting approach is better for reducing poverty and vulnerability. That said, poor and vulnerable people were considered in all cites during vulnerability analysis, so there is an awareness of their plight. Research conducted by TERI in 2013 (Sharma et al., 2013) reviewed the ACCCRN processes in the three original cities (Surat, Indore, Gorakhpur) and four expansion cities (Shimla, Bhubaneswar, Mysore, Guwahati). Reviewers noted that therelevantauthoritiesinIndore,Guwahati,Bhubaneswar, and Shimla considered the involvement of urban poor and community groups in the ACCCRN processes as impractical, as the issues under consideration were too technical. Engagement with community representatives was limited “to the initial stages when seeking inputs on identifying the primary risks and to an extent during implementation of the pilot projects in the core cities.” This demonstrates a failure to deeply implement a social learning paradigm in these cities, and also indicates the extent to which inclusive processes can be weakened to become merely forms of minimal consultation, and ways in which the elite can capture nominally inclusive processes. Evaluation discussions in India revealed a growing trend of city authorities relinquishing responsibility for upgrading slums by using various regulations (including environmental) for slum clearance and shifting poor people outside of city boundaries. There is a third concern about the extent to which poor and vulnerable lives have been improved by ACCCRN. This is more theoretical, but related to the relationship between resilience and poverty. In the field of resilience research, there is some doubt as to whether greater resilience and better lives for the poor are linked in the manner implied in the ACCCRN impact statement. Béné et al. (2014) expressed concern that the often technical and apolitical nature of resilience frameworks overlooks the social justice/transformative dimensions of resilience-building interventions. Thus the “bounce back from shocks” short-hand definition of resilience risks missing the transformative dimension of improving resilience through a more stable and empowered society. As evident from the Foundation’s own work and more widely in the field, the concept of resilience continues to evolve (Bahadur et al., 2010; Béné et al., 2014). The main schoolsofthoughtintheevolutionofresiliencethinking show a progression from bouncing back to equilibrium following a disturbance, to being an emergent property that retains function while undergoing change and has a capacity for learning, adaptation, and where necessary, transformation (Béné et al., 2012). In considering the distributional and wealth impacts of UCCR initiatives, ACCCRN and 100RC need to be alert to the social justice critiques of the urban resilience movement (Slater, 2014), and the implications of Piketty’s (2014) critique of the accumulation of capital. Urban systems are less resilient where there are disempowered citizens, poor social stability, and vulnerable groups with low levels of well-being. 7.3 Capacities of a cadre of champions ACCCRN has produced a cadre of motivated and appropriately skilled individuals in working groups and city teams in different cities, and has also built some capacity at national level and in a more limited way, at community level (e.g. in Danang, a storm-resistant housing project led to the consolidation of a group of volunteers that understands technical requirements of storm-resistant housing). However, the degree to which this cadre will be active after funding stops is unclear, and the extent to which it is “a cadre” depends in part on whether being linked through an ACCCRN network is a prerequisite for being a cadre. City-level capacity. In ACCCRN’s early engagement in cities, it interacted with what were seen as the more interested parties. Thus, many of the key individuals in cities who have taken some of the UCCR ideas on board were often already champions in their own sectoral spaces, and have engaged with the ACCCRN opportunity as it sufficiently overlapped with their own perspectives or ambitions. Bringing them together in a working group and undertaking social
- 79. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 57 learning on UCCR has increased collective awareness of climate change, and individuals are able to share their knowledge and information, yet very few would be able to articulate a systemic understanding of the resilience concept. However, their skills on commu- nity-based planning and conducting vulnerability assessments were created or strengthened as a result of ACCCRN. There are some threats to the capacity built at city level, as officials get rotated out of key positions. For example in Indore, where rotation out of city positions is particularly prevalent, the City Commissioner, who was supportive of ACCCRN actions, has been transferred and his replacement has different priorities. Therefore, sustainable increase of capacities of key individuals may not necessarily result in a sustainable increase in the city’s action towards UCCR. Overall, the capacity of the implementing agency for all urban-related actions – the municipality – is not very robust. In Gorakhpur, there is increased awareness of climate change among select middle class civil society individuals in GEAG and some individuals in the community with whom they work, but no evidence of increased UCCR-related capacity on the part of city officials. National-level capacity. Capacities have been built at national level. In Indonesia, ACCCRN has influenced engagementandreplicationwithnationalandmunicipal institutions by coordinating the Indonesia Climate Alliance (ICA). Similarly, enhancing collaboration with the Association of Planners included developing a strategy that calls for climate change resilience training as a legal prerequisite for accreditation as an urban planner. This is seen as an essential move to converge spatial planning with RAN-API and vulnerability assessments. In Vietnam, NISTPASS, VIAP and VUPDA have significantly improved staff capacity and skills on UCCR. VIAP has trained over 200 professionals in mainstreaming resilience within the urban planning processes, which in turn will contribute to capacity at the city level. In 2011, ACCCRN awarded a grant to ICCCAD, via IIED, to support the institutional strengthening and organizational development of ICCCAD. Initially, the grant supported ICCCAD in implementing a series of UCCR short courses and developing an urban track within ICCCAD’s masters course on climate change adaptation, in order to help establish a diverse cadre of government staff, NGOs, donors, the media, and actors in the private sector. The second year focused on training of trainers – those running UCCR courses for others in their countries – and the third and fourth years focused on people in planning departments in municipal governments. Although the course is short (one week) and accommodates only approximately 12–15 people, which is a very modest number in the whole of Asia, it targets exactly the type of people who should form a growing cadre of UCCR experts and champions in the region. Course participants have formed an alumni group, which has potential to merge with the ACCCRN network. However, there is a disconnect in that very few direct ACCCRN stakeholders have participated in ICCCAD courses, making the linkage between ACCCRN and the alumni network more difficult. 7.4 City impact TheTORsask:“towhatextenthasimpactbeenachieved at city level through the portfolio of city projects and the resilience strategies? What were the reasons for this?” Whether there has been impact on the people’s lives in cities is addressed above. Here we consider the key indicator: “Number of ACCCRN cities that have demonstrated measurable improvements in resilient urban systems through infrastructure, knowledge and institutions.” This indicator is important, because it does not ask “have cities become more resilient as a result of ACCCRN.” It asks “are there improvements in cities’ resilience systems.” This is an important distinction, since ACCCRN has not yet developed any shared approach to measuring UCCR. Thus, in assessing impact, we have sought evidence that there have been improvements in cities’ resilience
- 80. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE58 systems as a result of ACCCRN interventions. In all cities, there have been CRSs and city projects resulting from them. In some cities, such as Surat, the CRS has all but disappeared from view, while in others, such as Bandar Lampung, there is little vision beyond project-level implementation of the CRS. This gives rise to concerns that funding limitations and national-level governance issues influencing political prioritization (and therefore funding) will limit sustainability. However, despite some questions over the durability of the CRS, there are some positive impacts. In all cities, the experience of multi-stakeholder and multi-disci- plinary working has been seen as valuable. Working group and city members wish to continue this way of working, and some have already extended this to other areas of their work. This is an institutional change. Whether the change also endures in the form of a city team is less clear, due to the funding dependency. This is true even for the CCCOs in Vietnam (other than in Quy Nhon). Other institutional changes have been policy changes. For example, as a result of the Kok River restoration in Chiang Rai, there has been a policy change favouring natural and ecological systems to preserve rivers, rather than hard infrastructure. There also have been knowledge and capacity changes, with awareness and understanding about climate change and responding to it (even if not fully UCCR) increasing significantly among a wide range of city stakeholders. The capacity of working groups and city teams to conduct community-based planning and SLDs, and to collaborate with sectoral departments have all increased. Finally, some of the city projects themselves have had substantive impacts. The city projects that include planning and modelling projects deliver immediate benefits once implemented. Those designed to predict stresses in order to help avoid them – such as the Surat Disease Surveillance System, the Surat flood early warning system, and the Can Tho dengue surveillance and response system – should deliver results quickly, as long as their operation is sustained. This is likely in places such as Surat, where the projects have been adopted into the government’s own program. The projects of a protective nature, such as flood control and coastal protection, are larger interventions with soft infrastructure and require maintenance. Some of the ACCCRN projects are successful, but have small footprints. The sustainability of individual projects, such as Danang’s storm-resistant housing project, is still unclear as, although it is run as a revolving fund (with probably a limited number of cycles), no donor has confirmed an interest in supporting the project when ACCCRN funding ceases. However, there is hope from ACCCRN that ADB or the city itself may try to scale this Initiative. Inevitably, the projects selected from the CRSs were of a scale that ACCCRN could fund and of a nature that could be supported by the particular grantees. Therefore,whilesomehavebeeninfrastructureprojects – e.g. boat lifts, typhoon-resistant housing – they have not been large-scale projects. Thus, there will also be the challenge of ensuring that the approach can properly identify and then support the implementation of major infrastructure projects. ACCCRN treads a slightly dichotomous path on its successes. In places, it acknowledges that the approach is about experimenting and developing models, yet in others it has a tendency to claim impact at scale. This is not necessarily supported in the evidence, and in the absence of more regularized assessment of city resilience, there is a risk of over-claiming impact. Our assessment here has tried to be balanced, and to illustrate this, we refer to the following example featured inArup,ISETandIIEDdocuments:“Converselyrelatively In all cities, the experience of multi- stakeholder and multi-disciplinary working has been seen as valuable.
- 81. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 59 poor cities can make choices that build resilience. Gorakhpur in North Eastern India, is working to build resilience at the ward level in response to annual waterlogging in poorer parts of the city. By improving solid waste management practices to unblock drains, and increasing drainage of waterlogged areas the city has reduced incidences of diseases such as malaria and Japanese encephalitis which are spread by vectors that breed in waterlogged areas.” However, upon checking, we found that there has never been a case of Japanese encephalitis in Mahewa ward where GEAG implements its ACCCRN work. And GEAG’s ward-level intervention (in one of 72 municipal wards) has not been picked up or scaled out by the city government. Ultimately, UCCR is about creating a mosaic of synergizing capabilities and actions, some small, some large. In this jigsaw, ACCCRN has so far provided some small pieces that have made some contributions to resilience, but it is not yet systematic, and it is not yet clear whether it is sustainable. Whether resilience has been improved can really only be known in relation to how a city responds to a shock or stress – this is the real proof of resilience. ACCCRN can demonstrate success in the face of such proof. In 2013, Surat faced hydro-meteorological conditions similar to those that caused the large flood in 2006. However, this time, the flood early warning system project End-to-End had been implemented. The dam authority was able to manage water levels much better, and a catastrophic flood did not occur. The “flood that didn’t happen” is a strong example of ACCCRN-related resilience building.
- 83. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 61 8 Sustainability In examining sustainability, we need to be clear as to what ACCCRN was expected to sustain. The TORs infer that it was individual and organizational capacity, policies and practices, possibly a network, and certainly Rockefeller’s reputational legacy. To this, we would add an increase in UCCR in the ACCCRN cities. Some of these are covered in preceding chapters since, in assessing results, a view on whether they are likely to sustain is needed. Analysis of ACCCRN and UCCR-related literature by Arup (Kernaghan and da Silva, 2014) suggests that four factors are required to sustain action on UCCR: • knowledge based on local experience, supported by engagement in wider networks • presence of champions and engaged stakeholders from across government • enabling policies and plans at city, state and national levels • access to financing through donors or city budgets. ACCCRN has been working on all four components, somemorethanothers,andtovariedextentsacrossthe cities (Table 7). Arup’s main conclusions include that: i) securing finance for sustaining UCCR is the greatest challenge, ii) with the exception of raising finance, the three cities in Vietnam have the most comprehensive approach to sustaining UCCR, iii) in Thailand, engaging stakeholders and the absence of supportive polices and plans are a challenge to sustaining action, and iv) progress in India is variable across cities, with Surat being best placed to sustain UCCR. A key assessment emerging from the evaluation on sustainability is that ACCCRN’s legacy is likely to center primarily around increased awareness of UCCR and the enhanced capacity of key stakeholders. Relationships induced by ACCCRN among key individuals will sustain but formal networks may not. It is also not yet clear whether formal institutions (e.g. CCCO) and projects set up will sustain. Sustainability is further discussed below, around Arup’s four factors. 8.1 Knowledge and learning As shown above, UCCR capacity has increased at all levels of the Initiative, including the grantees and the Foundation. The greatest challenges to sustaining this is at the city level, due to both staff turnover and dis- continuities in funding for UCCR-related actions. Therefore, ACCCRN has correctly turned attention increasingly to national climate change actors and forums as a means to sustain capacity, by feeding it in from the top. In Indonesia, ACCCRN is supporting the high-level ICA, and in Vietnam, VIAP has
- 84. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE62 developed curricula and is training urban planners in mainstreaming resilience within the urban planning processes. This will help create a sustainable (though small) cascade of capacity building from the center. In Thailand, where the TEI approach involves working closely with academics, TEI is positing providing support to research and teaching of UCCR topics at various universities, in order to enhance the body of knowledge and also provide informed graduates entering professional careers. Thailand’s universities have an official mandate to provide technical support and advice to cities. ACCCRN support is also enabling the development of academic collaborative networks nationally, and internationally with IIED and ISET. One challenge to sustaining capacity is the difficulty ACCCRN has had in finding and working with organizations, particularly within the region, that have the knowledge, skills and expertise to carry the Initiative forward. The two major grantees that have driven the thinking and delivery approaches for the Initiative are international – Arup and ISET. UCCR was new for everyone in 2008, and these organizations have grown and refined their expertise over the course of ACCCRN. But once ACCCRN concludes, there may again be a capacity deficit. In relation to sustainability, ACCCRN should have considered more regional grantees, such as Asia Institute of Technology (AIT), and could have built its capacity over the course of the Initiative. India Indonesia Vietnam Thailand Surat Indore Gorakphur Semarang Bandar Lampung CanTho QuyNhon DaNang HatYai ChangRai KNOWLEDGE Local experiences n o n n n Wider networks n n o o n n n n n Mechanisms for learning/ exchange n o n n n n n n STAKEHOLDERS City campions/entrepreneurs n n n n n n n n Engaged government leaders n n Academia, private sector, civil society n n n n n POLICIES/PLANS City wide & departmental goals/plans o o n n n n n o State policies/plans o o National policy/plans n n n n n n n n FINANCE City/municipal budgets n n n o o o n Donor finance n Private sector participation n o o o o o n , Table 7: Summary of current and proposed UCCR sustaining actions n Sustaining action in progress o Sustaining action in proposed. Source: Kernaghan and da Silva, 2014.
- 85. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 63 Lastly, on knowledge, the successes of ACCCRN will be undermined if there are attempts to reify the approach for the sake of legacy. The knowledge and the approach will continue to evolve and be taken up by others. The legacy is also contingent on keeping the ACCCRN knowledge repository available in on-line space. This is not mutually dependent on the networking function. The collection needs to be kept in as many libraries as possible and the un-gating fee should be paid for any gated materials, in order to widen the audience. 8.2 Stakeholders A key aspect of sustainability is whether the cadre of champions continues to promote UCCR and UCCR approaches. A number of activities are underway to do that, including linking to national organizations and training bodies. Sustainability should not be heavily predicated on the continued existence of an ACCCRN network. Where members of the cadre are in touch with each other and sharing ideas, it will assist sustaining UCCR, but it is not essential. The prospects for clusters of ACCCRN’s network of relationships being sustained upon completion of grant spending are most positive via membership of existing, mostly national, networks in countries such as Vietnam and Indonesia. These granteesandtheirnetworksareprogressivelyintegrating a climate change mitigation and adaptation lens into a broader urban development framework. The extent to which the concept of climate change resilience persists as an organizing principle of their work depends to some extent on the priorities of their next donor relationship. Does the ACCCRN network need to be resilient and sustain indefinitely into the future? Research into networks established by IDRC suggest that donor-sup- ported networks can be useful and valid time-bound channels for dissemination of knowledge, but only for as long as: • the knowledge remains current and useful • network members continue to see value in relating with each other and being part of the network • required resources remain available. The success and sustainability of the ACCCRN network depends on the willingness of its members to participate. Neither the MTE nor this evaluation have found evidence that ACCCRN partners are seeking to become more regionally networked. In Vietnam and Indonesia, there are emerging examples of interdependent and cooperative styles of working among a few ACCCRN-supported organizations, independentofACCCRNsupport.Buttheheterogeneity of country actors coupled with the external contexts in India and Thailand means these countries are less well placed for nurturing collaborative organizational relationships for knowledge sharing, advocacy, and building communities of practitioners. 8.3 Policies and plans The development process needs to be the main vehicle for addressing climate change. Hence building UCCR needs to be a mainstream activity, and mainstreaming needs to be centered around development planning (Roman et al., 2012). The extent to which this is the case appears variable. In some cities, such as Surat, there is little connection. In others, such as Bandar Lampung, the CRS informed the approach to adaptation in the Mid-term Regional Development Plan 2010-2015 of Bandar Lampung. Field research suggests that this was partly because: “all the key selected consultants [for the Mid-Term Plan] have been part of the City Team” and, where the consultants were different (such as for the Bandar Lampung’s Spatial Planning (RTRW) Document for 2010-2030), “the VA and the City Resilience Strategy are not” (Lassa and Nugraha, 2014). Uptake of the CRS into formal planning in Bandar Lampung is also attributed to defining UCCR as a planning and financial development issue, not only an environmental one, as well as having the CRS translated into “development language” through its incorporation into the local development plans by the Local Planning Development Board, BAPPEDA.
- 86. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE64 More generally, CRSs have not been integrated into formal planning, even if they are still in use. The CRSs provide a very valid bottom-up, multi-stakeholder perspective on UCCR, but they need to link to, and better integrate with, formal top-down city plans, to which financing is attached. If the CRSs remain peripheral to mainstream planning, then they are much more likely to become moribund examples of an unsustainable donor-driven process. 8.4 Finance UCCR is a complex multi-piece jigsaw with only a few pieces as yet on the board. The ACCCRN cities need further funds to add more resilience-building actions to the puzzle and thus to improve ACCCRN’s UCCR legacy in those cities. Funds may come from many sources – the private sector, municipal bonds, development funds from central government, foundations, and bilateral aid grants, or loans from multilateral development banks, and other sources of innovative climate finance. To obtain funds from any of these sources, cities will require an up to date CRS (preferably linked to the medium-term city plan), investable UCCR projects and an awareness of how to go about securing the funding they require. To improve the sustainability of results from ACCCRN and better secure its legacy, cities need support in putting these in place. As a funding source, the UCCR Trust Fund (UCCRTF), which aims to work in 25 cities across six countries in the region, presents perhaps the most potential to help cities realize their UCCR ambitions. While cities should be selected by ADB country office staff, ACCCRN is keen that ACCCRN cities are included. Of the 12 cities selected for a fast start, none are ACCCRN cities. However, the CDIA is working with some ACCCRN cities to accelerate progress and/or link them to others funders, such as KfW Bankgruppe (the German Development Bank). The extent to which ACCCRN cities access UCCRTF funds will have an implication for the ACCCRN legacy in those cities, though some constraints are recognized here, since these are secondary cities which are often limited in their fiscal mandates by national policy. CDIA is a mechanism for early engagement with cities to help with pre-feasibility assessments of projects. Its assessment of ACCCRN cities revealed a low level of project readiness for investment, as only four cities were ready. This is an important finding. ACCCRN cities, having gone through the processes culminating in a CRS, were not deemed to have reached a readiness for investment. This points to a deficiency in either the nature of the CRS as UCCR “investment catalogues”, or in the post-CRS work to develop UCCR project designs. ACCCRN needs to pay attention to these areas to support sustainable UCCR results, and CDIA will be reviewing city resilience planning processes to assess how they may better generate more bankable projects. The UCCRTF is structured around three components: planning, investment, and knowledge and learning. While some cities will be ready to commence with investment, many of those where ACCCRN may be best placed to work in concert with ADB will need help to get ready through planning for investment in line with the ADB-based project cycle and its Urban Operational Plan – i.e. to have UCCR project opportunities ready for funding at the right stage of the cycle. This role has a wider potential, since the 2014 Bellagio meeting with donors identified the shortage of good proposals for UCCR projects as a key gap – donors have funds to spend, cities have a need, but there is a proposal deficit. ACCCRN cities need further funds to add more resilience-building actions to the puzzle and thus to improve ACCCRN’s UCCR legacy in those cities.
- 87. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 65 9 Conclusions The evaluation finds ACCCRN to be a successful Initiative which has developed a relevant approach to UCCR, tested and adapted this approach in a range of cities, built capacities to strengthen UCCR, produced a good body of published documentation on ACCCRN, and leveraged new funding and actors for building UCCR. As might be expected in a large and complex enterprise such as this, not everything has gone as planned. The sustainability at city level has yet to be tested, it has proven difficult to establish a network of UCCR practitioners, leveraging a fully collegiate way of working and realizing the potential comple- mentarity of grants has been sub-optimal, and it has taken longer than anticipated to mesh together The Rockefeller Foundation, DFID and ADB ways of working to operationalize the UCCRTF. At the time of the evaluation ACCCRN has a further period of granting and subsequent implementation to complete. Our assessment of progress against the results framework is therefore summative at the time of the evaluation. 9.1 Outcomes Outcome 1 - Capacity The capacity of city partners to plan, finance, coordinate, and implement climate change resilience strategies has improved in all cities. ACCCRN’s multi-stakeholder approach and its iterative, learning-by-doing modality have created new, more resilient relationships among city actors, and in most cities, improved the ownership of and commitment to building UCCR. The model of entering cities through a city climate change working group (of various modalities) was a pragmatic starting point. In some cities a sound reading of the local political economy has led to institutionalization of working group activities into municipal government operations. In others, the working group remains peripheral to municipal governance. In general, capacities have been better in technical areas than in social engagement and governance. Capacities have tended to be built in individuals rather than in whole institutions. However, the capacity of all the grant recipient organizations has improved through their involvement in ACCCRN. The city working groups have acquired new skills around, for example, vulnerability analysis and development of resilience strategies, enabling them to better cope with the risks implied by climate change. However, it is less evident that capacities to deal with the uncertainty aspects of climate change have been emphasized. In different countries, climate change is viewed through adaption, mitigation, and response lenses, which dilute the resilience perspective. City partners have demonstrated their capacities through implementing a range of city
- 88. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE66 projects. In general these have been technically sound. What remains is for the working groups to engage in some deep end-of-cycle reviews on the extent to which they have contributed to resilience. This review is critical for their capacity building and the wider embedding of the experience. Outcome 2 – Knowledge, learning, collaboration Individualandsharedlearning,andpracticalknowledge to build urban climate change resilience have been built. The theory of change and results framework assumed that this knowledge and learning would deepen the quality of awareness, engagement, and application by ACCCRN cities and other stakeholders, i.e. that the capacity should be put into action. As noted above, the learning and capacity building has been more individual than organizational. However, there are some broader increases in appreciation of UCCR – for example, where systems-wide UCCR project such as early warning and surveillance systems have been implemented, and where UCCR ideas have been taken up into planning and policy, as has been seen in Vietnam and Indonesia. ACCCRN and its partners have invested strongly in documenting the experiences and lessons. The IIED research and publication grant has been effective in helping ACCCRN to achieve a presence in the academic space around urban climate change, particularly around the operationalization of supporting cities in preparing for climate change. There is now a significant body of documentation on ACCCRN, and this has been shared globally online and at conferences. There are some obstacles to regional sharing due to language barriers, and nationally due to the technical level of the writing. ACCCRN will need to ensure that the material continues to be available after 2016, but the Foundation should also recognize that UCCR is a new and emerging field, along with resilience generally. ACCCRN is contributing to this space, and should not expect to reify UCCR around the ACCCRN experience. Outcome 3 – Expansion, networking, scaling up The creation of a network was previously an Outcome 2 aim, but this was revised in 2013 to be a networking aim under Outcome 3. The achievement of the network/networking objectives in ACCCRN has been challenging – the multi-faceted aims of a network combined with the diversity and geographic spread of potential members has meant the motivation and binding forces for such a network have been difficult to attain, with momentum mainly being maintained by ACCCRN funding. A new attempt at creating a network of country and regional networks is underway, but it is not certain that the mistakes to date have been sufficiently addressed in design – in particular the aim to have an ACCCRN network as part of the Initiative’s legacy. Where there is more prospect of success is in country-level networks that link to pre-existing groups and structures. Achievements on scaling, replication, and leverage have been much better. ACCCRN received a funding extensionaftertheMid-TermEvaluation,andpartofthis was granted to ICLEI to expand ACCCRN to 40 more cities (including in new countries), using a lighter-touch approach and leveraging their connections to city government. This expansion is showing progress on the ground. Similarly, Mercy Corp in Indonesia has extended the approach to six more cities, and ISET has levered new funds from USAID to implement an ACCCRN-based approach in four new cities in Vietnam and Thailand. A major result on leverage has been the establishment of the UCCRP with DFID, ADB, and USAID. At country level, there are examples of national commitment to UCCR-related issues and some new city governments wanting to invest in UCCR. However, overall, the influence ACCCRN has achieved at the national level has been variable, related to the extent to which climate change has been given priority in policy-making circles and the strategy ACCCRN has taken in engaging in this space. Progress has been
- 89. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 67 slower in Thailand and India, and in general, ACCCRN was not as well tuned into engaging in policy spaces in discovery and early implementation. There has more recently been some gap-filling grant-making to address this, but particularly in India, city-level work is disconnected from the national-level. Outcome 4 – Organizational excellence, management, accountability and learning Outcome 4 focused on The Rockefeller Foundation’s ACCCRN Team and its operation, relevance, accountability, and contribution to wider Foundation learning. The evaluation found that the Foundation’s ACCCRN Team structure has been relevant, and allowed the Foundation to gain in-depth knowledge of UCCR and how it gets put into practice. This placed the team in a strong position to lever other actors. The Mid-Term Evaluation identified concerns with the bilateral management of grantees. The SAG responded to this by creating opportunities for sharing and reflection, though more at a tactical level. There has been less country-level reflection between partners, notably in India, though new forums, for example in Indonesia, have stimulated sharing. Overall, the evaluation considers that the potential complemen- tarity and synergies of the mix of grantees that were brought together have not been realized. 9.2 Impacts Immediate impact - a diverse range of UCCR approaches, processes, and practices ACCCRN has generated a variety of models (i.e. processes, approaches and practices) around a core ACCCRN approach. These have been tailored by grantees and host cities according to the expertise and orientation of the facilitating grantee, and in response to the national and city context. With at least seven emerging “models,” ACCCRN’s objective to develop and test a “diverse range of effective approaches, processes, and practices” is fully met. As summarized under Outcome 3, there have been a number of examples of scaling and replication, so this aim is also met. The development of the CRSs incorporated the priorities of poor and vulnerable communities through vulnerability analysis. However, in some cities, the CRS subject matter was considered “too technical” and poor and vulnerable groups were not involved or able to directly voice their views. Having used vulnerability analysis for the CRSs, it is less clear how the unfolding plans and models being implemented address the concerns of poor and vulnerable communities, and how the effect of UCCR projects on them is monitored. Ultimate impact - resilience and capacity of a growing number of Asian cities is enhanced, and through this work the lives of poor and vulnerable (men and women) are improved The summative assessment of impact is that a growing number of Asian cities have improved UCCR. This is not to say they are substantially more resilient, but they have a number of smaller pieces in place in the larger UCCR jigsaw. Quantifying the improvement in UCCR is difficult, not least because ACCCRN and its cities have not adopted a measurement framework for this. It is also not certain yet how much the UCCR improvement under ACCCRN will sustain. Recommended activities during consolidation in relation to mainstreaming CRSs into cities’ master planning and thence getting the priority projects into budget forecasts and finance raising activities will be critical to this. The ultimate impact anticipates improvements in the lives of poor and vulnerable men and women. In the absence of any clear qualitative or quantitative baseline data or post-project data on people’s lives in the cities, it has not been possible to assess the extent to which the lives of poor and vulnerable people have improved.
- 90. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE68 ACCCRN needs to consider whether it wishes to invest in data collection over the next 18 months to facilitate a study that can answer this question directly. 9.3 Successes The evaluation identified a number of particular ACCCRN successes. Crystallizing UCCR – a relevant, operational approach The Rockefeller Foundation was a pioneer in setting up ACCCRN as an initiative to understand and build UCCR. The development of a multi-stakeholder-based process of studies (including vulnerability analysis) and collective reflection on these, leading to the production of a city resilience strategy (CRS) is seen as an appropriate mechanism to help cities appreciate UCCR and how to build it. In this novel field, working with an emergent strategy and following a learning-by-doing approach was entirely appropriate. The resultant learning on how to operationalize city-level intervention on UCCR is valued by a range of stakeholders. A body of UCCR literature ACCCRN has created a large body of published research and documentation of the ACCCRN process and experience. Most of this is empirical and related to the operationalization of UCCR, rather than the theoretical aspects of resilience. Knowledge has been one of the major outcomes from ACCCRN. UCCR Partnership ACCCRN has been able to leverage funds and contribution in kind at international, national, subnational, and city levels. The formalization of the Urban Climate Change Resilience Partnership (UCCRP) and its attendant Trust Fund represents a major success for ACCCRN. Managed by the Asian Development Bank (ADB), implementation guidance is jointly developed with ACCCRN inputs. UCCRP is funded jointly by DFID ($140 million) and USAID ($5 million). Reputational legacy ACCCRN collectively – both its grantees and participating cities – emerges with a strong reputation for innovation in the field of urban climate change resilience. Furthermore, The Rockefeller Foundation emerges with good reputational capital for having been an early, committed, and professionally engaged funder in this important and emerging field. 9.4 Challenges The evaluation also found some aspects of ACCCRN that could have been better. Networking As ACCCRN has progressed, the conception of the network in ACCCRN has become increasingly complex – a “catch-all” for ACCCRN’s proposed functions. While theinterdependentlinksinproposednetworkobjectives are conceptually coherent, the heterogeneous target membership (including individuals and organizations), geographicscope,languageandculturaldiversities,and expectations of multiple functions all mitigate against attracting active membership engagement. The value proposition may remain positive for as long as donor resources provide opportunities for capacity building and related travel to conferences and events, but seem ultimately hard to sustain. The emergence of more active national networks, linked to existing groupings that have local identities and purposes, appears to be more realistic than a multi-faceted ACCCRN - oriented network. Collaborative working Once the cities were chosen, the mix of grants and grantees was a broadly opportunistic approach. From the perspective of an action-research experiment, the spread of grants was generally appropriate. However, major granting has been exclusively to single grantees, with little incentive to collaborate. Thus,
- 91. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 69 although collaborative efforts did occur, the situation may be characterized as generally collegial rather than collaborative. While time and resources were committed to shared learning, for example through SAG meetings, more attention should have been invested in developing more conscious and shared reflection on how the program was unfolding in relation to the theory of change and program objectives, and making course corrections as deemed necessary. A key area where this might have made a difference is the attention paid to policy traction once it was evident that a theory of change predicated on knowledge and capacity did not reflect the de facto change pathway, and alone would not sufficiently influence policy. Closing the experimental loop Experimentation in resilience building in a young field wasappropriate.Whenitbecameapparent,forexample, that cross-learning (city-to-city or more widely) was going to be challenging and only likely to occur in a few specific instances, research could have been done to identify the necessary and sufficient conditions that enable cities to learn from other cities, and to make an assessment of the relative value and cost-benefit of such learning. The opportunity to close the experimental loop has so far been missed. Individual city experiences have been written up, and there has been analysis within and between countries, but not across distinct models (e.g. the seven practical ACCCRN models identified by the evaluation). These can be seen as attempts to derive generalized answers to the question “what works?” However, given the diversity of context and partner approach, the missing analysis is to answer the question “what works where, and why?” Closing the CRS cycle CRSs are interesting, largely bottom-up plans designed to identify particular vulnerabilities to climate change and prioritize a set of city actions to address them, i.e. they were planning and resource allocation exercises, with the intended consequence of focusing city efforts on building resilience. However, they are not well linked into cities’ formal medium-term development plans, master plans, spatial plans, regulatory frameworks, or finance mechanisms. Therefore, the way cities think about UCCR has not become institutionalized and a key intended consequence of the CRSs has not eventuated; ACCCRN did not focus energies on institutionalizing CRSs in this way. The CRS is being used as a framework for the city projects rather than mainstreamed as part of an integrated and systemic preparation for dealing with climate change. The longer-term consequence of the CRS not being mainstreamed is that it is side-lined (as has happened in some cities) or even forgotten. The process of creating the CRSs has been as important as the documents themselves, if not more so. It has brought stakeholders together with a common purpose around producing a strategy document. As a secondary consequence, this catalytic effect has added momentum to the working groups/city teams. This is an important finding on the “soft” side of what is needed for UCCR, but it does not diminish the unrealized need to institutionalize UCCR into formal city planning and budgeting. While the ACCCRN approach encompasses some elementsrequiredtobuildUCCR,thesearenotsufficient. The elements that are present include community involvement, multiple stakeholders, and iterative processes. However, the elements not yet sufficiently addressed include the link to formal city development planning, use of political economy analysis, and the ability to leverage project finance over and above ACCCRN’s own city project funding. ACCCRN needs to explore ways to integrate bottom-up engagement and contextual implementation with formal planning and policy systems, analysis and financing. Are we resilient yet? The particular designation “Urban Climate Change Resilience” (UCCR) has not gained much traction outside the ACCCRN circle. However, resilience as a concept has gained much currency since ACCCRN started. Climate change discussions, such as the IPCC AR5, use the term “resilience” freely, and the idea of
- 92. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE70 “resilient cities” is well accepted. The multi-dimension- ality of UCCR makes it a difficult concept to articulate, and it is hard for city stakeholders to know whether their city is indeed becoming more resilient. ACCCRN has working models and definitions of UCCR, but the extent to which resilience is being built in cities is very subjective, often limited to the emergent experience from city projects. Care is needed to ensure that cities are not led to believe that any projects encompassing city greening, environmental improvement, mitigation, climate proofing, environmental education, DRR, or adaptation will automatically lead to greater resilience. There is a need for a feedback loop that indicates, “if we do these things, it will have these effects on UCCR.” In order to do this, two things are missing: a more routine approach to city-led M&E (of the outcomes of UCCR actions), and an assessment framework, against which cities can judge whether they are becoming more resilient. Capable cities The ACCCRN approach entails working through climate change working groups. These have been successful in some cities, and less so in others. Although various initiatives are broadening the cadre of UCCR champions, the working groups remain the locus of UCCR capacity in the cities. These groups face two challenges: i) sustaining themselves once funds expire, not because they are expensive to run, but because many have become oriented towards city projects only, and ii) acting as the resilience focal point for cities. UCCR is complex, multi-scalar, and ultimately an aggregation of many small resilience-building actions. Individual projects may not demonstrate resilience given its whole-system nature. There is an oversight function at the systems level that entails appraising whether projects will indeed contribute to resilience, and involves working on UCCR measurement at project, system and city level. It is not clear how these functions will be dealt with in the future. Policy traction IntheACCCRNcities,thepromiseoffinancialassistance from The Rockefeller Foundation through ACCCRN city grants and the technical assistance from the ACCCRN partners were key enablers that facilitated or ensured the cities’ initial buy-in. From that stage, it was considered (in the theory of change) that knowledge, empirical evidence and hands-on capacity building would achieve commitment. This overlooked the political economic realities of cities. It also overlooked the importance of the national policy context to the motivations of cities to engage. More supportive policy environments (as in Vietnam) have stimulated better uptake of UCCR than less-well oriented environments (as in India). The weaker awareness of the policy and governance contexts for UCCR in the earlier stages of the Initiative hindered progress.
- 93. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 71 10 Lessons and Recommendations Based on the learning purpose of the evaluation TORs, this chapter draws on the evaluation findings particularly related to the outcomes and impact of ACCCRN to identify pertinent lessons for ACCCRN and The Rockefeller Foundation and to make rec- ommendations that address the consolidation and sustainability of ACCCRN’s achievement, the wider field of UCCR, and The Rockefeller Foundation’s focus on resilience and its own operational modality. 10.1 Lessons for ACCCRN Having developed and tested an approach to UCCR, the important lessons for ACCCRN at this stage are about how to consolidate the existing practice and ensure it is sustainably replicated and scaled-up. What will make UCCR praxis cohere for a whole range of stakeholders? Three main factors (Sumner et al., 2009) are considered important for achieving this: messaging (developing narratives that are relevant, memorable, and credible for different audiences), networking, and strategic opportunism (identifying windows of opportunity for impact and influence). These factors map onto ACCCRN’s Outcomes 2 and 3 with the following lessons. • Messaging. Resilience is a difficult concept, and UCCR more so. ACCCRN messaging is undoubtedly credible, but much of it has prioritized city projects. Also, much has been presented in lengthy documents, targeted at more academic audiences, which practitioners have not found accessible. The use of diagrams and schematics has helped improve accessibility and thus memorability. But, while UCCR is, at a meta-level, instinctively a “good thing,” neither UCCR nor the ACCCRN approach has been effectively communicated. The narratives around city projects are better, as they are generally case studies, and thus more naturally stories. But, there is a tension between communicating on the SLD-CRS (a fairly tightly managed process, with an obvious dénouement) and on city projects (a messier situation, and in some cases, with a less distinct end point). The lesson thus is around consideration of messaging to non-technical practitioners and policy audiences, crafting messages tailored to different audiences, and messaging in appropriate languages and cultural styles. • Networking. ACCCRN has given much attention to networking, although not always successfully. The lesson is to pay attention to the reasons
- 94. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE72 people want to get connected and how this happens, focusing on the types of people who drive expansion – connectors, experts, and salesmen (Gladwell, 2009). Good analysis is needed as to who may potentially be the universe of networkable people and organizations, why they may want to interact, and who can play what role in ensuring peoples’ needs are met. Networking should also pay attention to some of the tenets of social capital thinking (Putnam, 2000) – does networking aim to bond to similar actors in similar situations and create reciprocity, or to bridge to dissimilar actors in different situations, often in order to access resources and information at greater scale, or connect to more influential actors? The value proposition of the to-be-refreshed ACCCRN network is not yet clear in terms of opportunities and resources it may provide to practitioners. The evidence is that peer-to-peer country networks are emerging as the most functional model for ACCCRN. Small networks can have subcritical masses, thus consideration also needs to be given to the people who will promote and scale the UCCR message. • Strategic opportunism. ACCCRN has been strategic more than opportunistic and, in some places, it has been both. ACCCRN has been strategic at creating its own opportunities, convening donors at Bellagio twice, forming a partnership with ADB, DFID, and USAID, and hosting and presenting at conferences. However, overall it has not been close enough to national and global policy processes to identify the strategic opportunities for promoting the UCCR agenda as they arise. There are some exceptions at country level but having grantees close enough to these processes to seize opportunities was not integral to the core ACCCRN design. The lesson is to use political economy analysis (see Section 2.3) at every level during the early stages of engagement, and work with grantees who have the capacity to use this analysis to inform strategic opportunism. 10.2 Lessons for the Rockefeller Foundation What are the lessons and observations from this evaluation that are relevant and generalizable to the Foundation? ACCCRN helped shape and achieved results in a new and complex area. The use of emergent strategy and experimentation were appropriate management approaches in this environment. Progress was consequently modest, at least in the first half in the Initiative, but this was time well invested. The lesson for the Foundation is that experimentation requires flexibility in terms of time, staff, and financial resources, with the available time related to the complexity and novelty of the field. ACCCRN developed a generic approach, but issued parallel grants to a number of organizations to run with the approach and consciously adapt it to context. It has resulted in at least seven “ACCCRN models” that are currently in execution. This diversity was envisaged in the ACCCRN design, and contextually adapted models are more likely to be accepted and endure. One-size- fits-all approaches historically fail. A key aspect of tailoring to local context is understanding and relating to the local and national political economy. The Foundation has periodically raised concerns about Foundation staffing levels in ACCCRN and the role they play. The MTE found that staffing levels were broadly in line with other initiatives. This evaluation has found that the proximity of Foundation staff to the Initiative has given it a very sound and experiential appreciation of the issue and its solutions, and thence the credibility to be convincing when using the Foundation’s convening power to influence the uptake of ACCCRN ideas and approaches. The phasing of grant types that resulted from ACCCRN’s emergent strategy has worked well. This
- 95. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 73 has broadly included: initial exploratory and shaping grants to develop an approach; major grants to country coordinators to engage and catalyze action in cities; grants to country coordinators for sub-granting for the implementation of city projects; and then a range of gap-filling, including uptake and scaling grants to target actors, to increase the amount and spread of documents on ACCCRN, and to undertake training on UCCR. In retrospect, grants on M&E, documentation, and policy targeting should have started earlier in the Initiative. ACCCRN gave grants to individual organizations, and then invested much time and energy in trying to get the grantees to work together. This is paradoxical. It resulted in collegiality, but not collaboration. Grantees were not fully aware of the objectives of each other’s grants,despitebeingencouragedtoworktogether.This created suspicion, duplication, and other inefficiencies. Initiatives should consider collaborative grants and improved levels of transparency about grants within an initiative. The UCCR Partnership and Trust Fund with the ADB, DFID and USAID is an important result of ACCCRN that demonstrates the credibility of the UCCR concept, and of the Foundation’s convening and influencing power. Operationalizing the idea has been difficult. Developing joint guidance on operating the trust fund, and achieving agreements from four funders with quite different policy approaches, financing models, and operational requirements has been time-consuming, both in understanding the differences and then reaching a consensus. ACCCRN cities reported being confused by The Rockefeller Foundation having two different streams of urban resilience funding. They see ACCCRN as being process-led, inclusive, and adaptive, and 100RC as starting with some elements that may not fit with the current approach to urban resilience. A number of stakeholders identified the contrast between ACCCRN’s iterative and interactive process running its course over months, or a year or more, and 100RC aiming to cover some of the same ground in a one-day workshop. Some city stakeholders had interpreted 100RC as “ACCCRN 2.0”–anopportunitytoaccessafurtherstreamoffunding to continue city projects. For them, these projects have been a major component of ACCCRN, and given their limited development budgets, an important funding source. They would certainly like more funding for city resilience projects. The lesson is that there is a need for a clear narrative, shared by ACCCRN and 100RC of the differences and complementarities between the two initiatives. It is also important to develop a narrative for the ACCCRN cities that do not qualify for 100RC funds, so that this is not seen as “failure.” Countries and cities have bought into ACCCRN and thus into UCCR at different levels. Climate change and disaster risk reduction have been major hooks. For 100RC, climate change is less of a hook and therefore will need to carefully consider how “urban resilience” will be communicated to city stakeholders. Since the concept is explicitly broader than climate change, what are the national and city policy priorities with which 100RC can align to get that buy-in? A lesson from ACCCRN is to be pragmatic about how cities take on board the core UCCR concept, accepting that climate adaptation is a foundation from which climate resilience can grow. A final observation for the Foundation – ACCCRN is gender neutral. It does not appear to have lived up to the Foundation’s vision of, and support for, gender equality through the way the Initiative has been programed and developed. 10.3 Recommendations This is a summative evaluation of ACCCRN undertaken as it enters its final consolidation phase, so the scope for further granting and other significant change is limited. Detailed recommendations were made through the Mid-Term Evaluation in 2011 when some course correction was possible. At this stage, the main objective is to bring the Initiative to conclusion with a significant and lasting impact.
- 96. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE74 The TORs invited recommendations covering: • nurturing and sustaining ACCCRN’s achievements • key considerations for ACCCRN’s consolidation phase • implications of the achievements and challenges of the Initiative for UCCR partners and the field of UCCR • implications of the achievements and challenges of the Initiative for the work of the Foundation on resilience, and The Rockefeller Foundation model of operation. The first eight recommendations address consolidation, sustainability and the application of learning from ACCCRN to UCCR partners and the field of UCCR. The remaining five recommendations address The Rockefeller Foundation’s wider focus and work on resilience, and Rockefeller Foundation’s model of operation. CONSOLIDATION, SUSTAINABILITY, AND THE APPLICATION OF THE LEARNING FROM ACCCRN The Rockefeller Foundation’s ACCCRN team has thoughtcarefullyabouthowtoworktowardsACCCRN’s conclusion and its priorities for the consolidation phase. The evaluation concurs with a number of these priorities28 but is concerned that the emphasis on a renewed push for an ACCCRN Network will not meet the needs of consolidation and sustainability. A more tangible and specific set of actions is required to maximize progress in the closing period of implementation and ensure the future potential. The first recommendation focuses on this need and the role the team can play. Human capital development and knowledge resources 1. The Rockefeller Foundation’s ACCCRN team should coordinate the development of an action plan that ensures human capital development on UCCR is maximized during consolidation. • It is essential that ACCCRN take a human capital perspective during consolidation to ensure a 28 ACCCRN Team Retreat Communique, November 2013 coherent set of actions resulting in a UCCR knowledge and capability legacy. These actions should include the following. • Working with all grantees to ensure they have explicit plans for how they can best lever the capacity they have developed in UCCR during ACCCRN to take forward the knowledge and skills, and act as UCCR ambassadors. • Ensuring that all grantees involved in training, at country, regional, and international levels, institutionalize the UCCR courses they are running into the routine curricula of their orga- nizations. Curricula should be fully developed and available for future use. Ideally all training materials should be open access. • Supporting the country-level UCCR networks that are emerging as effective ways of devel- oping human capital through knowledge shar- ing as a cost-effective way to support The Rockefeller Foundation’s legacy in the UCCR space. 2. The Rockefeller Foundation’s ACCCRN team should work with the publicons, networking, and communication grantees to ensure that ACCCRN’s documentary knowledge resources are made as accessible as possible, and that future access is secured with an agreed long term location. • These resources29 should all be the public do- main, and in easily accessible repositories that are likely to have indefinite funding. • The navigability of the ACCCRN website should be improved, the extent of published resources on the site increased, and the search function upgraded. • To ensure continued online accessibility, the ACCCRN document repository should be mirrored or cached with a number of other relevant urban and climate change portals or repositories. 29 Papers, case studies, CRSs, vulnerability analyses, project reports, tools and toolboxes
- 97. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 75 • There is a consistent view that ACCCRN mate- rials are too academic for the broad audience. A focus for consolidation should be to ensure ACCCRN materials are packaged in a way that the practical user-audience30 will not find “too complex and Northern.” This should include production of material in national languages, culturally appropriate styles, and practitioner usable forms. Undertake resilience reviews 3. Country coordinators should facilitate the working groups in each of the original 10 ACCCRN cities to undertake a resilience review. The Rockefeller Foundation’s ACCCRN Team should track progress of these reviews and identify a means to collate results. These resilience reviews would: • assess the extent to which vulnerabilities iden- tified in the CRS have been addressed, • assess changes in UCCR in the city according to the parameters used in the CRS, • assess the extent to which city projects con- tributed to these changes in UCCR • pilot the use of the City Resilience Index (CRI) metrics as a means to get a comparable as- 30 By-and-large these are not IIED’s research partners or the focus of ISET’s publications sessment of cities’ resilience, and as a baseline for the future • revise the CRS accordingly. To nurture and sustain ACCCRN’s achievements, there is a need to undertake some consolidating actions related to the CRSs and city projects to ensure that the investments to date continue to play a key role in building UCCR. CRSs were intended to be living documents. However, they were generally used instrumentally to identify city projects for ACCCRN funding, and with only a few exceptions, have not been revisited, reviewed, or revised. Similarly, the city projects have not been comprehensively reviewed. Also, there has been no systematic attempt to assess the extent to which cities have become more resilient as a result of ACCCRN’s intervention, if at all. This recommendation will close the “plan (CRS) – do (city project) – review” loop in the 10 cities (Figure 9). This is not a recommendation for “more studies.” The Initiative and the cities need to systematically know what has changed as a result of the city projects and the CRS. The CRI needs to be tested and feedback obtained from potential users – including an assessment of their capacity to use this framework for planning, monitoring, and decision-making. This could all be done by grantees, but that would not itself be FIGURE 9: Closing the CRS loop with a resilience review Form UCCR working group Vulnerability assessment Shared Learning Dialogue City resilience strategy ACCCRN City projects City resilience strategy Source: Verulam Associates, 2014.
- 98. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE76 resilience-building behavior – the intention here is that the Resilience Reviews are done with and by the cities, as part of the learning dimension of UCCR. Resilient planning and finance 4. ACCCRN partners should work closely with the city climate change working groups/city teams and the respective city governments to link or integrate the CRS with formal planning and budgeting processes, and thence, prioritize a further round of UCCR-building projects. • ACCCRN processes, particularly the CRS planning processes, are not well mainstreamed into city planning, budgeting, and finance processes. In these situations, the CRS will have been a one-off donor-funded process that has built some hands-on capacity and funded a small number of projects. To achieve sustainability and the prospect of resources continuing to be channeled to UCCR-building actions in the ACCCRN cities, the CRSs need to link to formal city planning and finance systems (Figure 10). Finance can thus be accessed in an on-going manner for cities’ city projects, as opposed to time-bound ACCCRN city projects. Close-out key research questions 5. The Rockefeller Foundation’s ACCCRN team should sponsor and facilitate an inclusive process to close-out the key research question of “what works where, and why,” through a con- solidated analysis of its natural experiment. • The diversity of the many country-city-partner combinations is a significant and distinctive value embedded within ACCCRN. To realize this, the ACCCRN partners need to conduct cross-sec- tional analysis that focuses on identifying and explaining the conditions for success. This is a key piece of learning that is currently missing from ACCCRN. There has been a missed opportunity to assess ACCCRN as a natural experiment and try to determine “what works, where and why” – this is the research question that needs to be closed out for partners to maximize the learning from the Initiative. • The aim of this analysis would not be to derive “the answer.” As indicated by a wicked problem/ complex systems analysis, there are likely to be many answers. One approach may be to produce a menu of options, rather than a single recipe, to FIGURE 10: Linking CRSs with overall planning and finance Form UCCR working group Vulnerability assessment Shared Learning Dialogue City resilience strategy ACCCRN City projects Resilience review Finance Integrate with city devmt plan ‘City’ city projects Source: Verulam Associates, 2014.
- 99. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 77 help build UCCR. This analysis would consider both first and second generation models, and reflect that new actors in new city contexts (such as 100RC) are likely to wish to move more quickly and more simply than ACCCRN has done during its experimentation. Thus consideration of what constitutes the irreducible core of the ACCCRN approach ought to be one analytical lens. Another ought therefore to be the extent to which process facilitation is a rate limiter, and how this can be amplified or multiplied. • This analysis would elucidate a political economy- informed narrative of what actually happened,31 to determine which contextual factors and ACCCRN mechanisms led to city processes working well or less well. It would also help consolidate ACCCRN by bringing the range of city experiences together at a time when the Initiative risks fragmentation. • Involving practitioners and policy actors in the analysis would improve its utilization. Involving all partners at city, country, and regional levels could lead to an unmanageable process, so it may need to be a tiered process. Bringing the major partners together for a final and self-critical analysis across all the interventions could provide strong closure for the Initiative, with very good analytical materials to leave as a legacy. With good facilitation, this could avoid partners’ approaches being competitive. There is a broad and firm appetite among grantees to collaborate on a final cross-sectional analysis of the drivers of success in building UCCR that integrates the views of the major grantees, such as Arup’s urban systems thinking, IIED’s urban poverty focus, and ISET’s eco-social transformation perspective. Up-scale the influencing 6. The Rockefeller Foundation should commit senior staff time from within and outside the ACCCRN team, and leverage its ACCCRN con- nections, to influence the content and outputs from forthcoming major global processes related to urban development and resilience, with HAB- ITAT-III as the primary focus. 31 This could usefully be informed by a realist approach (Pawson et al., 2004) • ACCCRN needs to go out with the best possible contributions on the international platform. While it may be argued this is beyond the expectations of ACCCRN’s design criteria, there is an opportunity here that ACCCRN should grasp to broaden its footprint. Few other urban climate change initiatives carry the weight of eight years’ work and $59 million investment into developing and opera- tionalizing an approach to UCCR. • There are four large international processes underway, which ACCCRN has the potential to influence as part of its legacy. • UNFCC COP21. Set in Paris in December 2015, this aims to set new long-term binding climate agreements. • Post-2015 Sustainable Development Goals (SDGs). There has been a strong campaign to include an SDG on cities32 . • The post-Hyogo Framework. A new Hyogo Framework for Action (HFA-II) will be con- sidered at the World Conference on Disaster Risk Reduction to be held in Sendai, Japan, in March 201533 . • HABITAT-III. The UN will hold the Third United Nations Conference on Housing and Sustainable Urban Development (Habitat III) in mid-2016. This meeting, one of the first global conferences after the SDG meet- ing, aims to generate a “New Urban Agenda for the 21st century.” It is the opportunity to chart new pathways in response to the chal- lenges of urbanization and the opportunities offered in implementing the SGDs. The ra- tionale for focusing on this process, which is “urban” rather than “climate change” is based on several factors: it is furthest in the future, 32 At the working group on SDGs in New York in July 2014, it was agreed to consider Goal 11 (of 17) on Sustainable Cities and Human Settlements – Make cities and human settlements inclusive, safe, resilient, sustaina- ble”– at the forthcoming UN General Assembly. While the SDG proposes sustainable rather than resilient cities, resilience is one of its aims. 33 This will be a global framework for disaster risk reduction (DRR) that aims to: integrate DRR into sustainable development; develop and strengthen institutions, mechanisms and capacities to build resilience to hazards; and systematically incorporate risk reduction approaches into emergency preparedness, response and recovery programs.
- 100. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE78 giving more time to plan, its urban focus is consistent with The Rockefeller Foundation’s “urban resilience” orientation, and bringing climate to an urban stage may offer opportu- nities that differ from those that occur when bringing urban to a climate stage. Nonethe- less, ACCCRN may need to achieve greater clarity of message to influence a more explic- itly urban audience. If ACCCRN is able to inform the content and outputs from these processes, then where national buy-in has been slow, the global agenda can have some downward pressure, not least if it influences the way climate finance is used. A specific coordinated strategic approach is required in each case combining the Foundation’s convening power and status and grantees’ learning and city experience. At a minimum, media and communication resources should be assembled around these opportunities. Communicating exit and supporting partners 7. The Rockefeller Foundation’s ACCCRN team should have a more explicit plan with clear phased activities to smooth the end of the Initia- tive, preparing and supporting partners through the transition. This should include the following. • Clear and early messaging to all partners – grantees, city partners, other donors – cov- ering timing, funding arrangements, links to other programs, the plans for the ACCCRN Network and country-level networks, and ac- cess to the knowledge legacy. • Support to country coordinators to work with the city working groups to develop their own consolidation plans.34 The current sense is that ACCCRN consolidation planning is pri- marily Rockefeller Foundation and grantee 34 These plans could be the focus for longer term funding scenarios and linking the CRS to formal systems. centric. Less consideration has been given to what ACCCRN consolidation looks like for the city partners, or how they may need to adapt to a post-ACCCRN world. Handing over the baton 8. The Rockefeller Foundation’s ACCCRN team should ensure that bridges to other sources of support and funding to cities are in place in good time to be effective post-ACCCRN. This includes: • as part of grantees’ exit plans, being clear about what support they will continue to ex- tend to cities, and on what basis • making links to other potential sources of Rockefeller Foundation funding, including the parameters for obtaining and using that fund- ing, particularly how the ACCCRN cities will relate to 100RC • making it clear to ACCCRN cities what other sources of donor and private funding are avail- able to help continue building UCCR, and how they might access this. ACCCRN will conclude in 2016, but it is certain that the cities will not be substantially resilient at that point. Continued improvements in UCCR will in large part depend on the cities mobilizing their own finance more effectively and attracting new funds. ACCCRN has put some of the pieces in place to support this transition, notably the UCCRP (if ACCCRN cities do indeed access it), the CDIA support for project proposal preparation, and the second Bellagio donor convening on funding UCCR. However, without the ACCCRN entity, cities will be more dependent on their own resources and initiative to link to the opportunities. RECOMMENDATIONS FOR THE WORK OF THE FOUNDATION These five recommendations derive from observations on ACCCRN, but have broader implications, and relate to the general Rockefeller Foundation model of operation.
- 101. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 79 Better guidance on inter-initiative knowledge management 9. The Rockefeller Foundation should develop guidance on a more systematic and structured approach to on-going initiatives capturing their lessons, and to new initiatives accessing the knowledge that accumulates in on-going initia- tives. • ACCCRN’s experience has been instrumental in giving The Rockefeller Foundation the confidence to create major new initiatives on resilience. This is particularly so for The Rockefeller Foundation’s flagship centennial initiative (100RC), but ACCCRN is also an important antecedent in the thinking behind the Global Resilience Partnership (GRP). The core idea of “resilience” and for 100RC, “urban resilience,” had traction at a high level in The Rockefeller Foundation, and was attractive as the central idea in the new initiative. The practical op- erationalization and the contextual nuances did not transmit as well as the macro-concept. • The extent to which details and knowledge from ACCCRN informed new initiatives in search and discovery phases appeared patchy and inconsistent. This evaluation found that there is no shortage of information on initiatives in The Rockefeller Foundation’s Atlas digital filing system. However, accessing knowledge from initiatives on the ground is a challenge. The ACCCRN team recently produced drafts of its ACCCRN Points of View lessons documents; these are exactly the type of distilled knowledge that is useful to newer initiatives. • It is envisaged that ongoing initiatives could be charged with annually updating internal lesson papers on: the conceptual field, operationaliz- ing the concept, and initiative management. New initiatives in adjacent themes would be expected to review these, and importantly, spend time with the implementing team and key grantees to develop an appreciation of taking the initiative concept from the page to the field. Shape grant architecture to facilitate innovation 10. The Rockefeller Foundation should i) review the default structure of grant portfolios for ini- tiatives in which high levels of innovation are expected, and ii) examine the level of transpar- ency of grant information. • The evaluation found that ACCCRN’s grant architecture was paradoxical. The Initiative wanted grantees to work together and share learning, but all its grants were exclusive bilateral relationships with single organizations. This presented obstacles to collaborative working. While the Foundation expects “innovation dissonance” – productive tensions between people working through their differences and finding common ground in the innovation process – this is more difficult to resolve where the dissonance occurs between grants with somewhat similar aims, an element of commercial competition for resources, and a lack of transparency over grant objectives. Some creative tension is an innovation driver, but this can be achieved in more productive environments, such as collaborative and consortium grants, which can reduce the extent of unproductive friction. Likewise, a more transparent platform for initiatives in which grantees have better sight of each other’s documentation would facilitate cooperation. Make search and discovery proportionate 11. The Rockefeller Foundation should consider a more flexible duration for the discovery phase of initiatives, with more time being extended to initiatives judged to be more novel and more complex. • ACCCRN has pursued innovation in a new and complex field through an appropriate combination of emergent strategy and experimentation. This has helped ACCCRN shape and simultaneously achieve results in the field of UCCR.
- 102. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE80 • By taking a simultaneous learning and doing approach, ACCCRN progress was consequently modest during its earlier stages, as it progressed through iterative learning. This slower pace early on was time well spent, as it allowed ACCCRN to develop a sound approach, but it is a pace that appears less compatible with the current tighter initiative timeframes. Prioritize gender equity in all initiatives 12. The Rockefeller Foundation should strengthen the attention it gives to gender outcomes and the role of women and girls as change agents in all its grants and initiatives. • The Rockefeller Foundation is strongly committed to gender equity. But it appears to be missing an opportunity to realize greater gains in this area. The Rockefeller Foundation views women as change agents, but mainly addresses gender equality through women-specific grants. From the evidence of ACCCRN, where there has been ample opportunity to take a more gendered perspective, gender is not mainstreamed through “non gender” initiatives, and the potential to link with gender- specific grants, such as the Women’s Leadership on Climate Justice Network, has not been fully realized. Don’t lose the social aspects of resilience 13. The Rockefeller Foundation should ensure that all its resilience initiatives sufficiently combine the social and technical aspects of resilience. • The Rockefeller Foundation knows this but it has been addressed variably in ACCCRN. Vulnerability analyses and social learning are positives, the level of engagement with citizens has been patchy, and city projects have not consistently considered the poor and vulnerable or aspects of social justice, particularly when a techno-centric approach is taken. Resilience approaches that are overly scientific (e.g. driven by climate science) or techno-centric (i.e. dominated by technocrats and project staff) will fix problems, but may be less successful at building resilience. For example, these approaches may provide DRR-type fixes – hard solutions – but face challenges in also developing soft solutions. Resilience building needs a well-rounded approach that integrates top-down technical expertise and a bottom-up social orientation.
- 103. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 81 References 100 Resilient Cities website: http://www.100resilientcities.org. ACCCRN website: http://www.acccrn.org. ACCCRN (2010). City Resilience Strategy. Semarang’s ad- aptation plan in responding to climate change. ACCCRN, with ISET and MercyCorps Indonesia. ACCCRN (2013). ACCCRN City Projects. ACCCRN, Bangkok. ADB (2013). Gender and urban poverty in South Asia. Proceedings report of the 2012 subregional workshop, Mandaluyong City, Philippines. Asian Development Bank, Manila. Allen, A. & Yap, C. (2014). Midterm Reflection on IIED’s Role in Asian Cities Climate Change Resilience Network (ACCCRN). Development Planning Unit, UCL, London. Andrews, M., Pritchett, L. & Woolcock, M. (2012). Escaping Capability Traps through Problem Driven Iterative Ad- aptation (PDIA). CID Working Paper No. 240. Center for International Development at Harvard University. Anguelovski, I. & Carmin, J. (2011). Something borrowed, ev- erything new: innovation and institutionalization in urban climate governance. Environmental Sustainability, 3:1–7. Anguelovski, I., Chu, E. & Carmin, J. (2014). Variations in ap- proaches to urban climate adaptation: Experiences and experimentation from the global South. Global Environ- mental Change, 27; 156–167. APSC (2007). Tackling Wicked Problems – A Public Policy Perspective. Contemporary Government Challenges series. Australian Public Service Commission, Australian Government, Canberra. Arup (2103). Practical responses to building urban climate change resilience: Focus on sustaining action – What have we learnt from ACCCRN so far? Lessons from the Asian Cities Climate Change Resilience Network. Report on ACCCRN Knowledge Forum. Arup, London Arup (2014a). Quick reference guide for successful project delivery; Guide for ACCCRN grantees. Arup; London, Bangkok, Sydney. Arup (2014b). City Resilience Framework. City Resilience Index. Arup, London. Arup (2014c). Understanding networks for cities and climate change. ACCCRN Network, Bangkok. Australian Government (n.d.). Australian Aid: Promoting Prosperity, Reducing Poverty, Enhancing Stability. More Effective Aid: Changing How We Deliver. Australian Gov- ernment, Canberra. Available at: http://Aid.Dfat.Gov.Au/ Aidpolicy/Developmentpolicy/Pages/More-Effective-Aid. Aspx. Bahadur, A., Ibrahim, M. & Tanner, T. (2010). The resilience renaissance? Unpacking of resilience for tackling climate change and disasters. Strengthening Climate Resilience Discussion Paper 1. IDS, Brighton. Barr, J., Albertyn, C., Hong, VXN., Kundu, A. & Zaituni, F. (2012). Midterm Evaluation – The Rockefeller Founda- tion Initiative: Asian Cities Climate Change Resilience Network. Verulam Associates Ltd, for the Rockefeller Foundation. Béné, C., Wood, R., Newsham, A. & Davies, M. (2012). Resil- ience: New Utopia or New Tyranny? Reflection about the Potentials and Limits of the Concept of Resilience in Relation to Vulnerability Reduction Programmes. IDS Working Paper 2012, Number 405; CSP Working Paper Number 006. IDS, Brighton. Béné, C., Newsham, A., Davies, M., Ulrichs, M. & Godfrey- Wood, R. (2014). Review Article: Resilience, Poverty and Development. Journal of International Development; 26, 598–623. Brodzinsky, S. (2014). At the World Urban Forum in Medellín, resilience takes centre stage. The Guardian, Monday 14 April 2014. Guardian Resilient Cities website (sponsored by the Rockefeller Foundation). Available at: http://www. theguardian.com/cities/2014/apr/14world-urban-forum- medellin-resilience Brooks, N., Anderson, S., Burton, I., Fisher, S., Rai, N. & Tellam, I. (2013) An operational framework for Tracking Ad- aptation and Measuring Development (TAMD). IIED Climate Change Working Paper No. 5. Brown, A., Dayal, A. & Rumbaitis del Rio, C. (2012). From prac- tice to theory: emerging lessons from Asia for building urban climate change resilience. To be published in Envi- ronment and Urbanization Vol 24, No 2, October 2012. Brown, V.A., Harris, J.A. & Russell, J.Y. (2010). Tackling wicked problems through the transdisciplinary imagination. Earthscan, London.
- 104. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE82 Bulkeley, H. & Castán Broto, V. (2013). “Government by ex- periment? Global cities and the governing of climate change” Transactions of the Institute of British Geogra- phers, 38; 361-375. Castán Broto, V. & Bulkeley, H. (2013). A survey of urban climate change experiments in 100 cities. Global Environ- mental Change, 23; 92-102. Collins, K. & Ison, R. (2009). Jumping off Arnstein’s ladder: social learning as a new policy paradigm for climate change adaptation. Environmental Policy & Governance; Vol.19, Issue 6, pp 358–373. Cook, S. (2012). The Rockefeller Foundation ICLEI ACCCRN Process. Arup, London. DFID (2011). DFID’s Approach to Value for Money (VfM). DFID. London. Available at: www.gov.uk/government/up- loads/system/uploads/attachment_data/file/67479/DFID- approach-value-money.pdf. Davoudi, S., Crawford, J. & Mehmood, A. (2009). Planning for climate change: Strategies for mitigation and adaptation for spatial planners. Earthscan/James & James, London. Disaster Risk Management Team (2012). Building Urban Re- silience: Principles, Tools and Practice. East Asia and Pacific Infrastructure Unit (EASIN). The World Bank, Washington DC and AusAID, Canberra. Elsevier website: http://www.journals.elsevier.com/urban- climate/most-cited-articles/ Friend, R. & Moench, M. (2013). What is the purpose of urban climate resilience? Implications for addressing poverty and vulnerability. Urban Climate; Volume 6, pp 98-113. Friend, R. & MacKlune, K. (2013). Climate Resilience Frame- work. Putting Resilience into Practice. ISET, Boulder, Colorado, USA. Gladwell, M. (2000). The Tipping Point: How Little Things Can Make a Big Difference. Abacus, London. Hackmann, H., Moser, S. & St. Clair, A. L.(2014). The social heart of global environmental change. Nature Climate Change; 4, 653–655. IIED. (2014). Capturing complexities of urban resilience. Re- flect and Act Series, IIED, London. International Aid Transparency Initiative (IATI) website: http://www.aidtransparency.net/. ISET (2013). International Climate Resilience Case Study: Can Tho, Vietnam City Climate Resilience Action Plan. ISET-Vietnam, Hanoi, and TERI. (2012). Mainstreaming Cli- mate Resilience in Urban Areas. A Case of Gorakhpur City. Synthesis Report. TERI, New Delhi. IPCC Working Group II (2014). Climate Change 2014: Im- pacts, Adaptation and Vulnerability, Contribution to AR5. Available at: http://www.ipcc.ch/report/ar5/wg2/. Ison, R. & Collins, K. (2008). Public policy that does the right thing rather than the wrong thing righter. In: Analysing Collaborative and Deliberative Forms of Governance, 14 Nov 2008, The Australia National University, Canberra, Australia. Kasper, G. & Marcoux, J. (2014). The Re-Emerging Art of Funding Innovation. Stanford Social Innovation Re- view; Spring 2014 Kernaghan, K. & da Silva, J. (2014). Initiating and sustaining action: Experiences building resilience to climate change in Asian cities. Urban Climate, Vol 7, pp 47–63. Khan, K. & Joseph, K. (2013). Embracing the Paradoxes of Innovation. Stanford Social Innovation Review, Summer 2013. Kolko, J. (2012). Wicked Problems: Problems Worth Solv- ing. Excerpt in: Stanford Social Innovation Review, Mar 6, 2012. Knowledge Centre on Cities and Climate Change (K4C) website: http://www.kcccc.info/. Lassa, J. & Nugraha, E. (2014). Building Urban Resilience to Climate Change in the Secondary Cities in Indonesia. Working Paper No. 9, Institute of Resource Governance and Social Change, Indonesia. Available at: http://www. irgsc.org/pubs/wp.html. Levin, K., Cashore, B., Bernstein, S. & Auld, G. (2012). Over- coming the tragedy of super-wicked problems: constrain- ing our future selves to ameliorate global climate change. Policy Sciences; 45, 123-152. Levy, C. (1996). The Process of Institutionalising Gender in Policy and Planning: The ‘Web’ of Institutionalisation. Working Paper No. 74. Development Planning Unit, Uni- versity College London. McKenna, C. & Morrison, A. (n.d.). Guide 3: Natural experi- ments. Social Science Methods Series, Scottish Govern- ment Social Research Group. Office of the Chief Re- searcher, Edinburgh. Available at: http://www.scotland.gov. uk/Resource/Doc/175356/0091395.pdf. McLoughlin, C. (2009). Topic Guide on Political Economy Analysis, GSDRC. Mendizabel, E. (2006a). Understanding Networks: The Func- tions of Research Policy Networks, ODI Working Paper 271. ODI, London. Mendizabel, E. (2006b). Building Effective Research Policy Networks: Linking Function and Form, ODI Working Pa- per 276. ODI, London.
- 105. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 83 Mitchell, T. & Harris, K. (2012). Resilience, a risk management approach. Background note. Overseas Development In- stitute, London. Moench, M., Tyler, S. & Lage, J. (eds.) (2011). Catalyzing Ur- ban Climate Resilience. Applying Resilience Concepts to Planning Practice in the ACCCRN Program (2009–2011). ISET, Boulder, Colorado. Natsios, A. (2010). The Clash of Counter-Bureaucracy and Development. Essay, Center for Global Development, New York.Nutley, S., Walter, I. & Davies, H. (2007). Using Evidence: How Research Can Inform Public Services. Policy Press, London. OECD DAC (2010), Evaluating Development Co-Operation - Summary of Key Norms and Standards (2nd ed.). Net- work on Development Evaluation, OECD Development Assistance Committee: Paris. Pawson, R., Greenhalgh, T., Harvey, G. & Walshe, K. (2004). Realist synthesis: an introduction. RMP Methods Paper 2/2004. ESRC Research Methods Programme, University of Manchester. Pelling, M. & Manuel-Navarrete, D. (2011). From resilience to transformation: the adaptive cycle in two Mexican urban centers. Ecology and Society 16(2): 11. Piketty, T. (2014). Capital in the Twenty-First Century. Cam- bridge MA: Belknap Press. Putnam, R. (2000). Bowling Alone: The Collapse and Revival of American Community, New York: Simon and Schuster. RM-TWG (2014). Resilience Measurement Principles – To- ward An Agenda For Measurement Design. Technical Series No. 1. Resilience Measurement Technical Working Group; Food Security Information Network; World Food Programme. Radheshyam Jadhav, R. (2014). Union Government to Estab- lish Urban Institute of India. Times Of India, Feb 26, 2014. Available at: http://timesofindia.indiatimes.com/city/pune/ union-government-to-establish-urban-institute-of-india/ articleshow/31044810.cms Ramalingam, B. (2013). Aid on the Edge of Chaos: Rethink- ing International Cooperation in a Complex World. Oxford University Press, Oxford. Raworth, K. (2014). What is resilience? 6 icons tell the story. Exploring donut economics blog. Available at: http://www. kateraworth.com/2014/04/03/resilience/. Reed, S., Friend, R., Toan, V., Thinphanga, P., Sutarto, R. & Singh, D. (2013). “‘Shared learning’’ for building urban cli- mate resilience − experiences from Asian cities. Environ- ment and Urbanization; Vol 25, pp393-412. Rockefeller Foundation (n.d.). Initiative Summary: Asian Cit- ies Climate Change Resilience Network. Summary issued with Grant Letters. Rockefeller Foundation (2010). Introduction to ACCCRN. Prepared for Bellagio Donors Meeting, November 2010. Rockefeller Foundation (2014). Key Lessons from ACCCRN Regarding: Advancing and Embedding High Quality UCCR Approaches and Processes. Draft Point of View paper. Rockefeller Foundation, Bangkok. Rockefeller.org website gender equity: http://www.rockefel- lerfoundation.org/our-work/ current-work/gender-equity. Rockström, J. et al. (2009). A safe operating space for hu- manity. Nature, 461, pp472-475. Rodin, J. (2014a). The Resilience Dividend. Speech at the World Urban Forum, Medellin, Colombia, Apr 11, 2014. Rodin, J. (2014b). Announcing The Global Resilience Part- nership. Blogpost, Aug 04, 2014. Available at: http://www. rockefellerfoundation.org/our-work/current-work/global- resilience-partnership Román et al. (2012). Development policies as a vehicle for addressing climate change. Climate and Development 4(3): 251-260. Schön, D.A. (1983). The Reflective Practitioner. How profes- sionals think in action. Temple Smith, London. Sharma, D., Singh, R. & Singh, R. (2013). Urban Climate Resil- ience: A review of the methodologies adopted under the ACCCRN initiative in Indian cities. Asian Cities Climate Resilience. Working Paper Series 5. Teri, New Delhi and IIED, London. Sharma, D., Singh. R. & Singh, R. (2014). Building urban cli- mate resilience: learning from the ACCCRN experience in India. International Journal of Urban Sustainable Devel- opment, 6:2, 133-153. Simonsen, S., Biggs, R., Schlüter, M., Schoon, M., Bohensky, E., Cundill, G., Dakos, V., Daw, T., Kotschy, K., Leitch, A., Quin- lan, A., Peterson, G. & Moberg, F.(n.d.). Applying resilience thinking. Seven principles for building resilience in social- ecological systems. Stockholm Resilience Centre. Skinner, E. (2011). Gender and Climate Change, BRIDGE Cut- ting Edge Packs. IDS, Brighton. Slater, T. (2014).The resilience of neoliberal urbanism. Open Democracy website, 28 January 2014. Available at: www. opendemocracy.net/opensecurity/tom-slater/resilience- of-neoliberal-urbanism. Snowden, D. & Boone, M. (2007). A Leader’s Framework for Decision Making. Harvard Business Review; November 2007.
- 106. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE84 Stiglitz, J., Sen, A. & Fitoussi, J. (2009). The Measurement of Economic Performance and Social Progress Revisited: Reflections and Overview. Commission on the Measure- ment of Economic Performance and Social Progress. Available at: www.stiglitz-sen-fitoussi.fr. Sumner, A., Ishmael-Perkins, N. & Lindstrom, J. (2009). Mak- ing Science of Influencing: Assessing the Impact of De- velopment Research. IDS Working Paper 335, Institute of Development Studies, Brighton. Tanner, T., Mitchell, T., Polack, E. & Guenther, B. (2009). Urban Governance for Adaptation: Assessing Climate Change Resilience in Ten Asian Cities. IDS Working Paper 315. IDS, Brighton. TARU. (2010). Asian Cities Climate Change Resilience Network: India Chapter. Phase 2. Surat: City Resilience Strategy. TARU Leading Edge, Gurgaon & Gandhinagar, for ACCCRN. The Economist (2011). Defining poverty - Measure by mea- sure. Jan 20th 2011 Tyler, S., Nugraha, E., Nguyen, H.K., Nguyen, N.V., Delima, A., Thinpanga, P., Tran, T.T., Verma, S.S., Swanson, D. & Bi- zikova, L. (2014). Developing Indicators of Urban Climate Resilience. Climate Resilience Working Paper # 2. ISET, Boulder, Colorado, USA. UN (2014). Website for Rio+20 United Nations Conference on Sustainable Development, available at: http://www. uncsd2012.org/. UN DESA (2014). World Urbanization Prospects: The 2014 Revision, Highlights (ST/ESA/SER.A/352). United Nations, Department of Economic and Social Affairs, Population Division. UN-HABITAT (2011), Cities and Climate Change: Policy Directions. Global Report on Human Settlements 2011 Abridged Edition. United Nations Human Settlements Program. Earthscan, London. UN-HABITAT (2012). State of the World’s Cities Report 2012/2013: Prosperity of Cities. UN-HABITAT, Nairobi. UN-HABITAT (2013). Urbanization and Climate Change. Asia-Pacific Issue Brief Series on Urbanization and Cli- mate Change No. 1. United Nations Human Settlements Program, UN-HABITAT Bangkok. UN-HABITAT. (2014a). Guide: Planning for Climate Change. A Strategic, Values-Based Approach for Urban Planners. Tool Series, Cities and Climate Change Initiative, UN- HABITAT, Nairobi. UN-HABITAT. (2014b).Toolkit: Planning for Climate Change. A Strategic, Values-Based Approach for Urban Planners. Tool Series, Cities and Climate Change Initiative, UN- HABITAT, Nairobi. UNISDR Resilient Cities website: http://www.unisdr.org/cam- paign/resilientcities/. USAID (n.d.). Building Climate Resilient Cities. USAID. Verulam Associates & Rockefeller ACCCRN Team (2010). Results Framework ACCCRN Final Version 18 October 2010. Verulam Associates & ACCCRN (2013). Results Frame- work – Asia Cities Climate Change Resilience Network (ACCCRN). Draft Revision 1 May 2013. Verulam Associates (2014). 2013 ACCCRN Monitoring Syn- thesis Report, Verulam Associates Ltd. World Bank (2014). Strengthening Global Collaboration to Support Urban Resilience. World Bank blogpost 15th April, 2014. Available at: http://www.worldbank.org/en/ news/ feature/2014/04/15/stengthening-global-collabo- ration-for-urban-resilience. World Bank Institute. Value for Money in Public-Private Partnerships. Washington D.C. Available at: http://wbi. worldbank.org/wbi/news/2013/11/04/value-money-public- private-infrastructure-projects.
- 107. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE 85 ©LisaMurrayforRobinWyattVision
- 108. SUMMATIVE EVALUATION OF THE ROCKEFELLER FOUNDATION ACCCRN INITIATIVE86 www.rockefellerfoundation.org Supported by