Harmonising Measures of Conservation Success
One challenge facing both the conservation community and those who formulate national policies and programmes of work is how to evaluate the success of conservation efforts in order to identify those approaches that are most effective and distinguish them from less productive approaches.
This project, funded by the MacArthur Foundation, aims to address this challenge by building on and harmonising the work done by CCF member organisations to evaluate their own conservation impacts and, ultimately, to improve conservation practice by identifying successful approaches and factors that contribute to their success.
The objectives are to:
- Develop CCF's own harmonised approaches to measuring conservation success.
- Interact with the Conservation Measures Partnership in the U.S. to test and further develop their measures and compare them with those of CCF.
- Develop a longer term research plan to use these harmonised tools to assess what factors best predict the relative success or failure of conservation projects.
The project was implemented through a series of iterative steps involving the participation of over 40 individuals representing some 17 CCF member organisations. We:
- Agreed common definitions of conservation and conservation success.
- Agreed that a key to assessing conservation success is in finding ways to move measurement beyond activities and outputs to outcomes and impacts.
- Agreed that approaches to evaluating impacts would differ between different types of conservation action; agreed definitions of, and developed working groups addressing, each of seven broad categories of conservation activity.
- Discussed approaches and common issues & challenges in monitoring and evaluating the results of each type of conservation action.
- Developed and harmonised conceptual models of the relationship between each type of conservation activity and conservation impact.
- Developed a scorecard-style evaluation tool based on the models.
- Tested and refined the tool through trial evaluation of more than 25 conservation projects, both by project personnel and in group contexts that tested consistency among users in interpreting the questions.
- Developed a draft scoring system to enable semi-quantitative analysis of evaluated projects.
- Developed research questions and plan of analysis based on the harmonised tools developed.
Definitions and Concepts
Conservation is increasing the likelihood of persistence of native ecosystems, habitats, species and/or populations in the wild Conservation success is the achievement of this (without adverse impacts on human well-being).
Performance can be monitored at several levels
- Input – have you spent the money?
- Activity – what did you do with it?
- Output – what did it get you? How good was it?
- Outcome – what effect has it had on the problem (cf. what would have happened without it?)
- Impact – what is happening to the ecosystems/habitats/species/populations (biological change)
- It is common to monitor and report on inputs, activities and outputs, but assessing conservation success depends on evaluating outcomes and impacts.
Seven categories of conservation action
CCF members agreed that nearly all of their activities fit comfortably into one or more of the following broad categories of conservation activity. Most conservation projects involve several, but not all, of these activity types:
- Management of Sites, Habitats, Landscapes and Ecosystems – actions directly manipulating or managing a particular site.
- Management of species and populations – actions directly involving species themselves, such as clutch management, captive breeding, etc.
- Efforts to develop, adopt or implement policy or legislation - actions to establish frameworks within the processes of government, civil society or the private sector that make conservation goals official or facilitate their accomplishment; may include development, implementation and/or enforcement of legislation, management plans, sectoral policies, trade regulations, among others.
- Efforts to Enhance and/or Provide Alternative Livelihoods – actions to improve the well-being of people having impact on the species/habitats of conservation interest, including through sustainable resource management, income-generating activities, conservation enterprise, direct incentives.
- Training and capacity building – actions to enhance specific skills among those directly involved in conservation, includes both building individual skills and improving the many components of organisational capacity.
- Education and awareness-raising – actions directed at improving understanding and influencing behaviour among people not necessarily directly involved in conservation action. Covers all forms of communication, including campaigns, lobbying, educational and publicity/awareness programmes, and production of materials.
- Research and conservation planning – actions aimed at improving the information base on which conservation decisions are made, including survey, inventory, remote sensing, mapping, development of new technologies.
Common issues and challenges in assessing conservation success
- Participants in the thematic working groups of this project discussed the approaches used by their organisations in monitoring and evaluating the outcomes of each particular type of conservation action. Several issues were common to all seven working groups:
- Conservation projects or activities within them frequent lack clearly stated objectives.
- The assumptions about how a particular action will lead to conservation impact are rarely explicitly stated.
- Project rationales (problem analyses) are often coloured by donor priorities, so:
- Conservation-related objectives may not be not explicit in the project design (e.g. human development objectives may predominate).
- Monitoring and evaluation address donor priorities and explicit objectives, rather than conservation objectives.
- Addressing the conservation objectives therefore increases monitoring burden, and stretches the resources available.
- Conservation impact time frames are much longer than normal project cycles. Therefore,
- Projects rarely really assess conservation impacts.
- Support for post-project assessment is needed.
The working groups developed simple flow chart models to represent the generic processes by which different types of conservation activity can lead to conservation impact. Each model begins with problem identification and design of interventions and works through activities and a series of intermediate outcomes to conservation impact. While engagement with stakeholders is principally depicted at the early stages of each process, it should be understood to be fundamental throughout. The iterative feedback loops characteristic of adaptive management are also part of the process, but have not been drawn in order to maintain visual clarity. Key points of input from other types of conservation activity are shown by hexagons and grey arrows, but these are indicative rather than exhaustive. Download the latest Conservation Activity Conceptual Models (Word document) - scroll down to the bottom of this page to the "Documents" links to download.
The project has so far generated a fully functional CCF Project Evaluation Tool in Excel format and a supporting document containing key concepts and definitions. The following publications describe the tool and report some results from its trial use:
- Kapos, V., Balmford, A., Aveling, R., Bubb, P., Carey, P., Entwistle, A., Hopkins, J., Mulliken, T., Safford, R., Stattersfield, A., Walpole, M., & Manica, A. (2009) Outcomes, not implementation, predict conservation success. Oryx 43: 336-342.
- Kapos, V., Balmford, A., Aveling, R., Bubb, P., Carey, P., Entwistle, A., Hopkins, J., Mulliken, T., Safford, R., Stattersfield, A., Walpole, M., & Manica, A. (2008) Calibrating Conservation: New Tools for Measuring Success. Conservation Letters, 1, 155-164.
Details of a scoring system to accompany the tool are still being refined and will be released at a later date. We are also developing a more user friendly format for the tool and working to streamline it. We are seeking additional collaborators to test it further and share their comments and results with us. If you are interested in collaborating, please contact the project coordinator, Valerie Kapos.
We are developing a more aesthetically pleasing and user-friendly version of the CCF Project Evaluation Tool that will guide the user through one question at a time and make it easier to navigate within it.
We are seeking individuals and groups interested in using the tool to evaluate their existing or completed projects and to help improve the design of new projects. We hope to be able to support the wider use of the CCF approach through workshops to introduce it and as part of broader programmes of capacity building addressing project design and management.
We are also developing a set of research questions that can be addressed analytically once a sufficient body of projects evaluated using this method has been built up.
If you are interested in collaborating by:
- Using CCF's evaluation tool and contributing to its refinement by sharing your comments and evaluation results.
- Joining with us in using the results of evaluations to test hypotheses about effective conservation practice and approaches.
- Hosting a workshop or other form of introduction to the approach as a means of building capacity in conservation project design and evaluation.
Please contact Valerie Kapos for more details.
The project potentially involves all members of CCF. Its development and implementation have been overseen by a steering committee (see below) comprising representatives of the most actively involved organisations. More than 40 individuals from 17 CCF member organisations were part of the thematic working groups that addressed developed the framework and tools for assessing the conservation outcomes and impacts of different types of conservation activity, and still more were involved in testing these outputs.
The project and its follow-up are co-ordinated on a day-to-day basis by Valerie Kapos. To learn more about, or to participate in this work or collaborate with CCF please email her directly.
- Pete Carey (chair).
- Abigail Entwhistle / Ros Aveling / Matt Walpole – Fauna and Flora International.
- Alison Stattersfield / Roger Safford – Birdlife International.
- Teresa Mulliken – Traffic International.
- John Hopkins / Pete Brotherton – English Nature.
- Phil Bubb / Lera Miles – UNEP-WCMC.
- Andrew Balmford – University of Cambridge, Zoology Dept.
For further information on the project and its implementation, please contact:
Valerie Kapos, Coordinator,
CCF Measures of Success Project,
UNEP World Conservation Monitoring Centre,
219 Huntingdon Road,
Tel: +44 (0)1223 277314
Fax: +44 (0)1223 277136