Sign in or 
| Version | User | Scope of changes |
|---|---|---|
| Jul 3 2009, 4:55 AM EDT (current) | chachabooth | 5 words added, 3 words deleted |
| Jun 29 2009, 5:39 AM EDT | chachabooth | 111 words added, 88 words deleted |
Changes
Key: Additions Deletions
1) What is an indicator?
“An indicator is a quantitative or qualitative factor or variable that provides a simple and reliable means to reflect and monitor the changes [differences, improvements, developments, chachabooth] connected to an intervention.”[1] Indicators are therefore pieces of information that help to measure or monitor change. Peter Woodrow describes indicators as management tools that help to measure progress towards the desired results.[2] Indicators tell us what has changed, but not why this change occurred. We can say that indicators are like a specialized set of lenses – they help bring into focus what you want to watch for and study in greater detail.
Indicators should display a number of qualities:
2) Why and when do we use indicators?
Indicator development often provides great insight for a programme, because the development of indicators makes it necessary to think creatively about what one really wants to achieve and learn.[6] Indicators enable to approach the change initiated because of a specific peacebuilding initiatives. Indicators are at the heart of Planning, Monitoring, Evaluation and Learning as they make the process practical and more concrete. Indicators are used in different stages of PME & L: in project design/planning (through developing indicators for the different levels of the results chain the results become more concrete and tangible), for gathering baseline information, in monitoring and evaluation. After indicators have been developed in the planning stage of a programme/project, information is gathered – and the indicators validity tested – in the baseline study. Indicators can then be used for determining progress towards results in monitoring as well as in monitoring the context of the conflict. Evaluation again uses indicators and compares to the results of the baseline study in order to determine what change has taken place due to the intervention.
3) How can we develop indicators?
The development of appropriate and valuable indicators is an exercise that demands a lot of reflection and time. However, often indicator development takes place too late at the last minute when a proposal is due. Good indicators are context-specific, because concepts related to conflict transformation (trust, reconciliation) are embedded in the local context. Indicator development therefore should therefore strive to use a participatory approach.
Following Lederach, Neufeldt and Culbertson some simple questions can function as a starting point for indicator development:
1) Preparation:
Once you have generated a list of possible indicators, try to reduce the list to the best indicators. This can be done by comparing the indicators to the quality criteria mentioned above.
5) Testing indicators
Test the utility in decision-making of new and newly modified indicators as soon as possible to avoid the risk of relying on flawed or not useful indicators.
6) Set targets for indicators
You can define certain values for indicators that will be reached at a certain point in time. This helps to monitor the project over time.
Indicators need to be defined for all levels of the results chain (starting from actvities). Keep in mind that indicators may need to change over the course of the programme due to changes in the definition of what success is or changes in the direction of a programme.[8] Therefore, a regular reflection of used indicators is crucial. It is good to keep in mind that indicators will never be perfect – they are only means to help us see changes taking place.
4) Challenges and risks in indicator development and usage
The main challenge for developing indicators for peacebuilding initiatives is to operationalise concepts that are often highly complex and rather abstract, such as peace, reconciliation, justice or capacity. Also, by only focusing on indicators in monitoring, there is a danger to look through blinders and oversee significant changes that are taking place outside of what is measured with indicators.
The development of good indicators is dependent on the clarity of the programme design, especially the formulation of goals/objectives and the results chain. If objectives are vague and broad then the identification of clearly measurable indicators becomes very difficult.
There is a risk that the development of indicators becomes a “fill in the box” exercise that substitutes good analysis and interpretation.[9] Furthermore, there is also a risk of being tempted to confuse the achievement of objectives and the measurement of indicators. What has to be kept in mind is that indicators are pieces of information that should help to measure whether objectives have been achieved.[10]
5) References
Church, Cheyanne/Rogers, Mark M., Designing for Results. Integrating Monitoring and Evaluation in Conflict Transformation Programs (Chapter 4: Indicators), Search for Common Ground/United States Institute of Peace/Alliance for Peacebuilding, 2006, pp. 43-60. <http://www.sfcg.org/programmes/ilr/ilt_manualpage.html> (accessed 14 April 2009).
Lederach, John Paul/Neufeldt, Reina/Culbertson, Hal, Reflective Peacebuilding. A Planning, Monitoring, and Learning Toolkit (Chapter 7: Creating Indicators), Joan B. Kroc Institute for International Peace Studies and Catholic Relief Services East Asia Regional Office, 2007, pp. 37-42. <http://kroc.nd.edu/sites/default/file/reflective_peacebuilding.pdf> (accessed 14 April 2009).
Australian Government and Australian Agency for International Development, AusGuideline, Activity implementation 4.2 Baseline studies, 2005 <www.ausaid.gov.au/ausguide/pdf/ausguideline4.2.pdf> (accessed 14 April 2009)
Peter Woodrow, Myths and Realities of Evaluating Peacebuilding Work. Evolving Practice from the Reflecting on Peace Practice Project, powerpoint presentation given at the Life & Peace Institute Workshop "Measuring peace“, February 2008.
“An indicator is a quantitative or qualitative factor or variable that provides a simple and reliable means to reflect and monitor the changes [differences, improvements, developments, chachabooth] connected to an intervention.”[1] Indicators are therefore pieces of information that help to measure or monitor change. Peter Woodrow describes indicators as management tools that help to measure progress towards the desired results.[2] Indicators tell us what has changed, but not why this change occurred. We can say that indicators are like a specialized set of lenses – they help bring into focus what you want to watch for and study in greater detail.
Indicators should display a number of qualities:
- Generally, an indicator should concretely specify what is to be measured.
- Indicators shall be SMART:[3]
- Specific: clear, simple, single items of information, targeted (What is changing? Who is changing?)
- Measurable: items can be reliably quantified. This is safeguarded by: using specific units of measurement (What will be measured?); referring to a baseline/benchmark of comparison; using clearly defineddefine the word you wordsuse (e.g. “effective”, “appropriate”, “successful”)
- Attainable: data easily and cost-effectively gathered.
- Relevant: information relates to the different levels of results (see chain of results)
- Timely: data can be collected and analysed quickly enough to be useful in implementation management.
- Indicators should be reliable: Information is credible. Assumptions are clearly stated. There is a direct connection between the indicator and what you are trying to prove. Measuring the indicator is independent from who gathers information.
- Indicators should be attributable, i.e. measure changes that are clearly attributable to the specific peacebuilding initiative. It is good to ask yourself whether what the result would have occured without your engagement.
Example: A project aims to increase the political empowerment of women. A possible quantitative indicator would be: “Number of women elected parliamentarians in the next election.” while a qualitative indicator could be “increase in % in women parliamentarians’ belief that their voices are making a difference.” or “decrease in % of women’s perception that they are marginalised in decision-making.”[5]
2) Why and when do we use indicators?
Indicator development often provides great insight for a programme, because the development of indicators makes it necessary to think creatively about what one really wants to achieve and learn.[6] Indicators enable to approach the change initiated because of a specific peacebuilding initiatives. Indicators are at the heart of Planning, Monitoring, Evaluation and Learning as they make the process practical and more concrete. Indicators are used in different stages of PME & L: in project design/planning (through developing indicators for the different levels of the results chain the results become more concrete and tangible), for gathering baseline information, in monitoring and evaluation. After indicators have been developed in the planning stage of a programme/project, information is gathered – and the indicators validity tested – in the baseline study. Indicators can then be used for determining progress towards results in monitoring as well as in monitoring the context of the conflict. Evaluation again uses indicators and compares to the results of the baseline study in order to determine what change has taken place due to the intervention.
3) How can we develop indicators?
The development of appropriate and valuable indicators is an exercise that demands a lot of reflection and time. However, often indicator development takes place too late at the last minute when a proposal is due. Good indicators are context-specific, because concepts related to conflict transformation (trust, reconciliation) are embedded in the local context. Indicator development therefore should therefore strive to use a participatory approach.
Following Lederach, Neufeldt and Culbertson some simple questions can function as a starting point for indicator development:
- What do you want to learn about?
- How will you see whether the expected results actually occurred?
- Be specific and clear about your expected results.
- Identify the processes that you want to learn about, then think carefully how you will see and track the process.
- Think creatively about context and culture. Ask the people and partners you work with to help you develop indicators for the changes you are trying to promote.
- Consider carefully the timeframes of change.
- Be careful with nice phrases and jargon. Probe what is really going on.
1) Preparation:
- Objectives/expected results need to be defined as specific and attainable as possible.
- Articulate your theory(ies) of change. Knowing exactly how activities are supposed to be linked to results makes it possible to generate clear indicators.
- Keep an indicator bank (indicators that have been used before, by your or other organisations; borrow from other professions, e.g. public health)
- Make indicator development a continuous undertaking.
- Brainstorm all related things or dimensions that can be counted, measured, or sized.
- Ask yourself: How would we see that a result is obtained?
- Ask partners, local actors and stakeholders what they consider to be significant signals of change (participatory indicator development).
- Break issues (multidimensional concepts) into smaller components. E.g. Reconciliation can be broken down to the components mercy, justice, truth, and peace. Each of these components again is rather abstract and not directly measurable and therefore needs to be operationalised by finding indicators.
- Expand the use of proven social science methods for measuring qualitative factors of change (attitudes, opinions, perceptions…)
- When developing indicators, also keep in mind that sometimes your results - in particular the ones on output level - might already be observable and therefore might not need additional indicators.
- Keep focused: What do we need to know and what information will tell us what I need to know?
- If formulated objectives are not providing clarity, consider rewriting them.
- Once you have a couple of possible indicators, look for ways to make them increasingly simple.
Once you have generated a list of possible indicators, try to reduce the list to the best indicators. This can be done by comparing the indicators to the quality criteria mentioned above.
5) Testing indicators
Test the utility in decision-making of new and newly modified indicators as soon as possible to avoid the risk of relying on flawed or not useful indicators.
6) Set targets for indicators
You can define certain values for indicators that will be reached at a certain point in time. This helps to monitor the project over time.
Indicators need to be defined for all levels of the results chain (starting from actvities). Keep in mind that indicators may need to change over the course of the programme due to changes in the definition of what success is or changes in the direction of a programme.[8] Therefore, a regular reflection of used indicators is crucial. It is good to keep in mind that indicators will never be perfect – they are only means to help us see changes taking place.
4) Challenges and risks in indicator development and usage
The main challenge for developing indicators for peacebuilding initiatives is to operationalise concepts that are often highly complex and rather abstract, such as peace, reconciliation, justice or capacity. Also, by only focusing on indicators in monitoring, there is a danger to look through blinders and oversee significant changes that are taking place outside of what is measured with indicators.
The development of good indicators is dependent on the clarity of the programme design, especially the formulation of goals/objectives and the results chain. If objectives are vague and broad then the identification of clearly measurable indicators becomes very difficult.
There is a risk that the development of indicators becomes a “fill in the box” exercise that substitutes good analysis and interpretation.[9] Furthermore, there is also a risk of being tempted to confuse the achievement of objectives and the measurement of indicators. What has to be kept in mind is that indicators are pieces of information that should help to measure whether objectives have been achieved.[10]
5) References
Church, Cheyanne/Rogers, Mark M., Designing for Results. Integrating Monitoring and Evaluation in Conflict Transformation Programs (Chapter 4: Indicators), Search for Common Ground/United States Institute of Peace/Alliance for Peacebuilding, 2006, pp. 43-60. <http://www.sfcg.org/programmes/ilr/ilt_manualpage.html> (accessed 14 April 2009).
Lederach, John Paul/Neufeldt, Reina/Culbertson, Hal, Reflective Peacebuilding. A Planning, Monitoring, and Learning Toolkit (Chapter 7: Creating Indicators), Joan B. Kroc Institute for International Peace Studies and Catholic Relief Services East Asia Regional Office, 2007, pp. 37-42. <http://kroc.nd.edu/sites/default/file/reflective_peacebuilding.pdf> (accessed 14 April 2009).
Australian Government and Australian Agency for International Development, AusGuideline, Activity implementation 4.2 Baseline studies, 2005 <www.ausaid.gov.au/ausguide/pdf/ausguideline4.2.pdf> (accessed 14 April 2009)
Peter Woodrow, Myths and Realities of Evaluating Peacebuilding Work. Evolving Practice from the Reflecting on Peace Practice Project, powerpoint presentation given at the Life & Peace Institute Workshop "Measuring peace“, February 2008.
Sartorius, Rolf/Carver, Christopher, Monitoring, Evaluation and Learning for Fragile States and Peacebuilding Programs. Practical Tools for Improving Program Performance and Results, Social Impact, 2006. <http://www.socialimpact.com/resource-center/downloads/fragilestates.pdf> (accessed 14 April 2009)
Footnotes:
