Sign in or 
- EasyEdit
- Edit tags
-
(what's this?What are these tools?
People just like you can add or edit the content on this site. If you want to try editing, but aren't ready to add to this site, try our demo area.
Read more about editing pages at Wetpaint Central.
) - Report page
Share this
Activity: Cornerstones of the monitoring system
This activity builds on earlier activities ("Results chain exercises", "Peacebuilding Indicators" and "Developing a baseline plan").
Handout "How to do monitoring?"
Doing monitoring is based on the steps described earlier throughout this resource kit, namely: Theories of Change, results chains, indicators and baseline study. The results chain, together with your indicators, can be used as a guide for monitoring, in particular the implementation monitoring. If you include theories of change in your monitoring, this may help you to keep track on this dimension as well.
The data that you collected in the baseline study establishes a point of comparison. It is good to plan how you want to monitor your project/programme from the beginning. A monitoring plan can help to structure the process. Below, find an example framework that could be used for monitoring:
(Implementation) monitoring plan
Adapted from Church/Rogers, p. 91.
With this plan, the collection of data can be organised. This is, however, only the first step. Once the data has been collected, it needs to be interpreted and incorporated in decision-making processes. Also, this table does not include context monitoring or monitoring of assumptions. A good tool for monitoring can also be programme team reflection/monitoring sessions, facilitated and prepared by one staff member assigned for the PMEL process. This can also be a platform to discuss monitoring data and look at its implications for programme implementation. Besides looking at quantitative indicators these sessions could be prepared by asking all team members to use self-assessment tools or alternatively by asking them to identify the most significant changes that have taken place within the programme. Such sessions can easily also be conducted with partners and techniques for both self-assessment and for identifying most significant changes are described in the next section.
Practical tools for monitoring and data collection
Once you have developed a monitoring plan, you also need to decide on what tools you will use to collect the data that you need to learn (in order to inform decision-making). The most suitable tools will to a large extent depend on the context you are working in, the resources that you have available to do monitoring and whether you are mainly working on your own or together as a team.
Below we will present some different tools for data collection that can be very useful when monitoring.
Questionnaires
As long as the data that you want to collect is quantitative, tangible and easy to measure (rather than qualitative), a simple questionnaire asking your target group a number of questions is an easy way to do monitoring of activities. When you construct the questionnaire you need to remember to code or structure it so that you easily can summarise the results when doing data analysis. When using questionnaires for the evaluation of activities – such as workshops – it is also important to remember to hand them out to participants as soon as possible after the activity in order for participants not to forget what they thought or what the activity was about. If you use questionnaires to monitor processes it might, however, not be as important to distribute questionnaires directly after activities have taken place. In fact there can be reasons for why you would want to wait some time. Remember though that the monitoring of processes might demand the collection of more qualitative data than what a questionnaire easily allows you to collect. A good idea could then be to combine the questionnaire with other tools for qualitative data collection.
While the monitoring of indicators is important, a too narrow focus on pre-defined quantitative indicators does not guarantee that we learn from monitoring. The measurement of quantitative indicators often needs to be complemented with more qualitative forms of information gathering in order to capture the complex dimensions of capacity building and conflict transformation. Monitoring systems can be designed in a way that enables the measuring of results through both indicators and more “qualitative” forms of monitoring. This can be done through asking for the most significant change, self-assessment tools and reflection sessions
Asking for the most significant change[1]
In order to monitor different processes in contexts where the information you need is of a more qualitative nature or where people might not be used to filling in questionnaires a good tool for monitoring can be to ask for stories about the most significant change. Define a time frame and an area in which you want to monitor change and then ask your target group to describe to you the most significant change that has taken place during this time period and within the preset area. You can also ask them how this change happened if you want to specifically monitor your own assumptions about change. By asking these relatively open questions, the people you are interviewing are free to answer without being bound to your own indicators or guiding questions which allows you to see also those changes that you had not planned for as well as discover new risks within the programme in time. This method for monitoring also allows for stories of most significant change to be reported through other means than written text (e.g. images, drawings, taped interviews, videos and films) which might help in target groups that are illiterate or more used to oral traditions or who simply are not trained in formal report writing. You and your team can then sit together and analyse what changes were in line with your own visions of success, theory of change etc. Are there connections between your theory of change and how the target group describes how change happened, or do you need to adapt your theories of change to better fit with how the target group deals with and see change as occurring?
Self assessment tools
Self assessment can be a very useful tool for internal monitoring of programmes and change processes with partners as internal staff within a programme or a partner often tend to be more critical to their work than are external evaluators. One way of doing self–assessment in a structured manner is to let staff or partners fill in a results journal on a regular basis[2].
To track changes over time for example for each partner organisation you engage with, it seems useful to complete one results journal per partner. The results journal lists the indicators and asks for an assessment what the level of change with regards to indicators was (if there was any change). The level of change ranges from low (0-40 %), medium (40-80%) to high (80-100%), i.e. to what extent has the indicator been achieved. The results journal, in addition, encourages describing and reflecting on the change that has taken place and also asks for unexpected changes – that could easily be forgotten when just looking at pre-formulated results and indicators. In a last step, lessons learnt and suggestions for changes in the programme implementation can be noted. The results journal is completed by individual staff members (this can of course be extended to partners’ staff) and then discussed in common monitoring sessions taking place regularly (see below).
Results Journal
Adapted from Outcome Mapping's Outcome Journal. See Sarah Earl/Fred Carden/Terry Smutylo, Outcome Mapping. Building Learning and Reflection into Development Programmes, IDRC, 2001. <http://www.idrc.ca/en/ev-28389-201-1-DO_TOPIC.html> (accessed 29 May 2009)
Regular programme team reflection sessions
If you are working in a team a good tool for monitoring can be regular programme team reflection sessions, facilitated and prepared by one staff member assigned for the PME&L process. Such meetings can also be a platform to discuss monitoring data, do assumptions monitoring and look at implications for programme implementation. Besides interpreting data collected for quantitative indicators, these sessions could be prepared by asking all team members to use self-assessment tools. If you have collected change stories, these sessions provide the opportunity to review and reflect them. Such sessions can easily also be conducted with partners.
Remember: A sustainable monitoring system should be: simple to grasp, light to implement, provide useful information, help meeting reporting requirements.
Notes:
| Purpose: | To familiarise participants with the possible options for developing a monitoring system and to apply this knowledge to an example |
| Participants: | 4 to about 20 |
| Time: | 60-90 min. |
| Materials: | Flip charts Marker pens |
| Process: | 1. Introduce participants to how monitoring can be done in practice. Distribute the handout below. 2. Ask participants to study the handout carefully and make a list of possible elements of monitoring systems. 3. Participants now assemble in the group that they have worked with on "Results chain exercises", "Peacebuilding Indicators" and "Developing a baseline plan". 4. The groups' task is now to identify what elements the monitoring system for the project should comprise to enable learning. Here, they can draw on the work on results chains, indicators and baseline study and the knowledge they have gained about monitoring. Remind them to keep in mind that a good monitoring system should inform day-to-day decision making and that it should be as simple as possible while still fulfilling its learning function. Groups can note their ideas on flip charts. 5. Let groups present back to plenary how they would design the monitoring system. Encourage questions and answers from the other groups. |
| Note: | |
| Handout(s): | Handout "How to do monitoring?" (See below) |
| Source(s): | chachabooth@gmail.com |
Handout "How to do monitoring?"
Doing monitoring is based on the steps described earlier throughout this resource kit, namely: Theories of Change, results chains, indicators and baseline study. The results chain, together with your indicators, can be used as a guide for monitoring, in particular the implementation monitoring. If you include theories of change in your monitoring, this may help you to keep track on this dimension as well.
The data that you collected in the baseline study establishes a point of comparison. It is good to plan how you want to monitor your project/programme from the beginning. A monitoring plan can help to structure the process. Below, find an example framework that could be used for monitoring:
(Implementation) monitoring plan
| Monitoring focus: Result (on different levels: outcome, output) | Indicators | Means of Verification/Information source/Data collection tool | Frequency of data collection | Who collects data? |
With this plan, the collection of data can be organised. This is, however, only the first step. Once the data has been collected, it needs to be interpreted and incorporated in decision-making processes. Also, this table does not include context monitoring or monitoring of assumptions. A good tool for monitoring can also be programme team reflection/monitoring sessions, facilitated and prepared by one staff member assigned for the PMEL process. This can also be a platform to discuss monitoring data and look at its implications for programme implementation. Besides looking at quantitative indicators these sessions could be prepared by asking all team members to use self-assessment tools or alternatively by asking them to identify the most significant changes that have taken place within the programme. Such sessions can easily also be conducted with partners and techniques for both self-assessment and for identifying most significant changes are described in the next section.
Practical tools for monitoring and data collection
Once you have developed a monitoring plan, you also need to decide on what tools you will use to collect the data that you need to learn (in order to inform decision-making). The most suitable tools will to a large extent depend on the context you are working in, the resources that you have available to do monitoring and whether you are mainly working on your own or together as a team.
Below we will present some different tools for data collection that can be very useful when monitoring.
Questionnaires
As long as the data that you want to collect is quantitative, tangible and easy to measure (rather than qualitative), a simple questionnaire asking your target group a number of questions is an easy way to do monitoring of activities. When you construct the questionnaire you need to remember to code or structure it so that you easily can summarise the results when doing data analysis. When using questionnaires for the evaluation of activities – such as workshops – it is also important to remember to hand them out to participants as soon as possible after the activity in order for participants not to forget what they thought or what the activity was about. If you use questionnaires to monitor processes it might, however, not be as important to distribute questionnaires directly after activities have taken place. In fact there can be reasons for why you would want to wait some time. Remember though that the monitoring of processes might demand the collection of more qualitative data than what a questionnaire easily allows you to collect. A good idea could then be to combine the questionnaire with other tools for qualitative data collection.
While the monitoring of indicators is important, a too narrow focus on pre-defined quantitative indicators does not guarantee that we learn from monitoring. The measurement of quantitative indicators often needs to be complemented with more qualitative forms of information gathering in order to capture the complex dimensions of capacity building and conflict transformation. Monitoring systems can be designed in a way that enables the measuring of results through both indicators and more “qualitative” forms of monitoring. This can be done through asking for the most significant change, self-assessment tools and reflection sessions
Asking for the most significant change[1]
In order to monitor different processes in contexts where the information you need is of a more qualitative nature or where people might not be used to filling in questionnaires a good tool for monitoring can be to ask for stories about the most significant change. Define a time frame and an area in which you want to monitor change and then ask your target group to describe to you the most significant change that has taken place during this time period and within the preset area. You can also ask them how this change happened if you want to specifically monitor your own assumptions about change. By asking these relatively open questions, the people you are interviewing are free to answer without being bound to your own indicators or guiding questions which allows you to see also those changes that you had not planned for as well as discover new risks within the programme in time. This method for monitoring also allows for stories of most significant change to be reported through other means than written text (e.g. images, drawings, taped interviews, videos and films) which might help in target groups that are illiterate or more used to oral traditions or who simply are not trained in formal report writing. You and your team can then sit together and analyse what changes were in line with your own visions of success, theory of change etc. Are there connections between your theory of change and how the target group describes how change happened, or do you need to adapt your theories of change to better fit with how the target group deals with and see change as occurring?
Self assessment tools
Self assessment can be a very useful tool for internal monitoring of programmes and change processes with partners as internal staff within a programme or a partner often tend to be more critical to their work than are external evaluators. One way of doing self–assessment in a structured manner is to let staff or partners fill in a results journal on a regular basis[2].
To track changes over time for example for each partner organisation you engage with, it seems useful to complete one results journal per partner. The results journal lists the indicators and asks for an assessment what the level of change with regards to indicators was (if there was any change). The level of change ranges from low (0-40 %), medium (40-80%) to high (80-100%), i.e. to what extent has the indicator been achieved. The results journal, in addition, encourages describing and reflecting on the change that has taken place and also asks for unexpected changes – that could easily be forgotten when just looking at pre-formulated results and indicators. In a last step, lessons learnt and suggestions for changes in the programme implementation can be noted. The results journal is completed by individual staff members (this can of course be extended to partners’ staff) and then discussed in common monitoring sessions taking place regularly (see below).
Results Journal
| Monitoring period | ||
| Contributors to this monitoring update | ||
| Vision of success in the programme: | ||
| Outcomes | Indicators | Level of change |
| Low (L) Medium (M) High (H) LMH LMH LMH LMH | ||
| Outputs | Indicators | Level of change |
| LMH LMH LMH LMH | ||
| Description of the change: What happened? How did change happen? Why? | ||
| Who or what contributed to the change? | ||
| Sources of evidence | ||
| Unanticipated change | ||
| Lessons/ required programme changes/reactions | ||
Regular programme team reflection sessions
If you are working in a team a good tool for monitoring can be regular programme team reflection sessions, facilitated and prepared by one staff member assigned for the PME&L process. Such meetings can also be a platform to discuss monitoring data, do assumptions monitoring and look at implications for programme implementation. Besides interpreting data collected for quantitative indicators, these sessions could be prepared by asking all team members to use self-assessment tools. If you have collected change stories, these sessions provide the opportunity to review and reflect them. Such sessions can easily also be conducted with partners.
Remember: A sustainable monitoring system should be: simple to grasp, light to implement, provide useful information, help meeting reporting requirements.
Notes:
[1] Inspired by “The most Significant Change Technique” developed by Rick Davies for CCDB see R. Davies and J. Dart 2005 ”The most significant change MSC Technique, A guide to its use”
[2] The described results journal is adapted from Outcome Mapping's ”Outcome Journal” to better fit the PME&L system as it has been developed in this resource kit so far. For more information about Outcome Mapping please see Sarah Earl/Fred Carden/Terry Smutylo, Outcome Mapping. Building Learning and Reflection into Development Programmes, IDRC, 2001. <http://www.idrc.ca/en/ev-9330-201-1-DO_TOPIC.html> (accessed 29 May 2009); Sarah Earl/Fred Carden/Terry Smutylo, Brochure on Outcome Mapping. The Challenges of Assessing Development Impact, International Development Research Center (IDRC), 2001. <http://www.idrc.ca/uploads/user-S/10945665201om_pamplet_final.ppt> (accessed 29 May 2009)
chachabooth |
Latest page update: made by chachabooth
, Aug 26 2009, 4:25 AM EDT
(about this update
About This Update
Edited by chachabooth
1744 words added 3 words deleted view changes - complete history) |
|
More Info: links to this page
|
There are no threads for this page.
Be the first to start a new thread.
