Activity: process for planning evaluations for concrete peacebuilding engagementsThis is a featured page

Evaluation is nothing that should just “happens” to your project/programme/initiative. Instead, teams should use the opportunity to develop an evaluation plan already throughout the planning stage of a project or programme. Throughout this process, a number of decisions have to be taken. These sessions will focus on 5 key decisions:
1. What do we want to learn from the evaluation?
2. Who will the audience of the evaluation report be?
3. What type of evaluation is suitable for the kind of learning we aspire?
4. What evaluation approach is most appropriate for the kind of learning we want?
5. Who should do the evaluation?

Participants should have been introduced to evaluation in general before. Some of the sessions will include introduction material if necessary. One of the best resources to use for evaluation planning is:

Church, Cheyanne/Rogers, Mark M., Designing for Results. Integrating Monitoring and Evaluation in Conflict Transformation Programs, Search for Common Ground/United States Institute of Peace/Alliance for Peacebuilding, 2006, Chapter 8: Evaluation Preparation. <http://www.sfcg.org/programmes/ilr/ilt_manualpage.html> (accessed 14 April 2009).

The steps introduced have been condensed here for the group process.

Purpose: To plan how the project/programme should be evaluated.
Participants:The core participants of this activity are the team members dealing with the project or programme directly. Other staff, such as finance officers and adminitstrative staff, could also be involved. In addition, the process will benefit from the participation of local stakeholders, such as partner organisations.

Time: Steps 1 to 5 at least 60 min.
Steps 6 to 8 around 60 min.
Steps 9 and 10 at least 60 min.
Steps 11 to 13 at least 60 min.
Materials:Flip charts
Marker Pens
Masking tape
Index Cards
Pens
Process: 1. Introduce the work that will have to be done now to the participants.

2. Start with the most important question: What do we want to learn from evaluating the project/programme?

Ask participants to write learning points on index cards, completing the sentence “I want to learn…”. Collect them from participants when you have the feeling that participants have exhausted their ideas.

3. Read ideas out loud and organize them in thematic groups on a flipchart with the headline “We want to learn….” (for instance grouping them according to evaluation criteria). Feed possible missing points into the process by asking questions. Also, consider that there might be donor requirements that need to be considered.

4. Depending on the amount of learning points on the flip chart, the team might have to select the most important learning areas, i.e. the ones that will help improving the project in the next implementation round or benefit future projects. Note them on a separate flip chart.

5. Now you can focus on the next question: Who will the audience of the evaluation report be?
Let the group think about possible target audiences and list them on the flip chart. Make sure that the obvious – the team itself – is not forgotten. Also, draw a distinction between users and readers. Discuss why the mentioned target audiences will benefit from the evaluation report.

6. Now, you can focus on deciding what type of evaluation is most suitable for the kind of learning that you want to realize. It is likely that you first need to introduce the major types of evaluation to participants:
There are in principle three types of evaluation: formative, summative and impact evaluation. Formative evaluations occur around the middle of a project, summative evaluations take place near or at the end of a project. Impact evaluations occur some time after the project is complete to assess its results and, if feasible, the adaptability to change of those results.

7. In order to take a decision between formative and summative evaluations, distribute the handout with an overview of benefits for each type to participants (see below).

8. Engage the team in a discussion with the aim of reaching a decision between the types of evaluation. Before deciding on the matter, the team should also reflect what information the monitoring system provides and compare that to what a formative evaluation would provide.

9. After having decided which type of evaluation should be used, another decision needs to be taken: What evaluation approach is best suited to enable the kind of learning we want? For that purpose, you might have again to introduce evaluation approaches to the participants. In order to do so, Church and Rogers (Designing for Results) description of evaluation approaches from pages 114 to 121 can be very useful.

Organise the groups in pairs and ask them to familiarise themselves with at least 2 evaluation approaches. (Note: The group size will determine how many approaches each pair will deal with, make sure all six described approaches are discussed.) Let them prepare a 100 sec. presentation, where they outline when the approach is used (what you can learn about) and what strengths and weaknesses are.

10. After the presentations of the approaches, go back to the list “We want to learn about…” created in step 2, facilitate the discussion with the group and try to reach a conclusion about the best evaluation approach.

11. The last decision for the group process is the question of who should conduct the evaluation. This again entails a number of questions, but only one of them should be discussed in a group process. It is “Will the evaluators be internal or external?”

12. Use a game for taking this decision: Split the group in three sub-groups. One of the group will advocate for an external evaluator, the second will advocate for an internal evaluator and the third will observe the debate and judge which position is strongest. You can provide background material to the groups (Church and Rogers, pp. 126-128.)

13. Summarize the decisions taken throughout the sessions and discuss the follow-up in the evaluation planning. Ideally, one or two team members should be assigned to continue with the details of evaluation planning, including timing and development of ToR (see here Church and Rogers).
Note:


Handout(s): Handout "Benefits of Formative and Summative Evaluations"
Handout "External or Internal Evaluator" (Church and Rogers, <http://www.sfcg.org/programmes/ilr/ilt_manualpage.html> pp. 126-128)
Source(s):The activity builds on the content in Church, Cheyanne/Rogers, Mark M., Designing for Results. Integrating Monitoring and Evaluation in Conflict Transformation Programs, Search for Common Ground/United States Institute of Peace/Alliance for Peacebuilding, 2006, Chapter 8: Evaluation Preparation. <http://www.sfcg.org/programmes/ilr/ilt_manualpage.html> (accessed 14 April 2009).

Handout - Benefits of formative and summative evaluations
Benefits of Formative Evaluation
  • Contributes to reporting and accountability requirements during the project
  • Provides information to improve project before it is too late to make changes
  • Provides structured opportunity for reflection so that staff and resources are focused on the project
  • Helps clarify program strengths and weaknesses
  • Can provide information that assists realignment of project to the changing conflict context
Benefits of Summative Evaluation
  • Helps project sum up what it has achieved
  • Checks achievements against plans and obligations to donors and participants
  • Provides information as to why and how change occurred
  • Generates important information to drive learning
  • Can create documentation that captures approaches and lessons to be used in the wider organization
Taken from Church and Rogers, Designing for Results, p. 110.





chachabooth
chachabooth 		Latest page update: made by chachabooth , Aug 26 2009, 2:05 PM EDT (about this update About This Update chachabooth Edited by chachabooth

8 words added

view changes
- complete history)
Keyword tags: None (edit keyword tags)
More Info: links to this page

Threads for this page

Anonymous  (Get credit for your thread)

There are no threads for this page.  Be the first to start a new thread.