Evaluation
All work plans established for the action plans include a time frame within which the planned outputs and targeted results or impacts are expected to be achieved. At the end of that period, as the last step of the action planning process, a joint evaluation is arranged. The purpose is to compare the observed outputs, results and impacts achieved to outputs, results and impacts expected.
To do so, events such as final stakeholder workshops or small group sessions with the stakeholder groups which have been involved or impacted by specific actions at the sites can be arranged. But other settings such as household interviews or focus group discussions can also be used for evaluation purposes.
For such events, it is important that they have been agreed to at the start of the action plan implementation, and that they are announced to all well in advance and subsequently organised jointly with all the stakeholders. In practice, this may not always be possible, but then the evaluation should be planned at least in close collaboration between the research team and the local partner having the main responsibility for the implementation of the given action plan.
It is also important that the purpose and procedure of the evaluation is clear to all participants before commencing the actual evaluation. What do we mean by evaluation? What do we want to evaluate? By which (or whose) criteria? What do we need to know to do it? The basic task is of course is to assess whether the expected outputs, results and impacts were achieved or not. But it might also be interesting to try to reflect with the participants about how to explain the positive (or unsatisfactory) results, in order to become wiser and identify follow-up actions.
In practical terms, the setting should be carefully thought about to make sure all participants are comfortable in speaking out. The actual organization of the evaluation must fit the local setting. Should you organise a round table plenary inside an office building with presentations and discussions of results, or would it be more relaxed and productive to make observations and have discussions as you walk together to visit the site of the action plan? Are the involved stakeholder groups comfortable in sitting together, or would it be better to sit down with each of them in focus groups or by use of household interviews?
If you have opted for a formal stakeholder workshop, then it requires that you anticipate the need for a division of roles where competent facilitators and rapporteurs should be identified beforehand. The detailed organisation of the meeting should be done with them.
Some tools and frameworks for this are listed in the next section.
The tools used for the evaluation mostly fall within the following categories:
- Guidelines for the organisation of meetings and choosing the setup
- Tables and outlines for structuring the evaluation process (which questions to ask)
- Techniques for supporting the participants in answering the questions
There are many general guidelines available for Participatory Evaluation on-line already, such as the PLA notes series of the IIED at http://pubs.iied.org/14618IIED.html; the German Development Institute (former GTZ) guide on Participatory Monitoring and Evaluation (PAME) at http://www2.gtz.de/Dokumente/oe44/ecosan/en-guide-participatory-monitoring-evaluation.pdf, or the guidelines on Communicating and Reporting on an Evaluation from the Catholic Relief Services at http://www.crsprogramquality.org/storage/pubs/me/MEmodule_communicating.pdf.
At the HighARCS project we have been inspired by such guidelines. To fit our own situation, we have provided a brief on organising final stakeholder workshops. Moreover, suggestions for the site teams to reflect on the outcomes and why they turned out the way they did, as well as templates for tables summing up the lessons learned from the action plans have been used.
Examples of how they have been applied by the site teams, and which techniques they have actually used, will be uploaded when they have been reported.
NaN